Automate your API tests in continuous integration

A scenario built in the interface runs as it is in your pipeline. This guide covers scaling that up.

Prerequisites

• A Pro or Enterprise plan.
• An organisation token (rstk_…) to put in your CI’s secret vault. (Creating that token self-service from the dashboard is coming soon.)

The principle

RESTORM_TOKEN=$RESTORM_TOKEN restorm \
  --open ./api.restorm \
  --run "Tests de fumée" \
  --headless \
  --all-logs \
  --out run.log \
  --param baseUrl=$BASE_URL

Exit code 0 = success, 1 = the scenario failed, 2 = an invocation error, 3 = entitlement denied.

A virtual display is required

Restorm is a desktop application: even with no window, it needs a display server. On a Linux runner, prefix it with xvfb-run -a.

xvfb-run -a restorm --open ./api.restorm --run "Tests de fumée" --headless

Secrets

Never write a secret into the project. Declare your sensitive variables with the environment variable secret source; your CI’s vault injects them, Restorm reads them. See Secrets.

env:
  API_TOKEN: ${{ secrets.API_TOKEN }}

Parameterising per environment

Two complementary approaches:

1. A scenario parameter typed environment: --param Env=staging. The same scenario runs against any target.
2. Plain parameters: --param baseUrl=…, --param tenant=….

Conversion follows the parameter’s declared type, and an impossible conversion fails the launch immediately rather than running with a wrong value. See Variables and data.

Publishing the log

--out run.log writes the log as it goes. Publish it as an artefact, including when the job fails — that is precisely when it earns its keep.

- uses: actions/upload-artifact@v4
  if: always()
  with:
    name: journal-restorm
    path: run.log

Writing scenarios that read well in CI

A few habits that make all the difference when a red job shows up at 3 a.m.:

• Explicit assertion messages. The Assert action’s message field is what will appear in the log: write there what was expected.
• A log entry at the key steps. Without --all-logs, only the Log action’s entries are emitted: that is your narrative thread.
• Schema validation rather than field-by-field assertions. Wire the errors output of Schema validate onto a Log: you get the precise list of violations.
• Throw on the critical else outputs, so that the exit code reflects the failure.
• Retry around flaky network calls, rather than accepting intermittent tests. See Control.

Cleaning up

Chain the clean-up onto the main scenario’s done port: done waits for the whole subgraph to have finished. See Ports and links.

Traps worth knowing

Trap	Fix
The job waits for input	Supply all the parameters with --param; in headless mode, nothing can be asked for
A Toast action does not show	That is normal: it has no effect in headless mode. Use Log
The MCP server does not appear	That is intended: with no real display, it never starts
Exit 3	The token or the plan — the message says which of the four cases
The project file has moved	--open accepts a path relative to the repository: keep it relative

Complete examples

The GitHub Actions and GitLab CI pipelines are in Headless execution and CI.