Skip to content
DEVELOPERS

Make evaluation a release check.

Run a pinned suite from the CLI or REST API. Give CI the exit status and report it needs to decide what ships.

Pin a published suite in CI

From the source checkout, install dependencies and run npm link to make nore available. A personal workspace owner can issue a key with read and runs:write. Set NORE_URL to the HTTPS installation origin and supply NORE_API_KEY from a protected secret.

Replace the example suite and version with your own published values. Supply a unique RELEASE_KEY for each release attempt and keep it stable when retrying the same submission.

CLI gate command
nore run --suite wallet-safety --version 1 --gate strict \
  --idempotency-key "$RELEASE_KEY" --output report.json
ExitMeaning
0The run passed the selected gate.
1An assertion failed the gate.
2Invalid CLI usage.
3Skipped or partial coverage without infrastructure failure.
4Infrastructure failure, timeout or cancellation.
5API, network or filesystem error.

Archive the JSON report even if the gate fails. Strict mode fails on assertion failures. Critical mode can accept a run with noncritical failures when there are no skips or infrastructure failures.

Submit and read runs with REST

Use the workspace key as a bearer token. Read data.id from the submit response, assign it to RUN_ID, and poll that run until it is terminal. The report becomes available with terminal evidence.

Submit a run
curl --fail-with-body "$NORE_URL/api/v1/runs" \
  -H "Authorization: Bearer $NORE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $RELEASE_KEY" \
  -d '{"suite":"wallet-safety","version":1}'
Poll and read the report
curl --fail-with-body "$NORE_URL/api/v1/runs/$RUN_ID" \
  -H "Authorization: Bearer $NORE_API_KEY"

curl --fail-with-body "$NORE_URL/api/v1/reports/$RUN_ID" \
  -H "Authorization: Bearer $NORE_API_KEY"

Choose how to receive results

Poll the run for its status and the trace for paginated events. A personal workspace owner can register a signed webhook in Settings. Verify the signature, check its timestamp and deduplicate delivery IDs before processing a result. Deliveries are at least once.