# ReTibe MCP — a guide for Claude Code

Paste this file (or a link to it) into a Claude Code thread and it will connect to
the ReTibe platform and write you automated tests.

Russian original: `docs/CLAUDE_MCP_GUIDE.md`. The two are kept in step; where they
disagree, the Russian one was edited first.

---

## 1. Connecting

```bash
claude mcp add --transport http retibe https://retibe.com/mcp \
  --header "Authorization: Bearer YOUR_TOKEN"
```

Issue a token in your profile at https://retibe.com/profile → "API tokens". It is
shown once and cannot be recovered. Revoking is in the same place and takes effect
immediately.

**Two independent instances.** There is also `https://retibe.ru/mcp` — the same
code and the same set of tools, but a **separate database**: its own accounts, its
own scenarios, its own quota. A token from one does not work on the other; you get
`Invalid, revoked or expired`. Issue the token where your scenarios live.

To check the connection, ask Claude to call `retibe_whoami`. It should return your
email, your plan and what is left of your quotas.

---

## 2. How this differs from Playwright MCP

Worth understanding before you start, or your expectations will part company with
reality.

| | Playwright MCP | ReTibe MCP |
|---|---|---|
| How it works | drives a live browser | writes a scenario, runs it, reads the analysis |
| Sees the page between steps | yes | **no** |
| What survives the session | nothing | a saved scenario, a report, run history |
| Running it again | explain it again | one call by id |
| Where it executes | locally | on the platform, in its browsers |

**The upshot:** Playwright MCP is good for reconnaissance — "look at this page,
click around". ReTibe is good for pinning things down — "this path must always
work, check it on every release".

They are not competitors. The working combination is: scout with Playwright, pin it
down in ReTibe.

If Claude does not know your site's markup, have it look at the page first by
whatever means it has (Playwright MCP, WebFetch, or just open DevTools and hand it
the selectors). Guessing selectors blind is something it does badly.

---

## 3. How to ask

Works well:

> Write a ReTibe scenario: open https://example.com, check the heading is there, go
> to /pricing, make sure the form is visible. Validate it, run it, show me the
> result.

> Run webtest-123 failed. Work out why and suggest a fix.

> Go through my scenarios and find checks that do not actually check anything.

> Look at the "Release 2.4" checklist: what is still manual? Take the top case,
> write a scenario and link it to that case.

> Put the smoke set on every morning at 07:30 Moscow time. Show me what time the
> platform actually scheduled it for.

> What is scheduled for next week, and what failed last week?

Works badly:

> Test the site.

Too vague — Claude cannot see the page and will not guess what matters.

---

## 4. The right order of work

Tell Claude to follow this order; it is built into the tool descriptions.

1. **`retibe_env`** — which environments exist, and the parameter keys
2. **`retibe_actions`** — the action catalog, and the contract of each action it intends to use
3. **`retibe_reference`** — cross-cutting mechanics (templates, conditions, fixtures)
4. write the scenario
5. **`retibe_validate_scenario`** — before a browser starts; takes milliseconds
6. **`retibe_validate_scenario view:'storyboard'`** — "which of these steps can actually go red?"
7. **`retibe_run_scenario`** → **`retibe_run_result`**

Step 5 cannot be skipped. The platform **does not validate scenarios** — it will
accept a broken one and keep running it.

Step 6 answers a question the warning list does not. Forty warnings do not tell you
whether the test would notice your site breaking; the storyboard does, and it costs
the same milliseconds. If it says nothing can go red, the scenario is decorative and
running it is pointless.

If the account keeps checklists, there is a useful step zero before item 4 —
**`retibe_checklists`**, which answers "what is worth automating at all" — and after
you save the scenario, **`retibe_checklist_item op:link`** ties a case to it. The
link is a plain string the platform never validates; how it breaks is section 10.

How to name a scenario and how many to keep is section 14. Read it **before** item
4: the name and the top-level `id` are close to unchangeable afterwards, because
both the run history and the checklist link hang off them.

`retibe_examples` and `retibe_docs` are deliberately outside this order — they are
not always needed. Reach for them when an action is unfamiliar and the contract from
`retibe_actions` is not enough: `retibe_examples` gives a whole working scenario,
`retibe_docs` gives prose on a topic. How much to trust them is section 5.

---

## 5. How much to trust any of this

The reference tools have **different natures and different reliability**. This is
not pedantry — it decides what to believe when two of them disagree.

| Source | Where it comes from | Freshness |
|---|---|---|
| `retibe_actions` | extracted from the engine's own `switch` by parsing its AST | **high** — a test re-reads the engine on every run and fails on any drift |
| `data.*` field contracts | read out of the engine once, maintained by hand since | **medium** — accurate today, but nothing checks it against the engine automatically |
| `retibe_reference` | frozen text, compiled into the build | **low** — nothing regenerates it |
| `retibe_docs`, `retibe_examples` | files in `knowledge/` and `examples/`, shipped in the image | **medium** — edited by hand, never checked against the engine |
| a checklist's link to a scenario | a string somebody typed once | **none** — no foreign key, no trigger, no reconciliation; it breaks silently (section 10) |
| tool descriptions and prompts | written by hand | **low** — connected to the engine by nothing |

**What this means in practice.**

The action list is reliable. If `retibe_actions` did not return an action, the engine
does not have it: the catalog is machine-extracted from the dispatcher and guarded by
a drift test. Invented actions (`web_fill`, `web_type`, `web_hover`) are caught by the
validator.

Everything else is prose that may have fallen behind the code. Believe it until the
first contradiction with observed behaviour; when they disagree, the engine is right.

**The public endpoint can lag the repository.** Deploys are manual — `git pull` plus
a container rebuild, no CI/CD, and nothing checks the boundary. The endpoint does not
report its version or commit, so the discrepancy is invisible to a client. If a fix
to the validator or to a description is already on main but the behaviour has not
changed, it most likely just has not shipped yet.

---

## 6. The main thing to know about this engine

This is why the validator exists at all. The engine accepts almost anything, so a
broken test does not fail — it **passes green while checking nothing**. Claude is
told this in the tool descriptions, but it is worth knowing yourself.

**Checks that cannot fail.** `api_test`, `security`, `accessibility`, `seo_analysis`,
`graphql_test`, `load_test` and several others write the problem into findings while
the step itself stays `passed`. The API returned a 500 — the test is green. If a step
is meant to block a release, it needs a real assertion after it.

**`web_assert` does less than it looks like it does.** It checks visibility and
substring containment — and the substring only through `data.contains` or its alias
`data.expectedText` (the engine reads `data.contains ?? data.expectedText`). The
fields `assertion` / `expected` / `attribute` / `text` / `value` are **never read at
all**, so a step using them passes on any visible element with any text in it.

**There is no built-in absence assertion.** `visible: false` looks like "it is not
there" and means precisely the opposite: the wait is weakened to `state: "attached"`,
so the element must still be in the DOM, merely possibly hidden. None of the 48
actions asserts absence — but you can get there with `web_evaluate`: a script like
`if (document.querySelector('.popup')) { throw new Error('still there') } true`
genuinely fails the step, because the engine wraps the exception in
`web_evaluate failed:` and rethrows, provided the step does not carry `optional`.

**Assert the outcome, not the fact of the action.** Filling a form and clicking
"submit" is not yet a test: if nothing is checked afterwards, the steps pass green
even when the server refuses. You need a signal that **might not arrive** — a
redirect, an element appearing, a URL change. A good way to be sure a check is real
is to break the scenario deliberately and watch it go red.

**No screenshot is saved on a failing step** — the engine captures the screen after a
successful action. You will be diagnosing the failure from a neighbouring step's
frame.

**Visual regression fires in exactly two cases** — `web_scroll_screenshot` together
with `data.screenshotFullPage`, and `use_fixture` when the fixture carries
`visualRegression`. On `web_screenshot` it silently does nothing.

**And `visualRegression` is a STEP-level field, not a `data` one.** This is the
shape that works:

```json
{ "id": "full", "action": "web_scroll_screenshot",
  "visualRegression": true,
  "data": { "screenshotFullPage": true } }
```

Putting it in `data` is the natural mistake and the most expensive one here: the
validator does not look there, so it answers `valid: true` with not a single
warning, and no regression is captured. Verified against the live endpoint.

**Actions that do not exist.** The platform's built-in assistant documents
`web_fill`, `web_type`, `web_hover`, `web_select` and several more that the engine
does not have. The validator catches them and names the replacement.

**`{{params.*}}` do not work in Telegram scenarios** — the Telegram engine does not
resolve templates at all.

The validator surfaces most of this as warnings with a ready replacement:
`PHANTOM_ACTION` (with the exact substitute), `CANNOT_FAIL`, `ASSERT_FIELD_IGNORED`,
`VR_ON_WRONG_ACTION`, `TEMPLATE_IN_TELEGRAM`. If Claude ignored them, ask it to run
`retibe_validate_scenario` and go through each one.

**But four of these it cannot catch, as a matter of principle** — keep them in your
own head:

- **a scenario that checks nothing whatsoever.** Open, fill, click, and not a single
  assertion: `valid: true`, zero warnings;
- **a lone `visible: false`.** The validator mentions it only inside the `fix` of
  `ASSERT_FIELD_IGNORED`, i.e. only when an ignored field also ended up in `data`. On
  its own the step passes clean;
- **`visualRegression` put inside `data`.** The `VR_ON_WRONG_ACTION` check reads the
  step-level field, so it says nothing at all about `data.visualRegression` —
  `valid: true`, and no regression;
- **the contents of a fixture** — the body of `use_fixture` is not visible to the
  validator.

For the first there is the `/harden` prompt, which is written precisely as a check
for what the validator cannot see. And `view:'storyboard'` (section 11), which
answers it mechanically: it lists every step with a verdict on whether that step can
go red, and says how many can. `valid: true` together with "nothing can go red" is
the first item on that list, seen at a glance.

---

## 7. The tools

**Reference:** `retibe_actions` (the action catalog — the only machine-extracted
one), `retibe_reference` (cross-cutting mechanics), `retibe_docs` (an authoring
handbook, by section), `retibe_examples` (working scenarios as few-shot material),
`retibe_doctor` (the state of the run environment — on the public endpoint that
means the server's container, not your machine). Their reliability differs — see
section 5.

**Work:** `retibe_validate_scenario` (plus `view:'storyboard'` — section 11),
`retibe_run_scenario`, `retibe_run_status` (plus `wait_for` and `since` — section
12), `retibe_run_result`, `retibe_list_runs`, `retibe_stop_run`, `retibe_publish_run`
(needed in local mode, to publish a local run into reports and history; runs from
`retibe.com/mcp` are on the platform already).

**Platform:** `retibe_whoami`, `retibe_env`, `retibe_scenarios`,
`retibe_save_scenario`, `retibe_history`, `retibe_visual_review`.

**Checklists:** `retibe_checklists` (the coverage map — what is still manual),
`retibe_checklist_item` (link a case to a scenario, unlink it, record a manual
result). Section 10.

**Calendar:** `retibe_calendar` (what is scheduled and what has already run),
`retibe_schedule` (create, edit, pause, delete a schedule, and run one now).
Section 13.

**Prompts:** `/author` (write a scenario), `/debug` (diagnose a failure), `/harden`
(find checks that check nothing).

The repository has **22 tools and 3 prompts**. What the endpoint you are talking to
actually serves is whatever `/mcp` shows after connecting, and that is more
trustworthy than any number in this file.

---

## 8. Reading the result

- **`passed` / `failed`** — the test ran and the verdict is real
- **`infra_error`** — the scenario **did not execute**: the browser failed to start,
  or the deadline expired. There is no bug in the scenario to look for
- **`unknown`** — the run has not finished. This is **not** "passed"
- **`stepsObserved`** — an estimate, not a counter. Only `state` says a run finished.
  It became more accurate — a step is counted when it is entered rather than when a
  token changes — but it is still an estimate: step boundaries do not exist as data
  in this engine, see section 12
- **`steps`** — the step timeline with intervals. The intervals are estimates; what
  exactly stops them being measurements is in `timelineCaveats` on every response
- **`waitEndedBy`** — what ended the wait. `timeout` means "time ran out", not
  "nothing happened"
- **`next_offset`** — the cursor for your next poll

Screenshots come back as paths, not as images.

---

## 9. Limits

- **Telegram is authoring only.** Running requires a session created by an
  interactive login with an SMS code.
- **One test at a time** per account; bot and web tests block each other.
- Runs from here spend the same monthly quota as runs started from the web UI.
- A web test usually takes about a minute; the slowest reach ten. Calls never block:
  starting returns a `runId`, and you poll from there.

---

## 10. Checklists

A manual checklist is something the team already has: a list of cases, some of them
checked by hand every release. Both tools exist for one purpose: **make it visible
what is still manual, and let it be closed with a scenario.**

**The working cycle.**

1. `retibe_checklists` — the checklists with raw counters
2. `retibe_checklists id:N only:'unlinked'` — exactly the cases nothing covers
3. write a scenario for one of them, `retibe_validate_scenario`, `retibe_run_scenario`
4. `retibe_save_scenario` — save it on the platform
5. `retibe_scenarios id:…` — **read the saved one back**
6. `retibe_checklist_item op:'link'` — link the case to it

Item 5 cannot be skipped, and not out of tidiness: the key the platform later matches
the checklist on is taken **from the saved scenario**, not from the linking call.

### How the link works, and why it breaks

The platform ties a run to a case like this: the **checklist** has a `scenario_id`
field (a string), the **case** has a `step_id`. When a run finishes, the engine takes
`scenario.id || scenario.name` from whatever object started it and looks for
checklists with that `scenario_id`.

Everything else follows:

- **One checklist = one scenario.** The field is on the checklist, not on the case.
  Two cases about different scenarios are two checklists. `op:'link'` on a checklist
  that already points at a different scenario requires `confirm:true`, because it
  redirects **every** case in it.
- **No foreign key, no trigger, no reconciliation.** Rename the scenario, give it an
  `id` it did not have, delete it — the link is broken, silently and permanently. The
  platform reports this nowhere: a failed match looks exactly like "no matching
  cases". The only way to find out is `retibe_checklists verify_links:true`, with two
  caveats — it works on the listing (not on an opened checklist) and it checks
  against your 200 most recent scenarios, so `linksNotFound` means "not found", not
  "broken". The response says so. It matches all three forms the key can take: the
  scenario's own string `id`, the platform's numeric id, and the name. If none of the
  checklists points anywhere at all, the response says `nothingToVerify` rather than
  staying silent.
- **`scenarioKeySource` in the `op:'link'` response matters more than the key
  itself.** If it says `template id`, the scenario has no string `id` of its own and
  the key became the platform's numeric id. Such a link fires for a run started from
  the object `retibe_scenarios` returns, and does **not** fire for the same scenario
  passed inline — there the key is the name. If you want one key for both, give the
  scenario a stable top-level string `id` and link it again.
- **`checklist_items.scenario_id` and `scenario_hash` are dead columns.** They exist,
  they are indexed, and nothing reads them. The tool does not write them.
  `scenario_hash` is **not** the `historyId` that `retibe_history` takes.
- **Export/import severs the link.** A checklist export does not contain the
  checklist-level `scenario_id`, so a re-imported checklist looks wired up and
  receives no results.

### Where a case's results come from

Only from a run of the linked scenario **on the platform**. A local or composite run
does not stamp checklists at all, and `retibe_publish_run` does not fix that: it sends
the run id as the scenario name, and that key matches nothing.

The recording code coerces anything that is not `passed` to `failed` — a cancelled or
stopped run shows on the case as an honest failure. A case with no `step_id` takes the
run's overall verdict rather than its own step's.

### The coverage numbers

There are **three of them**, and they disagree:

| Where | Formula |
|---|---|
| the checklist listing | `automated / total` — `partial` is not counted |
| an opened checklist | `(automated + partial×0.5) / total` |
| the web UI | computes a fourth, in the browser, per project |

So the tool returns **raw counters** and names the formula beside them. A percentage
from a response can only be trusted together with its formula.

Two counters behave differently from how they look. `passed` and `failed` each count a
case if **either** of its two results matches — manual or automatic — so a case that
passed by hand and failed automatically is in both, and the sum can exceed `total`.
`automation_status` and `priority` are free text and the platform validates neither: a
value outside `manual/automated/partial/planned` drops out of every counter.

### `op:'record'` — careful

It records a **manual** result under the token owner's name. The platform has no
marker for "an agent did this": a human opening the checklist will see themselves as
the tester. The `[MCP]` prefix on the comment is the only thing distinguishing such an
entry. Execution rows **are not deleted** — there is no route for it. Hence
`confirm:true`.

If the call returns an error, **do not repeat it**. The route writes the execution row
first and updates the case afterwards, with no transaction: the result may already be
recorded. Read the case back with `retibe_checklists`.

### What is deliberately absent

Creating and deleting checklists and cases, and importing. A checklist is the team's
artefact, not the agent's; import additionally matches on an exact name and overwrites
what it finds. Set the structure up in the web UI; fill it with coverage from here.

---

## 11. The storyboard: which step can actually fail

`retibe_validate_scenario` with `view:'storyboard'`. A table per step, plus one line
of conclusion.

```
#  id     action        target           verdict
1  open   web_navigate  https://x.com    can fail
2  typo   web_clikc     #a               no such action — not in the action catalog for this kind
3  guard  web_assert    h1               may not run — guarded by skipIf, which the engine checks first
4  title  web_assert    h1.hero          asserts nothing — data.text is never read; no data.contains
5  sec    security      {checks}         always green — writes its problems to findings and reports passed

Only step 1 can fail; the other 4 cannot make this scenario go red.
```

`view` takes `findings` (the default, as before), `storyboard` and `both`.

**The verdicts.** Read worst to best; the one shown is the one that actually governs
the step:

| Verdict | What it means |
|---|---|
| `no such action` | the action is not in the catalog — the step throws rather than checking. Catches both a typo (`web_clikc`) and an invented action (`web_hover`) |
| `always green` | writes its problem into findings and reports `passed` regardless |
| `asserts nothing` | a `web_assert` whose text field the engine does not read |
| `failure ignored` | `optional:true`, and this action honours it |
| `may not run` | the step is guarded by `condition` or `skipIf` — nothing can be promised about it |
| `can fail` | a real gate: this step can go red |

**What this is not.** `can fail` is about the shape of the step, not about your site:
the engine can turn this step red, but whether it is the right check the storyboard
does not know. Reachability is not statically decidable (conditions, fixtures,
`optional` on four actions), so a guarded step's honest verdict is "may not run"
rather than "green".

Both guards count: the engine checks **`skipIf` before `condition`**
(`WebSiteTester.ts:4707`) and records a skipped step as a success — so a case shaped
like "`skipIf: {previousStepFailed: true}` plus a real assertion" looks like a gate
and in fact switches itself off the moment the previous step fails. An empty guard
(`condition: null`, `condition: {}`) does not count as a guard, because the engine
does not count it either.

The verdicts are derived from the same constants the validator itself uses
(`CANNOT_FAIL_ACTIONS`, `HONOURS_OPTIONAL`, `ASSERT_IGNORED_FIELDS`,
`PHANTOM_WEB_ACTIONS`). A second copy of those lists would have drifted from the
validator silently — which is exactly the defect all of this was written against.

The reliable way to know a check is real remains the same one: break the scenario
deliberately and watch it go red.

---

## 12. Polling a run: wait for a thing, not for a duration

`retibe_run_status` gained `wait_for`, `since` and `stall_ms`.

| `wait_for` | Returns when |
|---|---|
| `terminal` | the run reached a final state (the default, as before) |
| `step` | the next step boundary went by |
| `failure` | the first error-level line appeared — or the run ended |
| `stall` | the run is alive but has said nothing for `stall_ms` (30 s by default) |

The point is that a blind `wait_ms` spends the whole budget on a run that failed two
seconds in. `wait_for:'failure'` comes back immediately — and it is **not** a verdict:
`state` is still `running`, and the engine logs at error level on recoverable retries
too. What ended the wait is in `waitEndedBy`: `timeout` means "time ran out",
everything else means "this happened".

`since` is a cursor. The response carries `next_offset`; pass it to the next call and
the poll reads only what is new instead of re-reading the tail you already paid for.

**The step timeline.** The response gained a `steps` field:

```
✓ open   ██                       2.4s
✗ submit ████████████████████████ 31.7s
… verify                          ?
```

`✓` completed, `✗` failed, `–` skipped, `…` still running. Trust it exactly as far as
`timelineCaveats` says, and here is why: **step boundaries do not exist as data in
this engine.** There is no `stepStart` and no `stepEnd` — a step's state is recognised
by matching Russian prose in the log. Consequently:

- A step whose message did not match a pattern is simply absent from the timeline.
- A duration is the interval between two recognised lines, not time the engine
  reported spending. That makes it an estimate, not a measurement.
- `(retried)` means the engine entered the step more than once; the interval covers
  every attempt.
- The last, still-running step shows `?` rather than `0.0s`: printing a zero next to a
  step that has been hanging for a minute is worse than admitting there is no figure.

`stepsObserved` is still an estimate but a better one: a step is counted when it is
entered, so a retry no longer advances the counter. What this does **not** fix: a
step's token is `id || action`, so two steps without an `id` sharing an action remain
indistinguishable and count as one. The only cure is to give steps ids.

**On the public endpoint this works through the platform's progress channel**
(`GET /api/webtest/:testId/progress`, added alongside it). Before that, a remote poll
returned hard zeros and a sentence saying progress was unavailable. If the channel
does not answer, the response says plainly that **nothing is known** about the run —
which is not the same as "the run is quiet" — and `waitUnsupported` lists the
conditions it could not honour. The ring holds the last 500 lines: if your cursor is
older than that, the response says entries were evicted rather than going quiet.

---

## 13. The calendar and schedules

Two tools: `retibe_calendar` reads, `retibe_schedule` writes.

```
retibe_calendar                              # what is scheduled
retibe_calendar id:5                         # one schedule
retibe_calendar view:'entries' from:'2026-09-01' to:'2026-09-30'
retibe_schedule op:'create' name:'shop-smoke-nightly' scenario:42 \
                frequency:'daily' time:'07:30' timezone:'Europe/Moscow'
```

### There are only four frequencies

`once`, `daily`, `weekly`, `monthly`. The tool **does not accept** `cron`, and that is
not caution — the `cron` branch in the engine reads, in full:

```ts
// Simplified cron parser - for production use proper library like node-cron
// This is a placeholder that returns next hour
// TODO: Implement proper cron parsing
```

The expression is read exactly once — to check it is non-empty — after which the top
of the next UTC hour is returned. If you create such a schedule through the web API
directly:

- `scheduleConfig: {cron: "0 9 * * 1-5"}` → **200**, and the test runs **every hour,
  around the clock**, ignoring the expression. The quota is gone within a day.
- `scheduleConfig: {expression: "0 9 * * 1-5"}` → **400**: the route validates the key
  `expression` while the scheduler reads `cron`. So the key the error message points
  you at does not work at all.
- Such a schedule never appears on the calendar — for frequencies outside the four
  working ones the platform returns no entries whatsoever.

`interval` is named in the route's error text but has no branch in the scheduler, so it
is refused too.

### `nextRunAt` is the only honest answer to "when"

The platform computes it when the schedule is written. The tool returns it in the
response to `create` and `update` and **never recomputes it**. Check it with your own
eyes: if that is not the moment you expected, the schedule is wrong — and now is a
better time to find out than tomorrow.

**The timezone defaults to UTC.** Leave `timezone` out and `time: '09:00'` means 09:00
UTC. This is the most common cause of "the schedule did not fire". Pass an IANA name:
`Europe/Moscow`, `Asia/Almaty`.

For `monthly`, `day_of_month` is clamped to the length of the month: 31 means the last
day in February. For `daily` you can narrow to weekdays with `days_of_week`
(0 = Sunday).

### The run counters

`counts` in the response is `runs`, `passed`, `failed`, with `countsFrom` beside it
naming where they came from. There are two sources and they are not equivalent:

- **`tests table`** — counted from real runs. Trust this one.
- **`schedule columns`** — the schedule's own counters. Read these with suspicion:
  the method that was supposed to write `success_count` / `fail_count` was called
  from nowhere until `b297b63`, so for schedules created before that `passed` and
  `failed` stayed at zero however many times the schedule succeeded. Only `runs`
  moved.

The tool prefers the first source and falls back to the second only when statistics
could not be read — and says so in `countsFrom` rather than mixing them silently.

### A schedule holds a copy of the scenario

On creation the scenario's **steps** are copied into the schedule, not referenced. Edit
the scenario and the schedule keeps running the old version. To pull the new one in,
call `op:'update'` with the same `scenario`.

### What `view:'entries'` shows

The platform returns several different things under one key; the tool separates them
with a `kind` field:

- **`occurrence`** — a computed future firing. Nothing exists in the database for it;
  it is arithmetic over the schedule. The `end` field is not a prediction but a flat
  "plus five minutes" placeholder.
- **`run`** — a real run, past or in flight.

The caveats the tool returns alongside the data:

- Running tests are included **regardless of the range** — deliberately, so an active
  test is always visible.
- Finished runs are drawn only from the account's **500 most recent** tests. A run
  inside the range but older than those 500 is silently absent.
- A paused schedule still produces entries, with `status: 'paused'`.

### `op:'run_now'`

Fires the schedule immediately and returns a `testId` you **can poll**:
`retibe_run_status runId:'<testId>'`. This works as of `f413edd` — the scheduler starts
a web test through the same function that fills the progress channel, and ownership is
resolved from the `tests` table.

Roughly half of calls will return `Scheduler service is not available`. The cause is
not the schedule: the route reads the scheduler out of process memory, and it is only
set in the first of two workers, with connections distributed round-robin. **Just call
again** — the tool says so in the error text. Normal firings of the schedule are
unaffected: the scheduler lives where it lives.

A run from here spends the monthly quota like any other and is subject to the
one-test-at-a-time lock.

### What the tools do not do

Deleting requires `confirm:true` — the copy of the scenario inside the schedule goes
with it and cannot be restored (reports from past runs remain). If you only want it to
stop firing, use `op:'pause'`: the schedule keeps both its history and its next-run
time.

Pause and resume go through the update route rather than `/toggle`: toggle flips
whatever it finds, so two callers both asking to pause could leave the schedule
running.

---

## 14. Naming scenarios, and how many to keep

This section is about the things you cannot easily redo later. A scenario's name and
top-level `id` are not cosmetic: the run history and the checklist link both hang off
them.

### A project prefix, in the name and in the `id`

Give every scenario a **stable top-level string `id`** of the form
`<project>-<area>-<what it checks>`, lowercase, hyphen-separated:

```json
{
  "id": "shop-checkout-guest-purchase",
  "name": "shop / checkout — guest purchase",
  "startUrl": "https://shop.example.com",
  "steps": [ … ]
}
```

Why this rather than "Login test":

- **The `id` is the key the platform finds a checklist by.** The engine takes
  `scenario.id || scenario.name`. With no string `id` of its own, the key becomes the
  platform's numeric id — and then the link works for a run started from the object
  `retibe_scenarios` returns and does not work for the same scenario passed inline.
  With a stable `id` the key is the same in both cases (section 10).
- **`retibe_scenarios` is your coverage map.** The listing returns the name, the
  description and the step count. With prefixes, `search:'shop-checkout'` finds a whole
  area; without them, twenty "Login test 2" cannot be found at all.
- **`verify_links` checks against your 200 most recent scenarios.** A prefix makes it
  immediately clear which project an unmatched key belonged to.
- **Renaming orphans the history.** `historyId` is an md5 of the name, the URL, the
  step count and the action list. Change the name or add a step and the previous
  history is orphaned. So the name is chosen once.

Keep prefixes short and consistent within a project: `shop-`, `crm-`, `api-`. The area
is the second level: `shop-auth-`, `shop-checkout-`, `shop-catalog-`.

### There should be few scenarios, and they should be long

This is not about tidiness but about how the limits work.

| What | Why it bears on the number of scenarios |
|---|---|
| **One test at a time** per account, bot and web tests blocking each other | 40 one-minute scenarios is 40 strictly sequential minutes; they cannot overlap |
| **The quota counts runs, not steps** | ten 40-step scenarios is 10 runs; forty 10-step ones is 40 runs for the same coverage |
| **Fixed per-run overhead** | starting a browser and reaching the page is about a minute at p50 even for a tiny test. On short scenarios the overhead *is* the run |
| **`infra_error` is counted per run** | the more launches, the more chances one of them fails to start |

So aim for **a handful to a few dozen scenarios per project**, not hundreds. One
scenario is one whole user journey with its own setup: "arrived, logged in, added to
the cart, checked out, verified the email" is a single 30–50 step scenario, not five of
six steps each.

### Where a long scenario starts to hurt

The other side of it: a scenario that failed at step 3 tells you nothing about steps
4–40. So do divide — along these lines:

- **By shared setup.** Steps that need the same login and the same state live together.
  A different account or a different environment means a different scenario.
- **By what blocks a release.** Keep "it failed, we do not ship" apart from "it failed,
  file a bug". The first runs on every release, the second overnight.
- **By speed.** Do not mix a two-minute smoke with a ten-minute catalog walk, or you
  wait for the fast answer at the slow one's pace.

Inside a long scenario, decide deliberately what aborts the run: `critical` and
`stopOnFailure` are compared strictly against `true` (the validator warns if something
else ended up there). Without them a failed step does not stop the rest — sometimes
that is what you want, sometimes not.

### Give steps ids

In a long scenario this is required, not advisory:

- **The step timeline and `stepsObserved`** identify a step by `id || action`. Two
  steps without an `id` sharing an action are indistinguishable and count as one
  (section 12).
- **A checklist case links on `step_id`** — with no `id` on the step, a specific case
  cannot be tied to a specific check and takes the whole run's verdict instead
  (section 10).
- **The storyboard** names a step by its `id`; without one, a 40-row table is hard to
  read.

Step names follow the same principle as scenarios: `login-submit`, `cart-add`,
`checkout-assert-redirect`. Not `step-1`.

### How this maps onto checklists

Grouping and checklists are coupled more tightly than it looks: **one checklist points
at one scenario**. So a 40-step scenario covering 8 manual cases is one checklist,
where each case has its own `step_id`. That is the main practical argument for long
scenarios: the mapping from cases to checks comes out naturally. Split those same 8
cases into 8 scenarios and you need 8 checklists.

### A short instruction you can hand to Claude

> Work with ReTibe like this. Scenario names are `<project>-<area>-<what>`, lowercase,
> hyphen-separated, and put that same value in the top-level `id` field; our project
> prefix is `shop-`. Give every step a meaningful `id`. Do not breed small scenarios:
> one scenario is one whole user journey, 30–50 steps, split off only what needs
> different setup or a different run frequency. Always `retibe_validate_scenario`
> before running, then the same call with `view:'storyboard'` — and if nothing can go
> red, rewrite it rather than running it.

---

## 15. When something is wrong

| Symptom | Cause |
|---|---|
| `Missing API token` | The header did not arrive — check `--header` in the connect command |
| `Invalid, revoked or expired` | The token is revoked or wrong — or was issued on the other instance, see section 1 |
| `Usage limit exceeded` | The platform refused the run: web_tests or security_scan ran out, visible in `retibe_whoami` |
| `This account has used N of M API requests` | A gateway refusal on the api_requests quota — a separate counter, spent by the AI scenario generator rather than by MCP calls |
| The test is green but checks nothing | Run `/harden` over the scenario |
| `infra_error` | The scenario is not at fault; look at `retibe_doctor` and `retibe_whoami`, then try later |
| `... is missing from this checkout` | The endpoint is on an image older than `d1069cc` — `knowledge/` was not in it. Needs a redeploy, see section 5 |
| `No examples in this checkout` | The same thing, about the `examples/` directory |
| The validator says something different from this file | The endpoint has fallen behind the repository — deploys are manual, see section 5 |
| A case is linked but no results arrive | The run was local, or the key did not match. `retibe_checklists verify_links:true`, then check `scenarioKeySource` (section 10) |
| `Case N is not in checklist M` | A case id from a different checklist. The tool verifies membership before writing — the platform did not |
| `... already points at "..."` | One checklist points at one scenario. Either `confirm:true` and move every case, or use a separate checklist |
| `"step-N" is not a step id` | The step has no `id` of its own, or a different one. The engine addresses a step by `id || action`, as saved |
| The storyboard says nothing can go red | The scenario checks nothing. Go through every verdict from section 11 — this is not a false alarm |
| `steps` is empty while the run is going | No engine message matched a step-boundary pattern, or the progress channel did not answer. See `timelineCaveats` and `waitUnsupported` |
| "nothing is known about this run" | The progress channel did not answer — an old image, or Redis unreachable. This is **not** "the run is quiet"; the run's state is still honest |
| `wait_for:'failure'` returned but the run continues | By design: the engine logs at error level on recoverable retries too. Only `state` is a verdict |
| `Scheduler service is not available` | The route reads the scheduler from the memory of one worker out of two. Call again — the schedule itself is fine (section 13) |
| A schedule was created but did not fire | Check the `nextRunAt` from the response. The usual cause is a missing `timezone`, making 09:00 mean 09:00 UTC |
| The schedule exists but is not on the calendar | Its frequency is `cron` — the platform returns no entries for it. Recreate it as `daily`/`weekly` |
| The scenario was edited but the schedule runs the old one | A schedule holds a **copy** of the steps from when it was created. Update it with `op:'update'` and the same `scenario` |