# Diff two agent runs with reelier diff (exit 1 on drift) `reelier diff ` compares the last two runs of a compiled skill and turns any divergence into an exit code: it prints a per-step verdict, says `SAME` and exits 0 when the two runs are identical, and says `DRIFTED` and exits 1 the moment a step's outcome flips, a step is added or removed, or a step needed a different escalation level to pass. Drop that command into CI and a drifted agent run blocks the merge. The comparison is a pure read of the run records in `.reelier/runs/.jsonl` — no model, no network, no re-execution. It compares structure, outcomes, and escalation level, never the bound values a step returned, so a fresh API response on each honest run is not drift. This guide runs the whole thing end to end against real output; the failure mode it catches is defined at the linked glossary term. Markdown twin of https://www.reelier.com/guides/diff-two-agent-runs — a step-by-step Reelier how-to. Every command below is a real Reelier command and every output block is verbatim, never a mockup. ## Before you start - Reelier installed — `npm i -g reelier` (or swap `npx -y reelier@latest` for `reelier` in every command below). - A compiled skill with at least TWO recorded runs. `reelier diff` reads `.reelier/runs/.jsonl` and needs two run records to compare — replay the skill twice (or let a schedule tick twice). If you don't have a skill yet, compile one from a real session first. - Nothing else. The diff itself calls no model and makes no network calls — it only reads the receipts already on disk. ## 01 · Get two runs into the record `reelier run` re-executes a compiled skill's recorded tool calls and checks each assertion. At the default `--max-level 0` no model is constructed, so the replay costs 0 tokens and reproduces only the deterministic core of the workflow. Every run appends one receipt line to `.reelier/runs/.jsonl`. Run the skill twice — the first receipt becomes the baseline, the second is the candidate the diff will compare against it. (Replay is read-only by default; a recorded write step never re-fires unless you pass `--allow-writes`, and a destructive one still needs `--yes`.) The full record-and-replay loop is a guide of its own, linked below. ```bash reelier run registry-check.skill.md ``` ``` ✓ Step 1 — Call http.get [passed] 164ms PASSED: 1/1 steps ok, 0 failed, 164ms total ``` ## 02 · Diff the last two runs `reelier diff ` aligns the last two runs step by step and compares each step's outcome and escalation level. When every step matches, it prints one `=` line per step and a `SAME` summary, and exits 0. It never compares the data a step returned — only the shape of what happened. The compiled skill asserts a shape (`status == 200`), not a pinned value, so two honest replays seconds apart return different response bodies and still diff `SAME`. That is the point: same steps, fresh data is not drift. ```bash reelier diff registry-check ``` ``` = Step 1 "Call http.get": passed, unchanged. ✓ SAME — 1 step(s), every outcome and escalation level identical to the baseline. ``` ## 03 · Read the per-step verdict Each step gets one marked line. `=` means the step is unchanged. `✗` is hard drift — the outcome flipped (`passed → failed`), or a step was added or removed, i.e. the run did something structurally different. `~` is soft drift — same outcome, but the step had to escalate to a different level to reach it, so the world moved underneath and healing covered it (soft drift only appears once you opt into BYOK escalation above Level 0; the deterministic core never escalates). The summary line tallies them: `DRIFTED` splits the count into outcome/structure changes and steps that needed a different escalation level, so a strict CI gate can fail on any drift while a lenient one allows soft drift only. Timing is never counted as drift. ## 04 · Fail CI on the exit code The exit code is the whole integration surface: 0 on `SAME`, 1 on `DRIFTED` (and 1 when the two records aren't even the same skill). Add `reelier diff ` as a CI step after the replay and a non-zero exit fails the job — no extra assertions to wire up. To see a real `DRIFTED` verdict below, we forced one: we tightened the recorded contract from `status == 200` to `status == 404`, so the next replay against the live registry genuinely flipped `passed → failed`. In production the identical verdict fires on its own when an upstream API actually changes. Wiring this into a GitHub Actions job is covered on the platform page and the CI-gate guide, both linked below. ```bash reelier diff registry-check ``` ``` ✗ Step 1 "Call http.get": passed → failed — status == 404 failed: got 200. ⚠ DRIFTED — 1 outcome/structure change(s): #1 outcome-changed. ``` ## The receipt ``` $ reelier diff registry-check = Step 1 "Call http.get": passed, unchanged. ✓ SAME — 1 step(s), every outcome and escalation level identical to the baseline. $ reelier diff registry-check ✗ Step 1 "Call http.get": passed → failed — status == 404 failed: got 200. ⚠ DRIFTED — 1 outcome/structure change(s): #1 outcome-changed. ``` Both verdicts of the same skill, verbatim. The first diff ran over two clean replays of `registry-check` against the live npm registry and exited 0; then the recorded assertion was tightened from `status == 200` to `status == 404` to force a genuine `passed → failed` flip, so the second diff printed `DRIFTED` and exited 1. Produced on 2026-07-22 with `reelier` v0.14.0 in a scratch directory; the skill was compiled by `reelier from-session` from a one-call Claude Code-style transcript (a single `http.get`, shaped per `reelier/test/session.test.ts`). The diff itself constructed no model and made no network call — it only read the two receipts in `.reelier/runs/registry-check.jsonl`. ## FAQ **What exactly does reelier diff compare?** The last two run records in `.reelier/runs/.jsonl`, aligned by step number. It compares each step's outcome and escalation level plus the overall shape (steps added or removed) — never the bound values a step returned, and never timing. That is why 'same steps, fresh data' diffs `SAME`: only structure, outcomes, and healing level can drift. **I only have one run — it says 'Need at least 2 runs'.** `reelier diff` compares two run records, so it errors when the skill's `.reelier/runs/.jsonl` holds fewer than two. Replay the skill once more with `reelier run .skill.md`, then run the diff again — each replay appends one receipt. **What's the difference between a ✗ line and a ~ line?** `✗` is hard drift — the outcome flipped (`passed → failed`) or a step was added or removed; the run did something different. `~` is soft drift — same outcome, but reached by escalating to a different level, so the world shifted and healing absorbed it. A strict gate fails on either; a lenient one can allow soft drift. Soft drift only exists when you opt into BYOK escalation; the Level-0 deterministic core never escalates. ## Related - [Tool-call drift — the failure mode the diff is there to catch, defined](https://www.reelier.com/learn/tool-call-drift) - [Run receipt — the record diff reads; two of them are what it compares](https://www.reelier.com/learn/run-receipt) - [Guide: turn this exit code into a CI gate that blocks the merge](https://www.reelier.com/guides/ci-gate-for-agent-drift) - [Reelier for GitHub Actions — run the diff as a step on every push](https://www.reelier.com/for/github-actions) - [Guide: the full record → replay → diff loop, from a Claude Code session](https://www.reelier.com/guides/snapshot-test-claude-code-workflows) ## Sources - [seldonframe/reelier — the open-source (AGPL-3.0) CLI every command here runs](https://github.com/seldonframe/reelier) - [Exit status — the 0-is-success / non-zero-fails convention CI reads to block a merge](https://en.wikipedia.org/wiki/Exit_status) Other guides: [How to snapshot-test a Claude Code workflow](https://www.reelier.com/guides/snapshot-test-claude-code-workflows) · [How to record MCP tool calls with a wrap proxy](https://www.reelier.com/guides/record-mcp-tool-calls) · [How to replay an agent workflow at 0 LLM tokens](https://www.reelier.com/guides/replay-agent-workflow-zero-tokens) · [How to gate CI on agent drift](https://www.reelier.com/guides/ci-gate-for-agent-drift) · [Replay an agent skill on every PR with the Reelier GitHub Action](https://www.reelier.com/guides/github-action-agent-replay) · [Test your MCP server for regressions by replaying a golden run](https://www.reelier.com/guides/test-mcp-servers) · [Run a deterministic cron agent by scheduling a recorded replay](https://www.reelier.com/guides/deterministic-cron-agents) · [Build an agent audit trail from Reelier run receipts](https://www.reelier.com/guides/agent-audit-trail-receipts) · [Configure BYOK escalation levels: L0, L1, and L2](https://www.reelier.com/guides/byok-escalation-levels) · [Compile a skill from an existing session with reelier from-session](https://www.reelier.com/guides/from-session-compile) · [Convert an LLM agent skill into a deterministic, assertion-checked skill](https://www.reelier.com/guides/convert-agent-skill-deterministic)