Agent drift

01One name, several failure modes

The term arrived with agents in production, and it names an observation every operator makes eventually: an agent given the same job does not do the same job. The model version changes under the prompt. Retrieved context shifts. Accumulated memory nudges the plan. The agent improvises around a transient error and the improvisation sticks. No single step is wrong, and each looks like a reasonable fix in isolation — but the fixes compound into a workflow nobody reviewed.

Because the term is young, the taxonomies disagree at the edges. A production-monitoring treatment splits drift by cause: goal drift, context drift, reasoning drift, and collaboration drift — the last covering the agent's integrations with tools and other agents. A security glossary centers the workflow instead, defining agent drift as change in an agent's behavior, routing, or tool use that departs from the authorized plan. An academic study of multi-agent systems frames it as progressive behavioral degradation over extended interactions, with tool-usage patterns among its measured signals. The common thread across all three: drift is divergence from an intended baseline. Which exposes the practical problem — most agent behavior has no recorded baseline to diverge from.

02Tool-call drift: the subtype you can measure

Reasoning is hard to compare across runs; tool calls are data. Everything an agent does to the world outside its context window passes through them — a typed JSON request, a typed JSON response, a success or a failure. That layer can be recorded once and compared mechanically forever, which is why tool-call drift is the subtype worth building detection around: it is the part of agent drift with an objective before-and-after.

What counts as drift at that layer, concretely: a step's outcome changed (a step that passed now fails), the workflow's shape moved (a step was added or removed), or a step that used to run clean needed help to pass. What never counts: timing, and fresh data. Same steps, fresh values is not drift — it is the workflow working. A replayed report whose download count moved since last week has not drifted; a step whose response stopped matching its schema assertion has. Structure and outcomes, never data.

Reelier's diff splits the verdict further into hard and soft. Hard drift is an outcome or structure change — the run did something different. Soft drift is the same outcome reached at a different escalation level: the step still passed, but only because a model healed a stale expectation, which means the world moved underneath it. A strict CI gate can fail on both; a lenient one on hard drift only.

03Detection: record a baseline, diff every run after

Detection needs a baseline, so the loop starts by recording one. Capture a working run's tool calls — on Claude Code, reelier scan reads the transcript history in ~/.claude/projects; Cursor keeps its history in a SQLite database, so there is no history scan for it and capture is the reelier mcp --wrap recording proxy instead. Compile the recording into a SKILL.md with an assertion on every checkable step, then replay it with reelier run: deterministic, 0 LLM tokens at the default level, and read-only — recorded write steps do not re-fire without --allow-writes.

From the second run on, reelier diff <skill> compares the last two runs in .reelier/runs/<skill>.jsonl and prints a per-step verdict: SAME or DRIFTED, with the failing assertion as the why, exit code 1 on drift. That exit code is the product. Put replay-then-diff on the schedule and drift stops being a vibe — it is a red job naming the step that moved, instead of an agent quietly adapting around a change nobody approved.

The same mechanism catches drift from both directions. World drift: the workflow is frozen, upstream changes shape, an assertion fails loudly on replay. Agent drift proper: a new model ships — re-record the job on it and diff the fresh record against the frozen baseline, and steps added, removed, or flipped show exactly how the new model's behavior moved before it reaches production. When escalation is enabled (opt-in and BYOK: L1 re-evaluates the already-captured observation, L2 may re-execute a single read or idempotent-write step, destructive steps never escalate), the model still never edits the skill file — it proposes a narrow, validated patch, and the healing shows up in the next diff as soft drift rather than silence.

04What a diff cannot see

A tool-call diff sees the tool-call layer, full stop. Reasoning drift that produces identical tool calls is invisible to it, and the prose an agent writes or the judgment it exercises between calls is out of scope — evaluating those takes a model, and Reelier does not pretend otherwise. Detection is also only as good as capture: a gate built on a reconstructed history can miss what the reconstruction missed, which is why the wrap proxy — the lossless recording — is the higher-fidelity source than a scan.

And a replay-based gate guards a recorded workflow; it does not monitor an agent free-running in production. It answers a narrower question — did this known-good job still run the same — and answers it with an exit code. For the drift modes outside that boundary, the honest tools are evals and observability, not replay.