How it works

Lookback is a set of steps that run on GitLab after a benefits appeal is won. The steps are the same every time, and every step leaves something you can click.

Lookback is a GitLab Duo flow plus a CI pipeline. The flow does the reasoning (triage, test, fix). The pipeline does everything that must be exact and repeatable (replay, money, ledgers, releases). Agents never compute amounts and cannot merge.

From a hearing decision to money restored. The agents write code; the stage after merge is deterministic and runs without AI. Anything adverse to a household leaves the automatic path.

Follow the real run

This is the thread of work item #2, in order. Each row links to the live artifact.

#	What happened	Evidence
1	A hearing officer wrote up the decision as a work item and mentioned the Lookback account.Work item created with label `appeal::overturned`; a mention of `@ai-lookback-appeal-to-fix-gitlab-ai-hackathon` fired the flow trigger within one second.	Work item #2
2	Triage read the decision and the code, and said: the software is wrong, here is the rule, here is the right amount.Agent `triage` returned JSON: verdict `software_error`, rule area `excess_medical_deduction`, citation 7 CFR 273.9(d)(3), facts, recorded $23, correct $42.	Flow session on the work item
3	Reproduce wrote a test with the appellant's numbers and showed the current code fails it.Agent `reproduce` created `tests/regression/test_appeal_2.py`, asserting 4200 cents, and ran it: AssertionError, actual 2300.	The test
4	Fix changed one line and opened a merge request explaining who might be affected.Agent `fix`: `min(expenses, threshold)` became `max(0, expenses - threshold)` with the citation in a comment; branch `fix/appeal-2`; MR labeled `lookback::remediate`.	Merge request !1
5	The merge request's checks all passed, and it was merged.MR pipeline: unit and property tests, replayability gate, SAST, secret detection, container build and scan. The agent cannot merge; a human or auto-merge does.	MR pipeline
6	The fixed engine was deployed.Image pushed to Artifact Registry and deployed to Cloud Run as a tagged revision, with no stored keys (Workload Identity Federation).	Pipeline
7	Lookback found when the bug started: it tried the new test on each past release.Bisect over the engine's git history: 3 releases, 3 probes, first bad commit `08c5a271`.	Job `lookback-replay`
8	It replayed every decision the broken code had made and compared.Replay of 10,656 ledger records through the fix; 410 re-run on the engine that made them, all matched. Differences: OVERPAID 9, UNDERPAID 410, WRONGLY_DENIED 3.	Job artifact `summary.json`
9	It worked out what each family was owed and restored it. Overpayments went to a person.Remedy under 7 CFR 273.17: $25,861 to 413 households, $0 lost to the 12-month limit, 9 adverse findings held, 422 correction records appended to the ledger .	Release
10	A summary was posted back on the original work item.Release description and work item note rendered from `summary.json`.	Work item #2

Read what the agents actually said

Every session's trace is exported unedited (secrets redacted) into the repository, so the agent's reasoning is readable even when the platform is slow.

What the ledger looks like

Every decision is written down with the exact version of the code that made it. A fix never rewrites history; it adds a new line that says what the old one should have been.

Records are canonical JSON, hash-chained, with the engine commit and image digest. Corrections and adverse findings are appended and reference the record they concern.

Corrections and adverse findings are new records that point back. The chain makes any edit visible; the engine references make any record re-runnable.

Ask it anything, and it cannot lie about the numbers

A caseworker can ask GitLab Duo: "why does this household get $42?" or "what did the last remediation do?". The agent is not allowed to work the sum out itself; it has to ask the rules engine and quote the answer, regulation included. It cannot change anything. Today that works from GitLab's Duo CLI; the Duo Chat panel on GitLab.com cannot reach a project's own tool server yet, and there the agent declines rather than guesses.

The decider exposes an MCP server (Streamable HTTP, protocol 2025-11-25) with six tools, every one annotated readOnlyHint: true: decide_household, explain_decision, list_remediations, get_remediation, verify_ledger, lookback_guarantees. .gitlab/duo/mcp.json registers it for the project; agents/ask_lookback.yaml is the chat agent in the AI Catalog whose system prompt forbids computing an amount without a tool call. 12 transport conformance tests in tests/test_mcp.py; scripts/mcp_probe.py passes 14 checks against the live server. Proven on the platform's own client: the GitLab Duo CLI, run from this repository, loaded mcp.json, called explain_decision and quoted $42.00 with thirteen regulation steps (session 9261760). One platform limit, recorded as found: web sessions on GitLab.com (the chat panel and custom flows) cannot reach a project MCP server yet. A flow that declared the tools anyway (session 9237114) answered that it would not compute the amount without them, which is the guarantee holding under a missing tool.

And before any future change

Every proposed code change is tested against the last 90 days of real decisions. If any real household's result would change, the change is blocked unless it comes from an appeal or a change in the law.

The replay gate runs on every merge request that touches rules/. It fetches the ledger, verifies the hash chain, replays the window, and fails on unexplained flips. Explained flips are those whose MR references an appeal or a policy change.

Passed, merged

A pure refactor

Renamed a variable. 14,205 decisions replayed, none changed.

Merge request !2

Blocked, closed

A quiet rule change

Changed > to >= at the income limit, which would deny a household exactly at the limit. Caught by the unit tests. The replay gate passed, because no real household in the window sits exactly on the limit, which is why both layers exist.

Merge request !3

The parts

rules/        pure rules engine, integer cents, every step cites its regulation
ledger/       append-only hash-chained ledger; python -m ledger.cli verify
gates/        replayability gate (no clock, randomness, I/O, floats); replay gate
replay/       release builds, bisect, replay, cascade
remedy/       7 CFR 273.17: 12-month limit, asymmetric autonomy
lookback/     the post-merge stage: git history, record, stage, publish
decider/      HTTP service, one image per release; MCP server with six read-only tools
agents/       Ask Lookback, a Duo Chat agent that must call a tool before quoting an amount
.gitlab/duo/  Flow A (three agents), Flow B (two agents), Flow C (negative control), mcp.json, one-host sandbox config

The code is open, MIT licensed, and the README explains each part. See the repository.