For software that decides who gets help
When a person wins an appeal AppealWhat it is. A formal request to have a benefits decision reviewed, usually at a hearing with an independent officer.Why it matters. When an appeal is won, it is proof the system got something wrong. Lookback treats that proof as a bug report. because the software was wrong, Lookback finds the bug, fixes it, and issues the restoration the law requires for every other household the same bug harmed. It may act alone only in a person's favor.
What happened on this project
1 appeal led to 413 households restored, $25,861 in total, with nobody else having to appeal.
Real USDA audit households. How it was measured.
Her food benefit was cut by $19 a month because the system allowed a $35 medical deduction instead of the $150 she was owed. She took a day off work, waited six weeks, and won.
Same rule, same cut. He never knew it was wrong. He is still short every month. With Lookback, Ana's win reaches Ben automatically.
The law already says agencies must restore lost benefits automatically 7 CFR 273.17What it is. The federal rule for restoring lost SNAP benefits. The agency "shall automatically take action to restore any benefits that were lost. No action by the household is necessary."Why it matters. The duty already exists. What is missing is a way to find the people. Restoration also reaches back only 12 months from discovery, so time matters.Source. They just cannot find the people, and every month an error stays undiscovered is a month lost for good The 12-month limitWhat it is. Under 7 CFR 273.17, restoration reaches back at most 12 months from the date the agency discovers the loss.Why it matters. Every month a bug goes undiscovered is a month of benefits lost for good. Finding bugs faster is money..
A hearing officer records the outcome and mentions Lookback.
The mention fires a GitLab Duo flow trigger within a second.
An agent reads the decision and reproduces the error with a test. No reproduction, no change.
Triage returns a JSON verdict; reproduce writes a failing regression test.
The agent writes the fix. Tests, scans and a replay of 90 days of real decisions must pass; a person clicks merge.
Replayability gate, replay gate, SAST, secret and container scanning. Agents have no merge tool.
It finds when the bug started and replays every decision since.
Bisect over git history with the new test; replay the ledger; re-run a sample on the original engine.
It records what each family is owed under the law. It never takes money away.
7 CFR 273.17, 12-month limit; corrections appended; adverse findings held for a person.
Every link is a live GitLab artifact. The households are real USDA audit households USDA quality control sampleWhat it is. Each year USDA audits a sample of real SNAP households and publishes the de-identified results. The FY2024 file has 44,891 households.Why it matters. It is a real population with the government's own verified answers, so our numbers are measured on reality, not invented cases.Source; the hearing decisions are constructed and labeled ConstructedWhat it is. Data we built for the demo, labeled as such. The hearing decision is modeled on published New York decisions; months beyond a household's sampled month carry its circumstances forward.Why it matters. Headline numbers that depend only on real sampled data are reported separately from numbers that use constructed timelines..
Run 1: medical deduction underpaid
Mention to merge request in three and a half minutes. After merge: 10,656 decisions replayed, 3 releases bisected, 0 adverse actions taken Cascade replayWhat it is. Following one household forward in time, because a benefit is set at certification and paid every month after.Why it matters. A wrong certification harms every month that depends on it. Replaying one month at a time misses most of the money..
Run 2: a family wrongly denied
A lookup one household size too small, no test covering it. The agent found the line unaided; the replay gate named exactly the 19 families; cascade recovered the closed months. A first run with no hearing text returned "unclear" and did nothing.
Benchmarks, baselines and the command that reproduces each number
Restorations are automatic. Anything adverse to a household goes to a person, and no agent holds a tool that could do otherwise Asymmetric autonomyWhat it is. The system may restore money on its own. It may never take money away, create a debt, or close a case on its own.Why it matters. Automated clawbacks are what made Australia's Robodebt a national scandal. Lookback cannot do it, by construction..
0 adverse actions across 14,030 harmed decisions found. The remedy engine asserts it.
Every amount comes from tested rules code. Agents read appeals and write code; the Duo CLI quotes the engine.
Hash-chained ledgers Decision ledgerWhat it is. An append-only log of every decision the system made, with its inputs, outputs, and the exact code version that made it. Each record is chained to the previous one by a hash.Why it matters. You can replay any past decision exactly, and nobody can quietly edit or delete one. Anyone can run the verify command.; python -m ledger.cli verify recomputes the chain.
Tried to trick it 30 ways; forbidden tools found in any agent's toolset: 0. The full safety page
Real. The runs above happened on this GitLab project, with links to every artifact. The households are from USDA's public audit file. The hearing decisions are constructed and labeled.
No. AI reads the appeal and writes code. Every amount is computed by tested rules code, and anything adverse to a household goes to a person.
Its test must reproduce the error, every other test must pass, the replay gate must pass, and it cannot merge anything itself.
Fixing stops future harm. It does not repay past harm, and under the law the clock on that is 12 months.
Yes. Any rules service that records decisions with inputs and a code version can be replayed. SNAP is first because its rules, audit data and legal duty are all public.