Learn

The 60-second explainer

Some software decides whether people get food assistance, health coverage, or unemployment pay. It follows rules written in law. Sometimes the software gets a rule wrong, and everyone that rule applies to gets the wrong answer.

A few of those people appeal. When one of them wins, the agency fixes that one person's case. Everyone else stays wrong, because nobody looks for them.

Lookback changes what happens after that win. It treats the win as a bug report. An AI agent reads the hearing decision, writes a test that proves the bug, and fixes the code. Then a pipeline finds every past decision the broken code made, works out what each household is owed under the law, and restores it. If the bug had the opposite effect, overpaying people, Lookback stops and hands those to a person. It can never take money away on its own.

Federal law already requires agencies to restore lost benefits automatically when they discover an error. Lookback is a way to discover.

If you are new to this

What is a software decision?

A program takes facts about a person (income, household size, rent) and produces an answer (eligible, $291 a month). The same facts should always give the same answer.

What is an appeal?

A formal challenge to a decision. For SNAP it is called a fair hearing: an independent officer looks at the facts and the law and rules on whether the agency was right.

What is a bug?

A mistake in the program. Here it means the code does not do what the law says, for example applying a cap the law says does not apply to elderly households.

What is CI/CD?

Automation that runs every time code changes: tests, security scans, building and deploying. Lookback adds a step to it: finding and repaying the people a fixed bug harmed.

What is replay?

Running past decisions again, with the same facts, through the fixed code, and comparing. Where the answers differ, someone was harmed.

What is a ledger?

A record of every decision, in order, where each entry is locked to the one before it. You can check that nothing was changed, and you can re-run any entry.

Six weeks, or six minutes

Today: one person appeals

Notice arrives with the wrong amount
Appeal filed (most people never do)
About six weeks to a hearing
Decision: the agency was wrong
One household restored
Everyone else: nothing

With Lookback

Hearing officer records the outcome and mentions Lookback
Agent finds the bug, writes the test and the fix: 3.5 minutes
A person clicks merge; gates, scans, build, deploy, replay, remedy: 5 minutes of pipeline
413 households restored, $25,861
Overpayments held for a person, never acted on

Who it is for

State agencies

Meet the duty to restore automatically, and shorten the time an error goes undiscovered.

Vendors

Add a remediation stage to any rules engine that follows a small contract.

Legal aid

Turn one win into many.

Any team deciding money about people

Payroll, billing, insurance, fees. Same loop.

Glossary

7 CFR 273.17: The federal rule for restoring lost SNAP benefits. The agency "shall automatically take action to restore any benefits that were lost. No action by the household is necessary." Source
Agreement with USDA: On every in-scope audited household, our engine's benefit compared with the benefit USDA's reviewers calculated.
Analyst query: How claims systems do mass reprocessing today. An analyst selects records by reason code and date range, then re-runs them.
Appeal: A formal request to have a benefits decision reviewed, usually at a hearing with an independent officer.
Appeal rate: The share of people who were denied and formally challenged the decision. Source
Asymmetric autonomy: The system may restore money on its own. It may never take money away, create a debt, or close a case on its own.
Bisect: A search over past releases to find the first one where the appeal's test fails.
Cascade replay: Following one household forward in time, because a benefit is set at certification and paid every month after.
Cloud Run: Google Cloud's service for running containers. Each release of the rules engine becomes a tagged revision.
Colorado SNAP hearings: In FY2019 Colorado SNAP issued about 111,500 denials, cuts and overpayment claims. 262 reached a fair hearing. Source
Constructed: Data we built for the demo, labeled as such. The hearing decision is modeled on published New York decisions; months beyond a household's sampled month carry its circumstances forward.
Decision ledger: An append-only log of every decision the system made, with its inputs, outputs, and the exact code version that made it. Each record is chained to the previous one by a hash.
GitLab Duo flow: A multi-step AI workflow defined in YAML and run by GitLab. Lookback's flow has three agents, triage, reproduce and fix, each with the smallest set of tools it needs.
MCP server: Model Context Protocol, a standard way for an AI assistant to call tools on another system. Lookback's server offers six tools, and every one only reads.
MIT License: A permissive open-source license. Anyone may use, change and redistribute this code.
Overturn rate: The share of appeals that succeeded, fully or partly. Source
Recall and precision: Recall is the share of harmed households found. Precision is the share of flagged households that were really harmed.
Replay: Re-running past decisions through the fixed code, with the same inputs, and comparing outcomes.
Replay gate: Before any change merges, the last 90 days of real decisions are replayed through the proposed code.
SNAP: The Supplemental Nutrition Assistance Program, the US food assistance program, run by states under federal rules.
Supervised: The hackathon's middle autonomy level. Agents run a multi-step workflow on their own, clearing gates and making decisions; a person approves the outcome, not each step.
The 12-month limit: Under 7 CFR 273.17, restoration reaches back at most 12 months from the date the agency discovers the loss.
Tool governance: A GitLab setting that controls which actions an AI agent may take at all.
USDA quality control sample: Each year USDA audits a sample of real SNAP households and publishes the de-identified results. The FY2024 file has 44,891 households. Source
Waiting for appeals: What happens today. Each harmed household is found only if it appeals. We apply real appeal rates to the harmed count.