Your Retention Policy Deletes the Evidence You Will Need
A complaint about an automated decision arrives fourteen months after the decision. Somebody asks what the system was doing that day. The logs rotated at ninety days, the prompt was edited twice since, and the model version behind the endpoint was replaced by the provider in the spring. The decision is now unreconstructable — and every accountability obligation you have was written on the assumption that it would not be.
Every Obligation Assumes a Record Exists
Read the AI duties now landing on ordinary businesses and notice how many of them are retrospective. Explain the reason for an adverse decision. Demonstrate that a human was meaningfully involved. Show that the system performed within tolerance across groups. Respond to a request to correct an inference. Produce documentation to a supervisory body on request.
Not one of those is satisfiable from the live system. They are all satisfied from records, and they are all asked at a moment you do not choose. Logging is therefore not an engineering hygiene concern that governance borrows occasionally — it is the substrate the entire compliance posture rests on, and it is owned by a team whose default settings were chosen for debugging.
Which is why the failure is so consistent. Nobody decided to delete the evidence. A retention default set for disk cost quietly outranked an obligation nobody translated into a configuration value.
What the Record Has to Contain
The test for completeness is simple: could a competent person, with only this record, describe what happened and why the output was what it was? That usually requires:
- Time and system identity. Timestamp with timezone, the registered system this belongs to, and the environment. Ties the event back to your inventory entry.
- Exact model version. Not the family, the pinned version identifier. If you call an alias that the provider repoints, log whatever version string comes back in the response, because the alias tells you nothing after the fact.
- Prompt and configuration version. Template identifier plus revision, temperature and other sampling parameters, tool availability, retrieval index version. Prompt edits are code changes and need the same traceability.
- Input reference. A stable identifier and a hash rather than a copy, wherever the source system can be relied on to still hold it.
- Raw output, before post-processing. Including any score, probability or confidence. Teams routinely log the tidy final value and lose the number the decision was actually made on.
- Downstream action. What the workflow did: approved, routed, ranked, suppressed, escalated. The output alone does not establish the consequence to the person.
- Human involvement. Who reviewed, when, what they saw, and whether they accepted or overrode. Where a claim of meaningful human oversight is being made, this field is the claim.
- Failures and fallbacks. Timeouts, retries, guardrail blocks, and any default applied when the model did not return. Fallback paths produce decisions too, and they are almost never logged as decisions.
Versioning Is the Field Everyone Forgets
Of everything above, model and prompt versioning is the one most often missing and the hardest to reconstruct later. The reason is that both change through paths that are not treated as releases. A prompt gets a wording tweak in a config file. A provider deprecates a version and the alias silently moves. A retrieval index is rebuilt. Individually trivial; collectively they mean the pipeline that ran in March cannot be reassembled in September.
Two habits fix it. Pin explicit versions in production rather than calling floating aliases, so the change is a deliberate act with a date. And record prompts in version control with an identifier the log can reference, so "which instruction produced this" has an answer that does not depend on someone's memory of a Slack thread.
Resolving the Minimisation Conflict
Privacy counsel wants less personal data held for less time. Accountability wants a reconstructable history for years. These look irreconcilable when the audit log is designed as a copy of everything that passed through the system, and mostly dissolve when it is not.
Separate the two stores. The decision record — identifiers, versions, scores, outcomes, reviewer actions, hashes — is small, has a clear accountability purpose, and can be retained for the full period with minimal personal content. The payload — the applicant's text, the uploaded document, the transcript — stays in its source system on the retention schedule that already governs it. Your log points at it rather than copying it.
This has an operational benefit beyond the legal one: when a deletion request lands, you can honour it against the source without shredding the audit trail that proves how the system behaved. A design that mixes both forces a choice between two obligations every time someone exercises a right.
Setting the Retention Number
Work backwards from consequences rather than forwards from infrastructure defaults.
- List the claims each system could produce. Employment decisions, credit or pricing outcomes, insurance determinations, consumer protection, contract disputes.
- Take the longest applicable limitation period. That is a floor, not a target, and for employment and consumer matters it is measured in years.
- Add discovery lag. The clock often starts when the person learns of the harm, which for an opaque automated decision can be long after it was made.
- Check sector rules. Financial, health and employment record obligations may already prescribe a period that exceeds anything AI-specific.
- Write it into the system's inventory entry. Retention that lives only in a pipeline config is retention nobody reviews.
Then verify the number is real. Confirm the log survives the full period end to end — through the aggregation layer, the archive tier and the cold storage lifecycle rule that somebody set two years ago. Policies stating a retention period that the infrastructure silently contradicts are common, and they are worse than no policy, because they are a documented commitment you are not meeting.
Integrity Matters as Much as Existence
A record that anyone with production access can edit is worth much less in a dispute than one that cannot be quietly changed. Append-only storage, restricted write paths, and an access log for the log itself are inexpensive and convert your records from an assertion into evidence. The same measures protect your team: an unalterable trail is the thing that demonstrates a decision was not tampered with after the complaint arrived.
Also decide, before you need it, who can read these logs. Decision records about individuals are sensitive by construction, and a well-intentioned audit trail that half the company can query is a new privacy exposure of its own.
Common Questions
Do we need this for internal productivity tools?
Proportionately. A drafting assistant that produces text a human then edits carries little decision risk and needs little more than usage records. The moment output influences an outcome for a person — scheduling, performance, screening, access — it belongs on the full logging standard.
Is our normal application logging enough?
Rarely. Application logs are built for debugging: short retention, unstructured, focused on errors rather than successful decisions. AI decision records need the opposite profile — structured, long-lived, and complete for the ordinary path, not just the failures.
What about volume and cost at scale?
The decision record is small when it stores references and versions rather than payloads, so millions of rows remain inexpensive. The costs people fear come from logging full prompts and responses indefinitely, which is also the choice that creates the privacy problem. Trim what you store, not how long you keep it.
Should logs be available to the affected person?
Assume material parts will be. Access and explanation rights reach the reasoning behind a decision, and litigation reaches further. Write every field as though it will be read by the person it describes and by their representative, which is a good discipline for the free-text fields in particular.
How do we log a multi-step agent?
Per step, with a shared trace identifier. Record each tool call, each retrieved document version and each intermediate output, because with an agent the consequential mistake is usually several steps upstream of the final answer and is invisible in an end-to-end record.
What if a vendor holds the logs?
Get the retention period, the export path and the access terms in writing before you rely on it, then keep your own record anyway. Vendor retention is typically far shorter than your limitation exposure, and a contract termination can end your access to the evidence entirely.
What You Publish Is Also a Record
Retention statements, human-review claims and AI feature descriptions on your site are representations you can be held to — and they are read by buyers, by anyone assessing you, and by AI assistants summarising your practices to a prospect.
Run a free scan of your site to see what you are currently telling people about AI and data handling.