Most AI systems log something. Very few log the thing a regulator will actually ask for.
The test is specific: eleven months from now, a customer disputes a decision. The model has been upgraded twice, the prompt rewritten, the retrieval index rebuilt. Can you show what happened and why?
If the answer requires anyone to remember anything, the trail is not an audit trail.
The eight things to capture
- The input, as received. Not as normalised — the raw submission, so a dispute about what was actually provided has an answer.
- The retrieved context or features. Which documents came back, which features were computed, what values they held. This is the most commonly omitted item and usually the most important, because a wrong answer is far more often a retrieval failure than a reasoning one.
- Model and prompt version. Identifiers that resolve to an artefact you still hold. "GPT-class model, September" is not a version.
- Tool calls and their results. What the system did, in order, with what came back — including failures and retries.
- The raw output before any post-processing, plus the processed result.
- Confidence, and the threshold in force. A score means nothing without the cutoff it was compared against, and thresholds change.
- Human involvement. Whether it was reviewed, by whom, and whether they agreed or overrode. An override is the most valuable record in the system and the one most often left out.
- Timestamps on each stage, which is what lets you correlate with an incident later.
Why a debugging log will not do
The two are optimised for opposite things.
| Debug log | Audit trail | |
|---|---|---|
| Question answered | What is happening now? | What happened then, and why? |
| Time horizon | Hours to days | Years |
| Retention | Rotated | Fixed, defensible period |
| Granularity | Whatever helped an engineer | Fixed schema, every field, every time |
| Mutability | Freely rotated and pruned | Append-only |
Teams discover the difference at the worst moment: the incident is a year old, the logs rotated after ninety days, and the model that produced the decision no longer exists.
Versioning is the part that fails quietly
Logging "model version 4" is useless if version 4 has been overwritten. The artefacts have to be retained alongside the reference: the model or its identifier with the provider's own version, the exact prompt text, the retrieval index or a snapshot identifier, and the configuration in force.
This is why a model and prompt registry is not administrative overhead. It is the thing that makes the log mean something.
Reproducibility, and its honest limits
The strongest position is being able to re-run the decision and get the same result. That is achievable with deterministic settings and retained artefacts, and worth engineering for.
It is not always achievable — a provider may retire a model version, and non-determinism can be inherent. Where exact reproduction is impossible, the fallback is a complete record of what happened plus the reasoning artefacts, and being straightforward about the limit. A claim of reproducibility that fails under examination is worse than an accurate account of what was retained.
The log is a data store
An audit trail for a system processing personal data contains personal data. It therefore inherits every obligation the rest of the system carries: access control, retention limits, inclusion in deletion requests, and the same residency constraints.
The recurring failure is a logging pipeline built by engineers for debugging that quietly accumulates prompts and inputs outside the boundary the rest of the architecture respects — often in a third-party observability tool in another jurisdiction.
Retention, stated rather than assumed
Sector rules usually decide this rather than AI-specific ones: lending and insurance decisions commonly need to be defensible for years. That has to be balanced against storage limitation under the DPDP Act, which expects data not to be kept indefinitely without purpose.
The defensible answer is a stated period tied to a stated purpose, applied consistently, rather than either extreme.
Build it first
Retrofitting an audit trail onto a working system means reconstructing which version of what was in force during a period nobody instrumented — and the answer is usually that it cannot be done.
In regulated Indian sectors this is not hygiene. It is the difference between a system that can go live and a pilot that stays one.