Answering "Why Did It Say That?": The Replay Procedure

Every audit of computed answers eventually reduces to one question about one number on one date. The replay procedure is the rehearsed answer — and every step of it runs on artifacts that already exist.

The questions arrive in different costumes. A customer writes in: your tool told me something different in March. An internal reviewer samples last quarter's outputs. An examiner asks what the calculator showed a specific user before a specific decision. Underneath, the ask is identical — reconstruct a computed answer, prove the reconstruction is faithful, and explain any difference from today's behavior. Firms that store envelopes can answer with a procedure instead of a scramble. Here it is, step by step, with the failure branches named.

The five steps

Step one: retrieve. Pull the stored envelope by your join keys — conversation, account, document. It carries the model, spec version, echoed inputs, outputs, sentinels, assumptions, cited constants, and the record hash; the trace-storage and envelope-storage guides exist so this step is a query, not an investigation.

Step two: verify integrity. Recompute the SHA-256 over the canonical JSON of model, specVersion, inputs, and outputs — the envelope's own record field states the recipe — and compare to the stored hash. A match proves the record is exactly what was written on the day it was written. This is the step that converts "our database says" into "here is the demonstration."

Step three: replay, if the version allows. Check the model's current spec version. If it matches the stored one, re-run the stored inputs against the live API: the outputs must reproduce byte for byte, and now the historical answer has been independently regenerated — the strongest form of substantiation available for a computed number. If the version has moved, skip to step four; the free API runs current versions only, and the hash verification in step two already established the record's fidelity without a re-run.

Step four: explain any delta. When today's answer differs from the stored one, the public changelog carries the explanation: the entries between the stored version and the current one state what moved, when, and why. The response to the disputant writes itself — the answer you received was computed under version X, whose behavior is documented here; version Y changed this, for this reason, on this date. A difference stops being an anomaly and becomes a cited history.

Step five: package. Assemble the envelope, the hash verification, the replay output or changelog citation, the relevant spec, and — where the answer relied on regulatory constants — the registry rows with their periods and sources. That bundle answers the who-what-when-how-verified questions in one artifact set, all of it reproducible by the recipient.

Rehearse before anyone asks

A procedure that runs first during a real dispute is a procedure that fails in front of an audience. The rehearsal is cheap: quarterly, sample a handful of stored envelopes, run the five steps end to end, and file the dated result. The rehearsal catches storage rot, join-key gaps, and drifted assumptions about the API while the stakes are zero — and the file of successful rehearsals is itself evidence, the kind that makes an examiner's sampling exercise short. Firms with assistant surfaces should rehearse through the trace store too: the conversation-to-trace-to-envelope join (/writing/tool-call-traces) is part of the path the real question will travel.

• ✓Write the five steps into your operations runbook with named owners.
• ✓Verify hashes at write time so step two never surprises you.
• ✓Rehearse quarterly on sampled envelopes; file the dated results.
• ✓Treat hash mismatches and missing envelopes as incidents about your pipeline, disclosed as such.
• ✓Keep the changelog bookmarked — step four is a citation, not a negotiation.

Why this procedure is even possible

Most computed products cannot run this procedure, and the reason is architectural: their answers never carried enough context to be reconstructed. The envelope was designed as the audit object this procedure consumes — self-describing, hashed, versioned, cited. The procedure is therefore less a compliance innovation than the harvest of decisions made upstream: reject-never-clamp keeps stored inputs meaningful; determinism keeps replays exact; public versioning keeps deltas explainable. Firms inherit the whole chain by storing one object per communicated answer. The chain's last link — running the procedure calmly when it matters — is the part no vendor can supply.