Version Pinning in CI: Failing the Build When Behavior Would Drift

Every Worthune response stamps the specVersion that produced it and a SHA-256 record over the answer. Those two fields are a drift detector — if your CI stores them.

The fear behind every vendor integration is silent change: the number your product showed in March is not the number the same inputs produce in June, and nobody told you. Worthune's contract answer is that behavior changes only with a spec version bump, published in the changelog. This piece turns that contract into a CI job: a fixture that fails the build the moment behavior would differ from what you integrated against.

The fixture

Pick a handful of representative calls per model you depend on — the happy path, one edge your product cares about, one sentinel case. For each, store the request inputs, the response outputs, the specVersion, and the record.sha256, exactly as returned on the day you integrated. Commit the fixture next to your integration tests.

for each fixture: response = POST /api/v1/models/{model} (fixture.inputs);  assert response.specVersion == fixture.specVersion;  assert response.record.sha256 == fixture.sha256

Reading the two failure modes

A specVersion mismatch is the designed outcome. Model behavior changed, the version says so, and the changelog entry says why and what moved. Your job is a review, not a firefight: read the entry, re-run your fixtures against the new version, decide whether your product copy or downstream logic cares, update the fixture, merge. The build being red until a human looks is the feature — the alternative is the June number diverging from the March number with nobody assigned to notice.

A hash mismatch at the same spec version is a different animal. The record is computed over the model name, the spec version, your inputs, and the outputs, in canonical key-sorted JSON — so if the version matched and the hash did not, the outputs changed under a pinned version. The platform's dual-implementation harness and changelog discipline exist precisely so this does not happen; if your fixture ever shows it, you have found something worth reporting, and your build being red is correct.

Cadence and hygiene

Run the drift job on a schedule — daily or weekly — not just on your own commits, because the thing it watches is external. Keep the fixture small enough to read in a diff: five to ten cases per model is plenty, since the platform itself verifies hundreds per model per release; your fixture is not re-verifying the math, it is verifying that the math you integrated against is the math still running. When a version bump lands, update the fixture in its own commit with the changelog link in the message — that commit history becomes your integration's behavioral audit trail, for free.

• ✓Store inputs, outputs, specVersion, and record.sha256 per fixture case.
• ✓Assert version first, hash second — they mean different things.
• ✓Include one triggered-sentinel case for each model that documents a sentinel.
• ✓Run on a schedule, not only on push.
• ✓Update fixtures in dedicated commits linking the changelog entry.

The local-verification variant

For the paranoid tier (compliance environments, regulated deployments), add a third assertion that does not call the network at all: recompute the SHA-256 from the stored fixture fields and confirm it matches the stored hash. That check proves your fixture itself has not been tampered with or corrupted — the stored evidence stays evidence. The SDKs ship the canonical-JSON helper; the recipe is also stated in every response, so the check is a few lines in any language.

What this buys at review time

When a compliance reviewer or an enterprise customer asks how you know the vendor has not changed the math under you, the answer stops being a paragraph about trust and becomes a link to a CI job: here is the fixture, here is the schedule, here is the commit history of every version bump we reviewed, with changelog links. The question that sinks most vendor integrations in review — unmonitored behavioral dependency — arrives pre-answered.