The response envelope was designed as an audit artifact, but it doubles as a telemetry source. The observability question is which fields belong in your logs, which belong in your metrics, and which belong nowhere.
A product that embeds verified math has a new dependency to watch, and the standard dashboards do not quite fit: uptime and latency describe whether answers arrive, not whether they are the answers you integrated against. The useful telemetry comes from the envelope itself. Every response already carries the model name, the spec version, the sentinel annotations, and a hash — structured fields that turn into metrics with no parsing. Here is the layer, from what to record to what should page someone.
What to log per request
Log the envelope's metadata, not its contents. For every call: the model name, ok true or false, and your own latency measurement; for successes, the specVersion, any triggered sentinel fields, and the record.sha256. For failures, log the field names from the errors array — not the values the user submitted, just which fields failed. That set reconstructs everything operational about the call while keeping the log free of financial details.
The five metrics
| Metric | Source | What movement means |
|---|---|---|
| Validation failure rate by field | errors array field names | Your form design has a gap — a control permits what the contract rejects |
| Sentinel trigger rate | sentinels with triggered true | A product signal — how often real users hit never-breaks-even or not-by-100 answers |
| 429 count | HTTP status | Should sit at zero — any sustained value means a loop, not load |
| Latency p50 and p95 | Your own clocks | The ordinary health signal — budget it like any dependency |
| Observed specVersion per model | Envelope specVersion | Any change is a release you should be reading the changelog for |
Three of the five deserve a note. The validation-failure metric is form-design telemetry, not an error budget: a spike on one field after a UI change means the change let out-of-domain values through, and the fix is in the control, not the API. The sentinel metric is the strange one — it measures honesty events. A refinance tool whose Infinity rate jumps in a rising-rate month is working correctly and telling you something about your users' situations; alert on discontinuities, not on the level. And the specVersion metric is your second drift detector: the CI fixture catches version changes on a schedule, while production observation catches them the moment live traffic sees one.
What pages a human
- 429s sustained above zero — find the loop, not the limit.
- ok false rate spiking on a field that was quiet — a form or mapping regression.
- specVersion observed in production differing from the version your fixture pinned.
- Latency breaching the budget you set when you designed the calling path.
- Any response failing your own hash re-verification, if you verify at write time — this should never fire, which is why it should page.
The last item costs one SHA-256 per stored envelope and is the highest-signal alert on the list precisely because its expected rate is zero. A hash mismatch at write time means your serialization mangled the envelope between receipt and storage — a bug you want at write time, not at audit time.
Dashboards nobody has to trust
The quiet advantage of envelope-driven observability is that the numbers are reproducible. A latency chart is your measurement; nobody else can check it. But a specVersion timeline, a sentinel rate, a validation-failure profile — those derive from structured fields any second system would extract identically, and the stored envelopes back-test all of them. When a stakeholder asks how often the tool tells users a refinance never breaks even, the answer comes with the same provenance as the answers themselves.
Worthune's own posture here is worth copying: the platform publishes its aggregate usage at /api/v1/telemetry — daily hit counts per surface, thirty-day window — with only aggregate counters stored, never IPs, user agents, or inputs. Aggregate-only is a design position, not a limitation: the operational questions worth answering are about shapes and rates, and shapes and rates need no personal detail. An integrator whose math-layer telemetry would survive publication is holding exactly the right data.
Sources
- [1] Worthune public telemetry endpoint. https://worthune.com/api/v1/telemetry
- [2] Worthune API documentation. https://worthune.com/docs
- [3] Worthune writing: Version Pinning in CI. https://worthune.com/writing/version-pinning-ci