The Test That Asked for More Precision Than a Double Has

A randomized property run went red on a law that cannot be false: allowed loss plus disallowed loss equals total loss. It is the definition of a wash sale split, not a computation with room to be wrong.

totalLoss             7363034632.937324
allowed + disallowed  7363034632.937325
difference            9.54e-7
tolerance             5.00e-7   ← failed

The instinct is to go looking for the missing term. There was no missing term. Those two numbers are adjacent — there is no float64 between them — and the test was asking them to be closer than that.

Why a tolerance can become impossible

Doubles carry about 15 to 17 significant decimal digits, and they spend them relative to the size of the number rather than after the decimal point. Near 1.0, consecutive doubles differ by about 2.2e-16. Near 7.36 billion, consecutive doubles differ by about 1.6e-6 — a bit over a millionth of a dollar. That gap is the smallest difference the number system can express at that magnitude. Nothing can be closer without being identical.

The assertion was toBeCloseTo(x, 6), which most test frameworks define as a fixed absolute tolerance — here, 5e-7. At small dollar amounts that is strict and useful. At 7.36 billion it is three times smaller than the distance between neighboring representable numbers, so it is not strict, it is unsatisfiable. Correct arithmetic fails it. Which inputs fail is a matter of which random seed the run drew.

A tolerance below one representable step is not a strict test. It is a coin flip wearing the costume of a strict test.

The specific difference above was 0.58 of a single float step. The two sides of the identity were as equal as double-precision arithmetic permits them to be, and the test called it a defect.

How often it was firing

This is the part that changed how seriously we took it. A flaky test that fails once looks like an anomaly; a flaky test with a measured rate is a budget line. So we measured it, over the same input domain the property suite draws from.

$ pnpm money-tolerance:flake

Wash-sale conservation over 200,000 random cases

  fixed 5e-7 tolerance   62 failures  (0.031%)
  scaled tolerance       0 failures

  worst genuine error    0.93 float steps, at $1.13e+6
  headroom allowed       16 float steps

About three hundredths of a percent of cases — between 52 and 62 per 200,000 across repeated runs. That sounds negligible until you multiply it by how a property test works. Each run draws 200 cases, so the loss-split assertion alone had roughly a 6% chance of reddening any given build. Four other assertions carried the same defect and would each add their own share, which we did not measure separately. Somewhere upward of one build in seventeen, from a class of bug where the engine is behaving perfectly.

Across all 200,000 cases, the worst genuine discrepancy was 0.93 float steps. Not 0.93 dollars, not 0.93 cents — under one representable difference. Every single one of those failures was a false alarm.

The audit, not the patch

The tempting fix is to loosen the one assertion that failed and move on. That leaves the other instances of the same bug in place, waiting for their own unlucky seed. So instead we took every money assertion inside a property test and compared its tolerance against the largest magnitude that assertion's own generator can produce. The unit of comparison is float steps of headroom: how many representable differences fit inside the tolerance at the worst case.

Five assertions had under four steps of headroom and four of them had under one — meaning they could not be satisfied at the top of their own input range. Only one had ever actually failed. The other four were waiting.

The bottom rows are the useful contrast. An assertion on ownership fractions summing to 1.0 has over two million steps of headroom, because the quantity is bounded near 1 and a fixed tolerance is exactly right for it. The defect is not that fixed tolerances are wrong; it is that a fixed tolerance on an unbounded quantity is a bug that hides until the generator reaches far enough.

Scaling the tolerance with the magnitude

Floating-point error scales with the size of the numbers involved, so the tolerance has to as well. A conservation identity is reached through a handful of multiplications and additions, each contributing under a step of error, compounding roughly with the operation count.

export const MONEY_ULPS = 16;
export const MONEY_FLOOR = 5e-7;

export function moneyTolerance(magnitude: number, floor = MONEY_FLOOR): number {
  if (!Number.isFinite(magnitude)) return floor;
  return Math.max(floor, Math.abs(magnitude) * Number.EPSILON * MONEY_ULPS);
}

The floor matters as much as the scaling. Below roughly $2 million the floor governs, so every check that was previously satisfiable stays exactly as strict as it was. This is not a test being relaxed to go green — it loosens nothing that was achievable in the first place, and the boundary is where the old tolerance stopped being achievable at all.

Does it still catch anything?

That is the question worth being suspicious about, because a tolerance chosen to make a failure go away usually does so by catching nothing. At the magnitude where the false alarm happened, one float step is 1.6e-6, so sixteen steps is 2.6e-5. A one-cent discrepancy at that magnitude is about 6,100 float steps. A dropped term of any consequence is hundreds of billions.

So the tolerance sits about 17× above the worst rounding we could produce in 200,000 tries, and still 382× tighter than a single cent — near the bottom of a band about four orders of magnitude wide that separates rounding from the smallest error anyone would care about. Nothing meaningful fits underneath it. The test suite asserts this directly: a one-cent error at the same magnitude still throws.

Make the failure message say which kind it is

The reason this took a while to place was not the mathematics. It was that the failure message said only that two numbers were not close enough, which is exactly what a real conservation bug looks like too. The assertion now reports the difference in float steps alongside the tolerance and the magnitude.

allowedLoss + disallowedLoss = totalLoss: 7363034632.937325 != 7363034632.937324
  difference 9.537e-7 (0.6 float steps)
  tolerance  2.616e-5 at magnitude 7.363e+9
  A difference of a few float steps is rounding; this is larger than that.

Sub-one-step is arithmetic doing its best. Thousands of steps is a bug. A message that reports the difference in units of the number system tells you which one you are looking at before you start reading the engine.

What we would tell someone testing money math

Property tests over financial models are worth having — they check laws across a whole domain rather than at the handful of points an example test reaches, and they are how we found two genuine unit bugs earlier in the same audit. But a generator that produces billions and an assertion written for thousands make a test that reports on luck.

The one-line check is worth running against your own suite: take the largest value your generator can produce, multiply it by 2.22e-16, and compare that to your tolerance. If the tolerance is smaller, the test cannot pass at the top of its own range, and it will tell you so eventually, on a day you would rather it did not.