Every Stand-In Is a Theory

I spent a day rebuilding a test suite that had been quarantined for a week, and the two hardest bugs in it were the same bug wearing different clothes. Neither was in the code under test. Both were in the things standing in for the real ones.

The suite had been pulled from the run because it hung. Not failed — hung, in an infinite render loop, straight through a sixty-minute CI budget until the job was killed. The cause, once someone found it, was a few characters wide. The mock replacing a hook returned an object literal written inline, so every call produced a new object. The component had an effect keyed on that object. New object, effect fires, render, new object.

That mock returned the correct fields with the correct values every single time. What it got wrong was being a different object each time, and nothing in the shape of the thing says otherwise. The real hook returns the same reference until its inputs change. That is not a value, it is not in the signature, and it was the only property the component actually cared about.

That fix predated me. I read it in the history before I started, which makes what happened next worse.

The second one I walked into myself

Rebuilt suite, seventy-three tests. Most went green on the first run and the rest crashed inside React's scheduler in a way that made no sense from my assertions.

The test environment carries a polyfill for `MessageChannel`, because the runtime it executes in doesn't provide one. It's about eight lines. You post a message on one port, the handler on the other port receives it. Correct in every respect I would have thought to check.

It delivered synchronously.

React's scheduler uses a message channel precisely to get a deferred callback — it posts a message in order to yield, and expects to be re-entered later, from a clean stack. Deliver immediately and it gets re-entered from inside itself, and the whole thing comes apart. The polyfill implemented the interface exactly and inverted the one property the interface never mentions: that message delivery is asynchronous. It is asynchronous by definition. That is most of why the thing exists.

Same bug. A stand-in that was right about every value and wrong about a property that isn't a value.

Three tries to fake eight lines

First fix: deliver on a `setTimeout`. Asynchronous — correct on exactly the axis I had just been taught about. It broke three other suites, because `setTimeout` is a thing fake timers control and message delivery isn't. I had fixed the timing dimension and broken the controllability dimension.

Second fix: deliver on a microtask. Asynchronous, and invisible to fake timers. That held.

Then I read my own diff and found I had made both ports deliver, where the original only did the one direction React uses. A fake that does more than the real thing is also a wrong fake; it just fails later and further away. I cut it back to the minimum.

Three attempts to write eight lines of a thing whose entire public behaviour is "post a message, someone gets it."

What I would generalise

A real object has an unbounded number of properties. A stand-in has the ones you listed. Writing a mock is therefore not a mechanical act — it is a claim about which properties of the original are load-bearing, made under uncertainty, and the bugs live in the ones you never thought to name.

They are systematically the same kind of property, too. Not values. Values are easy, values are what the type tells you about, values are what you would think to check. The ones that bite are identity — is this the same object as last time — and timing, does this happen now or later, and controllability, can the test's other machinery reach into it. None of those appear in a signature. All of them are things real code depends on constantly and silently.

The same shape turned up once more that day, one level higher, and I didn't recognise it as the same shape until afterwards. The suite passed locally, twice, and then failed in CI. Not flakily: the tests were hitting a per-test time limit, because the CI runner is roughly two and a half times slower than my laptop and several tests mount a real form for each of eight steps. My machine had been standing in for the runner. It matched on every dimension I had thought about and differed on one I hadn't — how fast it is.

Which is the part that isn't really about mocks. Anything you put in the place of the real thing — a fake, a polyfill, a staging environment, your own laptop, a summary of a document you didn't read — is a theory about which of its properties matter. The theory is usually right, which is why we keep doing it. When it is wrong, it will not be wrong about a value. It will be wrong about something you never thought of as a property at all.