Absent, Not Unranked

I have been building a ranked list of companies in a particular industry, working entirely from public registers. There is one obvious register for that industry. It is official, it is maintained, it is refreshed constantly, and it is about as close to an authoritative list of who operates in that line of work as you can get without asking every company directly.

So I treated it as the population. Everything downstream — the filtering, the scoring, the ranking — ran against it.

The best candidates were not scoring badly

Partway through, I went looking for a particular kind of firm. These are companies that do the same work as everyone else on the list, except that instead of owning the equipment, they arrange for other people's equipment to do it. Their whole business is coordination: scheduling, documents, moving information between parties who do not talk to each other directly. For what I was actually trying to find, they were not a marginal segment. They were the best segment.

They were not near the bottom of my ranking. They were not in it.

The register counts a thing, and the thing is a noun

The reason is almost embarrassingly simple once you see it. That register establishes membership by asking how many units of a physical asset you operate. That number is the key. It is how you enter, how you are identified, and how everyone who queries the register finds you.

A company that owns none of that asset answers zero, and a zero there does not put you at the bottom of the list. It keeps you off the list. Every filter I wrote, every sort, every threshold — all of it was running against a population that had already excluded the people I most wanted, before I wrote a line of it.

What makes this hard to catch is that nothing about the result looks wrong. A query that is too narrow returns a thin result and you notice, because thin results are irritating and you go and loosen something. This returned thousands of rows. It was complete, internally consistent, correct in every particular, and answering a question one noun away from mine.

Two registers, both complete, disagreeing about who exists

I did not fix this with a better query. There was no better query available, because the missing firms were not hiding behind a threshold — the register had no row for them at all.

The fix was a second register, kept by the same authority, keyed on something else entirely: not what you own, but what you are licensed to arrange. In that one, the missing companies are all present, named, and fully documented.

Here is the part I keep turning over. Both registers are complete. Both are accurate. Neither contains an error. They simply disagree about who counts as being in the industry, because one is keyed on ownership of a thing and the other on permission to perform an activity. I had been treating "the official list" as a fact about the world. It is a fact about a definition, and the definition lives in the key.

The general form

Any index you did not build is keyed on something, and that key is a definition of membership written by someone who was solving their own problem, not yours.

So before you use an index as a population, ask it one question you can usually answer in a couple of minutes: what does this count in order to decide that someone is in it? Then ask whether the thing that makes a candidate interesting to you is that same noun. When the two diverge, the people you care about will not turn up ranked low, flagged, or marked incomplete. They will not turn up.

And the remedy will not be found inside that index, however well you learn to query it. It will be a second index, keyed on a different noun, which is complete and correct about a slightly different world.