Your Hardest Dependency Is Somebody Else’s Data Entry

We spent a quarter building provenance we were genuinely proud of, and then lost an afternoon to a supplier who wrote a lot number two different ways.

That is the whole lesson, and I want to be precise about why it is not a story about one careless supplier.

You can build immaculate provenance internally — stable identifiers, an append-only record, a defensible join, the whole thing — and be defeated entirely by a supplier who typed an identifier one way on the invoice and another way on the label.

This is the part nobody puts on a slide. The interesting engineering is supposed to be the cryptography, or the schema, or the distributed-systems problem. It is actually reconciliation: deciding what to do when two records that must describe the same physical thing do not agree, and doing it in a way that neither silently guesses nor blocks the entire shipment.

Both failure modes are real and they pull against each other. Guess, and you produce evidence you cannot support. Block, and you have built something that makes people's day worse until they route around it — which they will, and then you have no data at all.

What I have found workable is refusing to treat it as a binary. A mismatch is not an error and it is not a pass. It is a state: these records disagree, here is exactly how, here is what we would need to resolve it, and the shipment proceeds with that flag attached to that unit.

The flag is the product. It travels with the item, it is visible at the point where somebody needs to rely on the evidence, and it does not quietly evaporate because a batch job decided the strings were close enough.

The broader lesson I have taken from it: when your system's correctness depends on inputs you do not control, the design goal is not correctness. It is legibility about where correctness is uncertain. You cannot make the supplier type it right. You can absolutely make sure nobody downstream mistakes a guess for a fact.