Why I like awkward data

Clean data is lovely. Awkward data is where the interesting work usually is, because it forces the interface to become honest.

The myth of the neat source

Clean data is lovely. Awkward data is where the interesting work usually is.

Public data especially has a way of looking simple from a distance. Rubbish collections. Toilet locations. Vessel positions. These sound like solved categories. Surely the information exists somewhere. Surely it is structured consistently. Surely there is an endpoint, a schema, a neat export, a sensible update rhythm.

Then you actually start building.

One council uses a searchable endpoint. Another hides useful information behind a page designed for humans. Another has a PDF. Another changes terminology. One field is missing. One date format is different. One source works until a request looks too unlike a browser.

The project is not just showing the data

Suddenly the project is not “show the data.” It is “understand the habits of several small systems that were never designed to agree with each other.”

I find that frustrating in the moment, but satisfying in the work. Awkward data forces decisions. What can be trusted? What should be shown? What should be hidden? When should the interface admit uncertainty? How do you make something feel simple without pretending the source is cleaner than it is?

A good interface can simplify a messy source without lying about it.

Honesty is a design feature

That last part matters. A good public tool should not dump complexity onto the user, but it also should not lie. If coverage is patchy, say so through the design. If a council source needs special handling, build the special handling. If a field is unavailable, do not invent confidence.

This is where design and engineering become the same job. The data model shapes the interface. The interface reveals what the data model failed to solve. The user does not care which layer caused the problem; they just know whether the answer feels clear.

The translation layer

I think of a lot of this work as translation. Not translation between languages, but between systems and people.

The source might think in route IDs, zone codes, timestamps, receiver coverage, facility attributes, or provider-specific labels. The person using the tool thinks in questions: what goes out tonight, where can I go nearby, what is happening on the water?

The product lives in the space between those two ways of seeing. If it is built well, the person does not need to inherit the source’s weirdness. They get the part that matters, in a form that makes sense for the moment they are in.

Why I keep coming back to it

That is why I like awkward data. It gives you a real system to respect, not a perfect demo to decorate.

It also keeps the work grounded. There is no room for fake polish when the source is inconsistent. You have to make practical decisions, test assumptions, and build interfaces that can handle the shape of the real world.

Clean data is easier to present. Awkward data is more useful to understand. And when you can turn it into something calm, honest, and usable, the result feels genuinely earned.

← Being a jack of all trades, properly The daily art challenge changed my standards →