AI-Ready Data¶
More and more data is read by AI agents, not people. An analyst asks an agent for "office rent growth by market". The agent finds a table, joins it to another, and quotes a number in a memo. Nobody opens the methodology PDF.
The problem¶
Agents can call data. They often misread it:
- As-of dates. Which release is this? Is there a newer one? Two tables from different releases get mixed.
- Units. Is
0.012one percent or a fraction? Is412.5square feet or thousands of square feet? - Meaning. What is one row? Is a blank missing or zero? Which column replaced the one that was removed?
- Joins. Which key joins these tables, and what double-counts?
- Licensing and cost. May this be shown to a client? What does one more request cost?
People skim, ask a colleague and build a feel for a dataset over years. Agents read everything in front of them, follow it literally, and fail on whatever is implicit. A rule that lives in a support email or an analyst's head does not exist for them. They often sense trouble and ship the number anyway, because they have nothing to check against.
What AI-ready data needs¶
Facts about the data: what a reader must know to use it correctly, such as "rent_growth is a decimal, not a percent" or "office includes flex space from 2026Q2". They are not results drawn from the data ("rents rose 3%"); those are answers, and the facts are what make the answers right.
These facts are specific (a column, a source, a date), dated (so they can expire), placed next to the data (so they travel with it) and checkable: a small script (a probe) can re-test the claim against the data on each build, so the claim cannot silently stop being true.
A schema the reader can trust: the column names, how each value is stored (its type, filled from the data), its unit, and which columns were renamed or removed, so a change between two releases is spotted rather than guessed.
And a way for agents working on different parts of the pipeline to talk to each other. The agent that builds a dataset needs to know who reads it; the agents that read it need to hear when a number they quoted moves, ask the producer for a fix, and reuse a decision a person already made instead of re-deriving it. See Communication.
How it works¶
- Write the facts next to the data. A small YAML file (or folder) next to the data: grain (what one row is), units, as-of dates, joins, hazards, use cases and dated facts. See the spec.
- Check them against the data. A checker compares the file with what the data actually contains: a column that is not there, a unit that is not a valid unit, a typo in a key. Mistakes are reported with line numbers. Problems warn; they never block anyone from working.
- Deliver them to agents with citations. A tool shows the facts where the agent works, as a short card per dataset. Each fact has a stable id, so a number in a report can cite the fact it relied on.
- Keep the agents in touch. When a dataset is rebuilt, the projects that import it get a change notice their agents confirm once checked; questions go to the producer as requests; decisions are settled by a person and reused. See Communication.
The spec does not replace data contracts or semantic-model formats. It builds on open standards where they exist (UCUM units, ISO 4217 currencies, ODPS pricing plans, Apache Ossie relationships, a price mapped to ODCS) and adds only what they have no place for, such as an as-of date per source, a status and date on every statement, and a citation id. See Related standards for which element comes from where.
Where to start¶
- I publish data. For data vendors: what you write, how much work it is, what clients' agents get.
- I build pipelines or analysis. Write your first file: declare a dataset step by step and check it.
- I want to see the result. What an agent sees: the card an agent reads.
- I write about this. Start with the Overview and The full picture.
Or read on: four mistakes, fixed.