As-of dates¶
What an as-of date is. The as-of date says which release of a vendor source the data reflects. A file the vendor
released for the quarter ending 30 September 2026 has the as-of date 2026-09-30, whenever you downloaded or loaded
it. It is not the same as a fact's as_of, which is the date a statement was written or decided.
datasets.<name>.dates_asof maps each source name to where that source's as-of date comes from. It lets every answer
say which release it used, and lets a reader notice two sources at different as-of dates being mixed. Source names
are described once in project.sources.
For a data vendor, your own release is the source: name your product once and give the as-of date of the release you ship.
Which form to use¶
Each source uses exactly one of four forms:
constant: you ship or load files and know the release date.constant: "2026-09-30". Simple; update it with each new release, because nothing notices a stale constant.column: the date is in the data, one per row or per group.column: asof_date. Stays right when new data arrives. Prefer it when your load step can write the date into a column.param: a pipeline builds the dataset and takes the as-of date as a setting (a parameter of the run).param: asof. The tool reads the value from the record of each build.from_input: the dataset is built from another dataset that already declares the as-of date.from_input: marketdata-example/Markets. One vendor pull then reads the same in every project.
Example¶
datasets:
Rents:
dates_asof:
vendor_a:
param: asof
cadence: quarterly
lag_days: 20
received: received_at
vendor_b:
column: survey_date
by: market_id
vendor_c:
constant: "2026-09-30"
vendor_d:
from_input: marketdata-example/Markets
source: vendor_d
vendor_e: asof # shorthand for param: asof
Definitions¶
Each value is either a string (shorthand for param) or an object. An object has exactly one of param, column,
constant, from_input; otherwise it is an error.
| Key | Type | Required | Priority | Check | Description |
|---|---|---|---|---|---|
dates_asof.<source> |
string or object | no | Critical | - | One entry per source. A string is shorthand for param. |
dates_asof.<source>.param |
string | one of four | Critical | an implementation may check the parameter exists | A setting of the pipeline run that builds the dataset holds the as-of date. |
dates_asof.<source>.column |
column name | one of four | Critical | - | A column of the data holds it. |
dates_asof.<source>.dataset |
string | no | Critical | error if set without column |
With column: read the column from another dataset. |
dates_asof.<source>.constant |
string | one of four | Critical | - | A fixed as-of date. |
dates_asof.<source>.from_input |
<project>/<Dataset> |
one of four | Critical | - | Take the as-of date an upstream dataset recorded. |
dates_asof.<source>.source |
string | no | Critical | - | With from_input: the upstream dataset's name for the source (default: the same name). |
dates_asof.<source>.where |
string | no | Critical | - | With column: a row filter. Draft: the exact semantics are not specified yet. Intended for reading the date only from the rows a source fills. |
dates_asof.<source>.by |
string | no | Critical | - | With column: a grouping column. Draft: the exact semantics are not specified yet. Intended for an as-of date that differs per value of that column, for example per market. |
dates_asof.<source>.cadence |
daily, weekly, monthly, quarterly, semiannual, annual |
no | Useful | error on any other value | How often the source releases. Display only; never implies a release is due. |
dates_asof.<source>.received |
string | no | Useful | - | Where the time you received or downloaded the data is recorded (a pipeline setting). Tells "release date" apart from "when we got it". |
dates_asof.<source>.lag_days |
integer >= 0 | no | Useful | error if negative | Typical days from period end to the vendor's release. Display only. |
cadence and lag_days are shown on the card ("quarterly, ~20 days after period end"). Nothing uses them yet to
say that a newer release should exist.
As-of date versus build¶
The as-of date is not a fingerprint of the data. Two things identify what an answer used:
| Answers | Where | |
|---|---|---|
As-of date (dates_asof) |
Which release of each source does this data reflect? | This file; shown on the card. |
| Build id | Exactly which data is this? Each build has a timestamp, and the rows of two builds can be compared. | Tooling, not this file. |
Both are needed. A vendor can re-issue a file under the same as-of date with one value changed: the as-of date stays the same, the build id changes, and comparing the builds finds the changed row. A citation records both.