Skip to content

As-of dates

What an as-of date is. The as-of date says which release of a vendor source the data reflects. A file the vendor released for the quarter ending 30 September 2026 has the as-of date 2026-09-30, whenever you downloaded or loaded it. It is not the same as a fact's as_of, which is the date a statement was written or decided.

datasets.<name>.dates_asof maps each source name to where that source's as-of date comes from. It lets every answer say which release it used, and lets a reader notice two sources at different as-of dates being mixed. Source names are described once in project.sources.

For a data vendor, your own release is the source: name your product once and give the as-of date of the release you ship.

Which form to use

Each source uses exactly one of four forms:

  • constant: you ship or load files and know the release date. constant: "2026-09-30". Simple; update it with each new release, because nothing notices a stale constant.
  • column: the date is in the data, one per row or per group. column: asof_date. Stays right when new data arrives. Prefer it when your load step can write the date into a column.
  • param: a pipeline builds the dataset and takes the as-of date as a setting (a parameter of the run). param: asof. The tool reads the value from the record of each build.
  • from_input: the dataset is built from another dataset that already declares the as-of date. from_input: marketdata-example/Markets. One vendor pull then reads the same in every project.

Example

datasets:
  Rents:
    dates_asof:
      vendor_a:
        param: asof
        cadence: quarterly
        lag_days: 20
        received: received_at
      vendor_b:
        column: survey_date
        by: market_id
      vendor_c:
        constant: "2026-09-30"
      vendor_d:
        from_input: marketdata-example/Markets
        source: vendor_d
      vendor_e: asof          # shorthand for param: asof

Definitions

Each value is either a string (shorthand for param) or an object. An object has exactly one of param, column, constant, from_input; otherwise it is an error.

Key Type Required Priority Check Description
dates_asof.<source> string or object no Critical - One entry per source. A string is shorthand for param.
dates_asof.<source>.param string one of four Critical an implementation may check the parameter exists A setting of the pipeline run that builds the dataset holds the as-of date.
dates_asof.<source>.column column name one of four Critical - A column of the data holds it.
dates_asof.<source>.dataset string no Critical error if set without column With column: read the column from another dataset.
dates_asof.<source>.constant string one of four Critical - A fixed as-of date.
dates_asof.<source>.from_input <project>/<Dataset> one of four Critical - Take the as-of date an upstream dataset recorded.
dates_asof.<source>.source string no Critical - With from_input: the upstream dataset's name for the source (default: the same name).
dates_asof.<source>.where string no Critical - With column: a row filter. Draft: the exact semantics are not specified yet. Intended for reading the date only from the rows a source fills.
dates_asof.<source>.by string no Critical - With column: a grouping column. Draft: the exact semantics are not specified yet. Intended for an as-of date that differs per value of that column, for example per market.
dates_asof.<source>.cadence daily, weekly, monthly, quarterly, semiannual, annual no Useful error on any other value How often the source releases. Display only; never implies a release is due.
dates_asof.<source>.received string no Useful - Where the time you received or downloaded the data is recorded (a pipeline setting). Tells "release date" apart from "when we got it".
dates_asof.<source>.lag_days integer >= 0 no Useful error if negative Typical days from period end to the vendor's release. Display only.

cadence and lag_days are shown on the card ("quarterly, ~20 days after period end"). Nothing uses them yet to say that a newer release should exist.

As-of date versus build

The as-of date is not a fingerprint of the data. Two things identify what an answer used:

Answers Where
As-of date (dates_asof) Which release of each source does this data reflect? This file; shown on the card.
Build id Exactly which data is this? Each build has a timestamp, and the rows of two builds can be compared. Tooling, not this file.

Both are needed. A vendor can re-issue a file under the same as-of date with one value changed: the as-of date stays the same, the build id changes, and comparing the builds finds the changed row. A citation records both.