Skip to content

Examples

All data is synthetic. YAML style: block-style maps; short lists of column names inline.

Minimal

The smallest useful file: what one row is, and the unit of the main column.

version: 1
project:
  name: marketdata-example
datasets:
  Rents:
    purpose: Quarterly asking rent per market and sector.
    grain: [market_id, sector, period_yq]
    columns:
      rent:
        description: asking rent
        currency: USD
        per: year

A vendor-shipped file

A data vendor ships this file with each delivery. No pipeline: the as-of date is a fixed date, updated with each release, or read from a column. Licence and pricing are stated so a client's agent can read them.

version: 1
project:
  name: marketdata-example
  owner: data-product-team
  license: Licensed to subscribers; not for redistribution.
  sources:
    vendor_a: Quarterly rent file
datasets:
  Rents:
    purpose: Quarterly asking rent per market and sector.
    grain: [market_id, sector, period_yq]
    profile:
      period: period_yq
      entities: [market_id]
      coverage_by: [sector]
    dates_asof:
      vendor_a:
        constant: "2026-09-30"
        cadence: quarterly
        lag_days: 20
    columns:
      market_id:
        description: market code (text; keep leading zeros)
      sector:
        description: property sector
      period_yq:
        description: calendar quarter, written 2026Q3
      rent:
        description: asking rent per square foot
        currency: USD
        per: year
      rent_growth:
        description: quarter-on-quarter rent growth
        unit: decimal
    do_not:
    - text: compare rent across sectors without adjusting for lease length
      columns: [rent, sector]
    not_exported:
    - what: lease-level rows
      where: the Leases product
      why: sold separately
    pricing:
    - name: Standard API
      unit: Pay-per-use
      price: '0.01'
      price_currency: USD
      price_unit: request
    facts:
    - id: office-incl-flex
      statement: "office includes flex space from 2026Q2: compare office within one side of 2026Q2 only"
      as_of: 2026-07-15
      severity: caution
      status: reviewed
      columns: [sector]

A pipeline-built dataset

A team builds Rents with a pipeline that takes the as-of date as a setting (param), and adds vacancy from a second vendor whose release date is fixed (constant). This is the file behind the card on What an agent sees.

version: 1
project:
  name: marketdata-example
  description: Synthetic rents for the docs.
  owner: data-team
  sources:
    vendor_a: Quarterly rent file from vendor A
    vendor_b: Annual vacancy survey from vendor B
datasets:
  Rents:
    purpose: Quarterly asking rent per market and sector.
    grain: [market_id, sector, period_yq]
    profile:
      period: period_yq
      entities: [market_id]
      coverage_by: [sector]
    dates_asof:
      vendor_a:
        param: asof
        cadence: quarterly
        lag_days: 20
      vendor_b:
        constant: "2026-06-30"
        cadence: annual
    columns:
      market_id:
        description: market code (text)
      sector:
        description: property sector
      period_yq:
        description: calendar quarter
      rent:
        description: asking rent
        currency: USD
        per: year
        as_of_sources: [vendor_a]
      rent_growth:
        description: quarter-on-quarter rent growth
        unit: decimal
        as_of_sources: [vendor_a]
      vacancy:
        description: vacancy rate
        unit: "%"
        as_of_sources: [vendor_b]
    use_cases:
    - id: compare-markets
      use_case: compare markets
      then: use rent_growth within one sector; never across sectors
      pitfall: sectors differ in lease length
      columns: [rent_growth, sector]
    do_not: [join on market names]
    removed_columns:
    - name: rent_old
      removed: 2026-10-29
      use_instead: rent
    license: CC-BY-4.0
    facts:
    - id: office-incl-flex
      statement: "office includes flex space from 2026Q2: compare office within one side of 2026Q2 only"
      as_of: 2026-10-31
      severity: caution
      status: reviewed
      reviewed_by: data-team
      columns: [sector]
    - id: covid-window
      statement: how to treat 2020-2021 is undecided
      as_of: 2026-10-31
      status: decision-open
      options: [exclude 2020-2021, keep all quarters]
    - id: q3-resent
      statement: the 2026Q3 file was re-sent on 2026-11-02
      as_of: 2026-11-02
      expires: 2027-01-31
  Markets:
    purpose: One row per market.
    grain: [market_id]
    columns:
      market_id:
        description: market code (text)
      market_name:
        description: display name; not a key
relationships:
- name: rents_to_markets
  from: Rents
  to: Markets
  from_columns: [market_id]
  to_columns: [market_id]
  ai_context: join on market_id, never on market names

vacancy uses unit: "%"; pct means the same and is also accepted. For a dataset in another project, write to: as <project>/<Dataset>, for example marketdata-example/Markets.