Enforce the Promise: dbt, OpenMetadata and ODCS

A data contract is a promise written in a file. It might say that a table will always have these columns, that an identifier will never be empty, or that the data refreshes every day. The promise is clear, it is versioned, and the team reviews it. By itself, though, it protects nothing. This lab gives each of three tools one clear job, then builds the part that actually checks the promise and stops whatever would break it.

By the end, you'll be able to…

• Explain why dbt and OpenMetadata play different roles, and why neither of them is a contract.
• Build and test models with dbt, and tell a dbt model contract apart from an ODCS contract.
• Ingest dbt's artifacts into OpenMetadata to get lineage, documentation that stays in sync, and test results.
• Read and write an ODCS v3.1.0 contract, and run it against live data.
• Enforce the contract at the CI gate and on a schedule in the catalog, and know which layer is really protecting the data.

What you'll learn

The path

Setup: get the pieces running

Four pieces, each with one job. None of them doubles up: dbt never catalogs, OpenMetadata never transforms, and the contract file never enforces itself.

Step 1 · Start Docker Desktop

Open Docker Desktop and wait until it reports that the engine is running. Until then, every docker command fails with a "cannot connect to the Docker daemon" error.

docker info   # succeeds once the engine is up

Step 2 · Start the shop database

# from this lab folder
cd ..\openmetadata-lab\sample-db
docker compose up -d
docker ps --filter name=shop_postgres   # expect 0.0.0.0:5433->5432

Step 3 · Install dbt and the Data Contract CLI

# back in this lab folder (dbt-openmetadata-odcs-lab)
python -m venv .venv
.\.venv\Scripts\Activate.ps1          # macOS/Linux: source .venv/bin/activate
pip install "dbt-core~=1.11.0" dbt-postgres "datacontract-cli[postgres]"
dbt --version                         # dbt Core 1.11.x, adapter postgres 1.11.x

Step 4 · Copy the dbt profile into place

dbt only reads a file named exactly profiles.yml, so copy the example across before you connect. Skip this and you will see “Could not find profile named 'shop'”.

Copy-Item dbt\profiles.example.yml dbt\profiles.yml
# macOS/Linux: cp dbt/profiles.example.yml dbt/profiles.yml

Step 5 · Point dbt at the shop DB and test it

$env:DBT_PROFILES_DIR = "$PWD\dbt"   # macOS/Linux: export DBT_PROFILES_DIR="$(pwd)/dbt"
cd dbt
dbt debug                          # expect: All checks passed!

dbt, the worker

Use dbt to build clean, tested models from the raw shop tables. Pay attention to the three files it leaves behind, because those files are the entire bridge to OpenMetadata.

shop.public.orders (raw)          shop.public.customers (raw)
        │                                 │
        ▼                                 ▼
   stg_orders (view)                stg_customers (view)     ← staging: clean 1:1
        │
        ▼
   daily_revenue (table)                                     ← mart: the dashboard reads this

The mart daily_revenue is dbt's version of the hand-written v_daily_revenue view from the OpenMetadata lab. The numbers are the same, but now the table is materialized, tested and covered by a contract. Everything dbt builds goes into a separate analytics schema, so raw and transformed data stay apart.

Do thisBuild the models

powershellCopy

# from the dbt folder, with the venv active and DBT_PROFILES_DIR set in Setup
dbt deps          # fetch dbt_utils (range tests)
dbt build         # run + test every model, in dependency order

Confirm dbt actually created objects in the warehouse. The shop database runs in Docker, so query it through the container (no local psql needed):

powershellCopy

docker exec shop_postgres psql -U shop_user -d shop -c "select * from analytics.daily_revenue;"

You built real objects in the database. That is the worker doing its job, and it is something OpenMetadata could never do, because it builds nothing.

Two kinds of “contract” already live in this project

This is the distinction teams miss most often, so it helps to see it with the code in front of you.

With contract: enforced: true on the mart, dbt checks at build time that the output columns match the declared names and types exactly. If they drift, the build fails before the table is ever created.

Do thisBreak the shape on purpose

This one is a file edit, not a command to paste in the terminal. Open the mart model in your editor (it lives in this lab's dbt folder):

fileCopy

dbt\models\marts\daily_revenue.sql

Find the revenue line near the bottom and force the wrong type by adding ::text:

sql · daily_revenue.sqlCopy

    count(*)::bigint          as orders_count,
    sum(total_amount)::text   as revenue      -- was: sum(total_amount) as revenue

Save the file, then build just that model:

powershellCopy

dbt build --select daily_revenue

dbt refuses, because the output column no longer matches the contract:

Now put the line back to sum(total_amount) as revenue, save, and rebuild to return the lab to health:

powershellCopy

dbt build --select daily_revenue

You just watched Layer 1 enforcement work inside dbt: the problem never made it as far as the table.

The three files dbt leaves behind

dbt run             # manifest.json + run_results.json
dbt test            # updates run_results.json with test outcomes
dbt docs generate   # catalog.json (+ refreshes manifest.json)
ls target/*.json    # manifest.json, catalog.json, run_results.json all present

OpenMetadata, the librarian

Hand dbt's artifacts to OpenMetadata and watch it observe. It builds lineage through your models, syncs the documentation you wrote in YAML, and shows your dbt test results in the catalog.

dbt run/test/docs generate               OpenMetadata
        │                                       │
   target/manifest.json    ──ingest──▶   lineage (raw → stg → mart)
   target/catalog.json     ──ingest──▶   column types + docs, in sync with code
   target/run_results.json ──ingest──▶   test pass/fail history

Open OpenMetadata first

Everything in this lesson happens in the OpenMetadata web app, which is a different thing from Docker Desktop. Docker only keeps the containers running in the background; the clicking is all in your browser.

1. Go to http://localhost:8585 in a browser.
2. Sign in with the default lab credentials: user admin@open-metadata.org, password admin.
3. Find your shop database service under the gear icon → Settings → Services → Databases. From the OpenMetadata lab it is called online_shop.

Ingest the dbt artifacts

OpenMetadata's built-in ingestion runs inside a container, which cannot read the target/ files sitting on your Windows host. The simplest way around that on a local setup is the metadata ingest-dbt command: it runs on your host, where the files already are, and pushes everything to OpenMetadata over its API.

Do thisIngest with metadata ingest-dbt

1. Install the OpenMetadata ingestion package into the same venv you used for dbt. It must match your server's major.minor version, or it refuses to connect. This lab runs OpenMetadata 1.12.x, so pin the client to 1.12:

powershellCopy

pip install "openmetadata-ingestion[dbt]==1.12.*"
# the client version must match the OpenMetadata server (check yours in Settings)

2. Get an API token: in OpenMetadata, go to Settings → Bots → ingestion-bot and copy its JWT token.

3. Add an openmetadata block to the vars: section of dbt\dbt_project.yml (add a vars: section if there isn't one):

yaml · dbt_project.ymlCopy

vars:
  openmetadata_host_port: "http://localhost:8585/api"
  openmetadata_jwt_token: "<paste the ingestion-bot token>"
  openmetadata_service_name: "online_shop"

4. You already generated the artifacts in Lesson 1, so from the dbt folder run:

powershellCopy

metadata ingest-dbt

It finds target/, connects to OpenMetadata, and reports the models it processed. This is a local lab, so pasting the token is fine; for anything real, use an environment variable instead of writing it into the file.

docker cp dbt\target\manifest.json    openmetadata_ingestion:/tmp/dbt/manifest.json
docker cp dbt\target\catalog.json     openmetadata_ingestion:/tmp/dbt/catalog.json
docker cp dbt\target\run_results.json openmetadata_ingestion:/tmp/dbt/run_results.json

Do thisSee what the librarian now knows

1. Lineage. Open analytics.daily_revenue and go to Lineage. You should see public.orders, then stg_orders, then daily_revenue, drawn from the depends_on graph in the manifest.
2. Docs in sync. The column descriptions you wrote in _marts.yml are now in the catalog. Change the YAML, re-run and re-ingest, and the catalog updates.
3. dbt tab. The model shows its compiled SQL from the manifest.
4. Test results. The unique, not_null and range outcomes from run_results.json appear as pass or fail.

You didn't teach OpenMetadata any of this. It observed all of it from the records dbt left behind.

ODCS, the contract

Write the formal agreement that binds the producer of daily_revenue to its consumers, using the open standard the industry has now settled on.

ODCS is not the dbt model contract

You now have two contracts on one table, and they do different jobs:

Anatomy of the contract

A single document covers the whole agreement. The servers block is what makes it executable, since that is where the CLI connects.

apiVersion: v3.1.0
kind: DataContract
id: daily-revenue-001
name: Daily Revenue Contract
version: 1.0.0
status: active
domain: revenue
servers:
  - server: shop-postgres
    type: postgres
    host: localhost
    port: 5433
    database: shop
    schema: analytics          # the dbt-built schema, not public
schema:
  - name: daily_revenue
    physicalType: table
    properties:
      - name: order_date
        physicalType: DATE
        primaryKey: true
        required: true
        unique: true
      - name: revenue
        physicalType: NUMERIC
        required: true
        classification: restricted
    quality:
      - type: library
        metric: rowCount
        mustBeGreaterThan: 0
      - type: sql
        description: Revenue must never be negative.
        query: SELECT COUNT(*) FROM daily_revenue WHERE revenue < 0
        mustBe: 0
team:
  members: [{username: bmillet, role: Owner}]
slaProperties:
  - {property: frequency, value: "1", unit: d}
  - {property: freshness, value: "24", unit: h}
support:
  - {channel: "#data-revenue", tool: slack}

That SLA, ownership and support section is the part that could never live in a dbt model contract. It is an agreement between people, not a check on the shape of a table.

pip install check-jsonschema
check-jsonschema --schemafile https://raw.githubusercontent.com/bitol-io/open-data-contract-standard/main/schema/odcs-json-schema-latest.json contracts\daily_revenue.odcs.yaml

Enforcement: three layers

Turn the contract from a promise on paper into one that protects. A contract is only worth as much as the check behind it, so the real question is not whether to verify but when. There are three moments to choose from.

Layer 1: enforce at the CI gate (before)

Block the break in the pull request, before it can ever reach the data. On every PR that touches dbt/ or contracts/, three gates run against a throwaway Postgres, cheapest first:

1 · Schema gate → is the ODCS file valid ODCS 3.1.0?   (check-jsonschema)
2 · Build gate  → dbt build (model contract + data tests)
3 · Data gate   → does the LIVE built data satisfy the ODCS contract?  (datacontract test)

python -m venv .venv-contract
.venv-contract\Scripts\pip install "datacontract-cli[postgres]"

Do thisRun the three gates locally

powershellCopy

# gate 1 -- validate the ODCS document
check-jsonschema --schemafile https://raw.githubusercontent.com/bitol-io/open-data-contract-standard/main/schema/odcs-json-schema-latest.json contracts\daily_revenue.odcs.yaml
# gate 2 -- build and test the models
cd dbt ; dbt build ; cd ..
# gate 3 -- run the contract against the live data (CLI from its own venv)
$env:DATACONTRACT_POSTGRES_USERNAME = "shop_user"
$env:DATACONTRACT_POSTGRES_PASSWORD = "shop_pass"
.venv-contract\Scripts\datacontract test contracts\daily_revenue.odcs.yaml

Expect 13 checks green: the three quality rules plus the schema checks the CLI derives from the contract's properties.

Do thisBreak the contract and watch the gates go red

Play the careless insider: insert a bad order straight into the warehouse, on a fresh day so that day's revenue comes out negative. (Putting it on a day that already has sales would only lower the total, not push it below zero — aggregation can hide bad source rows.)

powershellCopy

docker exec shop_postgres psql -U shop_user -d shop -c "INSERT INTO public.orders (customer_id, order_date, status, total_amount) VALUES (1, '2025-05-10', 'completed', -50.00);"

Rebuild before you re-test. The mart is a table: if you skip dbt build, the CLI happily validates the stale data and everything stays green. A green check is only as fresh as the last build.

powershellCopy

cd dbt ; dbt build ; cd ..
.venv-contract\Scripts\datacontract test contracts\daily_revenue.odcs.yaml

Both layers catch it: dbt build finishes with ERROR=1 (the model's own data test — the worker's perimeter), and the contract goes red:

outputCopy

data contract is invalid, found the following errors:
1) Revenue must never be negative.: Actual was 1, expected = 0

In a real PR this exit code blocks the merge. Clean up by deleting the bad row and rebuilding, then confirm green again:

powershellCopy

docker exec shop_postgres psql -U shop_user -d shop -c "DELETE FROM public.orders WHERE total_amount < 0;"
cd dbt ; dbt build ; cd ..
.venv-contract\Scripts\datacontract test contracts\daily_revenue.odcs.yaml

Layer 1 does have a limit: it only sees what goes through code, so a manual insert like the one you just made slips straight past CI — you only caught it because you ran the checks by hand. That is exactly why you also want Layer 3, which runs them on a schedule.

Layer 3: validate in the catalog on a schedule (after)

The data is already sitting in the warehouse. On a schedule, OpenMetadata re-runs the contract checks and flags anything that failed. You gain a history and SLA reporting that CI cannot give you. OpenMetadata added contracts in v1.9 (August 2025) and full ODCS 3.1 import and export in v1.10 (October 2025).

Do thisImport the ODCS contract into OpenMetadata

1. Open localhost:8585 (login admin@open-metadata.org / admin), search daily_revenue and open the table under the online_shop service.
2. Click the Contract tab. It starts empty — click the ˅ arrow on Add Contract and choose Import ODCS, then upload contracts\daily_revenue.odcs.yaml. Wait for Schema Validation: Passed. If it fails with Cannot deserialize ... ArrayList at the team block, your contract uses the object form — switch to the array form (see Lesson 3).
3. Click Import. The contract now shows in the tab: description, SLA (freshness / latency / retention), and the schema with PRIMARY_KEY and NOT_NULL constraints.

Now audit what survived. On a real run against OpenMetadata 1.12.6: description ✔, schema and constraints ✔, SLA partially (frequency not surfaced) — and all three quality rules silently dropped. No error, no warning. The catalog looks protected while nothing checks revenue. That is the March 2026 pitfall, reproduced live.

Do thisRecreate the quality rules as native test cases and schedule them

Since the import dropped the quality rules, mirror them as native OpenMetadata test cases via the API. The lab ships a script that reads the ingestion-bot JWT you already stored in dbt/dbt_project.yml in Lesson 2:

powershellCopy

# from the lab root (quote the path if it has spaces), dbt venv active
python scripts\create_om_test_cases.py

Expected: [ok] table found, [ok] test suite, then three [ok] test case created lines for contract_row_count_gt_0, contract_orders_count_positive and contract_revenue_never_negative.

1. See them. On the table page: Data Observability tab → Data Quality sub-tab. You'll count 9 test cases: 6 dbt tests already green (those results were synced from dbt by Lesson 2's ingestion — the librarian reporting what the worker ran) and your 3 contract_* tests with no result yet: nothing in the catalog has executed them.

2. Schedule them. Still under Data Quality, click the Pipelines sub-tab (shows 0) → Add Pipeline. Name it (for example dq test), leave the test-case selection on All, set the schedule (for example every day at 12:00 AM, after the nightly dbt build), and submit. You should see "has been created and deployed successfully".
3. Run it once now. Back on the Pipelines list, use the row's Run action, wait about a minute, refresh: Recent Runs: Success, Status: Active.

4. Confirm. Switch back to Test Cases: 9 / 9 Successful, and the three contract_* tests now carry their own Last Run timestamp — executed by the catalog, not by dbt. Every scheduled run appends to this history: the pass/fail and SLA reporting view CI cannot give you.

The same promise is now enforced at two moments: blocked in CI before merge (Layer 1) and audited on a schedule in the catalog (Layer 3) — with the honest caveat that the catalog's copy of the quality rules is maintained by script until the ODCS import defect is fixed.

The full chain, end to end

ODCS contract   declared in YAML, versioned + reviewed in a PR
      │
   CI gate       check-jsonschema + dbt build + datacontract test   ← blocks any violation
      ▼
   dbt           builds + tests models, leaves manifest/catalog/run_results
      ▼
   OpenMetadata  ingests artifacts → lineage/docs/tests; imports ODCS → scheduled validation + SLA

A contract that protects, not just reassures

You have given three tools a clear job each and built the layer that checks the promise. Now step back and look at what each one really does, where the standard stops and the organisation has to take over, and how much anyone can honestly claim to know about this yet.

Each tool does exactly one job

The verdict

No one has five years of this

Keep your expectations grounded. ODCS 3.1.0 arrived in December 2025. OpenMetadata shipped contract enforcement in v1.9 (August 2025) and full ODCS support only in v1.10 (October 2025). The capability is measured in months, not years, so nobody has real tenure enforcing ODCS contracts in OpenMetadata. What does carry over is the judgment underneath the tools:

• Schema evolution: knowing which changes are safe, which break consumers, and how to version a breaking change without causing an incident.
• Quality gates: deciding what to test, where to test it, and which failures should block rather than warn. The rules matter more than the syntax.
• Ownership: negotiating the boundary between producers and consumers, and holding named owners to it. This is organisational work, not tooling.

Someone who can explain why a catalog holding a file is not the same as a contract being enforced understands more than someone who lists a tool they cannot have used for long. On this stack, weigh judgment over years served, because the years are not there yet for anyone.