Composable canonicals: from tribal data knowledge to versioned artifact

The central data team problem?

It’s a knowledge coordination problem. In a central warehouse or lake, one data team owns the data for the whole company. But that team does not know what the data means. The domain teams know: the people in payments, in product, in finance.

So the fix has always been about decentralization. If meaning lives in the domains, then ownership has to live there too. Put another way: decentralization is not a nice-to-have added on top. It is the direct answer to the knowledge problem. From embedded analysts to data mesh, everyone proposed some level of data and tech in the domain team to help convey the meaning to the central team.

Embedded analysts are common in most companies. Other companies go as far as creating an API-based contract between domain and core teams and forcing the domain teams to serve data through the contract-api. That’s expensive and complex, and complexity is often a sibling of failure.

Today, we have a new tool at our disposal - one that can shift technical capabilities into domain teams, or coordinate the knowledge transfer from domain to core teams: LLMs, of course.

I previously described how canonical models coupled with a description of the concepts and their relationships creates agentic-native canonical meaning units.

In this post, we move from concept to architecture that you can apply to a data. stack.

Composable canonicals

The idea of this post is to take these self documented canonical knowledge blocks and combine them into a business canonical block

To recap, the idea of a canonical block was to create a machine readable “tribal knowledge” or domain knowledge that can survive across departments and people and that can be used to generate reporting from raw data

The way we build the blocks:

1. Grab raw data and infer its schema with dlt. This creates a technical profile about the structure of the data and what is contained.
2. Bootstrap an ontology - the agent searches its memory, the internet, and asks you questions to understand what the data is about.
3. Generate a data model, review and possibly refine the original ontology.

This gives you a raw data pipeline, a transformation to canonical, and a knowledge document (the ontology) that instructs the agent what the data means.

The same way that we create a data source canonical model, we can also use these models as sources for creating the business canonical.

What a business canonical is, and how it differs from a source canonical

Let's make this concrete with three systems many teams actually run, and a real flow between them.

Suppose HubSpot is our source of truth for who a customer is. Slack is where people get support and chat with us. Luma is where we run events and collect guests — new people show up there first, and we later add them into HubSpot as contacts. So a single human can appear in all three, at different stages, under different names, with no shared ID.

Each source is its own canonical

A source canonical models one system in that system's own language. It has three parts: the raw tables we load, the transformation that shapes them, and the ontology that says what the entities mean inside that system.

Luma canonical — top of the funnel. Raw Luma gives us events, registrations, and guests. The ontology says: a Guest is a person who registered for or attended an event, keyed by the email they signed up with. In Luma's world, "customer" doesn't exist yet — there are only Guests, some of whom we'll later decide are worth pulling into the CRM.

• events — an event we hosted (name, date, capacity)
• registrations — a guest signed up for an event (email, name, RSVP status, attended y/n)
• guests — the deduplicated person as Luma knows them (email, name, first-seen date)

HubSpot canonical — the source of truth. Raw HubSpot gives us contacts, companies, and deals. The ontology says: a Contact is a person the company has a real relationship with; the lifecycle stage (lead, MQL, customer) is HubSpot's opinion of where they are. This is the system that gets to say the word "customer" officially.

• contacts — the person of record (email, name, lifecycle_stage, owner)
• companies — the account a contact belongs to
• deals — revenue attached to a contact/company

Slack canonical — the support and community surface. Raw Slack gives us members, channels, and messages. The ontology says: a Member is a person in our community/support workspace, keyed by the email on their Slack profile; their activity is a signal of engagement and support load, not of sales stage.

• members — a person in the workspace (email, display name, joined date)
• channels — support, onboarding, general
• activity — messages, joins, reactions per member

Each of these three is complete and correct on its own. The Luma team can decide what "attended" means without asking anyone. HubSpot owns "lifecycle stage." Slack owns "active member." Nobody has to negotiate, because each definition is true only inside its own system.

The business canonical composes them into one Customer

Now the VP of Growth asks the question no single system can answer: "Which Slack members who joined in Q1 became qualified leads after attending a couple of our events?"

Lucky, you have your business canonical layer that resolved the identity - in this layer lives the “customer” table which is created with some logic from the other 3 - for example, let’s say email is the customer’s identifier, and we merge all 3 sources into a master that contains some common columns such as email, first_seen, last_seen, events_attended, qualified_at etc. By having a customer table, you might be able to answer the question directly from the table or within a join or 2. Without it, you’d be defining the customer on the fly, eventually leading to multiple sources of truth.

An example of business canonical rules - how do we define a customer?

• Identity / match key. The same Person is matched across systems on email, but as users change emails over time, we sometimes edit records or re-map them.
• Precedence. When two systems disagree on a field, we declare a winner. Name and lifecycle come from HubSpot (source of truth). First-touch date comes from whichever system saw them first — usually Luma. Support engagement comes from Slack. This coalesce logic is the seam, written down.
• Stage mapping. The ontology encodes the journey: a Luma Guest who gets added to HubSpot becomes a Contact; a Contact who crosses HubSpot's threshold is a Qualified Lead; a Slack Member active in support channels is an Engaged customer. The business canonical is where "Guest → Contact → Qualified Lead" is defined once, not re-derived in every query.

The resulting Customer is the company-wide customer and the VP's question becomes a filter over that one table.

do it with dltHub

. The dlthub layers support this workflow from ingestion to production

• Ingestion with dlt, the ingestion standard for data engineers
• Transformation with dlthub’s composable transformations
• Via the AI harness, model to canonical or let anyone on the team build ingestion like a senior.
• The context catalog enables end to end automation of building, deployment, and maintenance.
• And when deployed the pipelines are orchestrated and run on our managed infrastructure which ensures low cost of operation.