· Updated · 8 min read
Unity Catalog and the Limits of a Standalone Catalog
Unity Catalog provides a lakehouse with shared discovery, access controls, and lineage. Pipeline checks still decide what may land and who fixes a failure.
A catalog can control who sees registered data without deciding whether that data should have landed. That distinction affects who can stop bad data and who must respond. The common view puts governance and data quality mainly in a downstream catalog program. A catalog governs accepted data, while the pipeline decides what may enter and its operating team fixes failures.
What is Unity Catalog?
Unity Catalog names Databricks’ built-in governance layer and a related community catalog project. Both organize data assets and metadata. The Databricks product also controls access, records lineage, and logs activity across attached workspaces. The project organizes assets and permissions through interfaces that different data engines can use.
Both use three levels to group assets. The Databricks product uses catalog, schema, and object. The community project uses catalog, schema, and asset. These levels name assets, record where they belong, and provide objects for permissions.
What does Unity Catalog govern, and where does that job end?
Unity Catalog governs how people find, access, and trace governed assets; its job ends at the pipeline’s decision to accept incoming data. A catalog-level browse permission can expose metadata without granting access to the underlying data. Other access controls grant privileges, apply policies to tagged data, filter table rows, mask columns, or restrict access by workspace.
Unity Catalog records table-level and column-level lineage across attached workspaces. Its lineage records include identities and event times when the system can infer lineage. Its audit table, currently in public preview as of 13 November 2026, records activity details for audit work.
These controls govern registered assets; they do not make the pipeline’s initial acceptance decision. Databricks can compare a table’s recent row count with its history after loading. It marks the table incomplete when the row count falls below the expected range. That quality monitoring helps after acceptance, but it does not replace an acceptance check.
| Capability | What it controls or records | Business decision |
|---|---|---|
| Discovery | Names and metadata | What can people find? |
| Access | Permissions and policies | Who can use it? |
| Lineage | Sources and targets | Where did it come from? |
| Audit records | Identities and activity | Who did what? |
| Object hierarchy | Catalogs, schemas, objects | Where does it belong? |
The two Unity Catalogs share a name and common concepts, but their documented scopes differ. Databricks created the community project and later contributed it to the LF AI & Data Foundation. The Databricks product is a managed governance layer within Databricks. The community project shares its API heritage but operates as a separate catalog server.
Both organize assets through catalogs and schemas, record metadata, and enforce documented access controls. The Databricks product works across workspaces attached to its metastore. The project documents connections for engines such as Spark, DuckDB, Trino, and Daft. It can operate on a local computer or on AWS, Microsoft Azure, and Google Cloud.
| Name | What it is | What it organizes | Where it operates |
|---|---|---|---|
| Databricks Unity Catalog | Built-in governance layer | Data and AI assets | Attached Databricks workspaces |
| Unity Catalog project | Separate catalog server | Tables, volumes, functions, models | Local or major clouds |
Its Iceberg interface leads to a separate question: what belongs to an Iceberg table and what belongs to its catalog?
How do Iceberg catalogs and the REST Catalog fit?
An Iceberg table, an Iceberg catalog, and the Iceberg REST Catalog do separate jobs. The table specification defines metadata for schemas, partition settings, properties, and snapshots. Each snapshot identifies a table state and its complete set of data files.
The catalog supplies the table location and can hold the pointer to its current metadata file. The REST Catalog defines a common web interface for configuration, namespaces, and table operations. Different engines can use compliant Iceberg catalogs through one client implementation. The interface does not make those catalog products equivalent.
| Part | Job | Information handled | What the specifications define |
|---|---|---|---|
| Iceberg table | Represent table state | Schemas, partitions, snapshots | Table metadata |
| Iceberg catalog | Locate current metadata | Location and current pointer | Catalog role |
| REST Catalog | Connect engines and catalogs | Namespaces and operations | Client-server behavior |
Who decides what data may land?
The team that runs the receiving pipeline decides what data may land. Acceptance checks therefore belong where data enters. The UK Government Data Quality Framework is government guidance on managing data quality across its lifecycle. It calls for improvements as early in that lifecycle as possible.
If incoming data breaks a rule, it does not proceed as accepted data. Discovery, permissions, lineage, and later measurements remain valuable after acceptance.
An acceptance rule can use only information available when data enters. A later check remains necessary when its answer depends on information that becomes available after that point. It complements rather than replaces the acceptance decision.
Once the acceptance point is clear, leadership must assign the failure response.
Who owns the fix when data fails?
Responsibility follows the pipeline that accepted the data. The team operating that pipeline owns its acceptance rule and failure response. It changes the pipeline or its source handling when the rule fails.
The catalog serves a different responsibility. It exposes metadata, permissions, lineage, and activity records so other groups can find the asset and understand its history. Separating remediation from the team that can change the pipeline moves action away from the source.
The framework calls for accountability throughout the data lifecycle and correction at source. The remaining question is how this division works with Databricks in practice.
How does dltHub work with Databricks and Unity Catalog?
dltHub uses schema contracts in the pipeline, loads tables into Databricks, and makes them accessible through Unity Catalog. Its schema contracts control new tables, new columns, and changes to column properties. A contract in freeze mode raises an error and loads no data when an incoming change is not allowed. The pipeline owner defines the contract and responds to the failure.
dltHub then loads data into Databricks as Delta tables by default or as Apache Iceberg tables. The Databricks destination makes those tables accessible through Unity Catalog. The operator provides the Databricks environment and catalog; storage and credential settings depend on where the pipeline runs.
Checks after loading provide another control. dltHub data-quality checks, currently in public preview as of 30 October 2026, measure uniqueness, missing values, allowed values, and table metrics. Current checks record and expose outcomes after loading; they do not block that load.
| Stage | What dltHub does | What Unity Catalog does | Who acts |
|---|---|---|---|
| Acceptance | Enforces schema contracts in freeze mode | No role in the load decision | Pipeline owner |
| Loading | Writes Delta or Iceberg tables | Makes loaded tables accessible | Pipeline owner |
| After loading | Runs quality checks | Exposes governed assets | Pipeline owner |
| Governance | Publishes the loaded result | Controls access and records lineage | Governance and operating teams |
11 Pythonic Data Quality Recipes for every day provides 11 Python patterns with runnable notebooks. A separate metadata guide explains how people, pipelines, and agents use connected metadata context.
The Unity Catalog Pages and Genie integration is currently in private preview, as of 5 November 2026. It reads business definitions and asks people to clarify uncertain terms. It then builds the ingestion pipeline, model, transformation, and metric view. It checks loaded data against the definitions and sends proposed terms for human review. The overview article and technical walkthrough show this flow.
Catalog governance covers finding, controlling, and tracing accepted data. Pipeline governance covers the decision to accept incoming data.
The catalog shares the governed result. The pipeline decides what may enter, and its owner fixes what fails.
Frequently asked questions
Does Unity Catalog have separate licensing?
No separate license is required as of 13 November 2026. Unity Catalog is included with Databricks Premium and Enterprise tiers. Customers pay for platform compute and storage.
Does Unity Catalog govern AI assets?
Yes. Its generally available asset governance covers models, MCP servers, and functions. Connections are also documented as governed objects. Unity Catalog governance applies access controls, supports discovery, and captures lineage for relevant assets.
How is Unity Catalog different from a traditional data catalog?
Unity Catalog does more than record searchable metadata. It enforces access during Databricks interactions, captures lineage across attached workspaces, and records activity for audit work.
Book a call to try dltHub with Databricks and Unity Catalog.
What is dltHub?
A catalog records what landed; dltHub decides what may land. The agentic data layer on top of any data warehouse, on managed infrastructure.
Was this page helpful?