Discover

· Updated · 9 min read

Schema drift vs schema evolution: a policy for each change

This guide treats schema drift as the source event and schema evolution as the chosen response. It shows what your pipeline should do with each kind of change and how to watch for changes in meaning.

Summarize withClaudeChatGPT

At 2 a.m., a source changes a column's type; your stop rule catches it, and the run fails. The next night, a source keeps every column name and type, but the values now mean something else. That run stays green. The first failure is loud, bounded, and reversible, while the second loads incorrect data into the reports that read it. A common answer is to let every change evolve automatically and repair the consequences later in dbt or a dashboard. That answer treats both nights the same.

What is schema drift vs schema evolution?

Schema drift is a structural or semantic change in incoming data. Schema evolution is the policy-driven response that accepts the change, stops the run, or drops the affected data. This guide uses the pair as a working distinction: drift is the event your source sends, and evolution is the response you choose before it arrives.

Drift comes in two forms. A structural change alters the schema: a field disappears, appears, or gets a new name. A semantic change alters what the values mean or which values are valid, while the names and types stay usable. Kramer et al. use the same split for pipelines that receive data in batches, and call the response to a change adaptation. Their paper sets out a vision for future pipelines, so its ideas are proposals, not tested results.

This guide covers changing inputs to data pipelines. It separates two jobs: noticing a significant change, and deciding, applying, and checking a response to it. A pipeline can do the first job well and still have no plan for the second. That plan depends on the kind of change.

What should your pipeline do with each kind of change?

Decide per change, by blast radius and reversibility. Blast radius is how much data and how many downstream users a change can affect, and reversibility is how hard the change is to undo. Accept a low-risk change that you can undo. Stop the run when a change breaks a downstream contract. Drop affected data only under a rule you declared. Send a change in meaning to a person.

Kramer et al. describe four levels of schema robustness: whether an operator, one processing step, keeps working after the schema changes. The levels build on each other, so they describe what an operator can survive, not four separate kinds of source change. The table maps each level to a response under the rule above.

Robustness levelWhen it appliesSource exampleHow you detect itResponse
IndifferenceThe operator does not use the propertyA change to an unused fieldCheck which operators use itAccept, if no downstream contract uses it
Loss of informationThe new schema is a subset of the oldA property disappearsSchema comparisonStop or drop, by contract and impact
Information changeExisting information changes, such as a renameA property gets a new nameSchema comparisonMap the rename before processing continues
Information addedThe new schema is a superset of the oldA new property appearsSchema comparisonAccept if low-risk and reversible

Type changes fall outside these four levels, so treat them by the contract rule: if a type change breaks a downstream contract, stop the run, as on the first night above. A rule like this works only when you declare it before the run, and a declared check needs a threshold that you already know. No row in the table catches a value that changes meaning while its name and type stay the same.

Why can a pipeline run and still be wrong?

A schema check sees only names and types. A structural change alters those, so a comparison catches it. A semantic change alters what values mean, which values are valid, or how they are distributed, and execution can continue without an error.

Kramer et al. name the two outcomes. In the first, an operator stops working, and the run ends with a runtime error. In the second, the pipeline still runs, but an operator produces incorrect results. Nothing at the execution layer reports the second outcome.

Their eye-tracking example shows how this happens. When the experiment's physical screen is replaced, Fixation-X and Fixation-Y stay in the data, but their distribution and valid intervals change. The schema stays the same, and the existing processing produces incorrect results. No structural check can say whether the new coordinates still mean what the analysis assumes. To see that change, you compare the values, not the schema.

How do you detect when values change meaning?

Compare a profile of each batch with the batch before, and learn the limits from past runs. A schema comparison finds structural differences, and a profile comparison finds changes in statistics and error patterns. Kramer et al. propose a data profile and an error profile for each batch, with a diff against the previous batch. Each profile holds metadata, so the diff does not compare raw records:

ProfileWhat it records
Data profileProperty names and types
Missing and distinct counts
Minimum, maximum, mean, median, standard deviation, quantiles
Value-frequency distributions
Error profileAffected properties, error types, and their counts

Fixed thresholds miss metrics that vary naturally, so learn the limits from past runs. Auto-Validate by-History (AVH), a research method from a 2023 paper, learns an acceptance interval for each column's metrics from earlier runs. It keeps only the checks that stay under a false-positive budget that you set. In its evaluation on 2,000 recurring Microsoft pipelines, the authors report competitive results with seven days of history and the best precision with 28 days. Deequ, a system for declarative data-quality checks on Spark, keeps a history of the metrics it computes. Its hasNoAnomalies check fails when a new value is anomalous against that history. These methods tell you that behavior changed, not what a value means for your business. Detection also gets harder when two changes arrive in the same batch.

What if several changes happen together?

Work out how the changes depend on each other before you apply any, rather than applying them in the order your detectors report them. When changes coincide, Kramer et al. propose finding the dependencies between them first, so that the remaining work forms a set of independent change steps.

Their example combines a rename with a distribution shift. A device update renames Fixation-X to Fixation-Screen-X, and a new screen changes the value distribution. First, establish that Fixation-Screen-X is the renamed Fixation-X. Then judge the distribution change against that same property. In the other order, the old field looks removed, and the new one looks added, and the link between them is lost.

The rule from this example is to resolve identity before you judge meaning. Even in the right order, an automated fix can make a pipeline run again without making its data right again.

When should self-healing stop for human review?

Stop when the decision is about correctness, not operation. Automation can restore a run for a failure it has seen before and that an approved response covers. Kramer et al. show the split: adapting to the rename lets execution continue, while the replaced screen needs its intended data quality restored. They call a pipeline that detects and reports change self-aware, and keep the term self-adapting for one that triggers or performs the adaptation.

Send three kinds of decision to a person: values that change meaning, quality that only a domain expert can judge, and fixes with a wide or hard-to-reverse effect. In the eye-tracking case, a person must decide the new valid intervals and the cleaning parameters.

Every proposed fix should carry the diagnosis, the blast radius, and the track record. The diagnosis tells the reviewer what failed and why. The blast radius shows what the fix touches, and the track record shows how often the fix worked before. Review fails when the reviewer rubber-stamps a proposal without checking that evidence. A contract enforced before load is one way to make the structural half of this policy concrete.

How does dltHub enforce a schema-change policy?

dltHub enforces a contract you declare before load, then explains a failed run so that a person can decide the fix. Schema contracts set one of four modes on three scopes: tables, columns, and data_type. dltHub enforces them before data reaches the destination. In the default evolve mode, schema evolution keeps old columns and adds new ones. That default accepts every change, so switch tables that feed a downstream contract to freeze. The first table shows the contract modes, and the second shows the default for each source change.

ModeWhat happens to violating data
evolveAccepted, and the schema changes
freezeDataValidationError; nothing from that load lands
discard_rowThe row is dropped; other rows load
discard_valueThe value is dropped; the row loads
Source changeDefault behavior
Field addedA new column is added
Value cannot be coerced to the typeA variant column is created
Field removedThe column stays and gets no new values
Field renamedA new column; the old column stays

A removed or renamed field leaves an old column with no new values. A nullable: False hint on that column fails the run before load when later records omit the field, but it cannot tell you which change caused it. When a freeze contract fails, this sequence follows:

  1. The run fails, and nothing from that load reaches the destination.
  2. A run-failure alert goes to Slack or email.
  3. The job-inspector agent, currently in preview as of 1/11/26, reads the trace, logs, and job definition, then returns the likely cause, its confidence, a proposed fix, and supporting evidence.
  4. You review the proposal and apply or reject the fix. The agent is read-only.

The failure and the alert detect and report the change, which Kramer et al. call self-aware. The job inspector then proposes the adaptation, and a person decides. On dltHub's autonomy roadmap from September 2026, repair without approval comes later, and only for failures seen before. Each action is rated by blast radius and reversibility. Dataset repair often stays with a reviewer, because deciding that successful-looking data is wrong needs human judgment.

What should you decide before the next source change?

Write down a response for each kind of change before the next run. Start with one production table. Record what happens when an unused property changes, when a property disappears, when one is renamed, and when one is added: accept, stop, or drop, and whether a person reviews it. Handle type changes and changes in meaning separately, outside the four levels.

Keep a person on changes in meaning and on fixes that are wide or hard to reverse. A recovered run proves that the pipeline operates, but restored meaning still takes judgment.

Book a call to try dltHub on one table: set a freeze contract, change the source, and watch job-inspector explain the failed run.

Was this page helpful?