Skip to main content
Version: devel

Data quality 🧪

dltHub

This page is for dltHub Feature, which requires a license. Join our early access program for a trial license.

warning

🚧 This feature is under development. Interested in becoming an early tester? Join dltHub early access.

dltHub will allow you to define data validation rules at the YAML level or using Pydantic models. This ensures your data meets expected quality standards at the ingestion step.

Example: Defining a quality contract in YAML​

You can specify quality contracts to enforce constraints on your data, such as expected value ranges and nullability.

engine_version: 10
name: scd_type_3
tables:
  customers:
    columns:
      category:
        data_type: bigint
        nullable: false
        quality_contracts:
          expect_column_max_to_be_between:
            min_value: 1
            max_value: 100

Key features​

With dltHub, you will be able to:

  • Define data tests and quality contracts using YAML configuration or Pydantic models.
  • Apply both row-level and batch-level validation.
  • Enforce constraints on distributions, boundaries, and expected values.

Stay tuned for updates as we expand these capabilities! 🚀

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.