Skip to main content
Version: devel

dlt.common.libs.sqlglot

from_sqlglot_type

def from_sqlglot_type(sqlglot_type: DATA_TYPE) -> TColumnType

View source on GitHub

Convert a SQLGlot DataType to dlt column hints.

reference: https://dlthub.com/docs/general-usage/schema/#tables-and-columns

to_sqlglot_type

def to_sqlglot_type(dlt_type: TDataType,
precision: Optional[int] = None,
scale: Optional[int] = None,
timezone: Optional[bool] = None,
nullable: Optional[bool] = None,
use_named_types: bool = False) -> DataType

View source on GitHub

Convert the dlt data_type and column hints to a SQLGlot DataType expression.

if use_named_type is True: use "named" types with fallback on "parameterized" types. else: use "parameterized" types everywhere.

Named types:

  • have some set attributes, e.g., DataType.Type.DECIMAL64 has precision=16 and scale=4.
  • can be referenced directly in SQL (table definition, CAST(), which could make SQLGlot transpiling more reliable.
  • may not exist in all dialects and will be casted automatically during transpiling
  • can have limited expressivity, e.g., named types for timestamps only included precision 0, 3, 9

Parameterized types:

  • instead of DECIMAL64, a generic DataType.Type.DECIMAL is used with DataTypeParam expressions attached to represent precision and scale. SQLGlot is responsible for properly compiling and transpiling them.
  • parameters might be handled differently across dialects and would require greater testing on the dlt side.

reference: https://dlthub.com/docs/general-usage/schema/#tables-and-columns

set_metadata

def set_metadata(sqlglot_type: sge.DataType,
metadata: TColumnSchema) -> sge.DataType

View source on GitHub

Set a metadata dictionary on the SQGLot DataType object.

By attaching dlt hints to a DataType object, they will be propagated until the DataType is modified.

get_metadata

def get_metadata(sqlglot_type: sge.DataType) -> TColumnSchema

View source on GitHub

Get a metadata dictionary from the SQLGlot DataType object.

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.