dlt.common.libs.sqlglot
from_sqlglot_type
def from_sqlglot_type(sqlglot_type: DATA_TYPE) -> TColumnType
Convert a SQLGlot DataType to dlt column hints.
reference: https://dlthub.com/docs/general-usage/schema/#tables-and-columns
to_sqlglot_type
def to_sqlglot_type(dlt_type: TDataType,
precision: Optional[int] = None,
scale: Optional[int] = None,
timezone: Optional[bool] = None,
nullable: Optional[bool] = None,
use_named_types: bool = False) -> DataType
Convert the dlt data_type
and column hints to a SQLGlot DataType expression.
if use_named_type is True:
use "named" types with fallback on "parameterized" types.
else:
use "parameterized" types everywhere.
Named types:
- have some set attributes, e.g.,
DataType.Type.DECIMAL64
hasprecision=16
andscale=4
. - can be referenced directly in SQL (table definition, CAST(), which could make SQLGlot transpiling more reliable.
- may not exist in all dialects and will be casted automatically during transpiling
- can have limited expressivity, e.g., named types for timestamps only included precision 0, 3, 9
Parameterized types:
- instead of DECIMAL64, a generic
DataType.Type.DECIMAL
is used withDataTypeParam
expressions attached to representprecision
andscale
. SQLGlot is responsible for properly compiling and transpiling them. - parameters might be handled differently across dialects and would require greater testing on the dlt side.
reference: https://dlthub.com/docs/general-usage/schema/#tables-and-columns
set_metadata
def set_metadata(sqlglot_type: sge.DataType,
metadata: TColumnSchema) -> sge.DataType
Set a metadata dictionary on the SQGLot DataType object.
By attaching dlt hints to a DataType object, they will be propagated until the DataType is modified.
get_metadata
def get_metadata(sqlglot_type: sge.DataType) -> TColumnSchema
Get a metadata dictionary from the SQLGlot DataType object.