dlt.dataset.dataset
Dataset Objects
class Dataset()
Access to dataframes and arrow tables in the destination dataset via dbapi
ibis
def ibis(read_only: bool = False) -> IbisBackend
Get an ibis backend for the dataset.
This creates a connection to the destination.
The read_only
flag is currently only supported for duckdb destination.
schema
@property
def schema() -> dlt.Schema
dlt schema associated with the dataset.
If no provided at dataset initialization, it is fetched from the destination. Fallbacks to local dlt pipeline metadata.
tables
@property
def tables() -> list[str]
List of table names found in the dataset.
This only includes "completed tables". In other words, during the lifetime of a pipeline.run()
execution, tables may exist on the destination, but will only appear on the dataset once
pipeline.run()
is done.
_ipython_key_completions_
def _ipython_key_completions_() -> list[str]
Provide table names as completion suggestion in interactive environments.
sqlglot_schema
@property
def sqlglot_schema() -> SQLGlotSchema
SQLGlot schema of the dataset derived from the dlt schema.
destination_dialect
@property
def destination_dialect() -> TSqlGlotDialect
SQLGlot dialect of the dataset destination.
This is the target dialect when transpiling SQL queries.
dataset_name
@property
def dataset_name() -> str
Name of the dataset
is_same_physical_destination
def is_same_physical_destination(other: dlt.Dataset) -> bool
Returns true if the other dataset is on the same physical destination helpful if we want to run sql queries without extracting the data