common.destination.dataset

SupportsReadableRelation Objects

class SupportsReadableRelation(Protocol)

[view_source]

A readable relation retrieved from a destination that supports it

columns_schema

Known dlt table columns for this relation

df

def df(chunk_size: int = None) -> Optional[DataFrame]

[view_source]

Fetches the results as data frame. For large queries the results may be chunked

Fetches the results into a data frame. The default implementation uses helpers in pandas.io.sql to generate Pandas data frame. This function will try to use native data frame generation for particular destination. For BigQuery: QueryJob.to_dataframe is used. For duckdb: `DuckDBPyConnection.df'

Arguments:

• chunk_size int, optional - Will chunk the results into several data frames. Defaults to None
• **kwargs Any - Additional parameters which will be passed to native data frame generation function.

Returns:

• Optional[DataFrame] - A data frame with query results. If chunk_size > 0, None will be returned if there is no more data in results

arrow

def arrow(chunk_size: int = None) -> Optional[ArrowTable]

[view_source]

fetch arrow table of first 'chunk_size' items

iter_df

def iter_df(chunk_size: int) -> Generator[DataFrame, None, None]

[view_source]

iterate over data frames tables of 'chunk_size' items

iter_arrow

def iter_arrow(chunk_size: int) -> Generator[ArrowTable, None, None]

[view_source]

iterate over arrow tables of 'chunk_size' items

fetchall

def fetchall() -> List[Tuple[Any, ...]]

[view_source]

fetch all items as list of python tuples

fetchmany

def fetchmany(chunk_size: int) -> List[Tuple[Any, ...]]

[view_source]

fetch first 'chunk_size' items as list of python tuples

iter_fetch

def iter_fetch(chunk_size: int) -> Generator[List[Tuple[Any, ...]], Any, Any]

[view_source]

iterate in lists of python tuples in 'chunk_size' chunks

fetchone

def fetchone() -> Optional[Tuple[Any, ...]]

[view_source]

fetch first item as python tuple

limit

def limit(limit: int, **kwargs: Any) -> "SupportsReadableRelation"

[view_source]

limit the result to 'limit' items

head

def head(limit: int = 5) -> "SupportsReadableRelation"

[view_source]

limit the result to 5 items by default

select

def select(*columns: str) -> "SupportsReadableRelation"

[view_source]

set which columns will be selected

__getitem__

def __getitem__(
        columns: Union[str, Sequence[str]]) -> "SupportsReadableRelation"

[view_source]

set which columns will be selected

__getattr__

def __getattr__(attr: str) -> Any

[view_source]

get an attribute of the relation

__copy__

def __copy__() -> "SupportsReadableRelation"

[view_source]

create a copy of the relation object

DBApiCursor Objects

class DBApiCursor(SupportsReadableRelation)

[view_source]

Protocol for DBAPI cursor

native_cursor

Cursor implementation native to current destination

SupportsReadableDataset Objects

class SupportsReadableDataset(Protocol)

[view_source]

A readable dataset retrieved from a destination, has support for creating readable relations for a query or table