dlt.destinations.impl.lancedb.utils

create_in_filter

def create_in_filter(field_name: str, array: pa.Array) -> str

View source on GitHub

Filters all rows where field_name is one of the values in the array

If array is dictionary-encoded (pa.DictionaryType) we emit the distinct values stored in its dictionary.

write_records

def write_records(records: DATA,
                  *,
                  db_client: DBConnection,
                  table_name: str,
                  vector_field_name: str,
                  write_disposition: Optional[TWriteDisposition] = "append",
                  merge_key: Optional[str] = None,
                  remove_orphans: Optional[bool] = False,
                  delete_condition: Optional[str] = None) -> None

View source on GitHub

Inserts records into a LanceDB table with automatic embedding computation.

Arguments:

• records - The data to be inserted as payload.
• db_client - The LanceDB client connection.
• table_name - The name of the table to insert into.
• merge_key - Keys for update/merge operations.
• write_disposition - The write disposition - one of 'skip', 'append', 'replace', 'merge'.
• remove_orphans bool - Whether to remove orphans after insertion or not (only merge disposition).
• filter_condition str - If None, then all such rows will be deleted. Otherwise, the condition will be used as an SQL filter to limit what rows are deleted.

Raises:

• ValueError - If the write disposition is unsupported, or id_field_name is not provided for update/merge operations.