dlt.destinations.impl.lance.configuration

DEFAULT_LANCE_BUCKET_URL

active run dir

LanceStorageConfiguration Objects

@configspec(init=False)
class LanceStorageConfiguration(FilesystemConfigurationWithLocalFiles)

View source on GitHub

namespace_name

Name of subdirectory in bucket_url to use as namespace root. Leave empty to use bucket_url as namespace root.

branch_name

Name of branch to use for read/write table operations. Uses main branch if not set.

options

Options to pass to storage client. See available options at https://lance.org/guide/object_store/#general-configuration.

Will be merged with credentials-derived options (if present), with options taking precedence in case of conflicts.

LanceEmbeddingsCredentials Objects

@configspec
class LanceEmbeddingsCredentials(CredentialsConfiguration)

View source on GitHub

api_key

API key for embedding model provider.

LanceEmbeddingsConfiguration Objects

@configspec
class LanceEmbeddingsConfiguration(BaseConfiguration)

View source on GitHub

credentials

Credentials for embedding model provider. Leave empty if authentication is not required (e.g. local providers).

vector_column

Name of column to store vector embeddings in.

provider

Provider of model used to generate embeddings, e.g. cohere or openai.

Find all providers at https://github.com/lancedb/lancedb/tree/main/python/python/lancedb/embeddings.

name

Name of model used by provider to generate embeddings, e.g. embed-english-v3.0.

max_retries

Number of retries for embedding requests. Set to 0 to disable retries.

kwargs

Additional provider-specific keyword arguments passed to EmbeddingFunction.create().

LanceClientConfiguration Objects

@configspec
class LanceClientConfiguration(WithLocalFiles,
                               DestinationClientDwhConfiguration)

View source on GitHub

storage

Storage configuration including URI and cloud credentials.

embeddings

Optional embeddings configuration to add a vector embedding column. Leave empty to skip embedding generation.