dlt.destinations.impl.lance.configuration
DEFAULT_LANCE_BUCKET_URL
active run dir
LanceCatalogType
Lance catalog type discriminator. Extend with ie. with "rest as they become available
LanceStorageConfiguration Objects
@configspec(init=False)
class LanceStorageConfiguration(FilesystemConfigurationWithLocalFiles)
namespace_name
Name of subdirectory in bucket_url to use as namespace root. Leave empty to use bucket_url as namespace root.
options
Options to pass to storage client. See available options at https://lance.org/guide/object_store/#general-configuration.
Will be merged with credentials-derived options (if present), with options taking precedence in case of conflicts.
DirectoryCatalogCredentials Objects
@configspec(init=False)
class DirectoryCatalogCredentials(FilesystemConfigurationWithLocalFiles)
Optional filesystem location for Lance directory catalog (__manifest root).
When bucket_url is empty, the catalog colocates with data (storage)
Setting bucket_url explicitly enables
Lance multi-base layout: __manifest in one bucket, data in another.
options
Object-store options for the catalog filesystem. Merged with credentials-derived options.
LanceCredentials
Polymorphic credentials resolved per catalog_type. Expand as new catalogs are added.
DirectoryCatalogCapabilities Objects
@configspec
class DirectoryCatalogCapabilities(LanceCatalogCapabilities)
manifest_enabled
V2 catalog tracking via a shared __manifest Lance table.
Enables fast listing and nested namespaces. Disable to fall back to directory scanning if concurrent writers on the same root cause manifest contention.
dir_listing_enabled
V1 fallback: discover tables by scanning directories for .lance suffixes.
LanceEmbeddingsCredentials Objects
@configspec
class LanceEmbeddingsCredentials(CredentialsConfiguration)
api_key
API key for embedding model provider.
LanceEmbeddingsConfiguration Objects
@configspec
class LanceEmbeddingsConfiguration(BaseConfiguration)
credentials
Credentials for embedding model provider. Leave empty if authentication is not required (e.g. local providers).
vector_column
Name of column to store vector embeddings in.
provider
Provider of model used to generate embeddings, e.g. cohere or openai.
Find all providers at https://github.com/lancedb/lancedb/tree/main/python/python/lancedb/embeddings.
name
Name of model used by provider to generate embeddings, e.g. embed-english-v3.0.
max_retries
Number of retries for embedding requests. Set to 0 to disable retries.
kwargs
Additional provider-specific keyword arguments passed to EmbeddingFunction.create().
LanceClientConfiguration Objects
@configspec
class LanceClientConfiguration(WithLocalFiles,
DestinationClientDwhConfiguration)
credentials
type: ignore[assignment]
storage
Storage configuration for table data (bucket, credentials, options, namespace subpath).
branch_name
Name of branch to use for read/write table operations. Uses main branch if not set.
embeddings
Optional embeddings configuration to add a vector embedding column.