dlt.destinations.impl.lance.configuration
DEFAULT_LANCE_BUCKET_URL
active run dir
LanceStorageConfiguration Objects
@configspec(init=False)
class LanceStorageConfiguration(FilesystemConfigurationWithLocalFiles)
namespace_name
Name of subdirectory in bucket_url to use as namespace root. Leave empty to use bucket_url as namespace root.
options
Options to pass to storage client. See available options at https://lance.org/guide/object_store/#general-configuration.
Will be merged with credentials-derived options (if present), with options taking precedence in case of conflicts.
DirectoryCatalogCredentials Objects
@configspec(init=False)
class DirectoryCatalogCredentials(FilesystemConfigurationWithLocalFiles)
Optional filesystem location for Lance directory catalog (__manifest root).
When bucket_url is empty, the catalog colocates with data (storage)
Setting bucket_url explicitly enables
Lance multi-base layout: __manifest in one bucket, data in another.
options
Object-store options for the catalog filesystem. Merged with credentials-derived options.
RestCatalogCredentials Objects
@configspec
class RestCatalogCredentials(CredentialsConfiguration)
Credentials for connecting to a Lance REST Namespace server.
uri
Base URI of the Lance REST Namespace server, e.g. http://127.0.0.1:2333.
api_key
API key to authenticate to the Lance REST Namespace server. Mapped to x-api-key HTTP header.
auth_token
Bearer token to authenticate to the Lance REST Namespace server. Mapped to Authorization: Bearer <auth_token> HTTP header.
LanceCredentials
Polymorphic credentials resolved per catalog_type.
DirectoryCatalogCapabilities Objects
@configspec
class DirectoryCatalogCapabilities(LanceCatalogCapabilities)
manifest_enabled
V2 catalog tracking via a shared __manifest Lance table.
Enables fast listing and nested namespaces. Disable to fall back to directory scanning if concurrent writers on the same root cause manifest contention.
dir_listing_enabled
V1 fallback: discover tables by scanning directories for .lance suffixes.
LanceEmbeddingsCredentials Objects
@configspec
class LanceEmbeddingsCredentials(CredentialsConfiguration)
api_key
API key for embedding model provider.
LanceEmbeddingsConfiguration Objects
@configspec
class LanceEmbeddingsConfiguration(BaseConfiguration)
credentials
Credentials for embedding model provider. Leave empty if authentication is not required (e.g. local providers).
vector_column
Name of column to store vector embeddings in.
provider
Provider of model used to generate embeddings, e.g. cohere or openai.
Find all providers at https://github.com/lancedb/lancedb/tree/main/python/python/lancedb/embeddings.
name
Name of model used by provider to generate embeddings, e.g. embed-english-v3.0.
max_retries
Number of retries for embedding requests. Set to 0 to disable retries.
kwargs
Additional provider-specific keyword arguments passed to EmbeddingFunction.create().
LanceNamespacePool Objects
class LanceNamespacePool()
Dispenses a shared namespace, session and storage options to job clients.
LanceClientConfiguration Objects
@configspec
class LanceClientConfiguration(WithAttachableEngine, WithLocalFiles,
DestinationClientDwhConfiguration)
credentials
type: ignore[assignment]
storage
Storage configuration for table data. Required for "dir" catalog, optional for "rest".
branch_name
Name of branch to use for read/write table operations. Uses main branch if not set.
embeddings
Optional embeddings configuration to add a vector embedding column.
always_refresh_views
Recreates views to get fresh data and schema of the tables for each query
manifest_enabled
@property
def manifest_enabled() -> Optional[bool]
Manifest mode setting of the catalog, None when the catalog has no such setting.
fingerprint
def fingerprint() -> str
Returns a fingerprint of the configured storage.
data_location
def data_location() -> str
Returns the Lance catalog root which identifies the namespace.
can_write_from
def can_write_from(other: DestinationClientConfiguration) -> bool
Lance does not have an engine that can write. dlt is that engine,
and returning False here enforces its usage.