extract.concurrency

FuturesPool Objects

class FuturesPool()

[view_source]

Worker pool for pipe items that can be resolved asynchronously.

Items can be either asyncio coroutines or regular callables which will be executed in a thread pool.

submit

def submit(pipe_item: ResolvablePipeItem) -> TItemFuture

[view_source]

Submit an item to the pool.

Arguments:

• pipe_item - The pipe item to submit. pipe_item.item must be either an asyncio coroutine or a callable.

Returns:

The resulting future object

resolve_next_future

def resolve_next_future(
        use_configured_timeout: bool = False) -> Optional[ResolvablePipeItem]

[view_source]

Block until the next future is done and return the result. Returns None if no futures done.

Arguments:

• use_configured_timeout - If True, use the value of self.poll_interval as the max wait time, raises concurrent.futures.TimeoutError if no future is done within that time.

Returns:

The resolved future item or None if no future is done.

resolve_next_future_no_wait

def resolve_next_future_no_wait() -> Optional[ResolvablePipeItem]

[view_source]

Resolve the first done future in the pool. This does not block and returns None if no future is done.