db

Database utils module.

class mltb2.db.AbstractBatchDataManager

Bases: ABC

Abstract base class for batch processing of database data.

This class (respectively an implementation of it) is intended to be used in conjunction with the BatchDataProcessor.

abstract load_batch() → Sequence

Load a batch of data from the database.

Return type:

Sequence

abstract save_batch(batch: Sequence) → None

Save a batch of data to the database.

Parameters:

batch (Sequence) –

Return type:

None

class mltb2.db.BatchDataProcessor(data_manager: AbstractBatchDataManager, process_batch_callback: Callable[[Sequence], Sequence])

Bases: object

Process batches of data from a database.

Parameters:

• data_manager (AbstractBatchDataManager) – The data manager to load and save batches of data.

• process_batch_callback (Callable[[Sequence], Sequence]) – A callback function that processes one batch of data.

run() → None

Run the batch data processing.

This is done until the data manager returns an empty batch. For each batch the process_batch_callback is called. Data is loaded by using an implementation of the AbstractBatchDataManager.

Return type:

None