paidiverpy.pipeline

paidiverpy.pipeline#

__init__.py for pipeline module.

Submodules#

Classes#

Pipeline

Pipeline builder class for image preprocessing.

Package Contents#

class paidiverpy.pipeline.Pipeline(config_params: dict[str, Any] | paidiverpy.config.config_params.ConfigParams | None = None, config_file_path: str | None = None, config: paidiverpy.config.configuration.Configuration | None = None, metadata: paidiverpy.metadata_parser.MetadataParser | None = None, steps: list[tuple[str, type, dict[str, Any]]] | None = None, track_changes: bool | None = None, logger: logging.Logger | None = None, raise_error: bool = False, verbose: int = 2)[source]#

Bases: paidiverpy.Paidiverpy

Pipeline builder class for image preprocessing.

Parameters:

  • config_params (dict | ConfigParams, optional) – The configuration parameters. It can contain the following keys / attributes: - input_path (str): The path to the input files. - output_path (str): The path to the output files. - metadata_path (str): The path to the metadata file. - metadata_type (str): The type of the metadata file. - track_changes (bool): Whether to track changes. - n_jobs (int): The number of n_jobs.

  • config_file_path (str) – The path to the configuration file.

  • config (Configuration) – The configuration object.

  • metadata (MetadataParser) – The metadata object.

  • steps (list[tuple], optional) – The steps of the pipeline.

  • track_changes (bool) – Whether to track changes. Defaults to None, which means it will be set to the value of the configuration file.

  • logger (logging.Logger) – The logger object.

  • raise_error (bool) – Whether to raise an error.

  • verbose (int) – verbose level (0 = none, 1 = errors/warnings, 2 = info).

run(from_step: int | None = None, close_client: bool = True, save_images: bool = False) → None[source]#

Run the pipeline.

Parameters:

  • from_step (int, optional) – The step to start from. Defaults to None, which means the pipeline will start from the last runned step.

  • close_client (bool, optional) – Whether to close the client. Defaults to True.

  • save_images (bool, optional) – Whether to save the images after running the pipeline. Defaults to False.

Raises:
process_custom_algorithm(step_params: dict[str, Any], config_index: int) → paidiverpy.custom_layer.custom_layer.CustomLayer[source]#

Process a custom algorithm.

Parameters:

  • step_params (dict) – The parameters of the custom algorithm.

  • config_index (int) – The index of the configuration.

Raises:

  • ValueError – If the file path is not provided.

  • ValueError – If the file does not exist.

  • ValueError – If the custom algorithm does not have a ‘run’ method.

Returns:

An instance of the custom algorithm class.

Return type:

CustomLayer

export_config(output_path: str | None = None) → None | str[source]#

Export the configuration to a yaml file.

Parameters:

output_path (str, optional) – The path to save the configuration file.

Returns:

The config file as string if output_path is None,

otherwise None.

Return type:

None | str

add_step(step_name: str, step_class: type, parameters: dict[str, Any], index: int | None = None, substitute: bool = False) → None[source]#

Add a step to the pipeline.

Parameters:

  • step_name (str) – Name of the step.

  • step_class (str | type) – Class of the step.

  • parameters (dict) – Parameters for the step.

  • index (int, optional) – Index of the step. It is only used when you want to add a step in a specific position. Defaults to None.

  • substitute (bool, optional) – Whether to substitute the step in the specified index. Defaults to False.