Usage

You can run your preprocessing pipeline using Paidiverpy in several ways, typically requiring just one to three lines of code:

1. Python Package: Install the package and utilize it in your Python scripts.

   In [1]: from paidiverpy.pipeline import Pipeline
   
   In [2]: pipeline = Pipeline(config_file_path="../examples/config_files/config_simple2.yml")
   
   In [3]: pipeline.run()
   ☁ paidiverpy ☁  |       INFO | 2024-11-04 17:49:24 | Running step 0: raw - OpenLayer
   ☁ paidiverpy ☁  |       INFO | 2024-11-04 17:49:25 | Step 0 completed
   ☁ paidiverpy ☁  |       INFO | 2024-11-04 17:49:25 | Running step 1: colour_correction - ColourLayer
   ☁ paidiverpy ☁  |       INFO | 2024-11-04 17:49:25 | Step 1 completed
   

   In this example, we instantiate the Pipeline class and pass a configuration file containing the pipeline information and run the pipeline.

   For more details on the configuration file format, refer to the Configuration File section. The images will be processed as NumPy arrays.

   To view the pipeline details, simply print the pipeline object:

   In [5]: pipeline
   

   Raw

   Type: Raw

   ➔

   Colour_correction

   Type: Colour

   {
                   "name": "colour_correction",
                   "step_name": "colour",
                   "mode": "grayscale",
                   "test": false
                   }

   {
               "name": "raw",
               "step_name": "raw",
               "sample_data": "benthic_csv",
               "input_path": "/home/tobfer/.paidiverpy_cache/benthic_csv/images",
               "metadata_path": "/home/tobfer/.paidiverpy_cache/benthic_csv/metadata/metadata_benthic_csv.csv",
               "metadata_type": "CSV_FILE",
               "image_open_args": "PNG",
               "append_data_to_metadata": "/home/tobfer/.paidiverpy_cache/benthic_csv/metadata/appended_metadata_benthic_csv.csv",
               "output_path": "output",
               "n_jobs": 1,
               "rename": "datetime",
               "sampling": [
                 {
                     "name": "sampling",
                     "step_name": "sampling",
                     "mode": "percent",
                     "test": false,
                     "params": {
                         "value": 0.1
                     }
                 }
               ],
               "convert": [
                 {
                     "name": "convert",
                     "step_name": "convert",
                     "mode": "bits",
                     "test": false,
                     "params": {
                         "output_bits": 8
                     }
                 }
               ]
               }

   To see a thumbnail of the output images, run the following code:

   In [6]: pipeline.images
   

   Step: raw ►

   Image: 20180611T041441.000Z.PNG ►

   

   Image: 20180611T041601.000Z.PNG ►

   

   Image: 20180611T041821.000Z.PNG ►

   

   Image: 20180611T041831.000Z.PNG ►

   

   Image: 20180611T042201.000Z.PNG ►

   
   Step: colour_correction ►

   Image: 20180611T041441.000Z.PNG ►

   

   Image: 20180611T041601.000Z.PNG ►

   

   Image: 20180611T041821.000Z.PNG ►

   

   Image: 20180611T041831.000Z.PNG ►

   

   Image: 20180611T042201.000Z.PNG ►

   

   To save the output images in the specified output directory from the configuration file, use the following command:

   In [7]: pipeline.save_images(image_format="png")
   

   Please refer to our API reference section for more details on the available methods and their usage.

2. Command Line Interface (CLI): Execute the package via the command line.

   You can run the package using the CLI with the following command:

   paidiverpy -c "../examples/config_files/config_simple2.yml"
   

   This command runs the pipeline according to the configuration file, saving output images to the directory defined in the output_path. Please refer to our CLI and Docker reference section for more details on the available methods and their usage.

3. Docker: Use the Docker image to run the package.

   After building or pushing the Docker image, you can run the package with the following command:

   docker run --rm \
     -v <INPUT_PATH>:/app/input/ \
     -v <OUTPUT_PATH>:/app/output/ \
     -v <METADATA_PATH>:/app/metadata/ \
     -v <CONFIG_DIR>:/app/config_files/ \
     paidiverpy -c /app/examples/config_files/<CONFIG_FILE>
   

   In this command:

   • <INPUT_PATH>: Local directory containing input images (as defined in the configuration file).

   • <OUTPUT_PATH>: Local directory where processed images will be saved.

   • <METADATA_PATH>: Local directory containing the metadata file.

   • <CONFIG_DIR>: Local directory containing the configuration file.

   • <CONFIG_FILE>: Name of the configuration file.

   Please refer to our CLI and Docker reference section for more details on the available methods and their usage.

4. GUI: Use the graphical user interface to run the package.

   You need to have the panel package installed to use the GUI. If you don’t have it installed, you can do so with the following command:

   pip install panel
   

   With the package installed, you can run the package using the GUI by executing the following command:

   paidiverpy -gui
   

   The GUI will open in your default web browser, where you can select your configuration file and run the pipeline interactively. More details on the GUI can be found in the Graphical User Interface (GUI) section.