paidiverpy.utils.object_store

paidiverpy.utils.object_store#

This module contains utility functions for interacting with object storage.

`define_storage_options`(→ dict[str, str])	Define storage options for reading metadata file.
`get_file_from_bucket`(→ bytes)	Get a file from an object store bucket.
`create_client`(→ boto3.client)	Create a boto3 client for S3.
`check_create_bucket_exists`(→ None)	Check if a bucket exists.
`upload_file_to_bucket`(→ None)	Upload an in-memory file to an object store bucket.
`path_is_remote`(→ bool)	Check if the path is a remote path.

paidiverpy.utils.object_store.define_storage_options(path: str | pathlib.Path) → dict[str, str][source]#

Define storage options for reading metadata file.

paidiverpy.utils.object_store.get_file_from_bucket(file_path: str, storage_options: dict[str, str] | None = None) → bytes[source]#

Get a file from an object store bucket.

Parameters:

file_path (str) – Full S3 path (e.g., “s3://my-bucket/path/to/image.png”).
storage_options (dict) – Storage options for reading metadata file. Defaults to None.

Returns:

The file content.

Return type:

bytes

paidiverpy.utils.object_store.create_client() → boto3.client[source]#

Create a boto3 client for S3.

paidiverpy.utils.object_store.check_create_bucket_exists(bucket_name: str, client: boto3.client) → None[source]#

Check if a bucket exists.

Parameters:

paidiverpy.utils.object_store.upload_file_to_bucket(file_obj: io.BytesIO, output_path: str, client: boto3.client) → None[source]#

Upload an in-memory file to an object store bucket.

Parameters:

file_obj (io.BytesIO) – In-memory file object.
output_path (str) – Full S3 path (e.g., “s3://my-bucket/path/to/image.png”).
client (boto3.client) – The boto3 client for S3.

paidiverpy.utils.object_store.path_is_remote(path: str | pathlib.Path) → bool[source]#

Check if the path is a remote path.