lensai_profiler package

Submodules

lensai_profiler.metrics module

class lensai_profiler.metrics.Metrics(framework='tf')

Bases: object

calculate_brightness(image)

Calculate the brightness of an image.

Parameters:

image – A TensorFlow tensor representing an RGB image.

Returns:

A TensorFlow tensor containing the mean brightness of the image.

calculate_channel_histogram(image)

Calculate the histogram of the channels of an image.

Parameters:

image – A TensorFlow tensor representing an image.

Returns:

A TensorFlow tensor containing the histogram of the image channels.

calculate_channel_mean(image)

Calculate the mean of each channel of an image.

Parameters:

image – A TensorFlow tensor representing an image.

Returns:

A TensorFlow tensor containing the mean of each channel.

calculate_sharpness_laplacian(image)

Calculate the sharpness of an image using the Laplacian operator.

Parameters:

image – A TensorFlow tensor representing an RGB image.

Returns:

A TensorFlow tensor containing the sharpness of the image.

calculate_snr(image)

Calculate the Signal-to-Noise Ratio (SNR) of an image.

Parameters:

image_tensor – A TensorFlow tensor representing an RGB or RGBA image.

Returns:

A TensorFlow tensor containing the SNR of the image.

process_batch(images)

Process a batch of images and calculate various metrics.

Parameters:

images – A TensorFlow tensor representing a batch of images.

Returns:

A tuple containing the brightness, sharpness, channel mean, SNR, and channel histogram of the batch.

lensai_profiler.metrics.calculate_percentiles(x, probabilities, lower_percentile=0.01, upper_percentile=0.99)

Calculates percentiles from a PMF (Probability Mass Function) represented as two separate lists.

Parameters:

• x – List containing the x-values (possible values) in the distribution.

• probabilities – List containing the probabilities corresponding to the x-values.

• lower_percentile – Float between 0 and 1 (inclusive) specifying the lower percentile (default 0.01).

• upper_percentile – Float between 0 and 1 (inclusive) specifying the upper percentile (default 0.99).

Returns:

A tuple containing the lower and upper percentiles (x-values, float).

lensai_profiler.metrics.get_histogram_sketch(sketch, num_splits=30)

Reads a binary file, deserializes the content, and extracts the PMF.

Parameters:

• sketch – A probabilistic data structure representing the sketch of the distribution.

• num_splits – Number of splits for the PMF (default: 30).

Returns:

A tuple containing x-axis values and the PMF.

lensai_profiler.sketches module

class lensai_profiler.sketches.Sketches

Bases: object

A generic class to manage and update KLL sketches for custom-defined metrics.

Supports both TensorFlow and PyTorch tensors for compatibility with different deep learning frameworks.

compute_thresholds(lower_percentile=0.1, upper_percentile=0.99)

Compute the lower and upper percentile thresholds for all registered KLL sketches.

Parameters:

• lower_percentile (float) – Lower percentile value (default is 0.1).

• upper_percentile (float) – Upper percentile value (default is 0.99).

Returns:

A dictionary containing the computed thresholds for all sketches.

Return type:

dict

load_sketches(base_path)

Load all KLL sketches from binary files.

Parameters:

base_path (str) – Path to the base directory from which sketches will be loaded.

pt_update_sketches(**kwargs)

PyTorch-specific method to update KLL sketches by directly calling update_sketches.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding PyTorch tensor.

publish_sketches(folder_path, endpoint_url, sensor_id='reference')

Compress a folder and post it to an endpoint, then clean up intermediate files.

Parameters:

• folder_path (str) – The path to the folder to compress and post.

• endpoint_url (str) – The URL of the endpoint to post to.

• sensor_id (str) – The value for the ‘sensorid’ header.

register_metric(metric_name, num_channels=1)

Register a custom metric by name. Each metric will have a corresponding KLL sketch.

Parameters:

• metric_name (str) – The name of the custom metric.

• num_channels (int) – The number of channels for the metric (e.g., 3 for RGB images). Default is 1.

save_sketches(base_path)

Save all registered KLL sketches to binary files for later use.

Parameters:

base_path (str) – Path to the base directory where sketches will be saved.

tf_update_sketches(**kwargs)

TensorFlow-specific method to update KLL sketches using tf.py_function.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding TensorFlow tensor.

update_kll_sketch(sketch, values)

Update a KLL sketch with values from a tensor.

Parameters:

• sketch (datasketches.kll_floats_sketch) – The KLL sketch to be updated.

• values (TensorFlow or PyTorch tensor) – The tensor containing values to update the sketch with.

update_sketches(**kwargs)

Update all registered KLL sketches in parallel using the provided metric values.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding data tensor.

Module contents

class lensai_profiler.Metrics(framework='tf')

Bases: object

calculate_brightness(image)

Calculate the brightness of an image.

Parameters:

image – A TensorFlow tensor representing an RGB image.

Returns:

A TensorFlow tensor containing the mean brightness of the image.

calculate_channel_histogram(image)

Calculate the histogram of the channels of an image.

Parameters:

image – A TensorFlow tensor representing an image.

Returns:

A TensorFlow tensor containing the histogram of the image channels.

calculate_channel_mean(image)

Calculate the mean of each channel of an image.

Parameters:

image – A TensorFlow tensor representing an image.

Returns:

A TensorFlow tensor containing the mean of each channel.

calculate_sharpness_laplacian(image)

Calculate the sharpness of an image using the Laplacian operator.

Parameters:

image – A TensorFlow tensor representing an RGB image.

Returns:

A TensorFlow tensor containing the sharpness of the image.

calculate_snr(image)

Calculate the Signal-to-Noise Ratio (SNR) of an image.

Parameters:

image_tensor – A TensorFlow tensor representing an RGB or RGBA image.

Returns:

A TensorFlow tensor containing the SNR of the image.

process_batch(images)

Process a batch of images and calculate various metrics.

Parameters:

images – A TensorFlow tensor representing a batch of images.

Returns:

A tuple containing the brightness, sharpness, channel mean, SNR, and channel histogram of the batch.

class lensai_profiler.Sketches

Bases: object

A generic class to manage and update KLL sketches for custom-defined metrics.

Supports both TensorFlow and PyTorch tensors for compatibility with different deep learning frameworks.

compute_thresholds(lower_percentile=0.1, upper_percentile=0.99)

Compute the lower and upper percentile thresholds for all registered KLL sketches.

Parameters:

• lower_percentile (float) – Lower percentile value (default is 0.1).

• upper_percentile (float) – Upper percentile value (default is 0.99).

Returns:

A dictionary containing the computed thresholds for all sketches.

Return type:

dict

load_sketches(base_path)

Load all KLL sketches from binary files.

Parameters:

base_path (str) – Path to the base directory from which sketches will be loaded.

pt_update_sketches(**kwargs)

PyTorch-specific method to update KLL sketches by directly calling update_sketches.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding PyTorch tensor.

publish_sketches(folder_path, endpoint_url, sensor_id='reference')

Compress a folder and post it to an endpoint, then clean up intermediate files.

Parameters:

• folder_path (str) – The path to the folder to compress and post.

• endpoint_url (str) – The URL of the endpoint to post to.

• sensor_id (str) – The value for the ‘sensorid’ header.

register_metric(metric_name, num_channels=1)

Register a custom metric by name. Each metric will have a corresponding KLL sketch.

Parameters:

• metric_name (str) – The name of the custom metric.

• num_channels (int) – The number of channels for the metric (e.g., 3 for RGB images). Default is 1.

save_sketches(base_path)

Save all registered KLL sketches to binary files for later use.

Parameters:

base_path (str) – Path to the base directory where sketches will be saved.

tf_update_sketches(**kwargs)

TensorFlow-specific method to update KLL sketches using tf.py_function.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding TensorFlow tensor.

update_kll_sketch(sketch, values)

Update a KLL sketch with values from a tensor.

Parameters:

• sketch (datasketches.kll_floats_sketch) – The KLL sketch to be updated.

• values (TensorFlow or PyTorch tensor) – The tensor containing values to update the sketch with.

update_sketches(**kwargs)

Update all registered KLL sketches in parallel using the provided metric values.

Parameters:

**kwargs – Keyword arguments where the key is the metric name and the value is the corresponding data tensor.

lensai_profiler.calculate_percentiles(x, probabilities, lower_percentile=0.01, upper_percentile=0.99)

Calculates percentiles from a PMF (Probability Mass Function) represented as two separate lists.

Parameters:

• x – List containing the x-values (possible values) in the distribution.

• probabilities – List containing the probabilities corresponding to the x-values.

• lower_percentile – Float between 0 and 1 (inclusive) specifying the lower percentile (default 0.01).

• upper_percentile – Float between 0 and 1 (inclusive) specifying the upper percentile (default 0.99).

Returns:

A tuple containing the lower and upper percentiles (x-values, float).

lensai_profiler.get_histogram_sketch(sketch, num_splits=30)

Reads a binary file, deserializes the content, and extracts the PMF.

Parameters:

• sketch – A probabilistic data structure representing the sketch of the distribution.

• num_splits – Number of splits for the PMF (default: 30).

Returns:

A tuple containing x-axis values and the PMF.