lfcnn.generators package

Submodules

lfcnn.generators.abstracts module

class lfcnn.generators.abstracts.BaseGenerator(data, data_key, label_keys, augmented_shape, generated_shape, reshape_func, batch_size, range_data=None, range_labels=None, data_percentage=1.0, model_crop=None, augment=False, psnr=None, to_mono=False, shuffle=False, use_mask=False, mask_type=None, fix_seed=False)

Bases: Sequence

Parameters:

• data (Union[str, dict]) – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key (str) – Key of the data in the .h5 file.

• label_keys (Union[str, List[str], None]) – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape (Tuple[int, int, int, int, int]) – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape (List[Tuple[int, …]]) – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func (Callable) – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size (int) – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data (Union[float, int, None]) – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels (Union[float, List[Optional[float]], None]) – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage (float) – Decimal percentage of data to use.

• model_crop (Optional[tuple]) – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment (Union[bool, dict]) – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr (Optional[int]) – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono (bool) – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle (bool) – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask (bool) – Whether to apply a random coding mask to the input data.

• mask_type (Optional[str]) – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed (bool) – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

__getitem__(index)

Generate one batch of data and label(s)

Returns:

Tuple data, dict(label_name=label, …) where multi-label outputs are passed as a dictionary.

__len__()

Denotes the number of batches per epoch.

property augment

property augmented_shape

property batch_size

property data

property data_key

property data_percentage

property epoch

property fix_seed

property generated_shape: list

Return type:

list

property indices

property input_shape

property label_keys

property label_names

property model_crop

on_epoch_end()

Is called by Keras at the end of each epoch.

process_data(lf_batch, labels, curr_indices)

Processes a batch of sample and labels.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Batch of corresponding labels.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch, labels. The number of labels returned does not have to match the number of input labels, for example when labels are created from the light field batch itself (such as using the central view or the full light field as a label for an auto-encoder).

property range_data

property range_labels

read_curr_data_dict(indices_read)

Read a batch of sample and label(s) from a data dictionary.

Parameters:

• indices_read (Union[List[int], slice]) – List of indices or slice to read.

• Returns – sample: A batch of data (light fields) read from the data dict. labels: A list of labels read from the data dict.

Return type:

Tuple[any, List[any]]

read_curr_data_path(indices_read)

Read a batch of sample and label(s) from .h5 file.

Parameters:

• indices_read (Union[List[int], slice]) – List of indices or slice to read.

• Returns – sample: A batch of data (light fields) read from the .h5 file. labels: A list of labels read from the .h5 file.

Return type:

Tuple[any, List[any]]

reshape_data(lf_batch, labels)

Reshape a batch of light fields and labels.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Batch of corresponding labels.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of reshaped light field batch, labels.

property shuffle

shuffle_idx()

property to_mono

property total_len

property use_mask

lfcnn.generators.generators module

class lfcnn.generators.generators.CentralAndDisparityGenerator(downsample=False, *args, **kwargs)

Bases: BaseGenerator

Generates light field batches and central view as well as disparity labels.

Parameters:

• data – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key – Key of the data in the .h5 file.

• label_keys – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage – Decimal percentage of data to use.

• model_crop – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask – Whether to apply a random coding mask to the input data.

• mask_type – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

process_data(lf_batch, labels, curr_indices)

Processes a batch of light fields and labels.

Here, the light field’s central view and the disparity are returned as the labels.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Single element list containing a batch of disparities.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple (lf_batch, [disp_batch, central_view_batch]).

class lfcnn.generators.generators.CentralGenerator(downsample=False, *args, **kwargs)

Bases: BaseGenerator

Generates light field batches and central view labels.

Parameters:

• data – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key – Key of the data in the .h5 file.

• label_keys – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage – Decimal percentage of data to use.

• model_crop – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask – Whether to apply a random coding mask to the input data.

• mask_type – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

process_data(lf_batch, labels, curr_indices)

Processes a batch of light fields and labels.

Here, the light field’s central view is returned as the labels.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – None.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple (lf_batch, [central_view_batch]).

class lfcnn.generators.generators.DisparityGenerator(*args, **kwargs)

Bases: BaseGenerator

Generates light field batches and central view disparity labels.

Parameters:

• data – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key – Key of the data in the .h5 file.

• label_keys – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage – Decimal percentage of data to use.

• model_crop – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask – Whether to apply a random coding mask to the input data.

• mask_type – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

process_data(lf_batch, labels, curr_indices)

Processes a batch of light fields and disparity as label.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Single element list containing a batch of disparities.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple (lf_batch, [disp_batch]).

class lfcnn.generators.generators.LfDownSampleGenerator(du=1, dv=1, ds=2, dt=2, dch=1, bw=False, *args, **kwargs)

Bases: BaseGenerator

Generates downsampled light field batches and originally sized (and thus superresolved w.r.t. to the input) light field labels. Currently, downsampling is performed by nearest neighbor interpolation without anti-aliasing due to perfomance.

TODO: Add different downsampling strategies.

Parameters:

• du – Downsample factor of u-axis.

• dv – Downsample factor of v-axis.

• ds – Downsample factor of s-axis.

• dt – Downsample factor of t-axis.

• dch – Downsample factor of channel-axis.

process_data(lf_batch, labels, curr_indices)

Processes a batch of sample and labels.

Here, no label inputs are assumed. The input light field is downsampled while the label corresponds to the original light field. This can be used to train CNNs for light field super-resolution. By “downsampling” we here refer to a simple sub-view of the light field, taking every x-th element along the corresponding axis. No downsampling in the sense of signal processing (e.g. accounting for aliasing) is performed.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Batch of corresponding labels. Here, should be an empty list.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_downsampled_batch, [lf_batch]. The light field batch is also returned as the label.

class lfcnn.generators.generators.LfGenerator(*args, **kwargs)

Bases: BaseGenerator

Generates light field batches and light field labels.

Parameters:

• data – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key – Key of the data in the .h5 file.

• label_keys – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage – Decimal percentage of data to use.

• model_crop – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask – Whether to apply a random coding mask to the input data.

• mask_type – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

process_data(lf_batch, labels, curr_indices)

Processes a batch of sample and labels.

Here, no label inputs are assumed. The light field batch is also returned as the label list.

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Batch of corresponding labels. Here, should be an empty list.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch, [lf_batch]. The light field batch is also returned as the label.

class lfcnn.generators.generators.PredictGenerator(*args, **kwargs)

Bases: BaseGenerator

Generates light field batches for model prediction.

Parameters:

• data – Either a system path to a dataset .h5 file or an already loaded dictionary containing data and possibly labels. The data needs to be in the shape (N, u, v, s, t, num_ch), where N is the total number of available data, (u, v) is the angular, (s, t) the spatial and (num_ch) the spectral shape of the light field.

• data_key – Key of the data in the .h5 file.

• label_keys – Key or list of keys of the corresponding label(s). Specify None, if no label is read from the .h5 file.

• augmented_shape – Shape of the light field after augmentation. Must be of form (u, v, s, t, lambda). Here, (u, v) is the angular, (s, t) the spatial and lambda the color or spectral component. The angular component should be odd-valued. This guarantees the existence of a well-defined central view.

• generated_shape – List of Shapes of the light field that the generator generates. Can be a single element list for single input models.

• reshape_func – A reshape function. Reshapes the light field (and possibly the labels) into the final, desired shape. Performs the transformation from augmented_shape to generated_shape.

• batch_size – Size of the batches to generate. Depending on the amount of data, the very last batch of an epoch might be smaller.

• num_labels – Number of labels that the generator generates. This can be more than the length of label_keys, for example when labels are created dynamically.

• range_data – Dynamic range of the input data. If not None, the data is normalized (divided by range_data).

• range_labels – List of dynamic ranges of the input labels. If not None, the corresponding label is normalized (divided by range_label[i]). Length of the list has to match num_labels.

• data_percentage – Decimal percentage of data to use.

• model_crop – For models with spatial cropping, e.g. due to ‘valid’ padding, specify the crop (c_s, c_t) that is cropped from the spatial borders.

• augment – Whether to perform online augmentation. Either a single bool, or dictionary of bool values for every available augmentation key, see lfcnn.generator.utils.AUGMENTATIONS

• psnr – Optionally add noise to the input resulting in the specified PSNR in decibels. If None, no noise is added.

• to_mono – Whether to convert to light field to mono. Currently only implemented for RGB light fields using the CIE standard.

• shuffle – Whether to shuffle the indices. The current epoch is used as a seed to be reproducible and to guarantee that the same shuffle across multiple processes when multiprocessing is used. Note that shuffling decreases the reading speed (in particular when reading from a file) and might limit the training performance.

• use_mask – Whether to apply a random coding mask to the input data.

• mask_type – When applying a coding mask, defines which mask to use. See Also: :func:lfcnn.generators.utils._create_lf_mask() Defaults to “random”.

• fix_seed – Whether to use a constant seed for random ops. Can be set to obtain reproducible results, most importantly for validation and testing generators.

process_data(lf_batch, labels, curr_indices)

Processes a batch of sample and labels.

Here, no label inputs are assumed. No label is returned

Parameters:

• lf_batch (any) – Batch of light fields.

• labels (List[any]) – Batch of corresponding labels. Here, should be an empty list.

• curr_indices (List[int]) – List of current sample indices.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch, None.

lfcnn.generators.reshapes module

lfcnn.generators.reshapes.lf_crosshair(lf_batch, labels)

Generates a crosshair view of the light field. Strictly speaking, this is not a pure reshape, but generates a list of output views.

(batchsize, u, v, s, t, lambda) to [(batchsize, v, s, t, lambda), # horizontal (batchsize, u, s, t, lambda), # vertical (batchsize, sqrt(u**2 + v**2), s, t, lambda), # lr-ud diagonal (batchsize, sqrt(u**2 + v**2), s, t, lambda),] # lr-du diagonal

Return type:

Tuple[dict, List[any]]

Returns:

Tuple of dict(lf_batch_reshaped), labels

lfcnn.generators.reshapes.lf_crosshair_channel_stacked(lf_batch, labels)

Generates a channel-stacked crosshair view of the light field as for example used by the Epinet disparity estimator. Strictly speaking, this is not a pure reshape, but generates a list of output views.

(batchsize, u, v, s, t, lambda) to [(batchsize, s, t, u, lambda), # horizontal (batchsize, s, t, v, lambda), # vertical (batchsize, s, t, sqrt(u**2 + v**2), lambda), # lr-ud diagonal (batchsize, s, t, sqrt(u**2 + v**2), lambda)] # lr-du diagonal

Return type:

Tuple[dict, List[any]]

Returns:

Tuple of dict(lf_batch_reshaped), labels

lfcnn.generators.reshapes.lf_crosshair_stacked(lf_batch, labels)

Generates a stacked crosshair view of the light fields. Strictly speaking, this is not a pure reshape, but generates a list of output views.

(batchsize, u, v, s, t, lambda) to [(batchsize, s, t, u*lambda), # horizontal (batchsize, s, t, v*lambda), # vertical (batchsize, s, t, sqrt(u**2 + v**2)*lambda), # lr-ud diagonal (batchsize, s, t, sqrt(u**2 + v**2)*lambda)] # lr-du diagonal

Return type:

Tuple[dict, List[any]]

Returns:

Tuple of dict(lf_batch_reshaped), labels

lfcnn.generators.reshapes.lf_distributed(lf_batch, labels)

The labels are the original light field in its original shape. The light field subapertures are contained in the first non-batch axis. This can be used for pseudo 4D convolutional networks.

(batchsize, u, v, s, t, lambda) to (batchsize, u*v, s, t, lambda).

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch_reshaped, labels

lfcnn.generators.reshapes.lf_identity(lf_batch, labels)

Returns the input batch and labels.

Return type:

Tuple[any, List[any]]

Returns:

lf_batch, labels

lfcnn.generators.reshapes.lf_subaperture_channel_stack(lf_batch, labels)

The light field subapertures are stacked retaining color/spectrum channels. This can be used for pseudo 3D convolutional networks. The light fields are reshaped from

(batchsize, u, v, s, t, lambda) to (batchsize, s, t, u*v, lambda).

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch_reshaped, labels

lfcnn.generators.reshapes.lf_subaperture_stack(lf_batch, labels)

The light field subapertures are stacked together with color/spectrum channels. The light fields are reshaped from

(batchsize, u, v, s, t, lambda) to (batchsize, s, t, u*v*lambda).

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch_reshaped, labels

lfcnn.generators.reshapes.lf_subaperture_stream(lf_batch, labels)

The light field subapertures are used for a multi-input stream. I.e., the generated input is a list of the light field’s subapertures corresponding to a transformation/reshape from

(batchsize, u, v, s, t, lambda) to [(batchsize, s, t, lambda), …, (batchsize, s, t, lambda)] with u*v elements.

Return type:

Tuple[any, List[any]]

Returns:

Tuple of lf_batch_reshaped, labels

lfcnn.generators.utils module

class lfcnn.generators.utils.AugmentResult

Bases: object

Class to hold results of random augmentation operations. This can be used if a corresponding label augmentation has to be performed with the same parameters.

The result, for example a randomly chosen scaling factor or a permutation list, is stored in the result attribute. The result attribute should always hold a dictionary. The value can then be obtained by calling an AugmentResult instance with the corresponding dictionary key.

add_result(key, value)

property result

lfcnn.generators.utils._is_iterable(x)

Check whether variable is iterable

lfcnn.generators.utils._reset_seed(seed=None)

Re-sets the Numpy random seed, if necessary.

lfcnn.generators.utils._set_seed(seed=None)

Sets the Numpy random seed, if necessary.

lfcnn.generators.utils._to_list(x)

Convert a variable into a list. If input is not iterable, make it iterable (and mutable).

lfcnn.generators.utils.disp_batch_augment(disp_batch, augment, aug_res)

Perform a full disparity batch augmentation in accordance with the light field augmentation lf_batch_augment().

Not all light field augmentation have a analogous disparity augmentation. For example, the disparity is invariant under light field channel weighting and permutation.

Parameters:

• disp_batch (any) – Disparity batch of shape (b, s, t)

• crop_only – Which augmentations to perform. Cropping is always performed.

• aug_res (AugmentResult) – AugmentResult instance which holds the results from the previous light field batch augmentation.

Returns:

disp_batch_augmented Augmented disparity batch.

lfcnn.generators.utils.disp_batch_crop_spatial(disp_batch, aug_res)

Crop the disparity batch to target spatial shape according to the crop region that the light field batch was cropped with.

Parameters:

• disp_batch – Batch of disparities. Shape (b, s, t).

• aug_res (AugmentResult) – Augmentation result of the corresponding lf_batch augmentation.

Returns:

lf_cropped Shape (b, u, v, s_crop, t_crop, lambda).

lfcnn.generators.utils.disp_batch_flip(disp_batch, aug_res)

Flip disparities when corresponding light fields have been flipped.

Parameters:

• disp_batch – Batch of disparities Shape (b, s, t).

• aug_res (AugmentResult) – Augmentation result of the corresponding lf_batch augmentation.

Returns:

disp_batch_flipped Shape is unchanged.

lfcnn.generators.utils.disp_batch_rotate(disp_batch, aug_res)

Random rotation through 0, 90, 180, 270 degree.

Parameters:

• disp_batch – Batch of disparities. Shape (b, s, t).

• aug_res – Augmentation result of the corresponding lf_batch augmentation.

Returns:

disp_batch_rotated The shape is unchanged.

lfcnn.generators.utils.disp_batch_scale(disp_batch, aug_res)

Random scaling with a scale factor within scale_range. For speed, the 0 order spline interpolation (nearest) is used. Note that all light fields in a batch are scaled with the same scale factor such that shape compatibility is assured.

Parameters:

• disp_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• aug_res (AugmentResult) – Augmentation result of the corresponding lf_batch augmentation.

Returns:

disp_batch_scaled Shapes (b, s_scaled, t_scaled, lambda)

lfcnn.generators.utils.get_batch_seeds(curr_indices, fix_seed, offset=0)

Create a list of seeds for the current batch. The data index is used as a seed, so when the seed is fixed, all random batch operations are reproducible.

Return type:

List[Optional[int]]

lfcnn.generators.utils.lf_batch_add_noise(lf_batch, psnr, curr_indices, fix_seed)

Apply noise to a batch of light fields (in-place).

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• psnr – Peak signal-to-noise ratio to achieve in decibels.

• curr_indices – List of current light field indices.

• fix_seed – Whether to fix the seed of all random operations.

Returns:

lf_batch_coded Shape is unchanged.

lfcnn.generators.utils.lf_batch_augment(lf_batch, augmented_shape, curr_indices, fix_seed, augment, aug_res=None)

Perform a full light field batch augmentation.

Augmentation consists of (in the presented order)

• Angular crop (depending on augmented_shape)

• Spectral crop (depending on augmented_shape)

• Random horizontal and vertical flip (if crop_only is False)

• Random rotation by multiples of 90° (if crop_only is False)

• Random channel weighting (if crop_only is False)

• Random channel permutation (if crop_only is False)

• Random spatial scaling (if crop_only is False)

• Spatial crop to final augmented_shape.

Parameters:

• lf_batch (any) – Light field batch of shape (b, u, v, s, t, num_ch)

• augmented_shape (tuple) – Target shape after augmentation.

• curr_indices (List[int]) – List of current light field indices.

• fix_seed (bool) – Whether to fix the seed of all random operations.

• augment (dict) – Dictionary specifying which augmentations to perform. Cropping to final shape is always performed.

• aug_res (Optional[AugmentResult]) – AugmentResult instance which can be used for possible label augmentations.

Returns:

lf_batch_augmented Augmented light field batch of shape (b, augmented_shape)

lfcnn.generators.utils.lf_batch_code(lf_batch, mask_type, curr_indices, fix_seed)

Apply spectral coding mask to a batch of light fields (in-place).

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• mask_type – Which mask to use as string. See Also: :func:_create_lf_mask()

• curr_indices – List of current light field indices.

• fix_seed – Whether to fix the seed of all random operations.

Returns:

lf_batch_coded Shape is unchanged.

lfcnn.generators.utils.lf_batch_crop_angular(lf_batch, crop_shape)

Crop the light field batch to target angular shape. The light field is always cropped around the center.

Parameters:

• lf_batch (any) – Batch of light fields. Shape (b, u, v, s, t, lambda).

• crop_shape – Size of the angular cropped data. Shape (u_crop, v_crop). Should be odd-valued, otherwise behaviour might not be as expected.

Return type:

any

Returns:

lf_cropped Shape (b, u_crop, v_crop, s, t, lambda).

lfcnn.generators.utils.lf_batch_crop_spatial(lf_batch, crop_shape, batch_seeds, aug_res=None)

Crop the light field batch to the target spatial shape. For better memory performance, the same crop is applied to all light fields in the batch.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• crop_shape – Size of the spatially cropped data. Shape (s_crop, t_crop).

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_cropped Shape (b, u, v, s_crop, t_crop, lambda).

lfcnn.generators.utils.lf_batch_crop_spectral(lf_batch, num_ch_cropped, batch_seeds, aug_res=None)

Crop the light field and disparity to target spectral/color shape. A random subset of the color channels is used.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• num_ch_cropped (int) – Number of spectral/color channels in the cropped light field. Needs to be smaller than input number of channels.

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_cropped Shape (b, u, v, s, t, num_ch_cropped).

lfcnn.generators.utils.lf_batch_flip(lf_batch, batch_seeds, aug_res=None)

Flip light fields in light field batch horizontally with a 50:50 chance. Vertical flipping is not performed since it is equivalent to rotation by 180 degrees and horizontal flipping.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_flipped Shape is unchanged.

lfcnn.generators.utils.lf_batch_gamma_channels(lf_batch, gamma_range, batch_seeds, aug_res=None)

Randomly weigh the color/spectral channels. The light field values are multiplied by a random weight factor which is for every channel chosen uniformly from the weight_range interval.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• gamma_range – Tuple (a, b). A gamma value is sampled from a uniform distribution over [a, b]

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_weighted Shape is unchanged.

lfcnn.generators.utils.lf_batch_permute_channels(lf_batch, batch_seeds, aug_res=None)

Randomly permute the order of color/spectral channels. :param lf_batch: Batch of light fields.

Shape (b, u, v, s, t, lambda).

Parameters:

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_permuted Shape is unchanged.

lfcnn.generators.utils.lf_batch_rotate(lf_batch, batch_seeds, aug_res=None)

Rotate each light field randomly by 0, 90, 180 or 270 degrees.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_rotated The shape is unchanged.

lfcnn.generators.utils.lf_batch_scale(lf_batch, scale_range, batch_seeds, aug_res=None)

Random scaling with a scale factor within scale_range. For speed, the 0 order spline interpolation (nearest) is used. Note that all light fields in a batch are scaled with the same scale factor such that shape compatibility is assured.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• scale_range – Tuple (a, b). A scaling factor is sampled from a uniform distribution over [a, b]

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_scaled Shapes (b, u, v, s_scaled, t_scaled, lambda)

lfcnn.generators.utils.lf_batch_to_mono(lf_batch)

Convert light field batch to mono. Throughout LFCNN, the channel axis is kept explicitly.

Parameters:

lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

Returns:

lf_batch_mono of shape (b, u, v, s, t, 1).

lfcnn.generators.utils.lf_batch_weigh_channels(lf_batch, weight_range, batch_seeds, aug_res=None)

Randomly weigh the color/spectral channels. The light field values are multiplied by a random weight factor which is for every channel chosen uniformly from the weight_range interval.

Parameters:

• lf_batch – Batch of light fields. Shape (b, u, v, s, t, lambda).

• weight_range – Tuple (a, b). A weight is sampled from a uniform distribution over [a, b]

• batch_seeds – Seeds for the current batch.

• aug_res (Optional[AugmentResult]) – Result of the random augmentation operations. Can be used to pass to label augmentation.

Returns:

lf_batch_weighted Shape is unchanged.

lfcnn.generators.utils.model_crop_spatial(labels, model_crop)

For models with spatial cropping (e.g. padding=’valid’, crop the spatial dimensions of the given labels.

lfcnn.generators.utils.shape_wrapper(input_shape)

Converts a generated_shapes attribute into the form needed by LFCNN. I.e. if input is a single tuple, will warp result in a list, if input is a single list, will convert to a tuple and wrap in a list, if input is a list of lists, will convert inner lists to tuples, if input is a list of tuple, nothing changes.

Parameters:

input_shape (Union[list, tuple]) – Shape-linke input.

Return type:

List[tuple]

Returns:

List of tuples of shapes.

lfcnn.generators.utils.single_seed_setter(f)

Decorator function to ensure proper seeding.

Module contents

The LFCNN generators module.

lfcnn.generators.get(generator)

Given a generator name, returns an lfcnn generator instance.

Parameters:

generator (str) – Name of the generator.

Returns:

Generator instance.