In the previous sections, you learnt how to leverage the SDKs to display different image in 4V using a Quad, as well as convert other supported image formats to a Quad using the built in Image decoders and synthesizers.

In this section, you will learn about the decoding process, and how to leverage the SDK to perform operations on the different image formats in your android applications.

What is a Multiview Image?

The MultiviewImage is a collection of ViewPoints, and global metadata that affects all views like gain, convergence and depth of field. It is a decoded representation of an image that can be rendered on a Leia Lightfield display.

A ViewPoint represents the image and disparity map from a single camera view point.

The Multiview Image Decoder can be used to decode the different image formats supported by the SDK and obtain a Multiview Image instance.

The Multivew Image provides a unified API for loading the different image formats.

The Android Media SDK provides a set of operators that work with the Multiview image, allowing you to perform operations like View synthesis, disparity estimation and depth of field.

The Multiview Image can be displayed on the screen using the QuadView as we seen in the previous sections.

Multiview Image Decoder

The MultiviewImage decoder decodes different image types to return a MultiviewImage instance.

Using the Multiview Image Decoder

You can obtain an instance of the MultiviewImageDecoder by calling MultiviewImageDecoder.getDefault()

val multiviewImageDecoder = MultiviewImageDecoder.getDefault()

MultiviewImageDecoder decoder = MultiviewImageDecoder.getDefault();

Decoding Images

The MultiviewImageDecoder provides different methods to decode images. In the previous sections, we decoded a LIF image using raw byte[] as well as decoded a SBS image by using the File Uri.

1. Decoding with Uri

This is the most recommended method for decoding the different image formats due to its compatibility with all image formats.

You can call the following function to decode using the file Uri.

val multiviewImage = multiviewImageDecoder.decode(context, fileUri, 1280 * 720)

MultiviewImage multiviewImage = multiviewImageDecoder.decode(context, fileUri, 1280 * 720);

The function takes in 3 parameters-

• The Application context.

• The File Uri or contentUri for the image on the device's internal storage.

• The output resolution for all the decoded viewpoints in the Multiview image. This rescales all views to the set number of pixels, if necessary.

Optionally, if you know the encoded image file contains ViewPoints with the same resolution, you can decode without specifying the output pixels.

val multiviewImage = multiviewImageDecoder.decode(context, fileUri)

MultiviewImage multiviewImage = multiviewImageDecoder.decode(context, fileUri);

2. Decoding with raw bytes

You can also decode images using raw bytes. However, this method is only supported for Leia Image file types. (H4VBinary & H4V_XMP)

val multiviewImage = multiviewImageDecoder.decode(imageBytes, 1280 * 720)

MultiviewImage multiviewImage = multiviewImageDecoder.decode(imageBytes, 1280 * 720);

Optionally, if you know the encoded image file contains ViewPoints of the same resolution, you can decode without specifying the output pixels.

val multiviewImage = multiviewImageDecoder.decode(imageBytes)

MultiviewImage multiviewImage = multiviewImageDecoder.decode(imageBytes);

Detect File Types

You can also use the MultiviewImageDecoder to detect the image file type.

val decoder = MultiviewImageDecoder.getDefault()
val fileType = decoder.getFileType(context, fileUri)

MultiviewImageDecoder decoder = MultiviewImageDecoder.getDefault();
MultiviewFileType fileType = decoder.getFileType(context, fileUri);

This function takes in the File Uri and returns a MultiviewFileType. The returned Multiview File types are categorized below

Decoder Compatibility

To ensure decoding compatibility for SBS (2x1) and Quad (2x2) images, you need to make sure the files for these image formats follow proper naming conventions.

• SBS image file names must have a suffix of _2x1

• Quad image file names must have a suffix of _2x2

Multiview Operations

We can now perform a several operations on the Multiview Image obtained from the decoder. This includes disparity estimation and View Synthesis.

You can perform these operations on the Multiview Image using the MultiviewSynthesizer2. You can obtain an instance of this class as shown below.

val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)

MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);

Disparity Estimation

Images decoded using the Multiview Image Decoder may or may not have depth information, depending on how they were encoded. In order to generate multiple views for a Multiview Image, we need to estimate disparity.

You can populate the disparity maps in a Multiview Image using the MultiviewSynthesizer

val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)
synthesizer.populateDisparityMaps(multiviewImage)

MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);
synthesizer.populateDisparityMaps(multiviewImage);

View Synthesis

The final steps to display a Multiview image on a Leia Lightfield device is to generate multiple views. This step performs View Synthesis to generate 4 viewpoints and returns a Bitmap representing a Quad.

val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)
val quadBitmap = synthesizer.toQuadBitmap(multiviewImage)

MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);
Bitmap quadBitmap = synthesizer.toQuadBitmap(multiviewImage);

The Quad bitmap can be used to display the image on the device using the QuadView

quadView.setQuadBitmap(quadBitmap)

quadView.setQuadBitmap(quadBitmap);

​