How To Deliver HDR-Like Video In An SDR Package Delight your audience with the best video experience possible.
Nathan Veer
Director of Product Marketing
What is Pseudo HDR?
HDR, or High Dynamic Range video retains more detail in an image’s brightest and darkest portions than standard dynamic range (SDR). It also displays a broader gamut of colors and texture.
Some of the limitations of HDR are less about the footage itself and more about the infrastructure required to view it. For example, many devices still do not support HDR footage. Add to that the higher bandwidth needed to deliver it and you have significantly limited who can watch HDR originated or produced content.
Visionular provides HDR-like quality video while keeping it in an SDR format. We refer to this as Pseudo HDR. You can input your HDR source footage, but your output is delivered as an SDR stream with “HDR-like” quality. Alternatively, you can even input SDR source footage, and it will be enhanced to achieve a more HDR-like quality, but of course, still being an SDR formatted stream. This means you no longer have to deal with the typical HDR limitations.
(Significanly Darker Areas) (Significantly Lighter Areas)
In essence, our Pseudo HDR approach is a module in the pre-processing stage, where details are enhanced in the darkest and brightest areas of an image. We use an effective histogram transformation to achieve this. However, we take it one step further. With the introduction of more details, typically you would need more bitrate or data to support these added details. Otherwise, you would introduce artifacts.
Additionally, each video frame is being processed by its own histogram transformation. As a result, inconsistencies could be introduced across successive frames. When compressed, these inconsistencies will only be amplified, potentially showing up as a flicker. What makes our approach unique is that the Pseudo-HDR pre-processing stage works hand in hand with our adaptive encoding workflow. Together it not only delivers an HDR-like quality video but can pre-empt any potential artifacts or inconsistencies, encoding it as efficiently as possible at the lowest possible bitrate.
So how do we do this? We start by using a conventional scheme referred to as Adaptive Histogram Equalization (AHE). By extracting multiple histograms from localized regions of a single frame and using these local histograms to achieve boosted contrast levels, the result is a significantly enhanced dynamic range approaching that of HDR.
(Adaptive Histogram Equalization is a region-based histogram transformation tool)
In addition to the AHE method, we apply a contrast limit to address artifacts such as noise that can surface during the histogram transformation process. This contrast limit clips the enhanced histogram to prevent these artifacts and make them less noticeable. The value at which the histogram is clipped is called “Clip Limit, also known as Contrast Limit AHE (CL-AHE).
(The value at which the histogram is clipped is referred to as “Clip Limit”)
The histogram localization transformation happens in the RGB domain, especially for images. However, for video, the input is mostly in the YUV domain. We have developed a direct YUV domain histogram transformation, leveraging the theory behind probability distribution while considering the conversion matrix between RGB and YUV channels. By deriving nonlinear probability functions, we are able to apply Pseudo HDR in the YUV domain without the YUV to RGB conversion. This helps to prevent any potential artifacts from color space conversion. To the best of our knowledge, this work is the first one to formulate the histogram transformation directly in the YUV domain.
(We use a direct YUV histogram transformation by leveraging the theory behind probability
distribution together considering the conversion matrix between RGB and YUV channels.)
Below is a flow chart of how we apply these elements in the Pseudo HDR module. This module has a “strength” parameter between 0 and 1. The higher the value, the stronger the Pseudo HDR effect that is applied.
Additionally, we combine the output of the Pseudo HDR with the original source video through the use of adaptive weighted sum. This allows us to address problem areas such as skin tones that, when subjected to a histogram transformation, can produce undesirable artifacts. We proactively compare different regions between the output and the source and apply skin protection to those specific regions. This adaptive region-based Pseudo HDR approach provides an overall better quality image.
In the example below, you can see the source video before it was fed through the Pseudo HDR process, and on the right, you can see the results of this processing. Both videos are SDR, but the Pseudo HDR clip on the right has visually improved contrast.
This is especially powerful because we have included the Pseudo HDR capability in our Aurora encoder workflow. Because it is part of our existing pre-processing stage, the benefits of Pseudo HDR are amplified by Aurora‘s capabilities. Together with our pre-processing the encoder will adaptively adjust the bitrate allocation and tune its rate control to avoid artifacts.
By applying Pseudo HDR together with adaptive region-based compression achieve 10-15% lower bitrate at the same visual quality. Additionally, it will apply our de-banding processing to address any banding artifacts that may have surfaced when using Pseudo HDR. Finally, all of this can be done in real-time to further maximize the value.
Now you can deliver HDR-like image quality on SDR devices to make your video stand out and increase engagement. The combination of these technologies as available in Visionular Intelligent Encoders introduces a big step forward for video streaming services and platforms seeking to delight their users with the best video experience possible.
Continue reading...
At Foothill Ventures, we believe in startup companies that ride the transformative power of major technology shifts such as deep learning in computer vision. Visionular’s founders are world-class technologists in their field of video codec and AI-driven optimization. We feel privileged to support their adventure with our resources and experience.
I invested in Visionular because the team is at the forefront of innovations in video encoding and image processing for real-time low latency video communications and premium video streaming applications.
