MSU Codec Study Full HD 2020 Results
AV1 codec implementations compared.
- The Visionular Aurora1 AV1 encoder beat all AV1 implementations for the second year.
- Aurora1 proved to be twice as efficient as x264.
- Using the Aurora1 AV1 encoder, video services can move to the royalty-free AV1 codec standard without incurring burdensome operational costs.
- Aurora1 enables services to save more than 50% on their delivery bitrate while maintaining the same visual quality.
challenge.
The annual Moscow State University Codec Study completed by the Graphics and Media Lab Video Group represents the best round up of video codec implementations in the video streaming and delivery industry. Video encoding engineers worldwide rely on this comprehensive and objective study to guide their codec selection decisions and the best implementation for their use case.
Visionular first participated in the 2019 study where our Aurora1 AV1 encoder came out on top for all objective metrics, a remarkable achievement, given the implementation was somewhat new.
For this year’s study, 2020, we again participated. The competition was steep, and yet, Aurora1 came out on top or among the top three across all of the ‘Best quality’ comparisons using SSIM, PSNR, and VMAF quality metrics. We also joined the winning group for Best speed-quality tradeoff using SSIM.
solutions.
This immense effort to evaluate 20 video encoders covering H.264 AVC, H.265 HEVC, AV1, VP9, and VVC utilized 50 video sequences. The process of video sequence selection involved voting among the participants, organizers, and an independent expert. The MSU team analyzed more than 1,500,000 video sequences and selected representative examples to choose the test set.
Video encoder selection and codec standard are dependent on the use case. What works well for VOD may not be appropriate for a real-time low latency deployment. Thus, the MSU comparison consists of two parts, corresponding to fast encoding and offline encoding.
For each use case, codec developers could provide encoding parameters. If a developer declined to provide codec parameters, evaluators either used the same parameters from our prior study or, if none were available, they did their best to choose the optimum parameters for the test. Each encoder tested had to satisfy the minimum speed requirement for their respective use case as follows while encoding at 1080p resolution.
- Online (30 fps)
- Offline (1 fps)
All comparisons used a machine based on an Intel Core i7-8700K (Coffee Lake)processor @ 3.7GHz with 32 GB of RAM running Windows 10 or Ubuntu. The YUV-SSIM metric was used as the primary objective quality indicator, though PSNR and VMAF were also measured.
results.
Across SSIM, VMAF, and PSNR, Aurora1 placed first or in the top three, always beating the incumbent AV1 encoding solutions that were evaluated.
To understand how the results were tabulated, the following is an explanation. View a free abridged version of the report by clicking here.
The MSU Video Quality Measurement Tool (VQMT) was used to calculate objective metrics for the encoded streams. The tool can be downloaded or purchased here.
To replicate the results, run the command: vqmt -in “{original_yuv}” IYUV {width}{height} -in “decoded_yuv” IYUV {width}{height} metrics_list -subsampling -json -json_file “{json_filename}” -threads 3
Where {original_yuv} is the encoded stream name, {width} and {height} are the size of the encoded stream in pixels, metrics_list is a list of metrics to measure (e.g., “-metr ssim_precise YYUV -metr ssim_precise UYUV -metr ssim_precise VYUV”), and {json_filename} is the name of the output file containing the metric results.
RD Curves
Relative-bitrate shows the average bitrate dependence on relative encoding time for a fixed quality output. The y-axis shows the ratio of a codec’s bitrate under test to the reference codec’s bitrate for a fixed quality.
A lower value indicates a better-performing codec. For example, a value of 70% means the codec can encode the sequence using 30% fewer bits than the reference codec (x264 was used in 2019 while x265 was the reference used for this study in 2020).
The x-axis shows the relative encoding time. Larger values indicate a slower codec. A value of 2.5 means the codec works 2.5 times slower, on average than the reference codec (x265).
The Aurora1 AV1 encoder achieved:
- 72% reduced bitrate at the same quality over x264 using SSIM.
- 34% reduced bitrate at the same quality over x265 using SSIM.
- Achieved a 23% bitrate savings over SVT-AV1, and a massive 46% savings over rav1e.
- Best performing codec for quality/speed (SSIM).
The following graph compares coding efficiency of the Aurora1 AV1 encoder with the other AV1 implementations in the study, as well as x264, x265, and VP9 using the VMAF quality metric.
The following graph compares coding efficiency of the Aurora1 AV1 encoder with the other AV1 implementations in the study, as well as x264, x265, and VP9 using the PSNR quality metric.
SSIM comparison of Aurora1 AV1 encoder with other AV1 implementations as well as x264, x265, and VP9.
presentation. Demuxed 2020 AV1 optimization talk.
