AV1 Had a Huge Q4 - Ten Updates
Since the founding of Visionular, we have been passionately focused on building the AV1 standard by developing the best AV1 codec implementation possible. Aside from assembling the largest dedicated group of codec engineers in the industry, we have taken every opportunity to promote the AV1 standard as active members of the AOM.
In the fourth quarter of this year, the team was especially busy. Here are a few of the more notable AV1 centric activities that we led or took part in.
In October, Mark Donnigan moderated a panel at the Agora.io RTE2020 virtual conference titled “Selecting The Best Video Codec To Scale Your Apps for RTE” where he was joined by: Shawn Zhong, Chief Scientist, Agora – Jerome Jiang, Software Engineer, Google – Josh Barnard, Technical Director, iStreamPlanet – Pierre Seigneurbieux, VP, Media Engineering, BlueJeans – Rui Zhang, Distinguished Engineer, Cisco Systems.
During the panel, Jerome Jiang, a key codec engineer behind Google DUO AV1 solution on the Google WebM team, shared critical insights about the performance of AV1, particularly for the user scenarios of real-time communications (RTC).
Video of the full panel discussion.
At the same RTE2020 conference, Zoe Liu gave a talk titled “AV1 for Real-Time Screen Content.” This presentation focused on applying Aurora1’s real-time Screen Content Coding (SCC) optimization and features. Screen content represents a significant category of online Internet video content, including but certainly not limited to presentation/desktop sharing in video conferencing, remote desktop services, computer-generated graphics applications, and live game content streaming.
Zoe’s talk demonstrated that using Aurora1, AV1 is able to encode at more than 45FPS for 1080p screen content videos using just a single core of a standard PC processor while achieving a bitrate saving of more than 80% compared to that of x264. This is a significant achievement since it demonstrates that AV1 bitstreams require just one-quarter of the bitrate needed by H.264 to generate the same level or even better visual quality at an encoding speed that is fast enough to meet real-time requests. It confirms that AV1 is ready for deployment in RTC scenarios.
Video of the full presentation by Zoe Liu.
At the Demuxed conference in October, Zoe Liu was chosen by a special selection committee to give a presentation titled “Decoder Complexity Aware AV1 Encoding Optimization.” This talk pointed out that AV1 encoder optimization not only needs to work from the perspectives of visual quality, bitrate saving, and encoding speed, but also the DCA (Decoder Complexity Aware). Read more here.
Combining CAE (Content-Adaptive Encoding) and DCA provides a big benefit, especially under the circumstance where AV1 can only be decoded using a software decoder. This enables AV1 to be deployed in practical scenarios so that end users can enjoy the advantages of AV1 at the earliest possible time.
As a simple example, the macroblock division method in AV1 can range from 4×4 to 128×128. This means that we can avoid the segmentation of image blocks that are too large or too small while ensuring sufficient coding efficiency and significantly reducing decoding complexity. This allows the decoder to perform real-time decoding on low-end devices. By taking into account decoder complexity, we can more easily drive deployments of AV1 today.
Video of the full presentation by Zoe Liu.
Mark Donnigan led a panel with Zoe Liu at the OTT Executive Summit in November titled “AV1, HEVC, LCEVC, or VVC? Navigating a Multi-Codec World,” with panelists including Tom Vaughan, Vice President, Strategy at Beamr – Jan De Cock, Director of Codec Development at Synamedia – Fabio Murra, SVP, Product Marketing, and Solutions at V-Nova.
In addition to an insight-rich discussion about the state of codec adoption in the video ecosystem, Zoe presented statistics from Visionular’s existing customers who have deployed Visionular’s AV1 encoder solutions to show market acceptance and demonstrate that AV1 is ready for deployment in wide user scenarios covering VOD, live streaming, and low delay RTC use cases.
Video of the full panel discussion.
Visionular continued our busy speaking circuit with a talk given by Zoe Liu at the Mile-High Video virtual conference in December titled “CAE-Based Intelligent Optimization Algorithms for AV1 Encoding.” This presentation focused on the use of machine learning as a valuable part of the encoder for further optimizing AV1 performance. Different approaches were compared and discussed, including the use of pre-processing enabled by machine learning, core encoder optimization through the use of neural network-based algorithms, machine learning for post-processing, and the possibility of an end-to-end neural video coding (NVC) process for next-generation video codec standards such as AOM/AV2.
Zoe also shared her perspective on how existing rule-based coding tools can improve by using a data-driven approach that leverages powerful neural networks to learn robust and efficient mapping functions for more compact content representation. By creating harmony between learning-based and conventional rule-based tools, AV1 performance can be enhanced further to achieve an excellent balance between coding efficiency and encoding speed.
Zoe Liu joined a panel at LiveVideoStack San Francisco in December called “New Standards, New Fields, New Applications- the New Norm of Multimedia Industry in Post-pandemic Era,” with Dan Rayburn, Streaming Media Expert & Analyst, Chairman NAB Streaming Summit – and, Stefan Lederer, Bitmovin CEO/Co-Founder.
Zoe presented that under the pandemic, explosive growth in real-time communications generated video makes AV1 an excellent codec standard for user scenarios, including video conferencing and distance learning collaborative working environment, remote medical, etc. Zoe pointed out that AV1 also provides tremendous potential in the post-pandemic era, considering its 100+ new coding tools, royalty-free licensing model, and ubiquitous browser support.
In December, MSU released their Video Codecs Comparison 2020 Part 1: FullHD, Objective study ranking our AV1 encoder, Aurora1, above all the other encoders that participated. 21 in total, representing a wide variety of video codec standards, including AV1, HEVC/H265, H264, VP9, and proprietary codecs. The MSU annual codec evaluation is the most acknowledged video codec evaluation report for academia and the video industry worldwide. 2020 was the second year that we came out on top. Read the 2019 results here.
Visionular was featured in a blog post by Dan Rayburn (Streaming Media Expert & Analyst, Chairman NAB Streaming Summit):
In comments made by Zoe Liu, and as a result of the extensive work that Visionular is doing across the world, Dan reported that almost all UGC, RTC, and premium video streaming services in Asia are using HEVC, and many are adopting AV1, or working towards the adoption of AV1.
The point of Dan’s article is to show that the Asia market is aggressively embracing the deployment of advanced codecs, starting with HEVC and AV1. Zoe pointed out, “they see the bitrate savings of HEVC and now AV1 as a huge competitive driver. A new codec standard like HEVC or AV1 allows them to increase resolution while upgrading their video quality, even while the bandwidth needed stays the same or goes down. The resulting consumer experience benefit is well understood, and services are willing to do a little extra work today because the user benefits far outweigh the risk. Asian video companies’ mindset is not to adopt the latest standard is to admit that a service is not trying to be the best.”
Visionular was featured in the Streaming Media AV1 Comparison Report written by Jan Ozer. READ: AV1 Has Arrived: Comparing Codecs from AOMedia, Visionular, and Intel/Netflix
In the article published in September, Jan wrote, “Through extensive evaluation, it has been confirmed that under the objective SSIMPLUS metrics, Visionular’s AV1 encoder presents superior coding efficiency performance compared to two other open-source AV1 encoders including libaom and SVT-AV1, as well as more superior than the widely acknowledged H264 open-source encoder x264 and the HEVC open-source encoder x265.”
For an even more up-to-date analysis, you’ll want to read the MSU 2020 Codec Study results.
Proceedings of the IEEE – An invited paper under review for this top IEEE scientific journal, titled “Advances In Video Compression System Using Deep Neural Network: A Review And Case Studies,” co-authored by Dandan Ding, Zhan Ma, Di Chen, Qingshuang Chen, Zoe Liu, and Fengqing Zhu.
Zoe Liu teamed up with her collaborators including Dandan Ding, Professor of Hangzhou Normal University and an advisor to Visionular – Zhan Ma, Professor of Nanjing University – Fengqing (Maggie) Zhu, Professor of Purdue University – who all contributed to this paper that provides a thorough overview of AI technologies applied to video compression. Significant results have been experienced and collected using different AV1 codec approaches, targeted to demonstrate that AI has a great potential to improve the AV1 codec’s performance further. The paper presents three major functional blocks, including pre-processing, coding, and post-processing, to maximize the end-user quality of experience (QoE) under a constrained bitrate budget. The authors reviewed recent technical advances in video compression systems, emphasizing deep neural network (DNN)-based approaches, and presented three comprehensive case studies. The paper explains how machine learning facilitated techniques can more effectively encode and enhance video frames with pre-processing and post-processing.
Continue reading...
At Foothill Ventures, we believe in startup companies that ride the transformative power of major technology shifts such as deep learning in computer vision. Visionular’s founders are world-class technologists in their field of video codec and AI-driven optimization. We feel privileged to support their adventure with our resources and experience.
I invested in Visionular because the team is at the forefront of innovations in video encoding and image processing for real-time low latency video communications and premium video streaming applications.
