AV1 Screen Content Coding
How to get a clean screen content with AV1 Screen Content Coding
Click the orange play button to watch what AV1 will do to text for your RTC app. Learn how it can deliver sharper and clearer text at lower bitrates than any other AV1 encoder. You'll never go back to H.264.
More about AV1 Screen Content Coding
Screen content is unique from standard video in the RTC use case. For example, almost every pixel will change when switching from one slide to the next. However, practically no changes occur once on that slide until the following slide change.
Applying the encoding algorithms for standard video to screen content is ineffective. The trade-off between quality and encoding efficiency is not the same. However, combining AV1’s core coding tools with our intelligent algorithms delivers higher-quality screen content using 50% fewer bits.
Screen Content Coding: fewer bits, higher quality
You no longer have to worry about screen content quality in your remote worker presentations. Take your online team collaboration software to the next level and deliver clean, crisp screen content. Make your remote presentation pop on large stage presentations.
Technology
Our team has worked hard to ensure that the Aurora1 AV1 encoder dynamically applies the best tools in the best way, to screen content. Unlike other codecs, AV1 supports special tools for screen content coding in the main profile, giving unparallel performance compatible with all compliant decoders. These tools include:
Intra Block Copy (IntraBC)
This tool allows data to be copied within the same frame, just like motion compensation and can produce huge bit rate savings. We use a highly-optimized hash-based search to identify suitable blocks.
Warping, Rotation, and Zoom
Common in animations, non-translational motion is not effectively coded in prior standards to AV1. Expanding the motion search space requires intelligent optimization to be used effectively, however.
Palette Coding
Palette coding provides an extra block prediction that uses a small palette of individual colors and codes labels instead of pixel values. We use fast routines for identifying suitable blocks and palette sizes.
Chroma from Luma
Chroma and luma samples are often correlated, especially in the colorful images typical of screen content, and predicting the chroma from the luma is often helpful.
Screen Content Coding (SCC) is more than just a set of tools, however. We need to make use of these tools without excessive complexity, and SCC can require a different approach to how the video is coded, even if specific screen tools are not used:
Adaptive Mode Decisions for Screen Content
The best way to investigate coding modes can depend on the source characteristics and when (and when not) to use screen content tools is critical to maintaining high speed with game-changing performance.
Categorization & Identification of Screen Content-specific Motion
Motion compensation has been the mainstay of MPEG encoding for a long time. Screen content presents its own specific kind of motion, which is found more rarely in general video – like scrolling, sliding, and rotating. Exploiting these motions efficiently lowers bandwidth greatly.
Behind The Curtain
The Comparisons
Take a look where Aurora1 is compared to x264 with 1080p30 screen content. Testing was performed on an Intel Core i7 processor with both encoders configured to use a single thread at 100% utilization.
A tool that the Aurora1 AV1 encoder leverages is single and compound motion compensation modes. Altogether there are 128 different methods to detect motion when you switch to compound mode, enabling Aurora1 to stabilize an image and reduce unnecessary movement, reducing the number of constantly changing pixels.
Enabling SCC in Aurora1 can reduce screen content bitrates by more than 50% or up to 500kbps, which is impossible with any other video codec standard, or AV1 encoder implementation including libaom RT.
When Aurora1 is compared to Open H264 across a wide test set of videos, Aurora1 achieved a greater than 50% bitrate reduction while operating just 38% slower. This shows that video engineers can confidently switch from the aging and less efficient Open H264 encoder to AV1 so that they can enjoy better quality and a tremendous reduction in bandwidth.
What about speed? A common concern with AV1 and any next-gen codec standard is speed. While this argument has been around for some time, it’s somewhat outdated. Read this post to learn about how AV1 speed has improved. WebRTC testing across various platforms, including cloud (data center), desktop, and mobile were conducted using the following settings and operational conditions:
- Video camera output encoded at 24 FPS with screen content at 12 FPS. Resolutions between 720p and 1080p.
- With standard screen content, Aurora1 preserved the original quality at 1080p and 100kbps. During intense screen content motion, the bitrate rarely exceeded 500kbps.
- CPU usage was reasonable, enabling smooth playback even on entry-level i5 PCs.
- With Aurora1’s Scalable Video Codec (SVC) the bitrate was 35% of OpenH264 or VP9 at the same visual quality. AV1 offers an exciting set of tools for optimizing content for real-time delivery using WebRTC. With the Aurora1 encoder, you can achieve lower bitrates at the same quality requiring less processing power, making AV1 a realistic option for any WebRTC application.
Customer Use
The team is at the forefront of advancements and innovations in video encoding and image processing for real-time low latency video communications and premium video streaming applications.
Agora launched in 2007, seeking to provide merchants with a smart eCommerce solution. Today Agora.io is a developer platform that provides broadcast, voice, and video calls and RTC cloud solutions for mobile and web applications through their software development kit. Their SoundNet real-time interactive technical services cover more than 200 countries and regions around the world (NASDAQ: API)
- Visionular Improved RTC quality and codec efficiency using AV1 and Screen Content Coding.
- Using Screen Content Coding, they reduced the needed bitrate for RTC by 70+%, resulting in savings in network bandwidth transmission.
- They significantly improved the viewing experience reducing network bottleneck issues, especially for ultra-low latency streaming (sports, betting, etc).
- Agora was recognized as the first RTC service provider to adopt and implement the latest AV1 codec standard.