The future of video codecs

Video codecs are compression tools that allow distributors to condense video files or live video to deliver efficiently across a range of networks. Codecs compress video by removing redundant information; the goal is to preserve the original video quality whilst reducing the amount of data that must be sent over the network. The compressed information is then decompressed at its destination for viewing.

Streaming employs several different types of video (or audio) codecs, and the codecs used will have a direct impact on the quality of the video. The volume of data needing to be sent over the network, as well as the speed (latency) in which this content is delivered.

The most advanced MPEG / ITU codec currently in use by the broadcast industry is H.265 (HEVC) which debuted in 2013. It boasts the ability to take a video file and compress it to half the size than its predecessor (H.264 AVC), without significant loss in quality.

However, HEVC is now eight years old, and there are questions around whether it is still fit for purpose. Today, there is demand to increase not only the pixel count, but to increase framerates too.

So how can these complexity implications be handled? Can tomorrow’s codecs solve the bandwidth issue? And what codec options exist on the market?

VVC is the natural evolution of H.265 (HEVC). It can reduce the bandwidth needed to transmit video by approximately 37% compared to HEVC but the encoding complexity is increased approximately 10x. The complexity increase is a concern.  Since when combined with resolution and / or framerate increases, such as migration from HD to UHD or even 8K, the jump in computational complexity can approximate exponentially.

The combined increases in codec complexity and video resolutions / framerates is a key challenge for encoding solutions of the future. For sure, these increases outpace advances in generic server performance and may signal a need to return to more application specific hardware solutions in the future to deliver a reasonable encoding performance, efficiency and density goals.

HEVC has suffered from patent issues and three separate patent pools now exist for it. AV1 was created by an alliance of industry leaders (the ‘alliance for open media’) which includes some of the largest software and streaming providers. This codec therefore underpins many of the largest video streaming and on demand video services.

EVC a ‘pick and mix’ or ‘a la carte’ encoding standard where you have considerable choice concerning what you implement. The main driver for providing this flexibility is to avoid the crippling patent issues seen with HEVC, and which may well affect VVC too.

MPEG has defined an EVC baseline profile which is intended to be royalty free because the included tools either have patents expired or are provided license free. However, there is currently no guarantee the baseline profile will be free of patent claims!

The main profile is not royalty free and replaces the ‘free to use’ tools of baseline profile with higher performance alternatives which will be subject to patent claims. However, the main profile toolset is designed to be ‘a la carte’ so it will be easy to remove tools that cannot be licensed cost effectively.

The main profile is also intended to outperform HEVC and delivers approximately 27% greater efficiency for a 4.5 x increase in complexity compared to HEVC. The baseline profile is approximately 34% more efficient than AVC for a 42% increase in complexity compared to AVC.

Licensing arrangements are not expected to conclude until before early-mid 2022.

LCEVC is not a stand-alone compression standard and cannot perform a useful function by itself. This alone makes it very different to other codec standards. The objective of LCEVC is to take an existing encoding solution (any codec can be used) and combine this with LCEVC which adds an ‘enhancement’ layer.

The sum of the two (the original codec plus LCEVC) should result in having a lower total complexity than using the original codec by itself, whilst significantly outperforming a solution using the original codec alone.

Two key concepts underpin this approach: The first is to operate the base codec at a downscaled resolution to reduce complexity. The (downscaled and encoded) output is then upscaled again and compared to the source. The difference between these is the error or ‘residual’, and this is encoded using a different codec standardised within LCEVC and which is specifically ‘tuned’ to encode residual data efficiently.

At the decoder end, the decoder will decode the base layer and on-board GPU capacity is normally used to decode the enhancement layer. The result is an efficient compression solution that focuses on managing the complexity required to achieve the performance increase and makes it possible for many legacy decoders to support LCEVC. The solution therefore promises a way to utilise untapped resources in deployed consumer equipment to increase video capability beyond the original design limits of the decoder.

Despite the technological hurdles facing these new video codecs, they could become instrumental to empower the next wave of video innovation. Including making widespread use of high resolution 4K and 8K formats with HFR possible.

Better quality video demands new standards, and more efficient codecs mean delivering the same high-quality picture we’re familiar with but using far less bandwidth. However, high encoding efficiency comes at the price of higher video latency. Some newer applications, such as gaming, cannot tolerate so codec choice and network performance is still critically important to define on an individual application basis.

Appear’s X and XC Platforms support a very wide range of codecs to suit most applications throughout every segment and have won acclaim in head-2-head tests against all our major competitors when it comes to VQ per bitrate on existing codecs (AVC and HEVC). Appears encoding solutions also offer incredible electrical and thermal efficiency and very high service density.

We’re excited to see what the future holds and believe we will witness the greatest advances yet seen in computational technology so far during this decade.