A new video encoding/decoding API can replace AMD and Nvidia's encoders on all platforms, truly one API to rule them all

The Khronos Group is a collaboration between nearly 200 companies, working together to produce various APIs for computing. Its most famous software, such as OpenGL, WebGL, and Vulkan, are all about graphics but for the past few years the consortium has been working on doing the same for video. It's just announced that Vulkan Video, a low-level API for handling video streams, now fully supports the hardware acceleration of the H.264 and H.265 standards for decoding and encoding.

If you're wondering what on Earth an API is, think of it as a translation service. It acts as a middleman between a developer's software and the hardware's drivers, making it far easier to produce an application that makes full use of a device's features. For example, games are typically written in C# or C++ but the instructions in the code are written for whatever API is being used. That could be Direct3D or Vulkan when it comes to the graphics.

Where Direct3D is restricted to Windows-based PCs, Vulkan is available across a range of platforms, such as Linux, Android, MacOS, iOS, and so on. Anything written with Vulkan in mind will work just the same on any of those systems, provided the hardware properly supports Vulkan. However, there hasn't really been a comprehensive cross-platform API for handling video decompression and compression.

That was until the Khronos Group released Vulkan Video a few years ago but the initial version only offered decoding, and even then it wasn't a core feature of the API: It was supported through the use of extensions, which are essentially API-snippets that are proprietary to one vendor's hardware.

With this new release, decoding and encoding of the H.264 and H.265 video format standards are now a core part of Vulkan Video. The decoding of the AV1 format will be coming soon, though exactly when isn't clear, and encoding will be for a later point in the future.

But right now, developers can write their video recording and streaming applications using Vulkan Video and it should work equally as well, no matter what system it's being run on.

I say 'should' because to make use of any hardware acceleration of video decoding/encoding, drivers need to be updated to recognise the API and its instructions. At the moment, only Nvidia has any drivers that support the new Vulkan Video update, and even then, just in its beta drivers. AMD and Intel will apparently update their drivers 'soon' but your guess is as good as mine as to when that will happen.

One of the first applications to be modified for the new API release will be FFmpeg, a free cross-platform tool for recording and streaming video. It's not fully ready yet but one of the developers has confirmed that it's currently being worked on.

You may well be thinking 'So what?' at this point, though.

After all, the number of games that use Vulkan for graphics, compared to Direct3D, is pretty small but having an API that isn't restricted to any one vendor's hardware or any specific platform means that developers will have a much easier time of things making software that will work the same, on any machine.

If the makers of FFmpeg are happy to make the jump, you can be sure that others will follow too. It could potentially could mean your favourite streaming software works fast and bug-free, no matter what CPU or GPU you have in your gaming PC.

The only problem is that everything will still depend on how well vendors implement the new API in the drivers. No matter how good the app is, if the drivers aren't great then any software relying on them will struggle.

We'll certainly be keeping an eye on Vulkan Video to see how things pan out and if it turns out that AMD, Intel, and Nvidia have done a good job of things, we'll let you know.