Overlaying Video With Transparency While Wrangling Cross-Browser Support

As websites are becoming more and more dynamic when it comes to design, there is sometimes a need to incorporate complex, animated elements. There are many ways to do that from CSS transitions to 3D rendering on canvas, and animated SVG. But it is often easier to use a <video> since they can be rather efficient and, with them, just about anything visual is possible.

But what if you a need transparent background on that video, so that it could overlay and appear to interact with other content on the page? It’s possible, but can be tricky, especially cross-browser. Let’s explore that.

Here’s an example

To see how transparent video overlays work, I’ve prepared an example that I hope feels both fun and relatable. The idea is to integrate video with interactive content underneath so that the content behind the video changes as the slider is advancing. It is a slider advertising dog food with a real video of a dog on the overlay and I’m sure that realistic, cute dog would improve the conversion rate!

You can follow along with this article to recreate this demo by yourself. Part of what you’ll need to create a video like this is an “image sequence” (a bunch of alpha-transparent images we’ll stitch together into a video). I’m providing the images I used for the example.

Download images

Exploring possibilities

To achieve this effect, I started with a search for any kind of solution that would allow me to insert animated content with transparent background on the page.

GIF

The first thing that came up was a GIF. Even though the GIF format was introduced over 30 years ago, it is still widely used. Unfortunately, it has its limitations. While it works perfectly for simple and small animated graphics, it is not that great for colorful, long video footage, it has limited color space, and will grow a lot in size for complex videos.

APNG

The next option I’ve looked at was APNG, which for some reason, is not as popular as GIF but is much better. (I didn’t even know it existed, did you?) APNG works similarly to animated GIF files while supporting 24-bit images and 8-bit transparency. It is also backward-compatible with regular PNG. If APNG is not supported, the first frame is displayed. It is supported by popular browsers, but no support in Internet Explorer 11 (yup, some people still have to support that). While it seemed like the way to go for a moment, I was not satisfied with its file size and performance.

Fun fact: Apple adopted the APNG format as their preferred format for animated stickers in iOS 10 iMessage apps.

Loop through PNGs

Next, a crude, but working, idea came to mind: use a bunch of PNGs and loop through them with JavaScript to imitate the video. But it requires using a lot of data transfer (each video frame would be a separate image) and resources to animate (you could drain a user’s battery quickly or make their computer fan go crazy).

I quickly abandoned that idea, but it turned out to be useful later. I’ll get back to that at the end of the article.

WebM with alpha transparency

As all the formats for <img> failed here, I started looking into videos. I found an article about alpha transparency in Chrome video published in 2013, which announces Google Chrome support for WebM with an alpha channel. It even shows an example and shares tips on how to use it. I went through the article and it immediately felt like the way to go. I was even more convinced after converting my image sequence to WebM because, while GIF was weighing in at 5.8 MB, WebM with transparency,using the same frame rate and full colors was only 540 KB! That’s more than 10 times smaller while offering better performance and quality. Awesome!

The joy did not last long though. As soon as I opened the website on my iOS Phone I realized that I should’ve started with checking the browser compatibility. Unfortunately, Safari (iOS and macOS) doesn’t support transparency in WebM. While my video was working perfectly on Chrome, Firefox and Edge, Safari greeted me with an ugly black background.

The search continues…

A good Solution

Thankfully, I found a video from WWDC 2019 announcing HEVC video with Alpha support for Safari starting in iOS 13 and macOS Catalina. It seemed to provide the same functionality as WebM but with support on Apple devices. So I decided to use the hybrid solution: HEVC for Safari and WebM for other browsers.

It seems like the perfect solution. But now we have two tasks:

1. Create an HEVC with alpha-transparency
2. Detect for Safari (and the version) to serve the correct format

Creating a transparent video

We will start with the easy part: creating a WebM file. If you want to create, convert, or even edit video files, FFmpeg is your friend. It is an extremely powerful open source multimedia framework, and if you have anything to do with multimedia files, I’d recommend starting there because it’s capable of so many things. I can say that FFmpeg has helped me reduce video file sizes over 10 times without any visible image quality degradation. But let’s get back to transparency.

In my experience, if you need to include animated elements into website layout, you get them as a set of video frames in PNG. Even while working on my example project, a tool that removes background from videos generated a set of images for me. So I will continue here assuming we’re building the video from a set of images (remember, you can download our set), but even if you’ve got a video file instead, the process will look similar (adjust FFmpeg options to your needs).

We are now ready to create a video. Using the command line, we’ll navigate to the folder that contains the PNG files and run the following command:

ffmpeg -framerate 25 -i unscreen-%3d.png -c:v libvpx-vp9 -pix_fmt yuva420p output.webm

You can adjust the arguments to fit your needs:

• framerate: how much images will be used for 1s of the output video
• -i unscreen-%3d.png: input file name and format. My files have numbers from 001 to 150 so i used %3d as a mask to select all files with three digits in the name.
• -c:v: specifies the codec to use. I want the video to be encoded with VP9, which is supported by most web browsers.
• -pix_fmt: specifies pixel format to be used. In our cas, it should support an alpha channel. You can see all supported formats if you run ffmpeg --pix_fmts.
• output.webm: provides the desired output file name as a last argument

There are a lot more options available, but I won’t dive into details as it would take probably more than one article to go through them. Those provided in the example command work fine for our use case.

After the process is finished, we should see a new output.webm file, and if you open it in a supported browser, you’ll see a video. Easy, right?

Creating HEVC Alpha

Since we have a WebP file ready, it’s time to move to the second format we will be using: HEVC with alpha transparency. Unfortunately, at the time of writing, FFmpeg doesn’t support HEVC, so we have to use another tool. As far as I know, the only way to create HEVC with alpha is to use the Finder or Compressor on Mac. If you have a PC, you’ll probably have to ask someone with Mac to do it for you. The Compressor app is only provided with Final Cut Pro, so I won’t be using it, although it may be worth considering if you need custom settings.

Since macOS Catalina, the Finder has a built in Encode Media tool to convert videos. It is simple (and free), so it feeds our needs.

The Finder expects a video file as an input, so first, we have to convert the image sequence to ProRes 4444. This is an important step, as the Encode Media tool won’t accept just any video — it will fail unless you provide the format it accepts. It gave me a headache or two until I found out the correct encoding for the input file.

We can create input video using FFmpeg. As we did when creating WebM, we just have to run FFmpeg with the proper arguments:

ffmpeg -framerate 25 -i unscreen-%3d.png -c:v prores_ks -pix_fmt yuva444p10le -alpha_bits 16 -profile:v 4444 -f mov -vframes 150 output.mov

From this point on, a Mac is needed. But the process is simple and does not involve typing anything in terminal, so even if you ask a non-techy friend with a Mac to do it for you, I’m sure they can manage.

Having ProRes4444 video ready, you can navigate to the containing folder in Finder, right-click on it, and in the context menu, select Encode Selected Video Files.

A window will appear. Select the desired HEVC quality (there are two to choose from), and check the option to Preserve Transparency. Then click the “Continue” button.

After a moment you should see a new file, which is encoded in HEVC with alpha. It’s done! Opening this file in Safari confirm that the transparency works.

Last but not least, it’s time to use videos on the website!

Serving transparent videos for all browsers

We can use <video> element to serve videos. If the browser supports the given format in the src attribute, it will just work. But as I’ve mentioned, we need to serve HEVC for Safari and WebM for other browsers.

There is one more thing to take care of: some browsers support WebM or HEVC itself but without transparency. Those browsers will play the video but with a black background instead of an alpha channel. For example, Safari 12 will play HEVC without transparency. That’s unacceptable for us.

Normally, I’d use the <source /> element to provide the video along with multiple formats as fallbacks, and the browser would pick the one it supports. However, since we want to show HEVC only on specific Safari versions, we will have to set the video src attribute instead using JavaScript.

HEVC with transparency is supported in iOS 13 and Mac Safari 13, so let’s start by detecting those. The recommended way to adjust website features based on support is by detecting the existence of an API rather than looking at the User Agent, but I didn’t find any simple way to detect if the browser supports transparency in video format. Instead, I came up with my own solution. It’s not pretty, but it does the job.

function supportsHEVCAlpha() {
  const navigator = window.navigator;
  const ua = navigator.userAgent.toLowerCase()
  const hasMediaCapabilities = !!(navigator.mediaCapabilities && navigator.mediaCapabilities.decodingInfo)
  const isSafari = ((ua.indexOf('safari') != -1) && (!(ua.indexOf('chrome')!= -1) && (ua.indexOf('version/')!= -1)))
  return isSafari && hasMediaCapabilities
}

This targets Safari 13 and above by looking at navigator.mediaCapabilities, which is not supported in older versions, as well as looking that the browser is Safari at all. If the function returns true, then I’m good to go with HEVC alpha. For any other case, I’ll load WebM video.

Here’s an example of how this comes together in HTML:

<video id="player" loop muted autoplay playsinline></video>

<script>
const player = document.getElementById('player');
player.src = supportsHEVCAlpha() ? 'output.mov' : 'output.webm';
</script>

If you’re curious about those loop muted autoplay playsinline arguments on video element, those are about replicating a GIF-like experience:

• Without muted, the video will not play without user interaction.
• With playsinline, on iOS, the video plays right where it is in the layout instead of opening up in fullscreen.
• With loop, the video repeats over and over.
• With autoplay, it will start playing the video automatically on page load (as long as we also have muted in place).

That’s it! We have a lightweight, performant and high-quality solution for transparent videos, that works on most modern browsers (at least the last two recent versions of Chrome, Firefox, Safari and Edge). We can add some basic HTML and CSS to integrate static content with it and make it a little bit more real-life, or just use the same idea that’s in my demo. That wasn’t too bad, was it?

Hey, but what about IE 11? My whole company is using it!

In general, I’d recommend limiting a feature like this to modern browsers and hiding the video in IE. The video is probably not a critical element of the website. Although, I used to work on a commercial project where IE 11 support was a must and I was forced to figure something out to show the transparent video there. In the end, I ended up cycling through PNG images with JavaScript. I reduced the amount of frames and switched between them using a timer. Sure, the performance was awful, but it worked. The video was quite important to the whole design so we intentionally decided to sacrifice performance on IE 11 to provide the full experience.

Summary

I hope that I’ve saved you some time researching the idea of alpha-transparent video and that now you’re be able to incorporate animated elements like this on your website. I bet you’ll need it someday!

Do you have a different idea of how to use transparent video? Or maybe some experience with that already? Let me know in the comments.

---

The post Overlaying Video With Transparency While Wrangling Cross-Browser Support appeared first on CSS-Tricks.

You can support CSS-Tricks by being an MVP Supporter.

source https://css-tricks.com/overlaying-video-with-transparency-while-wrangling-cross-browser-support/