How to Create High Framerate Puppeteer Screencasts With Audio, Using Browserless

If you’re using Playwright or Puppeteer to automate screencasting, you’ve probably come across issues like:

• No audio capturing
• Low exported frame rates
• Limited browser support
• Complexities associated with the API
• Unexpected crashes
• Inability to debug issues easily
• Continuous browser lags
• Extensive use of resources 

You end up wasting too much time and resources trying to get these automations running—and you’re left with no choice either. We’ve heard these complaints and built a an easy to use Screencast API.

In this article, we’ll review the capabilities of this feature and show you how to set it up and start screencasting in minutes.

The DIY option: Set up Puppeteer and CDP yourself

Usually, if you want to set up browser automation tasks that involved screencasting, you have to use Puppeteer's Page.screencast() method. Puppeteer provides the high-level API to control headless Chrome, while CDP is the operation’s backbone. It does work, but there are key shortcomings:

Low frame rate

The Puppeteer-CDP setup has a casting ability that uses Base64-encoded PNG or JPEG screen data and casts one frame at a time. With such low frame rates, you’ll end up with choppy visuals because the entire process is resource-intensive and slow. This impacts the final result.

Lack of audio

Also, this DIY method relies only on CDP, so it doesn’t support audio capture. You get multiple frames per second—but if you’re recording a video like a movie or an advertisement, this is a significant drawback.

Network intensity

The constant transmission of large amounts of data (e.g., Base64 encoded images) strains network resources. This is especially true when dealing with high-resolution content or prolonged recordings.

The simpler option: Browserless’s screencast API

As you can see, there are several drawbacks to the DIY approach, and it gets even more complicated when you want to venture out and use different browsers or add heavier tasks to the plate. That’s why we built the screencast API.

In this case, we use a custom Chrome extension which lets you generate WebM video files with audio. You can experience benefits like:

• Better frame rates: The extension offers a smoother recording experience and reduces the choppiness you see in the DIY solution. It’s great for cases where you need to record dynamic multimedia content.

• Audio capture: Unlike the DIY setup, the Browserless screencast feature supports audio recording. If you’re recording webinars, tutorials, or multimedia content, you can continue using Puppeteer or Playwright to do it on CDP.
• ‍Recording flexibility: The Chrome extension also allows you to select specific tabs or parts of the browser to record. This provides better control over what you record and avoids unnecessary data capture.

How to use the Screencast API

First, you need to establish a connection with Browserless using the API. Make sure you have all the files to set up the Chrome extension. You can do that by using the following code:

import fs from "fs";
import pptr from "puppeteer-core";
const sleep = (ms) => new Promise((res) => setTimeout(res, ms));
const wsEndpoint = `wss://chrome.browserless.io?token=GOES-HERE&headless=false&stealth&record=true`;
const browser = await pptr.connect({ browserWSEndpoint: wsEndpoint });
const page = await browser.newPage();
await page.goto("https://browserless.io");

// The magic happens here
const cdp = await page.createCDPSession();
await cdp.send("Browserless.startRecording");
await sleep(15000);
const response = await cdp.send("Browserless.stopRecording");
// ☝️ The response is a string containing a valid webm file
const file = Buffer.from(response.value, "binary");
await fs.promises.writeFile("./recording.webm", file);
await browser.close();

Note: Add record=true to your connection string before using the API. You can also change the height and width of your browser by calling the page.setViewport (with dimensions) and setting --window-size in the URL.

The code imports the Puppeteer function, connects the WebSocket Endpoint, and ultimately lets you connect the API to the browser. You need to specify that you intend to record the session, so mention the parameters within the API call—for example, the browser tabs and desired frame rate.

Once the session begins, Browserless’s API records it, captures the audio/video and saves it as a WebM file.

Potential limitations of the API

When you’re running the API, consider these aspects so that you don’t run into issues:

• You need a stable network connection and file system permissions to run the API. It’s a requirement because of the data-intensive nature of screencasting, especially for longer recordings or high-resolution content.
• Follow the documentation properly before setting up the extension, as it can be tricky. You’ll need to have all the necessary files in the same browser, so cross-check everything while running the API.
• You’ll also need to use additional monitoring or debugging tools to make sure the screencasting process runs smoothly.

4 use cases for automated screencasting

Here are a few use cases for screencasting—apart from the usual testing process:

1. HTML5 advertisements

Many businesses create interactive ads using HTML5. But it’s challenging to present it on multiple platforms like a web browser and TV since the dimensions might not carry or it only supports video formats. 

In this case, you can use the screencasting capability to capture the live ads and convert them into video files with audio. This way, you can display these ads on different devices—even if it doesn’t support HTML5.

2. Ad quality control

Quality control in terms of content and security standards is crucial in the advertising industry. For example, you don’t want to run an inappropriate ad on a business website or with the wrong messaging.

To avoid this, you can test your ads by using screencasting. You can review these ads in real-time and verify if the placements and messaging/copy align with your brand’s requirements. It’s also great for detecting any display issues before it’s live.

3. Recording video calls

Let’s say you’ve built a feature that includes video calls, and you want to add a recording option, you can use this API. You can connect it to your browser and get a video file at the end of it. This is an excellent option for recording calls such as interviews or training sessions.

4. Snippets for social sharing

Give your users a way to create video snippets for sharing, such as for an activity that they’re streaming. For example, they can share snippets like game highlights on social media. Usually, you load the game, run it, and then record a video, but you can automate that process with the API.

Add screencasting without the hassle using Browserless

While Puppeteer and CDP do offer options for screencasting, the sheer resources it takes to run and manage them break the setup. Even if it doesn’t break it, you’ll experience lags and other shortcomings.

Instead of DIY-ing it, why not try Browserless instead? All you have to do is set up the API and let us handle the rest. You also have the added advantage of recording at higher frame rates, accessing audio, and setting up programmatic controls for recording.

Currently, it’s available on our Enterprise plan only—so if you’re ready to take it for a spin, check out our documentation first.