Introduction to headless browser testing

Headless testing is a way of running browser UI tests without the head, which in this case means that there’s no browser UI, no GUI of any sorts. This is useful since when running tests, especially in a CI environment, there is nobody “watching” the visuals, so there is no need to have the extra overhead of the browser GUI.

One of the biggest reasons for using a headless browser/carrying out headless testing is performance, since it lets you run tests more quickly in a real browser environment. Headless browsers avoid draw operations, which handle rendering of the UI and their various pixels on the screen. With headless testing, we ignore those draw operations and the headless engines just run the same tests in the background without a need for a user interface.

How does headless testing help you as a developer?

By using headless testing, you’re guaranteed a more lightweight, less resource intensive, and scripted automation that executes quickly. Instead of going through each page and checking for errors manually, you can write a UI test and integrate it into your build process, then run it in the background.

Headless testing helps you to fix some of the potential quirks that would have been a problem for your users. It helps to make sure that an app can be shipped with little or minimal bugs.

Lastly, with Headless Testing, you can generate screenshots and PDFs of websites, scrape content from websites, automate form submission, and simulate keyboard input.

Headless Testing Tools

There are various tools out there to help with headless testing, however, we’ll only be focusing on the ones below because of their ease of use.

PhantomJS
Nightmare
Headless Chrome
Puppeteer

PhantomJS

PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards.

It’s suitable for headless website testing, screen capturing, page automation and network monitoring.

One major use case of PhantomJS is headless testing of web applications. It is suitable for general command-line based testing, within a precommit hook, and as part of a continuous integration system.

PhantomJS supports headless testing with the use of CasperJS. CasperJS is a navigation scripting & testing tool for PhantomJS. It allows you to build full navigation scenarios using high-level functions and a straight forward interface to accomplish all sizes of tasks.

CasperJS comes with a basic testing suite that allows you to run full featured tests and also allows you to capture data from web pages simply that don’t contain APIs by web scraping.

To get started with PhantomJS and CasperJS, you’ll first have to install PhantomJS globally.

# To install PhantomJS, run the command below
brew install phantomjs

Create a project folder to carry out the tests in. In that folder, initialize npm with npm init -y . The next thing to do is, adding CasperJS to the project.

npm i casperjs

Next up, let’s use CasperJS to scrape Google and display the results from a Google search in the terminal.

Create a file named caspergoogle.js and edit with the following code:

In the code block above, we use CasperJS to scrape Google.com. We use CasperJS’ API to simulate typing in a query on Google.com and then we display the result on the terminal.

We first created a links array and after that, we created an instance of Casper.

The getLinks() function, first gets the title of each Google search result with the document.querySelectorAll('h3 a') and aggregate the result links in a standard Array. We wait for the page to be loaded with the this.waitForSelector() function and we then simulate typing into Google a search query with the this.fill() function.

The results are displayed in the terminal with this.echo inside the casper.run() function.

Run the command casperjs caspergoogle.js to see the outputs from the code. If everything works fine, you should see the results from the Google search displayed nicely in your terminal.

Nightmare

Nightmare is a high-level browser automation library from Segment. Nightmare exposes a few simple methods that mimic user actions (like goto, type and click), with an API that feels synchronous for each block of scripting, rather than deeply nested callbacks.

It was originally designed for automating tasks across sites that don’t have APIs but is most often used for UI testing and crawling.

To start using Nightmare, you’ll have to install the Nightmare package.

npm i nightmare

Let’s write a basic test using Nightmare. This test checks if a website loads successfully or not. To do this, we’ll need mocha . Mocha is a Javascript testing framework that runs on Node and on the browser.

Install Mocha and Nightmare as development dependencies with the commands below:

npm install --save-dev mocha
npm install --save-dev nightmare

In the same folder, create a file named test.js and edit it with the following code:

In the code block above, we wrote a test that checks if a website get’s loaded successfully without any errors. In the second describe() block, we write a test case in which we use Nightmare to navigate to a site and see if there’s any error.

Headless Chrome

Headless Chrome is a way to run the Chrome browser in a headless environment so as to aid automated tests. It is available in the latest versions of Google Chrome (59+).

Headless Chrome brings all modern web platform features provided by Chromium and the Blink rendering engine to the command line. It’s a way of interacting with websites without having to actually have a window up on the screen.

It’s very easy to get started with Headless Chrome, as all you need is a terminal and the latest version of Chrome.

chrome --headless --disable-gpu

There’s a possibility that the command above will return a chrome command not found error. To fix that, you’ll need to create an alias for your Chrome installation.

Run the command below to add the chrome command as an alias.

## For MacOS
## Use this for Google Chrome
alias chrome="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
## Use this for Chrome Canary
alias chrome-canary="/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"
## Use this for Chromium
alias chromium="/Applications/Chromium.app/Contents/MacOS/Chromium"

Let’s carry out some basic tasks with Headless Chrome.

To create a PDF of a particular page.

chrome --headless --disable-gpu --print-to-pdf
https://www.logrocket.com/

That should generate a PDF titled output.pdf of the entire website.

Taking screenshots

chrome --headless --disable-gpu --screenshot
https://www.logrocket.com/

That should generate a screenshot titled screenshot.png of the visible viewport of the website.

To take a screenshot with a custom size, you easily add the --window-size flag to the command.

chrome --headless --disable-gpu --screenshot --window-size=1280,1696
https://www.logrocket.com/

If you’d like to take Headless Chrome a step further and carry out tasks/tests programmatically, by writing Node code, this is where Puppeteer comes in.

Puppeteer

Puppeteer is a Node library developed by the Chrome team. It provides a high-level API to control Headless Chrome. It’s similar to other automated testing libraries like Phantom and NightmareJS above, but it only works with the latest versions of Chrome (Chrome 59+).

The Puppeteer API can be used to take screenshots, create PDFs, navigate pages, and fetch information from pages. We’ll write some example code to see Puppeteer in action.

Taking a screenshot

Install Puppeteer with the npm package:

npm i puppeteer

Also, create a JavaScript file in which we’ll write code to demonstrate Puppeteer, you can name it puppeteer.js. Edit the newly created file with the code below.

The code block above helps to take a screenshot of a particular site (logrocket.com) using Puppeteer. The entire code is wrapped in an async function and a browser is launched. A page is then created and we use the goto() function to navigate to the website we’d like to take a screenshot of.

If you’d like a full page screenshot, you can easily do that by setting fullPage to be true inside the page.screenshot() function.

await page.screenshot({ path: 'https://logrocket.com', fullPage: true})

Run the command node puppeteer.js in your terminal and you should see a screenshot titled LR.png in your project directory.

Screenshot from Puppeteer code

Generate a PDF

In the same folder, create a JavaScript file named puppeteerpdf.js and edit with the following code:

The code block above helps to generate a PDF of a site using Puppeteer. The entire code is wrapped in an async function and a browser is launched. A page is then created and we use the goto() function to navigate to the website we’d like to generate a PDF of.

The waitUntil: 'networkidle' line of code means that the navigation of the site is only considered to be finished when the network activity stays “idle” for at least networkIdleTimeout ms (which defaults to 1000ms). The page.pdf() function creates the PDF file.

Run the command node puppeteerpdf.js in your terminal and you should see a PDF file titled YE.pdf in your project directory.

Extracting values from a page

The last thing we’ll do with Puppeteer is programmatically navigate to a page, automate a form submission as well as keyboard input, and then displaying the results from the form submission.

Create a file named puppeteersearch.js and edit with the code below:

In the code block above, Puppeteer launches a headless browser and navigates to google.com. We then define the query to be typed using the page.type() function and simulate a click using the page.click() function.

The page.waitForSelector() function is used to wait for a selector so as to check if the desired content is loaded. In this case, the h3 a selector is used.

The page.evaluate() function allows us to run scripts in the context of the page. The function above gets all the links from the Google search result and stores them in an array.

Finally, we log the result to the console. Run the command node puppeteersearch.js and you should see the results from the automated Google search in your terminal.

Conclusion

In this article, we’ve seen how Headless Testing can help you as a developer and also explored some Headless Testing tools and examples.

Headless Testing can be a very useful tool in web development. With Headless Testing, you can generate screenshots and PDFs of websites, scrape content from websites, automate form submission, and simulate keyboard input.

When combined with a headless browser, it allows you to do whatever you can do in a full-fledged browser, without needing the browser.

Further Learning Resources

200’s only : Monitor failed and slow network requests in production

Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third party services are successful, try LogRocket. https://logrocket.com/signup/

LogRocket is like a DVR for web apps, recording literally everything that happens on your site. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.

LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. .