Performance testing in the browser

Last Updated on by

Post summary: Approaches for performance testing in the browser using Puppeteer, Lighthouse, and PerformanceTiming API.

In the current post, I will give some examples how performance testing can be done in the browser using different metrics. Puppeteer is used as a tool for browser manipulation because it integrates easily with Lighthouse and DevTools Protocol. I have described all tools before giving any examples. The code can be found in GitHub sample-performance-testing-in-browser repository.

Puppeteer

Puppeteer is a tool by Google which allows you to control Chrome or Chromium browsers. It works over DevTools Protocol, which I will describe later. Puppeteer allows you to automate your functional tests. In this regards, it is very similar to Selenium but it offers many more features in terms of control, debugging, and information within the browser. Over the DevTools Protocol, you have programmatically access to all features available in DevTools (the tool that is shown in Chrome when you hit F12). You can check Puppeteer API documentation or check advanced Puppeteer examples such as JS and CSS code coverage, site crawler, Google search features checker.

Lighthouse

Lighthouse is again tool by Google which is designed to analyze web apps and pages, making a detailed report about performance, SEO, accessibility, and best practices. The tool can be used inside Chrome’s DevTools, standalone from CLI (command line interface), or programmatically from Puppeteer project. Google had developed user-centric performance metrics which Lighthouse uses. Here is a Lighthouse report example run on my blog.

PerformanceTimings API

W3C have Navigation Timing recommendation which is supported by major browsers. The interesting part is the PerformanceTiming interface, where various timings are exposed.

DevTools Protocol

DevTools Protocol comes by Google and is a way to communicate programmatically with DevTools within Chrome and Chromium, hence you can instrument, inspect, debug, and profile those browsers.

Examples

Now comes the fun part. I have prepared several examples. All the code is in GitHub sample-performance-testing-in-browser repository.

  • Puppeteer and Lighthouse – Puppeteer is used to login and then Lighthouse checks pages for logged in user.
  • Puppeteer and PerformanceTiming API – Puppeteer navigates the site and gathers PerformanceTiming metrics from the browser.
  • Lighthouse and PerformanceTiming API – comparison between both metrics in Lighthouse and NavigationTiming.
  • Puppeteer and DevTools Protocol – simulate low bandwidth network conditions with DevTools Protocol.

Before proceeding with the examples I will outline helper functions used to gather metrics. In the examples, I use Node.js 8 which supports async/await functionality. With it, you can use an asynchronous code in a synchronous manner.

Gather single PerformanceTiming metric

async function gatherPerformanceTimingMetric(page, metricName) {
  const metric = await page.evaluate(metric => 
     window.performance.timing[metric], metricName);
  return metric;
}

I will not go into details about Puppeteer API. I will describe functions I have used. Function page.evaluate() executes JavaScript in the browser and can return a result if needed. window.performance.timing returns all metrics from the browser and only needed by metricName one is returned by current function.

Gather all PerformaceTiming metrics

async function gatherPerformanceTimingMetrics(page) {
  // The values returned from evaluate() function should be JSON serializable.
  const rawMetrics = await page.evaluate(() => 
    JSON.stringify(window.performance.timing));
  const metrics = JSON.parse(rawMetrics);
  return metrics;
}

This one is very similar to previous. Instead of just one metric, all are returned. The tricky part is the call to JSON.stringify(). The values returned from page.evaluate() function should be JSON serializable. With JSON.parse() they are converted to object again.

Extract data from PerformanceTiming metrics

async function processPerformanceTimingMetrics(metrics) {
  return {
    dnsLookup: metrics.domainLookupEnd - metrics.domainLookupStart,
    tcpConnect: metrics.connectEnd - metrics.connectStart,
    request: metrics.responseStart - metrics.requestStart,
    response: metrics.responseEnd - metrics.responseStart,
    domLoaded: metrics.domComplete - metrics.domLoading,
    domInteractive: metrics.domInteractive - metrics.navigationStart,
    pageLoad: metrics.loadEventEnd - metrics.loadEventStart,
    fullTime: metrics.loadEventEnd - metrics.navigationStart
  }
}

Time data for certain events are compile from raw metrics. For e.g., if DNS lookup or TCP connection times are slow, then this could be some network specific thing and may not need to be acted. If response time is very high, then this is indicator backend might not be performing well and needs to be further performance tested. See How to do proper performance testing post for more details.

Gather Lighthouse metrics

const lighthouse = require('lighthouse');

async function gatherLighthouseMetrics(page, config) {
  // ws://127.0.0.1:52046/devtools/browser/675a2fad-4ccf-412b-81bb-170fdb2cc39c
  const port = await page.browser().wsEndpoint().split(':')[2].split('/')[0];
  return await lighthouse(page.url(), { port: port }, config).then(results => {
    delete results.artifacts;
    return results;
  });
}

The example above shows how to use Lighthouse programmatically. Lighthouse needs to connect to a browser on a specific port. This port is taken from page.browser().wsEndpoint() which is in format ws://127.0.0.1:52046/devtools/browser/{GUID}. It is good to delete results.artifacts; because they might get very big in size and are not needed. The result is one huge object. I will talk about this is more details. Before using Lighthouse is should be installed in a Node.js project with npm install lighthouse –save-dev.

Puppeteer and Lighthouse

In this example, Puppeteer is used to navigating through the site and authenticate the user, so Lighthouse can be run for a page behind a login. Lighthouse can be run through CLI as well but in this case, you just pass and URL and Lighthouse will check it.

puppeteer-lighthouse.js

const puppeteer = require('puppeteer');
const perfConfig = require('./config.performance.js');
const fs = require('fs');
const resultsDir = 'results';
const { gatherLighthouseMetrics } = require('./helpers');

(async () => {
  const browser = await puppeteer.launch({
    headless: true,
    // slowMo: 250
  });
  const page = await browser.newPage();

  await page.goto('https://automationrhapsody.com/examples/sample-login/');
  await verify(page, 'page_home');

  await page.click('a');
  await page.waitForSelector('form');
  await page.type('input[name="username"]', 'admin');
  await page.type('input[name="password"]', 'admin');
  await page.click('input[type="submit"]');
  await page.waitForSelector('h2');
  await verify(page, 'page_loggedin');

  await browser.close();
})();

verify()

const perfConfig = require('./config.performance.js');
const fs = require('fs');
const resultsDir = 'results';
const { gatherLighthouseMetrics } = require('./helpers');

async function verify(page, pageName) {
  await createDir(resultsDir);
  await page.screenshot({
    path: `./${resultsDir}/${pageName}.png`,
    fullPage: true
  });
  const metrics = await gatherLighthouseMetrics(page, perfConfig);
  fs.writeFileSync(`./${resultsDir}/${pageName}.json`,
    JSON.stringify(metrics, null, 2));
  return metrics;
}

createDir()

const fs = require('fs');

async function createDir(dirName) {
  if (!fs.existsSync(dirName)) {
    fs.mkdirSync(dirName, '0766');
  }
}

A new browser is launched with puppeteer.launch(), arguments { headless: true, //slowMo: 250 } are put for debugging purposes. If you want to view what is happening then set headless to false and slow the motions with slowMo: 250, where time is in milliseconds. Start a new page with browser.newPage() and navigate to some URL with page.goto(‘URL’). Then verify() function is invoked. It is shown on the second tab and will be described in a while. Next functionality is used to log in the user. With page.click(‘SELECTOR’), where CSS selector is specified, you can click an element on the page. With page.waitForSelector(‘SELECTOR’) Puppeteer should wait for the element with the given CSS selector to be shown. With page.type(‘SELECTOR’, ‘TEXT’) Puppeteer types the TEXT in the element located by given CSS selector. Finally browser.close() closes the browser.

So far only Puppeteer navigation is described. Lighthouse is invoked in verify() function. Results directory is created initially with createDir() function. Then a screenshot is taken on the full page with page.screenshot() function. Lighthouse is called with gatherLighthouseMetrics(page, perfConfig). This function was described above. Basically, it gets the port on which DevTools Protocol is currently running and passes it to lighthouse() function. Another approach could be to start the browser with hardcoded debug port of 9222 with puppeteer.launch({ args: [ ‘–remote-debugging-port=9222’ ] }) and pass nothing to Lighthouse, it will try to connect to this port by default. Function lighthouse() accepts also an optional config parameter. If not specified then all Lighthouse checks are done. In the current example, only performance is important, thus specific config file is created and used. This is config.performance.js file.

Puppeteer and PerformanceTiming API

In this example, Puppeteer is used to navigating the site and extract PerformanceTiming metrics from the browser.

const puppeteer = require('puppeteer');
const { gatherPerformanceTimingMetric,
  gatherPerformanceTimingMetrics,
  processPerformanceTimingMetrics } = require('./helpers');

(async () => {
  const browser = await puppeteer.launch({
    headless: true
  });
  const page = await browser.newPage();
  await page.goto('https://automationrhapsody.com/');

  const rawMetrics = await gatherPerformanceTimingMetrics(page);
  const metrics = await processPerformanceTimingMetrics(rawMetrics);
  console.log(`DNS: ${metrics.dnsLookup}`);
  console.log(`TCP: ${metrics.tcpConnect}`);
  console.log(`Req: ${metrics.request}`);
  console.log(`Res: ${metrics.response}`);
  console.log(`DOM load: ${metrics.domLoaded}`);
  console.log(`DOM interactive: ${metrics.domInteractive}`);
  console.log(`Document load: ${metrics.pageLoad}`);
  console.log(`Full load time: ${metrics.fullTime}`);

  const loadEventEnd = await gatherPerformanceTimingMetric(page, 'loadEventEnd');
  const date = new Date(loadEventEnd);
  console.log(`Page load ended on: ${date}`);

  await browser.close();
})();

Metrics are extracted with gatherPerformanceTimingMetrics() function described above and then data is collected from the metrics with processPerformanceTimingMetrics(). In the end, there is an example how to extract one metric such as loadEventEnd and display it as a date object.

Lighthouse and PerformanceTiming API

const puppeteer = require('puppeteer');
const perfConfig = require('./config.performance.js');
const { gatherPerformanceTimingMetrics,
  gatherLighthouseMetrics } = require('./helpers');

(async () => {
  const browser = await puppeteer.launch({
    headless: true
  });
  const page = await browser.newPage();
  const urls = ['https://automationrhapsody.com/',
    'https://automationrhapsody.com/examples/sample-login/'];

  for (const url of urls) {
    await page.goto(url);

    const lighthouseMetrics = await gatherLighthouseMetrics(page, perfConfig);
    const firstPaint = parseInt(lighthouseMetrics.audits['first-meaningful-paint']['rawValue'], 10);
    const firstInteractive = parseInt(lighthouseMetrics.audits['first-interactive']['rawValue'], 10);
    const navigationMetrics = await gatherPerformanceTimingMetrics(page);
    const domInteractive = navigationMetrics.domInteractive - navigationMetrics.navigationStart;
    const fullLoad = navigationMetrics.loadEventEnd - navigationMetrics.navigationStart;
    console.log(`FirstPaint: ${firstPaint}, FirstInterractive: ${firstInteractive}, 
      DOMInteractive: ${domInteractive}, FullLoad: ${fullLoad}`);
  }

  await browser.close();
})();

This example shows a comparison between Lighthouse metrics and PerformanceTiming API metrics. If you run the example and compare all the timings you will notice how much slower the site looks according to Lighthouse. This is because it uses 3G (1.6Mbit/s download speed) settings by default.

Puppeteer and DevTools Protocol

const puppeteer = require('puppeteer');
const throughputKBs = process.env.throughput || 200;

(async () => {
  const browser = await puppeteer.launch({
    executablePath: 
      'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
    headless: false
  });
  const page = await browser.newPage();
  const client = await page.target().createCDPSession();

  await client.send('Network.emulateNetworkConditions', {
    offline: false,
    latency: 200,
    downloadThroughput: throughputKBs * 1024,
    uploadThroughput: throughputKBs * 1024
  });

  const start = (new Date()).getTime();
  await client.send('Page.navigate', {
    'url': 'https://automationrhapsody.com'
  });
  await page.waitForNavigation({
    timeout: 240000,
    waitUntil: 'load'
  });
  const end = (new Date()).getTime();
  const totalTimeSeconds = (end - start) / 1000;

  console.log(`Page loaded for ${totalTimeSeconds} seconds 
    when connection is ${throughputKBs}kbit/s`);

  await browser.close();
})();

In the current example, network conditions with restricted bandwidth are emulated in order to test page load time and perception. With executablePath Puppeteer launches an instance of Chrome browser. The path given in the example is for Windows machine. Then a client is made to communicate with DevTools Protocol with page.target().createCDPSession(). Configurations are send to browser with client.send(‘Network.emulateNetworkConditions’, { }). Then URL is opened into the page with client.send(‘Page.navigate’, { URL}). The script can be run with different values for throughput passed as environment variable. Example waits 240 seconds for the page to fully load with page.waitForNavigation().

Conclusion

In the current post, I have described several ways to measure the performance of your web application. The main tool used to control the browser is Puppeteer because it integrated very easily with Lighthouse and DevTools Protocol. All examples can be executed through the CLI, so they can be easily plugged into CI/CD process. Among the various approaches, you can compile your preferred scenario which can be run on every commit to measure if the performance of your applicaition has been affected by certain code changes.

Related Posts

Read more...

Build a REST API with Express on Node.js and run it on Docker

Last Updated on by

Post summary: Code examples how to create RESTful API with Node.js using Express web framework and then run it on Docker.

Code below can be found in GitHub sample-nodejs-rest-stub repository. This is my first JavaScript post, so bare with me if something is not as perfect as it should be.

Node.js

Node.js is a JavaScript runtime built on Chrome’s V8 JavaScript engine. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

Create Node.js project

Node.js project is created with npm init, which guides you through a wizard with several questions.

In the end package.json file is created. This is the file with all your project’s configuration.

{
  "name": "sample-nodejs-rest-stub",
  "version": "1.0.0",
  "description": "Sample Node.js REST API",
  "main": "app.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Lyudmil Latinov",
  "license": "ISC"
}

Once created it is good to add some JavaScript code to test that project is working. I called the file app.js and it will be later extended. It does nothing, but writing Hello world! to the console.

'use strict';

console.log('Hello world!');

File with JavaScript code can be run with node app.js command. You can change package.json file by adding start script and then run the application with npm start:

"scripts": {
  "start": "node app.js",
  "test": "echo \"Error: no test specified\" && exit 1"
}

Express

Express is a web framework for Node.js. It is the most used one. In order to use express it has to be added as a dependency and saved to package.json file.

npm install express --save

Change app.js in order to verify Express is working correctly. The ‘use strict’ literal is used for enabling ECMAScript 5 strict mode, which has several restrictions, like warnings are thrown as errors, usage of undeclared variables is prohibited, etc. I would prefer using constants declared with const whenever possible. Express module is assigned to express variable with require(‘express’) directive. Then new express object is created and assigned to app variable. HTTP GET endpoint that listens to ‘/’ is configured with app.get(path, callback) function. Callback is a function that is called inside another function, in our case inside get() function. In the current example, callback has arguments req and res which gives you access to Express’ Request and Response objects. What is done below is that send([body]) function on the response is called, which returns the result. Socket that listens for incoming connections is started with app.listen(path, [callback]) function. More details can be found in Express API reference documentation.

'use strict';

const express = require('express');
const app = new express();

app.get('/', (req, res) => {
    res.send('Hello World!');
});

app.listen(3000, () => {
    console.log('Server up!');
});

If you run with npm start you should see Server is up! text. Firing GET request to http://localhost:3000 should return Hello World! response.

Add REST API

Functionality is a sample Person service that is used also in Build a RESTful stub server with Dropwizard and Build a REST API with .NET Core 2 and run it on Docker Linux container posts.

The first step is to include body-parser, an Express middleware which parses request body and makes it available as an object in req.body property.

npm install body-parser --save

Express middleware is series of function calls that have access to req and res objects. Middleware is used in our application. I will explain as much as possible, if you are interested in more details you can read in Express using middleware documentation.

Person class

A standard model class or POJO is needed in order to transfer and process JSON data. It is standard ECMAScript 6 Person class with constructor which is then exported as a module with module.exports = Person.

Person repository

Again there will be no real database layer, but a functionality that acts as such. In the constructor, a Map with several Person objects is created. There are getByIdgetAllremove and save functions which simulate different CRUD operations on data. Inside them, various Map functions are used. I’m not going to explain those in details, you can read more about maps in JavaScript Map object documentation. In the end, PersonRepository is instantiated to a personRepository variable which is exported as a module. Later, when require is used, this instance will be accessible only, not the PersonRepository class itself.

Person routes

In initial example routes and their handling was done with app.get(), here express.Router is used. It is complete middleware routing system. See more in Express routing documentation. Router class is imported const Router = require(‘express’) and new instance is created const router = new Router(). Registering path handlers is same as in application. There are get(), post(), etc., functions resp. for GET and POST requests. Specific when using the router is that it should be registered as application middleware with app.use(‘/person’, router). This makes router handle all defined in it paths which are now under /person base path. Current route configuration is defined as a function with name getPersonRoutes which takes app as an argument. This function is exported as a module.

Application

Important bit here is require(‘./routes/personRoutes’)(app) which uses getPersonRoutes function and registers person routes.

person.js

'use strict';

class Person {
    constructor(id, firstName, lastName, email) {
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.email = email;
    }
}

module.exports = Person;

personRepository.js

'use strict';

const Person = require('../json/person');

class PersonRepository {
    constructor() {
        this.persons = new Map([
            [1, new Person(1, 'FN1', 'LN1', 'email1@email.na')],
            [2, new Person(2, 'FN2', 'LN2', 'email2@email.na')],
            [3, new Person(3, 'FN3', 'LN3', 'email3@email.na')],
            [4, new Person(4, 'FN4', 'LN4', 'email4@email.na')]
        ]);
    }

    getById(id) {
        return this.persons.get(id);
    }

    getAll() {
        return Array.from(this.persons.values());
    }

    remove() {
        const keys = Array.from(this.persons.keys());
        this.persons.delete(keys[keys.length - 1]);
    }

    save(person) {
        if (this.getById(person.id) !== undefined) {
            this.persons[person.id] = person;
            return "Updated Person with id=" + person.id;
        }
        else {
            this.persons.set(person.id, person);
            return "Added Person with id=" + person.id;
        }
    }
}

const personRepository = new PersonRepository();

module.exports = personRepository;

personRoutes.js

'use strict';

const Router = require('express');
const personRepo = require('../repo/personRepository');

const getPersonRoutes = (app) => {
    const router = new Router();

    router
        .get('/get/:id', (req, res) => {
            const id = parseInt(req.params.id);
            const result = personRepo.getById(id);
            res.send(result);
        })
        .get('/all', (req, res) => {
            const result = personRepo.getAll();
            res.send(result);
        })
        .get('/remove', (req, res) => {
            personRepo.remove();
            const result = 'Last person remove. Total count: '
                + personRepo.persons.size;
            res.send(result);
        })
        .post('/save', (req, res) => {
            const person = req.body;
            const result = personRepo.save(person);
            res.send(result);
        });

    app.use('/person', router);
};

module.exports = getPersonRoutes;

app.js

'use strict';

const express = require('express');
const app = new express();
const bodyParser = require('body-parser');

// register JSON parser middlewear
app.use(bodyParser.json());

require('./routes/personRoutes')(app);

app.listen(3000, () => {
    console.log("Server is up!");
});

Debug with Visual Studio Code

I started to like Visual Studio Code – an open source multi-platform editor maintained by Microsoft. Once project folder is imported, hitting F5 starts to debug on the project.

External configuration

External configuration from a file is a must for every serious application, so this also has to be handled. A separate config.js file is keeping the configuration and exposing it as a module. There is versionRoutes.js file added which is reading configuration value and exposing it to the API. It follows the same pattern as personRoutes.js, but it has config as function argument as well. Also, app.js has to be changed, import config and pass it to getVersionRoutes function.

config.js

'use strict';

const config = {
    version: '1.0'
};

module.exports = config;

versionRoutes.js

'use strict';

const getVersionRoutes = (app, config) => {
    app.get('/api/version', (req, res) => {
        res.send(config.version);
    });
};

module.exports = getVersionRoutes;

app.js

'use strict';

const express = require('express');
const bodyParser = require('body-parser');
const config = require('./config/config');
const app = new express();

// register JSON parser middlewear
app.use(bodyParser.json());

require('./routes/personRoutes')(app);
require('./routes/versionRoutes')(app, config);

app.listen(3000, () => {
    console.log("Server is up!");
});

Code style checker

It is very good practice to have consistency over projects code. The more important benefit of using it is that it can catch bugs that otherwise will be caught later in when the application is run. This is why using a code style checker is recommended. Most popular for JavaScript is ESLint. In order to have it has to be added as a dependency to the project:

npm install eslint --save-dev

Notice the –save-dev option, this creates a new devDependencies node in package.json. This means that project needs this packages, but just for development. Those dependencies will not be available if someone is importing your project. Entry in scripts node in package.json file can be added: “lint”: “eslint .”. This will allow you to run ESLint with npm run lint. ESLint configuration is present in .eslintrc file. In .eslintignore are listed folders to be skipped during the check.

.eslintrc

{
  "extends": "eslint:recommended",
  "parserOptions": {
    "ecmaVersion": 6
  },
  "env": {
    "es6": true,
    "node": true
  },
  "globals": {
  },
  "rules": {
    "quotes": [2, "single"]
  }
}

.eslintignore

node_modules

package.json

"scripts": {
  "start": "node app.js",
  "test": "echo \"Error: no test specified\" && exit 1",
  "lint": "eslint ."
},
...
"devDependencies": {
  "eslint": "^4.15.0"
}

app.js

'use strict';

const express = require('express');
const bodyParser = require('body-parser');
const config = require('./config/config');
const app = new express();

// register JSON parser middlewear
app.use(bodyParser.json());

require('./routes/personRoutes')(app);
require('./routes/versionRoutes')(app, config);

app.listen(3000, () => {
    /* eslint-disable */
    console.log('Server is up!');
});

During the check, some issues were found. One of the issues is that console.log() is not allowed. This is a pretty good rule as all logging should be done to some specific logger, but in our case, we need app.js to have text showing that server is up. In order to ignore this error /* eslint-disable */ comment can be used, see app.js above.

Docker file

Docker file that packs application is shown below:

FROM node:8.6-alpine

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY package.json ./
RUN npm install

COPY . .

EXPOSE 3000
CMD ["npm", "start"]

Docker container that is used is node:8.6-alpine. Folder /usr/src/app and is made current working directory. Then package.json file is copied into container and npm install is run, this will download all dependencies. All files from the current folder are copied on docker image with: COPY . .. Port 3000 is exposed so it is, later on, available from the container. With CMD is configured the command that is run when the container is started.

Build and run Docker container

Docker container is packaged with tag nodejs-rest with the following command:

docker build . -t nodejs-rest

Docker container is run with exposing port 3000 from the container to port 9000 on the host with the following command:

docker run -e VERSION=1.1 -p 9000:3000 nodejs-rest

Notice the -e VERSION=1.1 which sets an environment variable to be used inside the container. The intention is to use this variable in the application. This can be enabled with modifying config.js file by changing to: version: process.env.VERSION || ‘1.0’. If environment variable VERSION is available then save it in version, if not use 1.0.

'use strict';

const config = {
    version: process.env.VERSION || '1.0'
};

module.exports = config;

If invoked now /api/version returns 1.1.

Conclusion

In the current post, I have shown how to make very basic REST API with Node.js and Express. It can be very easy run on Docker container.

Related Posts

Read more...