Extend Puppeteer e2e tests with UserWay accessibility analysis

UserWay CI/CD injects accessibility analysis into existing end-to-end tests through the @level-ci/a11y-puppeteer npm package. This library works with Puppeteer’s Page instance, collects accessibility insights during e2e test executions, and saves reports in the level-ci-reports directory.

1. Install @level-ci/a11y-puppeteer package

Install the @level-ci/a11y-puppeteer package in the root of your Puppeteer e2e testing project.

npm install --save-dev @level-ci/a11y-puppeteer

For more information about installation of @level-ci/a11y-puppeteer package refer to the documentation

2. Update Puppeteer e2e tests with levelSetup

Import and invoke the levelSetup() function in your Puppeteer’s test files in order to initialize and configure accessibility analysis. It is recommended to create a setup file or a test initialization file, e.g. setupPuppeteer.ts and add levelSetup invocation into it. Then just import setupPuppeteer.ts in all of your tests.

import { levelSetup } from '@level-ci/a11y-puppeteer'
 
test.describe('navigation', () => {
  test.beforeAll(async () => {
    levelSetup({
      elementScreenshots: true,
      printViolationsTable: true,
      pageScreenshot: true,
      reportPath: './level-ci-reports',
    })
  })
})

For more information about configuration of @level-ci/a11y-puppeteer package refer to the documentation

3. Update your Puppeteer e2e tests with levelAnalyze

Adding the levelAnalyze() function call to your existing end-to-end tests allows you to perform static page analysis at any point during test execution. With each function invocation, a report of accessibility violations will be saved in the level-ci-reports directory. As a rule of thumb, you should invoke levelAnalyze() at the end of every e2e test case.

const { levelAnalyze } = require('@level-ci/a11y-puppeteer')
 
test('example test', async ({ page }) => {
  await page.goto('https://website.com')
 
  await levelAnalyze(page)
})

Note: levelAnalyze() accepts multiple arguments and can be parametrized in order to meet your specific conditions and requiremens. Here is more advanced usage:

levelAnalyze(page, {
  strict: false,
  saveReport: 'json',
  elementScreenshots: true,
  reportPath: './level-ci-reports',
})

For more information about levelAnalyze() configuration options visit Puppeteer API doc

Note: Every levelAnalyze() function invocation counts towards consuming project Self-Hosted scans.

To see more examples of @level-ci/a11y-puppeteer package usage visit one of our sample projects

4. Optional configuration

• Update tsconfig.json. If you use TypeScript, add @level-ci/a11y-puppeteer to the types section in your tsconfig.json.

{
  "compilerOptions": {
    "types": ["puppeteer", "@level-ci/a11y-puppeteer"]
  }
}

• Update .gitignore. Ignore reports generated by the levelAnalyze function in your git commits by adding the report directory to the .gitignore file.

level-ci-reports

5. Verify configuration

It is always a good idea to validate changes made in the steps above before committing them into repository. In order to verify if your puppeteer configuration is correct - there are are few steps which can be performed on local developer environment:

• Make all necessary changes described above
• Run your e2e tests locally
• Make sure there is no error and tests pass successfully
• Check directory level-ci-reports/reports for JSON files. Number of report files should be equal to number of cy.levelAnalyze invocations in your e2e tests. Example of a expected report file name level-ci-report-lvoeobzh.json

For more information about troubleshooting of @level-ci/a11y-puppeteer package refer to the documentation

Prerequisites

• @level-ci/a11y-puppeteer compatible with Chrome and Chromium only
• Puppeteer version 22.14.0 or higher is required
• Node.js v16.20.2 or higher