Extend Playwright e2e tests with UserWay accessibility analysis

UserWay CI/CD injects accessibility analysis into existing end-to-end tests through the @userway/a11y-playwright npm package. This library works with Playwright's Page instance, collects accessibility insights during e2e test executions, and saves reports in the uw-a11y-reports directory.

1. Install @userway/a11y-playwright package

Install the @userway/a11y-playwright package in the root of your Playwright e2e testing project.

npm install --save-dev @userway/a11y-playwright

For more information about installation of @userway/a11y-playwright package refer to the documentation

2. Update Playwright e2e tests with setupUserway

Import and invoke the setupUserway() function in your Playwright's test files in order to initialize and configure accessibility analysis. It is recommended to create a setup file or a test initialization file, e.g. setupPlaywright.ts and add setupUserway invocation into it. Then just import setupPlaywright.ts in all of your tests.

import { setupUserway } from '@userway/a11y-playwright'
 
test.describe('navigation', () => {
  test.beforeAll(async () => {
    setupUserway({
      screenshots: false,
      printViolationsTable: true,
      reportPath: './uw-a11y-reports',
    })
  })
})

For more information about @userway/a11y-playwright package refer to the documentation: https://docs.cicd.qa.userway.dev/playwright/getting-started#installation (opens in a new tab)

3. Update Playwright e2e tests with userwayAnalysis

Adding the userwayAnalysis() function call to your existing end-to-end tests allows you to perform static page analysis at any point during test execution. With each function invocation, a report of accessibility violations will be saved in the uw-a11y-reports directory. As a rule of thumb, you should invoke userwayAnalysis() at the end of every e2e test case.

const { userwayAnalysis } = require('@userway/a11y-playwright')
 
test('example test', async ({ page }) => {
  await page.goto('http://localhost:5000')
  await userwayAnalysis(page)
})

Note: userwayAnalysis() accepts multiple arguments and can be parametrized in order to meet your specific conditions and requiremens. Here is more advanced usage:

await userwayAnalysis(page, {
  strict: false,
  screenshots: true,
  savePageOrigin: true,
  saveReport: true,
})

For more information about userwayAnalysis() configuration options visit: https://docs.cicd.qa.userway.dev/playwright/api#api (opens in a new tab)

To see more examples of @userway/a11y-playwright package usage visit sample project repository https://bitbucket.org/userway/sample-test-project (opens in a new tab) TODO dedicated public repo

4. Optional configuration

Update tsconfig.json. If you use TypeScript, add @userway/a11y-playwright to the types section in your tsconfig.json.

{
  "compilerOptions": {
    "types": ["playwright", "@userway/a11y-playwright"]
  }
}

Update .gitignore. Ignore reports generated by the userwayAnalysis function in your git commits by adding the report directory to the .gitignore file.

uw-a11y-reports

5. Verify configuration

It is always a good idea to validate changes made in the steps above before committing them into repository. In order to verify if your Playwright configuration is correct - there are are few steps which can be performed on local developer environment:

• Make all necessary changes described above
• Run your e2e tests locally
• Make sure there is no error and tests pass successfully
• Check directory uw-a11y-reports/reports for JSON files. Number of report files should be equal to number of userwayAnalysis invocations in your e2e tests. Example of a expected report file name uw-a11y-report-lvoeobzh.json

For more information about troubleshooting of @userway/a11y-playwright package refer to the documentation: https://docs.cicd.qa.userway.dev/getting-started#troubleshooting (opens in a new tab)

Prerequisites

• @userway/a11y-playwright compatible with Chrome and Chromium only
• Playwright version 15.3.0 or higher is required
• Node.js v16.20.2 or higher