React setup and CI/CD for End-to-End (E2E) testing with Playwright, TypeScript, and GitHub Actions

This article will be divided into 2 parts, an introduction with some general information and practical code examples with explanations. The project shown is using create-react-app. If you are interested only in deployment to GitHub Actions, jump to the end for a sample working YAML file.

As we all know, bugs in production cause confusion and irritation to our clients, but most importantly it’s possible to lose money… To avoid this, testing must be an important part of every application. Sadly most developers don’t like the idea of writing tests, or they start when there are already problems and issues raised by clients or businesses.

In this article, we are not gonna go into details about types of testing or bugs, but at least mention some of the most annoying ones in my opinion – regression bugs. To explain it a little it’s a bug that was already fixed in previous deployments and in newer versions it appears again, and to make it spicier it’s found and reported by your clients… To avoid this we can increase our E2E testing coverage. There are more established frameworks to choose from as Cypress, but Playwright is gaining credibility from the community and is my personal favorite for the moment. It’s easy to set up, the documentation is well structured and a lot of integrations are available.

Enough small talk, let’s see how to set it up, save the login state between all tests, and write our first test.

First, let’s install the needed packages. If you are using npm.

npm install @playwright/test dotenv

If you are using yarn (as I am)

yarn add @playwright/test dotenv

Dotenv is used to load environment variables in our tests and setup. We need to update the scripts in our package.json file. Let’s add two scripts

"scripts": {
  "start": "react-scripts start",
  "test:e2e": "playwright test",
  "test:e2e:report": "npx playwright show-report"
}

We assume that our header (navbar) has two buttons — login/register, which redirects us to a login/register page. Each of them has an additional data-attribute added by us. I prefer to add to all buttons and inputs data-testid.

{/* 
  let's assume we have useNavigate hook used above at our component
  or whatever we use for navigation
 */}
<button data-testid="loginButton" onClick={() => navigate('/login')}>Login</button>

Time to create our global setup file on a root level. We need to launch a chromium instance and a new page. Go to our application URL and get to the login page. In there we have 2 inputs and 1 button. Each input has a name prop (email, password) and the button has a data-testid. It’s a standard flow in which we enter our email, and password and click on submit (login) button. After successful login we are redirected to our private part of the application, to wait for this and have our cookies set we need to do await page.waitForNavigation().

Important note here is to have this login state saved for all our tests, we need to save it in a file, let’s say — storageState.json. I suggest you add this to .gitignore, as you really don’t need it in Git.

import type { FullConfig } from '@playwright/test';
import { chromium } from '@playwright/test';
require('dotenv').config();

// Both constants be easily extracted in environment variables
const EMAIL = '[email protected]';
const PASSWORD = 'my-test-password';

async function globalSetup(config: FullConfig) {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  // Url - http://localhost:3000 | https://my-website.com
  await page.goto(process.env.REACT_APP_CLIENT_URL);
  // Clicking this will redirect us to http://our-url/login
  await page.click('data-testid=loginButton');
  // After redirection we are looking for inputs with specific names
  await page.locator('input[name="email"]').fill(EMAIL);
  await page.locator('input[name="password"]').fill(PASSWORD);
  await page.locator('data-testid=loginSubmit').click();
  await page.waitForNavigation();
  // Save signed-in state to 'storageState.json'.
  await page.context().storageState({ path: 'storageState.json' });

  await browser.close();
}
export default globalSetup;

We have the global setup, but we need to pass it to our configuration, which we don’t have yet. Time to create a file named playwright.config.ts at the root level again. It can look a little bit too much at first, but it’s a more or less one-time configuration. As you can see below, I have divided some of the logic depending on the NODE_ENV as tests are gonna be triggered in GitHub Actions, and I want to reduce the bill (if you don’t care about reducing the price, it’s should be simple to update the code). Locally I like to test against multiple browsers and viewports, as to increase the chances to catch a bug (you can even add more). In my setup, I am using cookies for authentication, but it will work with localStorage too.

You don’t need to start your react app locally, as you can see config.webServer is doing it automatically for us.

import type { PlaywrightTestConfig } from '@playwright/test';
import { devices } from '@playwright/test';
require('dotenv').config();

// See https://playwright.dev/docs/test-configuration.
const config: PlaywrightTestConfig = {
  globalSetup: require.resolve('./globalSetup.ts'),
  testMatch: /.*.e2e.ts/,
  // Maximum time one test can run for. 
  timeout: process.env.NODE_ENV === 'production' ? 40000 : 220000,
  expect: {
    /**
     * Maximum time expect() should wait for the condition to be met.
     * For example in `await expect(locator).toHaveText();`
     */
    timeout: process.env.NODE_ENV === 'production' ? 40000 : 220000,
  }, 
  fullyParallel: true, // Run tests in files in parallel
  forbidOnly: !!process.env.CI, // Fail the build on CI if you accidentally left test.only in the source code.
  retries: process.env.CI ? 2 : 0, // Retry on CI only
  workers: process.env.CI ? 1 : undefined, // Opt out of parallel tests on CI.
  reporter: 'html', // See https://playwright.dev/docs/test-reporters
  use: {
    headless: true, // If set to false, you are gonna see a browser open when testing
    actionTimeout: 0, // Maximum time actions such as `click()` can take. Defaults to 0 (no limit). 
    // Base URL to use in actions like `await page.goto('/')`. 
    baseURL: process.env.REACT_APP_CLIENT_URL,
    // Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer 
    trace: 'on-first-retry',
    launchOptions: {
      slowMo: process.env.NODE_ENV === 'production' ? 0 : 50
    },
    storageState: 'storageState.json' // Load signed-in state from 'storageState.json' for all tests.
  },
  projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome']
      }
    },
    {
      name: 'webkit',
      use: {
        ...devices['Desktop Safari']
      }
    },

    // Test against mobile viewports. 
    {
      name: 'Mobile Safari',
      use: {
        ...devices['iPhone 12']
      }
    }
  ]
  // Folder for test artifacts such as screenshots, videos, traces, etc. 
  // outputDir: 'test-results/',
};

if (process.env.NODE_ENV !== 'production') {
  // Run your local dev server before starting the tests 
  config.webServer = {
    command: 'yarn start', // with npm -> npm run dev
    port: 3000
  };
  config.projects = [
    ...(config.projects || []),
    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox']
      }
    },
    // Test against mobile viewports. 
    {
      name: 'Mobile Chrome',
      use: {
        ...devices['Pixel 5']
      }
    },
    {
      name: 'Google Chrome',
      use: {
        channel: 'chrome'
      }
    }
  ];
}

export default config;

Where you put your tests is really a personal preference, mine is next to the component with the component name and “.e2e.ts” at the end. We need to do a little explanation of how our app works.

We have a route “my-app/blog/list” and inside is a list of blog posts and a button at the top with “data-testid=openCreateBlogButton”. Create button opens a dialog for a new post. The dialog has an input field with a name attribute“name=title”, and a submit button with “data-testid=createBlogButton”. For the sake of making this tutorial shorter and simpler, let’s assume that this is enough to create a blog post (you can easily add more fields). By clicking the create blog button the dialog is closed and the post is appended at the top of our list.

So we have an overview of how our application works. Time to write our first test. Let’s create a file named “BlogList.e2e.ts”, to add our logic inside.
We want to go to our route before every test, and to have a sample title predefined so we can test against it.

import { test, expect } from '@playwright/test';

const BLOG_TITLE = 'My Awesome Testing Blog';

test.beforeEach(async ({ page }) => {
  await page.goto('/blog/list');
});

test.describe('Create a blog post', () => {
  test('it should create a blog post', async ({ page }) => {
    // Opens our dialog with title input
    await page.locator('data-testid=openCreateBlogButton').click();
    // Fills up the title name with our predefined one
    await page.locator('input[name="title"]').fill(BLOG_TITLE);
    // Submits and creates our blog post
    await page.locator('data-testid=createBlogButton').click();
    // As we already know that the post should appear in the list, we want
    // to test if it's already visible.
    await expect(
      page.locator(`text=${BLOG_TITLE}`).first()
    ).toBeVisible();
  });
});

You can check if everything is working correctly for now with the script we added (this assumes you have your BE started and it’s working correctly)

yarn test:e2e # with npm -> npm run test:e2e

Time to continue with our GitHub Actions integration

Let’s create a .github/workflows directory in the root level of your applications if this directory does not already exist. As we are interested in playwright only right now, let’s create a file named “playwright.yml”. And this is how it should look.

Depending on our business requirements we can choose a different strategy for when the tests are to be run. Ours is to test and trigger actions on every push or pull request (PR) made to our master branch.

name: Playwright E2E Testing
on:
  push:
    branches: [master]
  pull_request:
    branches: [master]
jobs:
  e2e:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 16
      - name: Install dependencies
        run: yarn install --frozen-lockfile # with npm -> npm ci
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
      - name: Run Playwright tests
        run: yarn test:e2e # with npm -> npm run test:e2e
        env:
          REACT_APP_CLIENT_URL: https://my-blog-url.com
          NODE_ENV: production
      - uses: actions/upload-artifact@v3
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Actually, that’s it about workflows. Integrating with GitHub Actions is as simple as that. It can be upgraded to wait for a build in Vercel to be successful or other dependencies, but this is a sample one that will get you started.

I hope I saved you some time, as this took me a while to set it up per my needs for the first time. Don’t forget that this might be annoying, but it will definitely save you a lot of headaches in the future and make your product more stable.