Answer

Configuring Test Retries in Playwright Test

To manage failing or flaky tests, you can use the test retries feature in Playwright Test. By default, failing tests are not retried. But you can change this by specifying the number of retry attempts.

Enabling Test Retries via Command Line

Use the --retries flag followed by the number of retry attempts. Here's how:

npx playwright test --retries=3

Configuring Test Retries in Configuration File

You can also set the number of retry attempts in the playwright.config.ts file.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: 3,
});

Understanding Test Categories

Playwright Test categorizes tests into "passed", "flaky", and "failed". You can detect whether a test is being retried at runtime using testInfo.retry.

Setting Retries for Specific Tests

To specify retries for a specific group of tests or a single file, use test.describe.configure().

import { test } from '@playwright/test';

test.describe(() => {
  test.describe.configure({ retries: 2 });

  test('test 1', async ({ page }) => {
    // ...
  });

  test('test 2', async ({ page }) => {
    // ...
  });
});

Running Dependent Tests in Order

Use test.describe.serial() to ensure that dependent tests always run together and in order.

import { test } from '@playwright/test';

test.describe.configure({ mode: 'serial' });

test.beforeAll(async () => {
   /* ... */
});

test('first good', async ({ page }) => {
   /* ... */
});

test('second flaky', async ({ page }) => {
   /* ... */
});

test('third good', async ({ page }) => {
   /* ... */
});

By enabling test retries and configuring them appropriately, you can control how many times failing or flaky tests should be retried. For more on handling flaky tests, check out this post.