Answer

Fixing Flaky Tests with @playwright/test

Flaky tests can be a nuisance, but don't worry, @playwright/test has got you covered. The key to tackling flaky tests is using test retries.

Test Retries

Test retries allow you to automatically re-run a test when it fails. This is especially handy when dealing with flaky tests that fail intermittently. You can configure test retries in your configuration file.

import { test } from '@playwright/test';

test.use({
  retries: 3,
});

When all tests pass, they run in order in the same worker process. If a test fails, @playwright/test discards the worker process along with the browser and starts a new one. The new worker process starts by retrying the failed test and continues from there.

Web First Assertions

In addition to using test retries, it's crucial to use web first assertions such as toBeVisible() instead of isVisible(). This makes your assertions more reliable and less likely to cause flakiness.

Debugging Tests

For local debugging, you can use the VS Code extension to debug your tests live. Just right-click on the line next to the test you want to run, and it will open a browser window and pause at the breakpoint.

import { test } from '@playwright/test';

test('my test', async ({ page }) => {
  await page.goto('https://ray.run');
  // Set a breakpoint here in VS Code
  const title = await page.title();
  test.expect(title).toBe('Ray');
});

You can also use the Playwright inspector by running your tests with the --debug flag. This allows you to step through your code while viewing actionability logs and editing locators live.

For CI failures, use the Playwright trace viewer. It gives you a full trace of your tests as a local Progressive Web App (PWA) that can be easily shared.

Remember, flaky tests are not a dead-end. With @playwright/test, you have the tools to tackle them head-on. Happy testing!