How can I use soft assertions and fixtures in Playwright Test to save artifacts even if an expect fails after a test has passed?

Answer

Sure, you can save artifacts in @playwright/test even if an expect fails after a test has passed. This is possible through the use of "soft assertions" and fixtures.

Soft assertions are checks that don't immediately stop the test execution when they fail. Instead, they're compiled and displayed as a list when the test ends. You can use the expect.soft function for this. Here's an example:

await expect.soft(page.getByTestId('status')).toHaveText('Success');

Even if some checks fail, you can continue making more checks:

await expect.soft(page.getByTestId('eta')).toHaveText('1 day');

You can check for any soft assertion failures using test.info().errors. If this array has a length greater than 0, it means there were failed soft assertions:

expect(test.info().errors).toHaveLength(0);

If there were any failures, you can save artifacts related to them. For example, you can save logs or attach files with additional information about the failure.

Playwright Test provides hooks called fixtures that allow you to define actions before and after each test runs. You can use these fixtures to collect logs during tests and save them as artifacts when necessary. Here's an example:

export const test = base.extend<{ saveLogs: void }>({
  saveLogs: [async ({}, use) => {
    const logs = [];
    debug.log = (...args) => logs.push(args.map(String).join(''));
    debug.enable('myserver');

    await use();

    if (testInfo.status !== testInfo.expectedStatus) {
      const logFile = testInfo.outputPath('logs.txt');
      await fs.promises.writeFile(logFile, logs.join('\n'), 'utf8');
      testInfo.attachments.push({
        name: 'logs',
        contentType: 'text/plain',
        path: logFile
      });
    }
  }, { auto: true }],
});

In this example, the saveLogs fixture collects logs during the test execution. If the test status doesn't match the expected status, it saves the collected logs as an attachment.