Answer

Handling Test Failures in @playwright/test

Test failures when mocking returned data using page.route and route.fulfill in @playwright/test can be due to a few reasons.

Timeout Issues

One common reason is that the operation is taking longer than the maximum operation time set. By default, this value is set to 0, which means no timeout. If your operation exceeds this limit, it will result in a test failure.

// Set the default navigation timeout
browserContext.setDefaultNavigationTimeout(5000);

You can adjust the timeout settings using methods like setDefaultNavigationTimeout() or setDefaultTimeout().

Event Completion

Another reason could be related to when you consider an operation succeeded. The 'waitUntil' option allows you to specify when an operation should be considered successful. It defaults to 'load', but there are other options available such as 'domcontentloaded', 'networkidle', and 'commit'.

// Wait until the DOM is fully loaded
page.goto('https://ray.run', { waitUntil: 'domcontentloaded' });

Mocking Responses

Lastly, while mocking responses with route.fulfill(), ensure the response object passed has all necessary fields filled correctly, especially the status code which defaults to 200.

// Mock a response
await page.route('https://ray.run/api/data', route => route.fulfill({
  status: 200,
  body: JSON.stringify({ key: 'value' })
}));

Remember to set an appropriate timeout limit and carefully choose what event should signify successful completion of an operation. For more tips on handling flaky tests in Playwright, check out this blog post.