← Back to Discord Forum

Visual Comparison failing on tests that are run in parallel

garland2953posted in #help-playwright
Open in Discord

Hi, I have a .spec file that has 3 tests within it. When I run these tests individually either using the test runner plugin for vscode, or when running the file in serial mode the pixel comparisons will pass. If I run the file in parallel mode, the visual comparisons will fail. I will not share the actual expected and diff as I do not wish to expose project I am working on. However, the actual and expected screen shots are identical, and the diff seems to show a comparison with completely the wrong screen shot.

Has anyone else ever encountered this? If so, how did you resolve it?

This thread is trying to answer question "Why do visual comparisons fail when tests are run in parallel mode, and how can this issue be resolved?"

7 replies

Maybe you're reusing pages or something? Without looking at the code hard to pinpoint an issue


I will send a redacted snippet shortly


import { test, expect } from "@playwright/test";

test("1", async ({ page }) => { await expect(page).toHaveScreenshot("1.png", { fullPage: true, }); });

test("2", async ({ page }) => { await expect(page).toHaveScreenshot("2.png", { fullPage: true, }); });

test("3", async ({ page }) => { await expect(page).toHaveScreenshot("3.png", { fullPage: true, }); });


Note on this, I am acessing these assertions throught wrappers, however I don't see that being the issue as they execute fine when run serially

have you tried not using wrappers? is the issue persists then?

if it doesn't then you'll know where to look


The issue persists without the wrappers


Why Visual Comparisons Fail in Parallel Mode

When you run tests in parallel mode, visual comparisons can fail. This is because parallel tests are executed in separate worker processes and can't share any state or global variables. This can lead to unexpected results due to interference between tests.

Visual comparisons depend on capturing screenshots or videos of the application under test. In parallel mode, multiple instances of the application may be running at the same time, each capturing its own screenshots or videos. This can introduce timing issues and race conditions.

Additionally, running tests in parallel mode can lead to resource contention issues. If multiple tests require heavy CPU usage or network bandwidth at the same time, it can lead to slower execution times and potential failures in visual comparisons.

How to Address These Issues

To improve the reliability and accuracy of visual comparisons when running tests in parallel mode, you can:

  1. Ensure proper synchronization mechanisms: If your testing framework supports it, use synchronization mechanisms like locks or semaphores to control access to shared resources during visual comparison steps.
// Example using Playwright Test
import { test } from '@playwright/test';

test('visual comparison', async ({ page }) => {
  // Lock access to shared resources
  await page.evaluate(() => {
    // Your synchronization code here

  // Your visual comparison code here
  1. Use unique identifiers: Assign unique identifiers to captured screenshots or videos to avoid conflicts.
// Example using Playwright Test
import { test } from '@playwright/test';

test('visual comparison', async ({ page }) => {
  // Capture screenshot with unique identifier
  await page.screenshot({ path: `screenshot-${Date.now()}.png` });
  1. Optimize resource usage: Analyze the resource requirements of your tests and optimize their execution to minimize contention.

For more tips on how to optimize your Playwright tests, check out this blog post.

Related Discord Threads

AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.