Rayrun
← Back to Discord Forum

Visual comparison 1px difference - fail

michaldrajnaposted in #help-playwright
Open in Discord
michaldrajna
michaldrajna

Hi, I use visual testing. I get an error:

`Error: screenshot comparison failed:

Expected an image 132px by 46px, received 133px by 46px. `

There is a difference of 1px. I set toMatchSnapshot in the config: { maxDiffPixels: 1.01 }, but the test failed anyway.

Why?

This thread is trying to answer question "Why does the visual test fail due to a 1px difference even when maxDiffPixels is set to 1.01 in the config?"

4 replies

maxDiffPixels ignores errors only if the pixels are the same count, when the images are different like in your case, it'll fail no matter what percentage/count is set. Any ideas why the images are different sizes?

michaldrajna
michaldrajna

I don't know why. It'll just be some bad rendering of the elements - a bit too much space between div elements or something like that. Almost unrecognizable to the eye.

Is it the same when you add { fullPage: true } ? We had some issues between devices and just.. started using the playwright docker image, now it's kind consistent(but a bit slow to start)

michaldrajna
michaldrajna

I compared just elements sooo. I dunno how to fix it really without Docker like you.

Answer

Understanding Visual Test Failures in @playwright/test

If you're wondering why your visual test fails due to a 1px difference even when maxDiffPixels is set to 1.01, it's because maxDiffPixels specifies the acceptable amount of pixels that could be different between compared images. However, maxDiffPixels doesn't support decimal values. It expects an integer value representing the maximum number of pixels that can differ between images.

// Incorrect usage of maxDiffPixels
const config = {
  maxDiffPixels: 1.01
};

Also, consider the threshold for color difference, determined by the threshold option. This represents an acceptable perceived color difference in the YIQ color space between corresponding pixels in compared images. The default value for threshold is 0.2 unless configured differently with TestConfig.expect.

// Correct usage of maxDiffPixels and threshold
const config = {
  maxDiffPixels: 1,
  threshold: 0.2
};

Remember, other factors can contribute to visual test failures apart from just pixel differences and color thresholds. These could include variations in rendering across different devices or environments, dynamic content changes on web pages, or even issues with capturing screenshots accurately.

So, when configuring visual tests with Playwright's snapshot assertions and TestConfig options like maxDiffPixels, consider not only pixel differences but also factors like color thresholds and other potential sources of variation that may affect image comparison results.

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.