Rayrun
← Back to Discord Forum

what’s the best way to go about visual comparison

How do I execute screenshot test

This thread is trying to answer question "How do I execute a screenshot test using Playwright without it failing?"

35 replies

keeps failing

@skorp32

Whats failing?

Where failing?

Whats the error message?

What have you tried to fix it?

Error: Screenshot comparison failed:

621 pixels (ratio 0.01 of all image pixels) are different.

.aleksandaraleksandrov
.aleksandaraleksandrov

hello godsown, look at the properties toHaveScreenshot

@.aleksandaraleksandrov what of just screenshort

Call log:

  • expect.toHaveScreenshot(landingpage.png) with timeout 60000ms
  • generating new stable screenshot expectation
  • taking page screenshot
  • disabled all CSS animations
  • waiting 100ms before taking screenshot
  • taking page screenshot
  • disabled all CSS animations
  • 498 pixels (ratio 0.01 of all image pixels) are different.
.aleksandaraleksandrov
.aleksandaraleksandrov

await expect(page).toHaveScreenshot({ maxDiffPixels: 1000 }); try something like this.

await expect(page).toHaveScreenshot( 'landingpage.png',{ fullPage: true}, { maxDiffPixels: 1000 }) ;

cannot take three properties

.aleksandaraleksandrov
.aleksandaraleksandrov

await expect(page).toHaveScreenshot( 'landingpage.png',{ fullPage: true, maxDiffPixels: 1000 }) ;

.aleksandaraleksandrov
.aleksandaraleksandrov

First time the test always fails, because it is making the screenshot. The second time it is comparing the screenshot to the previously taken screenshot

Error: Screenshot comparison failed:

Expected an image 1280px by 720px, received 1280px by 10878px. 12825961 pixels (ratio 0.93 of all image pixels) are different.

Call log:

  • expect.toHaveScreenshot(landingpage.png) with timeout 60000ms
  • verifying given screenshot expectation
  • taking page screenshot
  • disabled all CSS animations
  • Expected an image 1280px by 720px, received 1280px by 10878px. 12826060 pixels (ratio 0.93 of all image pixels) are different.
  • waiting 100ms before taking screenshot
  • taking page screenshot
  • disabled all CSS animations
  • captured a stable screenshot
  • Expected an image 1280px by 720px, received 1280px by 10878px. 12825961 pixels (ratio 0.93 of all image pixels) are

failed twice

can i just take screenshot instead

@_godsown: https://playwright.dev/docs/screenshots

Will just go for normal screenshots then

.aleksandaraleksandrov
.aleksandaraleksandrov

I guess the first screenshot is 1280px by 720px, and the second is on the whole page. This is why, things are not working out.

.aleksandaraleksandrov
.aleksandaraleksandrov

Just do a test 2 times in a row, without changing anything and add maxDiffPixels: 10000 or something that might help the big gap in the pixels

It’s forever failing let me just do screenshots test @.aleksandaraleksandrov

Are you available for a collaboration debugging @.aleksandaraleksandrov

.aleksandaraleksandrov
.aleksandaraleksandrov

sure

Should I drop a google meet

@.aleksandaraleksandrov ?

.aleksandaraleksandrov
.aleksandaraleksandrov

Problem solved.

@.aleksandaraleksandrov i have tagged you in one of my query please have a look

man_bear_pig_79
man_bear_pig_79

Image comparison tests for us is also very unreliable ... @.aleksandaraleksandrov how did you manage to fix your issues?

.aleksandaraleksandrov
.aleksandaraleksandrov

I can get into a call for 5 min if you have the time?

.aleksandaraleksandrov
.aleksandaraleksandrov

@man_bear_pig_79

man_bear_pig_79
man_bear_pig_79

hey man am sorry gotta go to a meeting might be back in an hour or so

.aleksandaraleksandrov
.aleksandaraleksandrov

I will be available in 4-5 hours on my lunch break, write on personal message.

Answer

Running a Screenshot Test with Playwright

First off, make sure you've got Playwright installed in your project. If not, you can add it with this command:

npm install @playwright/test

Next, create a test file, say example.spec.ts, and import the necessary functions:

import { test, expect } from '@playwright/test';

Now, define a test case. Here's an example:

test('example test', async ({ page }) => {
  // Your test code goes here
});

In the callback function, navigate to the webpage you want to test:

await page.goto('https://ray.run');

To capture a screenshot of the entire page, use the page.screenshot method:

await page.screenshot({ path: 'screenshot.png' });

This will save a full-page screenshot as "screenshot.png" in your project directory. If you want to capture screenshots of specific elements, use locators:

await page.locator('.header').screenshot({ path: 'header-screenshot.png' });

This captures a screenshot of an element with class "header". You can customize your screenshots with additional parameters to page.screenshot.

Finally, run your screenshot tests with the Playwright test runner:

npx playwright test

On first run, Playwright Test generates reference screenshots. Subsequent runs compare against these and report differences. Remember to review and update your screenshots when necessary with the --update-snapshots flag. Happy testing!

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.