Rayrun
← Back to Discord Forum

Visual tests on CI with Windows

Hi everyone,

We're using Playwright as Integration tests and we want to add a layer of visual testing because our goal is to test several browsers and devices. We are using node to run Playwright and we are developing in Windows. The idea is to create the expected snapshots on our computer, so we can verify the quality and be sure everything looks right. And then when a new merge request is opened for a new feature, the CI is running to verify if everything looks how we expect it to be.

So we decided to go with Docker. We are building an image with the following steps :

  • use windows docker image with Playwright
  • copy our code with the new MR code
  • get expected screenshots from the folder
  • install packages Then we save the docker in the GitLab registry and we call this image in our CI to do the tests.

It feels a little bit too much like we feel we have a lot of work and construction to do for something that's not that trustworthy in time. It's a valid answer now, but will it be an answer in several dev sprints? So my question is, what do you suggest/recommend for Visual testing with a pipeline/CI environment?

Thank you for your time and thank you in advance for your answers

This thread is trying to answer question "What do you suggest/recommend for Visual testing with a pipeline/CI environment?"

0 replies

Answer

Integrating Playwright for Visual Testing in CI/CD Environments

Playwright is a robust tool for visual testing in CI/CD environments. It supports multiple programming languages and allows you to automate browser actions and validate the visual appearance of your web application across different browsers.

Jenkins

If you're using Jenkins, use Docker agents to run your tests. Use the Playwright Docker image provided by Microsoft. Here's an example of how to configure the agent in a Jenkins pipeline:

pipeline {
  agent {
    docker {
      image 'mcr.microsoft.com/playwright:v1.XX.X-jammy'
    }
  }
  stages {
    stage('e2e-tests') {
      steps {
        sh 'ci'
        sh 'npx playwright test'
      }
    }
  }
}

Bitbucket Pipelines

For Bitbucket Pipelines, you can also use Docker images as build environments. Here's an example configuration for running Playwright tests:

image: mcr.microsoft.com/playwright:v1.XX.X-jammy

pipelines:
  default:
    • step:
        name: Run Playwright Tests
        script:
          • ci
          • npx playwright test

GitLab CI

For GitLab CI, you can also use the Playwright Docker image. GitLab CI supports sharding tests between multiple jobs for faster execution. Here's an example configuration:

stages:
  • test

tests:
  stage: test
  image: mcr.microsoft.com/playwright:v1.XX.X-jammy
  parallel: 7
  script:
    • npm ci
    • npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

Azure Pipelines

For Azure Pipelines, integrating Playwright is straightforward. For Windows or macOS agents, no additional configuration is required. For Linux agents, you can use a Docker container with Azure Pipelines support or install all necessary dependencies using command line tools.

Playwright provides a reliable solution for visual testing in any pipeline/CI environment. For more information, check out our blog post on the rapid adoption of Playwright in software QA.

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.