Rayrun
← Back to Discord Forum

How to run the same test file multiple times in parallel?

Is there a way to make Playwright or Playwright Test run the same test file multiple times in parallel? The test file contains multiple tests which are and must be executed sequentially, which means that the parallelization shall only take place on file level (same file multiple times) but not within the file. If there is no built-in support in Playwright for that (haven't found anything in the docs), what's the recommended way of achieving this, short of duplicating the test file with multiple different file names (but same content) -- because we're talking about a parallelization level of several thousand times...

This thread is trying to answer question "How to run the same test file multiple times in parallel?"

14 replies

+1 I have a very similar question I would like to run the same test file, multiple times, but sequentially (or in parallel, but limit them with workers). I tried wrapping the tests within a for loop, but doing that it runs the 1st test n Times then the 2nd n Times and so on...

@kaspar01 Have your tried using test.describe.configure({ mode: 'parallel' }); ? As described here: https://playwright.dev/docs/test-parallel#parallelize-tests-in-a-single-file

Each parallel run is supposed to use a different test user or test a different variant or setting of your app?

jeflopodev

@p01_ In my case, I guess so... I would probably want to fetch data from database for each run.

jeflopodev

That's what I was trying:

import { test, expect } from '@playwright/test';

// with fullyParallel: false
test.describe.configure({ mode: 'serial' });

test.describe('Test Describe', () => {
  let page;
  let browser;  

  test.beforeAll(async ({ browser }) => {
    page = await browser.newPage();
  });

  test.afterAll(async () => {
    await page.close();
  });

  for (let i = 1; i <= 2; i++) {
    test(`test A ${i}`, async ({ page }) => {
      // ...
    });

    test(`test B ${i}`, async ({ page }) => {
      // ...
    });
  }
})
image.png
jeflopodev

Instead... I would like the execution order to be:

test A 1
test B 1
test A 2
test B 2
...
jeflopodev

ps. I feel bad for hijacking @kaspar01 thread, but it's quite a similar question/doubt and been three months of inactivity after his question. But please, If he joins us again... Help him first.

Instead of using multiple test() in the for loop, you could use a single test with multiple await test.step(...)

This especially makes sense as you intent to run the "inner tests" sequentially. Test steps are designed for this.

What do you think? Does that help? Work?

https://playwright.dev/docs/api/class-test#test-step

@p01_: @p01_ Thank you sir. Yes. It definitely makes sense. I'm gonna read the docs (further) & try again with `step`.

@p01_ Thank you sir. Yes. It definitely makes sense. I'm gonna read the docs (further) & try again with step.

This seems to work. I was using the hooks (beforeAll / afterAll ) to try to control how/when the browser initializes / teardown. But I've realized that Playwright Test, while using the { page } fixture, it already creates the window. I'm still closing it in the afterAll hook though.

// @ts-check
import { test, expect } from '@playwright/test';

test.describe('My Test', () => {

  // let browser;
  // let page;

  // test.beforeAll(async ({ browser }) => {
  //   page = await browser.newPage();
  // });

  test.afterAll(async ({ page }) => {
    await page.close();
  });

  for (let i = 1; i <= 2; i++) {
    test(`${i} - My Test instance`, async ({ page }) => {
      await test.step(`Go to Google`, async () => {
        await page.goto('https://www.google.com');
      });
    
      await test.step(`Go to Linux dot org`, async () => {
        await page.goto('https://www.linux.org');
      });
    });
  }
})

Oh, and when using the beforeAll hook, it was creating a window for each iteration of the loop beforehand (in the example above, 2 windows), but both tests were executed in one of them.

Glad that helped 🎉 Pretty sure you can remove the afterAll as well, because Playwright opens and close a browser context and page per test() 😉

@jeflopodev No worries about hijacking, didn't notice the replies here earlier. Sorry. I'd also raised an issue on PW GitHub (https://github.com/microsoft/playwright/issues/22550) on this and indeed, the suggested solution is to simple enclose your test code by a loop that leads to the code get executed X times in parallel. And that works quite well.

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.