How to run the same test file multiple times in parallel?

+1 I have a very similar question I would like to run the same test file, multiple times, but sequentially (or in parallel, but limit them with workers). I tried wrapping the tests within a for loop, but doing that it runs the 1st test n Times then the 2nd n Times and so on...

@kaspar01 Have your tried using test.describe.configure({ mode: 'parallel' }); ? As described here: https://playwright.dev/docs/test-parallel#parallelize-tests-in-a-single-file

Each parallel run is supposed to use a different test user or test a different variant or setting of your app?

@p01___ In my case, I guess so... I would probably want to fetch data from database for each run.

That's what I was trying:

import { test, expect } from '@playwright/test';

// with fullyParallel: false
test.describe.configure({ mode: 'serial' });

test.describe('Test Describe', () => {
  let page;
  let browser;  

  test.beforeAll(async ({ browser }) => {
    page = await browser.newPage();
  });

  test.afterAll(async () => {
    await page.close();
  });

  for (let i = 1; i <= 2; i++) {
    test(`test A ${i}`, async ({ page }) => {
      // ...
    });

    test(`test B ${i}`, async ({ page }) => {
      // ...
    });
  }
})

Instead... I would like the execution order to be:

test A 1
test B 1
test A 2
test B 2
...

ps. I feel bad for hijacking @kaspar01 thread, but it's quite a similar question/doubt and been three months of inactivity after his question. But please, If he joins us again... Help him first.

Instead of using multiple test() in the for loop, you could use a single test with multiple await test.step(...)

This especially makes sense as you intent to run the "inner tests" sequentially. Test steps are designed for this.

What do you think? Does that help? Work?

https://playwright.dev/docs/api/class-test#test-step

@p01___ Thank you sir. Yes. It definitely makes sense. I'm gonna read the docs (further) & try again with step.

This seems to work. I was using the hooks (beforeAll / afterAll ) to try to control how/when the browser initializes / teardown. But I've realized that Playwright Test, while using the { page } fixture, it already creates the window. I'm still closing it in the afterAll hook though.

// @ts-check
import { test, expect } from '@playwright/test';

test.describe('My Test', () => {

  // let browser;
  // let page;

  // test.beforeAll(async ({ browser }) => {
  //   page = await browser.newPage();
  // });

  test.afterAll(async ({ page }) => {
    await page.close();
  });

  for (let i = 1; i <= 2; i++) {
    test(`${i} - My Test instance`, async ({ page }) => {
      await test.step(`Go to Google`, async () => {
        await page.goto('https://www.google.com');
      });
    
      await test.step(`Go to Linux dot org`, async () => {
        await page.goto('https://www.linux.org');
      });
    });
  }
})

Oh, and when using the beforeAll hook, it was creating a window for each iteration of the loop beforehand (in the example above, 2 windows), but both tests were executed in one of them.

Glad that helped 🎉 Pretty sure you can remove the afterAll as well, because Playwright opens and close a browser context and page per test() 😉

@jeflopodev No worries about hijacking, didn't notice the replies here earlier. Sorry. I'd also raised an issue on PW GitHub (https://github.com/microsoft/playwright/issues/22550) on this and indeed, the suggested solution is to simple enclose your test code by a loop that leads to the code get executed X times in parallel. And that works quite well.