Rayrun
← Back to Discord Forum

How does playwright determine how to split tests during sharding?

I have over 150 tests in total, I'm currently trying to implement sharding in my CI test runs, it seems the split between the tests is quite uneven, one shard would have 7 tests, while another could have 42, is there anyway to make the shard split more even?

This thread is trying to answer question "How does playwright determine how to split tests during sharding?"

4 replies

Also I'm using a single browser atm, chromium

File a bug issue. It shards for me in equal amount and I've never heard of anyone else having such issue.

As for how it determines, I think it basically total amount / number of shards

They are definitely not even, I also had a case where one of the shards was just empty and had no tests in it.

I'll file that bug, thanks

Answer

Sharding Tests in Playwright

Sharding tests in Playwright is a breeze. You can use the --shard command line option to specify the shard index and total number of shards. For instance, if you want to split your test suite into three shards, you can use the following commands:

npx playwright test --shard=1/3
npx playwright test --shard=2/3
npx playwright test --shard=3/3

This way, you can distribute your tests across multiple machines or processes.

Balancing Your Shards

To balance your shards, adjust the number of shards (y) based on your needs. If you have a lot of tests and want each shard to contain roughly the same number of tests, increase y. If some tests take longer than others, decrease y to balance their execution time across shards.

Remember, Playwright doesn't automatically determine an optimal shard split. You're in the driver's seat here.

Controlling Parallelism

By default, Playwright runs test files in parallel with separate worker processes. You can disable parallelism by allowing just a single worker at any time. Use the workers: 1 option in the configuration file or pass --workers=1 as a command line argument.

// configuration file
{
  "workers": 1
}
// command line
npx playwright test --workers=1

You can also parallelize within a single file using test.describe.configure(). By default, tests in a single file run in order. But if you have many independent tests in a single file, you can run them in parallel.

test.describe.configure({ mode: 'parallel' })

Remember, parallel tests run in separate worker processes and can't share any state or global variables. Happy testing with Playwright!

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.