Rayrun
← Back to Discord Forum

How to approach "semi-parallelism"

I have encounter following issue and I would love to hear your perspectives on this. Did someone try to solve something similar?

The problem: Lets say I have test cases A, B and C. It is perfectly fine to run those in parellel. Now I would like to test those cases in multiple browser configurations, which means multiple projects and multiple test runs of the same cases. However due to some data dependencies (lets not focus on the exact reason please) I cannot run the same test in more instances at the same time (so its fine to have 3 workers to run A-B-C, but it is not possible to run A-A-A). How to approach this? I considered those solutions:

  1. Disable parallelism - this is easy way out, but not prefered one because I don't want to lose this nice feature
  2. Disable parallelism for specific files - this is not possible to my knowledge
  3. Split test run to only run a single project at a time - this is pretty easy to setup by npm commands and it would nicely solve my problem. But I also need to preferably get one test report from all runs. Playwright has ability to merge reports using "blob" option. However every blob report is named "report.zip" and overwrite the previous one. According to the documentation there is only option to give a blob custom folder location, not a custom name. In this case I would have to generate every blob to a different folder. But to merge them, they need to be in one folder. So I would have to come up with some script to rename those reports and move them to one folder, which is little bit of an overkill in my mind. (I and i have no idea how would this work in CI)
  4. Sharding - but to my knowledge there is no way how to split tests to shard in a custom manner
  5. <Your suggestion goes here>

Bonus: Someone here previously run into similar issue as described in 3) when trying to run tests in monorepo with multiple workspaces and then merge the reports. No solution was find in this post.

This thread is trying to answer question "How to approach running test cases in multiple browser configurations in parallel when the same test cannot be run in more instances at the same time due to data dependencies?"

3 replies

It's undocumented but you can give blobs different name via PWTEST_BLOB_REPORT_NAME environment variable.

Thx, that was what I was looking for! However, when I use custom names (report-chrome.zip, report -firefox.zip etc.), it still overwrites existing report file in the folder even if it has different name. So i still only have file from the last run 🤯

Ahh, playwright probably clears blob directory like it clears test-results. You may still need different folders for each report then 😦

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.