Rayrun
← Back to Discord Forum

How to programmatically change the order in which tests are executed?

I am working on an abstraction for surfacing flaky tests (https://github.com/lucgagan/playwright-havoc/). and one of the common causes of flaky tests is interdependency of tests. I would like to surface those issues by forcing the tests to run in a different than usual order (e.g. backwards). How could I achieve this programmatically?

This thread is trying to answer question "How can one programmatically change the order in which tests are executed to surface issues related to interdependency of tests?"

6 replies

and one of the common causes of flaky tests is interdependency of tests. Isn't that mostly solved by Playwright Test encouraging, by design, tests to not be interdependent?

The way you could achieve this is by removing the serialisation of tests that'd be required for them to be interdependent

Although not sure about the value here given that users would most likely know their tests are interdependent and therefore that any change to their order or removing serialisation will break them

Isn't that mostly solved by Playwright Test encouraging, by design, tests to not be interdependent?

There is nothing stopping tests from being dependent on each other at the moment.

I was just debugging a case the other day where another test created an authenticated session and it was leaking to the next test

There is nothing stopping tests from being dependent on each other at the moment.

That's pretty much the point of browser contexts, each test gets it's own context & page, unless you purposely re-use a page object between tests: https://playwright.dev/docs/test-retries#reuse-single-page-between-tests https://playwright.dev/docs/browser-contexts

I was just debugging a case the other day where another test created an authenticated session and it was leaking to the next test That's not a flaky test though, that issue would be the same every test run.

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.