Rayrun
← Back to Discord Forum

Is there a way to run _only_ tests marked as "fixme"?

The use case is that sometimes we need to debug if test is flaky or not.

This thread is trying to answer question "Is there a way to run only tests marked as 'fixme'?"

2 replies

I use the following command locally and in a GH pipeline:

npx playwright test --project=chromium --grep "@flaky" --max-failures 0 --retries 0 --trace on

This just runs my flaky tests once but adds a trace since I assume failure. I also run this on my CI build but do not include this status in my overall test result success.

Not sure if this is what you are looking for. We also have a separate pipeline to allow testers to work on flaky tests since sometimes the test is only flaky in an environment rather than flaky on every run.

If you need further information let me know.

Good suggestiong but i would avoid the @ sign more as it will be a pain depending on the environment you need to run in. Might be i'm lazy and avoid trouble if possible, and simply use "--myTag" and then for the regex -g".--(a|b|c|d)." where the a ,b is the tag you added... This should work. While my concern you might be on a fools errand simply because if i'm remembering, test marked fixme the runner will skip but you want to run them, so drop the fixme and update the test description with the tag suggested and you should have what you want. yep double checking, 'fixme' will skip so would suggest to avoid as suggested..

https://playwright.dev/docs/test-annotations#use-fixme-in-beforeeach-hook

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.