Rayrun
← Back to Discord Forum

Don't skip the rest of the tests, if any one of the test is failed

Is there a way, that my tests should not skip, if one of the test is failed, using playwright testruner

This thread is trying to answer question "Is there a way to prevent tests from being skipped if one fails when using the Playwright test runner?"

13 replies

is soft assertions what you are looking for?

Nope @debbieobrien , In mocha or jest, to achieve the same we set bail = false in the resepective config file, can I do the same with playwright test runner ?

I am not seeing for the soft assertions, I want my rest of the tests to be executed even when any of the test is failed in between, as my tests are completely independent.

can you please suggest for the above

all tests should still run even if one fails. if test 1 fails test2 will still run unless you have set it up that one depends on the other with project dependencies but out of the box all tests run even if one test fails. soft just allows the failed test to continue running.

@debbieobrien Sorry usually don't direct message, but was wondering if you all might provided some setting on this. Meaning should an expect().toBeXXXX() hard kill the variation. Sure one can use expect.soft() but in development might be times i want to allow test to get past one expct in order to more easily write my tests. Then in an offical test run i any of the plain 'expect().toBeXXX()' fail fast:

So a setting in the PW.Config.ts.... { name: "Desktop", use: { ...devices["Desktop Chrome"], viewport: { width: 1200, height: 6500 }, baseURL: ENV.URL, locale: "en-US", expect: [(FailFast|FailSoft), ScreenShot, other,...?]

Lack of a beter way my regex pattern for the options... FailFast -- Current behavior to end current test FailSoft -- Any expect will behave as a "expect.soft(); ScreenShot -- would create a screen shot of the UI at the point the exception occurred, could be used with expect.soft()...

Thanks just hope i'm not abusing the ping 🙂

no worries. You can configure your own so that you always do soft assertions and then change it to true but this is at test level. https://playwright.dev/docs/test-assertions#expectconfigure What you are looking for looks like a feature request to me as you want it in the config. Why don't you submit that as a request on our GitHub and the team can look into it.

This is a piece of code, where in one of the test, i am using the soft assertion and intentionally failing the soft assertion. I am expecting when the soft asserting failed, the next test(usecase), should not be skipped. But, here it is skipped. please help me, how to achieve it

test.describe.configure({ mode: 'serial' }) I believe because of the serial more the next test is skipped

I need to run my test in serial mode and if any test fails, the other tests should not be skipped, is ther a way? can you help @ondra123.

@saikiran.k If you don't run in serial mode but restrict to 1 worker in your config file, this will give the result you want. Tests will still be run in sequence because there is just one worker and if any test fails, the rest will still be run.

@sashaknits , If I do so, my test suite run time in affected. Is there any way other than this?

@saikiran.k: No, the only other solution would be to design your tests such that they can run independently and then you wouldn't need to use serial mode. If a test fails, even on a soft assertion, the runner assumes that following tests must be skipped because of being in serial mode.

No, the only other solution would be to design your tests such that they can run independently and then you wouldn't need to use serial mode. If a test fails, even on a soft assertion, the runner assumes that following tests must be skipped because of being in serial mode.

okay thanks for the info @sashaknits

Answer

Handling Test Failures in Playwright

In Playwright, you can handle test failures in two ways: fail fast or fail soft.

Fail Fast with test.fail()

The test.fail() function in Playwright marks a test as "should fail". When this test runs, Playwright ensures it fails. This is useful for acknowledging broken functionality until it's fixed. Here's how you use it:

import { test } from '@playwright/test';

test('not yet ready', async ({ page }) => {
  test.fail();
  // ... rest of your code
});

Fail Soft with Soft Assertions

If you want to continue running all tests even if some fail, use soft assertions. These don't immediately terminate the execution when failed. Instead, they compile and display a list of failed assertions once all tests have ended. Here's an example:

import { expect } from '@playwright/test';

await expect.soft(page.getByTestId('status')).toHaveText('Success');
// ... continue executing more checks or actions

With soft assertions, even if one assertion fails, other checks or actions will still be executed. Any subsequent failures will be recorded and displayed at the end of the test execution.

So, depending on your needs, you can use test.fail() for a fail fast approach, or soft assertions with expect.soft() to continue executing all tests and fail soft.

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.