Blog Ask AI Tools Videos QA Wiki Discord

Don't skip the rest of the tests, if any one of the test is failed

Is there a way, that my tests should not skip, if one of the test is failed, using playwright testruner

This thread is trying to answer question "Is there a way to prevent tests from being skipped if one fails when using the Playwright test runner?"

13 replies

debbieobrienSeptember 12 1:48 PM

is soft assertions what you are looking for?

debbieobrienSeptember 12 1:48 PM

https://playwright.dev/docs/next/best-practices#use-soft-assertions

saikiran.kSeptember 12 2:04 PM

Nope @debbieobrien , In mocha or jest, to achieve the same we set bail = false in the resepective config file, can I do the same with playwright test runner ?

I am not seeing for the soft assertions, I want my rest of the tests to be executed even when any of the test is failed in between, as my tests are completely independent.

can you please suggest for the above

debbieobrienSeptember 12 3:11 PM

all tests should still run even if one fails. if test 1 fails test2 will still run unless you have set it up that one depends on the other with project dependencies but out of the box all tests run even if one test fails. soft just allows the failed test to continue running.

d3333333September 12 4:01 PM

@debbieobrien Sorry usually don't direct message, but was wondering if you all might provided some setting on this. Meaning should an expect().toBeXXXX() hard kill the variation. Sure one can use expect.soft() but in development might be times i want to allow test to get past one expct in order to more easily write my tests. Then in an offical test run i any of the plain 'expect().toBeXXX()' fail fast:

So a setting in the PW.Config.ts.... { name: "Desktop", use: { ...devices["Desktop Chrome"], viewport: { width: 1200, height: 6500 }, baseURL: ENV.URL, locale: "en-US", expect: [(FailFast|FailSoft), ScreenShot, other,...?]

Lack of a beter way my regex pattern for the options... FailFast -- Current behavior to end current test FailSoft -- Any expect will behave as a "expect.soft(); ScreenShot -- would create a screen shot of the UI at the point the exception occurred, could be used with expect.soft()...

Thanks just hope i'm not abusing the ping 🙂

debbieobrienSeptember 12 4:53 PM

no worries. You can configure your own so that you always do soft assertions and then change it to true but this is at test level. https://playwright.dev/docs/test-assertions#expectconfigure What you are looking for looks like a feature request to me as you want it in the config. Why don't you submit that as a request on our GitHub and the team can look into it.

saikiran.kSeptember 21 6:18 AM

This is a piece of code, where in one of the test, i am using the soft assertion and intentionally failing the soft assertion. I am expecting when the soft asserting failed, the next test(usecase), should not be skipped. But, here it is skipped. please help me, how to achieve it

ondra123.September 22 12:28 PM

test.describe.configure({ mode: 'serial' }) I believe because of the serial more the next test is skipped

saikiran.kSeptember 24 7:00 AM

I need to run my test in serial mode and if any test fails, the other tests should not be skipped, is ther a way? can you help @ondra123.

sashaknitsSeptember 25 8:36 AM

@saikiran.k If you don't run in serial mode but restrict to 1 worker in your config file, this will give the result you want. Tests will still be run in sequence because there is just one worker and if any test fails, the rest will still be run.

saikiran.kSeptember 25 9:23 AM

@sashaknits , If I do so, my test suite run time in affected. Is there any way other than this?

sashaknitsSeptember 25 9:57 AM

@saikiran.k: No, the only other solution would be to design your tests such that they can run independently and then you wouldn't need to use serial mode. If a test fails, even on a soft assertion, the runner assumes that following tests must be skipped because of being in serial mode.

No, the only other solution would be to design your tests such that they can run independently and then you wouldn't need to use serial mode. If a test fails, even on a soft assertion, the runner assumes that following tests must be skipped because of being in serial mode.

saikiran.kSeptember 25 10:30 AM

okay thanks for the info @sashaknits

Open in Discord

Answer

Handling Test Failures in Playwright

In Playwright, you can handle test failures in two ways: fail fast or fail soft.

Fail Fast with `test.fail()`

The test.fail() function in Playwright marks a test as "should fail". When this test runs, Playwright ensures it fails. This is useful for acknowledging broken functionality until it's fixed. Here's how you use it:

import { test } from '@playwright/test';

test('not yet ready', async ({ page }) => {
  test.fail();
  // ... rest of your code
});

Fail Soft with Soft Assertions

If you want to continue running all tests even if some fail, use soft assertions. These don't immediately terminate the execution when failed. Instead, they compile and display a list of failed assertions once all tests have ended. Here's an example:

import { expect } from '@playwright/test';

await expect.soft(page.getByTestId('status')).toHaveText('Success');
// ... continue executing more checks or actions

With soft assertions, even if one assertion fails, other checks or actions will still be executed. Any subsequent failures will be recorded and displayed at the end of the test execution.

So, depending on your needs, you can use test.fail() for a fail fast approach, or soft assertions with expect.soft() to continue executing all tests and fail soft.

Related Discord Threads