Do you have retries enabled?

Test retries are a way to automatically re-run a test when it fails. This is useful when a test is flaky and fails intermittently. Test retries are configured in the configuration file.

https://playwright.dev/docs/next/test-retries

For specific tests you can use test.describe.configure({ retries: 2 });

I'm not quite sure how this works if you need to wait a specific time before the next test attempt after a test failure, though

Ah, actually, that article has a solution: if (testInfo.retry) So you could have a waitForTimeout(60 * 5 * 1000) or so inside the testInfo.retry conditional block

Ooh thank you! I'll check that out and let you know if it works or not.

I know I can detect if the test is a retry or not, but can I determine which retry it is?

https://playwright.dev/docs/next/api/class-testinfo#test-info-retry

Specifies the retry number when the test is retried after a failure. The first test run has testInfo.retry equal to zero, the first retry has it equal to one, and so on. Learn more about retries. that's for testInfo.retry

Awesome

I'm getting an error that says "Playwright Test did not expect test.describe.configure() to be called here." What I'm trying to do is set up a function in a separate file that I can call in the beforeEach section of all of my test files, so I don't have to copypaste it all over the place. Is that not possible?

Current code:

That's the beforeEach section, right? I don't think that's possible, I think it's supposed to be something like this:

test.beforeEach(...) {
  ...
}

test.describe( () => {
  // All tests in this describe group will get 2 retry attempts.
  test.describe.configure({ retries: 2 });

  test('test 1', async ({ page }) => { ... }
  test('test 2', async ({ page }) => { ... }
})

Where I think you'd be importing the individual tests (e.g. 'test 1') from another file

The beforeEach is inside the describe group though.

Oh, right. In that case, something like this(?):

test.describe(() => {
  // All tests in this describe group will get 2 retry attempts.
  test.beforeEach(...) {
    ...
  }
  test.describe.configure({ retries: 2 });
  ...

Sorry, I didn't read what I wrote properly, fixed it 😅

No, wait. Argh.

Third time's the charm with these edits, I hope

So it has to be called outside the beforeEach???

I think so, since it's a configuration - I think it's similar to if you ran the tests from the command line with the --retries=2 flag

Hmm... I might still be able to make this work if it's a variable..

Or wait, can it be a variable in the first place?

If you are saying that you absolutely only want to retry if there's a problem with the network, I'm not sure I can help - what I've written will retry if there is any problem at all 😅

How do I make a code block again?

Three `

Do you think I could do something like

const numRetries = 2;
test.describe.configure({ retries: numRetries });

I think so

Alright I'll try that. I was worried about the JSON compatibility

Hmm. The variable is fine, but it seems to be determining the value before the beforeEach function runs instead of after.

I wasn't sure if it would do that, but it does make sense

One (probably bad) idea you could do is add some kind of special handling for when the tests return error and then see if the error... code(?) or message(?) is the one for when there's a network problem

Or if it's ok that you don't get error when there's a network error, you could loop that "element detector" a few times with increasing timeouts

I don't know that I can even tell if there's a network problem though. I have plenty of ways to tell that the page is blank, so that's what I'm hoping I can use

One of my coworkers suggested a long timeout. I was hoping I wouldn't have to use it, but...

There are also ways in Playwright to detect specific requests, so if you can identify a request that tells you that you've got the page, you could try that

If you do go the timeout route, I suggest something like this:

for (let i = 0; i < 10; i++) {
  //code
  page.waitForTimeout(400 * 60 * i);
}```
So that the timeout starts small and increases with every loop

The thing about requests I'm talking about is this: https://playwright.dev/docs/next/api/class-browsercontext#browser-context-route I've never used it in actual tests, mind, but...

Where does the timeout variable go in page.waitForSelector()?

page.waitForSelector({timeout: 1000})

So it would be page.waitForSelector("element", {timeout: 1000})"?

I think so, yeah

Great, thanks