Test passes while debugging but fails during normal execution

maybe adding await page.pause(); in between your test instructions can help ?

I believe Debug mode is different. First it has no timeout. Secondly, it will wait until an element is presented, while actual execution might not

I could do that...see what's happening right before the button is pushed.

that's what I was thinking (that the runner may not be waiting). But what's weird is that the save just looks like it is occurring. I even get a message saying the record has been updated. But the database doesn't change. It's baffling.

For mine one, I use traceview to see where it fail, and add page timeout to do a quick test to see if it's the issue:

in config file use: { trace: 'retain-on-failure', },

Sometimes too the save from frontend, takes a while to finish. We click 'Save'/'Create' button, but the request is still firing. Then the test end close the browser and interrupt the creation process. I have had to wait for specific UI element like the spinner to appear and dissapear

ok, I'll give this a whirl tomorrow. Thank you.

I've examined the trace carefully, found one more layer of cache that I had thought was already off and turned it off and still the data was not being saved.

The only thing that worked was to add a sloMo of 1000. With that, the test runs just fine outside of debug mode.

Unfortunately, it doesn't seem to be possible to slow down just a section of code. There are a few issues asking for it but it doesn't seem to have been implemented yet.

Not sure what to try next.

@andre_at_performant_labs I know it’s not ideal to add timeouts but if you don’t want to slow down all tests you can try adding time outs only for that test.

Yes, that's a good idea, even if it's suboptimal. Better than adding a sloMo of 1000.

Hey André, have you found a better solution for this? I got the same problem, that the tests are passing in UI and debug mode, but not during normal execution.

Unfortunately I did not. I had to add timeouts.

sad :/

but thanks!

I've fixed my problem. I put the "await" keyword at the wrong position. Maybe it will help you as well. Wrong: expect( await page.getByText(testEntryInternaText)).not.toBeVisible()

Right: await expect( page.getByText(testEntryInternaText)).not.toBeVisible()

I little tip - the await word has to be where the used method is:

example 1: await expect(page.locator('')).toBeVIsible() - here the await is next to be expect because it is waiting for the toBeVisible()

example 2: expect(await page.locator('').textContent()).toBe("something") - here the await is with textContent() because it is waiting for the locator and the textContent().

This is how I use the awaits, might be wrong and sorry if I misslead somebody.

Oh, good.

When I look at your video, one things come out: toContain is not a web assertion and will assert immediately instead of awaiting for it to reach the desired state. Would explain your debug mode passing but not when you execute normally

So I highly recommend you to leverage web assertions as they work best for this kind of flakiness you're experiencing (will wait for the desired state a certain amount of time). Try toContainText or toHaveText instead, with the await. VS code actually pointed this out as underlining the unnecessary await with toContain.

https://playwright.dev/docs/api/class-playwrightassertions

There's toBeHidden as well, FYI 🙂