Playwright Test Fail in CI But Pass Locally

Sometimes machines in CI are límited to run with 8 workers. Should works fine with only one worker

Was seeing the same if not more errors when it was running with only 1 worker on the CI pipeline, when I increased it to 8 it showed less errors but still erroring 30000ms timeout as Page closed

Generally CI machines a usually minimal configs, running more then 1 or 2 may be problematic, dictated more by the cores on the box or total thread, 2 core iwth 2 thread give 4 possible, and one would be max. Slog through the docs PW will suggest no more then 25% of your possible threads should be used... One might get away with using 50% or 75% if you can get away with it, but not likely. In the 1.39 release video or was it the PW Testing Services in videos they kind of touch on and showed running with toooooooo many workers. Start with 25% as per the convention then step up.

With 8 workers are the errors shown in the unit tests?

Too you can try without video and trace do you have some repo to check?

Hi, the others are right that more workers makes it more tricky. Each worker will be more resource constrained.

Don't know why it seemed different for you in practice.

Anyway if tests fail in CI, using the trace files and trace viewer are the best way to debug: https://playwright.dev/docs/trace-viewer-intro

I prefer setting trace to 'retain-on-failure', but if you have retries configured, you can choose 'on-first-retry'.

That is interesting, so i tried lowering to 1 worker again but i still see the same thing, all unit tests pass and the e2e tests fail saying Test timeout of 30000ms exceeded. Error: locator.innerHTML: Page closed Error: locator.textContent: Page closed etc....

Yes unit tests that i created always pass. The 2 e2e tests which i built for implementing e2e tests the first two being for authentication are failing on CI but not on local

I will try the trace viewer today thank you

I noticed if i set the globalTimeout to 0 and the timeout to 120000 it helps make some of the e2e tests pass

Got that idea from a github issue where a bunch of users have been having this issue for almost 2 years surprised that playwright hasnt patched this

https://github.com/microsoft/playwright/issues/13090

I think you can split your test unit tests don't require a lot of CPU. For e2e tests you need to run with only one worker and check the trace and/or video.

Simply standing up a machine and running tests on it, really need to ensure the machine has enough memory, cpus and what ever is needed. Especially if the only problem you can can say at this point is "Works on my machine but not in CI" Would be exactly the issue if there isn't enough memory or cpu... Even better on your local machine, pop out a memory chip or to or find a way to get you machine to mirror the CI machine.

Oh ya, most often the only time i see the click() failed with "Traget Closed" is something failed adn the browser itself is gone...

I tried that and all the unit tests remain passing but the e2e tests didn't. So i tried to change the timeout to 0 as well since it was still timing out after 120000 and then i saw that 1 part on e2e passed(setup project which helps setup the authentication). However, none of the other e2e tests were even listed which is weird. The CI ran for 2 hours and nothing happened to the other 2 e2e tests

Once i changed the timout to 0, the errors dissapeared. So i don't think it was a Target Closed issue but the browser closed because the time ran out. However, now the tests just don't run on CI. P.S. I have changed to 1 worker as you advised on both local and CI

What is your last error? timeout is more related with a lot of workers you can check in the videos or trace as attachment or you can try the trial of microsoft playwright test service that can run up to 50 workers

@apis3445 Sorry for late response, got put on more urgent situation. Now finally back to working on backlog stuff.

For 2 hours it just remains here with no errors, 2 hours being the max timeout for the pipeline at which point it crashes

Your timeout is 2 hours?

Might be me, seeing attempts to fix the issue there is little understanding of what is actually happening, at best you mention at one point you were going to look at the traces... After that only comments about looking at timeout values.

@dand33 Sorry forgot to mention that here, i couldn't get traces viewed on ci. I need to get CI or playwright to not exit the CI on test fail so it can run the npx playwright show-trace xxxfile as the following step i also doubt that will work cause how do an hmtl page load on CI 😕

I set that manually to try to get rid of the 30s timeout error. But even if i set it to the max, the CI pipeline that is hosted has a max timeout of 2hrs so it just crashes after 2hrs of waiting to get the button to click

However, i've since removed that and it is throwing the 30s timeout error again

You shouldn't be trying to view the trace within CI.

CI should run your tests and produce a trace file (.zip). This should be stored as a pipeline artifact. You can then download the trace file and view it using the trace viewer.

Thank you, looks like there might be an issue with getting playwright working with Svelte

After viewing the trace i realized it was failing on CI on a import path so googled it and found this issue thats open atm https://github.com/microsoft/playwright/issues/19411