Rayrun
← Back to Discord Forum

Horrible time getting Playwright to pass tests in Azure Devops pipeline

As the title, saids, I am having a horrible time getting playwright (and playwrite test) to run in Azure pipeline running Ubuntu (Image: ubuntu-20.04). While my tests run and pass locally with the same exact setup (using webserver, same npm script command) my tests pass on my local. But in a PR with the same code base the tests will fail. I tried adjusting timeouts both in config and webserver, retries to 2, and slomo but to no success. Finally, I just placed skip on all my describes so no tests run... and I still get another error!

Error: Timed out waiting 480000ms from config.webServer.

I don't get this on my local (ubuntu linux) with the same tests having skip! I mean the same exact test run on my local will just skip and finish. I don't get this since it is all Mircosoft, Azure should be Playwright friendly.

At the very least, why can't I skip all the tests on and not have that webserver timeout error? Again, same thing runs on my local with no errors.

This thread is trying to answer question "How can the user get Playwright tests to pass in an Azure Devops pipeline running Ubuntu?"

14 replies

Do you have 480000 timeout in webServer config? It works locally because web server starts without problem but on CI perhaps it doesn't start quite well and Playwright waits for web server to launch and return 200.

You can try adding stdout and stderr to web server config, perhaps it will show web server errors. Otherwise, investigate failures with web server.

@skorp32: yes, I mean timeout in webServer config

yes, I mean timeout in webServer config

ok, thanks. Also... there 6 gigs of memory on the agent with about 3 gigs free memory when I ran free -h. Is that enough for playwright?

Should be enough for a few workers but Playwright is more CPU dependant, there should be 2 logical CPU cores for each worker if I'm not mistaken.

I have workers: CI? 1 : 5,

so just one 1 worker

for in the pipeline

Is 1 worker enough for a test suite in the pipeline? I have about 34 tests

if tests are somewhat fast then it should be enough for now

I set the stdout to pipe and got the below. From docs, stderr alrerady goes to pipe

[WebServer] 
** Angular Live Development Server is listening on localhost:8080, open your browser on http://localhost:8080/ **

[WebServer] 
✔ Compiled successfully.
[WebServer] ✔ Browser application bundle generation complete.
[WebServer] 
Initial Chunk Files | Names   | Raw Size
runtime.js          | runtime | 13.00 kB |

5 unchanged chunks

Build at: 2023-06-15T17:45:37.386Z - Hash: 86736805c660f512 - Time: 4532ms
[WebServer] 
✔ Compiled successfully.
Error: Timed out waiting 480000ms from config.webServer.

Any ideas, please, what is wrong? Again, I don't get this timeout on my local

hm, logs look good. in that case I would suggest to file a bug in playwright github repo

perhaps im missing something

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.