Rayrun
← Back to Discord Forum

Playwright Tests Timeout in GitHub Actions but it works locally

Hey everyone, I'm facing an issue with our Playwright tests when running them in GitHub Actions. Here are some details about the problem:

  • Locally, all our Playwright tests pass without any issues.
  • However, when we run the same tests in GitHub Actions, they consistently exceed the timeout limit.
Screenshot_2023-10-27_at_19.15.05.png
Screenshot_2023-10-27_at_19.17.40.png

This thread is trying to answer question "Why are Playwright tests timing out in GitHub Actions but not when run locally, and how can this issue be resolved?"

21 replies
dirvinautomation_16636

Do you have any artifacts to share that could point us to the reason? I'm not seeing any issue based on the screenshots you've provided.

your test seem to fail pretty early when its trying to reach the url did you try to increase the timeout beween each action ? i had the same problems back in the days and some timeout increased fixed it

Hi, I'd collect traces/screenshots for these tests and upload them as artifacts to github. I'm pretty sure it'll help you to understand the root cause of the issue.

@zwonkoh: when i have the same timeout,It passes locally but not in github actions

when i have the same timeout,It passes locally but not in github actions

i had the problem that my tests passed 90% locally and when i let them run on a build server it failed after 20sec

it depends on the performance of the machine so you have to make the tests so stable that it runs on every other machine without problems

thats why you should try to make the timouts very high, jsut to test if it works then

lucky_rabbit_44106

Can u check you the trace file for any differences in UI like a popup of something when running in Github. It's likely that or maybe something related to URL not being accessible etc.

@zwonkoh: should I increase the timeout in the configuration file of playwright or github action. export default defineConfig({ timeout:70000, testDir: './tests', /* Run tests in files in parallel */ fullyParallel: true, /* Fail the build on CI if you accidentally left test.only in the source code. */ forbidOnly: !!process.env.CI, /* Retry on CI only */ retries: process.env.CI ? 2 : 0, /* Opt out of parallel tests on CI. */ workers: process.env.CI ? 1 : undefined, /* Reporter to use. See https://playwright.dev/docs/test-reporters */ reporter: 'html', /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */ use: { /* Base URL to use in actions like `await page.goto('/')`. */ // baseURL: 'http://127.0.0.1:3000', /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */ trace: 'retain-on-failure', screenshot:'only-on-failure', },

should I increase the timeout in the configuration file of playwright or github action. export default defineConfig({ timeout:70000,

testDir: './tests', /* Run tests in files in parallel / fullyParallel: true, / Fail the build on CI if you accidentally left test.only in the source code. / forbidOnly: !!process.env.CI, / Retry on CI only / retries: process.env.CI ? 2 : 0, / Opt out of parallel tests on CI. / workers: process.env.CI ? 1 : undefined, / Reporter to use. See https://playwright.dev/docs/test-reporters / reporter: 'html', / Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. / use: { / Base URL to use in actions like await page.goto('/'). */ // baseURL: 'http://127.0.0.1:3000',

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'retain-on-failure',
screenshot:'only-on-failure',

},

i did it in playwright but i see you already have 7s right ?

@zwonkoh: yes

yes

Github actions runners are slow and low memory. It causes problems, especially if you run tests parallel.

If you have company, enterprise account you can join beta program and request better runners. We are using ubuntu-4c-16gb-150ssd

If no - increasing timeout and not running in parallel are your best options

I have increased the timeout ,but I'm still getting the same timeout error .

Yes. Did you try not to run in parallel ? How much you increased timeout? Your app(website) is being run in container or you have external instance?

I setted fullyparallel as false and increased the timeout to 120000

OK, so I guess something wrong with Github -> app connection. As others said - screenshots, videos, artifacts and debug info

Also I assume ou run them locally also in ubuntu ?

You can also try to run Github action locally: https://github.com/nektos/act

Good luck!

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.