Tests fail on CI but pass on local
Hi all, looking for help on a few issues and hoping to solve this in the right way(if one exists). Im using javascript/TypeScript.
- When tests are run locally they pass. However, when i run the same tests are run on CI pipeline (Azure DevOps) some pass and some fail.
- Upon Setting trace: "retain-on-failure". i can get the trace on local and disect the issue. However, im unable to open the trace file which is generated from the pipeline.
- Wanting the JUnit results to have the trace in it as an attachment?
I have attached what my pipeline looks like on DevOps and my config file on VSCode.
Thanking all playwrightians in advance. 🙂
This thread is trying to answer question "Why do tests pass locally but fail on the CI pipeline, and how can the trace file generated from the pipeline be opened and attached to the JUnit results?"
Hi, Azure DevOps does not support JUnit attachments. (Not sure if Playwright even puts references to the traces in the JUnit output, but it does not matter.)
You need a separate step for uploading the zip files with the traces somewhere. I suppose the easiest way to get started is to use the publish artifacts task for that.
I believe others are publishing the HTML report + trace files to GitHub Pages. That would be a more convenient way to access them.
We are publishing them to Azure blob storage, but I cannot share the code.
I don't know if Azure DevOps has the equivalent of GitHub Actions Artefacts to generate and give access to thing along a build.
However I can recommend you to use Argos + Playwright (https://argos-ci.com/playwright). It uploads failures screenshots and traces and give you instant access to it when you run test on CI. We made it for this specific use case!
Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.