Rayrun
← Back to Discord Forum

Running Playwright tests using Azure Test Plans

  1. I've been running Playwright tests that are linked to Azure Test Plans UI using a third-party library. When I run these tests locally, the status of the test-case updates automatically in the Azure Test Plans UI.
  2. However, I'm encountering issues when trying to run these tests in a release pipeline on Azure DevOps. The tests are failing and causing errors.
  3. According to the logs, the test run is being aborted, which is causing the task to fail. The error message I'm seeing is "System.Exception: The test run was aborted, failing the task."
  4. Interestingly, my build pipeline completes successfully and all tests pass. This leads me to believe that the issue might be with the release pipeline or its interaction with Azure Test Plans.
  5. I've shared my playwright.config.ts and azure-pipelines-pw.yml files for reference. These files contain the configurations for the Playwright tests and the Azure Pipelines build respectively.

For more details, you can check out my original post: https://stackoverflow.com/questions/76396854/is-there-any-way-to-run-a-test-using-the-azure-test-plans-if-the-test-runner-is

This thread is trying to answer question "Why are Playwright tests that are linked to Azure Test Plans UI failing when run in a release pipeline on Azure DevOps, and how can this issue be resolved?"

8 replies

Hi, were you able to figure this out? 🙂

@ritavdas if you are still looking for an answer, please note that I just answered this in the stack overflow.

Thanks man !! , I will try that out

ntodorov99

@guru_s , @ritavdas hey guys, what task could replicate the VS Test task?

ntodorov99

I think in the release pipiline we should run the tests with npm task, but will that create a test run and update the tests outcome in Test Plans.

@ntodorov99 as you already know, with Playwright we have to use https://www.npmjs.com/package/@alex_neo/playwright-azure-reporter to update the test case status.. so we no need to use VS Test tasks as its not meant for this framework.

Yes, this is true, it just that we loose the capability to start testcases directlly from Test Plan

@ntodorov99 just in case if you are trying to achieve this "start test case execution from test plan in Azure" and got succeeded , please do share with me.. thank you

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.