Rayrun
← Back to Discord Forum

How to add test results as artifacts in Bitbucket CI pipeline

haridas_67812posted in #help-playwright
Open in Discord
haridas_67812

Hello everyone,

I need some guidance on how to add test-result directory as an artifact in bitbucket CI pipeline. I'm using below yml file to configure the pipeline

image: mcr.microsoft.com/playwright:v1.38.0-jammy

pipelines:
  pull-requests:
    '**':
      - step:
          name: Build and Test
          caches:
            - node
          script:
            - npm ci # Install dependencies
            - npx playwright install --with-deps
            - npx playwright test # Run your Playwright tests

      - step:
          name: Store test results.
          script:
            - ls .
          artifacts:
            - test-results/*
            - 'test-results/*'

Also attached is my playwright.config.js file content.

module.exports = defineConfig({
  testDir: "./tests",
  /* Run tests in files in parallel */
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [['junit', { outputFile: process.env.CI ? 
    `${process.env.BITBUCKET_CLONE_DIR}/test-results/results.xml` : 'test-results/results.xml' }]],
  use: {
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'Draw Tests',
      use: { ...devices["Desktop Chrome"] },
      //testMatch : 'test data/draw',
      retries: 2,
      testDir: 'src/tests/draw',
    },
  ],
});

I tried several combinations of the reporters (html, xml, junit etc), outputFile and different debug options to check for the test-results directory on the CI-runner. ls . and ls -R ../.. commands just shows the repository and few other directories but not the test-results directory. I'm not able to find the test-result not sure if this is generated or not. How I can ensure test-results directory gets generated and attached as an artifact.

Thanks in advance.

Haridas.

This thread is trying to answer question "How can I ensure a test-results directory gets generated and attached as an artifact in a Bitbucket CI pipeline?"

2 replies

Bitbucket only creates artifacts when a pipeline step runs successfully. Obviously you’re interested in the artifacts when playwright fails, so the only option is adding || true to the bitbucket pipeline (not behind my pc atm so cant check exactly where) in order to ALWAYS pass the pipeline. Side effect is that each pipeline will look successful, but could actually be failing. Not ideal…

haridas_67812

okay. I was expecting test-results directory to be present at the current working directory on the CI but it's not there. Do you know, how to let CI runner to create the directory and results into this directory? Do I need any other settings/configurations?

Related Discord Threads

TwitterGitHubLinkedIn
AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.