Multiple reporters only running 1 of the 2 listed

Based on silly mistakes I've made myself is ./reporter-file.ts definitely the right path relative to the config file?

Scratch that, my mistake for commenting before finishing reading your message 🤦‍♂️

If:

['html'],
    ['./reporter-file.ts']
  ],

isn't working that sounds like a bug to me and might be worth raising an issue on the PW Test github repo, if it's not already an issue that's been raised! https://github.com/microsoft/playwright/issues?q=is%3Aissue+is%3Aopen

Thanks! I was wondering if that may be the case or if it was something silly on my end. Thank you for taking a look!

I just noticed we are on playwright/test version 1.32.3 and 1.39 is the newest. When I updated it, it broke a bunch of files and won't run our e2e tests. Once I get that working, I'll post again with the results of the multiple reporter issue in case anyone else has the same issue. I'll post a link to a bug ticket if I have to open one too.

Nice one, always great to see other people caring about contributing 🙂

@cod3r_54237 Can you guide me how to configure Grafana/Hosted Graphite dashboard

@cod3r_54237 Yep, would appreciate for some help in implementing the custom reporter to grafana

@samsuthen @ondra123. I'm sorry I missed your messages!

I followed the doc here https://playwright.dev/docs/test-reporters#custom-reporters Using the HTML reporter and custom reporter. Keep in mind, when you send the metrics, they MUST have \n after each metric so graphite can separate multiple metrics (even if you're only sending 1 at a time)

import net from 'net';
class CustomReporter implements Reporter {
  private socket: net.Socket;
  private graphiteHost = ‘your graphite URL goes here';
  private graphitePort = 'port number goes here';
  private metricList: string[] = [];
  constructor() {
    this.socket = net.createConnection(this.graphitePort, this.graphiteHost);
  }
  onTestEnd(test: TestCase, result: TestResult) {
    const metricName = `hosted-graphite-key-goes-here.playwright.${test.title}.${result.status}`;
    const metricValue = 1;
    const timestamp = Math.floor(Date.now() / 1000);
    this.metricList.push(`${metricName} ${metricValue} ${timestamp}`);
  }
  onEnd() {
    this.metricList.forEach((metric) => {
      this.sendMetric(metric);
    });
    // Close the connection
    this.socket.end();
  }
  private sendMetric(metric: string) {
    this.socket.write(`${metric}\n`);
  }
}
export default CustomReporter;```

Update: Updated from 1.32.3 to 1.40.1 and it's having a slightly different issue. With 2 reporters listed, it ignores the html reporter and always runs my custom reporter for HG metrics. Does not show the HTML results no matter what order I list them. Bug reported here: https://github.com/microsoft/playwright/issues/28718

Option 2 here worked for me: https://github.com/microsoft/playwright/issues/18945#issuecomment-1322734486

Looking at your original post this looks wrong... reporter: [ ['html', './reporter-file.ts'] ],

Shouldn't it be an array of arrays of reported as per the docs?

reporter: [ ['list'], ['json', { outputFile: 'test-results.json' }] ],

Could be an issue but might be best to follow what is in the docs?

Coming back to this, @dand33 you are correct, it should have been an array. I forgot to update what I did:

const config: PlaywrightTestConfig = {
  reporter: process.env.CI ? [['html'], ['./graphite-reporter.ts']] : 'html', // If CI runs, html and graphite-reporter execute, otherwise html locally
};

If it working great, but the one thing that might be problematic if your reporter used take in arguments, which the format for the reporter are in an object. I.E.

reporter : [ [ 'reporterToUse1', { json object for reporter1 arguments } ], [ 'reporterToUse1', { json object for reporter2 arguments } ], ] // close reporter array...

And the json object might be what ever that reporter wants...