The Quality Assurance (QA) field witnesses constant flux, with tools and frameworks vying for attention. Amid this vast selection, @playwright/test carves its niche. While established tools like Cypress and Selenium continue to anchor QA processes, @playwright/test gains momentum.
This article dissects this trend, contrasting @playwright/test adoption rates with those of Cypress and Selenium. By examining statistics, community engagement, and industry sentiment, we unlock reasons behind @playwright/test's swift rise among QA professionals.
Quantitative Adoption Metrics
A compelling way to understand the adoption rate of @playwright/test compared to Cypress and Selenium is to look at concrete numbers. The GitHub stars and NPM download rates offer clear evidence of this growing trend.
GitHub Stars
The number of GitHub stars can indicate a project's popularity and community support:
- Selenium took 8 years to reach 20,000 stars.
- Cypress took 4 years to achieve the same milestone.
- Remarkably, Playwright took just 1 year to amass 20,000 stars.
As of today, the numbers stand as follows:
- Playwright commands an impressive 52,645 stars.
- Cypress, while still popular, trails with 42,760 stars.
- Selenium, despite its earlier dominance, lags behind with 26,391 stars.
It's worth noting that Playwright's GitHub stars have been growing at a faster rate than Cypress and Selenium. This is a strong indicator of Playwright's growing popularity and the increasing interest it garners from QA engineers worldwide.
NPM Downloads
NPM downloads further substantiate the adoption rates of these tools:
- Selenium required 6 years to hit 2 million weekly downloads.
- Cypress achieved this in half the time, at 3 years.
- Astoundingly, @playwright/test reached 2 million weekly downloads in just 1 year.
As of writing this article, the download statistics present an interesting picture:
- Selenium has 2.2 million weekly downloads, a number which is on the decline from its peak of 3 million weekly downloads in 2022.
- Cypress stands strong at 5 million weekly downloads, its highest ever.
- @playwright/test matches Selenium with 2.2 million weekly downloads but with an upward trajectory.
The rapid increase in @playwright/test's GitHub stars and weekly NPM downloads provides compelling evidence of its swiftly growing popularity in the QA community.
GitHub Pull Requests
Another important indicator of a tool's adoption is the number of pull requests it receives. This metric can provide a glimpse into the level of community engagement and the tool's evolution.
- Playwright has been averaging 9 pull requests per day.
- Cypress has been averaging 2 pull requests per day.
- Selenium has been averaging 1 pull request per day.
This data further underscores the growing interest in Playwright, with the tool receiving more than four times the number of pull requests as Cypress and Selenium.
2022 Survey Data
The State of JS survey is a popular annual survey that provides insights into the JavaScript ecosystem. The 2022 survey included a question on end-to-end testing tools, with Playwright, Cypress, and Selenium as the options. The results were as follows:
- Interest in Playwright has increased from 69% to 73% (a 6% increase)
- Interest in Cypress has dropped from 72% to 69% (a 4% drop)
- Usage of Playwright has increased from 9% to 15% (a 67% increase)
- Usage of Cypress has remained at 42%
- Retention of Playwright has increased from 92% to 94% (a 2% increase)
- Retention of Cypress has dropped from 91% to 84% (a 7% drop)
- Awareness of Playwright has increased from 33% to 50% (a 51% increase)
- Awareness of Cypress has decreased from 83% to 82% (a 1% drop)
Each ratio is defined as follows:
- Retention: would use again / (would use again + would not use again)
- Interest: want to learn / (want to learn + not interested)
- Usage: (would use again + would not use again) / total
- Awareness: (total - never heard) / total
As a reminder, this is a survey of JavaScript developers, not QA engineers. The survey results are still useful to understand the general trend of the JavaScript ecosystem.
Shoutout to Jean-François Greffier for sharing this data.
Other GitHub Metrics
Other noteworthy metrics include:
| Metric | Playwright | Cypress | Selenium |
|---|---|---|---|
| Releases | 106 | 309 | 83 |
| Open Issues | 638 | 1.3k | 163 |
| Open PRs | 15 | 47 | 37 |
| Contributors | 434 | 443 | 689 |
| Wathcers | 399 | 626 | 1k+ |
| Forks | 2.9k | 2.9k | 7.6k |
Security Considerations
In an era where data breaches and security threats are rampant, the security of the tools we employ is of paramount importance. The Snyk advisor score, which calculates the health of npm packages based on historical variables, is an effective way to compare the security of Playwright, Cypress, and Selenium.
- Playwright stands out with a perfect score of 100/100, indicating top-notch health and security.
- Cypress, while still having a high score, lags slightly behind at 89/100. A vulnerability in a dependency that remained unaddressed for several years seems to have affected its score.
- Selenium also performs well, with a score of 98/100.
It's worth noting that these scores are influenced by factors such as the time taken to fix a vulnerability. The exceptional score of Playwright demonstrates its commitment to security, not having been impacted by any security alerts so far. This makes it an even more appealing choice for QA engineers conscious about security.
Community Engagement
In today's interconnected world, social media presence and community engagement often serve as measures of a tool's popularity and influence. Analyzing Twitter followers and Discord community size provides another lens through which to view the adoption of Playwright, Cypress, and Selenium.
Twitter Followers
Despite being the newest tool among the three, Playwright has quickly garnered a sizeable following:
- Playwright, despite only joining Twitter in May 2020, has already accumulated 10,000 followers.
- Cypress, which joined Twitter in May 2014, has a slightly larger following of 21,000.
- Selenium, the oldest of the three, having joined Twitter in May 2009, leads with 23,000 followers.
Although newer to the scene, the rate at which Playwright has amassed Twitter followers is indicative of its growing influence and popularity in the QA community.
Discord Community
Discord serves as a hub for real-time interaction, knowledge sharing, and community building. Playwright's Discord community has shown impressive growth:
- Playwright's Discord boasts 5,050 members, with 600 online at the time of writing this article.
- Cypress Discord has 9,950 members, with 567 online at the time of writing this article.
- Selenium appears not to have a Discord community.
The strong community engagement on Discord underscores Playwright's rise in popularity and the active interest it garners from QA professionals worldwide. This social proof further underlines why Playwright is fast becoming a favored tool in the software QA landscape.
Stack Overflow
Stack Overflow is a popular platform for developers to ask and answer questions. The number of questions asked and answered on Stack Overflow can provide a glimpse into the popularity of a tool.
- Playwright has 2,169 questions asked and 1,043 questions answered.
- Cypress has 9,609 questions asked and 5,780 questions answered.
- Selenium has 155,408 questions asked and 91,564 questions answered.
While Playwright's numbers are lower than those of Cypress and Selenium, it's worth noting that Playwright is the newest tool among the three. The fact that it has already amassed over 2,000 questions asked and 1,000 questions answered on Stack Overflow is a testament to its growing popularity. Plus, with the advent of ChatGPT, we can expect more of the questions to be answered by AI in the future (check Ask AI if you have not already).
Geographic Distribution
The geographic distribution of a tool's user base can provide further insights into its adoption. The following map shows the geographic distribution of Playwright users based on GitHub issues:
We can see that US users account for 22% users, following by 8% in Germany, 7% in India, 6% in the UK, and 5% in Canada. The distribution of Cypress and Selenium users is similar, with the US, Germany, India, and the UK being the top four countries.
Notable Companies Using Playwright
To provide further insight into the real-world application of Playwright, several globally recognized organizations and institutions have adopted this tool for their testing needs.
- WordPress: WordPress has recently migrated their end-to-end tests to Playwright. As a prominent open-source content management system, this transition serves as a testament to Playwright's flexibility and robust testing capabilities.
- Adobe: Adobe, known for its suite of creative and multimedia software, uses Playwright for its spectrum web components. By doing so, Adobe ensures the seamless functionality and integration of these web components across their offerings.
- ING Bank: ING Bank uses Playwright to test their web components. Ensuring a seamless online banking experience for their customers, Playwright helps them maintain a reliable and user-friendly interface.
- Facebook: One of the world's largest social networking platforms, Facebook, uses Playwright in their React JS library.
- NASA: Playwright is used by NASA for end-to-end testing in Open MCT (Open Mission Control Technologies).
- Johnson and Johnson: Johnson and Johnson, a multinational corporation specializing in pharmaceutical and consumer packaged goods, is in the process of migrating to Playwright for end-to-end testing in their BodilessJS toolset.
- VMWare: VMWare, a leader in cloud infrastructure and digital workspace technology, uses Playwright for integration testing of Tanzau Kubeapps.
- Elastic: Elastic, a search company known for their SaaS offerings for search, logging, security, observability, and analytics, uses Playwright for their testing needs.
These case studies underline the wide-ranging and powerful applications of Playwright across various sectors and industries. They demonstrate the trust that many world-leading organizations put in Playwright for their testing requirements, further cementing Playwright's position as a go-to tool for modern end-to-end testing.
Special thanks to Reddit user fellim who contributed these examples.
Reasons for Switching to @playwright/test
With @playwright/test gaining traction rapidly, let's understand the reasons behind this shift. Below, we outline the key factors making @playwright/test the preferred choice for many QA engineers.
Fastest Test Execution
@playwright/test outpaces competitors like Cypress, Selenium, and Puppeteer in test execution time.
Browser Contexts for Isolated Tests
@playwright/test utilizes the concept of browser contexts to run tests in isolation rapidly, ensuring high-performance testing.
Rich and Robust Testing Scenarios
@playwright/test supports testing scenarios for multi-tab, multi-user, multi-origin/domain, and iframes, increasing the tool's flexibility and adaptability.
Comprehensive Test Artifacts
The HTML report in @playwright/test offers a detailed view of test execution results, including visual diffs, traces, error logs, video recordings, and screenshots.
VS Code Extension for Ease of Use
The VS Code extension simplifies test writing and debugging, with features to run tests, debug step by step, explore selectors, and record new tests (codegen).
Built-In Visual Regression Testing
@playwright/test supports visual regression testing through its built-in toMatchScreenshot() method.
Supports Async/Await Syntax
Async test code in @playwright/test uses standard JavaScript async/await syntax, aligning with modern JavaScript practices.
Test Code Reuse and Emulation
@playwright/test provides the ability to re-use signed-in state and offers emulation for mobile devices, user agents, locales & timezones, permissions, geolocation, and dark/light mode.
Intuitive Network Interception
With its network interception feature, @playwright/test enables easy stubbing and mocking of network requests.
Comprehensive Language Support
@playwright/test supports JavaScript, TypeScript (no transpilation required), .NET, Python, Java, and Go (supported by the community).
And many more...
From browser compatibility to Docker support, @playwright/test's extensive feature set, versatility, and modern design have facilitated its rapid adoption among QA engineers. As it continues to evolve, its user base will likely keep expanding.
User Case Studies
Real-life examples and user experiences bring abstract ideas into the tangible realm. Let's explore two compelling case studies that shine a light on the benefits of @playwright/test in practice.
Experience from Michael Lynch
Michael Lynch, an experienced software engineer, shared his positive experience with @playwright/test after transitioning from Cypress. Despite only being a few hours into using Playwright, he found it to be a substantially better experience than Cypress. The API clarity, testing setup simplicity, and speed made him 50-100% more productive in Playwright than in Cypress.
He notes:
"Going forward, I’ll be testing all of my new apps with Playwright. I’ll likely even port some of my old Cypress tests to Playwright for apps where my tests have crept above the five-minute mark."
This powerful testament emphasizes the effectiveness and productivity gains that come with @playwright/test. For those using Cypress, Lynch strongly recommends exploring Playwright, likening the jump from Cypress to Playwright as substantial as from Selenium to Cypress.
Read Experience from Michael Lynch
Migration from Selenium to Playwright by Edwin Takahashi
Edwin Takahashi, another software engineer, also shared his experience of migrating from Selenium to Playwright. He observed a significant improvement in test execution speed.
In his words:
"For this canned example, the best execution time I have seen from a serial 15 run was 700ms for Playwright, but 1400ms for Selenium. I would bet the median difference would be greater than 2x if the tests were run 100 times each."
His experience further underscores the notable speed and performance advantage that @playwright/test has over Selenium, a factor that can significantly impact project timelines and overall productivity.
Read Migration from Selenium to Playwright by Edwin Takahashi
Conclusion
In a world where software quality assurance is critical to an application's success, the importance of a powerful, flexible, and fast test automation framework cannot be overstated. @playwright/test is setting a new standard in the industry, thanks to its extensive and unique features, high performance, and a design that aligns with modern development practices.
Its rapid adoption is testament to the value it brings to QA engineers and teams. It's not just about the speed of test execution or the breadth of testing scenarios. It's also about the intuitive syntax, strong community support, seamless VS Code integration, and a slew of other features that make testing feel less like a chore and more like an integral part of the development process.
Whether you're a fan of Selenium, a Cypress enthusiast, or new to the world of testing, @playwright/test undoubtedly deserves a place in your toolbox. As the tool continues to mature and gain further industry traction, we're excited to see how it will shape the future of end-to-end testing.
Are you ready to take your testing experience to the next level with @playwright/test? Trust me, it's worth a shot. Happy testing!
