Why Playwright with TS/Python/Java has different features?

Most of the differences comes from Playwright Test, the test runner that is available only in JS/TS

Hi @jfgreffier , sorry for necroing this post, but can you explain further about the test runner available only in JS/TS?

For context, I have researched on the JS/TS version of playwright and currently looking into the python version of playwright to get a good picture for my team. I am seeing some pretty glaring disparity between the TS/JS and Python playwright.

For example, in the TS/JS playwright, the playwright.config.ts makes configuration so much easier, but in python, we have to programmatically structure in the conftest.py to get screenshots and trace files for failing tests.

Another example is the sync/async nature for our test execution. In TS/JS playwright, it is standard to use async functions with promises and whatnots. That makes sense. However, according to my understandings, Python is more synchronous by default (There are async functionalities with additional packages). Hence, in the python playwright documentation, there is both async and sync examples. The biggest problem I have is that pytest testing framework recommends fixtures and the yield syntax, which is very powerful and great to dow setup and teardown on many levels. But for someone who is trying to do sync test executions, we cannot utilize fixtures the way that pytest recommends because we would getting an error from Playwright stating that we are trying to make sync calls in async loop. (Further investigation indicates that the yield syntax is similar to awaits but is a python generator for functions to be used later on; hence, we were getting the error). Initial workaround to this problem seemed to be just use return statement, but we would loose the placeholder functionality that yield provides to executing teardowns.

Overall, it just seems like Python playwright is the forgotten child that needs some more love and ideally, the different languages testing frameworks should have parity functions all across.

I understand that limitations and priorities dictate the current state of things, and I don't mind async functions, but it would help to me and others to making better decisions if we can get some sort of public roadmap.

@clee350 I think it's because the ecosystem Microsoft builds: VSCode - Microsoft GitHub - Microsoft Azure - Microsoft Playwright - Microsoft Typescript = ?

@maxsudik I guess. But if we use the standard Agile dev structure (PM, devs and etc). It may seem to be that the Java and Python versions are too much for the team to handle at this point. Tbh, from a maintainability perspective I would have gotten rid of both java and python version as I would need to hire experts of 2 other languages that have their limitations due to how things are setup that are dictated by others, such as pytest.

that's a question to Playwright team

I think the Playwright browser automation library is pretty much on feature parity between the different ecosystems. You have reliable automation with auto-waiting actions, auto-waiting assertions, codegen/recording, trace files. That's pretty decent already. Better than what you get from Selenium or Cypress (which only is available in the NodeJS ecosystem).

For most ecosystems, integration is offered with the most popular test runner(s). Yes, for NodeJS/TypeScript the situation is even more luxurious, with a dedicated @playwright/test runner, html reporter, ui mode, vscode extension. But calling TypeScript 'the forgotten child' is a bit an exaggeration in my view.

If you are fine with NodeJS/TypeScript and want all the bells and whistles, then go with NodeJS/TypeScript. If your team is much more at home in Python, and hates JS/TS, then go with Python, accepting that you will miss a few bells and whistles.

I think the one of the main differences across languages , PW started off as a node.js, more as a fork from puppeteer, last i saw puppeteer is only JS. So those using PW who are not JS, should be thankful PW includes and covers it... I don't wander out of my area much but what else do people use...? Also thought i remember reading, but my mind is going, effectively as PW is natively JS/TS and will always get the best support. Then again no reason someone couldn't build a VS Extension for a given language and and associated test runner... In the end, would suspect the PW team is rather small, they need to prioritize what/when to include and make the biggest impact for what stuff to enable and support, plus bug fixes and maintenance to ensure current support for the browsers and OS... Also can't see them as a huge revenue stream... So the PW has the joy of looking at what makes the most sense to cover the most people. Myself, would just prefer C#, but for tests, JS seems easy and quick enough...

I'd be curious what other frameworks out that which work reliably exist and what languages do they support? Would suspect JS/TS and most things are testing web site and JS/TS are more the language of the web...

@maxsudik TypeScript is Microsoft too...

Team is just 5,6 people, you cannot have all things right away unfortunately.

@dand33 I think you meant Playwright Python in your second statement.

That is true. That people should be thankful and that I do understand that the team is limited on the time and priorities, manpower and etc. AKA the iron triangle. However, it is a decision that they made for adding support for any other languages. By the act doing the providing the language support, they are signed up to supporting the tooling like any other codebase that are being used for development and testing. The expectation is still there. Hence, like I said, it would be great to see a roadmap to make it easier for folks to decide which to go with. Marketing could use some work.

If I am the project manager of Playwright, I would have gutted out the python and the java version of PW and be singularly focus on more on providing the best and take more of Cypress's user bases. I am not biased on using JS since I came from a Java/C# and JS background with some experience using Python.

Just look at the current trend... https://madnight.github.io/githut/#/pull_requests/2023/2

@bandito9274 I just clicked on the link and it seems to be the overall general trend on the languages. Not really Playwright specific. It could that people are using python for something else for web or for AL/ML (Not sure if Python is still popular tooling for AI/ML). It doesn't show correlation of people using Playwright via python versus the other languages.

Out of curiosity, I went ahead and took a look at the stats. Here are the two links from PyPl and npm stats specifically for Playwright: https://pypistats.org/packages/playwright https://npm-stat.com/charts.html?package=playwright&from=2020-12-01&to=2023-09-27

From the graphs, it seems like Playwright is immensely more popular on JS rather than on Python by a couple of magnitudes. by 4x (just looking from last month's numbers)

Do note, these are statistics are for downloads which may not be a good indications of actual unique usages.

I think having Playwright available for major languages other than JS/TS is a great thing, and a big differentiator to Cypress for example. As said before, automation is pretty much on-par whatever the language you use. The team is indeed small, so maybe the community can help for Python or Java ?

If you look at playwright-java repo, something is on the way 👀

@clee350 to be specific Playwright Test comes with UI mode, configuration file, easy parallelism, codegen for tests, image snapshot, VSCode extension... That's what I meant by saying that the main differences between python and JS/TS versions come from Playwright Test.

Do you want to create issues for what is missing to Playwright Python ? Or upvote existing ones ? Let's make it even simplier : what is the main feature that is missing to Playwright Python ?

Interesting:

(https://npm-stat.com/charts.html?package=playwright&from=2023-08-01&to=2023-09-01) Total number of downloads between 2023-08-01 and 2023-09-01: package downloads playwright 6,955,149

PyPI for Playwright: Downloads last month: 1,482,055

Might suggest only one month may not be a good window, wouldn't get people like me unless we averaged out over a few months... Simply because we run for a while with a given version and then decide after 3 or 4 months to upgrade PW.

I much prefer JS/TS over Python for a few reasons, and if one needed to give up the advantages JS had i'd be using C# for the benefits it provides. Python have LINQ?

So I just spent the past 2.5 weeks writing my team's first Playwright tests in Python, and this has led me to a couple of observations relevant to this thread:

First, the Python API is better developed than the Python documentation. There are some things that are just not well-documented on the Python site, or where the documentation is misleading.

Second, there's no rule that you have to write your Playwright tests in the same language that your underlying code is in, since PW is driving a browser automation rather than interacting with your function calls as say pytest would do. So we have a Python/Django backend and a React frontend, I write PW tests in Python because I work on backend and I'm more comfortable in Python than JS, but if there were features I wanted that only existed in JS, then I'd just write tests in JS.

Great point(s), at the end of the day PW is still just a tool to get something done. Do what best fit your needs... I need coffee, and discord don't serve that, have a great day...

From the graphs, it seems like Playwright is immensely more popular on JS rather than on Python It depends how/what for do you use Playwright... I'm coding a lot of backend class, function and method for my specifications... I'm call it constructors. I do not know how easy would it be to use JS for coding the same "supportive" codes... but with Python... it's super easy. That's why I'm a little bit sad, when a feature only released just for JS and not for Python

Hi @jfgreffier, Sorry for the late reply. Just recovering from being sick mid-week till now. Over the weekend, I reflected some more and perhaps, the documentation was where that I felt that I had to spend quite a bit of time in Python vs. in JS/TS. The main functionalities are definitely there in the python version as well as the JS version, but it was a trial and error to get them be working.

I would like to submit recommendations. For example, Rather than just having people attempt to read the entire Pytest fixture documentation. I think we can bridge the mental mapping by put up a page for conftest.py file and how to set it up as an example. Another one, which I am not able to currently solve, is the reusing the state for okta login. My team would love it if we can show off the power of reusing state. Finally, It would be nice if we can post a test execution flow as a visual diagram. Here is an example of what I did for showing the testing execution flow using playwright.

@bandito9274 Backend as in API calls. Maybe stuff like mocks and stubbing?