Introducing Auto Playwright: Automating Playwright Tests with AI

Auto Playwright revolutionizes test automation by integrating AI with Playwright, offering a seamless, intuitive way to automate web testing tasks. This open-source tool simplifies test creation with plain-text AI-driven prompts, supporting all browsers compatible with Playwright.
  1. How Auto Playwright Streamlines Your Testing Workflow
    1. A Practical Example
      1. Testing User Registration
        1. Complex Navigation and Data Retrieval
        2. Integrating AI into Daily QA Work
          1. Limitations of Auto Playwright
            1. How Auto Playwright Works
              1. Why Auto Playwright?
                1. Getting Started
                  1. Join the Auto Playwright Community
                    1. Conclusion: A Step Forward in Test Automation
                      1. ZeroStep

                        In the ever-evolving landscape of software development, the need for efficient and effective testing solutions has never been more critical. Today, I am thrilled to introduce Auto Playwright, an open-source project that integrates the power of AI into your testing workflow.

                        How Auto Playwright Streamlines Your Testing Workflow

                        Auto Playwright is designed to run Playwright tests using AI, offering a seamless and intuitive way to automate testing tasks. The setup is straightforward:

                        1. Installation: Begin by installing the auto-playwright dependency using npm:

                          $ npm install auto-playwright -D
                        2. Configuration: Auto Playwright leverages OpenAI's capabilities. Export the API token as follows:

                          $ export OPENAI_API_KEY='sk-..."
                        3. Usage: Import and use the auto function in your tests:

                          import { test, expect } from "@playwright/test";
                          import { auto } from "auto-playwright";

                        A Practical Example

                        Consider this scenario: you want to test a web application's search functionality. With Auto Playwright, you can do this with simple, human-like instructions:

                        test("executes query, action and assertion", async ({ page }) => {
                          await page.goto("/");
                          // `auto` can query data
                          // In this case, the result is plain-text contents of the header
                          const headerText = await auto(
                            "get the header text",
                            { page, test },
                          // `auto` can perform actions
                          // In this case, auto will find and fill in the search text input
                          await auto(
                            `type "${headerText}" in the search box`,
                            { page, test },
                          // `auto` can assert the state of the website
                          // In this case, the result is a boolean outcome
                          const searchInputHasHeaderText = await auto(
                            `is the contents of the search box equal to "${headerText}"?`,
                            { page, test },

                        Testing User Registration

                        Imagine automating a user registration process. You can instruct Auto Playwright to navigate to the registration page, fill in the user details, submit the form, and then verify the success message. For example:

                        test("User registration", async ({ page }) => {
                          await auto("go to registration page", { page, test });
                          await auto("fill in name, email, and password", { page, test });
                          await auto("submit the registration form", { page, test });
                          const registrationSuccess = await auto("is there a 'Registration successful' message?", { page, test });

                        Complex Navigation and Data Retrieval

                        Auto Playwright can navigate through a series of pages and retrieve specific information. For instance, testing a shopping cart feature might involve adding items to the cart, navigating to the cart page, and confirming the items and prices.

                        test("Shopping cart functionality", async ({ page }) => {
                          await auto("add item 'X' to cart", { page, test });
                          await auto("go to shopping cart page", { page, test });
                          const itemsList = await auto("list all items in the cart", { page, test });
                          const totalPrice = await auto("what is the total price?", { page, test });
                          expect(itemsList).toContain('Item X');

                        Integrating AI into Daily QA Work

                        In quality assurance (QA) and software testing, people often ask, "How do you use AI every day in your work?" Before, there weren't many clear examples. But now, Auto Playwright is changing that. It brings AI right into the heart of test automation. This means QA experts can use AI to make test development much faster and more efficient. Auto Playwright does more than just automate usual tasks. It changes the way we think about testing. It makes it easier to run even the most complex tests smoothly and dependably. For the first time, using AI in QA work really speeds things up and makes tests more accurate and thorough.

                        Limitations of Auto Playwright

                        Auto Playwright is innovative but not yet a full replacement for traditional testing due to slower AI processing times and the cost of API calls. While it's not ideal for high-frequency testing, it excels in speeding up test development and assisting in writing tests. It's best used for less frequent, detailed testing scenarios rather than constant, large-scale use.

                        How Auto Playwright Works

                        Auto Playwright operates by harnessing the capabilities of OpenAI's technology to transform plain text instructions into a series of function executions. Here's a closer look at its workflow:

                        1. Plain Text Instructions: You begin by writing tests in simple, plain text. These instructions describe the actions or checks you want to perform on your web application, like "click the login button" or "verify the page title".

                        2. Translation by OpenAI SDK: Auto Playwright uses the OpenAI SDK, specifically the runFunctions feature, to interpret these instructions. This tool analyzes the text and determines the necessary functions that need to be executed to perform the described tasks.

                        3. Function Executions: Once the plain text is translated, Auto Playwright carries out the sequences of functions on the web application. This might involve interactions like clicking, filling out forms, navigating to different pages, or fetching and verifying content.

                        4. Feedback and Results: As these functions are executed, Auto Playwright provides feedback or results based on the actions performed. This might include confirmation of successful actions, data retrieved from the page, or the results of checks and assertions made.

                        In essence, Auto Playwright acts as an intelligent intermediary between your plain text commands and the complex code required to automate web testing, making the process more intuitive and accessible.

                        Why Auto Playwright?

                        1. Ease of Implementation: Auto Playwright allows for rapid creation of tests using plain text prompts, significantly reducing the time and effort required in traditional methods.

                        2. Flexibility with Supported Browsers: It supports every browser that Playwright does, ensuring wide compatibility.

                        3. Customizable Options: You can fine-tune your testing with additional options like enabling debug mode or choosing an OpenAI model.

                        4. Diverse Functionality: From actions like clicking links to queries and assertions, Auto Playwright covers a wide range of testing needs.

                        5. Cost-Effective: This library is free, with the only costs arising from OpenAI usage, which are minimal due to HTML sanitization.

                        Getting Started

                        To get started with Auto Playwright:

                        1. Follow the Setup Instructions: Install the package and set up your OpenAI API key.
                        2. Write Your Tests: Utilize the auto function with clear, descriptive prompts.
                        3. Run and Observe: Execute your tests and observe the AI-driven automation in action.

                        Join the Auto Playwright Community

                        I encourage you to try out the library in your projects and see the difference AI-driven testing can make. Your feedback and contributions are invaluable; together, we can continuously improve and evolve Auto Playwright. Try it, test with it, and join our community in shaping the future of AI-powered test automation.

                        Conclusion: A Step Forward in Test Automation

                        Auto Playwright represents not just a tool but a leap forward in the way we approach test automation. It democratizes testing by making it more accessible, less cumbersome, and more aligned with the dynamic nature of web development. As the digital landscape continues to evolve, tools like Auto Playwright will be at the forefront, shaping the future of testing and development.

                        This is the future of test automation - simplified, efficient, and powered by AI.


                        This project draws its inspiration from ZeroStep. ZeroStep offers a similar API but with a more robust implementation through its proprietary backend. Auto Playwright was created with the aim of exploring the underlying technology of ZeroStep and establishing a basis for an open-source version of their software. For production environments, I suggest opting for ZeroStep.

                        Here is a side-by-side comparison between Auto Playwright and ZeroStep:

                        CriteriaAuto PlaywrightZeroStep
                        Uses OpenAI APIYesNo[^3]
                        Uses plain-text promptsYesNo
                        Uses functions SDKYesNo
                        Uses HTML sanitizationYesNo
                        Uses Playwright APIYesNo[^4]
                        Uses screenshotsNoYes
                        Uses queueNoYes
                        Uses WebSocketsNoYes
                        Implements parallelismNoYes
                        Allows scrollingNoYes
                        Provides fixturesNoYes
                        Thank you!
                        Was this helpful?

                        Now check your email!

                        To complete the signup, click the confirmation link in your inbox.

                        Subscribe to newsletter

                        You will receive a weekly email with the latest posts and QA jobs.

                        AboutQuestionsDiscord ForumBrowser ExtensionTagsQA Jobs

                        Rayrun is a community for QA engineers. I am constantly looking for new ways to add value to people learning Playwright and other browser automation frameworks. If you have feedback, email luc@ray.run.