Introducing Auto Playwright: Automating Playwright Tests with AI

In the ever-evolving landscape of software development, the need for efficient and effective testing solutions has never been more critical. Today, I am thrilled to introduce Auto Playwright, an open-source project that integrates the power of AI into your testing workflow.

How Auto Playwright Streamlines Your Testing Workflow

Auto Playwright is designed to run Playwright tests using AI, offering a seamless and intuitive way to automate testing tasks. The setup is straightforward:

1. Installation: Begin by installing the auto-playwright dependency using npm:

   $ npm install auto-playwright -D

2. Configuration: Auto Playwright leverages OpenAI's capabilities. Export the API token as follows:

   $ export OPENAI_API_KEY='sk-..."

3. Usage: Import and use the auto function in your tests:

   import { test, expect } from "@playwright/test";
   import { auto } from "auto-playwright";

A Practical Example

Consider this scenario: you want to test a web application's search functionality. With Auto Playwright, you can do this with simple, human-like instructions:

test("executes query, action and assertion", async ({ page }) => {
  await page.goto("/");

  // `auto` can query data
  // In this case, the result is plain-text contents of the header
  const headerText = await auto(
    "get the header text",
    { page, test },
  );

  // `auto` can perform actions
  // In this case, auto will find and fill in the search text input
  await auto(
    `type "${headerText}" in the search box`,
    { page, test },
  );

  // `auto` can assert the state of the website
  // In this case, the result is a boolean outcome
  const searchInputHasHeaderText = await auto(
    `is the contents of the search box equal to "${headerText}"?`,
    { page, test },
  );

  expect(searchInputHasHeaderText).toBe(true);
});

Testing User Registration

Imagine automating a user registration process. You can instruct Auto Playwright to navigate to the registration page, fill in the user details, submit the form, and then verify the success message. For example:

test("User registration", async ({ page }) => {
  await auto("go to registration page", { page, test });
  await auto("fill in name, email, and password", { page, test });
  await auto("submit the registration form", { page, test });
  const registrationSuccess = await auto("is there a 'Registration successful' message?", { page, test });
  expect(registrationSuccess).toBe(true);
});

Complex Navigation and Data Retrieval

Auto Playwright can navigate through a series of pages and retrieve specific information. For instance, testing a shopping cart feature might involve adding items to the cart, navigating to the cart page, and confirming the items and prices.

test("Shopping cart functionality", async ({ page }) => {
  await auto("add item 'X' to cart", { page, test });
  await auto("go to shopping cart page", { page, test });
  const itemsList = await auto("list all items in the cart", { page, test });
  const totalPrice = await auto("what is the total price?", { page, test });
  expect(itemsList).toContain('Item X');
  expect(totalPrice).toMatch(/\$\d+/);
});

Integrating AI into Daily QA Work

In quality assurance (QA) and software testing, people often ask, "How do you use AI every day in your work?" Before, there weren't many clear examples. But now, Auto Playwright is changing that. It brings AI right into the heart of test automation. This means QA experts can use AI to make test development much faster and more efficient. Auto Playwright does more than just automate usual tasks. It changes the way we think about testing. It makes it easier to run even the most complex tests smoothly and dependably. For the first time, using AI in QA work really speeds things up and makes tests more accurate and thorough.

Limitations of Auto Playwright

Auto Playwright is innovative but not yet a full replacement for traditional testing due to slower AI processing times and the cost of API calls. While it's not ideal for high-frequency testing, it excels in speeding up test development and assisting in writing tests. It's best used for less frequent, detailed testing scenarios rather than constant, large-scale use.

How Auto Playwright Works

Auto Playwright operates by harnessing the capabilities of OpenAI's technology to transform plain text instructions into a series of function executions. Here's a closer look at its workflow:

1. Plain Text Instructions: You begin by writing tests in simple, plain text. These instructions describe the actions or checks you want to perform on your web application, like "click the login button" or "verify the page title".

2. Translation by OpenAI SDK: Auto Playwright uses the OpenAI SDK, specifically the runFunctions feature, to interpret these instructions. This tool analyzes the text and determines the necessary functions that need to be executed to perform the described tasks.

3. Function Executions: Once the plain text is translated, Auto Playwright carries out the sequences of functions on the web application. This might involve interactions like clicking, filling out forms, navigating to different pages, or fetching and verifying content.

4. Feedback and Results: As these functions are executed, Auto Playwright provides feedback or results based on the actions performed. This might include confirmation of successful actions, data retrieved from the page, or the results of checks and assertions made.

In essence, Auto Playwright acts as an intelligent intermediary between your plain text commands and the complex code required to automate web testing, making the process more intuitive and accessible.

Why Auto Playwright?

1. Ease of Implementation: Auto Playwright allows for rapid creation of tests using plain text prompts, significantly reducing the time and effort required in traditional methods.

2. Flexibility with Supported Browsers: It supports every browser that Playwright does, ensuring wide compatibility.

3. Customizable Options: You can fine-tune your testing with additional options like enabling debug mode or choosing an OpenAI model.

4. Diverse Functionality: From actions like clicking links to queries and assertions, Auto Playwright covers a wide range of testing needs.

5. Cost-Effective: This library is free, with the only costs arising from OpenAI usage, which are minimal due to HTML sanitization.

Getting Started

To get started with Auto Playwright:

1. Follow the Setup Instructions: Install the package and set up your OpenAI API key.
2. Write Your Tests: Utilize the auto function with clear, descriptive prompts.
3. Run and Observe: Execute your tests and observe the AI-driven automation in action.

Join the Auto Playwright Community

I encourage you to try out the library in your projects and see the difference AI-driven testing can make. Your feedback and contributions are invaluable; together, we can continuously improve and evolve Auto Playwright. Try it, test with it, and join our community in shaping the future of AI-powered test automation.

• Discord
• GitHub
• Newsletter

Conclusion: A Step Forward in Test Automation

Auto Playwright represents not just a tool but a leap forward in the way we approach test automation. It democratizes testing by making it more accessible, less cumbersome, and more aligned with the dynamic nature of web development. As the digital landscape continues to evolve, tools like Auto Playwright will be at the forefront, shaping the future of testing and development.

This is the future of test automation - simplified, efficient, and powered by AI.

ZeroStep

This project draws its inspiration from ZeroStep. ZeroStep offers a similar API but with a more robust implementation through its proprietary backend. Auto Playwright was created with the aim of exploring the underlying technology of ZeroStep and establishing a basis for an open-source version of their software. For production environments, I suggest opting for ZeroStep.

Here is a side-by-side comparison between Auto Playwright and ZeroStep: