AI Test Script Generation: Convert Manual Cases Into Playwright Tests

An AI-generated test can click the correct button and still verify the wrong result. A script that reaches an order confirmation has not necessarily checked the purchased quantity or total. Converting a manual case into Playwright therefore starts with preserving its approved expected behavior, then making that behavior executable against elements that exist.

Treat AI test script generation as a proposal-and-review workflow. Supply the case, application context and project conventions; inspect the candidate; run it; and accept only the coverage you can explain. The model can suggest code, but the requirement owner decides what the application should do.

Freeze the Manual Case Before Asking for Code

Choose one small, approved case rather than a folder of loosely written steps. A useful conversion input identifies prerequisites, test data, actions and expected results separately. Resolve phrases such as “complete checkout successfully” before generation, because they leave the model to invent the acceptance criteria.

Consider this fictional manual case, MUG-01 v1, for a local demonstration page:

• Prerequisites: a fresh page with no order placed and a mug unit price of USD 12.00. Quantity initially displays one.
• Test data: quantity two, with no tax, delivery charge or discount in this example.
• Steps: enter two in Quantity, then select Place Order.
• Expected result: the confirmation reads “Order accepted: 2 mugs; total USD 24.00”.

The expected total is fixed by the case, not calculated by copying the application’s pricing expression into the assertion. A reviewer can compare the script directly with the approved outcome. This case covers one positive quantity path. It says nothing about payment, stock, invalid quantities, backend persistence or whether a real order reaches a fulfillment system.

Keep the source version alongside the proposed script. For teams managing manual coverage in Jira, Sherlock Test Management’s product documentation describes human-approved, versioned test baselines and completed runs linked to the version they executed. That can provide the reviewed source-case context for this workflow. It does not establish a Playwright exporter or automation integration; generating and running the script remains a separate task.

Give the Model Locator Evidence and a Bounded Request

Before requesting code, inspect the authorized test environment. Record the actual field labels, button roles and accessible names, relevant containers, and agreed test IDs. Include the application state in which those elements exist. A familiar selector copied from another project is not evidence about your DOM.

For the example below, the inspected fixture contains a Quantity label associated with a numeric input, a button named Place Order, and a status paragraph whose data-testid is order-result. The generated test should use those contracts. If a real screen has several Place Order buttons, scope the locator to the intended order or dialog instead of hiding ambiguity with first().

Provide a representative existing test or seed fixture, the approved application entry point, authentication setup, supported data-reset mechanism, package versions and test configuration. Do not paste production credentials or customer records into a prompt. If necessary context is missing, ask the generator to list the gap rather than fabricate an account, API or selector.

A bounded request can be copied and adapted:

Convert approved case MUG-01 v1 into one Playwright TypeScript test. Preserve the prerequisites, quantity two and exact expected confirmation supplied above. Use the inspected DOM contracts and the project’s fixture pattern. Await every Playwright action and web-first assertion. Do not invent selectors, requirements, credentials or data-reset APIs. Return the complete candidate, a mapping of each expected result to its assertion, and any unresolved assumptions. Do not change the expected result to make execution pass.

Playwright’s documented test agents offer one implementation path: a planner explores an application using a seed test and optional requirements context; a generator converts the Markdown plan into test files and verifies selectors and assertions live. The documentation still warns that generated tests may contain initial errors. Review the plan before generation and review the resulting code afterward. Ordinary Playwright codegen records browser interactions and selected assertions; that recording workflow should not be described as an AI model.

Convert the Case Into a Self-Contained TypeScript Test

The following complete file uses a deliberately fictional DOM, installed into a fresh Playwright page. It needs no application server, login or network service. The fixture’s page.evaluate function implements the tiny demonstration application; the test beneath it implements the approved manual case. In a real project, replace the fixture with your existing application setup, not an invented production equivalent.

Use a separate empty local directory with Node.js and npm available. Install the tested package versions with the following command:

npm install --save-dev --save-exact @playwright/test@1.64.1 typescript@7.0.2 @types/node@26.6.5

Install the Chromium headless shell with the next command. If you use a custom browser-cache path, supply the same PLAYWRIGHT_BROWSERS_PATH value during installation and every test run.

npx playwright install chromium --only-shell

Save all the following TypeScript lines, from the import through the final closing statement, as order.spec.ts. They are plain text so they can be copied directly into that file.

import { test as base, expect, type Page } from '@playwright/test';
const test = base.extend<{ shop: Page }>({
shop: async ({ page }, use) => {
await page.setContent('<main><label for="quantity">Quantity</label><input id="quantity" type="number" min="1" value="1"><button type="button">Place Order</button><p role="status" data-testid="order-result">No order placed</p></main>');
await page.evaluate(() => {
const input = document.querySelector('input') as HTMLInputElement;
const button = document.querySelector('button') as HTMLButtonElement;
const result = document.querySelector('[role="status"]') as HTMLParagraphElement;
button.addEventListener('click', () => {
const count = Number(input.value);
result.textContent = 'Order accepted: ' + count + ' mugs; total USD ' + (count * 12).toFixed(2);
});
});
await use(page);
}
});
test('MUG-01 v1: order two mugs for USD 24.00', async ({ shop }) => {
const quantity = shop.getByLabel('Quantity', { exact: true });
const placeOrder = shop.getByRole('button', { name: 'Place Order', exact: true });
const result = shop.getByTestId('order-result');
await expect(quantity).toHaveValue('1');
await expect(result).toHaveText('No order placed');
await expect(placeOrder).toHaveCount(1);
await quantity.fill('2');
await placeOrder.click();
await expect(result).toHaveText('Order accepted: 2 mugs; total USD 24.00');
});

The DOM queries inside page.evaluate belong to the demonstration application, not the test’s locator strategy. The test interacts through the observed label, role/name and test ID. Its initial assertions verify the declared starting state, while the final assertion checks the business result after submission. The literal expected confirmation is intentionally separate from the application’s multiplication.

Save this complete configuration as tsconfig.json beside the test. It enables strict type checking for this file without emitting JavaScript:

{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noEmit": true,
"types": ["node"],
"skipLibCheck": true
},
"include": ["order.spec.ts"]
}

Review the Oracle and Awaited Assertions

The oracle is the approved rule used to judge the result. Here it requires the correct quantity and total in the confirmation. Merely checking that Place Order was clicked, that a status paragraph exists, or that a screenshot was saved would leave that outcome unchecked. A broad “contains Order accepted” assertion would also miss an incorrect total.

Playwright’s awaited web-first assertions retry the condition until it succeeds or times out. That is why the final check awaits expect(result).toHaveText(...). Avoid replacing it with a fixed sleep or an immediate isVisible() value check. Waiting can accommodate rendering delays, but it cannot decide whether USD 24.00 is the right requirement.

Review the candidate for missing awaits, swallowed assertion failures, test.only, skips, unexpected network calls and invented helpers. Ensure every approved expected result has a meaningful assertion. Where the manual case is ambiguous, return it for clarification instead of letting a successful run approve the model’s interpretation.

Separate Browser Isolation From Application Data

The typed shop fixture establishes the page before await use(page), then hands that page to the test. Playwright’s built-in page belongs to a fresh browser context for each test; the runner tears that context down afterward. The example has no server data to clean up, and the fixture reinstalls the same initial DOM on every invocation.

A real application needs a separate data contract. Fresh cookies and local storage do not reset an order table or allocate an independent customer account. Use your application’s supported test-data provisioning and cleanup mechanism, with records owned by the current test. Put related setup and teardown together in a fixture, and avoid depending on a previous test to create the order. Shared worker-level resources need an explicit concurrency plan; a new browser context does not prevent two tests from editing the same server record.

Run the Candidate and Investigate What Failed

Type checking and browser execution are separate gates. Playwright transforms TypeScript to JavaScript without checking its types. Run the compiler explicitly, then run the actual spec rather than treating test collection as execution:

npx tsc -p tsconfig.json --noEmit
npx playwright test order.spec.ts --workers=1 --retries=0 --trace on --reporter=list

This command uses the default Chromium browser without assuming that a project named chromium exists in a configuration. Check your installed CLI help before adapting flags. Keeping whole-test retries at zero makes the first acceptance run easier to interpret; web-first assertions still retry their own conditions.

For this article, the exact displayed candidate was exercised in an isolated local project with Node.js v24.18.0, Playwright 1.64.1, TypeScript 7.0.2 and Chromium headless shell 156.0.8078.4. Strict type checking exited successfully, and the candidate run reported one passed test. A separate run with --repeat-each=3 reported three passed executions with fresh test contexts and zero whole-test retries. These observations concern this toy page only. No customer application, Playwright generator agent, external model provider or CI pipeline was exercised.

A preplanned negative exercise checked that the oracle could detect the missing outcome: the same test ran against the fixture before its confirmation handler was added. The click completed, but the final assertion failed because the paragraph still read “No order placed”. After implementing the handler, the unchanged expected confirmation passed. This demonstrates sensitivity to that missing behavior; it does not establish complete fault detection or validate the requirement independently.

When a real candidate fails, classify the failure before asking AI to repair it:

Observation	Inspect	Acceptance Decision
Multiple elements match an action	DOM state, accessible names and intended container	Correct the locator scope without changing the case
Click completes but total is wrong	Approved pricing rule, input data and displayed result	Reject the result; investigate a product or test defect
Test passes alone but fails in a suite	Shared accounts, records and fixture lifecycle	Fix the data contract and rerun independently and together
Agent output skips the scenario	Failure evidence, skip reason and intended coverage	Keep the case unverified until an explicit human disposition

Record the first failure and compare any proposed repair with the reviewed candidate. Preserve the oracle unless the requirement owner approves a genuine requirement change. A locator repair may be appropriate when an accessible name changes; deleting the price assertion merely to obtain green output would remove the promised coverage.

Use the trace to inspect the action locator, DOM snapshots, source and error details. The --trace on option records the local no-retry run. A policy that records only on the first retry would not capture a run with no retry. Open a retained artifact with npx playwright show-trace path/to/trace.zip, using its actual path, and review sensitive application data before sharing it.

Playwright documents that its healer may return a skipped test when it believes functionality is broken. A skipped scenario is not passing product verification. That documented behavior does not mean the healer always weakens assertions; it means reviewers must inspect the resulting status and coverage rather than accepting an apparently finished repair loop.

Accept the Conversion as Reviewed Coverage

Accept a generated script only when its evidence connects back to the approved case. The review record should preserve:

• The manual case ID, version, prerequisites, data and expected results.
• The inspected locator contracts and the exact candidate revision.
• The assertion mapping and the fixture’s browser-state and server-data responsibilities.
• Separate type-check and actual execution outcomes, with relevant logs or traces.
• A human decision on remaining assumptions, skipped coverage and additional cases.

A green run shows that the implemented assertions held in that environment. It cannot prove that the oracle was correct, that persistence works, or that every requirement is covered. Extend coverage through separately approved cases for invalid inputs, permissions and other business rules rather than silently expanding one generated script.

Start with one approved case whose business outcome is precise enough to reject a wrong result. Give the generator real application context, then require a reviewable candidate and an actual run. The useful deliverable is executable, explainable coverage with a human acceptance decision attached.