API Testing with AI: From OpenAPI Specs to Playwright Tests

APIs have clear contracts, which makes them a great match for AI. If you give it an accurate description, it can suggest many scenarios quickly. In this lesson you will practise API testing with AI, starting from an OpenAPI description and finishing with tests that run against our free practice API.

📖 What You’ll Learn

  • What an API and an OpenAPI spec are
  • Which scenarios every endpoint needs
  • How to write Playwright API tests
  • How to check generated API tests

💡 What Is an API?

Explain Like I’m 5: API

Imagine: In a restaurant, you tell the waiter your order. The waiter brings food from the kitchen.

What it really means: An API lets one program ask another for data or actions.

In testing: A test can ask for a user directly, with no browser.


🔍 What Is an OpenAPI Spec?

Explain Like I’m 5: OpenAPI Spec

Imagine: A menu lists every dish, what comes with it and the price.

What it really means: An OpenAPI spec describes every endpoint, its inputs and its responses.

In testing: It gives AI real facts to work from.

openapi: 3.0.3
info:
  title: Practice API
  version: 1.0.0
paths:
  /users/{id}:
    get:
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: integer
            minimum: 1
      responses:
        '200':
          description: The user was found
        '400':
          description: The id is not valid
        '404':
          description: The user does not exist

This short example was written for this lesson. Always use the real spec for your own API.


🧩 Scenarios for API Testing with AI

Explain Like I’m 5: Trying Every Door

Imagine: You check that your key opens the door, that the wrong key does not and that a locked door stays locked.

What it really means: Test valid requests, bad requests and access rules.

In testing: One happy-path test is never enough.

Scenario typeExample
Valid requestExisting user returns 200
Not foundUnknown ID returns 404
Invalid inputNon-numeric ID returns 400
ValidationBad field values return 422
AuthenticationMissing token returns 401
Response bodyFields and values are correct
ContractResponse matches its schema

🌍 Real-Life Example: Testing the Practice API

Our API Testing Practice page offers a free REST API. These tests use its real responses.

import { test, expect } from '@playwright/test';

const BASE_URL = 'https://sqatools.in/wp-json/sqa-api/v1';

test('GET user 1 returns the expected user', async ({ request }) => {
  const response = await request.get(`${BASE_URL}/users/1`);

  expect(response.status()).toBe(200);
  const body = await response.json();
  expect(body.data.id).toBe(1);
  expect(body.data.name).toBe('Asha Verma');
});

test('GET an unknown user returns 404', async ({ request }) => {
  const response = await request.get(`${BASE_URL}/users/999`);

  expect(response.status()).toBe(404);
  const body = await response.json();
  expect(body.error.code).toBe('not_found');
});

test('invalid user details return all validation errors', async ({ request }) => {
  const response = await request.post(`${BASE_URL}/validate/user`, {
    data: { name: '', email: 'not-an-email', age: 10 },
  });

  expect(response.status()).toBe(422);
  const body = await response.json();
  expect(body.error.code).toBe('validation_failed');
  expect(body.error.details).toHaveLength(3);
});

Learn more in API Testing with Playwright.


📌 How to Check Generated API Tests

Explain Like I’m 5: Reading the Label

Imagine: A jar says “strawberry jam”. You still open it to check that it is really jam.

What it really means: A successful status code does not prove the data is right.

In testing: Check important fields, not only the status.

Review each test for these things:

  1. Does the expected status come from the spec or a real decision?
  2. Does it check important response values?
  3. Does it create its own data and clean up afterwards?
  4. Can it run alone and in any order?
  5. Does it contain any secrets?

Specs can be wrong too

If the API and its spec disagree, do not simply change the test. Report the difference, and find out which one is correct.


🧪 Try It Yourself

  1. Open the API Testing Practice page.
  2. Pick an endpoint, and send a request by hand.
  3. Run the prompt below using its real details.
  4. Add the tests to your Playwright project and run them.
  5. Change one expected value, and confirm that the test fails.

📋 Ready-to-Copy Prompt

You are a senior API test engineer. Using only the details below, list
test scenarios first. Do not write code yet.

API description or OpenAPI excerpt:
[paste description]

Real sample responses:
[paste responses with no secrets or personal data]

Return a table with: ID, Endpoint, Scenario Type, Request, Expected
Status, Expected Body Check, Source.

Rules:
- Cover valid, invalid, not-found, validation and auth scenarios.
- Mark any expected result without a source as "NEEDS CONFIRMATION".
- Do not invent endpoints or fields.

Wait for me to approve the scenarios. Then write Playwright Test code
in TypeScript using the request fixture.

⚠️ Common Mistakes

  • Testing only happy paths
  • Checking only the status code
  • Letting AI invent fields or endpoints
  • Sharing real tokens in prompts
  • Writing tests that depend on each other

🎯 Key Takeaways

  • An accurate spec gives AI strong input.
  • Cover valid, invalid, validation and access scenarios.
  • Check response values, not only status codes.
  • Report any differences between the API and its spec.

🎤 Interview Questions

1. What is OpenAPI?

Model answer: OpenAPI is a standard format for describing REST APIs, including endpoints, parameters, request bodies and responses. Teams use it for documentation, client generation and test design.

2. What would you test for a REST endpoint?

Model answer: I would test valid requests, invalid inputs, validation rules, not-found cases, authentication, authorisation, response data and schema. I would also consider idempotency and error formats where relevant.

3. What is the difference between 401 and 403?

Model answer: A 401 means the request is not authenticated or the credentials are missing or invalid. A 403 means the server knows who the caller is, but that caller lacks permission.

4. How do you use AI for API testing?

Model answer: I give it an accurate spec and sample responses, ask for scenarios first and review them. Then I generate tests and verify the expected results, data setup and cleanup.

5. How do you test an API with Playwright?

Model answer: I use the request fixture to send HTTP requests. I check the status and parse the body to assert important values. Playwright can also combine API setup with UI tests.


📝 Assignment

Write at least six tests for two practice endpoints, including one validation failure and one not-found case. Compare the live responses with the user schema. Record any scenario that AI missed.


❓ FAQ

What if my API has no OpenAPI spec?

You can use requirements, developer notes and approved sample responses. Treat anything AI infers from samples as unconfirmed.

Can I run load tests on the practice API?

No. It is a shared learning resource. Use it for functional tests only, and see Lesson 29 for safe performance testing.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with SQL Database Validation with AI: Check Data Like a Tester.

Migrate Selenium to Playwright with AI: A Safe Step-by-Step Plan

AI can convert a Selenium test in seconds, but a line-by-line copy can carry old problems into a new tool. A good migration improves the suite while keeping its coverage. In this lesson you will learn how to migrate Selenium to Playwright with AI using a small pilot and clear proof at each step.

📖 What You’ll Learn

  • When a migration makes sense
  • How Selenium ideas map to Playwright
  • How to convert a test without copying its problems
  • How to prove the new tests are equal or better

💡 Should You Migrate at All?

Explain Like I’m 5: Moving House

Imagine: Before you move, you ask whether the new house is better. Then you carry over only what you still need.

What it really means: A migration needs a clear reason and a clear plan.

In testing: Do not copy tests that nobody needs.

Good reasons include flaky timing, slow debugging or a better fit with your team’s skills. If your current suite is stable and valuable, a full migration may not be worth the cost. Read Playwright vs Selenium vs Cypress first.


🔍 How Concepts Change

Explain Like I’m 5: A New Language

Imagine: “Bonjour” means hello, but you do not translate a joke word by word. You translate the meaning.

What it really means: Convert the purpose of a test, not each line.

In testing: Remove sleeps instead of translating them.

SeleniumPlaywright
WebDriver instancepage fixture
find_element(By.ID, ...)page.locator('#id') or getByRole
Explicit waitsAuto-waiting and web-first assertions
time.sleep(3)Remove it, and assert the expected state
assert element.text == valueawait expect(locator).toHaveText(value)
Grid setupBuilt-in browsers and projects

🌍 Real-Life Example: One Test Converted

This Selenium test targets our Automation Practice Page.

import time
from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://sqatools.in/automation-practice-page/")
driver.find_element(By.XPATH, "//button[text()='Load Success']").click()
time.sleep(3)
message = driver.find_element(By.ID, "loader").text
assert message == "Data loaded successfully!"
driver.quit()

The message takes about three seconds to appear, so time.sleep(3) is a race that can fail. A direct translation would keep that fixed wait. A better conversion looks like this:

import { test, expect } from '@playwright/test';

test('Load Success shows the success message', async ({ page }) => {
  await page.goto('https://sqatools.in/automation-practice-page/');
  await page.getByRole('button', { name: 'Load Success' }).click();
  await expect(page.locator('#loader')).toHaveText('Data loaded successfully!');
});

The purpose stays the same, the XPath is gone and the fixed wait has been replaced by a retrying assertion. Playwright also closes the browser for you.


🛠️ How to Migrate Selenium to Playwright with AI

Explain Like I’m 5: Trying a Bite First

Imagine: You try one spoon of soup before you serve the whole pot.

What it really means: A small pilot shows problems early.

In testing: Convert a few important tests before planning the rest.

  1. List your tests. Mark each one to keep, fix or delete.
  2. Choose a pilot. Pick 5 to 10 tests that matter and vary in difficulty.
  3. Set up the foundation. Add config, a page object and test data.
  4. Convert one test at a time. Review each result.
  5. Compare. Run both suites and compare coverage and results.
  6. Measure. Record stability and run time on your own system.
  7. Scale gradually. Keep the old suite until the new one earns trust.

📌 Proving Equal Coverage

AI can drop a check without any warning. For each converted test, compare:

CheckQuestion
StepsDoes the new test cover the same behaviour?
AssertionsIs every old assertion kept or deliberately replaced?
DataIs the same test data used?
Negative casesAre error paths still covered?
FailureDoes the new test fail when behaviour breaks?

Keep old evidence

Do not delete the Selenium test until its replacement has passed reviews and repeated runs.


🧪 Try It Yourself

  1. Write or find a small Selenium test.
  2. Run the prompt below.
  3. Compare every assertion with the original.
  4. Run the Playwright test three times.
  5. Break the feature or expected value, and confirm that it fails.

📋 Ready-to-Copy Prompt

You are helping me migrate one Selenium test to Playwright Test with
TypeScript.

Selenium test:
[paste test]

Real HTML or page details:
[paste details]

Rules:
- Convert the behaviour, not each line.
- Remove fixed sleeps, and use web-first assertions.
- Prefer role-based and label-based locators from the provided HTML.
- Keep every original assertion, or explain why it changed.
- Do not add tests or framework layers.

After the code, give me a table: Original Check, New Check, Same or Changed.

⚠️ Common Mistakes

  • Migrating unused tests just to finish the list
  • Translating sleeps into Playwright waits
  • Running a big-bang migration without a pilot
  • Assuming that all assertions were carried over
  • Removing old tests too early

🎯 Key Takeaways

  • Decide why you are migrating before you begin.
  • Convert the purpose of a test, not each line.
  • Start with a pilot, then expand gradually.
  • Prove equal coverage for every converted test.

🎤 Interview Questions

1. How would you plan a Selenium to Playwright migration?

Model answer: I would review the existing suite, remove low-value tests and run a small pilot. Then I would build the foundation, convert in priority order, compare results and keep the old tests until the new ones are trusted.

2. How do waits differ between Selenium and Playwright?

Model answer: Selenium commonly uses explicit waits in test code. Playwright auto-waits before actions and offers retrying web-first assertions, so many fixed and manual waits can be removed.

3. How can AI help with migration?

Model answer: It can speed up conversion and suggest modern patterns. I give it real page details and rules, then compare assertions and coverage myself, because it may drop checks or invent locators.

4. How do you make sure a migrated test is correct?

Model answer: I compare steps, data and assertions with the original. I run it repeatedly and confirm that it fails when the behaviour breaks. I also compare results across both suites.

5. When would you not migrate from Selenium?

Model answer: I might not migrate if the suite is stable, the team has strong Selenium skills, or a required browser or integration is better supported there. The migration cost must be justified.


📝 Assignment

Choose one test from our Selenium with Python material, or write your own. Convert it with AI, complete the comparison table and run the result five times. Write three lines on what improved and what still needs attention.


❓ FAQ

Can Selenium and Playwright run side by side?

Yes. During a migration, this lets you build confidence before switching off the older tests.

Can AI convert a whole suite at once?

It can try, but reviewing a huge change is difficult. Small batches are safer and easier to verify.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with API Testing with AI: From OpenAPI Specs to Playwright Tests.

Page Object Model with AI: Design a Clean Playwright Framework

One test is easy. Two hundred tests with copied locators are painful. A framework keeps them tidy, and AI can do much of the typing. In this lesson you will use the Page Object Model with AI, while you stay in charge of the design decisions that matter.

📖 What You’ll Learn

  • What the Page Object Model is
  • A simple framework folder structure
  • How to refactor with AI in small steps
  • Which design decisions should stay with you

💡 What Is the Page Object Model?

Explain Like I’m 5: Page Objects

Imagine: A TV remote has one Volume button. You do not open the TV and touch the wires every time.

What it really means: A page object keeps a page’s locators and actions in one place.

In testing: When a button changes, you fix one line and not fifty tests.


🔍 A Simple Page Object

Here is a page object for our Automation Practice Page.

import type { Locator, Page } from '@playwright/test';

export class PracticePage {
  readonly page: Page;
  readonly loadSuccessButton: Locator;
  readonly loaderMessage: Locator;

  constructor(page: Page) {
    this.page = page;
    this.loadSuccessButton = page.getByRole('button', { name: 'Load Success' });
    this.loaderMessage = page.locator('#loader');
  }

  async goto(): Promise<void> {
    await this.page.goto('https://sqatools.in/automation-practice-page/');
  }

  async loadSuccess(): Promise<void> {
    await this.loadSuccessButton.click();
  }
}

And here is the test that uses it:

import { test, expect } from '@playwright/test';
import { PracticePage } from '../pages/practice-page';

test('Load Success shows the success message', async ({ page }) => {
  const practicePage = new PracticePage(page);

  await practicePage.goto();
  await practicePage.loadSuccess();
  await expect(practicePage.loaderMessage).toHaveText('Data loaded successfully!');
});

The test now reads like a set of steps. In this course we keep assertions in the tests, so the page object does not hide what is being checked.


🧩 A Simple Folder Structure

Explain Like I’m 5: A Tidy Bedroom

Imagine: Socks go in one drawer and books go on the shelf. You can find things without searching the whole room.

What it really means: Each kind of file gets one clear place.

In testing: Pages, tests, data and helpers stay separate.

tests/
  practice-page.spec.ts
pages/
  practice-page.ts
test-data/
  users.json
fixtures/
  pages.ts
playwright.config.ts

See Framework Folder Structure for a fuller layout.


🛠️ How to Use the Page Object Model with AI

Explain Like I’m 5: Moving House

Imagine: You move one box at a time and check that nothing broke. You do not throw the whole house onto a truck.

What it really means: Refactor a little, test, and then continue.

In testing: Convert one page and its tests before starting the next.

Ask AI to refactor one page at a time. Run the tests before and after each change. A refactor should change the structure, and it should not change what the tests prove.

Refactoring must keep behaviour

Compare your test count and results before and after. If assertions disappear, reject the change.


📌 Decisions That Stay With You

AI tends to add layers because they look professional. Too many layers make simple tests hard to read.

DecisionA good question to ask
Base classesDo we really share behaviour here?
HelpersIs it used more than once?
Assertions in page objectsDoes our team allow this?
FixturesDoes this remove real repeated setup?
Test dataWhere should it live, and who owns it?

Start simple. Add structure only when repetition causes a real problem. Our Page Object Model lesson explains the pattern in detail.


🌍 Real-Life Example: Over-Engineering

A tester asks AI to “create an enterprise framework” for six tests. The answer has base pages, factories, wrappers and a logging layer. It looks impressive, but a new teammate cannot follow it. The tester instead asks for one page object and one fixture. The result is smaller and easier to maintain.


🧪 Try It Yourself

  1. Take the two tests from Lesson 24.
  2. Commit your project.
  3. Run the prompt below.
  4. Review the diff, and confirm that the assertions are unchanged.
  5. Run the tests, and compare the results.

📋 Ready-to-Copy Prompt

You are a senior test automation engineer. Refactor these Playwright
TypeScript tests to use the Page Object Model.

Current tests:
[paste tests]

Rules:
- Create one page object only for the page used in these tests.
- Keep all assertions in the test file.
- Do not change what any test checks.
- Do not add base classes, wrappers or new packages.
- Keep the same locators unless one is clearly unstable.

Show the new folder structure and full file contents. Then list
anything you changed apart from moving code.

⚠️ Common Mistakes

  • Asking AI for a huge framework on day one
  • Refactoring many pages in one step
  • Hiding important assertions inside helpers
  • Adding base classes with no shared behaviour
  • Skipping the before-and-after test run

🎯 Key Takeaways

  • Page objects keep locators and actions in one place.
  • A simple folder structure is easier to maintain.
  • Refactor one page at a time, and check the results.
  • Design decisions remain a human responsibility.

🎤 Interview Questions

1. What is the Page Object Model?

Model answer: It is a design pattern that places a page’s locators and actions in a dedicated class. Tests call those methods, so a UI change usually needs an update in only one place.

2. Should assertions be placed inside page objects?

Model answer: Teams differ. I usually keep assertions in tests so that the test clearly shows what it verifies. Simple page-level checks may be acceptable if the team agrees on that convention.

3. How would you use AI to refactor a test framework?

Model answer: I would work in small steps, starting with one page. I would give it clear rules, run the tests before and after, and review the diff to confirm behaviour and assertions did not change.

4. What is over-engineering in test automation?

Model answer: It means adding layers or abstractions before they solve a real problem. This makes tests harder to read and maintain, so I add structure only when repetition or complexity justifies it.

5. How are fixtures useful in Playwright?

Model answer: Fixtures provide reusable setup and teardown for tests, such as creating page objects or signed-in users. They reduce repeated code and give each test the resources it needs.


📝 Assignment

Create page objects for two sections of the practice page. Add a fixture that provides one of them, then write two tests. Ask a friend to read a test and explain what it does without opening the page object.


❓ FAQ

Are page objects required in Playwright?

No. Small projects can work well without them. They become useful as repeated locators and actions grow.

Should I use fixtures or page objects?

They work well together. Page objects describe the page, and fixtures prepare what each test needs.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Migrate Selenium to Playwright with AI: A Safe Step-by-Step Plan.

Generate Playwright Tests with AI: Stable Locators and Real Assertions

AI will happily write a Playwright test for a page it has never seen. It will guess the locators, and the test may fail or, worse, pass for the wrong reason. In this lesson you will generate Playwright tests with AI using real page details, stable locators and assertions that prove something.

📖 What You’ll Learn

  • A safe workflow for generating a test
  • How to choose stable locators
  • What makes an assertion meaningful
  • How to prove a test can really fail

💡 How to Generate Playwright Tests with AI Safely

Explain Like I’m 5: Plan, Then Build

Imagine: You draw your sandcastle first. Then you build it. It is easier to fix a drawing than a castle.

What it really means: Agree the test plan before asking for code.

In testing: Review scenarios first, then generate one test.

  1. Write down the behaviour to test.
  2. Collect real details about the page.
  3. Ask AI for one test.
  4. Read the code.
  5. Run it.
  6. Break the behaviour, and confirm the test fails.

🔍 Give AI Real Page Details

AI cannot see your page unless you show it. You can paste the relevant HTML, use a browser tool such as Playwright MCP, or use Codegen to record real locators.

Here is real HTML from our Automation Practice Page:

<button onclick="loadAjax(true)">Load Success</button>
<p id="loader"></p>
<input type="text" id="username" name="username" placeholder="Enter username">
<input type="checkbox" id="python"><label for="python">Python</label>

🧩 Choosing Stable Locators

Explain Like I’m 5: Stable Locators

Imagine: “The boy in the red hat” works until he changes hats. “Asha” still works.

What it really means: A stable locator keeps working when the page’s appearance changes.

In testing: Find a button by its role and name, not by its position.

PreferExampleAvoid
Role and namegetByRole('button', { name: 'Load Success' })div > button:nth-child(2)
LabelgetByRole('checkbox', { name: 'Python' })Long XPath
PlaceholdergetByPlaceholder('Enter username')Auto-generated class names
Test IDgetByTestId('checkout')Position on the page

An ID such as #loader is fine when no better user-facing option exists. Learn more in Playwright Locators.


🛠️ Meaningful Assertions

Explain Like I’m 5: Checking the Real Result

Imagine: You press the light switch. Seeing the switch move is not enough. You check that the light came on.

What it really means: An assertion should prove the behaviour worked.

In testing: Check the success message, not only that a button exists.

WeakStronger
Button is visible after clickingSuccess message has the exact text
Page did not crashSaved value appears after reload
Response has a bodyResponse contains the expected data

🌍 Real-Life Example: Two Useful Tests

import { test, expect } from '@playwright/test';

const PAGE_URL = 'https://sqatools.in/automation-practice-page/';

test.beforeEach(async ({ page }) => {
  await page.goto(PAGE_URL);
});

test('Load Success shows the success message', async ({ page }) => {
  await page.getByRole('button', { name: 'Load Success' }).click();
  await expect(page.locator('#loader')).toHaveText('Data loaded successfully!');
});

test('a language checkbox can be selected', async ({ page }) => {
  const python = page.getByRole('checkbox', { name: 'Python' });

  await expect(python).not.toBeChecked();
  await python.check();
  await expect(python).toBeChecked();
});

Notice the check before the action. If the box was already selected, the second test would otherwise prove nothing.

await expect(...).toHaveText(...) retries until the text appears or the timeout ends. That removes the need for fixed waits.


🔑 Prove the Test Can Fail

Explain Like I’m 5: Testing the Smoke Alarm

Imagine: You press the test button on the smoke alarm. An alarm that never rings is no help.

What it really means: A test is only useful if it fails when the behaviour breaks.

In testing: Change the expected text briefly and watch it fail.

Try one of these on a temporary copy:

  • Change the expected text.
  • Click the wrong button.
  • Remove the action step.

If the test still passes, it is not checking what you thought.

Do not edit expected results to get green

When a test fails, find out why. The product may be wrong, and the test may have just found a real bug.


🧪 Try It Yourself

  1. Pick another feature from the practice page, such as the country dropdown.
  2. Copy its real HTML.
  3. Run the prompt below.
  4. Read the code, and run it.
  5. Make it fail on purpose, then restore it.

📋 Ready-to-Copy Prompt

You are a senior Playwright engineer. Write one Playwright Test in
TypeScript.

Behaviour to test: [behaviour]
Page URL: [approved test URL]
Real HTML from the page:
[paste relevant HTML]

Rules:
- Use only elements shown in the HTML.
- Prefer getByRole, getByLabel, getByPlaceholder and getByTestId.
- Use web-first assertions with await expect.
- Do not use fixed waits or CSS position selectors.
- The assertion must prove the behaviour, not only that an element exists.
- If the expected result is not provided, write a TODO and ask me.

After the code, tell me how to make this test fail on purpose.

⚠️ Common Mistakes

  • Letting AI guess locators it has not seen
  • Accepting toBeVisible as the only check
  • Using waitForTimeout to hide timing problems
  • Running a generated test only once
  • Never checking whether the test can fail

🎯 Key Takeaways

  • Give AI real page details, and generate one test at a time.
  • Prefer locators based on what users see.
  • Assert the outcome of the behaviour.
  • Prove that each new test can fail.

🎤 Interview Questions

1. How do you generate reliable Playwright tests with AI?

Model answer: I define the behaviour first and give the tool real HTML or browser access. I require stable locators and meaningful assertions, then read and run the test, and confirm it fails when the behaviour is broken.

2. Which Playwright locators do you prefer, and why?

Model answer: I prefer role, label and test ID locators because they match how users find elements or are designed for testing. They usually survive layout and styling changes better than long CSS or XPath selectors.

3. What are web-first assertions?

Model answer: They are Playwright assertions such as toHaveText that automatically retry until the condition is met or the timeout ends. This handles dynamic pages without fixed waits.

4. How can a generated test pass but be useless?

Model answer: It may only check that a page loaded, assert the wrong thing or never reach the behaviour. I review what it proves and verify that it fails when I break the behaviour.

5. What would you do if an AI-generated locator fails?

Model answer: I inspect the real page and see whether the locator was invented, the page changed or the element is not ready. I fix the cause using a stable locator, and I avoid adding waits just to hide it.


📝 Assignment

Generate three tests for different features of the practice page. For each test, record the locator choice, the assertion and how you proved that it can fail. Rewrite at least one weak assertion.


❓ FAQ

Can AI use my live browser to write tests?

Yes, tools such as Playwright Test Agents can explore a page. Keep them on approved test environments and review what they produce.

Should every element get a test ID?

Not necessarily. Use accessible roles and labels first, and add test IDs where they are not practical.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Page Object Model with AI: Design a Clean Playwright Framework.

AI Coding Assistants for Testers: Use Your IDE Safely

A chat window can give you code, but an assistant inside your editor can see your project. That makes it more useful, and it means you need clear rules. This lesson explains how AI coding assistants for testers work, how to guide them and how to keep control of what changes.

📖 What You’ll Learn

  • The three main ways assistants work in an IDE
  • How to give an assistant project rules
  • How to work in small, reviewable steps
  • Which permissions need extra care

💡 How AI Coding Assistants for Testers Work

Explain Like I’m 5: Three Helpers

Imagine: One helper finishes your sentence. One answers your questions. One tidies a whole room while you watch.

What it really means: Assistants can suggest a line, answer in chat or make changes across files.

In testing: The bigger the job, the more carefully you review it.

ModeWhat it doesBest forRisk
InlineSuggests the next linesSmall, familiar codeAccepting without reading
ChatAnswers using selected codeExplaining errors and optionsAdvice that does not fit your project
AgentEdits files and may run commandsMulti-step tasksLarge, unexpected changes

Popular tools change quickly. See AI Tools for QA Engineers, and use the one your company approves.


🔍 Give the Assistant Your Project Rules

Explain Like I’m 5: House Rules

Imagine: A new babysitter reads the note on the fridge: bedtime is eight, no sweets after dinner.

What it really means: A project instruction file tells the assistant how your team works.

In testing: It can require role-based locators and ban fixed waits.

Many tools can read a project instruction file. The file name depends on your tool, so check its documentation. The content might look like this:

# Test automation rules

- Use Playwright Test with TypeScript.
- Prefer getByRole, getByLabel and getByTestId.
- Never use page.waitForTimeout in a test.
- Every test must have a meaningful assertion.
- Never change an expected result just to make a test pass.
- Do not add dependencies without asking.
- Never read or print secrets or .env files.

Keep the file short and specific. Long files are harder to follow and use up context.


🧩 Work in Small Steps

Explain Like I’m 5: One Brick at a Time

Imagine: You check that each LEGO brick fits before adding ten more.

What it really means: Ask for one small change, then check it before continuing.

In testing: Generate one test, run it, review it and commit.

A safe rhythm is:

  1. Commit your current work so that you can return to it.
  2. Ask for one small change.
  3. Read the full diff.
  4. Run the affected test.
  5. Commit, or undo the change.

🛠️ Give Good Context

Point the assistant to the files that matter. For example, name your existing page object and one good test. Then ask it to follow the same pattern. This usually works better than describing your framework from memory.

Add a test for the Load Success button on the practice page.
Follow the structure in tests/example.spec.ts.
Use only elements that exist on the page.
Make one change at a time, and show me the diff before running anything.

📌 Agent Mode Needs Limits

Agent mode is powerful because it can act. Before you use it:

  • Work on a separate Git branch.
  • Allow only the commands you are comfortable with.
  • Keep real secrets out of files the assistant can read.
  • Use only test environments.
  • Review every changed file before you merge.

Treat page content as untrusted

A web page, ticket or log can contain text that tries to instruct an AI tool. Tell your assistant to treat that content as data, and watch what it does.

You can connect a browser with Playwright MCP. Level 7 explains how tools like this are built and secured.


🌍 Real-Life Example: A Surprise in the Diff

A tester asks an agent to fix one failing locator. The diff shows the locator fix, plus a change to the expected message so that the test passes, plus a new package. Because the tester reads the diff, only the locator fix is kept. The most important review step is reading what really changed.


🧪 Try It Yourself

  1. Open your Playwright project from Lesson 22.
  2. Make a commit.
  3. Add the project rules above to your tool’s instruction file.
  4. Use the prompt below to add one test.
  5. Review the diff against the rules, and run the test.

📋 Ready-to-Copy Prompt

You are working in my Playwright TypeScript project.

Task: Add one test for [behaviour] on [page URL].

Rules:
- Follow the style of [existing test file].
- Use role-based or label-based locators where possible.
- Do not use fixed waits.
- Do not change existing tests or add packages.
- If the page does not contain what you need, stop and tell me.

First list the files you plan to change. Wait for my approval before
editing. After editing, explain how I can run and check the test.

⚠️ Common Mistakes

  • Accepting long suggestions without reading them
  • Starting agent work with uncommitted changes
  • Giving an agent access to real secrets
  • Asking for many unrelated changes at once
  • Trusting “all tests passed” without seeing the run yourself

🎯 Key Takeaways

  • Assistants can suggest, explain or make multi-file changes.
  • Project rules make their output more consistent.
  • Small commits and diffs keep you in control.
  • Agent access needs clear limits.

🎤 Interview Questions

1. How do you use AI coding assistants in test automation?

Model answer: I use them for drafts, explanations and small refactoring. I provide examples from the project, make one change at a time, read the diff and run the tests myself before committing.

2. How do you make an assistant follow team standards?

Model answer: I use a short project instruction file with locator rules, banned patterns and test conventions. I also point it to good existing tests, and I enforce important rules with linting and code review.

3. What are the risks of agent mode?

Model answer: It can edit several files, run commands or add unexpected changes. I reduce the risk with a separate branch, limited permissions, no real secrets and full review before merging.

4. What would you check in a diff produced by AI?

Model answer: I check scope, locators, assertions, waits, new dependencies and secrets. I look especially for changed expected results or removed checks that could hide a defect.

5. How do you protect sensitive data when using coding assistants?

Model answer: I use approved tools and settings, keep secrets outside the project and use synthetic data. I also check what files the tool can access, and I follow company policy.


📝 Assignment

Write a project instruction file with at least six rules. Ask your assistant for two new tests, one at a time. Record every rule it followed or broke. Update the file to fix one problem, and try again.


❓ FAQ

Which AI coding assistant should I choose?

Use one approved by your company. Good review habits matter more, and they transfer between tools.

Will an assistant make me a worse coder?

It can if you stop thinking. Read every change, ask why it works and write some code yourself.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Generate Playwright Tests with AI: Stable Locators and Real Assertions.

TypeScript and Playwright Basics for Testers: A Git Primer

AI can write test code in seconds, but you must be able to read it and tell if it is right. This lesson covers the TypeScript and Playwright basics you need for the rest of Level 4, plus just enough Git to keep your work safe. If you are new to coding, take your time here.

📖 What You’ll Learn

  • The TypeScript you need to read a test
  • The parts of a Playwright test
  • Why async and await appear everywhere
  • Five Git commands for saving your work

💡 Why Learn This If AI Writes the Code?

Explain Like I’m 5: Reading the Code

Imagine: A robot can write your shopping list. If you cannot read, you will not know it wrote “ten thousand apples”.

What it really means: You must understand code to judge what AI wrote.

In testing: Reading a test shows you what it really checks.

You do not need to be a developer. You do need to follow a test line by line.


🔍 TypeScript in Five Minutes

Explain Like I’m 5: Types

Imagine: A toy box labelled “cars” should only hold cars. If someone puts in a sandwich, you notice.

What it really means: A type says what kind of value something should be.

In testing: TypeScript warns you when a test uses the wrong kind of data.

type TestUser = {
  username: string;
  age: number;
  active: boolean;
};

const user: TestUser = { username: 'asha', age: 34, active: true };

function isAdult(age: number): boolean {
  return age >= 18;
}

console.log(isAdult(user.age));

This prints true. Here is what each part means:

CodeMeaning
type TestUserDescribes the shape of some data
const userA named value that will not be reassigned
age: numberThis value must be a number
function isAdultA reusable piece of logic

For a full introduction, see TypeScript Variables.


🧩 Your First Playwright Test

Follow Install Playwright to set up your project. The usual start is:

npm init playwright@latest
npx playwright test

Here is a real test for our Automation Practice Page.

import { test, expect } from '@playwright/test';

test('Load Success shows a message', async ({ page }) => {
  await page.goto('https://sqatools.in/automation-practice-page/');
  await page.getByRole('button', { name: 'Load Success' }).click();
  await expect(page.locator('#loader')).toHaveText('Data loaded successfully!');
});
PartJob
test(...)Names one test
pageThe browser tab Playwright controls
getByRoleFinds an element the way a user sees it
expect(...)Checks the result

🛠️ Why Async and Await?

Explain Like I’m 5: Await

Imagine: You put bread in the toaster, then wait for it to pop before you butter it.

What it really means: Browser actions take time, and await waits for each one to finish.

In testing: Without await, the test rushes ahead and fails in odd ways.

A missing await is one of the most common bugs in AI-generated tests. Check for it every time.


📌 Git in Five Commands

Explain Like I’m 5: Git

Imagine: In a video game, you save before a hard level. If it goes wrong, you go back to your save.

What it really means: Git saves versions of your work so you can compare and undo.

In testing: You can try AI’s changes and return if they break something.

git init
git status
git add .
git commit -m "Add first Playwright test"
git diff

Run git diff before committing to see exactly what changed. This matters even more when AI has edited your files. Our Git and GitHub Tutorial goes much further.

Keep secrets out of Git

Never commit passwords, tokens or a real .env file. Add those files to .gitignore before your first commit.


🧪 Try It Yourself

  1. Install Playwright using the guide linked above.
  2. Copy the test from this lesson into a file ending in .spec.ts inside your tests folder.
  3. Run it with npx playwright test.
  4. Change the expected text, and confirm that the test fails.
  5. Use the prompt below to explain any line you do not understand.

📋 Ready-to-Copy Prompt

You are a patient automation mentor teaching a manual tester.

Explain this Playwright TypeScript test line by line in plain English:

[paste test]

Then tell me:
1. What behaviour this test really proves
2. What it does not prove
3. One way it could pass even if the feature is broken

Do not rewrite the test yet. Use simple words.

⚠️ Common Mistakes

  • Running AI code that you cannot read
  • Forgetting await before an action or web-first assertion
  • Committing secrets or node_modules
  • Making one giant commit instead of small ones
  • Changing a failing assertion just to get a pass

🎯 Key Takeaways

  • You need to read code well enough to judge it.
  • Types help catch mistakes early.
  • A test finds elements, performs actions and checks results.
  • Git lets you review AI changes and undo them.

🎤 Interview Questions

1. Why is TypeScript useful for test automation?

Model answer: It adds types, so editors can give better suggestions and catch many mistakes before tests run. This helps in larger frameworks and when reviewing generated code.

2. What do async and await do in a Playwright test?

Model answer: Playwright actions run asynchronously and return promises. The await keyword pauses that test step until the action finishes, so steps run in the intended order.

3. What are the main parts of a Playwright test?

Model answer: A test usually has a title, a page fixture, locators to find elements, actions such as click or fill, and assertions with expect to verify the outcome.

4. What is the difference between git add and git commit?

Model answer: git add puts selected changes in the staging area. git commit saves those staged changes as a new version in the repository history.

5. How do you check code that an AI tool has changed?

Model answer: I review the diff, read each changed line, run the tests and confirm the assertions still reflect the requirement. I also look for removed checks and accidental secrets.


📝 Assignment

Create a project with two tests for the practice page. Commit after each test with a clear message. Then break one test on purpose, read the failure, fix it and use git diff to review your change.


❓ FAQ

Should I learn JavaScript first?

It helps, but you can start here and fill gaps as you go. Our Playwright course covers JavaScript variables and more.

Do I need Python instead?

No. We use TypeScript in this course, and the same ideas apply to Playwright with Python.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with AI Coding Assistants for Testers: Use Your IDE Safely.

AI Testing Project for Beginners: Build a Complete Test Pack

You have learned many separate skills. Now you will combine them. This AI testing project for beginners guides you through a complete test pack for a demo website. When you finish, you will have real work to show in interviews, and honest numbers to talk about.

📖 What You’ll Learn

  • How to combine Levels 1 to 3 in one workflow
  • How to measure the real value of AI
  • How to keep a record of human review
  • How to present the finished project

💡 The Project Goal

Explain Like I’m 5: The Project

Imagine: You learned to cut, glue and colour. Now you make a whole birthday card.

What it really means: A project joins separate skills into one finished piece of work.

In testing: You go from a feature description to a complete test pack.

Your mission is to create a full test pack for one feature of the Dummy Booking Website. You may instead use the Dummy E-commerce Website. Use only made-up rules and data.


🔍 AI Testing Project: Your Eight Steps

StepWhat to produceLesson
1. Story reviewUser story, criteria and questionsLesson 12
2. Test planOne-page planLesson 13
3. Test designBoundaries, table and statesLesson 14
4. Test dataValid, invalid and boundary recordsLesson 15
5. ExplorationCharter and session notesLesson 16
6. Bug reportsOne or more clear reportsLesson 17
7. Regression choiceRisk table and skipped areasLesson 19
8. SummaryResults and recommendationLesson 20

Use the review checks from Lesson 10 at each step. Use your prompt library too.


🧩 Keep a Review Log

Explain Like I’m 5: Review Log

Imagine: When your homework is checked, the red marks show what was fixed.

What it really means: A review log records what AI produced and what you changed.

In testing: It proves that you were in control of the work.

Add one row for each step:

StepItems from AIAcceptedEditedRejectedWhat you added
Test design201253Two state tests

The numbers above are only an example. Yours must be real. In this example, 12 of 20 items were accepted unchanged, which is 60%.


🛠️ Measure Honestly

Explain Like I’m 5: Honest Measuring

Imagine: You time your race with a stopwatch. You do not just say, “I was very fast.”

What it really means: Measure real results and do not guess.

In testing: Record how long each task took, and what quality you got.

Record:

  • Time spent on each step, including your review
  • Items accepted, edited and rejected
  • Gaps that AI missed and you found
  • Mistakes that AI made and you caught

To compare time fairly, do one similar task without AI. Your result may be faster, slower or mixed. All of those are honest results.

Never polish the numbers

Interviewers often ask follow-up questions. A small real result that you can explain is worth far more than a big number you cannot defend.


📌 Organise Your Test Pack

booking-test-pack/
  README.md
  01-story-and-questions.md
  02-test-plan.md
  03-test-design.md
  04-test-data.json
  05-exploratory-session.md
  06-bug-reports.md
  07-regression-risk.md
  08-test-summary.md
  review-log.md
  prompts-used.md

The README should explain the feature, the tools used, your process and what you learned. Before sharing, make sure there is no private data or employer information. Our Git and GitHub Tutorial shows how to publish a project.


🧪 Try It Yourself

  1. Choose your feature, and write down one reason for choosing it.
  2. Complete Step 1, then stop and review your work.
  3. Continue through the steps, updating your review log each time.
  4. Run the prompt below when your pack is finished.
  5. Fix the most important problem it raises.

📋 Ready-to-Copy Prompt

You are a QA hiring manager reviewing a beginner's portfolio project.

Project description: [describe the project]
Files included: [list files]
Review log summary: [your real numbers]

Review it for clarity, test thinking and evidence of human judgment.

Return:
1. Three strengths, based only on what I shared
2. Three specific improvements
3. Five interview questions you would ask about this project

Do not assume that I did anything not described above.
Do not suggest adding numbers or results I do not have.

⚠️ Common Mistakes

  • Choosing a whole website instead of one feature
  • Skipping the review log
  • Publishing AI output without changing or checking it
  • Claiming time savings you did not measure
  • Forgetting to explain your decisions in the README

🎯 Key Takeaways

  • A single project can bring all your Level 1 to 3 skills together.
  • A review log shows your judgment, not only your output.
  • Honest numbers are stronger than impressive guesses.
  • A clear test pack gives you real examples for interviews.

🎤 Interview Questions

1. Tell me about a project where you used AI in testing.

Model answer: Use the structure of situation, task, action and result. For example: “I built a test pack for a booking feature. AI drafted scenarios and data, and I reviewed each step. I accepted some items, corrected others and added cases it missed.” Use only your real numbers.

2. How did you make sure AI output was correct?

Model answer: I traced each test to the requirement, checked expected results, reviewed the data and kept a log of changes. I also ran the tests myself instead of relying on the AI’s output.

3. What did AI miss in your project?

Model answer: Give a true example, such as a state transition or a business question that AI did not raise. Explain how you found it. This shows that you understand the limits of the tools.

4. Did AI save you time?

Model answer: Answer with your actual measurements, and include review time. A good answer might be: “Drafting was faster, but reviewing took longer than I expected, so the overall gain was moderate.”

5. How would you improve the project?

Model answer: I might add more features, automate the most stable checks and compare prompts over repeated runs. I would choose improvements based on the weak points that appeared in my review log.


📝 Assignment

Complete all eight steps and publish the test pack in a private or public repository, with no sensitive data. Practise explaining the project out loud in two minutes. This completes Level 3, and you are ready to begin AI-assisted automation in Level 4.


❓ FAQ

How long should this project take?

Take as long as you need to do it well. Focus on one feature, as depth matters more than size.

Can I use my real work project?

Only if your employer allows it, and with all confidential details removed. A demo website is the safer choice for a portfolio.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with TypeScript and Playwright Basics for Testers: A Git Primer.

Test Summary Report with AI: Write Results Stakeholders Understand

Your testing is only useful if people understand the result. Managers want to know what was tested, what is still risky and what you recommend. In this lesson you will write a test summary report with AI, turn technical results into plain language and create simple release notes.

📖 What You’ll Learn

  • What belongs in a useful test summary
  • How to write for different audiences
  • How to protect your numbers from AI mistakes
  • How to create clear release notes

💡 What Is a Test Summary Report?

Explain Like I’m 5: Test Summary

Imagine: After a school trip, you tell your parent the three most important things. You do not describe every step you took.

What it really means: A test summary gives the important results and what they mean.

In testing: It shows quality, open risks and your recommendation.

SectionThe question it answers
ScopeWhat did we test?
ResultsHow many tests passed, failed or were blocked?
Open defectsWhat is still broken?
RisksWhat might still go wrong?
Not testedWhat did we leave out?
RecommendationShould this go ahead, and with what conditions?

🔍 How to Write a Test Summary Report with AI

Explain Like I’m 5: Do Your Own Counting

Imagine: Before sharing sweets, you count them yourself. A guess could leave someone with none.

What it really means: Work out the numbers yourself, then give them to the AI.

In testing: Copy results from your test tool and verify every number.

AI can make arithmetic mistakes, so calculate and verify key numbers first. Then let the AI help with wording.


🌍 Real-Life Example: Release Results

Here are made-up results for one release.

MeasureValue
Tests planned200
Tests executed190
Passed172
Failed18
Blocked10
Pass rate of executed tests90.5%

Check them: 172 plus 18 equals 190 executed tests. Add 10 blocked tests, and you reach the 200 planned. The pass rate is 172 divided by 190.

A number alone can mislead. A 90.5% pass rate could be fine, or it could hide one critical payment failure. Always explain what failed. Link to your clear bug reports, and show any areas you skipped during risk-based regression testing.

A good summary could read:

“We tested the new checkout discount and executed 190 of 200 planned tests. Ten tests were blocked by the payment sandbox. One High defect remains: discounts are wrong for orders with mixed tax rates. We recommend releasing only after this defect is fixed and the blocked payment tests have passed.”


🛠️ Write for Your Audience

Explain Like I’m 5: Choosing Your Words

Imagine: You explain a game one way to your grandmother and another way to your best friend.

What it really means: Different readers need different details.

In testing: Leaders need risk and decisions, while developers need defect details.

AudienceWhat they need
Manager or sponsorStatus, risk and decision needed
Product ownerUser impact and open items
DeveloperDefect IDs, steps and evidence

📌 The Recommendation Is Yours

You make the call

AI can draft the wording, but it does not carry responsibility for the release. Your recommendation should reflect your judgment and your team’s rules.


🔑 Release Notes in Plain Language

Release notes tell users what changed. Write about the benefit, and avoid internal terms.

Internal noteRelease note
Fixed null pointer in discount serviceFixed an issue where some discount codes would not apply
Refactored cart APIDo not include it, unless users notice a change

Never include internal hostnames, security details or customer names.


🧪 Try It Yourself

  1. Use the made-up results in this lesson.
  2. Add two open defects of your own invention.
  3. Run the prompt below.
  4. Check every number against your data.
  5. Ask the AI to rewrite it for a developer, and compare both versions.

📋 Ready-to-Copy Prompt

You are a QA lead writing a test summary for a [audience].

Verified results:
[planned, executed, passed, failed and blocked counts]

Open defects: [ID, severity and one-line impact]
Not tested and why: [details]
Known risks: [details]
My recommendation: [your recommendation and conditions]

Write a summary under 150 words in plain language.

Rules:
- Use my numbers exactly. Do not calculate new numbers.
- Do not change my recommendation.
- Do not make anything sound safer than the facts show.
- If a fact is missing, write "[TO CONFIRM]".

⚠️ Common Mistakes

  • Reporting only a pass percentage
  • Letting AI calculate or change your numbers
  • Leaving out blocked tests and untested areas
  • Using technical language with non-technical readers
  • Making the result sound better than it really is

🎯 Key Takeaways

  • A summary explains results, risks and what happens next.
  • Verify the numbers before you give them to AI.
  • Write differently for different audiences.
  • The recommendation stays with people.

🎤 Interview Questions

1. What should a test summary report include?

Model answer: It should include scope, execution results, open defects, risks, anything not tested and a clear recommendation. It should help a reader understand quality and make a decision quickly.

2. Why can a high pass rate be misleading?

Model answer: It treats every test equally and ignores what was blocked or not tested. A single critical failure can matter more than many passed tests, so I always explain the impact of failures.

3. How do you communicate test results to non-technical stakeholders?

Model answer: I focus on user and business impact, remaining risk and the decision needed. I keep technical details short and link to them for readers who need more.

4. How do you use AI for test reporting safely?

Model answer: I verify the data first and give AI the final numbers. I tell it not to calculate or change them, then I review the wording. I also ensure it has not softened the risks.

5. What would you do if management wants to release despite a high-severity defect?

Model answer: I would make the impact and risk clear, with evidence and options such as a workaround or a limited release. The business may accept the risk, but the decision should be informed and recorded.


📝 Assignment

Using the same made-up results, write one summary for a manager and one for a developer. Then write three plain-language release notes. Check that every number is correct, and that the manager’s version can be read in under one minute.


❓ FAQ

How often should I send a test summary?

Follow your team’s rhythm. Many teams share short status updates during testing and a final summary before release.

Can AI read the results directly from my test tool?

Some tools offer this. Check their accuracy and data rules, and review the result as you would any other AI output.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with AI Testing Project for Beginners: Build a Complete Test Pack.

Risk-Based Regression Testing with AI: Choose What to Test

You rarely have time to run every test before a release. The real question is which tests matter most this time. In this lesson you will use risk-based regression testing with AI to score risk, choose tests and clearly explain what you are leaving out.

📖 What You’ll Learn

  • What regression testing and risk mean
  • How to score risk with two simple numbers
  • How AI can help map changes to tests
  • How to explain what you are skipping

💡 What Is Regression Testing?

Explain Like I’m 5: Regression Testing

Imagine: You fix a wobbly table leg. Then you check that the table still stands and nothing else came loose.

What it really means: Regression testing checks that changes did not break working features.

In testing: After a checkout fix, you also check payments and order emails.


🔍 What Is Risk?

Explain Like I’m 5: Risk

Imagine: Spilling water on the floor is a small problem. Spilling water on a laptop is much bigger.

What it really means: Risk combines how likely a problem is and how much harm it causes.

In testing: A likely bug in payments deserves more testing than a rare logo issue.

Use this simple formula:

Risk score = Likelihood × Impact

Score each from 1 (low) to 3 (high). Your team can adjust the scale.


🌍 Real-Life Example: A Checkout Release

This release changes the discount calculation.

AreaLikelihoodImpactScoreDecision
Order total and discounts339Test deeply
Payment236Test core and negative paths
Order confirmation email224Test main path
Login133Run smoke test
Profile picture111Skip this release

Likelihood rises when code has changed, is complex or has failed before. Impact rises when a failure affects money, safety, legal duties or many users.


🛠️ How to Do Risk-Based Regression Testing with AI

Explain Like I’m 5: Sorting Your Time

Imagine: You have only ten minutes before bedtime, so you put away the sharp toys first.

What it really means: Limited time should go to the most important risks.

In testing: Run the highest-risk tests first.

AI can help you to:

  • Connect a change list to features that may be affected
  • Suggest regression areas that you might forget
  • Draft a first risk table from your facts
  • Summarise your decision for stakeholders

AI does not know your real code dependencies unless you supply them. It also does not know which customers matter most. Ask developers what changed and what else uses that code.

Use real evidence

Base scores on change history, defect history, usage and developer input. An AI score with no evidence is only an opinion.


📌 Say What You Are Not Testing

Explain Like I’m 5: Being Honest

Imagine: You tell your parent, “I tidied my room, but I did not clean under the bed.” Now nobody is surprised.

What it really means: Share what you left out and what risk that creates.

In testing: Stakeholders can accept the risk or ask for more testing.

A clear statement could be:

“We did not retest profile pictures in this release because the code did not change and past defects are low. The remaining risk is low.”


🔑 Automated Regression

Run your stable automated suite on every build when you can. Risk-based choices then focus mostly on manual time and slower tests. Unstable tests weaken confidence, so see Retries and Fixing Flaky Tests.


🧪 Try It Yourself

  1. Imagine a change to the room search on the Dummy Booking Website.
  2. List five areas, and score them yourself.
  3. Run the prompt below with the same facts.
  4. Compare every score that differs by 2 or more.
  5. Decide which score is right, and write why.

📋 Ready-to-Copy Prompt

You are a senior QA engineer planning regression testing for a release.

What changed: [change list]
Known dependencies from developers: [dependencies]
Past defects: [defect history]
Important user journeys: [journeys]
Testing time available: [time]

Create a risk table with: Area, Likelihood (1-3), Impact (1-3), Score,
Evidence, Suggested Testing and Reason.

Then list what could be skipped and the risk of skipping it.

Rules:
- Calculate Score as Likelihood multiplied by Impact.
- Use only the information above as evidence.
- Write "[UNKNOWN]" where evidence is missing.
- Mark this as a draft for human review.

⚠️ Common Mistakes

  • Giving every area a “High” rating
  • Accepting an AI risk score that has no evidence
  • Ignoring features that share changed code
  • Skipping tests without telling anyone
  • Using the same regression pack for every release

🎯 Key Takeaways

  • Regression testing checks that changes did not break existing features.
  • Risk combines likelihood and impact.
  • Use real evidence from changes, defects and usage.
  • Always make skipped areas and remaining risk visible.

🎤 Interview Questions

1. What is regression testing?

Model answer: Regression testing confirms that recent changes have not broken existing behaviour. It can be a full suite or a selected set, depending on the change and the time available.

2. How do you prioritise regression tests with limited time?

Model answer: I score areas by likelihood and impact, using change details, defect history, usage and developer input. I test the highest risks first and clearly report what was not tested.

3. What is risk-based testing?

Model answer: It is an approach where testing effort follows the level of product risk. Critical, likely problems receive deeper coverage, while low-risk areas may receive lighter checks or none.

4. How can AI help with regression planning?

Model answer: It can map a change list to possible affected areas and draft a risk table. I verify dependencies with developers and base the scores on real data. The final decision remains with the team.

5. What is the difference between smoke testing and regression testing?

Model answer: A smoke test is a quick check that a build’s critical functions work well enough for further testing. Regression testing is broader and checks whether changes have broken existing behaviour.


📝 Assignment

Create a risk table for a made-up release with at least six areas. Write a two-sentence statement for each skipped area. Ask a friend to challenge one score, and record whether your evidence changes your view.


❓ FAQ

Should I always use a 1 to 3 scale?

No. Use the scale your team understands. A simple scale is often enough to support a good discussion.

Can AI select tests automatically?

Some tools use code changes and past results to select tests. They still need trusted data, and teams should check that important tests are not being missed.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Test Summary Report with AI: Write Results Stakeholders Understand.

Log Analysis with AI: Read Logs and Stack Traces as a Tester

Logs can look like a wall of strange text. Once you know what to look for, they become a timeline of what the system did. In this lesson you will practise log analysis with AI, learn to read a stack trace and turn a confusing error into a useful test finding.

📖 What You’ll Learn

  • What logs and log levels mean
  • How to read a stack trace
  • How to share only safe, relevant lines
  • How to confirm a possible cause

💡 What Is a Log?

Explain Like I’m 5: Logs

Imagine: A detective follows muddy footprints across the floor to see where someone went.

What it really means: A log is a timed record of what software did.

In testing: It shows which request arrived and where it failed.

Log levelMeaningExample
INFONormal activityRequest received
WARNSomething looks unusualA service responded slowly
ERRORAn operation failedThe order could not be saved
DEBUGExtra technical detailValues used by developers

🔍 How to Do Log Analysis with AI Safely

Explain Like I’m 5: Finding the Right Page

Imagine: You do not read a whole storybook to find one picture. You go to the right page.

What it really means: Share only the small part of the log that matters.

In testing: Find the time of the failure and copy nearby lines.

Follow these steps:

  1. Note the time and test action that failed.
  2. Find the same request ID or nearby timestamps.
  3. Copy only the relevant lines.
  4. Replace secrets and personal data with placeholders.
  5. Ask for possible explanations, and how to check them.

Remember the rules in AI Data Privacy for Testers. Use only approved tools, and never paste unsanitised production logs.


🌍 Real-Life Example: A Failed Order

This made-up log shows one request.

10:15:02 INFO  [req-42] POST /api/orders received
10:15:03 WARN  [req-42] inventory-service took 4800 ms
10:15:05 ERROR [req-42] Order failed: TimeoutException from inventory-service
10:15:05 INFO  [req-42] Response status 500

Read it from top to bottom. The same ID, req-42, connects each line. The order service asked the inventory service for help, that service became slow, and the request failed with an HTTP 500.


🛠️ How to Read a Stack Trace

Explain Like I’m 5: Stack Trace

Imagine: One child asks another for a toy, who asks a third. When the third child cannot find it, the message travels back along the line.

What it really means: A stack trace lists the chain of code that was running at the failure.

In testing: It helps developers find where the failure surfaced.

java.util.concurrent.TimeoutException: inventory-service did not respond
    at com.shop.InventoryClient.reserve(InventoryClient.java:88)
    at com.shop.OrderService.place(OrderService.java:41)
    at com.shop.OrderController.create(OrderController.java:23)

In a Java stack trace, start at the top. It gives the error type and message, followed by the most recent code that was running. Python is different. Its traceback puts the error on the last line. If you see “Caused by” in Java, read that part too, because it can reveal the original problem.


📌 From Possible Cause to Evidence

Explain Like I’m 5: Possible vs Proven

Imagine: The cookie is gone and your brother has crumbs on his shirt. That is a clue. It is not proof until you check.

What it really means: AI gives possible explanations, and evidence confirms one.

In testing: A timeout may come from a slow service, a network issue or bad configuration.

For our example, a tester could check:

  • Does the failure repeat when the inventory service is healthy?
  • Are other requests slow at the same time?
  • Does the problem happen with only some products?
  • Did a recent deployment or configuration change happen?

A symptom is not always the cause

The final error may be only the last thing to break. Look at what happened earlier in the timeline, and ask a developer to confirm the root cause.


🔑 Automation Failures

The same approach helps with failed UI tests. Combine the error message with the screenshot and trace. Our Playwright Trace Viewer lesson shows how to inspect a failed test step by step.


🧪 Try It Yourself

  1. Copy the made-up log from this lesson.
  2. Before using AI, write what you think happened.
  3. Run the prompt below.
  4. Mark each answer as confirmed by the log, or as a guess.
  5. Write a short finding for a developer, using only what you can support.

📋 Ready-to-Copy Prompt

You are a senior QA engineer helping me understand a failure.

What I was testing: [test action]
Expected result: [expected result]
Actual result: [actual result]

Sanitised log or stack trace:
[paste only relevant lines, with no secrets or personal data]

Please provide:
1. A plain-language timeline of what happened
2. The first line that looks unusual, and why
3. Up to 3 possible causes, each marked Low, Medium or High likelihood
4. For each cause, one check that could confirm or reject it

Rules:
- Separate facts from the log and your guesses.
- Do not claim a root cause without evidence.
- Tell me what other information you need.

⚠️ Common Mistakes

  • Pasting an entire log file into a chat
  • Sharing tokens, emails or customer data
  • Reporting AI’s guess as the root cause
  • Reading only the final error line
  • Ignoring the timestamps and request IDs

🎯 Key Takeaways

  • Logs are a timeline of system behaviour.
  • A request ID can connect related lines.
  • Stack trace reading order depends on the language.
  • AI suggests possibilities, and evidence confirms them.

🎤 Interview Questions

1. How do you investigate a failed test using logs?

Model answer: I find the time or request ID of the failure, review the related lines in order and look for the first unusual event. I compare this with the test evidence, then check likely causes or involve a developer.

2. What is a stack trace?

Model answer: A stack trace shows the sequence of function calls active when an error occurred. It helps identify where the failure surfaced. Reading order differs by language, so in Python the final line contains the error.

3. What are common log levels?

Model answer: Common levels are DEBUG, INFO, WARN and ERROR. DEBUG gives detailed diagnostics, INFO records normal events, WARN marks unusual conditions and ERROR records failed operations.

4. How can you safely use AI to analyse logs?

Model answer: I use an approved tool, send only the relevant lines and replace sensitive values before sharing. I ask for possible causes and checks, then verify them with real evidence.

5. How do you tell the difference between a symptom and a root cause?

Model answer: A symptom is the visible failure, such as an HTTP 500. A root cause explains why it happened. I trace events back in time, test possible explanations and confirm the cause with evidence and developers.


📝 Assignment

Create a made-up log with at least eight lines, two request IDs and one hidden problem. Ask a friend or an AI assistant to find the problem. Check whether the answer separates facts from guesses, and improve your prompt if it does not.


❓ FAQ

Do testers need to read code to use logs?

No. You can learn a lot from timelines, error messages and request IDs. Basic code reading will help as you move into automation.

Are AI log tools always accurate?

No. Treat any accuracy claim with care, and verify results against the raw evidence.


🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Risk-Based Regression Testing with AI: Choose What to Test.