QA Automation AI Agent Project: Vision, MVP Scope and Architecture

This is the level where everything comes together. Over the next 22 lessons you will build a working QA automation AI agent that reads a requirement, inspects a page, writes Playwright tests, runs them and reports the results. First you need a clear plan, because the biggest risk in any agent project is trying to build too much at once.

---

💡 The Goal of the QA Automation AI Agent

A person gives the system a requirement and an approved test environment. The system turns that into reviewed scenarios and Playwright tests. It runs them, diagnoses any failures and writes a report with evidence.

---

🔍 Define the MVP

An MVP, or minimum viable product, is the smallest version that truly works. Ours tests one workflow: login.

In the first version	Left for later
Web UI tests with Playwright and TypeScript	API, mobile and performance testing
One safe demo application	Any production system
Valid and invalid login scenarios	Many workflows at once
Human approval before writing or running code	Fully automatic runs
Screenshots and traces for diagnosis	Automatic changes to application code
A model behind a provider adapter	Dependence on one AI vendor

---

🧩 The Architecture

User requirement
      |
      v
Orchestrator (controls every step)
      |
      +--> Requirement parser --> Structured requirements
      +--> Explorer ------------> Real page evidence
      +--> Planner -------------> Traceable test scenarios
      +--> Generator -----------> Playwright test file
      +--> Validator -----------> Static checks and human approval
      +--> Executor ------------> Playwright test run
      +--> Failure analyst -----> Evidence-based diagnosis
      +--> Healer --------------> Proposed patch (approval needed)
      +--> Reporter ------------> Markdown, HTML and JSON report

You saw this design in Multi-Agent Orchestration for QA. Agents here are focused components. They are not chatbots that can do anything they like.

---

🛠️ What Each Part Is Responsible For

Part	Responsibility	It must never
Orchestrator	Controls order, approvals and limits	Skip an approval
Requirement parser	Structures the requirement	Invent missing details
Explorer	Reports what is on the page	Invent selectors
Planner	Creates traceable scenarios	Cover imaginary features
Generator	Writes the test code	Use unsupported locators
Validator	Rejects unsafe output	Trust the model
Executor	Runs approved commands	Run a model’s shell command
Failure analyst	Classifies failures	Claim a cause without evidence
Healer	Proposes the smallest fix	Change an expected result
Reporter	Summarises what happened	Hide a failure

---

📌 Your Build Plan

Phase	Work	Lessons
0	Check your environment and agree the plan	Setup
1	Create the repository and a real Playwright test	57 and 58
2	Define contracts and guardrails	59
3	Build each agent with a mocked model	60 to 66
4	Connect a real model	67
5	Add diagnosis, healing and reporting	68 to 70
6	Add MCP, CI and evaluation	71 to 73
7	Accept the MVP and present it	74 to 76

Notice how late the real model arrives. By then, the rest of the system is already tested.

---

🌍 Real-Life Example: The Agent That Did Everything

A team asks an AI to build a complete testing agent in one prompt. It produces thousands of lines that look impressive and never fully run. Nobody can tell which part is broken. Another team builds one tested piece at a time, and it has a demo in two weeks.

---

🧪 Try It Yourself

1. Write one sentence that describes what your agent will do.
2. List the login scenarios that you expect it to create.
3. Choose where each agent’s work should stop.
4. Mark every point where a person should approve.
5. Compare your notes with the tables above.

---

📋 Ready-to-Copy Prompt

You are a senior QA automation architect reviewing an AI agent plan.

My goal: [describe what the agent should do]
First workflow: [for example, login]
Approved test environment: [describe it with no secrets]

Tell me:
1. What should be in the first version, and what should wait
2. Where a person must approve
3. What could cause harm if the model is wrong
4. The smallest demo that would prove the idea works

Be strict about scope. Do not design anything beyond the first version.

---

⚠️ Common Mistakes

• Starting with the AI model instead of the test foundation
• Trying to cover every kind of testing at once
• Letting agents call each other freely
• Pointing the agent at a real production site
• Measuring success only by how much code it writes

---

🎯 Key Takeaways

• Build a small, testable workflow first.
• Code controls the order, and agents do focused jobs.
• Model output stays untrusted until it is validated.
• People approve anything that writes or runs code.

---

🎤 Interview Questions

1. What would you build first in an AI testing agent?

Model answer: I would first prove that ordinary Playwright tests run reliably. Then I would add typed contracts and a mocked workflow, and connect a real model only when the rest is stable.

2. What is an MVP for a QA agent?

Model answer: It is the smallest version that proves the idea, such as one login workflow on a safe application. It goes from a requirement to a reviewed, executed test and a report.

3. Why should an orchestrator control the workflow?

Model answer: It guarantees that validation and approval cannot be skipped. It also makes runs repeatable and easier to debug than agents deciding everything for themselves.

4. What would you leave out of the first version?

Model answer: I would leave out production access, automatic healing, application code changes and extra test types. Each of those adds risk before the core workflow is proven.

5. What is your rule for model output?

Model answer: I treat it as untrusted until validated by schemas, code checks and human review. The system should never act on it just because a model produced it.

---

📝 Assignment

Write a one-page project charter. Include the goal, the MVP scope, what is out of scope, the approved environment and your definition of done. Keep it in your project folder for the next lesson.

---

❓ FAQ

Do I need a paid AI model to start?

No. You will build almost everything with a mocked model, which is free and repeatable.

Can I use my own application instead of the demo?

Later, yes, if you are authorised to test it. Finish the project on the demo first so that any problems are easy to find.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with AI Agent Project Setup: Prerequisites and a Safe Local Environment.