How to Review AI-Generated Test Cases: A Beginner’s Guide
AI can write 50 test cases in under a minute. Checking them is now the important job. In this lesson you will learn how to review AI-generated test cases with three simple checks, then practise on a test set containing planted mistakes.
📖 What You’ll Learn
- The three problems to look for in AI test cases
- How to find missing coverage
- How to find invented behaviour and wrong results
- How to give feedback that improves the next answer
💡 How to Review AI-Generated Test Cases: Three Questions
Explain Like I’m 5: Reviewing AI Test Cases
Imagine: You check a packed school bag three ways. What is missing? What does not belong? What is broken?
What it really means: Look for gaps, invented content and wrong results.
In testing: Missing boundary cases, made-up rules and wrong expected results.
| Check | The question | Common example |
|---|---|---|
| Gaps | What is missing? | No boundary or role-based scenarios |
| Invented content | What was never asked for? | A rule that is not in the requirement |
| Wrong results | What is incorrect? | An expected result that breaks a rule |
🔍 Before You Read the AI’s Answer
Explain Like I’m 5: Make Your Own List First
Imagine: Before opening a lunchbox, guess what should be inside. Then you notice right away if the sandwich is missing.
What it really means: Think of key scenarios yourself before reading the AI’s answer.
In testing: Your own short list stops the AI from shaping your thinking.
Spend two minutes writing the top risks and scenarios you expect. A long, tidy AI answer can make you feel that everything is covered. Your own list protects you from that feeling.
🧩 Check 1: Find the Gaps
Use the test design techniques you already know. The ISTQB Foundation guide explains them if you need a refresher.
- Boundaries: values just below, on and just above each limit
- Negative paths: invalid, empty and unexpected input
- Roles: what different kinds of user can and cannot do
- States: actions on locked, expired or already-used items
- Failures: timeouts, retries and duplicate requests
🛠️ Check 2: Remove Invented Content
Explain Like I’m 5: Invented Content
Imagine: You ask for a story about a cat. A dragon appears, and nobody asked for one.
What it really means: The AI added something that your requirement never mentioned.
In testing: A test expects an OTP, but the story never mentions OTP.
Trace each test case back to its requirement. If you cannot find a source, do not simply delete it. Ask whether it is an invention or a real gap in the requirement. Sometimes the AI has raised a sensible question.
📌 Check 3: Verify the Expected Results
This is the most important check. For every expected result, ask three things:
- Does it match the business rule exactly?
- Can a tester actually observe it?
- Is it specific? “It works correctly” is not an expected result.
Calculate any numbers yourself. Do not trust the arithmetic in AI output.
🔑 Also Check Duplicates and Priority
AI often writes the same test in different words. Merge those cases. Then check that the priority reflects real business risk. A typing error and a failed money transfer should not both be “High”.
💡 Practice: Find the Planted Mistakes
Requirement: A customer can transfer money between their own accounts. The minimum per transfer is 100. The maximum per transfer is 50,000. The source and destination accounts must be different.
Below is an AI-style answer with mistakes planted in it. Try to find the problems before you read the answer key.
| ID | Scenario | Amount | Expected Result |
|---|---|---|---|
| TC1 | Transfer the minimum amount | 100 | Transfer succeeds |
| TC2 | Transfer above the maximum | 50,001 | Transfer succeeds |
| TC3 | Transfer with a valid amount | 5,000 | OTP is sent and the transfer succeeds |
| TC4 | Transfer a mid-range amount | 1,000 | Transfer works correctly |
| TC5 | Transfer one thousand between accounts | 1,000 | Transfer succeeds |
Answer Key
| Finding | Type | Fix |
|---|---|---|
| TC2 should be rejected | Wrong result | Expect an error and no money moved |
| TC3 adds an OTP | Invented content | Remove it, or ask the product owner |
| TC4 says “works correctly” | Weak result | State the updated balances |
| TC5 repeats TC4 | Duplicate | Merge them |
| No test below 100, such as 99 | Gap | Add a boundary test |
| No test at exactly 50,000 | Gap | Add a boundary test |
| No same-account test | Gap | Add a negative test |
Did you catch all seven?
Finding the wrong result and the missing boundaries takes practice. The more you review, the faster you will spot them.
🔍 Give Better Feedback to the AI
You can use your review to improve the next answer. Be specific, and quote the original rule.
Please correct the test cases using this review.
1. TC2: A transfer above 50,000 must be rejected. Correct the
expected result.
2. TC3: Remove the OTP step. It is not in the requirement.
3. Add boundary tests for 99 and 50,000.
4. Add a negative test where both accounts are the same.
Use only the requirement text. Do not add any new rules.Review the corrected version too. New answers can contain new mistakes.
🧪 Try It Yourself
- Choose a feature from the Dummy Banking Website.
- Write your own list of the top 5 scenarios first.
- Ask an AI assistant for 10 test cases, using made-up rules.
- Run the prompt below in a new chat for a second opinion.
- Do your own review, then compare your findings with the AI’s.
📋 Ready-to-Copy Prompt
Act as a strict QA reviewer. Review these test cases against the
requirement. Do not rewrite them.
Return a table with these columns:
Test Case ID, Problem Type, Evidence from Requirement, Suggested Fix
Problem types: Wrong Result, Invented Content, Duplicate, Weak Result.
After the table, list missing scenarios under "Gaps". Cover boundary,
negative, role and state-based cases.
If the requirement does not answer something, list it under
"Questions". Do not invent product behaviour.
Requirement:
[paste requirement]
Test cases:
[paste test cases]An AI review is only a second opinion. It can miss problems, and it can report problems that do not exist. You make the final decision.
⚠️ Common Mistakes
- Reading the AI’s list before forming your own view
- Judging test cases by how many there are
- Approving test cases because they are neatly formatted
- Deleting every unsupported item without asking why it appeared
- Accepting vague results such as “works as expected”
🎯 Key Takeaways
- Look for gaps, invented content and wrong results.
- Write your own short risk list before reading the AI’s answer.
- Expected results deserve the closest attention.
- Clear, specific feedback improves the next AI answer.
🎤 Interview Questions
1. How do you review AI-generated test cases?
Model answer: I first write down my own key risks. Then I check for gaps, trace each case to the requirement to catch invented content, and verify every expected result against the business rules. I also remove duplicates and check the priorities.
2. What is the most dangerous mistake in AI-generated test cases?
Model answer: A wrong expected result is especially dangerous because it can make a defect look like correct behaviour. If automated, that test may pass and hide the problem. That is why I verify expected results carefully.
3. What would you do if AI generated a test for a rule not in the requirement?
Model answer: I would not include it as an approved test yet. I would decide whether it was invented or whether it exposes a real gap. If it might be important, I would ask the product owner to confirm the expected behaviour.
4. How do you find coverage gaps in AI output?
Model answer: I apply techniques such as boundary value analysis and equivalence partitioning, and I look at roles, states and failure paths. I also compare the output with my own risk list and map tests back to each requirement.
5. Can you use AI to review AI-generated test cases?
Model answer: Yes, as a second opinion, ideally in a separate chat with the requirement included. It can suggest issues, but it can also miss some or be wrong. I use its findings as input and make the final decision myself.
📝 Assignment
Ask AI for 15 test cases for a feature with at least three rules. Review them using the three checks, and log every finding by type. Calculate what percentage of the AI’s cases you accepted without changes. Keep this number, because measuring acceptance will matter again in later levels.
❓ FAQ
How long should a review take?
It depends on the risk of the feature. A payment flow deserves a careful review. Time spent here is still worthwhile because it prevents wrong tests from entering your suite.
Should I fix test cases myself or ask the AI?
For one or two small fixes, it is usually quicker to edit them yourself. For repeated issues, give the AI specific feedback and improve your prompt.
🚀 Next Steps
Well done on finishing this lesson. Complete the assignment, then continue with Build Your QA Prompt Library: Save and Reuse Prompts That Work.