Generative AI and LLMs for Software Testers, Explained Simply
AI words appear in every QA job description now, but you do not need a maths degree to understand them. This simple guide to generative AI and LLMs for software testers explains what each term means, how these systems differ from the software you test every day, and why a tester’s mindset is a real advantage.
📖 What You’ll Learn
- The difference between AI, ML, deep learning, generative AI and LLMs
- Why AI-based software behaves differently from rule-based software
- Where testers already meet AI at work
- How to explain these terms clearly in an interview
💡 Why Testers Should Learn AI Basics
Testers meet AI in two ways. You use AI to do your work faster, such as drafting test cases or explaining an error. You also test AI when your product adds a chatbot, a recommendation engine or a smart search. Both need the same foundation: knowing what these systems are and how they fail.
🔍 What Are AI, ML, Generative AI and LLMs?
Explain Like I’m 5: AI
Imagine: A toy robot sorts your blocks by colour without your help. It looks like it is thinking.
What it really means: Software that does jobs we usually connect with human thinking.
In testing: A tool picks which tests to run first.
Explain Like I’m 5: Machine Learning
Imagine: Nobody gave you a list of rules for spotting a dog. You saw lots of dogs, and then you just knew.
What it really means: Software that learns from many examples. It does not only follow written rules.
In testing: A model studies old bugs and guesses where new bugs may hide.
Explain Like I’m 5: Deep Learning
Imagine: Friends stand in a line and pass a picture along. One spots lines. The next spots shapes. The last one shouts, “It’s a cat!”
What it really means: Machine learning with many layers. Each layer learns a little more than the last.
In testing: A visual tool recognises a button in a screenshot.
Explain Like I’m 5: Generative AI
Imagine: You have heard many bedtime stories. Now you can make up a new one of your own.
What it really means: AI that makes new text, code or pictures after learning from lots of examples.
In testing: AI writes a first draft of a bug report.
Explain Like I’m 5: LLM
Imagine: You have a friend who has read nearly every book. You say, “Once upon a…” and your friend says, “time!” They keep guessing the next word until the story ends.
What it really means: A generative AI trained on huge amounts of text. It predicts what text comes next.
In testing: ChatGPT, Claude or Gemini answers your question about Playwright.
Think of these terms as circles inside circles. AI is the biggest circle, and LLMs sit inside it. Here is the same list as a quick table.
| Term | Plain meaning | Example a tester will recognise |
|---|---|---|
| Artificial Intelligence (AI) | Any software that performs tasks we normally connect with human thinking | A tool that decides which tests to run first |
| Machine Learning (ML) | Software that learns patterns from data instead of following only hand-written rules | A model that predicts which module is likely to have defects |
| Deep Learning | ML that uses large, layered neural networks | Visual testing that recognises a button in a screenshot |
| Generative AI (GenAI) | AI that creates new content such as text, code or images | Creating test data or a draft bug report |
| Large Language Model (LLM) | A GenAI model trained on huge amounts of text that predicts what text should come next | ChatGPT, Claude or Gemini answering a question about Playwright |
Remember this one line
An LLM is a kind of generative AI, generative AI mostly uses deep learning, deep learning is a kind of machine learning, and machine learning is a kind of AI.
🧩 Rule-Based Software vs AI-Based Software
Explain Like I’m 5: Rule-Based vs AI-Based
Imagine: A light switch is rule-based. Press it and the light comes on, every time. A puppy is AI-based. It learned from you, and it may not do the same thing every time.
What it really means: Rule-based software follows fixed instructions. AI-based software learned its behaviour from data.
In testing: The switch has one right answer. With the puppy, you have to judge if it behaved well.
Most applications you have tested are rule-based. A developer writes the rule, and you verify it. AI-based software learns its behaviour, and that changes how you test it.
| Point | Rule-based software | AI-based software |
|---|---|---|
| How behaviour is created | Developers write the logic | The model learns from data |
| Same input, same output? | Normally yes | Often no |
| Expected result | Usually one exact value | Often a range of acceptable answers |
| Typical defect source | Code, configuration, data | Training data, prompts, model version, code |
| Typical check | expect(total).toBe(500) | Is the answer correct, safe and relevant? |
🛠️ Where You Already Meet AI at Work
- Chat assistants that draft test cases, SQL queries or bug reports
- Coding assistants in VS Code that suggest Playwright or Selenium code
- Visual testing tools that compare screens
- Self-healing tools that try to repair a broken locator
- Product features such as chatbots, fraud detection and product recommendations
You can see named examples in AI Tools for QA Engineers.
🌍 Real-Life Example: A Login Story
Imagine this requirement: a user can log in with an email and password, and the account locks after 5 failed attempts.
A rule-based login check has an exact expected result. After 5 wrong passwords, the account must be locked. Now imagine the same site has an AI support bot, and a user asks, “Why can’t I log in?” There is no single correct sentence. You check whether the reply is accurate, mentions the lock if relevant, avoids exposing another user’s data and stays polite.
Same product, two very different test approaches. That difference is the main idea of this lesson.
🧪 Try It Yourself
- Open any AI chat assistant you are allowed to use.
- Paste the prompt below.
- Read the answer and mark each scenario as rule-based or AI-based yourself before you look at the AI’s label.
- Note any place where you disagree with the AI, and decide who is right.
📋 Ready-to-Copy Prompt
You are a senior QA engineer teaching a beginner.
Here are 6 features of an e-commerce website:
1. Add item to cart
2. "Recommended for you" product list
3. Coupon code validation
4. Customer support chatbot
5. Order total calculation
6. Search results ranked by relevance
For each feature, tell me:
- Whether it is most likely rule-based or AI-based
- How the expected result would be defined
- One test idea
Return the answer as a table. If you are unsure about a feature, say so
and explain what you would need to know.⚠️ Common Mistakes
- Using “AI” and “LLM” as if they mean the same thing
- Assuming AI output is correct because it sounds confident
- Assuming every “smart” feature in a product is really AI, without asking how it works
- Testing an AI feature with exact text matches only
🎯 Key Takeaways
- AI contains ML, ML contains deep learning, and LLMs are one type of generative AI.
- AI-based software learns behaviour from data, so its output can vary.
- Testers both use AI and test AI, and this course covers both.
🎤 Interview Questions
1. What is the difference between AI and machine learning?
Model answer: AI is the broad goal of making software perform tasks that normally need human intelligence. Machine learning is one way to achieve it, where the system learns patterns from data instead of relying only on hand-written rules. All ML is AI, but not all AI is ML. A fixed rules engine could be called AI without learning anything.
2. What is generative AI?
Model answer: Generative AI creates new content such as text, code or images based on patterns it learned during training. In QA, I would use it to draft test scenarios or synthetic test data, and I would review the output before using it.
3. What is an LLM?
Model answer: A Large Language Model is a generative model trained on a very large amount of text. It predicts the most suitable next piece of text based on what came before. That is why it writes fluently, and also why it can be wrong while sounding confident.
4. How is testing an AI feature different from testing a normal feature?
Model answer: A normal feature usually has one exact expected result, so I can assert a specific value. An AI feature may give different valid answers, so I define acceptance criteria such as correctness, relevance and safety, then evaluate against a prepared set of examples.
5. How do you use AI as a tester?
Model answer: I use it as an assistant for first drafts, such as test ideas, test data and bug report wording. I stay responsible for the result, so I check it against the requirement, remove anything invented and add the domain cases it missed. Replace this with a real example from your own work.
📝 Assignment
Pick one application you use daily, such as a shopping or banking app. List 5 features, label each one rule-based or AI-based, and write how you would define the expected result for each. Keep this file, because you will extend it later in the course.
❓ FAQ
Do I need to know maths or coding for this level?
No. Level 1 needs no coding and no maths. Coding starts in Level 4, and the site’s Playwright with TypeScript course can help you prepare.
Is ChatGPT the same as AI?
No. ChatGPT is one product built on LLMs, and LLMs are only one part of the wider AI field.
🚀 Next Steps
Well done on finishing this lesson. Complete the assignment, then continue with How LLMs Work: Tokens, Context and Why Answers Change.