Build an AI Agent from Scratch: Your First One-Tool QA Agent

The best way to understand agents is to write one. In this lesson you will build an AI agent from scratch, with no framework and only one tool. It reads a test report and summarises the failures. It is small on purpose, because every real agent is this same idea with more parts.

---

💡 Plan a Tiny Agent

Block	Our agent
Role	Test report summariser
Goal	Summarise the failed tests
Tool	listFailedTests, which only reads data
Guardrails	Tool allowlist and step limit
Output	One short summary

---

🔍 Why Read-Only First?

---

🧩 Build an AI Agent from Scratch

This complete example has no dependencies.

type TestResult = { name: string; status: 'passed' | 'failed'; error?: string };
type Decision =
  | { type: 'tool'; tool: string }
  | { type: 'finish'; answer: string };

const report: TestResult[] = [
  { name: 'login with valid user', status: 'passed' },
  { name: 'login with wrong password', status: 'failed', error: 'Timeout' },
  { name: 'logout', status: 'passed' },
];

// The agent's only tool. It reads data and changes nothing.
function listFailedTests(): TestResult[] {
  return report.filter((test) => test.status === 'failed');
}

// A fake model, so this example is free and repeatable.
function fakeModel(goal: string, observations: TestResult[][]): Decision {
  if (observations.length === 0) {
    return { type: 'tool', tool: 'listFailedTests' };
  }
  const failed = observations[0];
  const names = failed.map((test) => test.name).join(', ');
  return {
    type: 'finish',
    answer: `${failed.length} failed: ${names}. Check the evidence before naming a cause.`,
  };
}

const ALLOWED_TOOLS = ['listFailedTests'];
const MAX_STEPS = 3;

function runAgent(goal: string): string {
  const observations: TestResult[][] = [];

  for (let step = 1; step <= MAX_STEPS; step++) {
    const decision = fakeModel(goal, observations);

    if (decision.type === 'finish') {
      return decision.answer;
    }
    if (!ALLOWED_TOOLS.includes(decision.tool)) {
      return `Stopped: tool not allowed: ${decision.tool}`;
    }

    observations.push(listFailedTests());
    console.log(`Step ${step}: called ${decision.tool}`);
  }

  return 'Stopped: reached the step limit';
}

console.log(runAgent('Summarise the failed tests'));

Step 1: called listFailedTests
1 failed: login with wrong password. Check the evidence before naming a cause.

---

🛠️ What Each Part Does

Part	Job
listFailedTests	The tool, which your code controls
fakeModel	Decides the next step
ALLOWED_TOOLS	Rejects any other tool request
MAX_STEPS	Guarantees that the run ends
runAgent	The agent loop that connects them

---

📌 Test Your Agent

Change fakeModel to try each of these:

• Ask for a tool called deleteTests, and confirm that it is refused.
• Always ask for a tool, and confirm that the step limit stops it.
• Empty the report, and decide what the summary should say.

You can apply the checks from Testing AI Agents here.

---

🔑 Replacing the Fake Model

A real model is an API call that receives the goal, tool descriptions and earlier results. It returns either a tool request or a final answer. Your loop and guardrails stay the same. In the next lesson, you will learn to validate what comes back, and Level 8 connects a real provider safely.

---

🌍 Real-Life Example: A Tempting Extra Tool

A tester’s agent works well, so they add a tool that deletes old reports. A wrong model decision now has real consequences. A safer design keeps the agent read-only and asks a person to carry out the deletion. Each new tool needs its own risk review.

---

🧪 Try It Yourself

1. Set up a project with Node.js and tsx.
2. Save the example as agent.ts.
3. Run it with npx tsx agent.ts.
4. Try each of the three bad situations above.
5. Add a tool that counts passed tests, and add it to the allowlist.

---

📋 Ready-to-Copy Prompt

You are a senior engineer reviewing my first AI agent.

Agent code:
[paste code with no secrets]

Check for: a missing stop condition, unsafe tools, tool requests that
are not validated, unhandled errors and places where results are trusted
without evidence.

Return a table with: Issue, Why It Matters and Smallest Fix.

Rules:
- Do not add a framework.
- Do not add new tools.
- Keep the agent read-only.

---

⚠️ Common Mistakes

• Beginning with write access or shell commands
• Trusting a tool name without checking the allowlist
• Adding a real model before testing the loop
• Letting a summary state a root cause without evidence
• Adding many tools before one works reliably

---

🎯 Key Takeaways

• A useful first agent can be very small.
• Start with read-only tools.
• Allowlists and step limits belong in code.
• A fake model lets you test the logic for free.

---

🎤 Interview Questions

1. How would you build an AI agent without a framework?

Model answer: I would write a loop that gets a decision from the model, validates and runs the requested tool, adds the result to the history and stops at completion or a limit.

2. Why start with read-only tools?

Model answer: They limit the possible damage while the agent is being developed. I add write or execution tools only after validation, permissions and human approval work.

3. How do you test agent logic without paying for a model?

Model answer: I replace the model with a scripted fake that returns chosen decisions. This makes tests fast, free and repeatable, including failure cases that are difficult to trigger.

4. What is a tool allowlist?

Model answer: It is a fixed list of tools that the agent can use. Any request outside it is rejected in code, which protects the system even if the model asks for something unexpected.

5. What is the benefit of building an agent from scratch?

Model answer: It teaches how the loop, tools and limits really work, and it makes later debugging easier. I can then decide with evidence whether a framework would add value.

---

📝 Assignment

Extend the agent with a second read-only tool that returns the error for one named test. Update the allowlist, handle an unknown test name and write three tests using a fake model. Use results from Playwright reporters as inspiration for your data.

---

❓ FAQ

Do I need an API key for this lesson?

No. The fake model means that you need only Node.js and a TypeScript runner.

Why TypeScript?

It fits the Playwright project in this course and its types make tool inputs and decisions clearer. The same design works in Python.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with LLM Structured Output: Validate AI Responses with a Schema.