MCP Tool Design for QA: Browser, File, Test Runner and Report Tools

A model can only be as careful as the tools that you hand it. A tool called run_command invites trouble, while run_login_tests is easy to control. Good MCP tool design gives a model few choices, and each one is safe and clear. In this lesson you will design the four tool families that your QA agent needs.

---

💡 MCP Tool Design Rules

Rule	Poor design	Better design
One clear job	do_testing	run_login_tests
Limited power	run_command	A fixed list of test commands
Strict input	Any string	An allowed list or pattern
Clear description	“Gets data”	“Returns failed test names. Read-only.”
Limited output	The full log	A summary and a file reference
Helpful errors	“Failed”	“Unknown project. Use chromium or firefox.”

---

🔍 The Four Tool Families

Family	Good first tools	Limits
Browser	Read a snapshot, take a screenshot	Approved hosts only, with test accounts
Filesystem	Read approved project files	Write only to the generated tests folder
Test runner	Run approved Playwright commands	Fixed commands, time limit, structured result
Reporting	Read report files, build a summary	Read-only, with secrets removed

These boundaries come from the project guide that you will follow in Level 8.

---

🧩 Reject Dangerous Input in Code

Two common risks are escaping from a folder and injecting extra commands. This program has no dependencies, and it shows both defences.

const PROJECTS = ['chromium', 'firefox'];

function safeReportPath(requested: string): string | null {
  if (/^[/\\]/.test(requested) || requested.includes(':') || requested.includes('\\')) {
    return null;
  }
  const parts = requested.split('/');
  if (parts.some((part) => part === '' || part === '.' || part === '..')) {
    return null;
  }
  return `reports/${requested}`;
}

// Returns separate arguments, so nothing is interpreted by a shell.
function buildTestCommand(project: string, spec: string): string[] | null {
  if (!PROJECTS.includes(project)) {
    return null;
  }
  if (!/^tests\/[a-z0-9-]+\.spec\.ts$/.test(spec)) {
    return null;
  }
  return ['npx', 'playwright', 'test', spec, `--project=${project}`];
}

console.log(safeReportPath('run-17/summary.json'));
console.log(safeReportPath('../../secrets.env'));
console.log(buildTestCommand('chromium', 'tests/login.spec.ts')?.join(' '));
console.log(buildTestCommand('chromium', 'tests/login.spec.ts; rm -rf /'));

reports/run-17/summary.json
null
npx playwright test tests/login.spec.ts --project=chromium
null

In real code, also resolve the final path and confirm that it stays inside the folder. That check should follow symbolic links too.

---

🛠️ Return Results That Models Can Use

Return a short summary that a model can read, plus structured data for your code. For large outputs, return a file reference and not the whole file. Long logs fill the model’s context and can hide the real error.

{
  "runId": "run-17",
  "exitCode": 1,
  "passed": 12,
  "failed": 1,
  "reportPath": "reports/run-17/index.html"
}

---

📌 Remember State with a Handle

The newest MCP revision has no protocol-level sessions. If a tool needs to continue earlier work, such as an open browser, it should return a handle. The model passes that handle to the next call. Make handles hard to guess, give them an expiry and return a clear error when one is unknown.

---

🔑 Add Tools in the Right Order

1. Start with one read-only browser inspection tool.
2. Add only the tools needed for your current workflow.
3. Define input schemas, output shapes, time limits and errors.
4. Add an allowlist for hosts and actions.
5. Add write and test-execution tools only after approvals work.
6. Log the tool name, redacted inputs, duration and status.

---

🌍 Real-Life Example: Too Many Similar Tools

A server offers get_report, fetch_report, read_report and load_report. The model picks between them almost at random. The team merges them into one tool with a clear description, and wrong calls drop. You can measure that kind of change using an LLM evaluation.

---

🧪 Try It Yourself

1. Add a test for safeReportPath using a Windows-style path.
2. Try a spec name containing a space.
3. Add a third allowed project and test it.
4. Rewrite one vague tool description from your own server.
5. Use the prompt below to review your tool list.

---

📋 Ready-to-Copy Prompt

You are a senior engineer reviewing MCP tools for a QA agent.

Tool list:
[paste names, descriptions and input schemas]

For each tool, check:
- Does it do one clear job?
- Is it read-only, or can it change something?
- Can its inputs reach anything outside the approved scope?
- Is the description clear enough for a model?
- Is its output size limited?

Suggest tools to merge, split or remove. Do not add a general command tool.
Return a table with: Tool, Problem, Risk and Better Design.

---

⚠️ Common Mistakes

• Exposing a general shell or file tool
• Building commands by joining strings
• Letting the model choose any file path
• Giving several tools nearly the same purpose
• Returning huge logs directly to the model

---

🎯 Key Takeaways

• Each tool should have one narrow job.
• Enforce input limits in code, not only in schemas.
• Keep the four tool families and their limits separate.
• Add powerful tools only after approvals work.

---

🎤 Interview Questions

1. What makes a good MCP tool?

Model answer: It has one clear purpose, a strict input schema, an accurate description, limited output and helpful errors. It also has only the permissions that it needs.

2. How would you design a tool to run tests?

Model answer: I would expose a fixed set of approved commands with validated options and a time limit. It would return structured results, and it would never accept a raw shell command.

3. How do you prevent path traversal in a file tool?

Model answer: I would reject unsafe input, resolve the full path and confirm that it stays inside the approved folder, following symbolic links. I would also limit file types and size.

4. Why should tools limit their output?

Model answer: Large outputs use up context, cost more and can bury the useful information. I return a summary and a reference to the full artifact.

5. In what order would you add tools to a QA agent?

Model answer: I would start with read-only inspection, add only what the workflow needs, then add allowlists. Write and execution tools come last, once human approval works.

---

📝 Assignment

Design one tool from each family for a login test workflow. For each, write its name, description, input schema, output, time limit, permissions and one test that tries to misuse it. Check the design against the agent building blocks.

---

❓ FAQ

How many tools should one server have?

As few as your workflow needs. Too many similar tools make wrong choices more likely.

Is a schema enough to make a tool safe?

No. A schema checks the shape of the input, but your code must still enforce paths, hosts and permissions.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with MCP Security for Testers: Permissions, Allowlists and Approvals.