Multi-Agent Orchestration for QA: Workflows and Frameworks

Once one agent works, it is tempting to create ten. More agents can help, but they also add cost and new ways to fail. In this lesson you will learn what multi-agent orchestration really means for QA. You will build a small orchestrator that stops for human approval.

---

💡 What Is Multi-Agent Orchestration?

---

🔍 Choose the Simplest Design

Design	Use it when	QA example
Single agent	One job with a small set of tools	Summarise failures
Workflow	The order of steps is known	Requirement to reviewed tests
Multi-agent	Jobs need separate tools, permissions or independent review	Generator and separate reviewer

Good reasons to split include separate permissions, independent review and jobs that are too large for one context. “It looks more advanced” is not a good reason. More agents mean a bigger bill, more waiting and more handovers for you to test.

---

🧩 The Golden Rule: Code Controls the Order

In the project you will build, the guide uses one clear rule. The orchestrator controls the sequence, and agents are focused components. An agent cannot skip validation or approval, because normal code decides what happens next.

---

🛠️ Build a Small Orchestrator

Each function below stands in for one focused agent. This keeps the example free and repeatable.

type Run = {
  requirement: string;
  scenarios: string[];
  code: string;
  errors: string[];
};

type Stage = { name: string; run: (state: Run) => void };

const stages: Stage[] = [
  {
    name: 'parse',
    run: (state) => {
      if (state.requirement.trim() === '') {
        state.errors.push('Requirement is empty');
      }
    },
  },
  {
    name: 'plan',
    run: (state) => {
      state.scenarios = [`${state.requirement}: valid`, `${state.requirement}: invalid`];
    },
  },
  {
    name: 'generate',
    run: (state) => {
      state.code = state.scenarios.map((title) => `test('${title}')`).join('\n');
    },
  },
  {
    name: 'validate',
    run: (state) => {
      if (state.code.includes('waitForTimeout')) {
        state.errors.push('Fixed sleeps are not allowed');
      }
    },
  },
];

function orchestrate(requirement: string, humanApproved: boolean): string {
  const state: Run = { requirement, scenarios: [], code: '', errors: [] };

  for (const stage of stages) {
    stage.run(state);
    if (state.errors.length > 0) {
      return `Stopped at ${stage.name}: ${state.errors.join('; ')}`;
    }
    console.log(`Finished: ${stage.name}`);
  }

  if (!humanApproved) {
    return `Waiting: ${state.scenarios.length} scenarios need human approval`;
  }
  return 'Approved: ready to run in the test environment';
}

console.log(orchestrate('login', false));
console.log(orchestrate('', true));

Finished: parse
Finished: plan
Finished: generate
Finished: validate
Waiting: 2 scenarios need human approval
Stopped at parse: Requirement is empty

The second run proves that a bad input stops the whole process before any later stage runs.

---

📌 Handoffs Need Contracts

Use the checks from LLM Structured Output at every handoff. Save the output of each stage too. When a run fails, you can then see exactly where it went wrong.

---

🔑 Do You Need a Framework?

Frameworks such as LangGraph can manage state, retries and handoffs for you. They can be useful for complex flows. They also add concepts to learn and can hide what is happening. Build the small version first, then adopt a framework if a real problem justifies it.

Build your own when	Consider a framework when
The flow is short and clear	The flow has many branches
You are still learning	You need saved, resumable runs
You want few dependencies	The team already knows the framework

---

🌍 Real-Life Example: The Polite Reviewer

A generator and a reviewer share the same prompts and context, so the reviewer agrees with nearly everything. Giving the reviewer its own instructions and only the final code improves it. The team also adds normal checks, such as a type check. Independent review needs real independence.

---

🧪 Try It Yourself

1. Run the example.
2. Change the generate stage so that it includes waitForTimeout.
3. Confirm that the run stops at validate.
4. Add a stage that saves each output to an array.
5. Draw your own QA workflow and mark the human gates.

---

📋 Ready-to-Copy Prompt

You are a senior AI systems architect reviewing a QA agent design.

Goal:
[describe the goal]

Proposed agents and tools:
[list them]

For each agent, decide whether it should remain separate, be merged or
become normal code. Explain why.

Then give me:
1. The simplest workable sequence
2. The data passed at each handoff
3. Where validation is needed
4. Where a human must approve
5. How the system stops safely

Do not recommend a framework unless the design clearly needs one.

---

⚠️ Common Mistakes

• Starting with many agents before one works
• Letting agents decide to skip validation or approval
• Passing unvalidated text between agents
• Ignoring the cost of extra handovers
• Adopting a framework before understanding the flow

---

🎯 Key Takeaways

• Most QA tasks do not begin with multiple agents.
• Code should control the order of important steps.
• Validate each handoff between agents.
• Use a framework only for a real need.

---

🎤 Interview Questions

1. What is multi-agent orchestration?

Model answer: It is the coordination of several focused agents, including who works when, what data they exchange and when the process stops. A controller manages that flow.

2. When would you choose multiple agents over one?

Model answer: I would split when jobs need different tools or permissions, need independent review or do not fit in one context. Otherwise, I would keep the simpler design.

3. Why should code control a QA workflow?

Model answer: Deterministic control ensures that validation and human approval cannot be skipped. It also makes runs easier to reproduce, test and debug.

4. How do you make handoffs between agents reliable?

Model answer: I use defined schemas, validate each output before passing it on and store each stage’s result. Then a failure can be traced to one specific step.

5. When would you use an orchestration framework?

Model answer: I would use one for complex branching, long-running state or team standards. For short flows, plain code is easier to understand and has fewer dependencies.

---

📝 Assignment

Design a three-agent flow for reviewing a failed test: collector, analyst and reporter. Write the input and output of each agent, the permissions and one human gate. Then explain whether plain code could replace any of them. Bring it to the Level 8 project.

---

❓ FAQ

Is multi-agent always better than a single agent?

No. It often costs more and is harder to debug. Choose it only when you can name the benefit.

Do the agents need to talk to each other directly?

No. In this course, they pass data through the orchestrator. This keeps the flow clear and easy to check.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with What Is an MCP Server? A Simple Guide for Software Testers.