AI Agent Guardrails: Contracts, Context and Safety Limits for QA

Agents become reliable when their inputs and outputs are precise. They become safe when code enforces their limits. In this lesson you will write the contracts that connect your agents, then add the AI agent guardrails that every later step depends on.

---

💡 Contracts Come First

Create src/schemas/scenario.schema.ts. Each scenario links back to the requirements it covers.

import * as z from 'zod';

export const ScenarioSchema = z.object({
  id: z.string().regex(/^TS-\d+$/),
  title: z.string().min(1),
  requirementIds: z.array(z.string()).min(1),
  priority: z.enum(['P0', 'P1', 'P2', 'P3']),
  preconditions: z.array(z.string()),
  steps: z
    .array(z.object({ action: z.string().min(1), expectedResult: z.string().min(1) }))
    .min(1),
  testDataReference: z.string().optional(),
  automationCandidate: z.boolean(),
  riskNotes: z.array(z.string()),
});

export const ScenarioPlanSchema = z.object({
  scenarios: z.array(ScenarioSchema).min(1),
});

export type TestScenario = z.infer<typeof ScenarioSchema>;
export type ScenarioPlan = z.infer<typeof ScenarioPlanSchema>;

The Zod schema checks data while the program runs, and z.infer creates the TypeScript type from it. You write the shape once.

---

🔍 Shared Types and Context

Create src/core/types.ts.

export type ProjectContext = {
  applicationName: string;
  baseUrl: string;
  environment: 'local' | 'test' | 'staging';
  framework: 'playwright';
  language: 'typescript';
  conventions: string[];
  allowedDomains: string[];
  prohibitedActions: string[];
  approvalRequiredFor: string[];
};

export type PageElement = { role: string; name: string };

export type PageObservation = {
  url: string;
  title: string;
  objective: string;
  elements: PageElement[];
};

export type FailedTest = {
  title: string;
  error: string;
  tracePaths: string[];
  screenshotPaths: string[];
};

export type TestRunResult = {
  runId: string;
  startedAt: string;
  finishedAt: string;
  exitCode: number;
  passed: number;
  failed: number;
  skipped: number;
  reportPath?: string;
  tracePaths: string[];
  screenshotPaths: string[];
  failures: FailedTest[];
};

Create src/core/agent.ts. Every agent has the same simple shape.

export interface Agent<Input, Output> {
  readonly name: string;
  run(input: Input): Promise<Output>;
}

Create src/config/project.ts. This gives all agents the same facts about the project.

import type { ProjectContext } from '../core/types.js';
import type { Env } from './env.js';

export function buildProjectContext(env: Env): ProjectContext {
  return {
    applicationName: 'Demo Login',
    baseUrl: env.BASE_URL,
    environment: 'local',
    framework: 'playwright',
    language: 'typescript',
    conventions: [
      'Use getByRole or getByLabel locators',
      'Read credentials from process.env.TEST_USER_EMAIL and process.env.TEST_USER_PASSWORD',
      'Never use fixed sleeps',
    ],
    allowedDomains: [new URL(env.BASE_URL).host],
    prohibitedActions: ['Testing production', 'Running shell commands from a model'],
    approvalRequiredFor: ['Writing generated code', 'Running generated tests', 'Applying patches'],
  };
}

---

🧩 AI Agent Guardrails in Code

You met this idea in AI Agent Building Blocks. Create src/core/guardrails.ts.

export class GuardrailError extends Error {}

export const LIMITS = {
  maxWorkflowSteps: 20,
  commandTimeoutMs: 120_000,
  llmTimeoutMs: 60_000,
  maxHealingAttempts: 1,
};

export function isAllowedUrl(url: string, allowedHosts: string[]): boolean {
  try {
    const parsed = new URL(url);
    return ['http:', 'https:'].includes(parsed.protocol) && allowedHosts.includes(parsed.host);
  } catch {
    return false;
  }
}

export function isSafeGeneratedPath(path: string): boolean {
  return /^tests\/generated\/[a-z0-9-]+\.spec\.ts$/.test(path);
}

export function redact(text: string): string {
  return text
    .replace(/\b(?:sk|key|token)[-_][A-Za-z0-9_-]{12,}\b/g, '[REDACTED]')
    .replace(/\bBearer\s+[A-Za-z0-9._-]+/gi, '[REDACTED]')
    .replace(/\b(password|passwd|pwd|api[_-]?key)(\s*[=:]\s*)\S+/gi, '$1$2[REDACTED]');
}

Guardrail	What it stops
isAllowedUrl	Visiting another host, even a look-alike
isSafeGeneratedPath	Writing outside the generated tests folder
redact	Secrets appearing in logs and reports
LIMITS	Runaway loops and hung commands

---

🛠️ Test Your Guardrails

Guardrails are the code that most needs tests. Create tests/unit/guardrails.test.ts.

import assert from 'node:assert/strict';
import { test } from 'node:test';
import { isAllowedUrl, isSafeGeneratedPath, redact } from '../../src/core/guardrails.js';

const hosts = ['127.0.0.1:4321'];

test('only the approved host is allowed', () => {
  assert.equal(isAllowedUrl('http://127.0.0.1:4321/login', hosts), true);
  assert.equal(isAllowedUrl('http://127.0.0.1:9999/login', hosts), false);
  assert.equal(isAllowedUrl('https://127.0.0.1:4321.evil.test/', hosts), false);
  assert.equal(isAllowedUrl('file:///etc/passwd', hosts), false);
  assert.equal(isAllowedUrl('not a url', hosts), false);
});

test('generated files stay in the approved folder', () => {
  assert.equal(isSafeGeneratedPath('tests/generated/login.spec.ts'), true);
  assert.equal(isSafeGeneratedPath('tests/generated/../../.env'), false);
  assert.equal(isSafeGeneratedPath('src/app.ts'), false);
});

test('secrets are removed from text', () => {
  assert.equal(redact('password=hunter2 ok'), 'password=[REDACTED] ok');
  assert.equal(redact('Authorization: Bearer abc.def'), 'Authorization: [REDACTED]');
});

Run the tests.

npm run test:unit

Look at the third URL in the first test. A careless check using startsWith would allow it. You practised this in MCP Security for Testers.

---

📌 The Minimum Guardrails

• Allow only the target host, and reject redirects elsewhere.
• Never test production without a separately approved policy.
• Use synthetic test accounts and data.
• Never run shell commands returned by a model.
• Validate every generated file path.
• Require approval before generated code is written or run.
• Set limits on time, retries and workflow steps.
• Record which model, prompt and evidence produced each result.

---

🌍 Real-Life Example: The Look-Alike Host

An agent is told to stay on the test site, and the check is url.startsWith('https://test.shop.com'). A link to https://test.shop.com.attacker.example passes the check. Parsing the URL and comparing the exact host closes that gap for good.

---

🧪 Try It Yourself

1. Create all the files and run the unit tests.
2. Add a test for a URL with a different port.
3. Add a test for a path that uses a backslash.
4. Add a new secret format to redact, and test it.
5. Run npm run typecheck.

---

📋 Ready-to-Copy Prompt

You are a security-minded QA engineer reviewing guardrail code.

Code:
[paste your guardrails with no secrets]

Try to find inputs that get past each function. Consider odd URLs, unusual
paths and secrets in different formats.

Return a table with: Function, Input That Gets Through, Risk and Fix.
Show a unit test for each problem that you find.

---

⚠️ Common Mistakes

• Letting agents exchange free text
• Writing a TypeScript type with no runtime check
• Relying on prompt instructions for safety
• Checking URLs with startsWith
• Leaving guardrails untested

---

🎯 Key Takeaways

• Contracts make agents replaceable and testable.
• One schema can give you both types and runtime checks.
• Guardrails belong in code and need their own tests.
• All page content is untrusted data.

---

🎤 Interview Questions

1. What is an agent contract?

Model answer: It is the defined shape of an agent’s input and output, checked at runtime. It lets agents be developed and tested independently.

2. What guardrails would you add to a QA agent?

Model answer: I would add host allowlists, safe output paths, command allowlists, secret redaction, approvals and limits on time, retries and steps. Each is enforced in code.

3. Why is startsWith unsafe for URL checks?

Model answer: A hostile address can begin with the approved text and continue to another domain. I parse the URL and compare the exact host.

4. What is the danger of untrusted page content?

Model answer: A page can include text aimed at the model, such as instructions to reveal data. I label it as data and keep all permissions in code.

5. Why record the model and prompt version?

Model answer: It makes results traceable. When quality changes, I can tell whether the model, a prompt or the application caused it.

---

📝 Assignment

Add a function that rejects any BASE_URL whose host is on a list of known production hosts. Write three unit tests, and call it from buildProjectContext.

---

❓ FAQ

Why Zod and not a hand-written validator?

You learned the idea by hand in Level 6. Zod is shorter and gives clearer errors for larger contracts.

Is the redact function complete?

No function can catch every secret format. It is one layer, so also avoid sending secrets to the model at all.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with AI Requirement Parser: Turn User Stories into Structured Data.