AI Agent Architecture for Production: API, Queue, Workers and Storage

A browser run can take minutes, so a request cannot simply wait for it to finish. A production AI agent architecture splits the work: one part accepts requests, and another does the slow work in the background. In this lesson you will learn each part, then build the database that also acts as your job queue.

---

💡 The Production AI Agent Architecture

Team member or tool
       |
       v
      API  ---> Database (runs, approvals, audit log)
                    ^
                    |
                 Worker ---> Orchestrator ---> Browser and tests
                    |
                    v
          Artifact storage (traces, screenshots, reports)

Part	Job
API	Accepts requests and shows status
Job queue	Holds work that is waiting
Worker	Runs one job at a time
Database	Stores runs, approvals and the audit log
Artifact storage	Keeps traces, screenshots and reports

Your AI agent orchestrator runs inside the worker, and it needs no changes.

---

🔍 Start Small

Stage	Database and queue	When it fits
First release	SQLite on one server	One team, one worker
Growing	PostgreSQL with a job table	Several servers or workers
Large	A managed queue and object storage	Many teams

We use SQLite, which is built into Node.js 22 and later. It needs no extra server. Its limit is clear: it suits a single server. Because your code uses a small store class, you can replace it later without rewriting the API or worker.

---

🧩 Build the Store

Create src/service/store.ts.

import { randomUUID } from 'node:crypto';
import { mkdirSync } from 'node:fs';
import { dirname } from 'node:path';
import { DatabaseSync } from 'node:sqlite';

export type RunStatus = 'queued' | 'running' | 'completed' | 'stopped' | 'interrupted';
export type ApprovalStatus = 'pending' | 'approved' | 'rejected' | 'expired';

export type RunRow = {
  id: string;
  status: RunStatus;
  requirement: string;
  environment: string;
  requested_by: string;
  stop_reason: string | null;
  report_path: string | null;
  state_json: string | null;
  lease_until: number | null;
  created_at: number;
};

export type ApprovalRow = {
  id: string;
  run_id: string;
  action: string;
  summary: string;
  status: ApprovalStatus;
  decided_by: string | null;
  expires_at: number;
};

export type TokenRow = { id: string; name: string; role: string; revoked_at: number | null };

const SCHEMA = `
CREATE TABLE IF NOT EXISTS runs (
  id TEXT PRIMARY KEY, status TEXT NOT NULL, requirement TEXT NOT NULL,
  environment TEXT NOT NULL, requested_by TEXT NOT NULL, idempotency_key TEXT UNIQUE,
  stop_reason TEXT, report_path TEXT, state_json TEXT, lease_until INTEGER,
  created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS approvals (
  id TEXT PRIMARY KEY, run_id TEXT NOT NULL, action TEXT NOT NULL, summary TEXT NOT NULL,
  status TEXT NOT NULL, decided_by TEXT, decided_at INTEGER, expires_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS audit_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT, at INTEGER NOT NULL, actor TEXT NOT NULL,
  action TEXT NOT NULL, run_id TEXT, detail TEXT
);
CREATE TABLE IF NOT EXISTS api_tokens (
  id TEXT PRIMARY KEY, name TEXT NOT NULL, role TEXT NOT NULL,
  token_hash TEXT NOT NULL UNIQUE, revoked_at INTEGER
);
CREATE TABLE IF NOT EXISTS settings (key TEXT PRIMARY KEY, value TEXT NOT NULL);
`;

export class ServiceStore {
  private readonly db: DatabaseSync;

  constructor(path: string) {
    if (path !== ':memory:') {
      mkdirSync(dirname(path), { recursive: true });
    }
    this.db = new DatabaseSync(path);
    this.db.exec('PRAGMA journal_mode = WAL; PRAGMA busy_timeout = 5000;');
    this.db.exec(SCHEMA);
  }

  ping(): boolean {
    return this.db.prepare('SELECT 1 AS ok').get() !== undefined;
  }

  createRun(input: {
    requirement: string;
    environment: string;
    requestedBy: string;
    idempotencyKey?: string;
  }): RunRow {
    const key = input.idempotencyKey ? `${input.requestedBy}:${input.idempotencyKey}` : null;
    this.db
      .prepare(
        `INSERT INTO runs (id, status, requirement, environment, requested_by, idempotency_key, created_at)
         VALUES (?, 'queued', ?, ?, ?, ?, ?) ON CONFLICT(idempotency_key) DO NOTHING`,
      )
      .run(`run-${randomUUID()}`, input.requirement, input.environment, input.requestedBy, key, Date.now());
    const row = key
      ? this.db.prepare('SELECT * FROM runs WHERE idempotency_key = ?').get(key)
      : this.db.prepare('SELECT * FROM runs ORDER BY rowid DESC LIMIT 1').get();
    return row as unknown as RunRow;
  }

  getRun(id: string): RunRow | undefined {
    return this.db.prepare('SELECT * FROM runs WHERE id = ?').get(id) as unknown as RunRow | undefined;
  }

  // One SQL statement, so two workers can never claim the same run.
  claimNextRun(leaseMs: number): RunRow | undefined {
    return this.db
      .prepare(
        `UPDATE runs SET status = 'running', lease_until = ?
         WHERE id = (SELECT id FROM runs WHERE status = 'queued' ORDER BY created_at, rowid LIMIT 1)
         RETURNING *`,
      )
      .get(Date.now() + leaseMs) as unknown as RunRow | undefined;
  }

  renewLease(id: string, leaseMs: number): void {
    this.db.prepare('UPDATE runs SET lease_until = ? WHERE id = ?').run(Date.now() + leaseMs, id);
  }

  finishRun(id: string, result: {
    status: RunStatus;
    stopReason?: string;
    reportPath?: string;
    stateJson?: string;
  }): void {
    this.db
      .prepare(
        `UPDATE runs SET status = ?, stop_reason = ?, report_path = ?, state_json = ?, lease_until = NULL
         WHERE id = ?`,
      )
      .run(result.status, result.stopReason ?? null, result.reportPath ?? null, result.stateJson ?? null, id);
  }

  // A crashed run may already have had side effects, so it is never retried automatically.
  interruptExpiredRuns(): number {
    const result = this.db
      .prepare(
        `UPDATE runs SET status = 'interrupted', stop_reason = 'The worker stopped during this run'
         WHERE status = 'running' AND lease_until < ?`,
      )
      .run(Date.now());
    return Number(result.changes);
  }

  createApproval(runId: string, action: string, summary: string, expiresAt: number): ApprovalRow {
    const id = `apr-${randomUUID()}`;
    this.db
      .prepare(
        `INSERT INTO approvals (id, run_id, action, summary, status, expires_at)
         VALUES (?, ?, ?, ?, 'pending', ?)`,
      )
      .run(id, runId, action, summary, expiresAt);
    return this.getApproval(id) as ApprovalRow;
  }

  getApproval(id: string): ApprovalRow | undefined {
    return this.db.prepare('SELECT * FROM approvals WHERE id = ?').get(id) as unknown as
      | ApprovalRow
      | undefined;
  }

  listPendingApprovals(): ApprovalRow[] {
    return this.db
      .prepare(`SELECT * FROM approvals WHERE status = 'pending' ORDER BY expires_at`)
      .all() as unknown as ApprovalRow[];
  }

  // Only a pending approval can change, so the first decision always wins.
  decideApproval(id: string, status: ApprovalStatus, decidedBy: string): boolean {
    const result = this.db
      .prepare(
        `UPDATE approvals SET status = ?, decided_by = ?, decided_at = ?
         WHERE id = ? AND status = 'pending'`,
      )
      .run(status, decidedBy, Date.now(), id);
    return result.changes === 1;
  }

  addAudit(actor: string, action: string, runId?: string, detail?: string): void {
    this.db
      .prepare('INSERT INTO audit_log (at, actor, action, run_id, detail) VALUES (?, ?, ?, ?, ?)')
      .run(Date.now(), actor, action, runId ?? null, detail ?? null);
  }

  listAudit(runId: string): Array<{ actor: string; action: string }> {
    return this.db
      .prepare('SELECT actor, action FROM audit_log WHERE run_id = ? ORDER BY id')
      .all(runId) as unknown as Array<{ actor: string; action: string }>;
  }

  addToken(name: string, role: string, tokenHash: string): void {
    this.db
      .prepare('INSERT INTO api_tokens (id, name, role, token_hash) VALUES (?, ?, ?, ?)')
      .run(randomUUID(), name, role, tokenHash);
  }

  findToken(tokenHash: string): TokenRow | undefined {
    return this.db
      .prepare('SELECT id, name, role, revoked_at FROM api_tokens WHERE token_hash = ?')
      .get(tokenHash) as unknown as TokenRow | undefined;
  }

  revokeToken(name: string): void {
    this.db.prepare('UPDATE api_tokens SET revoked_at = ? WHERE name = ?').run(Date.now(), name);
  }

  getSetting(key: string): string | undefined {
    const row = this.db.prepare('SELECT value FROM settings WHERE key = ?').get(key) as
      | { value: string }
      | undefined;
    return row?.value;
  }

  setSetting(key: string, value: string): void {
    this.db
      .prepare(
        `INSERT INTO settings (key, value) VALUES (?, ?)
         ON CONFLICT(key) DO UPDATE SET value = excluded.value`,
      )
      .run(key, value);
  }

  isPaused(): boolean {
    return this.getSetting('paused') === 'true';
  }

  close(): void {
    this.db.close();
  }
}

Design	What it prevents
claimNextRun is one statement	Two workers taking one job
idempotency_key is unique	A retried request creating a second run
Crashed runs become interrupted	Repeating side effects automatically
decideApproval needs a pending row	A later decision overriding the first
audit_log table	Losing track of who did what

---

🛠️ Test the Store

Create tests/unit/store.test.ts.

import assert from 'node:assert/strict';
import { test } from 'node:test';
import { ServiceStore } from '../../src/service/store.js';

const newRun = (store: ServiceStore, key?: string) =>
  store.createRun({ requirement: 'Login', environment: 'demo', requestedBy: 'asha', idempotencyKey: key });

test('a queued run can be claimed only once', () => {
  const store = new ServiceStore(':memory:');
  newRun(store);
  assert.equal(store.claimNextRun(60_000)?.status, 'running');
  assert.equal(store.claimNextRun(60_000), undefined);
});

test('the same idempotency key does not create a second run', () => {
  const store = new ServiceStore(':memory:');
  assert.equal(newRun(store, 'abc').id, newRun(store, 'abc').id);
});

test('a run with an expired lease is interrupted, not retried', () => {
  const store = new ServiceStore(':memory:');
  const run = newRun(store);
  store.claimNextRun(-1);
  assert.equal(store.interruptExpiredRuns(), 1);
  assert.equal(store.getRun(run.id)?.status, 'interrupted');
  assert.equal(store.claimNextRun(60_000), undefined);
});

test('the first decision on an approval wins', () => {
  const store = new ServiceStore(':memory:');
  const approval = store.createApproval(newRun(store).id, 'x', 'y', Date.now() + 1000);
  assert.equal(store.decideApproval(approval.id, 'rejected', 'ben'), true);
  assert.equal(store.decideApproval(approval.id, 'approved', 'cara'), false);
  assert.equal(store.getApproval(approval.id)?.status, 'rejected');
});

Run the tests with npm run test:unit.

---

🌍 Real-Life Example: The Double Run

A tester’s network drops just after they send a request, so their tool retries. Without protection, two identical runs start and file two sets of bugs. With an idempotency key, the second request simply gets the first run back. You learned why retries are risky in MCP Server Testing.

---

🧪 Try It Yourself

1. Create the store and run its tests.
2. Open a store with a file path, and create a run.
3. Stop your program and read the run again.
4. Add a test that claims two jobs in order.
5. Explain why crashed runs are not retried.

---

📋 Ready-to-Copy Prompt

You are a senior backend engineer reviewing a job queue built on one
database table.

Code:
[paste your store with no secrets]

Check whether two workers could ever claim the same job, whether a retry
could create a duplicate and what happens if a worker crashes.

Tell me the exact point at which this design would need PostgreSQL or a
managed queue. Do not suggest a change for a problem I do not have yet.

---

⚠️ Common Mistakes

• Making the caller wait for a browser run
• Adding several services before they are needed
• Retrying interrupted runs automatically
• Letting two runs share one output folder
• Skipping the audit log

---

🎯 Key Takeaways

• The API accepts work, and the worker does it.
• Start with one simple database.
• Claiming a job must be atomic.
• Keep a clear path for growing later.

---

🎤 Interview Questions

1. Why use a job queue for an AI agent?

Model answer: Runs are slow, so the API should accept a request and respond quickly. A queue holds the work, controls concurrency and survives restarts.

2. How do you stop two workers processing the same job?

Model answer: I claim it with one atomic update in the database. Only one worker can change a queued job to running.

3. What is an idempotency key?

Model answer: It is a value that identifies a request. If the same key arrives again, the system returns the original result and creates nothing new.

4. Why not retry a crashed run automatically?

Model answer: It may already have had side effects. I mark it as interrupted and let a person decide whether to run it again.

5. When would you move away from SQLite?

Model answer: I would move when I need more than one server, more concurrent writes or managed backups. The store interface makes that change contained.

---

📝 Assignment

Add a listRecentRuns method with a limit, and write a unit test for it. Then sketch how the store’s methods would map to PostgreSQL tables and identify one that would need to change.

---

❓ FAQ

Is SQLite really acceptable in production?

For one small service on one server, it can be a sensible start. Back it up, and know when to move on.

Do I need Redis or Kafka?

Not for this workload. A team’s handful of test runs fits easily in a database table.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with AI Agent as a Service: Wrap Your QA Agent in an API.