LLM Provider Adapter: Switch AI Models Without Rewriting Code

AI models change quickly. If provider-specific code is spread across your whole agent, each change becomes a painful rewrite. An LLM provider adapter keeps those details in one small place. In this lesson you will build one, add a safe retry and test it all with fake providers.

---

💡 What Is an LLM Provider Adapter?

Without an adapter	With an adapter
Provider code in every agent	Provider code in one file
Changing models touches many files	Changing models touches one class
Tests need a real model	Tests can use a fake provider

---

🔍 One Small Interface

Keep the interface small. A single generate method is enough for our example.

---

🧩 Build the Adapter and Retry

Each provider below is fake. The retrying provider returns a broken answer until it receives feedback, just as a real model sometimes does.

interface ModelProvider {
  name: string;
  generate(prompt: string): Promise<string>;
}

type Validate = (text: string) => string[];

const steady: ModelProvider = {
  name: 'steady',
  generate: async () => '{"result":"ok"}',
};

const retrying: ModelProvider = {
  name: 'retrying',
  generate: async (prompt) =>
    prompt.includes('Fix these errors') ? '{"result":"ok"}' : 'Sure, here you go!',
};

const broken: ModelProvider = {
  name: 'broken',
  generate: async () => 'I cannot produce JSON',
};

const mustBeJson: Validate = (text) => {
  try {
    JSON.parse(text);
    return [];
  } catch {
    return ['Response is not valid JSON'];
  }
};

async function generateStructured(
  provider: ModelProvider,
  prompt: string,
  validate: Validate,
  maxAttempts = 3,
): Promise<string> {
  let currentPrompt = prompt;

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const text = await provider.generate(currentPrompt);
    const errors = validate(text);

    if (errors.length === 0) {
      console.log(`${provider.name}: valid on attempt ${attempt}`);
      return text;
    }

    currentPrompt = `${prompt}\nFix these errors: ${errors.join('; ')}`;
  }

  throw new Error(`${provider.name}: no valid output after ${maxAttempts} attempts`);
}

async function main() {
  for (const provider of [steady, retrying, broken]) {
    try {
      await generateStructured(provider, 'Return JSON', mustBeJson);
    } catch (error) {
      console.log((error as Error).message);
    }
  }
}

main();

steady: valid on attempt 1
retrying: valid on attempt 2
broken: no valid output after 3 attempts

The calling code never changes, and only the provider does. This is the main benefit of the design.

---

🛠️ Adding a Real Provider Later

A real adapter implements the same interface. Inside generate, it calls the provider’s SDK and returns the text. Provider details such as the model name stay inside that class. You will connect one in the Level 8 project.

---

📌 Two Kinds of Retry

Problem	What to do
Timeout or rate limit	Wait, then retry the same request a limited number of times
Invalid JSON or schema	Send the errors back, within a limit
Invalid API key	Stop, because a retry cannot fix it
Model refuses	Stop and report the refusal

Each retry costs money. Log every attempt so that you can measure the cost and reliability of each provider.

---

🌍 Real-Life Example: A Model Is Retired

A team’s provider retires the model that it uses. With an adapter, one engineer changes the configuration and reruns the evaluation. Without one, model names and response parsing are scattered across twelve files. Running the LLM evaluation afterwards matters, because a new model can behave differently.

---

🧪 Try It Yourself

1. Run the example.
2. Change maxAttempts to 1 and see what changes.
3. Add a fourth fake provider that fixes itself on attempt 3.
4. Log the prompt length on each attempt.
5. Replace mustBeJson with the validator from the previous lesson.

---

📋 Ready-to-Copy Prompt

You are a senior TypeScript engineer reviewing an LLM provider adapter.

Code:
[paste code with no keys or secrets]

Check that:
- Provider details stay inside the adapter
- A fake provider can be used in tests
- Retries are limited
- Each error type is handled in the correct way
- Keys are not hard-coded or logged

Return a table with: Issue, Risk and Smallest Fix.
Do not change the public interface unless it is necessary.

---

⚠️ Common Mistakes

• Calling a provider SDK directly from every agent
• Making the interface far too large
• Retrying without any limit
• Retrying errors that cannot be fixed by waiting
• Assuming that a new model behaves the same way

---

🎯 Key Takeaways

• An adapter keeps provider details in one place.
• A small interface also makes testing easy.
• Handle network errors and bad output differently.
• Re-evaluate your agent after changing models.

---

🎤 Interview Questions

1. What is an LLM provider adapter?

Model answer: It is a thin layer that gives the application one interface for model calls. Provider-specific details remain inside it, so agents do not depend directly on a vendor.

2. How does an adapter help testing?

Model answer: It lets me replace the real provider with a fake one that returns scripted responses. Tests become fast, free and able to simulate invalid output and failures.

3. How would you handle invalid model output?

Model answer: I would validate it and resend the errors to the model for a limited number of attempts. If it still fails, I would stop with a clear error instead of using the output.

4. Which errors should not be retried?

Model answer: Invalid credentials and bad request formats usually need a fix, so I would stop. I would retry temporary failures such as timeouts and rate limits, with waiting and a limit.

5. What would you do after changing the model?

Model answer: I would rerun the evaluation set, compare quality, cost and speed, and review any new failures. An interface can stay the same while the model’s behaviour changes.

---

📝 Assignment

Add a FakeFlakyProvider that throws a timeout error once and then succeeds. Update generateStructured so that timeouts and validation failures are counted separately. Write tests for both paths.

---

❓ FAQ

Should I use a library instead of my own adapter?

A library can be a good choice once you know your needs. A small adapter is easy to understand and is enough for this course.

Can one project use several models?

Yes. For example, you might use a cheaper model for simple steps and a stronger one for harder analysis, and measure both.

---

🚀 Next Steps

Well done on finishing this lesson. Complete the assignment, then continue with Multi-Agent Orchestration for QA: Workflows and Frameworks.