Add an LLM Chat Feature Behind a Provider Contract, Not Across Your Codebase

A teammate pastes a model SDK call directly into a React component on Friday. On Monday the demo works, and by Wednesday you're asked to swap the model, add auth, stream responses, and explain why the API key leaked into the client bundle. That is the failure mode this article is about: the feature never had a contract, so every layer absorbed provider details it should never have seen.

Here is the working path I use now when a full-stack app needs an AI capability: define a provider contract first, implement one seam behind it, and only then let UI, storage, and auth talk to that seam. I'll walk through a vertical slice (chat endpoint → provider seam → streamed UI) that you can run locally and validate on a free tier before you commit budget.

The contract comes before the SDK

One file, no imports from any vendor:

// src/server/llm/contract.ts
export type ChatMessage = { role: 'user' | 'assistant' | 'system'; content: string };

export interface ChatRequest {
  messages: ChatMessage[];
  maxTokens?: number;
  userId: string; // required: usage attribution and rate limiting
}

export interface ChatProvider {
  streamChat(req: ChatRequest): AsyncIterable<string>;
}

export class ProviderError extends Error {
  constructor(
    public readonly code: 'RATE_LIMITED' | 'UNAVAILABLE' | 'BAD_REQUEST',
    message: string,
  ) {
    super(message);
  }
}

Three decisions in ten lines, and each one was made because something failed without it:

• userId is mandatory on the request. The first time I skipped this, I had no way to attribute cost or throttle a runaway retry loop in the frontend.
• The provider returns an AsyncIterable<string> of tokens, not a vendor response object. The moment a response shape leaks past the seam, swapping providers becomes a refactor across your codebase instead of a one-file change.
• Errors collapse into three codes your API layer can translate into HTTP responses (429, 503, 400) with Retry-After where appropriate. Vendor-specific error hierarchies stay inside the implementation.

One implementation behind the seam

// src/server/llm/http-provider.ts
import { ChatProvider, ChatRequest, ProviderError } from './contract';

export class HttpChatProvider implements ChatProvider {
  constructor(
    private readonly baseUrl: string,
    private readonly apiKey: string,
    private readonly model: string,
  ) {}

  async *streamChat(req: ChatRequest): AsyncIterable<string> {
    const res = await fetch(`${this.baseUrl}/chat/completions`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${this.apiKey}`,
      },
      body: JSON.stringify({
        model: this.model,
        messages: req.messages,
        max_tokens: req.maxTokens ?? 512,
        stream: true,
      }),
    });

    if (res.status === 429) throw new ProviderError('RATE_LIMITED', 'upstream limit');
    if (!res.ok || !res.body) throw new ProviderError('UNAVAILABLE', `status ${res.status}`);

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buffer = '';
    for (;;) {
      const { done, value } = await reader.read();
      if (done) return;
      buffer += decoder.decode(value, { stream: true });
      // parse server-sent events; yield token deltas only
      const lines = buffer.split('\n');
      buffer = lines.pop() ?? '';
      for (const line of lines) {
        if (!line.startsWith('data: ') || line.includes('[DONE]')) continue;
        const delta = JSON.parse(line.slice(6));
        const token = delta.choices?.[0]?.delta?.content;
        if (token) yield token;
      }
    }
  }
}

Note what this class does not know: auth sessions, the database, React, or which environment it's deployed to. Those belong to the layers around it.

Wiring the rest of the stack

The Express route owns auth, rate limiting, and persistence; it delegates the actual generation:

// src/server/routes/chat.ts
router.post('/api/chat', requireSession, async (req, res) => {
  const provider = req.app.locals.chatProvider as ChatProvider; // injected at startup
  const userId = req.session.userId;

  if (!(await rateLimiter.allow(userId))) {
    return res.status(429).json({ error: 'slow down', retryAfterSeconds: 30 });
  }

  res.setHeader('Content-Type', 'text/event-stream');
  try {
    let full = '';
    for await (const token of provider.streamChat({ messages: req.body.messages, userId })) {
      full += token;
      res.write(`data: ${JSON.stringify({ token })}\n\n`);
    }
    await db.messages.save(userId, full); // persistence lives here, not in the provider
    res.end();
  } catch (e) {
    if (e instanceof ProviderError && e.code === 'RATE_LIMITED') {
      return res.status(429).json({ error: 'provider busy', retryAfterSeconds: 10 });
    }
    return res.status(503).json({ error: 'assistant unavailable' });
  }
});

The frontend consumes the SSE stream with fetch + ReadableStream and never imports a model SDK. If you switch providers tomorrow, the client diff is zero.

Where I validate before paying for anything

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

For the pre-production pass, I point the seam at MonkeyCode, which currently offers free model access and a free server option, so the provider URL, key, and model name come from environment variables and nothing in the code changes between "free validation" and "paid production":

CHAT_BASE_URL=...
CHAT_API_KEY=...
CHAT_MODEL=...

The point of the free stage isn't benchmarking the provider — it's flushing out integration failures cheaply. Treat free availability as a gift with an unknown shelf life: don't hardcode its limits into capacity plans, and don't build a demo that only works because the quota happened to cover it.

My pre-production test list for this slice, all runnable against the free tier:

1. Unauthenticated request → 401, zero provider calls (assert with a stub provider).
2. Burst of 20 requests from one user → 429 with retryAfterSeconds, client backs off.
3. Upstream rate limit (stub the provider to throw RATE_LIMITED) → 503/429 to the client, never a raw stack trace.
4. Mid-stream disconnect → client sees a truncated stream and can re-request; partial text is not saved as a complete message.
5. Provider swap drill → point the env vars at a second provider and confirm the only file that changes is startup wiring.
6. Persistence check → saved messages are attributed to the right userId, and a second user cannot read them.

Test 5 is the one people skip and regret. If swapping providers takes more than an afternoon, your seam isn't a seam yet.

Limitations and who shouldn't do this

• A single-provider seam is overhead for a throwaway prototype that will never ship. Write the seam when the feature survives its second week.
• Streaming SSE behind some managed platforms has buffering quirks; verify flush behavior on your actual deployment target, not just locally.
• Free tiers are for integration validation, not load testing. You still owe yourself a real capacity estimate before launch, based on your own traffic model.
• If you need provider-specific features (tool calling with vendor-specific schemas, fine-tuned model management), a thin contract won't cover them. Extend the interface deliberately instead of punching holes in it.

Reusable delivery checklist

• [ ] Contract file has zero vendor imports
• [ ] userId required on every provider call
• [ ] Error codes mapped to HTTP responses in one place
• [ ] Rate limit + auth tested with a stub provider
• [ ] Provider swap drill done at least once
• [ ] Partial-stream handling verified on the client
• [ ] Persistence and permissions tested with a second user

Which handoff is least stable in your setup — the provider-to-API boundary, or the API-to-client stream? If you've hit a specific failure state there (status code, truncated SSE, swallowed error), I'd like to hear the exact response you got.