Vercel AI SDK Guide: Connect an OpenAI-Compatible API in Next.js

The Vercel AI SDK can connect to an OpenAI-compatible provider through @ai-sdk/openai-compatible. You create a provider with an API key and base URL, select a model by ID, and pass that model to generateText or streamText. The application code stays focused on prompts and tool behavior while the provider configuration stays in one server-side module.

The official OpenAI-compatible provider documentation defines createOpenAICompatible, baseURL, apiKey, and provider options. This guide applies that contract to a LinkModel-backed Next.js route.

This guide uses LinkModel as the example backend. Its current developer reference lists https://api.linkmodel.ai/v1 as the API root and bearer-authenticated OpenAI-compatible POST /chat/completions for chat models. Confirm the model ID and supported features in the LinkModel model reference before shipping.

For terminal-based media generation, see LinkModel CLI. For n8n orchestration around image and video workflows, see n8n workflow templates for AI media.

Install the AI SDK packages

In a Next.js project, install the core SDK and compatible provider:

npm install ai @ai-sdk/openai-compatible zod

Store credentials in server-side environment variables:

LINKMODEL_API_KEY=your_api_key
LINKMODEL_BASE_URL=https://api.linkmodel.ai/v1
LINKMODEL_MODEL=gpt-5.4-mini

Do not prefix a browser-exposed variable with NEXT_PUBLIC_. The provider must be created in server code so the key is not shipped to the client.

Create a provider module

// lib/linkmodel.ts
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

export const linkmodel = createOpenAICompatible({
  name: 'linkmodel',
  apiKey: process.env.LINKMODEL_API_KEY,
  baseURL: process.env.LINKMODEL_BASE_URL,
  includeUsage: true,
});

export const defaultModel = process.env.LINKMODEL_MODEL ?? 'gpt-5.4-mini';

The compatible provider adds a bearer authorization header when apiKey is set. Keep the provider name stable because provider-specific options are nested under that name. If you change the provider package or SDK major version, rerun the streaming and tool tests before deployment.

Build a streaming Next.js route

// app/api/chat/route.ts
import { streamText } from 'ai';
import { linkmodel, defaultModel } from '@/lib/linkmodel';

export async function POST(request: Request) {
  const body = await request.json();
  const prompt = typeof body.prompt === 'string' ? body.prompt.trim() : '';

  if (!prompt || prompt.length > 8000) {
    return Response.json({ error: 'A prompt of 1-8000 characters is required.' }, { status: 400 });
  }

  const result = streamText({
    model: linkmodel(defaultModel),
    system: 'Answer directly. If the evidence is missing, say so instead of guessing.',
    prompt,
  });

  return result.toTextStreamResponse();
}

streamText is designed for interactive applications such as chatbots. The route validates input before calling the paid provider, and the system instruction gives the model a clear fallback behavior. In a real chat UI, use the AI SDK UI helpers to send message history in the format expected by your installed SDK version.

If your project uses a different AI SDK release, check the current method name for converting a stream to a Response. The conceptual flow stays the same: call streamText, handle errors, and return the provider stream.

Add a typed tool

import { stepCountIs, streamText, tool } from 'ai';
import { z } from 'zod';
import { linkmodel, defaultModel } from '@/lib/linkmodel';

const result = streamText({
  model: linkmodel(defaultModel),
  prompt: 'Look up order 1842, then summarize its status.',
  tools: {
    lookupOrder: tool({
      description: 'Read the status of one authorized order. Never changes order data.',
      inputSchema: z.object({ orderId: z.string().min(1).max(40) }),
      execute: async ({ orderId }) => {
        // Call an authenticated service here and re-check the caller's access.
        return { orderId, status: 'verification_required' };
      },
    }),
  },
  stopWhen: stepCountIs(3),
});

The tool schema helps the model produce valid arguments, but it is not authorization. Check the authenticated user inside execute, validate the order, and return a structured error for missing or unauthorized records. For refunds, deletion, or outbound messages, add an approval flow instead of executing automatically.

For Python services using the same compatible-endpoint pattern, see the LangChain OpenAI integration. For teams that prefer visual orchestration around the resulting application, see the n8n AI chatbot guide.

Tool calling depends on model and provider compatibility. Test plain generation first, then one read-only tool, then multi-step tool loops. Do not assume that a provider's OpenAI-compatible chat route preserves every proprietary field or tool behavior.

Switch models through configuration

The provider instance can expose several approved models. Keep the mapping on the server so a browser request cannot select an unapproved model:

const models = {
  fast: linkmodel('gpt-5.4-mini'),
  reasoning: linkmodel('gpt-5.4'),
};

const selected = models[body.mode === 'reasoning' ? 'reasoning' : 'fast'];

Use an allowlist and default to the safer path. Do not let a request body select arbitrary provider model IDs. Record the selected model, latency, usage, tool calls, and final business outcome so you can evaluate routing decisions.

Error handling, retries, and streaming

Streaming introduces partial responses. Your UI should show a cancellable state and a clear error if the connection breaks after text has arrived. Set server and provider timeouts, and retry only transient errors. Never blindly retry a tool with side effects.

Use request IDs and idempotency keys for downstream writes. Log the error class and provider status, but redact API keys and sensitive prompt content. For usage accounting, use provider-reported usage when present and label estimates as estimates.

Test before production

Run a fixed test set covering:

• short answer and long answer;
• malformed or oversized input;
• provider timeout and 429;
• tool selection and invalid arguments;
• prompt injection;
• stream cancellation;
• model fallback;
• unauthorized side-effect request.

Measure time to first token, total latency, token usage, error rate, tool success, and accepted answer rate. A faster model is not the right default if it creates more retries or review work.

Keep the client and server contracts separate

The browser should send a user message to your own /api/chat route, not to LinkModel. That route validates the input, chooses an allowlisted model, and owns the provider key. A minimal client-side request can be:

const response = await fetch('/api/chat', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ prompt: message }),
});

if (!response.ok || !response.body) throw new Error('Chat request failed');

const reader = response.body.getReader();
const decoder = new TextDecoder();
let text = '';
for (;;) {
  const { value, done } = await reader.read();
  if (done) break;
  text += decoder.decode(value, { stream: true });
  renderPartialAnswer(text);
}

For a full conversational UI, use the AI SDK UI hooks and message protocol for your installed SDK version. The important boundary remains the same: browser input is untrusted, provider credentials stay server-side, and partial output is not proof that a downstream action succeeded.

Vercel AI SDK troubleshooting

401: confirm the key is available only in server runtime and that the base URL is the API root.

404: check that baseURL does not already include /chat/completions; the provider appends the path.

No stream: verify the route returns the SDK's response object and that the client is reading a text or UI stream compatible with your SDK version.

Tool calls fail: test the model without tools, inspect the request, and check the provider's supported tool-call schema.

Usage is missing: enable usage where supported and treat absent usage as unknown rather than inventing a precise bill.

Next step

Put the compatible provider in one server-only module, start with non-streaming or plain streaming text, and add tools after the basic route is observable. Use LinkModel CLI when a coding agent needs explicit image/video task states, and n8n workflow templates for AI media when the application needs external workflow orchestration.

Sources: Vercel AI SDK OpenAI-compatible provider, AI SDK streamText reference, AI SDK tool-calling guide, LinkModel's first API call, and LinkModel's model reference.