A production chatbot usually needs routing rather than one permanently chosen model. Use a fast, economical default for routine turns, escalate tasks that fail a confidence or validation gate, and keep a fallback for outages. Measure cost per resolved conversation, not cost per token in isolation.
A practical evaluation method
- Track first-token latency, full-response latency, tool success, and handoff rate.
- Cap context growth with retrieval, summarization, and cache-aware prompts.
- Evaluate safety and structured-output failures on real support intents.
Choosing an AI API for a Chatbot
A production chatbot lives or dies on three things: response quality, latency, and cost per conversation. The best choice balances all three — and usually means routing tiers, not picking one model. Here's how to decide and build.
Which Models
- Cheap, fast default → DeepSeek V4 Flash ($0.14/$0.28) or Gemini Flash-Lite — handle the bulk of everyday turns cheaply.
- Balanced quality → Claude Sonnet 4.6 or Gemini 3.5 Flash — for nuanced replies.
- Hard turns → escalate to Claude Opus 4.8 or GPT-5.6 only when needed.
The pattern: route the easy 90% to a cheap model, escalate the hard 10% — the biggest cost lever for a chatbot. See how to reduce AI API costs and how to choose an LLM.
Build Essentials
- Stream responses so replies feel instant — streaming guide.
- Prompt caching for your system prompt / knowledge — cuts cost and first-token latency dramatically (prompt caching).
- Tool use to fetch real data (orders, docs, weather) — function calling.
- Manage context window — summarize old turns so you don't resend everything each message (context window comparison).
- Watch latency — AI API latency guide.
Minimal Chat Call
curl -X POST https://api.linkmodel.ai/v1/chat/completions \
-H "Authorization: Bearer $LINKMODEL_API_KEY" -H "Content-Type: application/json" \
-d '{ "model": "deepseek-v4-flash",
"messages": [
{"role":"system","content":"You are a helpful support agent for Acme."},
{"role":"user","content":"Where is my order #1234?"}
],
"stream": true }'Through LinkModel, one key runs every tier above — so you can route cheap→premium and A/B models without new integrations. Confirm the chat endpoint in the docs; start from the Python or Node.js quickstart.
Cost Math
Estimate tokens per turn × turns per conversation × price. A Flash-tier model keeps most chats well under a cent; caching the system prompt slashes it further. Reserve premium models for escalations only.
Which to Pick
- Cheapest scalable bot → DeepSeek V4 Flash / Gemini Flash-Lite.
- Quality-sensitive (sales, advisory) → Sonnet 4.6, escalate to Opus/GPT-5.6.
- Building on a budget → start cheap, add a routing rule for hard turns.
Broader picks in best LLM API and cheapest LLM API.
Bottom Line
Don't pick one model — route tiers. Default to a cheap fast model, stream, cache the system prompt, add tool use, and escalate only hard turns. That's a chatbot that's both good and cheap to run.
Start free with a $1 credit and build your bot on one key.