llm
Browse all articles, tutorials, and guides about llm
Posts
Per-Branch AI Endpoints: Isolating Model Spend Across Prod, Preview, and CI
When previews, CI, and production all call models with the same key, you cannot tell what a preview cost or notice a runaway test until the invoice. Because a Neon branch is its own deployment with a usage ledger that lives in the branch's Postgres, model spend is attributed and isolated per environment. I proved it: a CI branch spent tokens while production stayed flat.
Model Fallback and Routing Without a Provider SDK Each
Models have outages, rate limits, and bad minutes. A resilient app falls back to another one, but building that across providers normally means a different SDK and error shape for each. Through one OpenAI-compatible gateway, fallback is a loop over model names. Here it is, tested against a real failure.
One Key for Claude, GPT, and Gemini: the Gateway Pattern
Using three model providers usually means three API keys, three SDKs, and three billing relationships sprayed across your code. An AI gateway collapses that to one credential and one OpenAI-compatible endpoint. I proved it on a Neon Function: the same call answered by GPT, Claude, and Gemini.
Your First Serverless LLM Call on DigitalOcean in 10 Minutes
DigitalOcean's Inference Engine gives you an OpenAI-compatible endpoint with pay-per-token pricing and no GPU to manage. Here is the fastest path from zero to a working call, with curl, Python, and Node, every snippet run against the live API.
The US Government Pulled Two Frontier Models Overnight. The Real Lesson Is About Your Stack
On June 12, 2026, an export-control directive forced Anthropic to disable Claude Fable 5 and Mythos 5 for every user worldwide, three days after launch. The policy fight is interesting. The operational lesson for anyone building on a single model provider is more urgent.