Drop-in with the OpenAI SDK
Keep the code you already have. Change one URL and the key.
AI Gateway puts your self-hosted models and providers like OpenAI, Groq and Together behind one OpenAI-compatible endpoint. It runs on your own servers. Your GPUs take the traffic first, vendors step in when needed, and you decide who uses what.
AI use grows faster than the rules around it.
You run models on your own GPUs. Teams also call OpenAI, Groq or Together with keys shared in chat threads. It works until it doesn't.
No failover. When a provider goes down, every app that depends on it goes down too.
No limits. One script can burn through a shared key and block everyone else.
No view of spend. Nobody can say who spent what, on which model, with which vendor.
No front door. Paid APIs get the traffic even when your own GPUs could have done the job.
AI Gateway is the one front door that fixes all four.
Live in four steps.
Step 01. Deploy it on your servers. AI Gateway runs inside your own infrastructure. Prefer not to run it yourself? We can host and manage it for you.
Step 02. Connect your providers. Register your GPU servers (vLLM, Ollama) and your vendor accounts. Give each model a name your team will recognise.
Step 03. Invite your team. Each person gets their own keys, their own token budget and a list of models they are allowed to use.
Step 04. Point your apps at one URL. Requests go to your GPUs first and move to a vendor automatically when needed. Every token and every dollar shows up in the dashboard.
One gateway. Five jobs done.
Everything you need to run self-hosted and third-party models as one reliable service.
Your GPUs first, vendors as backup
Use the hardware you already pay for, and spill over to paid vendors only when you need to.
Control who uses what
Set the rules once and the gateway enforces them on every request.
See every token and dollar
Know exactly where your AI budget goes.
Private by design
Your data passes through. It never stays.
Built for your team.
Register providers, publish a model catalogue and set each user's budget and model access when you invite them. Change the catalogue live, with no restart. Take a server out for maintenance without dropping a single request.
Before and after
| Without AI Gateway | With AI Gateway | |
|---|---|---|
| Integration | A different setup for every provider | One OpenAI-compatible endpoint |
| Your GPUs | Used separately from vendors | Used first, vendors as backup |
| Outages | A provider goes down, your app goes down | Requests move to the next provider automatically |
| Access | Shared keys with no limits | Personal keys, budgets and model access per user |
| Spend | Scattered across vendor invoices | Cost per user, key, model and provider in one dashboard |
| Privacy | Unclear who logs what | Prompts and responses never stored |
Works with the models you already run.
- OpenAI
- OpenRouter
- Groq
- Together
- vLLM
- Ollama
See how AI Gateway fits your setup.
Tell us what you run today and how many people need access. We'll get back to you to learn more about your setup.
Frequently asked questions.
What is AI Gateway?
One API that sits in front of your self-hosted GPU servers and third-party AI providers. Your apps send every request to one place, and the gateway routes it, applies your limits and records the cost.
Where does AI Gateway run?
On your own servers, inside your infrastructure. If you'd rather not run it yourself, we can host and manage it for you.
Do we need to change our code?
No. If you use the OpenAI SDK, change the base URL and the API key. Everything else stays the same.
Which models and providers can we use?
Any OpenAI-compatible provider, including OpenAI, OpenRouter, Groq and Together, plus your own servers running vLLM or Ollama. Anthropic and Gemini models are available through OpenRouter.
What happens when a provider goes down?
The gateway checks every provider on a schedule. If one is down, overloaded or rate-limited, requests move to the next one automatically, and unhealthy providers are taken out of rotation.
Do you store our prompts?
No. Prompts and responses are never logged or stored. We only record usage data such as date, model, key and token counts, so you can see usage and cost.
Can we limit how much each person uses?
Yes. Each user gets daily and monthly token budgets, each key gets a requests-per-minute limit, and admins choose which models each user can call. Costs are tracked in US dollars for reporting.
Which API endpoints are supported?
Chat completions, with and without streaming, embeddings and model listing.
Who gets access?
Your admins decide. They invite each user, who signs in with a 6-digit code sent by email. No passwords.
How do we get started?
Get in touch with the form above. We'll book a call, look at your setup and show you how AI Gateway fits.