# Models

> Call OpenAI, Anthropic, Google, Kimi, GLM, MiniMax and more through one gateway with your own provider keys, each provider in its own API. Kuiper routes, limits and records every call, can keep what each call said, and adds no charge.

Web page: https://kuiper.sh/docs/models/

## One gateway for your own keys

Kuiper's model gateway serves each provider's own API at one address: OpenAI's, Anthropic's and Google's Gemini API. Glove and the providers' own SDKs work as they are: you change their base URL and their key. You bring the provider keys. Kuiper sends each call to the right one, keeps the keys out of your units, limits and records every call, and shows you the tokens and what each call will cost you. Turn logging on and it keeps what each call said too: prompts, tool calls and their results, reasoning, images and audio. Kuiper adds no charge: your providers bill you directly, as they do today.

The gateway is at `https://gateway.kuiper.sh/models` and takes a [gateway key](https://kuiper.sh/docs/gateway-keys.md) (`kgw_…`) that includes the `models` service.

## Add a provider key

Provider keys belong to the organisation. Owners and admins add them in the dashboard, under **Access**, then **Providers**, or with the CLI. `--key -` reads the key from standard input, so it stays out of your shell's history:

```sh
kuiper providers add openai --key - < openai.key
kuiper providers add anthropic --key -
kuiper providers add kimi --field region=china --key -
kuiper providers add bedrock --field region=us-east-1 --key -
kuiper providers add custom --name my-vllm \
  --field base_url=https://llm.example.com/v1 --projects proxima --key -
kuiper providers test openai
kuiper providers list --service models
kuiper providers kinds --service models
```

- **The kind** is the provider: `openai`, `kimi`, `custom`. **The name** (`openai`, `my-vllm`) is the prefix of the key's model IDs, as in `openai/gpt-5` or `my-vllm/meta-llama/Llama-3.3-70B`. It defaults to the kind; a `custom` key needs one.
- **Projects.** A key is shared with every project unless `--projects` names some. `kuiper providers projects NAME a,b` or `--all` changes it.
- **Write-only.** Kuiper shows a key as its last four characters, never again. `kuiper providers set NAME --key -` replaces it and `kuiper providers rm NAME` removes it.
- **Status.** `test` lists the provider's models with the key and says `ok`, `invalid` (the provider refused the key) or `unreachable`. A call the provider answers with 401 marks the key `invalid` as well.

Members see the list but can't change it.

### Kinds

| Kind | Provider | Settings |
| --- | --- | --- |
| `openai` | OpenAI | |
| `anthropic` | Anthropic | |
| `google` | Google (the Gemini API) | |
| `openrouter` | OpenRouter | |
| `vercel` | Vercel AI Gateway | |
| `together` | Together AI | |
| `fireworks` | Fireworks AI | |
| `groq` | Groq | |
| `mistral` | Mistral | |
| `deepseek` | DeepSeek | |
| `xai` | xAI | |
| `baseten` | Baseten | |
| `kimi` | Kimi (Moonshot AI) | `region`: `international` (api.moonshot.ai, the default) or `china` (api.moonshot.cn) |
| `glm` | GLM (Z.ai, Zhipu) | `region`: `international` (Z.ai, the default) or `china` (Zhipu, open.bigmodel.cn) |
| `minimax` | MiniMax | `region`: `international` (api.minimax.io, the default) or `china` (api.minimax.cn) |
| `mimo` | Xiaomi MiMo | |
| `bedrock` | Amazon Bedrock | `region`: the AWS Region the key works in, such as `us-east-1` |
| `custom` | Custom endpoint | `base_url`: where its API is, such as `https://llm.example.com/v1` |

A key works on the platform it was made on: a China key needs `region=china`.

**Amazon Bedrock** takes a Bedrock API key, not AWS access keys. Kuiper calls Bedrock's Chat Completions, Responses and Messages APIs in the key's Region, with Bedrock's model IDs: `bedrock/openai.gpt-oss-120b-1:0`, `bedrock/global.anthropic.claude-sonnet-4-5-20250929-v1:0`.

**A custom endpoint** is a server of your own, such as vLLM, SGLang or an internal gateway. Kuiper sends it what it would send a provider, under its base URL: `/chat/completions`, `/responses`, `/embeddings`, `/messages` and the rest. The base URL must be `https://` on a public address: Kuiper never calls into private networks. A model on your own machine, in Ollama or LM Studio, needs a public HTTPS address in front of it, such as a tunnel.

## Point Glove or an SDK at Kuiper

### From a unit on Kuiper

Point your variables at the built-in `models` resource. Each running copy of the unit gets its own key and its region's gateway:

```sh
kuiper env set -p proxima -u agent \
  'MODELS_BASE_URL=${{models.BASE_URL}}' \
  'ANTHROPIC_BASE_URL=${{models.ANTHROPIC_BASE_URL}}' \
  'GEMINI_BASE_URL=${{models.GEMINI_BASE_URL}}' \
  'MODELS_API_KEY=${{models.API_KEY}}'
kuiper restart -p proxima -u agent
```

| Output | Value |
| --- | --- |
| `models.BASE_URL` | `https://gateway.kuiper.sh/models/v1`, or your region's gateway. The base URL for Glove's providers (all but `anthropic`) and OpenAI's SDK |
| `models.ANTHROPIC_BASE_URL` | `https://gateway.kuiper.sh/models`. The base URL for Glove's `anthropic` provider and Anthropic's SDK, which add `/v1/messages` |
| `models.GEMINI_BASE_URL` | `https://gateway.kuiper.sh/models`. The base URL for Google's Gen AI SDK, which adds `/v1beta/models/…` |
| `models.API_KEY` | A key made for this running copy (secret) |

Your provider keys never reach the unit. The unit's key is revoked when that copy stops, so a copy left on a lost machine can't call models.

### From anywhere else

Make a key for the project:

```sh
kuiper keys create laptop -p proxima --services models
export MODELS_BASE_URL=https://gateway.kuiper.sh/models/v1
export MODELS_API_KEY=kgw_…
```

The gateway reads the key from `Authorization: Bearer`, `x-api-key` (what Anthropic's SDK sends), `x-goog-api-key` or `?key=` (what Google's SDK sends).

### Glove

`createAdapter` takes a `baseURL` and an `apiKey`. Keep the provider and model you use, and give it the gateway and your gateway key:

```ts
import { createAdapter } from 'glove-core/models/providers';

const kimi = createAdapter({
  provider: 'kimi',                       // or openai, gemini, glm, minimax, mimo, …
  model: 'kimi-k2.5',
  baseURL: process.env.MODELS_BASE_URL,   // ${{models.BASE_URL}}
  apiKey: process.env.MODELS_API_KEY,     // ${{models.API_KEY}}
});

const claude = createAdapter({
  provider: 'anthropic',
  model: 'claude-sonnet-5-5',
  baseURL: process.env.ANTHROPIC_BASE_URL, // ${{models.ANTHROPIC_BASE_URL}}
  apiKey: process.env.MODELS_API_KEY,
});

const router = createAdapter({
  provider: 'openrouter',
  model: 'openrouter/anthropic/claude-sonnet-5-5', // your key's name, then OpenRouter's ID
  baseURL: process.env.MODELS_BASE_URL,
  apiKey: process.env.MODELS_API_KEY,
});
```

- A provider's own model names (`gpt-5`, `claude-…`, `gemini-…`, `kimi-…`, `glm-…`, `MiniMax-…`, `mimo-…`) find your key of that provider. Other names start with the key's name: `openrouter/…`, `vercel/…`, `my-vllm/…`.
- Reasoning fields (`reasoning_effort`, `reasoning_content`, thinking blocks) pass through untouched.
- Glove's `bedrock` provider signs requests with AWS credentials, which the gateway doesn't take. Use a `bedrock` key through `anthropic` or `openai` instead, with a model such as `bedrock/global.anthropic.claude-sonnet-4-5-20250929-v1:0`.
- For `ollama` and `lmstudio`, add the server as a `custom` key and use `my-ollama/llama3.2`.

### OpenAI SDK

```js
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: process.env.MODELS_BASE_URL, // ${{models.BASE_URL}}
  apiKey: process.env.MODELS_API_KEY,   // ${{models.API_KEY}}
});
const r = await client.responses.create({
  model: 'gpt-5-mini',
  input: 'Say hello',
});
```

```python
import os
from openai import OpenAI

client = OpenAI(base_url=os.environ["MODELS_BASE_URL"], api_key=os.environ["MODELS_API_KEY"])
r = client.chat.completions.create(
    model="glm-4.6",
    messages=[{"role": "user", "content": "Say hello"}],
    stream=True,
)
for chunk in r:
    print(chunk.choices[0].delta.content or "", end="")
```

### Anthropic SDK

```js
import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  baseURL: process.env.ANTHROPIC_BASE_URL, // ${{models.ANTHROPIC_BASE_URL}}
  apiKey: process.env.MODELS_API_KEY,      // ${{models.API_KEY}}
});
const msg = await anthropic.messages.create({
  model: 'claude-sonnet-5-5',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Say hello' }],
});
```

### Google Gen AI SDK

```js
import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({
  apiKey: process.env.MODELS_API_KEY,                     // ${{models.API_KEY}}
  httpOptions: { baseUrl: process.env.GEMINI_BASE_URL }, // ${{models.GEMINI_BASE_URL}}
});
const r = await ai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: 'Say hello',
});
```

```python
import os
from google import genai
from google.genai import types

client = genai.Client(
    api_key=os.environ["MODELS_API_KEY"],
    http_options=types.HttpOptions(base_url=os.environ["GEMINI_BASE_URL"]),
)
r = client.models.generate_content(model="gemini-2.5-flash", contents="Say hello")
```

## What the gateway serves

| Route | API |
| --- | --- |
| `POST /models/v1/chat/completions` | Chat Completions, streamed or not |
| `POST /models/v1/responses` | Responses, streamed or not |
| `GET`, `DELETE /models/v1/responses/{id}`, `POST …/{id}/cancel` | Stored responses, below |
| `POST /models/v1/embeddings` | Embeddings |
| `POST /models/v1/images/generations` | Image generation |
| `POST /models/v1/images/edits` | Image edits, as a form (or JSON, as xAI takes them) |
| `POST /models/v1/audio/speech` | Speech: the audio streams back |
| `POST /models/v1/audio/transcriptions` | Transcription, as a form |
| `POST /models/v1/messages` | Anthropic's Messages, streamed or not |
| `POST /models/v1/messages/count_tokens` | Anthropic's token counting |
| `POST /models/v1beta/models/{model}:generateContent` | Gemini, and `:streamGenerateContent` (with `?alt=sse` or without) |
| `POST /models/v1beta/models/{model}:countTokens` | Gemini's token counting |
| `POST /models/v1beta/models/{model}:embedContent` | Gemini's embeddings, and `:batchEmbedContents` |
| `GET /models/v1/models` | Every model your project's provider keys offer, as `name/model` |
| `GET /models/v1/calls/{id}` | One of your project's calls |

Gemini's routes work under `/models/v1/models/…` too. Every call is recorded under its route (`chat.completions`, `responses`, `embeddings`, `images.generations`, `images.edits`, `audio.speech`, `audio.transcriptions`, `messages`, `count_tokens`, `generate_content`). Counting tokens is recorded but isn't usage.

Your request goes to the provider as you wrote it. Kuiper changes only the model (to the provider's own ID), asks a Chat Completions stream for its token usage, and asks OpenRouter for its cost. Answers stream through as they come, and a client that goes away closes the provider's connection, so generation stops. Every answer carries `x-kuiper-call-id` and `x-kuiper-provider`, and the provider's rate-limit headers.

### Which provider serves what

| Kind | Chat Completions | Responses | Embeddings | Images | Audio | Messages | Gemini |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `openai` | yes | yes, and stored | yes | generations, edits | speech, transcriptions | | |
| `anthropic` | yes | | | | | yes, and count_tokens | |
| `google` | yes | | yes | generations | | | yes |
| `openrouter` | yes | yes | yes | | | | |
| `vercel` | yes | yes | yes | | | yes, and count_tokens | |
| `together` | yes | yes | yes | generations | speech, transcriptions | | |
| `fireworks` | yes | yes | yes | | | yes | |
| `groq` | yes | yes | | | speech, transcriptions | | |
| `mistral` | yes | | yes | | transcriptions | | |
| `deepseek` | yes | | | | | yes | |
| `xai` | yes | yes, and stored | yes | generations, edits | | | |
| `baseten` | yes | | | | | yes | |
| `kimi` | yes | yes | | | | yes (international) | |
| `glm` | yes | | | | | | |
| `minimax` | yes | | | | | yes, and count_tokens (international) | |
| `mimo` | yes | | | | | yes | |
| `bedrock` | yes | yes, and stored | | | | yes | |
| `custom` | yes | yes, and stored | yes | generations, edits | speech, transcriptions | yes, and count_tokens | |

A call to an API its provider doesn't offer is a 400 that names the key, the provider and the API.

### Forms

Image edits and transcriptions stream to the provider as they arrive. The gateway holds a form only until its `model` field has passed, up to 128 MiB, so the model can pick the key. A form is sent once: there are no fallbacks for it.

### Stored responses

Reading, removing or cancelling a stored response goes to the key that made it. The gateway that served the call remembers that for 30 days. Otherwise, name the key in an `x-kuiper-provider` header (every answer names its key there), or the project's only key whose provider stores responses is used.

## Which provider a model goes to

1. **`name/model`** goes to the provider key called `name`: `openai/gpt-5`, `or/moonshotai/kimi-k3`, `my-vllm/meta-llama/Llama-3.3-70B`. On Gemini's routes, where the model is in the path, write `name~model`.
2. **A provider's own model names** go to that provider, so an SDK only changes its base URL. `gpt-…`, `o1`, `o3`, `o4`, `chatgpt-…` and OpenAI's embedding, image and audio models go to `openai`. `claude-…` goes to `anthropic`. `gemini-…`, `gemma-…` and `text-embedding-004` go to `google`. `kimi-…` and `moonshot-…` go to `kimi`, `glm-…` to `glm`, `MiniMax-…` to `minimax` and `mimo-…` to `mimo`. On Anthropic's routes every other name without a prefix is Anthropic's, and on Gemini's, Google's.
3. Anything else is a 400 that lists your provider keys. A provider's own model with no key for that provider is a 400 that says to add one, and a key not shared with the project is a 403.

When the organisation has several keys of a provider, the one named after it is used, otherwise the oldest shared with the project.

## Fallbacks

Send an OpenRouter-style list and the gateway tries it in order:

```json
{ "model": "anthropic/claude-sonnet-5-5", "models": ["openai/gpt-5", "or/google/gemini-2.5-pro"], "messages": [] }
```

The next model is tried when one can't be reached, doesn't start answering in time, or answers 408, 429 or 5xx, as long as nothing has reached your client yet. There are at most three attempts. Once a stream has started there is no failover: a stream that breaks ends with an error in its API's format. A list works on every route whose body names the model; Gemini's routes, forms and stored responses make one attempt.

## Calls, usage and costs

Kuiper records every call: the model asked for, the provider key and model that answered, the status and any error, tokens, time to first token and latency, whether it streamed, the unit or gateway key, and its conversation. **Prompts and completions are never in a record.** They are kept only when you turn logging on (below).

```sh
kuiper models calls -p proxima --since 1h --status error
kuiper models calls -p proxima --has tool_calls,images --unit agent
kuiper models call 01JA… -p proxima
kuiper models usage -p proxima --by model
kuiper models list -p proxima
```

- **Tokens** are the provider's own count: input, cached input read and written, and output, with reasoning shown apart.
- **Estimated cost** is what the provider will bill for the call: the provider's own figure where it gives one, such as OpenRouter's, otherwise the tokens at Kuiper's list of prices for that model, otherwise nothing. It is never part of Kuiper's bill. Owners and admins see amounts; members see tokens.
- Calls are kept for 30 days. Hourly totals are kept for good, on the project's **Models** page and in the **Model calls** section of the organisation's [Usage](https://kuiper.sh/docs/usage.md) page.

The project's **Models** page in the dashboard shows usage by model, unit, key, provider or day, a call log with filters, and each call in a drawer. Agents connected over [MCP](https://kuiper.sh/docs/mcp.md) get `list_model_calls`, `get_model_call` and `get_model_usage`.

### Conversations

Every call belongs to a conversation. Name it yourself with an `x-kuiper-session` header, the same on every turn of one agent run:

```js
const client = new OpenAI({
  baseURL: process.env.OPENAI_BASE_URL,
  apiKey: process.env.OPENAI_API_KEY,
  defaultHeaders: { 'x-kuiper-session': runId },
});
```

Without one, a logged call's conversation is a key made from its system prompt and its first user message (`c_…`), which stays the same while the agent's history grows. Click a call's conversation on the Models page, or run `kuiper models calls --conversation KEY`, to see its calls in order.

## Log prompts, answers and traces

By default no prompt or completion is kept anywhere. An owner or admin turns logging on for a project and says how long logged calls are kept, from 1 to 30 days (7 unless you say):

```sh
kuiper models logging full -p proxima --retention 14
kuiper models logging metadata -p proxima
```

You can do the same on the project's **Models** page. With `full`, each call keeps:

- **Its trace.** The call in one shape, whichever API it used: the parameters (temperature, max tokens, reasoning effort or thinking budget, tool choice, response format), the tools offered, and every message in and out, in order. A message's parts are its text, the model's reasoning (or that it was encrypted), tool calls with their arguments, tool results, images, audio, files and refusals. With them come the tokens, with cache reads and writes and reasoning apart, time to first token, latency, output tokens a second, and why the answer ended.
- **Its raw request and answer**, each up to 1 MiB, as they were sent. A streamed answer is kept as the stream your client got.

Kuiper builds the trace after the call ends, so logging adds no latency. Images and audio sent inline are kept as files; an image you send as a URL stays a URL, and Kuiper never fetches it.

Your agent sends its whole history again on every turn, but Kuiper keeps each message once per project and each image or sound once per organisation. A long agent run costs about what its messages weigh once.

Owners and admins read logged calls:

- in the call's drawer on the **Models** page: the input, the reasoning, the output with images and audio players, and each tool call linked to its result and to the call that carried it;
- with `kuiper models call ID -p proxima`, which prints the messages (`--json` prints the call and its trace as JSON, `--raw` the request and answer as they were sent);
- with the `get_model_call` MCP tool;
- at `GET /v1/projects/{project}/models/calls/{id}`, whose answer includes `trace`.

Members see each call's record but never what it said.

## Limits

| Limit | Value | Past it |
| --- | --- | --- |
| Calls a minute per gateway key | 600 | 429 with `Retry-After` |
| Calls in flight per organisation | 100 | 429 with `Retry-After` |
| Until the provider starts answering | 300 s | the next fallback, else 504 |
| A whole call, stream included | 30 min | the stream ends with an error |
| A request body | 32 MiB | 413 |
| A form (image edits, transcriptions) | 128 MiB | 413 |

Your providers' own limits still apply to your keys, and their 429s come back to you with their headers.

## Errors

Errors come in the format of the API you called, so SDKs raise their usual exceptions: OpenAI's routes answer `{"error": {"message", "type", "code"}}`, Anthropic's `{"type": "error", "error": {"type", "message"}}` and Gemini's `{"error": {"code", "message", "status"}}`. A provider's own error is passed on as the provider sent it.

| Status | Code | Meaning |
| --- | --- | --- |
| 400 | `invalid_model` | The model names no provider key, there is no key for that provider, or the provider doesn't offer that API |
| 401 | `invalid_api_key` | No gateway key, or an unknown or expired one |
| 403 | `permission_denied` | The key lacks the `models` service, or the provider key isn't shared with the project |
| 429 | `rate_limited`, `too_many_calls` | A limit above |
| 502 | `provider_unreachable`, `provider_error` | The provider couldn't be reached |
| 503 | `unavailable` | The gateway can't reach Kuiper yet and doesn't have your provider keys |
| 504 | `timeout` | The provider didn't answer in time |