Integration
Cloud-to-Local Migration
Moving from a hosted AI API to this server is deliberately boring: the surfaces are behavioral clones (API Compatibility), so migration is a base URL, a credential, and a model name, per client. This is the runbook: the exact switch for each kind of client, the differences that actually matter, and the checklist that proves the cutover before anyone notices it happened. Migrating from another local runner instead is Coming from Ollama; still weighing the move at all is When to Run AI Locally.
1The switch, per client#
| Client | The change |
|---|---|
| OpenAI SDKs | base_url (or the OPENAI_BASE_URL environment variable) to http://your-server:PORT/v1, api_key to a key from the admin console |
| Anthropic SDKs | base_url to the server root, and the key through the AUTH-TOKEN option (ANTHROPIC_AUTH_TOKEN), not the api-key option; see section 2 |
| Claude Code | ANTHROPIC_BASE_URL to the server root, ANTHROPIC_AUTH_TOKEN to a key |
| Claude Desktop | Its own gateway dialog, not an environment variable: Claude Desktop |
| Responses-native tools | wire_api = "responses" plus the base URL: Responses and Vector Stores |
| Ollama clients | Point the client's host setting (commonly the OLLAMA_HOST environment variable) at the server root; the /api dialect answers as they expect |
| Frameworks (anything wrapping the SDKs above) | Find where the framework surfaces base URL and key; it inherits everything else (per-stack recipes) |
On a loopback-only install with anonymous access, the key can be a placeholder; beyond loopback, keys are real and mandatory (Going Live).
2Credentials behave differently, on purpose#
- One key model for every dialect. Keys come from the admin console, are shown once, and authenticate through the Bearer header everywhere.
- Anthropic's
x-api-keyheader does not authenticate here. It is read for attribution only; configure Anthropic clients with their auth-token option. This is the single most common migration stumble, and it presents as 401s from an SDK that "worked yesterday". - Read-only keys exist and are refused on mutating methods; migrations should mint a full-scope key per consuming system, not share one (Keys and Authentication).
- Provider JWTs can replace keys where you already run an identity provider: Single Sign-On.
3Mapping model names#
Cloud model names will not resolve here; three strategies, in order of preference:
- Name local models explicitly: the catalog ids (
GET /v1/models, or the admin console) go wherever the cloud name went. - Send no model at all: a request that omits the model gets the server's default for the capability, so the operator, not each client, decides what "the model" is (Choosing and Operating Models).
- Keep the old names temporarily: only Claude Desktop's gateway maps fixed cloud identities onto local models automatically; every other surface answers 404 for a model it does not serve, which is the honest signal to finish the mapping.
4What actually changes#
| Topic | Hosted API | Here |
|---|---|---|
| Rate limits and quotas | Enforced per account, priced per token | None; throughput follows your hardware, and saturation follows YOUR policy (queue or reject with Retry-After: Inference Capacity) |
| Context windows | The provider's per-model limits | The local model's own limits, reported per model; long-prompt refusals are sized honestly (Errors, Retries, and Jobs) |
| Data handling | The provider's terms | Nothing leaves by default; the exhaustive egress list is Where Data Lives |
| Feature set | Everything the provider ships | What the dialects serve, stated in API Compatibility; unsupported corners refuse by name instead of half-working |
| Model quality | Frontier hosted models | The local models you chose; re-run YOUR quality checks, not the provider's marketing (Measuring What Matters) |
The last row is the honest one: migration changes the model, not just the URL. Budget an evaluation pass on your own tasks; the Playground is the fastest place to run it before touching code.
5The cutover checklist#
- Stand the server up properly first: posture, TLS, keys (Going Live).
- Pull and pin the models, set the defaults, and verify
GET /v1/modelslists what you expect. - Point ONE client at the server in a non-production path; run its real traffic shape, including streaming and tool calls if it uses them.
- Watch the request trail in the admin console while it runs: every call, its key, its model, its latency, in one place.
- Flip clients one at a time by their base URL; keep the old configuration one revert away until each is proven.
- Then turn off what you left behind: cloud keys that no longer need to exist are the cheapest security win of the whole migration.
6Stated plainly#
- Per client, migration is a base URL, a credential, and a model name; the table in section 1 is the whole switch.
- The classic stumbles are the Anthropic auth-token nuance and unmapped cloud model names; both fail loudly and are fixed in minutes.
- Limits, quotas, and data-handling clauses stop being the provider's and become your hardware and your policy.
- The model changed too: prove quality on your own tasks before declaring victory.