Local Coding Assistants

How to run AI coding tools against this server instead of a hosted API: which provider type and base URL each tool wants, which models to serve for chat versus autocomplete, and the capacity settings that keep an editor's rapid-fire requests responsive. For developers who want Copilot-class assistance with zero source code leaving the machine, and for teams whose policy bans cloud code assistants outright.

1Why these tools connect unmodified#

Coding assistants are API clients. Each one speaks one or more of the three dialects this server serves side by side (API Compatibility), so the integration is provider settings, not plugins:

The Anthropic surface is the differentiator. Tools built on the Anthropic Messages API, including their tool-use loops and streaming, connect to this server as they are, which most local servers cannot offer. Claude Code and Claude Desktop have their own end-to-end guide, including the identity mapping and serving-pool floors: Claude Desktop.

Authentication is the Bearer header on every dialect. A tool whose Anthropic provider sends only the x-api-key header will not authenticate here (that header is read for attribution only): on a keyed server, use that tool's OpenAI-compatible provider instead, or run loopback-only with anonymous access, where no key is required.

2Prepare the server#

1. Pull the models. Models section: one chat model and, if your tool does autocomplete, one small fast model (section 3). Downloads run in the background.
2. Decide the posture. On the developer's own machine, the loopback default with anonymous access needs no key at all. Serving a team means the Network posture, where every call needs a key (Going Live).
3. Mint per-developer keys in Access when the server leaves loopback: one full-scope key per developer, so Requests attributes every call to a person and revoking one credential never breaks a teammate (Keys and Authentication).

3Picking models: chat, autocomplete, embeddings#

Three jobs with opposite requirements. The catalog in Models is the living source, and the memory-fit method is The Right Model for Your Machine; as of this writing, sensible starting points are:

Do not serve autocomplete from the chat model: a 30B-class model answers a completion request correctly but not in the time an editor waits, and every keystroke pause competes with your chat turns for the same slots. Two models, two jobs.

4Tool by tool#

The provider-level fields below are stable; menu layouts and file formats churn with releases, so each tool's own documentation is authoritative for where exactly a field sits.

Continue (VS Code, JetBrains)

Continue's models are declared in its config file (config.yaml under the .continue directory), one entry per model with a role list. All three roles point at this server:

models:
  - name: Coder
    provider: openai
    model: qwen3-coder:30b-a3b
    apiBase: http://your-server:PORT/v1
    apiKey: lmk_...
    roles: [chat, edit, apply]
  - name: Autocomplete
    provider: openai
    model: qwen3.5:2b
    apiBase: http://your-server:PORT/v1
    apiKey: lmk_...
    roles: [autocomplete]
  - name: Indexer
    provider: openai
    model: qwen3-embedding:0.6b
    apiBase: http://your-server:PORT/v1
    apiKey: lmk_...
    roles: [embed]

The embed role makes Continue's codebase indexing call this server's /v1/embeddings, so repository retrieval stays local too.

Cline and Roo Code (VS Code)

Both are agentic editors born on the Anthropic API, and both connect through their provider dropdown. Two working configurations:

• Anthropic provider with a custom base URL of http://your-server:PORT (the server root): the tools' native tool-use loop runs unmodified against the Messages surface. Use this on a loopback install with anonymous access, since these providers' key field may not send Bearer.
• OpenAI Compatible provider with base URL http://your-server:PORT/v1, your key, and a model id: the portable choice on a keyed server.

Agentic sessions here grow long fast (file contents, diffs, tool results accumulate every step), so give the serving model a generous context window (section 5).

aider (terminal)

aider treats any OpenAI-compatible endpoint as a provider through two environment variables and a model prefix:

export OPENAI_API_BASE=http://your-server:PORT/v1
export OPENAI_API_KEY=lmk_...
aider --model openai/qwen3-coder:30b-a3b

The openai/ prefix tells aider to route the request to the configured OpenAI-compatible base rather than to a known hosted provider.

Zed

Zed's settings.json declares custom providers under language_models: an OpenAI-compatible entry takes an API URL (http://your-server:PORT/v1) plus the models you want listed, each with its context window, and the Ollama entry takes the server root and discovers models on its own. The key is entered once in Zed's provider settings. Which providers a given Zed release exposes is version-dependent; both paths exist in current builds as of early 2026.

JetBrains AI Assistant

Recent releases run on local models through their Ollama integration: in the AI Assistant model settings, point the Ollama URL at http://your-server:PORT (the server root). The assistant discovers installed models over the Ollama dialect's listing endpoint and uses them for chat. Local-model support in JetBrains AI Assistant is version-dependent and has been expanding release by release; check the IDE's own settings for what your version offers.

5Keeping the editor responsive#

An editor is a bursty, concurrent client: autocomplete fires on typing pauses, a chat turn runs beside it, and an agentic task fans out tool calls. Capacity here is declared as slots times context window per model (Inference Capacity), and the defaults are tuned for neither extreme, so give each model an override in the Inference section:

• The autocomplete model wants slots over window: several concurrent slots with a modest window, so overlapping completion requests decode side by side instead of queueing behind each other. The panel prices the shape against the device before saving.
• The chat and agent model wants window over slots: agentic sessions carry entire files and accumulated tool output, and a window sized below the session is realized as a stopped context-limit result mid-task. Size the window from your longest real session, not from the model's trained maximum.
• A team multiplies concurrency. Set the slot count from peak simultaneous developers and keep the saturation policy on Queue, so a burst stretches latency instead of failing requests.
• Codebase indexing never competes. Embedding requests run on their own pinned pool per model, so a full repository index and interactive chat coexist by design.

Watch the result where it shows: queue depth on the dashboard and per-request timings in Requests, filtered by the developer's key.

6Stated plainly#

• Coding assistants are API clients: point their provider settings here and every prompt, completion, and indexed file stays on your hardware.
• The Anthropic Messages surface means Anthropic-native tools (Cline, Roo Code, Claude Code) connect unmodified; authentication is Bearer on every dialect.
• Serve two models: a coding-tuned or strong general model for chat and agentic work, a small fast one for autocomplete; embeddings index the codebase on their own pool.
• Responsiveness is a capacity setting, not luck: slots for the autocomplete model, window for the agent model, per-model overrides in the Inference section.
• One key per developer makes the request trail name people, and revocation surgical.