The Command Line
The lmkit executable has two jobs. Launched with no arguments, it IS the server. Launched
with a verb, it is the operator's toolbox: everything provisioning scripts, golden images,
and terminal diagnostics need, without opening a browser. lmkit --help lists every verb,
lmkit <verb> --help its options, and every verb exits with a meaningful code, so each one
doubles as a scriptable check. Verbs run OFFLINE against the installation's own state unless
marked live below; the live ones talk to a running server over HTTP. Every verb reads the
same settings file and environment the server reads, which is what section 5 is about.
1Provisioning: models before first run#
lmkit models
Lists the model catalog with per-model download state, and:
lmkit pull gemma4:12b
downloads one by its catalog id, with progress, into the configured model directory. This
pair is how an air-gap-bound or fleet image gets its models WITHOUT starting the server:
pull on the build machine, ship the image, and first run serves immediately. Exit codes:
pull returns 0 on success or already-downloaded, 1 on an unknown id or a failed download.
On a fleet with a shared model volume, the same pair provisions the volume once, from any
node (section 5).
2Diagnostics: status, doctor, logs, devices, version#
lmkit status(live): is a server running here, and which one? Probes the configured local HTTP then HTTPS ports (or--url), and prints the running server's version, engine, and boot id. With--api-key, it also queries readiness: models loaded, active inferences, queue depth. Exit 0 running, 2 not running, solmkit status && ...is a guard, not a grep.lmkit doctor: the preflight. One pass over everything an installation depends on: the state directory with the rule that elected it (pinned byLMKIT_STATE_DIR, beside a writable executable, or the per-user fallback) and its writability, the identity store the configuration selected (Identity store:, embedded or shared), a note when a data folder of earlier builds exists and whether it has been copied in, the model directory with downloaded count and free disk, visible compute devices, HTTPS certificate validity, Windows service registration, and a liveness probe. Every hard check prints[ok]or[!!]; exit 0 only when all pass, which makeslmkit doctorthe first line of any provisioning script and the first ask on any support thread.lmkit logs: the tail ofserver.logfrom the terminal:-n 200for depth,--followto stream new lines (rotation-safe). Exit 2 when no log exists yet. The Logs page remains the richer view; this is the SSH-only path.lmkit devices: the compute inventory (number, name, type, memory), for verifying what the runtime will see before anything loads.lmkit version: this EXECUTABLE's product and engine versions, offline.statusreports the RUNNING server's; after an upgrade the two differ until the service restarts, anddoctorpoints that out explicitly.
3The server as a unit: service, certificates, operators#
lmkit service install | uninstall | status(Windows, elevated for install and uninstall) registers this executable as the Windows service, removes it, or reports registration (exit 2 when not registered). This is how a machine crosses between desktop mode and service mode; on Linux and macOS, the systemd unit and launchd job are the equivalents. Desktop installs deliberately run without a service, since the tray runs the server per session.installrecordsSecurity:NetworkAccess=Networkin the settings file when nothing has set it yet, because a machine-wide service listening on loopback alone serves nobody; on a managed node the manifest declares that posture and the verb writes nothing. The service is registered, not started.lmkit cert-infoshows the HTTPS certificate (path, subject, thumbprint, validity, SANs);cert-export -o server.pemwrites the public certificate for distributing trust to other machines;cert-install/cert-uninstallmanage the OS trust store entry (normally automatic on first run; these exist for re-attempts and golden images).lmkit cert-acme --enable --domains kb.example.com --email ops@example.com --accept-tosconfigures automatic TLS via ACME / Let's Encrypt by writingappsettings.json(restart to apply), andcert-acme-statusreports whether it is active and, when it is not, why. UnderLMKIT_MANAGED_CONFIGthe verb answers the same refusal every configuration write answers: the manifest owns the settings, so the edit is declined and nothing is written. Going Live covers when ACME applies at all.lmkit admin create-owner | reset-password | listis break-glass operator management: it acts on the admin identity store directly, no sign-in required, because host access IS the credential.create-owner --username <name> --password <password>mints a new Owner (passwords are at least 12 characters);reset-password --username <name> --password <password>sets a new password, ends that account's sessions and re-enables a disabled account, and with--one-timeforces a change at the next sign-in;listprints the accounts with role, second-factor and disabled state. Exit 2 whenreset-passwordnames an operator that does not exist. A server running beside the store serves the change on the next sign-in. Context and the other two ways in are Operator Accounts, section 3.lmkit farm join | leave | statusis this node's farm membership, run with the server stopped forjoinandleave.join --store postgres --connection-string "..."copies this node's identity domain into the shared database (--no-copyskips it,--mergejoins a database that already holds accounts), writesAdmin:IdentityandDeployment:Mode=Farminto the settings file, and prints the domain key once (--domain-keysupplies your own) with the environment block every node needs and the prerequisites this node still has to meet.leaveis the reverse: the shared domain copied into a fresh embedded store, the file written back to a single node, and the environment variables to remove listed.statusprints the mode, the store, the node roster with each row's liveness, and the unmet farm prerequisites (exit 1 when any). Procedures: Farm Runbooks.
4Two adapters: the MCP bridge and the docs export#
lmkit mcp --url http://localhost:5189 --api-key $LMKIT_API_KEY
Many MCP clients launch their servers as CHILD PROCESSES speaking newline-delimited JSON on
standard input and output, and cannot open an HTTP session themselves. This verb is the
adapter: point such a client at lmkit mcp as the command, and it bridges stdio to the
running server's MCP endpoint, carrying session and protocol
headers across and forwarding the server's progress events back. Standard output stays
protocol-pure (diagnostics go to standard error), so the stream never corrupts. --path
overrides a relocated endpoint; omit --api-key when the server accepts anonymous local
calls.
lmkit docs export --out ./site --base-url https://docs.example.com/lm-kit-one
writes the documentation site, every guide and the complete REST API reference, as static
HTML ready to publish under your own docs domain (--base-url sets the canonical links and
the sitemap). It is the one verb that defers: the exported reference must be the OpenAPI
document the composed application actually serves, so the verb records the request, startup
composes every route, the exporter runs just before the server would start listening, and
the process exits. No server needs to be running, no model is loaded, and no port is bound.
5On a farm: the verbs and the environment they read#
Every verb starts the way the server starts: it loads the settings file and the environment, opens the admin identity store, and only then dispatches. That is what makes the verbs truthful on a single node, and it is the one thing to know before running them on a fleet node:
A verb resolves the state directory AND the identity store from the shell that invokes it,
never from the service it sits beside. On a service or container node those two facts
usually live where the service reads them: Environment= and EnvironmentFile= lines in
the systemd unit, the machine-wide variables the Windows service control manager hands the
service, the container's environment. An interactive shell opened without them resolves its
own state directory and, finding no Admin__Identity__Store, selects the embedded store.
lmkit admin create-owner then mints an Owner into a per-node admin-identity.db no server
serves, and lmkit doctor reports that store as healthy. Nothing fails; the verb simply
acted on an installation that does not exist. Two habits close the gap:
Run the verb with the node's environment. Under systemd, load the unit's environment file into a shell running as the service account before the verb, or let systemd do it:
sudo -u lmkit bash -c 'set -a; . /etc/lmkit-one/secrets.env; set +a; exec /opt/lmkit-one/lmkit admin list' sudo systemd-run --pipe --wait --uid=lmkit -p EnvironmentFile=/etc/lmkit-one/secrets.env /opt/lmkit-one/lmkit admin list(add the unit's
Environment=lines,LMKIT_STATE_DIRandLMKIT_MODELS_DIRamong them, the same way). In a container,docker exec <container> /opt/lmkit/lmkit admin listorkubectl exec deploy/<name> -- /opt/lmkit/lmkit admin listinherits the container's environment and needs nothing else. On Windows, setLMKIT_STATE_DIR,LMKIT_MODELS_DIRand theAdmin__Identity__*pair as machine-wide (system) variables, so the service and every new elevated prompt resolve the same installation (Windows Server Deployments, section 4). KeepingAdmin:Identityin the settings file under the pinned state directory works too: the service and any shell that resolves that directory read the same file.Read doctor's
Identity store:line before acting.embedded SQLite (admin-identity.db under the state directory, single node)in a shell on a fleet node means the environment is missing and the verb is about to act on a private store;shared postgres database (every node on this connection string forms one admin domain)(ormysql,sqlserver) means the verb is talking to the fleet. A shared database that cannot be reached stops the verb before it dispatches, with the store's error in the output: opening the store is part of startup and there is no fallback to the embedded one, so the failure is the diagnosis.
With the environment in place, the verbs that exist today do the following on a fleet:
lmkit doctoris the per-node acceptance test: the state directory this node resolved and which rule elected it (on a fleet the answer should bepinned by LMKIT_STATE_DIR, on local disk), whether it is writable, the identity store as above, the model directory (the shared volume) with its free disk, the devices this node sees, its certificate, its service registration and whether its server answers. Run it on every node; two nodes printing different state directory rules or different identity store lines have not been given the same environment.lmkit admin create-owner | reset-password | listact on the shared domain, so an Owner minted from one node signs in on every node, a reset ends that operator's sessions fleet-wide, andlistis the same list everywhere. On a shared store the admin surface never opens for a first visit, socreate-owneris one of the ways the first operator of a farm arrives (Operator Accounts, section 3).lmkit pullandlmkit modelsprovision the shared model volume once, from any node: withLMKIT_MODELS_DIRin the environment the file lands on the volume, andlmkit modelson every other node lists it as downloaded. A pull holds the SDK's cross-process file lock, so a second node pulling the same id at the same time waits and then finds the finished file rather than writing over it.lmkit cert-info | cert-export | cert-install | cert-uninstallact on THIS node's persisted certificate under its own state directory; every node has its own, which is why a fleet terminates TLS at the balancer (Reverse Proxy) or mounts one PFX on every node throughHttps:CertificatePath(password through the environment asHttps__CertificatePassword).cert-acmeis per node: each node keeps its own ACME account and certificate store under its state directory, and the ACME challenge for a shared name lands on whichever node the balancer picks, so ACME does not fit behind a balancer; a setHttps:CertificatePathdisables it, andcert-acme-statussays so. On a managed nodecert-acmedeclines to write, as section 3 describes.lmkit service install | uninstall | statusregisters this machine's Windows service; run it from an elevated prompt that carries the machine-wide variables, so the service it registers and the shell it runs in resolve the same state directory. On a managed node it leaves the settings file alone.lmkit farm statusis the farm's own acceptance test on a node: the mode this node resolved, the store it opened, every node's heartbeat row with its liveness, and the farm prerequisites this node fails, if any.farm joinandfarm leaverefuse while a server runs on the state directory, because they rewrite its settings file and its embedded store.
6Conventions, stated once#
- No arguments serves; any argument is a verb. There are no server-start flags: the server is configured by its settings file and the admin console, not by its command line.
- Exit codes are the API. 0 is success, 1 is failure, and 2 is the scripted-check
"false" (
statusdown,service statusunregistered,logsabsent,reset-passwordon a name nobody holds), so shell logic reads outcomes without parsing text. - Offline verbs act on the installation's own state (the same
state directory the server uses), which is why running them
as a different OS user than the service can see different paths;
doctorprints the paths it acted on precisely so that mismatch is visible. On a service or container node the same rule reaches the identity store: a verb run without the node's environment acts on a private embedded store (section 5). - Help is a served answer.
--helpandversionexit 0.
7Stated plainly#
- One executable, two jobs: no arguments is the server, a verb is the toolbox.
pullandmodelsprovision headlessly;status,doctor, andlogsoperate from any terminal;service,cert-*andadminmake hosts;mcpadapts stdio clients anddocsexports the documentation site.- Every verb reads the settings file and environment of the shell that runs it; on a fleet
node, give it the node's environment or it acts on a store nobody serves, and read
doctor's
Identity store:line to tell the two apart. - Every verb exits meaningfully, so the CLI composes into scripts as naturally as the API composes into pipelines.