Building and Testing Agents

An agent that only lives in a demo is a prompt; an agent in production is defined once, consumed by name from every client, and tested against evidence. This chapter is that lifecycle on this server (the cookbook runs it concretely in A Support Assistant with Memory): build the bundle, wire its tools, skills, and memory, consume it over the API, and verify what it actually does.

1Build: the definition#

Agents are defined in the admin console's Agents section (persisted configuration, so they survive restarts and ship in a settings file). The complete field set:

One rule makes the whole design predictable: the bundle supplies defaults, and anything the request states explicitly wins. Design agents accordingly: put the durable identity in the bundle, leave per-call variation to callers, and never rely on the bundle to FORBID anything (that is what the tools policy and skill allowlists are for).

2Consume: one field, every client#

• Discovery: GET /lmkit/v1/agents lists the definitions with their descriptions, skills, tool names, memory intent, and greeting, so pickers build themselves.
• Native chat: POST /lmkit/v1/chat with "agent": "support", streaming or not.
• OpenAI dialect: the same agent field rides /v1/chat/completions as an extension, so any existing OpenAI-SDK application adopts a server agent by adding one field, no other change. The same holds for skill, skill_inputs, server_tools, memory, and memory_store.
• The playground: every agent appears as a persona; picking it arms the greeting, the skill, the tools, and a per-conversation memory id when the agent wants one.

An unknown agent name is a named error on every path, never a silent fallback: a deploy that forgot to define the agent fails loudly on the first call.

3Test: evidence, not vibes#

Agents are software; test them like it. The server gives you determinism levers, live inspection, and after-the-fact evidence:

• Interactive first: the playground is the workbench: converse, watch tool activity as it happens, inspect the answer's shape, iterate on the skill in the admin editor (the live watcher applies edits immediately), and converse again.
• Reproducibility: chat accepts a seed, so a scripted probe suite can pin sampling and compare runs; request_id names each exchange so your harness can correlate its calls with the server's records.
• The loop's own record: every response summarizes tool activity as tool_events (streamed live as tool_use events), so an assertion like "the refund question must consult the policy document, never the web" is a check on structured output, not a guess.
• Memory checks: after a probe conversation, read the facts the agent stored (admin memory inspection) and assert what SHOULD have been remembered was, and nothing else; delete the probe id afterward.
• Server-side evidence: the requests and logs pages hold every exchange with timing, models, and outcomes, which is where "it behaved differently yesterday" gets answered.

A practical regression suite is therefore plain HTTP: a fixed set of conversations (seeded), assertions over answers and tool_events, memory-state checks where memory matters, run against a staging server on every skill or definition change. No special framework is required, because everything the agent does is visible in the API's own responses.

4Harden: the governance seams#

Production agents run inside the operator's decisions, each covered in its own guide: tool policy and egress decide what the loop may touch, skill allowlists and sources decide what procedure may load, memory stores decide what persists and the admin surface makes it correctable, and API keys decide who may converse at all. The agent definition itself deliberately holds no secrets and no permissions: it is a convenience bundle, and the security model stays where security lives.

5Iterate: the improvement loop#

Treat the definition, its skills, and its taxonomies as one reviewable unit: version them, change one lever at a time, and re-run the probe suite. When the agent's model is the bottleneck at volume, the models guide covers per-capability defaults and measurement, and fine-tuning turns your corrected transcripts into a compact model that speaks your domain natively. The corpus your agents generate under real traffic, with tool events and outcomes attached, is exactly the evidence that loop needs.

6Stated plainly#

• Build the durable identity into the bundle; leave variation to requests; enforce nothing through the bundle, everything through policy.
• Consumption is one field on any dialect, with API discovery and named refusals, so clients stay thin and deploys fail loudly.
• Testing is ordinary HTTP against determinism levers and structured evidence (seed, request_id, tool_events, memory inspection, request logs): agents are observable software here, not black boxes.