How Do LLMs Work in OpenClaw? A Generative AI Guide

In OpenClaw, the large language model is the reasoning layer, not the system. A local Gateway assembles context, exposes typed tools, calls a provider/model pair, and executes whatever tool calls come back. The model decides; the Gateway acts.

That separation is the whole point, and it is where most evaluations go wrong. Teams ask which model OpenClaw uses, as though the answer were a product decision. It is a configuration line. The decisions that determine whether an OpenClaw deployment is production-grade sit around the model: what context it receives, which tools it is allowed to see, what happens when a provider rate-limits mid-task, and who approved the action it just took on a host machine.

This guide walks the model layer as it is actually documented — the agentic loop, provider routing and failover, local inference, memory, and the permission boundary that decides what the model can reach. Every configuration key and behaviour below is taken from the official OpenClaw documentation, verified 22 September 2026.

What does the LLM actually do in OpenClaw?

The LLM does reasoning and tool selection. Everything else — connections, sessions, scheduling, delivery, permission enforcement — belongs to the Gateway.

OpenClaw’s documentation describes the Gateway as “the local control plane for sessions, tools, events, and channel connections.” A single long-lived Gateway “owns all messaging surfaces (WhatsApp via Baileys, Telegram via grammY, Slack, Discord, Signal, iMessage, WebChat),” exposes a typed WebSocket API, validates inbound frames against JSON Schema, and emits agent, chat, presence, health, heartbeat and cron events.

Read that list again and notice what is absent: nothing in it requires a language model. The Gateway is a message broker and policy engine with a model attached. This is why OpenClaw lets you swap model harnesses — Claude, Codex, local models — without changing any other component.

The practical consequence for an enterprise architect is that the model is the least sticky part of the stack, and the orchestration layer is the most. Model quality determines how well a task is planned. The Gateway determines what the planning can touch, who can trigger it, and whether the blast radius of a bad plan is a sandbox or a production filesystem.

Intelligence versus orchestration, stated plainly:

The second row is the one that gets an engagement paused. Model errors are visible and recoverable. Orchestration errors are quiet and expensive. It is the same separation that governs any enterprise AI agent build: the reasoning engine is rarely where the engineering effort concentrates.

How does the agentic loop work?

The loop is: assemble context, call the model, receive tool calls, check them against policy, execute, feed the results back, and repeat until the model stops asking for tools.

Concretely, in OpenClaw:

1. Context assembly. The Gateway builds the request from the system prompt, the session transcript, bootstrap memory files, any loaded skill instructions, and the schemas of the tools the agent is currently allowed to use.
2. Tool exposure — filtered before the call. This is the step most explainers skip. OpenClaw’s documentation is explicit: “The model only sees tools that survive the active profile, allow/deny policy, provider restrictions, sandbox state, channel permissions, and plugin availability,” and tool policy is “enforced before the model call.” A tool the model cannot see is a tool it cannot hallucinate its way into.
3. Reasoning. The model returns text, tool calls, or both. Tools are “typed functions the agent can call,” sent to the provider as structured function definitions.
4. Execution and observation. The Gateway runs the approved calls — on the host by default — and returns the results into the transcript as new context.
5. Iteration or stop. The loop continues until the model produces a final response with no further tool calls, or a limit, abort, or refusal ends it.

The difference between this and a chat completion is step 4. A chat model that writes rm -rf produces a string. An agent that writes rm -rf produces an outcome, unless a policy layer stands between the two. OpenClaw’s answer to that is tool permissions, not model restraint — and the ordering matters: policy first, model second.

How are LLM providers connected and routed?

Through a provider/model string and an ordered fallback chain. OpenClaw is model-agnostic by design, and the documented starter set covers more than 30 providers.

Configuration is JSON5. The documented minimum is a single line:

{
  agents: {
    defaults: {
      model: {
        primary: "anthropic/claude-opus-4-6"
      }
    }
  }
}

The starter set spans hosted providers (Anthropic, OpenAI, Google, Cohere, Mistral, xAI), infrastructure (Amazon Bedrock, Fireworks, Together AI, DeepInfra, NovitaAI), regional providers (Alibaba Model Studio, Qianfan, Qwen, Moonshot AI, StepFun), gateways (OpenRouter, Vercel AI Gateway, Cloudflare AI Gateway, LiteLLM), local runtimes (Ollama, LM Studio, vLLM, SGLang), and media generation (ComfyUI, Runway, Fal). Authentication is handled during openclaw onboard.

Model roles are separable

Rather than one model doing everything, OpenClaw exposes distinct keys:

• agents.defaults.model.primary — the reasoning model
• agents.defaults.model.fallbacks — ordered alternates
• agents.defaults.utilityModel — a lower-cost model for internal work
• agents.defaults.decisionModel — typed decisions and scoring
• agents.defaults.imageModel, agents.defaults.pdfModel — media and document handling
• agents.defaults.modelPolicy.allow — an allowlist restricting which models can be selected at all
• agents.entries.*.model — per-agent override, which also applies to subagents spawned by that agent

For anyone running this in a regulated environment, modelPolicy.allow is the key to read first. It is the difference between “we standardised on an approved model list” and “an operator typed /model and sent regulated data to an unreviewed endpoint.”

Failover is specified, not improvised

OpenClaw builds a candidate chain — the requested model first, then configured fallbacks deduplicated, then the configured primary appended if no override was supplied — and rotates auth profiles within a provider before advancing. Profile rotation runs OAuth tokens first, then static tokens, then API keys, least-recently-used within each tier.

Failover advances on auth failures, billing disables, rate limits and cooldown exhaustion, provider-overloaded signals, timeout-shaped errors, model-not-found including HTTP 404, and unclassified errors while candidates remain. It deliberately does not advance on context overflow, explicit aborts, or a final provider refusal — three cases where trying a different model would hide the real problem rather than solve it.

Before rotating, OpenClaw attempts bounded same-model recovery: up to 10 attempts for rate limits with exponential backoff capped at 30 seconds, and 8 retries within a 90-second window for other transient failures. Long Retry-After waits are capped at 60 seconds by default via OPENCLAW_SDK_RETRY_MAX_WAIT_SECONDS. Errors are classified as auth, rate_limit, overloaded, billing, server_error, timeout, model_not_found, or unclassified; billing failures disable a profile for 10 minutes.

One asymmetry is worth putting in a runbook. Configured defaults fall back automatically. A user’s explicit /model selection is strict — it “fails visibly instead of falling through to another configured model.” That is the correct behaviour, and it will still surprise an operator at 2am who assumed redundancy applied everywhere.

Can OpenClaw run on local LLMs?

Yes. OpenClaw documents managed llama.cpp, Ollama and LM Studio, plus high-throughput serving through vLLM, MLX or SGLang over OpenAI-compatible HTTP, and custom proxies such as LiteLLM.

A local provider is declared alongside its endpoint and context limits:

{
  agents: {
    defaults: {
      model: { primary: "lmstudio/my-local-model" }
    }
  },
  models: {
    providers: {
      lmstudio: {
        baseUrl: "http://127.0.0.1:1234/v1",
        apiKey: "lmstudio",
        api: "openai-responses",
        models: [{
          id: "my-local-model",
          contextWindow: 196608,
          maxTokens: 8192
        }]
      }
    }
  }
}

Three caveats from the documentation decide whether local inference survives contact with real work.

Memory is workload-dependent, not model-dependent. Requirements “depend on the model weights, context size, runtime, and other work on the host.” Managed llama.cpp enforces an 8 GiB minimum for smaller recipes and scales up from there. The documentation’s own advice is to test actual tasks before making a model your default, because a model that handles short prompts can still fail under a full agent workload carrying history and tool calls.

Tool calling is the failure mode, not generation quality. Smaller local models frequently break on structured tool invocation. OpenClaw mitigates this with Tool Search, which “defers schemas while preserving policy-approved capabilities” for stricter backends. Where a model consistently fails, tools can be disabled entirely or reduced with lean mode, which drops optional capabilities such as browser and image generation while keeping core tools.

Local does not mean safer by default. “Local models do not provide hosted providers’ safety filters.” Data residency improves; prompt-injection exposure does not. The compensating control is tool permissions, and it has to be configured deliberately.

The honest enterprise read: local inference is a sound answer to data residency and a poor answer to capability. A defensible pattern is a hosted reasoning model with a local utilityModel for routine internal steps, sized by evidence from your own workload rather than a public benchmark.

How does OpenClaw manage memory across Slack, WhatsApp and other channels?

Through files on disk plus transcript compaction. The documentation is blunt about it: “The model only remembers what gets saved to disk; there is no hidden state.”

Memory lives as plain Markdown in the agent workspace at ~/.openclaw/workspace:

• USER.md — stable preferences, communication style, relationships and project context, treated as directives. Loaded at session start on its own small budget.
• MEMORY.md — the curated long-term layer: durable facts and decisions. Loaded at session start on a standard budget. If it exceeds that budget, OpenClaw “keeps the file on disk intact but truncates the copy injected into context.”
• memory/YYYY-MM-DD.md — dated running notes. Today’s and yesterday’s load automatically on /new or /reset; older notes are indexed for search and retrieved on demand rather than injected every turn.
• DREAMS.md — background consolidation summaries for human review, controlled by plugins.entries.memory-core.config.dreaming.enabled.

Semantic search over notes uses memory.search.provider, which defaults to OpenAI — a detail worth catching before deployment, because it means note content can leave the host even when the reasoning model is local.

Working memory is the transcript, and compaction manages it

OpenClaw compacts automatically as a session approaches the model’s context limit, and again as overflow recovery when a provider returns a context-overflow error, after which it retries. /compact triggers it manually. Older turns are condensed into summaries while recent messages stay intact; tool calls and their matching results are kept paired, and if a boundary lands inside a tool block OpenClaw “moves the boundary so the pair stays together.” Full history stays on disk — compaction changes only what the model sees next. Images and non-text content get omission markers.

The relevant keys sit under agents.defaults.compaction: model to summarise with a cheaper model, mode (default "safeguard", which applies stricter quality audits), maxActiveTranscriptBytes as a size trigger, notifyUser, keepRecentTokens (default 20,000) for manual retention, and memoryFlush.enabled to write durable facts to MEMORY.md before a compaction discards them.

Two governance points follow directly. First, a memory layer that is plain Markdown on a host is readable by anything with filesystem access, and it accumulates whatever passed through chat — which, on a WhatsApp or Slack surface, includes content nobody classified. Second, because MEMORY.md is loaded as directives, it is a persistence mechanism for prompt injection: text that reaches memory reaches every future session. Treat that workspace as a data store under policy, not as scratch space.

How do tools and plugins connect the model to real systems?

Through a typed tool catalogue that the Gateway executes on the host by default, extended by plugins and instructed by skills.

The documented built-in categories:

The three extension mechanisms are not interchangeable, and conflating them is a common scoping error:

• Tools are callable actions — new capability.
• Skills are “a SKILL.md instruction pack loaded into the agent prompt” — repeatable procedure over existing capability, not new capability.
• Plugins add runtime infrastructure: “tools, skills, channels, model providers, speech, realtime voice, media generation, web search, web fetch, hooks, and other runtime capabilities,” built with the plugin SDK and distributed through ClawHub.

If a workflow needs a database, an internal API or an ERP, the answer is a plugin exposing a typed tool, with the procedure for using it written as a skill. Writing the procedure without the tool produces an agent that describes the work; adding the tool without the procedure produces an agent that does the work inconsistently. The build path for this is covered step by step in our guide to OpenClaw custom skill development, and the connection engineering behind the tool itself — write-back, permissions, audit — is the subject of AI integration into systems of record.

What has to be governed before this reaches production?

The permission boundary, the sandbox, the install policy and the trust model for inbound messages. The documentation states the starting assumption directly: “Treat inbound messages as untrusted input,” and tools “execute on-host by default; sandboxing is configurable.”

The controls that matter, by name:

• Tool policy. tools.deny / tools.allow globally, agents.entries.<agentId>.tools per agent, with per-agent overrides taking precedence. Enforced before the model call.
• Control-plane tools. gateway is owner-only because it reads configuration via config.schema.lookup / config.get and starts updates with update.run — restricted to avoid exposing secrets and host topology. cron creates persistent scheduled jobs and requires approval in untrusted contexts. For any agent exposed to untrusted content, the documentation’s guidance is to deny both by default.
• Sandboxing. Either containerise the whole Gateway in Docker, or run a host Gateway with tool-level isolation. agents.defaults.sandbox.workspaceAccess takes "none" (sandbox workspace only, the default), "ro", or "rw"; agents.defaults.sandbox.scope sets the isolation boundary to "agent", "session" or "shared". Paths are validated against normalised, canonicalised sources, with system directories and credential locations blocked. tools.elevated.allowFrom governs escapes from the sandbox — treat it as a named, reviewed exception, never a default.
• Lateral messaging. An agent with the message tool can send across conversations and providers by default: allowAcrossProviders and allowWithinProvider both default to true. Set both to false to confine an agent to its bound conversation. This is the control that stops a compromised or confused agent in a public channel from messaging a private one.
• Node execution. Running code on paired macOS nodes requires device pairing with an approval token, a global policy via gateway.nodes.commands.allow / deny, and per-node approval settings under exec.approvals.node.*. Approval mode “binds exact request context and, when possible, one concrete local script/file operand.”
• Supply chain. security.installPolicy requires operator approval for plugin and skill installation. A marketplace-installable capability that can call tools is a supply-chain surface; treat ClawHub items the way you treat any third-party package.
• Pairing. DM-capable channels require pairing approval, and node pairing is device-based with approval held in the device pairing store.

A model-layer decision and a permission decision are the same decision. Choosing a local model to keep data on-premise while leaving workspaceAccess: "rw", message unrestricted across providers, and gateway reachable from an agent that reads a shared Slack channel does not produce a private deployment. It produces a private model with a public blast radius. The wider threat picture is worth reading alongside this — we break it down in five OpenClaw risks every CTO should know.

An evaluation checklist for the model layer

Before an OpenClaw deployment goes past a pilot, the following should have written answers rather than defaults:

1. Which models are on modelPolicy.allow, who approved them, and how is the list reviewed?
2. What is the fallback chain, and does every model in it satisfy the same data-handling requirements as the primary? A compliant primary with a non-compliant fallback is a compliance gap that appears only under load.
3. Which model handles utilityModel, decisionModel, imageModel and pdfModel — and does memory.search.provider send note content somewhere the reasoning model does not?
4. What is on each agent’s allow list, and can any agent reaching untrusted input call exec, gateway, cron, or message across providers?
5. Is sandbox.workspaceAccess set to "none", and who is named on any tools.elevated.allowFrom exception?
6. Where does ~/.openclaw/workspace live, who can read MEMORY.md, is it backed up, and does it fall under retention policy?
7. What is the compaction configuration, and is memoryFlush.enabled set so durable decisions survive a compaction?
8. What is logged — model, tokens, tool calls, approvals — and can you reconstruct who or what authorised a given action after the fact?

Items 6 and 8 are the hardest to answer retrospectively. Data retention and action attribution are architectural properties, not features that can be added to a running deployment on request.

Where this leaves an enterprise evaluation

The model is a configuration value. The architecture is the product.

An OpenClaw deployment succeeds or fails on four things that have nothing to do with which model you picked: what the tool policy allows before the model is ever called, what the sandbox contains when a plan goes wrong, what the memory layer accumulates and who can read it, and whether you can reconstruct after the fact who authorised an action. Get those right and the model becomes a line you can change on a Tuesday. Get them wrong and no model choice compensates.

GrowExx works with enterprise teams on the layer this guide describes — model policy, tool permissions, sandboxing, and the evidence trail that makes an agent deployment auditable. See our OpenClaw skill development and implementation services, or talk to us about a model-layer and permission review of an agent pilot already running.

Frequently asked questions