OpenBot Keeps the Agent Out of the Sandbox: Tools Are Just JSON Requests

How OpenBot gives each Bot its own computer without letting the agent live inside it, and what a tool call actually looks like from the model's side, step by step.

Today I learned how OpenBot gives every Bot its own computer, and why the agent itself never moves into it.

An agent that runs code needs a sandbox, a throwaway computer where a bad command can’t hurt anything. Harrison Chase wrote about two ways to connect an agent to a sandbox:

1. Agent in the sandbox. The whole agent, its API key and its memory live inside the box. You send it messages.
2. Sandbox as a tool. The agent stays on your server. It only sends single actions into the box and reads back the result.

OpenBot uses pattern 2.

The Bot runs as its own service and talks to the OpenBot server over AG-UI. The model’s API key stays there. Threads and memory live in CopilotKit Intelligence and PostgreSQL.

The computer is a separate container: Chromium with its own browser profile, a /workspace folder, a shell, and an egress proxy that filters every outbound request. A supervisor creates, stops and resets one container per person, or one per Bot marked Isolated. Nothing else runs in there. The container holds no model key, no memory and no policy.

Between the Bot and the computer sits the gateway. The docs put it plainly: “The computer itself does not decide policy.”

This is the part that made it click for me. The model never touches the computer, and it can’t. A model only reads text and writes text.

A tool is a function the harness promises to run for the model. The model gets a menu in its prompt: a name, a description, and the arguments as a JSON schema. OpenBot defines its computer tools in server/src/computer/headless-tools.ts:

tool(
  "computer_run_command",
  "Run a bash shell command in your workspace. Pipes and && work; long output is truncated and long commands are stopped.",
  z.strictObject({ command: text }),
  async (input, { signal }) => ({
    ok: true,
    ...(await gateway.runCommand(seat, actor, input, signal)),
  }),
),

The model sees the first three arguments. The fourth one, the handler, runs on the server and never reaches the model. Every handler calls the gateway, never the computer directly.

The full menu is short: computer_snapshot, computer_click, computer_type, computer_read, computer_run_command, computer_read_file, computer_write_file, computer_request_help, and a few more. To the model, “my computer” is just that list.

Let’s step through it like a debugger. The person asks: “What files do I have?”

Step 1: the model writes a request instead of an answer. It returns a tool call and stops.

{ "name": "computer_run_command", "input": { "command": "ls" } }

Nothing has happened yet. The model is now waiting.

Step 2: the harness finds the handler. The server looks up computer_run_command and calls its handler with the input the model wrote. The handler already knows which Bot and which person it acts for. The server sets both before the model writes any arguments, so the model can’t ask for someone else’s computer.

Step 3: the gateway runs its four steps. For a shell command it looks like this (server/src/computer/gateway.ts):

1. Resolve the target: which container, and for a click, which element.
2. Decide against the policy. The policy is a list of CEL rules, so a deployment can refuse intent == "run_command" outright.
3. Write the audit row. This happens before the action runs, so a refused or failed attempt still shows up in the trail.
4. Act or refuse. Refused means an ActionRefusedError that names the rule. Allowed means an HTTP POST /exec to the container, carrying its COMPUTER_TOKEN.

Step 4: the result goes back as a message. The shell output comes back marked as untrusted, because a command can print anything it fetched from the web:

{ "ok": true, "stdout": "notes.md\nreport.csv\n" }

Step 5: the model reads it and decides again. It can call another tool or answer the person. That’s the whole agent loop: request, run, result, repeat.

Here’s a bigger turn you can step through yourself, one hop at a time, with the real payload at every hop.

{
  "role": "user",
  "content": "Go to alexop.dev and write a CSV of the last 5 blog posts."
}

• The model can’t go around the rules. Its only way to act is a tool call, and every tool calls the gateway.
• The box can’t steal anything. A prompt injection that takes over the shell finds no API key and no chat history. It also can’t reach the database, which sits on a separate Docker network.
• The box is disposable. If it crashes, the Bot keeps its memory, and the next call wakes a fresh one.
• The model has to be taught the tools. It can’t see the page, so OpenBot’s prompt spells out the order: call computer_snapshot first, then click or type with the refs it returned, and never invent one.

The price is a network round trip for every action. For a Bot that clicks through a website, I think that’s a fair trade.

I’m copying the same shape into my own project: the Bot runs on the server, and only the gateway is allowed to talk to the computer.