Live Desktop Sandboxes: VNC/GUI Integration in Better OpenClaw

Self-hosted AI agents can now do more than execute code in headless containers. With the latest Better OpenClaw update, your agents can spin up full GUI desktop environments and stream them directly into the Homespace dashboard via noVNC — no separate VNC client needed.

This post walks through the architecture: how OpenSandbox's create_desktop action connects to the Homespace UI through Caddy's WebSocket proxy, Convex-backed session tracking, and an embedded SandboxViewer component that supports live VNC streaming, Chrome DevTools tabs, and fullscreen mode.

Why GUI Sandboxes Matter for AI Agents

Headless code execution covers most use cases, but some tasks genuinely need a display:

• Browser automation — Watch your agent navigate a website in real time via Chrome or Firefox desktop sessions.
• GUI application testing — Agents can launch, interact with, and screenshot desktop applications.
• Data visualization — Matplotlib plots, Jupyter notebooks, and other tools that render to a display server.
• Debugging — See exactly what the agent sees when it reports errors in a GUI context.

Architecture Overview

Caddy WebSocket Proxy: flush_interval -1

Caddy automatically handles WebSocket upgrades in its reverse_proxy directive. However, for streaming connections like noVNC and KasmVNC, you need one additional directive: flush_interval -1.

Without it, Caddy buffers the HTTP response body before forwarding. This is fine for typical request/response patterns, but VNC streams continuous frame data over WebSocket. Buffering causes the remote desktop to appear frozen or laggy.

# Auto-generated Caddyfile for OpenSandbox
opensandbox.example.com {
    reverse_proxy opensandbox:8080 {
        header_up Host {host}
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-For {remote_host}
        header_up X-Forwarded-Proto {scheme}
        flush_interval -1
    }
}

Better OpenClaw's Caddy generator now detects the websocket: true flag on port definitions and automatically injects this directive. The Desktop Environment service (KasmVNC on port 6901) and OpenSandbox (noVNC proxy on port 8080) both have this flag set.

Session Tracking with Convex

Sandbox sessions are tracked in a dedicated sandboxSessions table in Convex. When an agent creates a desktop sandbox, the webhook handler in openclaw.ts receives the event and upserts a session record:

// sandboxSessions table schema
{
    sandboxId: string,      // OpenSandbox UUID
    novncUrl: string,       // noVNC WebSocket URL
    vncEndpoint?: string,   // Direct VNC endpoint
    devtoolsUrl?: string,   // Chrome DevTools URL (chrome image only)
    image: string,          // Container image
    resolution?: string,    // Display resolution
    status: "creating" | "running" | "terminated" | "error",
    taskId?: Id,            // Associated task
    agentId?: Id,           // Agent that created it
    tenantId: string,       // Tenant isolation
}

Sessions are indexed by tenant, status, task, and sandbox ID for efficient querying. The Homespace UI subscribes to these queries reactively — when an agent creates a sandbox, the viewer appears automatically.

The SandboxViewer Component

The heart of the UI integration is the SandboxViewer React component. It renders an embedded <iframe> pointing to the noVNC URL, wrapped with controls and status indicators:

• Status badge — Color-coded indicator (green = running, orange = creating, red = error).
• Refresh — Reloads the noVNC iframe without killing the sandbox.
• Fullscreen — Expands the viewer to fill the entire viewport.
• Open in new tab — Opens the noVNC URL directly for a standalone experience.
• Terminate — Stops the sandbox container via Convex mutation.

The component supports a compact mode (300px height) for embedding inline in the TaskDetailPanel, and a full-size mode (500px) for the dedicated Sandboxes page.

Chrome DevTools Integration

When the sandbox image is opensandbox/chrome, the agent can return a devtools_url alongside the VNC URL. The SandboxViewer detects this and adds a tab bar to switch between the Desktop view (noVNC) and Chrome DevTools — both rendered in the same iframe slot.

This is particularly useful for browser automation tasks where the agent needs to inspect network requests, console output, or DOM state while viewing the rendered page.

Where Sandboxes Appear

Desktop sandboxes are surfaced in three places in the Homespace dashboard:

1. Sandboxes page — A dedicated full-page view (via the nav bar) showing all sessions with a list sidebar and large viewer panel.
2. Task detail panel — Inline compact viewers appear in the "Desktop Sandboxes" section of any task that has active sessions.
3. Document preview tray — Documents of type sandbox render the SandboxViewer instead of text/code content.

Getting Started

To use desktop sandboxes in your stack, include opensandbox as a service and optionally add desktop-environment for KasmVNC:

# CLI
npx create-better-openclaw my-stack --services opensandbox --proxy caddy --domain my-ai.dev

# With the code skill pack (auto-includes opensandbox)
npx create-better-openclaw my-stack --skills code --proxy caddy --domain my-ai.dev

Once running, your agents can use the code-sandbox skill's create_desktop action to spin up desktop sessions. The noVNC viewer will appear automatically in Mission Control.

What's Next

• Screenshot capture — Periodic screenshots saved as documents for async review.
• Session recording — VNC session replay for debugging agent behavior.
• Multi-display — Side-by-side desktop sessions for parallel agent tasks.