Agent Vm

How the isolated agent MicroVMs on vanessa are built from Clan services: the host networking layer, the per-VM orchestration, the guest OS, and the secret flow that delivers API keys from host Clan vars into the guest.

This is reference documentation for the running system. The source of truth is always the code; see the table below.

Overview

Agent VMs are Clan service instances, not inventory.machines entries. They are declared in flake.nix and composed onto vanessa:

flake.nix inventory.instances
  -> @luxus/microvm-host on vanessa
     -> imports inputs.microvm.nixosModules.host
     -> creates bridge vmbr0 (10.200.0.1/24)
     -> provides NAT + DNS (dnsmasq) for guests

  -> @luxus/agent-browser on vanessa
     -> Chromium CDP 10.200.0.1:9222 (default) + Lightpanda :9223

  -> @luxus/agent-vm instances on vanessa
     -> microvm.vms.<name>
     -> modules/nixos/_agent-vm-guest.nix

vanessa is the only deploy target. Each @luxus/agent-vm instance defines exactly one microvm.vms.<vmName> on the host.

graph LR
  ext["external iface (br0)<br/>NAT"] --- host["@luxus/microvm-host<br/>vmbr0 · 10.200.0.1/24"]
  host --- cdp["agent-browser<br/>Chrome :9222 · Lightpanda :9223"]
  host --- jadorey["jadorey · pi"]
  host --- piff["pi-finalform · pi"]
  host --- tina["tina · pi"]

Isolation model

Each agent runs inside a QEMU MicroVM (microvm.hypervisor = "qemu", optimize.enable = false so the host uses stock qemu_kvm). The containment boundary per VM is:

• Private network segment — its own tap interface (vm-*) bridged to vmbr0, a static IP in 10.200.0.0/24, and a unique MAC.
• Read-only Nix store — /nix/store is shared in read-only via virtiofs (ro-store tag, mounted at /nix/.ro-store).
• Dedicated secret bundle — a per-VM virtiofs share mounted read-only at /run/host-secrets (see Secret flow).
• Persistent volumes — two raw images per VM: persist.img → /persist (4096 MB) and home.img → /home/luxus (4096 MB). The home image keeps agent state, ~/.pi/~/.hermes, npm/pnpm globals, zellij/shell history, and dotfiles across reboots.
• Project share — the host project directory is virtiofs-mounted read-write at /work (tag project).
• VSOCK SSH — a unique VSOCK CID per VM enables microvm -s <name> from the host; root login is prohibit-password (key only).
• Resource QoS — every agent VM runs in agent.slice with systemd CPU/Memory limits (see QoS & systemd ordering).

Guests authorize two SSH keys (the emily local key and the vanessa root / management key) for both luxus and root. The guest SSH host key is read directly from the persistent path /persist/ssh/ssh_host_ed25519_key, so ssh-keyscan stays stable across VM reboots.

VM inventory

Instance	Hostname	IP	Agent	Project (host = guest path)
agent-jadorey	jadorey	10.200.0.4	pi	/home/luxus/projects/jadorey
agent-pi-finalform	pi-finalform	10.200.0.6	pi	/home/luxus/projects/finalform
agent-tina	tina	10.200.0.7	pi	/home/luxus/projects/tina

Projects live on the host (Obsidian/rclone/git sync there). Guests mount the same path R/W via virtiofs. Browser engines are host CDP: Chromium (AGENT_BROWSER_CDP=http://10.200.0.1:9222) and Lightpanda (AGENT_BROWSER_CDP_LIGHTPANDA=http://10.200.0.1:9223).

Agent types

agentType (defined in services/agent-vm/default.nix) selects the CLI, default environment, expected secret names, and packages in modules/nixos/agent-vm-guest.nix. Active VMs today:

• pi — llm-agents flake (agent-jadorey, agent-pi-finalform, agent-tina).

Legacy types (hermes, opencode, claude) remain in _agent-vm-guest.nix for reuse but have no running instances (hermes MicroVM removed while package fails).

Host networking (@luxus/microvm-host)

services/microvm-host/default.nix (role server, on vanessa) provides the shared infrastructure:

• Imports inputs.microvm.nixosModules.host.
• Bridge vmbr0 — systemd-networkd netdev/network with host IP 10.200.0.1/24. Tap interfaces matching vm-* are attached to the bridge.
• NAT — networking.nat with externalInterface = "br0" and internalInterfaces = [ "vmbr0" ] (IPv4 only). This gives guests internet access out through the LAN-facing bridge.
• DNS — dnsmasq bound to vmbr0 + lo, forwarding upstream to 192.168.1.1. Guests point their resolver at the host gateway 10.200.0.1. dnsmasq is ordered after/requires the bridge device.
• Firewall — vmbr0 is a trusted interface; UDP/TCP 53 are opened for DNS.
• agent.slice — a systemd slice for all agent MicroVMs with accounting on and fair CPUWeight = 100, so runaway agents can’t starve the rest of the host (Home Assistant VM, Bambuddy, etc.).

The settings in flake.nix override the module defaults: externalInterface = "br0", bridgeName = "vmbr0", hostIP = "10.200.0.1", subnet = "10.200.0.0/24".

The physical host (modules/hosts/vanessa/default.nix) owns the br0 bridge itself (enp4s0 bridged into br0, shared with the Home Assistant VM) and trusts br0. It deliberately does not import the microvm host module — that belongs to @luxus/microvm-host.

Guest networking

In agent-vm-guest.nix, each guest configures a static uplink via systemd-networkd (20-uplink): Address = <ip>/24, Gateway = 10.200.0.1, DNS = 10.200.0.1, DHCP off, wait-online disabled.

Port forwarding

@luxus/agent-vm exposes a forwardPorts option that maps to networking.nat.forwardPorts (host port → <guestIP>:guestPort). Both current VMs set forwardPorts = [ ] (no host-side forwards); access is via the bridge IP or VSOCK SSH.

Per-VM orchestration (@luxus/agent-vm)

services/agent-vm/default.nix (role host) turns one instance into:

1. microvm.vms.<vmName> — restartIfChanged = true, autostart, and the guest config built by modules/nixos/agent-vm-guest.nix with the instance settings.
2. agent-vm-prep-<vmName>.service — a oneshot (RemainAfterExit) prep unit that builds the secret bundle and ensures the project root exists. It runs before both microvm@<vmName>.service and microvm-virtiofsd@<vmName>.service.
3. microvm@<vmName>.service overrides — requires/after the prep unit, placed in agent.slice, with the QoS limits applied.
4. NAT port forwards — only when forwardPorts is non-empty.

Secret flow

Secrets never reach the guest by mounting Clan paths directly — the host UID/GID/mode of /run/secrets/... may not map correctly to the guest user over virtiofs. Instead, the prep service materializes a VM-readable bundle.

flowchart LR
  vars["host Clan vars<br/>/run/secrets/vars/&lt;name&gt;/{key,token,value}"]
  prep["agent-vm-prep-&lt;vm&gt;.service<br/>(oneshot, before microvm)"]
  bundle["/var/lib/microvm-bundles/&lt;hostName&gt;/secrets<br/>0444 root:root"]
  viofs["virtiofs share (tag: secrets, read-only)"]
  guest["/run/host-secrets in guest"]
  env["exported env vars<br/>(zsh shellInit)"]
  hermes["/run/hermes/.env (tmpfs)<br/>hermes-gateway EnvironmentFile"]

  vars --> prep --> bundle --> viofs --> guest
  guest --> env
  guest --> hermes

Host side: building the bundle

agent-vm-prep-<vmName>.service (in services/agent-vm/default.nix):

• Clears and recreates /var/lib/microvm-bundles/<hostName>/secrets (0755 root:root).
• For secretFiles / environment: installs each secret as a flat file named by the env var (ANTHROPIC_API_KEY, …) mode 0444 root:root.
• Guest oneshot agent-env.service builds /run/agent.env from those files (after virtiofs mounts; not activationScripts).
• Ensures projectRoot exists on the host (0755 luxus:users).

Guest side: mount and export

modules/nixos/agent-vm-guest.nix mounts the bundle read-only via virtiofs:

source     = /var/lib/microvm-bundles/<hostName>/secrets
mountPoint = /run/host-secrets   (readOnly)

The zsh shellInit then exports variables three ways:

1. agent.secrets (per agent type) — export NAME=$(cat /run/host-secrets/NAME).
2. A scan of /run/host-secrets (and ~/.config/agent-secrets) that exports only files whose entire name is a valid UPPER_SNAKE env identifier — a stray *.sig/lowercase/dotfile can’t silently become a variable.
3. Declarative environment mappings — reads /run/host-secrets/<secretName>/{key,token,value} and exports under the chosen envName.

Secrets per VM (from inventory in modules/clan/services.nix)

• agent-jadorey / agent-tina / agent-pi-finalform (environment): typically OPENROUTER_API_KEY, ANTHROPIC_API_KEY, OPENCODE_API_KEY, and related AI keys (see inventory for the exact map per VM).

QoS & systemd ordering

Ordering enforced by @luxus/agent-vm and @luxus/microvm-host:

agent-vm-prep-<vm>.service (oneshot)
   before: microvm@<vm>.service, microvm-virtiofsd@<vm>.service
microvm@<vm>.service
   requires/after: agent-vm-prep-<vm>.service
   Slice: agent.slice
dnsmasq
   after/requires: sys-subsystem-net-devices-vmbr0.device

Per-VM limits (systemd, accounting enabled):

VM	CPUQuota	MemoryHigh	MemoryMax
agent-jadorey	250%	1152M	1280M
agent-pi-finalform	250%	1536M	1792M
agent-tina	250%	1152M	1280M

Source of truth

Concern	File
VM inventory and identity	flake.nix
Host bridge, NAT, DNS, microvm host import	services/microvm-host/default.nix
Per-VM host orchestration, prep, secret bundle, port forwards	services/agent-vm/default.nix
Guest OS config	modules/nixos/agent-vm-guest.nix
Physical host config	modules/hosts/vanessa/default.nix
Current ops notes	machines/vanessa/README.md

Operations

Run on vanessa:

sudo systemctl status microvm@agent-jadorey
sudo systemctl status microvm@agent-pi-finalform
sudo systemctl status agent-vm-prep-agent-pi-finalform
sudo journalctl -u microvm@agent-jadorey -f
ssh luxus@10.200.0.4            # via the bridge

Scripted runtime verification (run from emily after deploy):

scripts/verify-agent-vms.sh

Related documentation

• Machine & Service Inventory
• Secret Inventory
• Hindsight Stack
• Ops notes: machines/vanessa/README.md