Clan Vars

This document explains how Clan-managed secrets (clan.core.vars) are declared, selected per machine, turned into generators, and consumed as environment variables — and, crucially, why the generators must be assigned in a module’s config body rather than via importApply in the imports list.

It complements Vars and Environment, which covers the consumer side (env vars on normal machines vs. MicroVM agents). This document focuses on the evaluation architecture and the cross-architecture workaround tracked in issue #19.

TL;DR

Secrets are defined once in lib/secret-registry.nix.
Each machine selects a subset of keys in lib/machine-env-matrix.nix.
lib/vars-generators.nix is a pure function that turns a key list into the clan.core.vars.generators attrset.
Each machine assigns that attrset with config.clan.core.vars.generators = ... in its module body, computed in a let block — never via importApply inside imports.
Doing this in imports causes infinite recursion during Clan’s clanInternals vars selectors when the CLI runs on one architecture (e.g. aarch64-darwin) and evaluates machines of another (e.g. x86_64-linux).

The Pipeline

flowchart TD
  reg["lib/secret-registry.nix<br/>secret definitions (source of truth)"] --> matrix["lib/machine-env-matrix.nix<br/>per-machine key selection"]
  matrix --> gen["lib/vars-generators.nix<br/>pure: keys → generators attrset"]
  gen --> cfg["config.clan.core.vars.generators<br/>(machine module body)"]
  cfg --> sops[("SOPS-encrypted vars / deployed files")]
  cfg --> consumers["consumers"]
  consumers --> hjem["modules/shared/vars/hjem-session-variables.nix<br/>environment.sessionVariables (*_FILE)"]
  consumers --> shell["session-vars-spec.nix + render.nix<br/>bash/nushell extras"]
  consumers --> sec["lib/secrets.nix<br/>getPath / getVmPath / mkPasswordCmd"]

1. `lib/secret-registry.nix` — the source of truth

A function { pkgs ? null }: returning an attrset keyed by logical name (openrouter, anthropic, gmail, …). Two helpers build the entries:

mkPromptedSecret — a secret (or non-secret setting) populated from an interactive prompt. The generator script copies the prompt into the out file:
```
script = ''
  cat "$prompts/${promptAttr}" > $out/${fileAttr}
'';
```

Each entry carries metadata used downstream:

generator — the Clan generator name (e.g. anthropic-api-key).
envVar — the environment variable name (or null if not exported).
fileAttr — the output file name (e.g. key, token, password, name).
secret — whether the value is secret (affects _FILE vs. direct export).
files.<fileAttr> — Clan file declaration (secret, owner, mode).
prompts.<promptAttr> — prompt definition (for prompted secrets).
domain / kind — classification metadata.

Note the pkgs ? null default: the registry can be imported without a package set, which is what the pure consumers rely on (see below).

2. `lib/machine-env-matrix.nix` — per-machine selection

A plain attrset mapping machine name → list of registry keys. There is also a sharedCore list. For example, lea is intentionally minimal:

lea = [
  "cloudflare-account-id"
  "hindsight"
  "deepseek"
  "gmail"
  "icloud"
];

while emily selects a much larger set. This is how a machine controls exactly which secrets/vars it materializes.

3. `lib/vars-generators.nix` — pure keys → generators

A pure function { keys, pkgs }: that:

imports the registry (import ./secret-registry.nix { inherit pkgs; }),
selects entries by key (throwing on an unknown key),
maps each entry to a Clan generator via mkGenerator, copying only files, script, share, and (when present) prompts / runtimeInputs,
returns builtins.listToAttrs keyed by entry.generator.

The output is exactly the shape expected by clan.core.vars.generators.

4. Assignment in the machine module body

Each machine computes the generators in a let block and assigns them in config:

## modules/hosts/lea/default.nix
let
  envMatrix = import ../../lib/machine-env-matrix.nix;
  varsGenerators = import ../../lib/vars-generators.nix {
    keys = envMatrix.lea;
    inherit pkgs;
  };
in
{
  # IMPORTANT: set in the module body (as `config`), NOT via importApply in
  # the `imports` list. Putting it in imports causes infinite recursion during
  # Clan's clanInternals vars selectors when the CLI is on a different arch.
  clan.core.vars.generators = varsGenerators;
  ...
}

modules/hosts/emily/default.nix (aarch64-darwin) does the same with envMatrix.emily.

5. Consumers

The generators feed consumers that avoid unsafe cross-arch imports:

modules/shared/vars/hjem-session-variables.nix — Hjem module wired from modules/clan/hjem-users.nix when keys is set. Emits *_FILE paths for secrets and direct values for non-secrets from osConfig.clan.core.vars.generators.
lib/session-vars-spec.nix + lib/session-vars-render.nix — declarative shell extras (hydration, composed URLs) rendered to bash and nushell from one spec.
lib/secrets.nix { pkgs ? null }: — accessor helpers:
- getPath { osConfig, key } reads the deployed file path from osConfig.clan.core.vars.generators (the assigned config) — only safe on the actual target, not during introspection.
- getVmPath { key, mountPoint } builds a virtiofs-style path string purely.
- mkPasswordCmd { osConfig, key } → "cat <path>" for password-command consumers.

lib/session-vars-spec.nix declares shared + per-machine session-var steps; lib/session-vars-render.nix renders both bash and nushell from that single spec (kept pure for cross-arch introspection). lib/machine-session-extras.nix merges steps per machine and feeds extraSessionVars / extraNuSessionVars.

modules/clan/hjem-users.nix imports hjem-session-variables.nix as a Hjem module (when keys is set), not as a top-level let import — so generator paths are resolved inside Hjem’s module evaluation on the target, not during bare clanInternals filtering of machine source trees.

Why Generators Must Be Set in `config`, Not `imports`

This is the core of issue #19.

The cross-architecture evaluation model

The Clan CLI is frequently run on one machine (e.g. a Mac, aarch64-darwin) while it needs to introspect and deploy machines of another architecture (e.g. lea, x86_64-linux). To do this, Clan exposes clanInternals — a set of selectors that evaluate machine configuration to extract metadata (including the vars generators) without building the whole system closure.

These selectors evaluate the module fixpoint enough to read clan.core.vars.generators. Because the CLI is on a different architecture, this evaluation runs with a pkgs / system that does not match the target, and Clan filters the source tree it feeds into evaluation.

The recursion risk

The NixOS module system computes everything via a single recursive fixpoint (config depends on config). Two facts interact badly:

imports is resolved early, while the module list and the fixpoint are still being assembled. Modules in imports must be determinable before the final config exists.
importApply produces a module whose content is derived from arguments. If that content (the vars generators) is itself derived from evaluating parts of the configuration that Clan’s clanInternals vars selector is also trying to read, the selector must finish the fixpoint to know the imports, but it needs the imports to start the fixpoint.

Under same-architecture evaluation this can sometimes resolve, but under the cross-arch clanInternals path — with a filtered source tree and a mismatched pkgs — it closes into an infinite recursion: the vars selector triggers evaluation of the import that produces the vars, which re-triggers the selector.

The fix

Assign the generators as a normal config value computed in a let block:

let
  varsGenerators = import ../../lib/vars-generators.nix { keys = ...; inherit pkgs; };
in
{
  clan.core.vars.generators = varsGenerators;
}

This keeps the generators inside the ordinary config fixpoint (read by the selector as a plain value) instead of feeding them back through imports. The let block runs purely — vars-generators.nix only imports the registry and does listToAttrs — so the selector can read the result without re-entering import resolution.

The inline comments in both machine files document this requirement; do not “clean them up” into an importApply.

How Cross-Arch Evaluation Stays Safe

Several deliberate choices keep introspection from another architecture working:

Registry importable without pkgs — secret-registry.nix defaults pkgs ? null, and secrets.nix / session-vars-spec.nix import it with pkgs = null. No package set is forced during pure metadata reads.
Pure consumers — session-vars-spec.nix and session-vars-render.nix perform no derivation building and stay free of nixpkgs lib.
Hjem module for session paths — hjem-session-variables.nix reads generator paths from osConfig inside Hjem, not from a let-bound pure function in machine configuration.nix.
Activation-time secrets — actual secret values are read from Clan var files at activation time on the target, not at evaluation time. The machine comments note: “Secrets are now populated at activation time from Clan var files to avoid evaluation cycles during clan m u on mixed-architecture fleets.”
Narrow eval-time exceptions — getPath / mkPasswordCmd do read the path at HM evaluation time (e.g. accounts.email.*.passwordCommand), but the surface is intentionally tiny, so the risk of triggering global introspection cycles is low.

When This Pattern Is Needed

Use the full config-assigned, pure-function pattern when:

You have a mixed Darwin + Linux fleet managed by a single Clan CLI that runs on one architecture and deploys the others.
Machines need per-machine secret selection (different keys per machine).
You rely on clan vars / clan machines update introspection across architectures (the clanInternals selectors).

Simpler approaches suffice when:

Every machine shares the same architecture as the CLI host, and you are not hitting clanInternals cross-arch selectors — a static clan.core.vars.generators = { ... } (or even an imported module) typically works.
A machine needs all secrets — you can pass keys = null (or []) to vars-generators.nix, which selects the entire registry.
The value is a non-secret that can be exported directly (no _FILE indirection needed).

Examples

Adding a new secret end-to-end

Add the entry to lib/secret-registry.nix:

mistral = mkPromptedSecret {
  generator = "mistral-api-key";
  envVar = "MISTRAL_API_KEY";
  description = "Mistral API Key (from console.mistral.ai)";
  domain = "ai";
  kind = "api-key";
};

Add the key to each machine that should have it in lib/machine-env-matrix.nix:
```
emily = [ ... "mistral" ];
```
Nothing else changes — the existing config.clan.core.vars.generators assignment and the home.sessionVariables consumer pick it up automatically.

Reading a path on the target (safe site)

## Inside a module evaluated on the actual target:
secrets = import ../../lib/secrets.nix { inherit pkgs; };
passwordCommand = secrets.mkPasswordCmd { inherit osConfig; key = "gmail"; };

Anti-Patterns

❌ Setting generators via importApply in imports:
```
imports = [
  (lib.modules.importApply ../../lib/vars-generators.nix { keys = ...; })
];
```
Causes infinite recursion during cross-arch clanInternals. Use a let block + config.clan.core.vars.generators instead.
❌ Pure let-bound session var builders in machine configs — evaluating generator .path during machine let blocks can fail cross-arch or before clan vars generate. Use the hjem-session-variables.nix module instead.
❌ Forcing pkgs in pure metadata reads — always import the registry with pkgs = null from pure consumers.
❌ Baking secret values into the store at eval time — only paths (*_FILE) for secrets; populate values at activation time.

Troubleshooting

infinite recursion encountered while running clan vars … or clan machines update from a Mac targeting Linux machines — almost always a vars generator (or other vars-derived value) being introduced through imports / importApply instead of config. Move it into a let block and assign config.clan.core.vars.generators.
Unknown secret key in envMatrix: <key> (from vars-generators.nix) — the machine’s key list in machine-env-matrix.nix references a key not present in secret-registry.nix. Add the registry entry or fix the typo.
Generator '<name>' not configured for secret key '<key>' (from secrets.nix getPath) — the machine’s keys list omits a key that a consumer is trying to read at eval time. Add the key to the machine’s matrix entry.
Env var missing in the shell on the target — confirm the registry entry has a non-null envVar and the key is included in the machine’s machine-env-matrix list; secrets appear as <NAME>_FILE, non-secrets as <NAME>.
Path missing during introspection but present on target — expected. Path reads via getPath are only valid on the real target; introspection paths intentionally avoid resolving them.

lib/secret-registry.nix — secret definitions (source of truth)
lib/machine-env-matrix.nix — per-machine key selection
lib/vars-generators.nix — pure keys → clan.core.vars.generators
modules/shared/vars/hjem-session-variables.nix — Hjem environment.sessionVariables (*_FILE)
lib/session-vars-spec.nix — declarative session-var steps (shared + per-machine)
lib/session-vars-render.nix — renders steps to bash + nushell
lib/machine-session-extras.nix — per-machine step merge + render
lib/secrets.nix — getPath / getVmPath / mkPasswordCmd helpers
modules/clan/hjem-users.nix — Hjem user builder (wires hjem-session-variables.nix when keys set)
adr/0003-session-vars-pipeline.md — pipeline ADR incl. Clan vars vs sops-nix decision
modules/hosts/lea/default.nix, modules/hosts/emily/default.nix — the let-block + config assignment pattern
architecture/vars — consumer-side env var delivery (normal machines + MicroVM agents)