Fediversity

Feature Specification: Component spire

Created: 2026-10-07

Status: Implemented

Input: Reconstructed from the code at 093e68b6f (core/components/hosting-provider/spire/default.nix, core/components/hosting-provider/spire/spire.nix, core/components/hosting-provider/spire/retire-name.sh, core/effects/tf/incus/host/retire-ek.sh, core/resources/spiffe/default.nix, core/resources/spiffe/spire-agent.nix, core/setups/wiring/spire-roster-test.nix, core/checks/default.nix, core/checks/nix-unit-tests.nix).

User Scenarios & Testing (mandatory)

User Story 1 - Give each host a SPIFFE identity from its TPM (Priority: P1)

Nodes of a deployment need a workload identity to log in to OpenBao. The spire component runs the SPIRE server for the trust domain fediversity. The server attests agents with the TPM NodeAttestor in EK-hash allow-list mode. For each known host, spire-register allow-lists the EK hash and creates the entry spiffe://fediversity/host/<name>. The agents on the consumer nodes come from the spiffe resource. The component declares provides = [ "spiffe" ], so the agents of the group attest to this node.

Why this priority: without an entry, the agent gets no SVID, and every OpenBao cert login from that node fails (see openbao).

Independent Test: integrations-spire-openbao and integrations-spire-incus. The second test attests an incus-deployed node through its incus vTPM (see incus).

Acceptance Scenarios:

  1. Given consumerHostHashes = { host-a = "<hash>"; }, When the node boots, Then tmpfiles seeds /var/lib/spire-server/host-ek-hashes/host-a, and spire-register creates the host/host-a entry.
  2. Given a deploy that drops a <name> file with an EK hash into /var/lib/spire-server/host-ek-hashes, When spire-register runs, Then it writes the allow-list marker into /var/lib/spire-server/ek-hashes and creates the entry with parent spire/agent/tpm/<hash> and selector unix:uid:0.
  3. Given a host whose EK hash changes, When spire-register runs, Then it re-points the entry to the new parent and removes the old marker.
  4. Given two drop-files with the same EK hash, When spire-register runs, Then it keeps the newest name and retires the others.

User Story 2 - Retire identities of removed hosts (Priority: P2)

A host that leaves the deployment must lose its identity. Otherwise anything that holds its EK keeps the identity. spire-register retires each drop-dir name that is absent from hostRoster and otherDeploymentRoster. A deployment that removes its own instances calls spire-retire-name on destroy, through retire-ek.sh.

Why this priority: a stale identity is a security gap, but it does not stop a live deployment.

Independent Test: integrations-spire-incus retires the old name of a renamed node and a node absent from hostRoster. nix-unit-setups-spire-roster and nix-unit-setups-spire-registration test the rosters that the setups render.

Acceptance Scenarios:

  1. Given a non-empty hostRoster, When a drop-dir name is in neither roster, Then spire-register retires it.
  2. Given otherDeploymentRoster = [ "app" ], When the drop-dir holds abc123-app, Then spire-register keeps it.
  3. Given both rosters empty, When spire-register runs, Then it retires no name.
  4. Given two names with the same EK hash, When spire-retire-name removes one, Then the allow-list marker stays for the other.

User Story 3 - Publish the trust bundle (Priority: P3)

Agents and other consumers need the CA bundle of the server. The spire-bundle-export unit writes the bundle every 15 minutes. The deploy reads it from /var/lib/spire-server/ca-bundle.pem and gives it to each agent (see the spiffe resource). The component does not serve the bundle on the network.

Why this priority: an agent needs the bundle to attest, and keeps it current over its own stream after that.

Independent Test: integrations-cold-start. A node that deploys before its SPIRE server exists still finishes activation.

Acceptance Scenarios:

  1. Given a running server, When spire-bundle-export runs, Then it writes /var/lib/spire-server/ca-bundle.pem.
  2. Given a host that the deploy registers, When register-ek.sh runs, Then it reads the bundle on this node and writes it to the trust_bundle_path of the agent on that host.

User Story 4 - Recover after the server loses its state (Priority: P2)

An operator recovers the fleet after the server loses its state (/var/lib/spire/server, which holds the datastore and keys.json, or /var/lib/spire-server). There is no backup to restore (see Assumptions). The server starts with a new key, a new CA and an empty datastore, and each agent and OpenBao still trust the old CA.

Recovery steps, on the server node unless a step says otherwise:

  1. Start the server, or redeploy its node. tmpfiles makes the state directories and the drop-files of consumerHostHashes again.
  2. Run systemctl start spire-bundle-export.service, so that /var/lib/spire-server/ca-bundle.pem holds the new CA now and not after the next timer run (at most 15 minutes).
  3. Run systemctl restart spire-register.service when the drop-files in /var/lib/spire-server/host-ek-hashes are still there but the datastore is not. register-ek.sh does not restart spire-register for a drop-file that holds the same hash, so without this step the entries stay absent until the next change of a drop-file.
  4. Apply each deployment that registers hosts with this server. This includes each deployment that owns the hosts of otherDeploymentRoster. The apply writes the new bundle to the trust_bundle_path of each agent (terraform_data.trust_bundle). On the hypervisor host, the host-register-ek timer does this within 5 minutes. As an alternative to an apply, copy /var/lib/spire-server/ca-bundle.pem to /var/lib/spire-agent/trust-bundle.pem on each node, with mode 0444, as a temporary file that you move into place.

No step on the agent nodes is necessary after the bundle arrives. A running agent fails its next sync with the server (an unknown authority), and SPIRE stops the agent, because rebootstrap_mode is never. systemd starts it again (Restart=on-failure), spire-agent-trust-bundle removes the cached SVID that does not chain to the new bundle, and the agent attests again. Until the bundle arrives, the agent stops and starts in a loop.

OpenBao follows without a step: openbao-spire-ca-bundle runs when the bundle of its own agent changes, waits for that agent to attest to the new CA (see openbao), installs the bundle and re-seeds the cert auth roles. The listener needs no reload. Until then, each cert login fails and each node keeps its current token until that token expires.

Why this priority: until the recovery is complete, no node can log in to OpenBao, so no node gets a new secret or a new token.

Independent Test: the subtest "The fleet recovers after the SPIRE server loses its state" of integrations-spire-openbao deletes the state of the server, does steps 1 to 4, and makes sure that the running agent gets an SVID from the new CA with no step on its node, and that a cert login succeeds after OpenBao has the new bundle. integrations-openbao-spire-ca-bundle tests the wait of openbao-spire-ca-bundle.

Acceptance Scenarios:

  1. Given a server with a new CA, and an agent that runs and has the new bundle at its trust_bundle_path, When the agent syncs with the server, Then it stops, starts again, and holds an SVID that chains to the new bundle, with no step on its node.
  2. Given OpenBao with the old bundle, When a node logs in with an SVID from the new CA, Then the login fails and the node keeps its current token.
  3. Given OpenBao with the new bundle and re-seeded cert auth roles, When a node logs in, Then it gets a new token, and the credential daemon reads secrets with it.

Edge Cases

  • The server is not up yet: spire-register waits in a health-check loop. On failure it restarts after 30 seconds, with no start limit (StartLimitIntervalSec=0).
  • spire-server fails: spire-register only wants it, so it does not stop with it.
  • A retire runs twice: spire-retire-name is idempotent.
  • A name with an operator namespace prefix is gone but matches otherDeploymentRoster: the name stays until its owning deployment retires it.
  • The server loses its state: see User Story 4 for the recovery steps.

Requirements (mandatory)

Functional Requirements

  • FR-001: The component MUST declare components.spire with name = "SPIRE", package = pkgs.spire, and an implementation that imports spire.nix.
  • FR-002: The server MUST use the trust domain fediversity, listen on port 8081, use the SQLite datastore and the disk key manager.
  • FR-003: The server MUST attest agents with NodeAttestor.tpm and hash_path /var/lib/spire-server/ek-hashes.
  • FR-004: The drop-dir /var/lib/spire-server/host-ek-hashes MUST have mode 0700, and the allow-list dir MUST have mode 0755.
  • FR-005: consumerHostHashes MUST seed one drop-file for each host through tmpfiles.
  • FR-006: spire-register MUST be a oneshot with RemainAfterExit, Restart=on-failure and a 30 second delay.
  • FR-007: spire-register MUST, in order, wait for server health, prune by roster, retire duplicate-EK names, and reconcile entries.
  • FR-008: Each entry MUST be spiffe://fediversity/host/<name> with parent spire/agent/tpm/<hash> and selector unix:uid:0.
  • FR-009: The prune MUST match hostRoster exactly and otherDeploymentRoster as the entry or *-<entry>, and MUST be off when both are empty.
  • FR-010: spire-retire-name <name> [reason] MUST delete the entry and the drop-file, and remove the marker only when no other drop-file holds that hash.
  • FR-011: spire-retire-name MUST be on environment.systemPackages.
  • FR-012: spire-bundle-export MUST run from a timer every 15 minutes.
  • FR-013: The component MUST NOT serve the bundle on the network. The deploy reads it over incus exec or ssh.
  • FR-014: The component MUST declare provides = [ "spiffe" ], so the setups route the spiffe server address of the group to this node.

Key Entities

  • Drop-file: /var/lib/spire-server/host-ek-hashes/<name>, which holds the EK hash of one host.
  • Allow-list marker: a file in /var/lib/spire-server/ek-hashes that names an EK hash the TPM attestor accepts.
  • Host entry: spiffe://fediversity/host/<name>, the SPIFFE ID of one host.
  • Host roster (hostRoster): the host names this deployment provisions.
  • Other-deployment roster (otherDeploymentRoster): name stems that other deployments register under a namespace prefix.
  • Trust bundle: /var/lib/spire-server/ca-bundle.pem, which holds the current CA and the prepared next CA.

Success Criteria (mandatory)

Measurable Outcomes

  • SC-001: integrations-spire-openbao and integrations-spire-incus pass.
  • SC-002: integrations-cold-start passes.
  • SC-003: nix-unit-setups-spire-roster and nix-unit-setups-spire-registration pass.

Assumptions

  • The deploy writes drop-files for hosts that it provisions at deploy time, through register-ek.sh.
  • The setups render hostRoster and otherDeploymentRoster from the full group roster (see setups and wiring).
  • Consumer nodes run a SPIRE agent from the spiffe resource and have a TPM 2.0.
  • OpenBao trusts the bundle of this server for cert logins. It gets the bundle from its own agent (see openbao).
  • The server has no offsite backup. Its state is the datastore and keys.json, and the deploy makes all of the datastore again. A copy of keys.json is a copy of the CA key, which can sign an SVID for each host, and thus get each secret that a host can read in OpenBao. All restic repositories use the same password, thus a backup would give this key to each node that can read a backup bucket. The CA has a lifetime of 24 hours, thus a daily copy is near its end or expired when it is restored. The server node has no SPIRE agent, thus it cannot get the S3 key of a bucket from OpenBao KV.

Known Deviations

  • spire.nix has a TODO: the datastore is SQLite and not PostgreSQL.
  • The *-<entry> match in otherDeploymentRoster keeps names that no deployment holds, until the owning deployment retires them.