Fediversity

Feature Specification: Deployment method tf

Created: 2026-10-07 Status: Implemented Input: Reconstructed from the code at 093e68b6f (core/effects/tf/ssh/, core/effects/tf/common/, core/effects/common/lib.nix, core/effects/common/tf-verbs.nix, core/effects/env-config.nix, core/setups/tf-hosts.nix, core/setups/hosts-common.nix, core/checks/deployment/tf-hosts/, core/checks/deployment/tf-hosts-bootstrap/), issues #336, #723, #882

The tf method updates NixOS hosts that already exist, from an OpenTofu graph. The graph holds one terraform_data resource per node. Its local-exec provisioner runs the SSH transport of the ssh method (see ssh.md) against that node. The state lives on an HTTP backend. The setup is tf-hosts, and the effect is core/effects/tf/ssh/effect.nix.

The method creates no machine. Compared to ssh, it adds state: a node deploys again only when its trigger values change. The bootstrap wrapper bootstrap-apply.sh, the verdict and the env-key publishing are in spec.md. The verbs and the state rules are in ../004-deployment-programs/spec.md.

User Scenarios & Testing (mandatory)

User Story 1 - Deploy existing hosts with tracked state (Priority: P1)

An operator runs the tf-hosts program against a running HTTP state backend. OpenTofu switches each node whose configuration changed and records the result in the backend.

Why this priority: This is the only purpose of the method.

Independent Test: deployment-tf-hosts runs a terraform-backend on the deployer and deploys two targets through tf/ssh. It checks the state at the backend and that TLS ends on the ssl resource on both targets. apps-tf runs the all program of the tf-hosts setup against a terraform-backend on the deployer, and deploys the app nodes of a test group. The deployment-* checks call the effect directly, with their own setup.nix as caller. The JSON args of the deploy cannot hold groups, thus at deploy time boundary.deploymentPlan imports the caller again with the in-tree groups. Through the tf-hosts setup, these groups would replace the test-only groups of a check.

Acceptance Scenarios:

  1. Given a reachable backend at TF_HTTP_ADDRESS, When the operator runs the program, Then OpenTofu applies with -parallelism 10 and prints the outputs as JSON.
  2. Given a node whose nixos_conf, host, config_identity, host_key and ssh_opts did not change, When the operator applies again, Then OpenTofu does not replace that node's resource, and the host is not deployed again.
  3. Given a node whose closure changed, When the operator applies, Then OpenTofu replaces the resource and its provisioner deploys that node through run.sh.

User Story 2 - Bring up the graph's own state backend (Priority: P1)

The tf-hosts graph deploys the terraform-backend node that holds its own state. On a fresh provider, that backend does not exist yet. One BOOTSTRAP=1 run applies with local state, waits for the backend, migrates the state up and applies again.

Why this priority: Without it, a fresh provider cannot get its first state backend from the graph that needs it.

Independent Test: deployment-tf-hosts-bootstrap deploys a backend target and an app target from a deployer with no backend. It checks the phase markers on stderr, the state on the backend, the terraform.tfstate.migrated marker, and a steady re-run without BOOTSTRAP.

Acceptance Scenarios:

  1. Given an unreachable backend and BOOTSTRAP=1, When the operator runs the program, Then cycle 1 applies against local state under BOOTSTRAP_STATE_DIR/<STATE_KEY>.
  2. Given that cycle 1 started the backend node, When the backend serves, Then the wrapper migrates the local state to it, renames the local file to terraform.tfstate.migrated, and applies again.
  3. Given the .migrated marker, When the operator runs the program again, Then the wrapper takes the steady HTTP path.

User Story 3 - Run a lifecycle verb against the same state (Priority: P2)

An operator runs plan, destroy, state-list or another verb of tf-verbs.nix against the graph.

Why this priority: The operator must be able to inspect and remove the graph without a full apply.

Independent Test: integrations-plan-does-not-apply, integrations-destroy-detaches-state, integrations-reclaim-own-lock and integrations-steady-refuses-lost-state test these paths of bootstrap-apply.sh. deployment-tf-hosts runs the plan verb of the effect after the apply, and apps-tf runs the all-plan verb of the tf-hosts setup after the apply of all nodes. Both run it with -detailed-exitcode, thus a plan with changes fails the check.

Acceptance Scenarios:

  1. Given TF_VERB=plan, When the wrapper runs, Then it runs tofu plan through the same address derivation, node filter and tunnel as an apply, and changes nothing.
  2. Given TF_VERB=destroy on this self-backend graph, When the wrapper runs, Then it moves the state from the backend to local state first, and runs the destroy against the local copy.
  3. Given BOOTSTRAP=1 and a TF_VERB, When the wrapper starts, Then it refuses with exit 2.

Edge Cases

  • TF_HTTP_ADDRESS is not set: the setup uses http://localhost:8080/state/project1/example, unpinned. The bootstrap migrate then derives the reachable address.
  • The backend has no state for the key, but the local working dir holds a fleet's state: the apply refuses with exit 3. BOOTSTRAP_ALLOW_LOST_STATE=1 overrides. A verb other than apply gets a warning and runs.
  • The backend is unreachable and BOOTSTRAP is not 1: tofu init or the apply fails.
  • A stale lock that this user@host holds: TF_RECLAIM_OWN_LOCK=1 releases it before the verb.
  • A failed apply leaves errored.tfstate: the wrapper salvages it into the working dir.
  • DEPLOYMENT is set: the published-names script reads the submission, and the graph drops the nodes that the submission turns off.
  • A destroy: OpenTofu removes the resources from the state. The hosts keep their last configuration, because the graph has no destroy provisioner.
  • The backend still answers after a detached destroy: the wrapper warns that the backend now holds a superseded copy.

Requirements (mandatory)

Functional Requirements

  • FR-001: The method MUST run the OpenTofu graph tf/ssh through tfRun, with the opentofu package and the hashicorp_external provider from the store.
  • FR-002: The graph MUST hold one terraform_data.nixos per node, keyed by the dotted deployment-name, with triggers_replace = [nixos_conf, host, config_identity, host_key, ssh_opts]. A change of the pinned host key or of the SSH options alone thus deploys the node again.
  • FR-003: The provisioner of each resource MUST run run.sh with a bash from the Nix store, through the runner in deploy_runner (core/effects/ssh/deploy-runner.nix), with nixos_conf, username, host, key_file, host_key and ssh_opts in the environment.
  • FR-004: nixos_conf MUST come from writeConfig with effect = "tf-hosts" and multiNode = true.
  • FR-005: The graph MUST declare backend "http" {}. httpBackend MUST map to TF_HTTP_* with POST updates and LOCK/UNLOCK on the state address.
  • FR-006: The setup MUST read TF_HTTP_ADDRESS at evaluation. The address MUST count as pinned only when the variable is set.
  • FR-007: With an httpBackend, the all environment MUST suffix the state address with -<group>.
  • FR-008: The setup MUST set bootstrap = true, so the wrapper gets BOOTSTRAP_SELF_BACKEND=1.
  • FR-009: The effect MUST pass a publishedNamesExpr, so a DEPLOYMENT run reads its node set from the submission.
  • FR-010: Every verb of tf-verbs.nix MUST run through bootstrap-apply.sh with TF_VERB set and BOOTSTRAP empty.
  • FR-011: The wrapper MUST refuse BOOTSTRAP=1 together with TF_VERB (exit 2), and an apply over lost backend state (exit 3).
  • FR-012: The wrapper MUST keep its working dir at BOOTSTRAP_STATE_DIR/<STATE_KEY>, where STATE_KEY is the sanitized backend address.
  • FR-013: The tf entry of core/effects/env-config.nix MUST read SSH_USER and SSH_NODES, and the setup MUST layer the SSH identity of each node as ssh.md FR-012 states.

Key Entities

  • Node entry (TF_VAR_nodes): {nixos_conf, config_identity, username, host, key_file, host_key, ssh_opts}, keyed by the dotted deployment-name. host_key is the pinned public host key of the node, or empty to accept the key of the first connection. ssh_opts is the sshOpts of the node as a JSON list of bare Key=Value options, and run.sh puts -o in front of each item (FR-018 of ssh.md).
  • HTTP backend (httpBackend): address, pinned, the lock and unlock addresses and methods, the basic-auth user and password, and the retry settings.
  • State key (STATE_KEY): the backend address with /:?&=#@ and spaces replaced by _.
  • Migrated marker: terraform.tfstate.migrated in the working dir. It means that the backend holds the state.

Success Criteria (mandatory)

Measurable Outcomes

  • SC-001: After an apply, the backend holds the state, every changed node runs its closure, and deployment-tf-hosts passes.
  • SC-002: A second apply with no change deploys no node. After an apply, plan shows no changes: deployment-tf-hosts checks this for the plan verb of the effect, and apps-tf for the all-plan verb of the tf-hosts setup.
  • SC-003: One BOOTSTRAP=1 run on a fresh provider ends with the state on the backend, and deployment-tf-hosts-bootstrap passes.
  • SC-004: An apply never writes over a backend key that lost its state, unless the operator sets BOOTSTRAP_ALLOW_LOST_STATE=1.

Assumptions

  • Each host already runs NixOS and accepts SSH, as for the ssh method.
  • One operator applies a given state key at a time. The backend lock enforces this.
  • The terraform-backend node of the graph serves the address in TF_HTTP_ADDRESS, or the address that the migrate derives.
  • The deployer can build every closure, or substitute it.

Known Deviations

  • httpBackend defaults the basic-auth password to "fake-secret", and the address to a localhost example (#2220).