Fediversity

Threat model

This page tells what Fediversity must protect, from whom, and how the code protects it. For each threat, it gives the control in the code, or the issue that tracks the gap. The page uses STRIDE to sort the threats: spoofing (S), tampering (T), repudiation (R), information disclosure (I), denial of service (D) and elevation of privilege (E).

The page states the current state of the code on main. When you change a specification or a control on this page, update the page in the same pull request. To report a vulnerability, read SECURITY.md.

Scope

The scope is the scope of SECURITY.md:

  • the code in this repository, and the configuration that it gives to the applications that it deploys;
  • the development infrastructure that this repository deploys, for example git.fediversity.eu and its CI runners.

The applications themselves, for example Mastodon or PeerTube, are out of scope. Their configuration is in scope.

Assets

Asset Where it is Why it is important
Secrets in OpenBao The KV, PKI, LDAP and database engines of the OpenBao node of each hosting provider (core/components/hosting-provider/openbao/) They give access to all services of all deployments.
The OpenBao unseal key and root token The unseal key is sealed to the TPM of the OpenBao node. The root token is in /var/lib/openbao-init/root-token (openbao.nix). Each one gives full control of OpenBao.
Deploy credentials The SSH key of CD (core/keys/cd-ssh-key.pub is its public half), the Forgejo Actions secrets, the OIDC token of a CD run, and the operator deploy credentials (core/components/hosting-provider/deploy-credentials.nix) A deploy runs as root on the nodes.
Contributor keys core/keys/contributors/ They give root on the development infrastructure, and they decrypt the agenix secrets (core/keys/README.md).
agenix secrets core/resources/secrets/*.age, recipients in core/resources/secrets/secrets.nix Forge database and mail passwords, the runner registration token, the OpenPGP key for vulnerability reports.
The OpenTofu state The terraform-backend node of the hosting provider (core/components/hosting-provider/terraform-backend/) The state maps the infrastructure and can contain secrets.
User data The databases (core/resources/postgresql/), buckets (core/resources/garage/) and caches (core/components/shared/valkey/) of the applications, and their offsite backups Personal data of the end users of the applications (GDPR).
The Nix store and the binary caches Each node, the nix-serve cache on the incus host and its signing key in /var/lib/incus-bridge-cache/ (core/components/hosting-provider/incus.nix), the stores of the CI runners Each node runs what its store contains.
The source code and its history git.fediversity.eu (core/components/dev/forgejo/) CD deploys what is on main.
The SPIRE trust bundle and the node identities The SPIRE server of the hosting provider (core/components/hosting-provider/spire/) A node logs in to OpenBao with its SVID.

Parties

  • Hosting provider. It runs the incus host, the shared services (OpenBao, SPIRE, the state backend, PostgreSQL, Garage, Authelia, lldap, telemetry, windmill) and the panel and API. Its administrators are fediversity-admins in lldap, and the persons in ADMIN_USERS.
  • Operator. A customer of the hosting provider. It configures and deploys applications through the panel. It does not get shell access to the nodes.
  • End user. A person who uses an application of an operator, for example a Mastodon account. End users and operators are on the internet.
  • Contributor. A person with write access to the repository. A contributor whose key is in core/keys/contributors/ is also an administrator of the development infrastructure.
  • CI and CD runners. Hosts that run Forgejo Actions jobs (core/components/dev/forgejo-runner/). The same pool runs the checks of pull requests and the CD jobs that deploy.
  • The internet. All other parties, which includes the persons who open an account on git.fediversity.eu and send pull requests from a fork.
  • Upstream projects. nixpkgs, the pinned sources in npins/, and the public binary caches.

Trust boundaries

  1. Internet to the public services. The ports that each node publishes to everyone: HTTPS of the applications and the admin UIs, ACME on :80, DNS on :53, and SMTP on :25 on a node that takes DMARC reports.
  2. Internet to the administrative surface. SSH on :22 and the incus API on :8443. The packet filter admits only ADMIN_SOURCES to these ports (notes/private-access.md).
  3. Operator to the hosting provider. The panel and the API, and the code that a deploy evaluates for an operator.
  4. Operator deployment to the hosting-provider mesh. The nodes of an operator deployment and the nodes of the hosting provider share one incus bridge.
  5. Node to the secret store. Each node gets only the secrets of its own role from OpenBao.
  6. Forge to CI and CD. A pull request runs code on the runners. A push to main deploys.
  7. Contributors and administrators to the nodes. Root SSH keys and the agenix keys.
  8. Upstream and caches to the Nix store. The store paths that a node gets from a substituter.

Entry points

Entry point Who can reach it Control Code
HTTPS of applications and UIs Everyone TLS from the ssl resource. Authelia forward-auth on the admin UIs. core/resources/ssl/, core/components/shared/authelia/
:80 (ACME HTTP-01) Everyone, only on a node that gets a public certificate Published only for the ACME challenge core/setups/wiring/firewall-policy.nix
DNS :53 Everyone, only on a public named DNSSEC signing, update-policy for dynamic updates core/components/hosting-provider/bind.nix
SSH :22 ADMIN_SOURCES and the internal sources Packet filter, keys only, no passwords core/resources/firewall/, core/setups/nixos/networking.nix
incus API :8443 ADMIN_SOURCES and the internal sources Packet filter, client certificates, OIDC core/setups/wiring/firewall-policy.nix, incus specification
Panel Everyone Django login or OIDC, CSRF origin check panel/src/panel/settings.py, panel/nix/component/panel.nix
API The panel, and browsers with a panel token Authorization callback to the panel for each call api/src/fediversity_api/authz.py, API authorization specification
OpenBao Nodes of the mesh, administrators mTLS listener, cert, jwt and oidc auth, one policy for each role core/components/hosting-provider/openbao/
State backend The internal sources Packet filter, HTTPS through nginx core/components/hosting-provider/terraform-backend/
git.fediversity.eu Everyone Anubis bot gate, Forgejo accounts core/components/dev/forgejo/
Forgejo Actions Repositories of the fediversity organization, and pull requests to them Organization registration token, no secrets for fork pull requests core/components/dev/forgejo-runner/forgejo-runner.nix

Threats and controls

Internet to the public services

STRIDE Threat Control Gap
S, I An attacker reads or changes traffic to a service. Each served name gets a certificate from the ssl resource: ACME or the private CA in OpenBao (core/resources/ssl/, ssl specification). The panel sets forceSSL in production (panel/nix/configuration.nix). Traffic between services on the bridge is not always encrypted (#2179).
S An attacker guesses the password of an admin UI. Authelia regulation locks out after failed logins (core/components/shared/authelia/default.nix). The admin UIs need only one factor (#2181).
E An attacker uses a known vulnerability in a deployed package. A daily vulnix scan of the closure of each node fails on each CVE that is not in the triage list (.forgejo/workflows/vulnerability-scan.yaml). The daily run opens an issue for each new CVE of high or critical severity (.forgejo/workflows/vulnix-report.py). Each node has a CycloneDX SBOM of its runtime closure (core/setups/nixos/sbom.nix, system.build.sbom). The API does not give the SBOM of a deployment (#2176). Known CVEs wait for a fix (#2177, #2208). No release with a support period (#2178).
T An attacker uses the CORS rule of the website buckets. None. The website buckets have a CORS GET rule from * (Known Deviations of the Mastodon, PeerTube and Pixelfed specifications).
D An attacker floods a service. Anubis on the forge (core/components/dev/forgejo/anubis.nix), the ICMP rate limit of the node (core/setups/nixos/networking.nix). The nginx configuration of the services has no rate limit, except for the password login of the panel.

Internet to the administrative surface

STRIDE Threat Control Gap
S, E An attacker logs in with SSH. The input-gate chain admits :22 and :8443 only from adminSources and the internal sources (core/resources/firewall/nixos.nix). A node with a public address publishes no administrative port (core/setups/wiring/firewall-policy.nix). SSH takes keys only, PasswordAuthentication = false (core/setups/nixos/networking.nix). When ADMIN_SOURCES is empty, the administrative ports are open to all sources. The design permits this only on the management network (firewall-policy.nix). root can log in with SSH (#24, #295).
S A deploy connects to a false host. None. The ssh method turns off the host-key check (#2137).
E A user with OIDC access to incus changes all instances. The incus OIDC login goes through Authelia (core/components/hosting-provider/incus/oidc.nix). incus has no fine-grained authorization, so each OIDC user is an administrator (incus specification).

Operator to the hosting provider

STRIDE Threat Control Gap
S An attacker guesses the password of an operator on the panel. The panel checks the strength of a new password (panel/src/panel/settings.py, AUTH_PASSWORD_VALIDATORS). With OIDC, the login goes through Authelia. nginx limits the POST requests to /login/ and /admin/login/ to 5 a minute for each client address (panel/nix/configuration.nix, panel_login).
S, I An attacker on the network takes the session cookie of an operator. In production, nginx sends HTTP requests to HTTPS and sends an HSTS header, and the session and CSRF cookies have the Secure flag (panel/nix/configuration.nix).
S, E An operator acts on a deployment of a different operator. The API asks the panel for each call (api/src/fediversity_api/authz.py). The panel allows a call only for the deployments of the requester, and for a list without ids it gives the ids of the deployments of the requester, which api/ limits the list to (panel/src/panel/views.py, AuthzCallback). The callback checks the bearer token of the API with hmac.compare_digest. The panel component sets the token (panel/nix/component/panel.nix). Without the token, the callback takes requests from all callers and writes a warning.
E An operator makes a deploy run code that the hosting provider did not accept. The API accepts a module_source only from an allowlist of exact repository URLs, and only a revision on a branch (api/src/fediversity_api/source_allowlist.py). The default allowlist holds only this repository (api/nix/component/default.nix, trustedSourceUrls).
I An operator secret goes into the Nix store. Deploy credentials go to the deploy as environment variables, not into the configuration file. The API removes them from the windmill job (core/components/hosting-provider/deploy-credentials.nix). A secret that the operator types in the form goes into the Nix store and into the stored configuration (#2133).
I A secret is the same for all deployments. None. Fixed restic password, fixed state backend password and a PeerTube placeholder secret (#2136).
R Nobody knows who started an action. The api records an audit event with the actor for each action that changes a deployment, in the audit_event table and in the journal (api/src/fediversity_api/audit.py). #2182.

Operator deployment to the hosting-provider mesh

STRIDE Threat Control Gap
S A node uses the address of a different node. incus MAC, IPv4 and IPv6 filtering on each NIC (core/effects/tf/incus/instance.tf).
I, T A node of one operator reaches a node of a different operator. Each operator deployment gets its own /28, and an incus network ACL drops traffic to the rest of the operator prefix (core/effects/tf/incus/operator/bridge-isolation.tf, core/effects/tf/incus/bridge-acl/). Without NetBox, the deploy stops. incus drops all IPv6 on the bridge, because the bridge has no IPv6 subnet.
I, E A node of an operator reaches the services of the hosting provider. Each service asks for a credential: an SVID for OpenBao, scram-sha-256 over TLS for PostgreSQL (core/resources/postgresql/postgresql-provider.nix), passwords and ACLs for valkey (core/components/shared/valkey/valkey.nix). The bridge ACL lets operator nodes reach all ports of the hosting-provider nodes. The bridge subnet is an internal source on each node (core/setups/hosts-common.nix, internalSources), and the hypervisor trusts incusbr0 (core/setups/wiring/firewall-policy.nix) (#2223). LDAP is plain ldap:// (#2139), and its consumers bind as the lldap admin (#2140).
T, I A node reads or changes the OpenTofu state. The state is encrypted at rest with a key that the node makes (core/components/hosting-provider/terraform-backend/terraform-backend.nix). The backend is reachable only from the internal sources. The basic authentication of the backend accepts all requests (#2220). The default password is fixed (#2136). The lock is in memory (terraform-backend specification).
D A deployment loses the objects of its media buckets. When the operator enables it, a timer on the application node copies the media buckets each day into the backups bucket of the application, with a history of the replaced and deleted objects (core/resources/garage/media-backup.nix). A VM test does a restore (core/checks/integrations/media-backup/). The backup is off by default. It is in the same garage as the source (#2101). Its encryption password is only in OpenBao (#2102).
I A party that reads the backups bucket reads the media. rclone crypt encrypts the name and the content of each object with a password of the deployment from OpenBao. The backup key may read the media buckets, and may not write them. The metadata of an object, for example its content type, is not encrypted.
I A backup of an application keeps a deleted or erased record or object. All the backups of an application keep its data for 30 days (backupRetention.application in core/components/operator/app-resources.nix): restic prunes with --keep-within 30d, and the history of the media backup expires after retentionDays, which has the same default. media-backup-<app> erase removes an object from the media backup and from all of its history. After a restore of the media alone, the operator runs the cleanup of orphan media of the application. The operator runs erase and the cleanup. The database backups have no erase command, thus an erased record stays in them for 30 days. The database backups of lldap, Authelia and the panel keep the accounts of the hosting provider for 30 days too. The other backups of the hosting provider keep their data for approximately 6 months, for example the audit_event table of api/ and the users of Grafana and NetBox. After a restore from a backup that is older than an erasure, the operator does the erasure again before the service opens to its users. Nothing records the erasures, thus the operator keeps a list of them (hosting-provider-guide.md, "Erase personal data").

Node to the secret store

STRIDE Threat Control Gap
S A node reads the secrets of a different node. A node logs in to OpenBao with its SPIFFE SVID (cert auth). Each roster host has its own role and policy (core/components/hosting-provider/openbao/spire-auth.nix). The deploy gives each agent the SPIRE trust bundle over incus exec and ssh (core/effects/tf/incus/host/register-ek.sh). OpenBao gets the bundle from the Workload API of its own agent (core/components/hosting-provider/openbao/spire-ca-bundle.sh).
I An attacker reads the unseal key or the root token from the disk. The unseal key is sealed to the TPM, with no plaintext fallback (openbao.nix). The root token is in a file, and the reconciler state holds secrets (OpenBao specification). The data on the hypervisor disks is not encrypted (#2180).
D OpenBao or SPIRE loses its state. None. No escrow of the unseal key (#2102). No recovery after the SPIRE server loses its state (#2215). No replication (#2187).
R Nobody knows who read a secret. OpenBao writes each request and each response to the journal of its node as JSON, with secret values as HMAC-SHA256 hashes (openbao.nix). The journal goes to Loki (hosting provider guide). The audit log is not tamper-evident (#2270).

Forge to CI and CD

STRIDE Threat Control Gap
E A pull request from a fork gets the deploy credentials. Forgejo holds the workflows of a fork pull request until a maintainer approves them, unless the author has write access or a maintainer chose "always trust" for that user (Forgejo services/actions/trust.go). No workflow uses pull_request_target or issue_comment. A fork pull_request run gets no secrets and a read-only token. Only the organization registration token is used, so a repository of a different user cannot use the runners (core/components/dev/forgejo-runner/forgejo-runner.nix). After one approval, the job runs on the host (:host labels) and as the same service user as the CD jobs. Each account with write access runs code there with no review (#2228).
E A run that is not on main deploys production. The OpenBao jwt role accepts only a token of this repository on refs/heads/main. The token lives for 30 minutes (core/components/hosting-provider/openbao/cd-oidc.nix). CD and pull requests use the same runner hosts and the same Nix daemon. CD has no Incus project of its own (#1638).
T A CI build goes into a public cache. #92.
T The code that CD deploys is not reviewed. The branch protection rule on main has required_approvals: 0 and enable_push: true in the forge settings. Thus a user with write access can change main with no review (#291, #1487).
D The forge loses its data. No backups of Forgejo (#29).

Contributors and administrators to the nodes

STRIDE Threat Control Gap
E A lost contributor key gives root on all nodes. The keys are in the repository (core/keys/), so the history shows each change. Each contributor key is a recipient of the agenix secrets of each host (core/resources/secrets/secrets.nix, secrets specification). Administrators get sudo with no password and are trusted Nix users (core/setups/nixos/users.nix).
R Nobody knows which administrator did an action. The journal of each node goes to Loki (maintainer guide). The logs keep data for a limited time (hosting provider guide). The journal keeps the full client IP address.

Upstream and caches to the Nix store

STRIDE Threat Control Gap
T A changed source goes into a build. All sources are pinned with a hash in npins/sources.json.
T A node takes a false store path from a cache. The bridge cache listens only on the bridge, and the NIC filters stop address spoofing on the bridge. The host signs each path that the bridge cache serves, and a node accepts a path only with a signature from its host or from an upstream cache (require-sigs, core/setups/nixos/incus-bridge-cache.nix). A container that does not have the key of its host, for example a VM without the incus agent or an instance in a project with its own profiles, accepts unsigned paths in the bridge step of core/effects/incus/run.sh (--no-require-sigs) (#2277).

Accepted risks and open gaps

These issues track the gaps on this page:

The Known Deviations sections of the specifications give more gaps, for each component.

Maintenance

CONTRIBUTING.md tells when a pull request updates this page. The specifications are the reference for each control. This page links to them, and does not repeat them.