Fediversity

Feature Specification: Component forgejo-runner

Created: 2026-10-07

Status: Implemented

Input: Reconstructed from the code at 093e68b6f (core/components/dev/default.nix, core/components/dev/forgejo-runner/default.nix, core/components/dev/forgejo-runner/forgejo-runner.nix, core/setups/groups/dev.nix, core/checks/ci.nix), pull requests #1383, #1430, #1452, #1453, #1454, #1466, #1470, #1484, #1524, #1528, #1534, #1547, #1623, #1693, #1708, #1735, #1746, #1822, #1852, #1956, #1967, #1989 and #2017.

This spec describes one component of the dev group. The family spec ../spec.md describes what all components share. The group roster is in core/setups/groups/dev.nix, see ../../005-setups-and-wiring/spec.md.

User Scenarios & Testing (mandatory)

User Story 1 - CI jobs run on a memory tier (Priority: P1)

A workflow asks for a runner by label, for example ram-8g. The component runs a pool of Forgejo runner instances on each CI host. Each instance belongs to one memory tier, and systemd caps the instance at the MemoryMax of that tier. The instances run jobs natively on the host, with no Docker.

Why this priority: All CI of the repository runs on this pool.

Independent Test: nix-unit-setups-forgejo-runner-pool (core/setups/forgejo-runner-pool-test.nix) evaluates the pool from the roster, and the instances, MemoryMax and labels of each tier on the realised dev-fediversity-ci node, also with other pools. No check deploys the runner pool. core/checks/ci.nix reads the tier labels to route checks.

Acceptance Scenarios:

  1. Given a CI host in the pool, When it deploys, Then it runs per host 3 ram-64g, 2 ram-32g, 4 ram-16g, 5 ram-8g, 15 ram-4g, 6 ram-2g and 11 ram-1g instances.
  2. Given an instance, When it starts, Then it declares the labels native:host, ram-<N>g:host, nspawn:host and one host-not-<peer>:host per other pool member.
  3. Given an instance, When it registers, Then it connects to fediversity.forgejo-runner.forgeUrl (https://git.fediversity.eu in the dev group) with the token from the forgejo-runner-token secret and capacity = 1.

User Story 2 - A deploy does not cancel running jobs (Priority: P1)

The operator deploys a new runner configuration while jobs run. The runner units have restartIfChanged = false, so the switch does not stop them. The forgejo-runner-restarter service then drains and restarts the runners one at a time.

Why this priority: A cancelled job wastes hours of CI time and gives a false red result.

Independent Test: No check covers the drain.

Acceptance Scenarios:

  1. Given a change to any rendered runner unit, When the host switches, Then runnerUnitsStamp changes and the restarter runs.
  2. Given a rotated forgejo-runner-token.age, When the host switches, Then tokenStamp changes the runner units, and the restarter brings the pool onto the new token.
  3. Given a busy runner, When the restarter drains it, Then the job can run to jobTimeout (3 hours) before the runner stops.

User Story 3 - The host frees store space without blocking builds (Priority: P2)

CI fills the nix store. The ci-gc service frees space when the free space falls below a trigger. Builds do not wait for it.

Why this priority: A full disk fails every build on the host.

Independent Test: No check covers ci-gc.

Acceptance Scenarios:

  1. Given less than 60 GB free on /nix/store, When the ci-gc timer fires, Then ci-gc removes old result roots, deletes system generations older than 14 days, and collects garbage up to 100 GB free.
  2. Given at least 60 GB free, When the timer fires, Then ci-gc does nothing.

Edge Cases

  • fediversity.forgejo-runner.machine is not in fediversity.forgejo-runner.pool: an assertion fails the evaluation.
  • The timeouts break the order stopTimeout < jobTimeout <= shutdownTimeout < drainCap: an assertion fails the evaluation.
  • A roster entry in dev.nix has no cd = { deploy; runsJobs; }: the evaluation throws an error that names the host.
  • A result link under /tmp or /home that is older than 7 days is removed only when ci-gc runs under its trigger.
  • An act workspace under /var/lib/private/gitea-runner/*/.cache/act is removed after 1 day.
  • The disk fills before ci-gc frees space: builds fail with ENOSPC, because min-free = 0.

Requirements (mandatory)

Functional Requirements

  • FR-001: The component MUST declare components.forgejo-runner with name = "Forgejo Runner", package = pkgs.forgejo-runner and nixStore = "own".
  • FR-002: The component MUST NOT request a domain or TLS, because the host has no DNS entry.
  • FR-003: The component MUST request the secrets entry forgejo-runner-token with owner root.
  • FR-004: The component MUST import forgejo-runner.nix through the nixos-module resource.
  • FR-005: The module MUST expose fediversity.forgejo-runner.pool, a list of strings with default [ ], that holds every machine in the pool.
  • FR-005a: The component MUST have the option forgeUrl, with no default, and MUST set fediversity.forgejo-runner.forgeUrl from it. The module MUST also expose fediversity.forgejo-runner.machine, with no default, the roster name of the host.
  • FR-006: core/setups/groups/dev.nix MUST derive runnerPool from the machines with component = "forgejo-runner", MUST set fediversity.forgejo-runner.pool from it and fediversity.forgejo-runner.machine from the roster key, and MUST set forgeUrl to https:// plus the domain of the forge.
  • FR-007: Each runner instance MUST be named runner-<label>-<i> and MUST run with the MemoryMax of its tier.
  • FR-008: Each instance MUST build its server.connections from .runner before start, with a 30s fetch interval.
  • FR-009: Runner units MUST set restartIfChanged = false. forgejo-runner-restarter MUST have restartTriggers = [ runnerUnitsStamp ].
  • FR-010: runnerUnitsStamp MUST hash every rendered runner unit, and the units MUST carry tokenStamp, the hash of the file that the secrets resource gives for forgejo-runner-token.
  • FR-011: nix-daemon MUST run with MemoryMax = "200G".
  • FR-012: The module MUST set nix.gc.automatic = false and min-free = 0, and MUST run ci-gc from a timer with OnBootSec = "5min" and OnUnitInactiveSec = "5min".
  • FR-013: ci-gc MUST run with Nice = 19 and IOSchedulingClass = "idle".

Key Entities

  • runnerTiers: The list of memory tiers, each with label, count and memoryMax.
  • fediversity.forgejo-runner.pool: The pool members. The host-not-<peer> labels are the complement of fediversity.forgejo-runner.machine in the pool.
  • forgejo-runner-restarter: The service that drains and restarts the runners after a switch.
  • ci-gc: The GC service and timer for the host.
  • CI hosts: The roster machines fediversity-ci, dev2, dev3 and dev4 in core/setups/groups/dev.nix.

Success Criteria (mandatory)

Measurable Outcomes

  • SC-001: The dev group evaluates with all assertions of forgejo-runner.nix true.
  • SC-002: A deploy of a changed runner configuration cancels no running job.
  • SC-003: Free space on /nix/store returns above 100 GB after a ci-gc pass.

Assumptions

  • The pool hosts are reached by IPv4 literals and get their host name from the network resource.
  • The component requests forgejo-runner-token from core/resources/secrets/forgejo-runner-token.age. nix-unit-resources-secrets covers the secret routing, not the stamp.

Known Deviations

None.