Utility functions
Helper functions provided under core/lib and core/effects, one
section per file, rendered from their doc-comments. This covers every
function exercised by the project's nix-unit tests.
General utilities (core/lib/lib.nix): model evaluation and environment-variable overrides.
General utilities: model evaluation and environment-variable overrides.
fediversity.lib.optionalEnv
Read an environment variable, returning null when it is unset or empty.
Inputs
k- Name of the environment variable to read.
Type
optionalEnv :: String -> a | null
Example
optionalEnv "DEPLOY_HOST"
=> "fediversity-ci" # when DEPLOY_HOST is set
=> null # when DEPLOY_HOST is unset or empty
fediversity.lib.envOr
Read a JSON-encoded environment variable, falling back to default when it
is unset or empty. The variable's value is parsed with builtins.fromJSON,
so it may carry any JSON value (string, number, list, object).
Inputs
k- Name of the environment variable to read.
default- Value to return when the variable is unset or empty.
Type
envOr :: String -> a -> a
Example
envOr "PARALLELISM" 1
=> 4 # when PARALLELISM="4"
=> 1 # when PARALLELISM is unset
fediversity.lib.importDataFile
Load a hosting-provider data file, dispatching on extension: .toml is
parsed with builtins.fromTOML, .json with builtins.fromJSON, anything
else is import-ed as Nix. Returns a plain attrset suitable for
lib.recursiveUpdate.
.toml and .nix are the hand-authoring forms; .json is what a generator
emits, since it is the one round-trippable format that also carries null.
Inputs
path- Path to the data file (a Nix path or a string).
Type
importDataFile :: Path -> AttrSet
Example
importDataFile ../../examples/hosting-config.toml
=> { environments.mastodon.resources.garage.external = { ... }; ... }
fediversity.lib.getSomeAttrs
Like lib.getAttrs, but tolerates whitelisted keys that are absent from the
attribute set instead of erroring on them. Only the requested keys that
actually exist are returned.
Inputs
names- List of attribute names to pick.
attrs- Attribute set to pick them from.
Type
getSomeAttrs :: [String] -> AttrSet -> AttrSet
Example
getSomeAttrs [ "a" "c" ] { a = 1; b = 2; }
=> { a = 1; }
fediversity.lib.evalModel
Evaluate the data model together with an additional module, returning the
resulting config. Mirrors how the deployment code composes
data-model.nix with extra modules.
Inputs
module- An extra NixOS-style module merged on top of
data-model.nix(an attribute set, a function, or a path to either).
Type
evalModel :: Module -> AttrSet
Example
(evalModel { config.components = { }; }).components
=> { } # the merged data model's `config`, here with no components
fediversity.lib.moduleEnableMap
Module-driven default node enablement: enable every component that a roster
node runs and that has a declared components.<name> module in the group. A
cheap eval of the group's component module set yields the declared names,
which the roster's components are intersected with -- so a roster node whose
component has no module is left out here and deployed via its node-name
wiring instead.
The result is keyed by component rather than by node, matching
configuration.applications: several machines may run the same component
(the CI runner pool) and share one enable entry.
Inputs
components- The group's component module list (as passed to
evalModel). nodeComponents- The group's roster (
published ++ internal) as anode -> componentmap. A node that does not name a component maps to its own name.
Type
moduleEnableMap :: { components :: [Module]; nodeComponents :: { ${node} :: String; }; } -> AttrSet
fediversity.lib.evalOption
Evaluate a config value against a NixOS option declaration, applying its type checks and defaults.
Inputs
opts- An option declaration, as produced by
lib.mkOption. conf- The config value to evaluate against that declaration.
Type
evalOption :: Option -> a -> b
Example
evalOption (lib.mkOption { type = lib.types.int; }) 3
=> 3
fediversity.lib.cast
Evaluate a value against a NixOS option type, applying the type's defaults.
A thin wrapper around evalOption
that wraps the bare type in an option declaration for you.
Inputs
type- A NixOS option type (e.g.
lib.types.int, or asubmodule). a- The value to evaluate against
type.
Type
cast :: Type -> a -> b
Example
cast (lib.types.submodule { options.x = lib.mkOption { default = 1; }; }) { }
=> { x = 1; } # the submodule's default applied
fediversity.lib.registrarNsGroupName
The registrar nameserver group named after a domain: the domain with its
dots as dashes, since group names take no dots. One place, so the hosting
provider's registrar lane, which makes the group for its apex
(core/setups/hosts-common.nix), and the operator lane, which delegates
an operator's domain through that same group
(core/effects/tf/incus/operator/effect.nix), agree on the name.
Modular function type (core/lib/function.nix): the type-checked function type used throughout the data model.
Modular function type: the type-checked function type used throughout the data model.
Compared to plain nix functions, adds input type-checks at the cost of longer stack traces.
Usage:
{ lib, ... }:
{
options = {
my-function = lib.mkOption {
description = "My type-safe function invocation.";
type = lib.types.submodule PATH/TO/function.nix;
readOnly = true;
default = {
input-type = lib.types.int;
output-type = lib.types.int;
implementation = x: x + x;
};
};
};
config = {
my-function.apply "1"
};
}
A sample stack trace using this ends up like:
INVOKER.apply.<function body>function.nixINVOKER.wrapper.<function body>.outputINVOKER.implementation.<function body>
Env-adapter fold (core/lib/env-adapter.nix): the name-free driver that runs per-module envConfig readers.
Env-adapter fold: the name-free driver that runs per-module envConfig readers.
Some render inputs come from environment variables -- the structural pins
(self-pins.nix) and each effect method's env-derived arguments
(TF_HTTP_ADDRESS, BOOTSTRAP, NETBOX_*, the shared host/config vars). Each
module models that mapping as an envConfig { getEnv }: { ... } reader; this
file is the generic driver that runs those readers, plus the companions that
union the env var names they consume and the per-variable metadata a module
states about them.
Nothing here names an effect: these functions iterate whatever modules they
are handed, so a new effect shipping its own envConfig/envKeys is picked
up with no edits here.
fediversity.env.foldEnvConfig
Deep-merge the envConfig outputs of every module in a list.
Calls each present envConfig with the one impure getEnv edge and
recursiveUpdate-merges the results; modules without an envConfig
contribute nothing. getEnv is a parameter (defaulting to
builtins.getEnv) so a pure caller can drive the fold with a stub -- e.g.
getEnv = _: "" yields every module's pure defaults with no environment read
at all.
Inputs
modules- List of modules, each of which may carry an
envConfig { getEnv }: AttrSetreader. getEnv- The impure
name -> valueenv lookup (defaults tobuiltins.getEnv); pass a stub to fold purely.
Type
foldEnvConfig :: { modules :: [ Module ]; getEnv :: (String -> String) ? } -> AttrSet
Example
foldEnvConfig {
modules = [ { envConfig = { getEnv }: { address = getEnv "ADDR"; }; } ];
getEnv = name: { ADDR = "http://example"; }.${name} or "";
}
=> { address = "http://example"; }
fediversity.env.foldEnvKeys
Union every module's declared envKeys into one deduplicated list.
The key-set companion to foldEnvConfig:
where the value fold reads envConfig, this reads envKeys, the env-var
names a module consumes. core.lib.deployEnvKeys runs it over the render
fold's module set to give the front-end a single core-owned source of truth
for which env vars to forward into the render-eval subprocess.
Inputs
modules- List of modules, each of which may carry an
envKeys :: [ String ]list.
Type
foldEnvKeys :: { modules :: [ Module ]; } -> [ String ]
Example
foldEnvKeys {
modules = [ { envKeys = [ "A" "B" ]; } { envKeys = [ "B" "C" ]; } { other = true; } ];
}
=> [ "A" "B" "C" ]
fediversity.env.withEnvKeys
Derive a module's envKeys from the envVars schema it declares.
A module that states per-variable metadata -- what the variable is for, and
what an unset variable means -- has already named its keys, so this makes
the list instead of letting a hand-written second copy drift from it. The
names come out sorted, as lib.attrNames gives them: every consumer of the
published key sets compares or prints a set, none depends on the order a
module declares its variables in.
Inputs
module- A module carrying an
envVars :: { <NAME> = { description :: String; defaultText :: String ? }; }schema.
Type
withEnvKeys :: Module -> Module
Example
withEnvKeys {
envVars.ADDR = { description = "Where the state lives."; };
envConfig = { getEnv }: { address = getEnv "ADDR"; };
}
=> { envVars = { ... }; envConfig = <lambda>; envKeys = [ "ADDR" ]; }
fediversity.env.foldEnvVars
Merge every module's envVars schema into one variable-indexed dictionary.
The metadata companion to foldEnvKeys:
where the key fold yields the names alone, this yields what each name is
for, which is what the generated environment tables on the docs site are
built from. Two modules may declare the same variable -- overlapping key
groups are how a method composes its readers -- but only with identical
metadata, so no group's description can quietly win over another's.
Inputs
modules- List of modules, each of which may carry an
envVarsschema.
Type
foldEnvVars :: { modules :: [ Module ]; } -> AttrSet
Example
foldEnvVars {
modules = [ { envVars.A = { description = "First."; }; } { envVars.B = { description = "Second."; }; } ];
}
=> { A = { description = "First."; }; B = { description = "Second."; }; }
JSON schema helper (core/lib/schema-for-source.nix): frontend JSON schema generation.
JSON schema helper: frontend JSON schema generation.
Computes the frontend JSON schema from a configuration module. Exposes
schemaFromModuleType moduleType for build-time use from package.nix,
where the module type is already imported.
The converter is fediversity/module-schema (optionsToSchema), a standalone
nixpkgs-only library that emits standard JSON Schema draft 2020-12. It replaced
a Fediversity fork of clan-core's jsonschema.fromOptions (which coupled the
schema to clan's Config*Input def naming and its x-defaultText extension).
The two structural differences that shape this file:
optionsToSchemareturns an object root whosepropertiesare the option map keyed by option name, NOT wrapped under aConfigInput.properties$def. We rebuild that root here, so the downstream shape the api layer and the form renderer consume is stable no matter what the converter puts beside it.- the
deployment-methodattrTagunion emits each branch's settings INLINED atoneOf[i].properties.<tag>(not a$refto a per-tag$def). ThepinEffectcollapse andhostingLeavesprune below therefore operate on the inlined branch object directly -- no def-name resolution, no clan naming contract to track.
That post-processing is fediversity policy rather than a compliance gap in the
converter: the pinEffect collapse, the HOSTING_CONFIG leaf prune, the
applications required prune, and the alias/rename prune plus $defs GC all
encode product decisions with no upstream home.
optionsToSchema already emits fully compliant JSON Schema; this file is NOT a
second converter route and closes no compliance gap. It is a fediversity POLICY
layer stacked on ONE conversion. Of its four steps, three are product transforms
with no upstream home and one is a pure shape adapter:
pinEffectcollapse -- a deployment-form PRODUCT decision (a prod site serves a single pre-baked effect with no picker, issue #723). Fediversity policy, not a converter concern.hostingLeavesprune -- removesHOSTING_CONFIG-fixed fields so a provider-fixed value never reaches the form/wire (issue #723). Keyed on fediversity's hosting-config concept; not upstreamable.- alias/rename prune +
$defsGC -- drops the OLD paths ofmkAliasOptionModulealiases. The converter is RIGHT to emit them (an alias isvisible = true, a live option); we just keep aliases for back-compat while excluding them from the FORM. Fediversity migration policy. - wrap-into-object-root -- the only pure shape adapter. The converter emits an
object root of its own, but the three policy steps above prune properties out
of the map, so its
requiredset no longer describes what is left. The root is therefore rebuilt here, after the prunes, from a few lines that co-locate with the steps they follow.
So forking the converter would remove none of the pin/prune policy -- that is ours by nature -- and at most absorb the wrap-root adapter.
fediversity.schema.schemaFromModuleType
Compute the frontend JSON schema from a configuration module's type.
Walks the module type's sub-options with module-schema and wraps the
resulting option map into the conventional object-schema root, keeping any
$defs the converter emitted.
Renamed options are already visible = false (the converter drops hidden
options), but aliased ones are visible = true and would leak into the form
as duplicate fields, so the old (from) paths of the structured rename table
are pruned here.
Inputs
moduleType- A submodule option type (e.g. the
tf-incus-hostssetup'sconfigurationtype) whose sub-options become the schema's properties. renames- Optional structured rename table (as produced by
renames-for-source.nix); the old (from) top-level paths it lists are pruned from the schema'sproperties. Defaults to[ ]. pinEffect- Optional
deployment-methodunion tag (e.g."tf-incus-hosts") to collapse the emitted union to a single pre-baked effect: that branch's settings become top-level properties and thedeployment-methodkey is dropped altogether, so the schema says nothing about deployment methods and the form renders the effect's settings as ordinary fields. Which effect a pinned build serves is the build's own fact, recorded beside the schema (api/nix/package.nix'spinned-effect.json) rather than carried in the document. The tag names any union member -- a deployment method or a non-deploy effect -- hence "effect", not "method".null(default, and the bare-moduleTypeform) emits the whole union.
Type
schemaFromModuleType :: ModuleType -> AttrSet
schemaFromModuleType :: { moduleType :: ModuleType, pinEffect :: String?, hostingLeaves :: AttrSet? } -> AttrSet
hostingLeaves- Optional
HOSTING_CONFIG-fixed leaf paths as a{ <tag> = [ [seg...] ]; }map (as produced bycore/lib/hosting.nix'sfixedLeafPaths, already bucket-validated). Each listed path is pruned from that tag's inlined settings object (oneOf[i].properties.<tag>), so a provider-fixed field never enters the schema or the wire. Defaults to{ }(no pruning).
Example
schemaFromModuleType (lib.types.submodule { options.domain = lib.mkOption { type = lib.types.str; }; })
=> {
"$schema" = "https://json-schema.org/draft/2020-12/schema";
type = "object";
properties.domain = { type = "string"; };
# ...
}
Deployment effect helpers (core/effects/common/lib.nix): bash/env rendering used by the TF effects.
Deployment effect helpers: bash/env rendering used by the TF effects.
fediversity.effects.toBash
Render a Nix value as a string suitable for a double-quoted bash word.
Paths and null become their toString, strings pass through, and other
values are JSON-encoded. Backslashes and double-quotes are escaped so a
pre-existing \" round-trips through bash double-quotes.
Inputs
v- The value to render (a path,
null, a string, or any JSON-encodable value).
Type
toBash :: a -> String
Example
toBash "hello"
=> "hello"
toBash { a = 1; }
=> "{\\\"a\\\":1}" # JSON-encoded, with quotes escaped for bash
fediversity.effects.mapKeys
Map an attribute set's keys through a function, keeping the values.
Inputs
keyMapper- Function applied to each attribute name to produce the new name.
attrs- The attribute set whose keys to rewrite (applied as the second argument).
Type
mapKeys :: (String -> String) -> AttrSet -> AttrSet
Example
mapKeys (k: "TF_VAR_${k}") { host = "localhost"; }
=> { TF_VAR_host = "localhost"; }
fediversity.effects.filterNull
Drop attributes whose value is null.
Inputs
attrs- The attribute set to filter.
Type
filterNull :: AttrSet -> AttrSet
Example
filterNull { a = 1; b = null; }
=> { a = 1; }
fediversity.effects.withEnv
Render an attribute set as a space-separated list of key="value"
assignments, with each value passed through toBash.
Inputs
environment- Attribute set of environment-variable names to values.
Type
withEnv :: AttrSet -> String
Example
withEnv { GREETING = "hi there"; COUNT = 2; }
=> "GREETING=\"hi there\" COUNT=\"2\""