mirror of https://github.com/NixOS/nixpkgs.git synced 2025-11-10 01:33:11 +01:00

nixos/doc: add documentation for formats.hocon

2024-06-16 20:58:47 +02:00

14 KiB

Raw Blame History

Options for Program Settings

Many programs have configuration files where program-specific settings can be declared. File formats can be separated into two categories:

Nix-representable ones: These can trivially be mapped to a subset of Nix syntax. E.g. JSON is an example, since its values like {"foo":{"bar":10}} can be mapped directly to Nix: { foo = { bar = 10; }; }. Other examples are INI, YAML and TOML. The following section explains the convention for these settings.
Non-nix-representable ones: These can't be trivially mapped to a subset of Nix syntax. Most generic programming languages are in this group, e.g. bash, since the statement if true; then echo hi; fi doesn't have a trivial representation in Nix.

Currently there are no fixed conventions for these, but it is common to have a configFile option for setting the configuration file path directly. The default value of configFile can be an auto-generated file, with convenient options for controlling the contents. For example an option of type attrsOf str can be used for representing environment variables which generates a section like export FOO="foo". Often it can also be useful to also include an extraConfig option of type lines to allow arbitrary text after the autogenerated part of the file.

Nix-representable Formats (JSON, YAML, TOML, INI, ...)

By convention, formats like this are handled with a generic settings option, representing the full program configuration as a Nix value. The type of this option should represent the format. The most common formats have a predefined type and string generator already declared under pkgs.formats:

pkgs.formats.javaProperties { comment ? "Generated with Nix" }

A function taking an attribute set with values

comment: A string to put at the start of the file in a comment. It can have multiple lines.

It returns the type: attrsOf str and a function generate to build a Java .properties file, taking care of the correct escaping, etc.

pkgs.formats.hocon { generator ? <derivation>, validator ? <derivation>, doCheck ? true }

A function taking an attribute set with values

generator

: A derivation used for converting the JSON output from the nix settings into HOCON. This might be useful if your HOCON variant is slightly different from the java-based one, or for testing purposes.

validator

: A derivation used for verifying that the HOCON output is correct and parsable. This might be useful if your HOCON variant is slightly different from the java-based one, or for testing purposes.

doCheck

: Whether to enable/disable the validator check.

It returns an attrset with a type, generate function, and a lib attset, as specified below. Some of the lib functions will be best understood if you have read the reference specification. You can find this specification here:

https://github.com/lightbend/config/blob/main/HOCON.md

Inside of lib, you will find these functions

mkInclude

: This is used together with a specially named attribute includes, to include other HOCON sources into the document.

 The function has a shorthand variant where it
 is up to the HOCON parser to figure out what type
 of include is being used. The include will default
 to being non-required. If you want to be more
 explicit about the details of the include, you can
 provide an attrset with following arguments

 `required`

 :   Whether the parser should fail upon failure
     to include the document

 `type`

 :   Type of the source of the included document.
     Valid values are `file`, `url` and `classpath`.
     See upstream documentation for the semantics
     behind each value

 `value`

 :   The URI/path/classpath pointing to the source of
     the document to be included.

 `Example usage:`

 ```nix
   let
     format = pkgs.formats.hocon { };
     hocon_file = pkgs.writeText "to_include.hocon" ''
       a = 1;
     '';
   in {
     some.nested.hocon.attrset = {
       _includes = [
         (format.lib.mkInclude hocon_file)
         (format.lib.mkInclude "https://example.com/to_include.hocon")
         (format.lib.mkInclude {
           required = true;
           type = "file";
           value = include_file;
         })
       ];
       ...
     };
   }
 ```

mkAppend

: This is used to invoke the += operator. This can be useful if you need to add something to a list that is included from outside of nix. See upstream documentation for the semantics behind the += operation.

 `Example usage:`

 ```nix
   let
     format = pkgs.formats.hocon { };
     hocon_file = pkgs.writeText "to_include.hocon" ''
       a = [ 1 ];
       b = [ 2 ];
     '';
   in {
     _includes = [
       (format.lib.mkInclude hocon_file)
     ];

     c = 3;
     a = format.lib.mkAppend 3;
     b = format.lib.mkAppend (format.lib.mkSubstitution "c");
   }
 ```

mkSubstitution

: This is used to make HOCON substitutions. Similarly to mkInclude, this function has a shorthand variant where you just give it the string with the substitution value. The substitution is not optional by default. Alternatively, you can provide an attrset with more options

 `optional`

 :   Whether the parser should fail upon
     failure to fetch the substitution value.

 `value`

 :   The name of the variable to use for
     substitution.

 See upstream documentation for semantics
 behind the substitution functionality.

 `Example usage:`

 ```nix
   let
     format = pkgs.formats.hocon { };
   in {
     a = 1;
     b = format.lib.mkSubstitution "a";
     c = format.lib.mkSubstition "SOME_ENVVAR";
     d = format.lib.mkSubstition {
       value = "SOME_OPTIONAL_ENVVAR";
       optional = true;
     };
   }
 ```

pkgs.formats.json { }

A function taking an empty attribute set (for future extensibility) and returning a set with JSON-specific attributes type and generate as specified below.

pkgs.formats.yaml { }

A function taking an empty attribute set (for future extensibility) and returning a set with YAML-specific attributes type and generate as specified below.

pkgs.formats.ini { listsAsDuplicateKeys ? false, listToValue ? null, ... }

A function taking an attribute set with values

listsAsDuplicateKeys: A boolean for controlling whether list values can be used to represent duplicate INI keys
listToValue: A function for turning a list of values into a single value.

It returns a set with INI-specific attributes type and generate as specified below. The type of the input is an attrset of sections; key-value pairs where the key is the section name and the value is the corresponding content which is also an attrset of key-value pairs for the actual key-value mappings of the INI format. The values of the INI atoms are subject to the above parameters (e.g. lists may be transformed into multiple key-value pairs depending on listToValue).

pkgs.formats.iniWithGlobalSection { listsAsDuplicateKeys ? false, listToValue ? null, ... }

A function taking an attribute set with values

listsAsDuplicateKeys: A boolean for controlling whether list values can be used to represent duplicate INI keys
listToValue: A function for turning a list of values into a single value.

It returns a set with INI-specific attributes type and generate as specified below. The type of the input is an attrset of the structure { sections = {}; globalSection = {}; } where sections are several sections as with pkgs.formats.ini and globalSection being just a single attrset of key-value pairs for a single section, the global section which preceedes the section definitions.

pkgs.formats.toml { }

A function taking an empty attribute set (for future extensibility) and returning a set with TOML-specific attributes type and generate as specified below.

pkgs.formats.elixirConf { elixir ? pkgs.elixir }

A function taking an attribute set with values

elixir: The Elixir package which will be used to format the generated output

It returns a set with Elixir-Config-specific attributes type, lib, and generate as specified below.

The lib attribute contains functions to be used in settings, for generating special Elixir values:

mkRaw elixirCode: Outputs the given string as raw Elixir code
mkGetEnv { envVariable, fallback ? null }: Makes the configuration fetch an environment variable at runtime
mkAtom atom: Outputs the given string as an Elixir atom, instead of the default Elixir binary string. Note: lowercase atoms still needs to be prefixed with :
mkTuple array: Outputs the given array as an Elixir tuple, instead of the default Elixir list
mkMap attrset: Outputs the given attribute set as an Elixir map, instead of the default Elixir keyword list

pkgs.formats.php { finalVariable } []{#pkgs-formats-php}

A function taking an attribute set with values

finalVariable: The variable that will store generated expression (usually config). If set to null, generated expression will contain return.

It returns a set with PHP-Config-specific attributes type, lib, and generate as specified below.

The lib attribute contains functions to be used in settings, for generating special PHP values:

mkRaw phpCode: Outputs the given string as raw PHP code
mkMixedArray list set: Creates PHP array that contains both indexed and associative values. For example, lib.mkMixedArray [ "hello" "world" ] { "nix" = "is-great"; } returns ['hello', 'world', 'nix' => 'is-great']

[]{#pkgs-formats-result} These functions all return an attribute set with these values:

type

A module system type representing a value of the format

lib

Utility functions for convenience, or special interactions with the format. This attribute is optional. It may contain inside a types attribute containing types specific to this format.

generate filename jsonValue

A function that can render a value of the format to a file. Returns a file path.

::: {.note} This function puts the value contents in the Nix store. So this should be avoided for secrets. :::

::: {#ex-settings-nix-representable .example}

Module with conventional `settings` option

The following shows a module for an example program that uses a JSON configuration file. It demonstrates how above values can be used, along with some other related best practices. See the comments for explanations.

{ options, config, lib, pkgs, ... }:
let
  cfg = config.services.foo;
  # Define the settings format used for this program
  settingsFormat = pkgs.formats.json {};
in {

  options.services.foo = {
    enable = lib.mkEnableOption "foo service";

    settings = lib.mkOption {
      # Setting this type allows for correct merging behavior
      type = settingsFormat.type;
      default = {};
      description = ''
        Configuration for foo, see
        <link xlink:href="https://example.com/docs/foo"/>
        for supported settings.
      '';
    };
  };

  config = lib.mkIf cfg.enable {
    # We can assign some default settings here to make the service work by just
    # enabling it. We use `mkDefault` for values that can be changed without
    # problems
    services.foo.settings = {
      # Fails at runtime without any value set
      log_level = lib.mkDefault "WARN";

      # We assume systemd's `StateDirectory` is used, so we require this value,
      # therefore no mkDefault
      data_path = "/var/lib/foo";

      # Since we use this to create a user we need to know the default value at
      # eval time
      user = lib.mkDefault "foo";
    };

    environment.etc."foo.json".source =
      # The formats generator function takes a filename and the Nix value
      # representing the format value and produces a filepath with that value
      # rendered in the format
      settingsFormat.generate "foo-config.json" cfg.settings;

    # We know that the `user` attribute exists because we set a default value
    # for it above, allowing us to use it without worries here
    users.users.${cfg.settings.user} = { isSystemUser = true; };

    # ...
  };
}

:::

Option declarations for attributes

Some settings attributes may deserve some extra care. They may need a different type, default or merging behavior, or they are essential options that should show their documentation in the manual. This can be done using .

We extend above example using freeform modules to declare an option for the port, which will enforce it to be a valid integer and make it show up in the manual.

::: {#ex-settings-typed-attrs .example}

Declaring a type-checked `settings` attribute

{
  settings = lib.mkOption {
    type = lib.types.submodule {

      freeformType = settingsFormat.type;

      # Declare an option for the port such that the type is checked and this option
      # is shown in the manual.
      options.port = lib.mkOption {
        type = lib.types.port;
        default = 8080;
        description = ''
          Which port this service should listen on.
        '';
      };

    };
    default = {};
    description = ''
      Configuration for Foo, see
      <link xlink:href="https://example.com/docs/foo"/>
      for supported values.
    '';
  };
}

:::

14 KiB Raw Blame History

Options for Program Settings

Nix-representable Formats (JSON, YAML, TOML, INI, ...)

Module with conventional settings option

Option declarations for attributes

Declaring a type-checked settings attribute

14 KiB

Raw Blame History

Module with conventional `settings` option

Declaring a type-checked `settings` attribute