The previous setup caused all renewal units to be triggered upon
ever so slight changes in config. In larger setups (100+ certificates)
adding a new certificate caused high system load and/or large memory
consumption issues. The memory issues are already a alleviated with
the locking mechanism. However, this then causes long delays upwards
of multiple minutes depending on individual runs and also caused
superfluous activations.
In this change we streamline the overall setup of units:
1. The unit that other services can depend upon is 'acme-{cert}.service'.
We call this the 'base unit'. As this one as `RemainAfterExit` set
the `acme-finished-{cert}` targets are not required any longer.
2. We now always generate initial self-signed certificates to simplify
the dependency structure. This deprecates the `preliminarySelfsigned`
option.
3. The `acme-order-renew-{cert}` service gets activated after the base
unit and services using certificates have started and performs all acme
interactions. When it finishes others services (like web servers) will
be notified through the `reloadServices` option or they can use
`wantedBy` and `after` dependencies if they implement their own reload
units.
The renewal timer also triggers this unit.
4. The timer unit is explicitly blocked from being started by s-t-c.
5. Permission management has been cleaned up a bit: there was an
inconsistency between having the .lego files set to 600 vs 640
on the exposed side. This is unified to 640 now.
6. Exempt the account target from being restarted by s-t-c. This will
happen automatically if something relevant to the account changes.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
Split tests up based on certain use cases:
- http01-builtin: Tests most functionality of the core module, such
as the systemd and hashing components, whilst utilising lego's built
in http01 resolution mechanis.
- dns01: Tests only that DNS01 renewal works as expected.
- nginx: Tests nginx compatability
- httpd: Tests httpd compatability
- caddy: Tests caddy compatability
Over time, we added a lot of setup services to the ACME module, namely:
- acme-selfsigned-ca.service: Creates the selfsigned CA certificates
used to generate selfsigned certs for each configured cert.
- acme-fixperms.service: Ensures permissions correctness on certs after
system configuration changes.
- acme-lockfiles.service: Create lockfiles used to implement
maxConcurrentRenewals.
These numerous setup services complicated the dependency chain for any
cert renewal, and also made it difficult to track responsibility for
specific setup steps, for example, creating /var/lib/acme or setting
permissions of shared folders.
This change proposes a new acme-setup.service which encapsulates the
functionality of the previous 3 services into one. The service is still
defined in 3 separate chunks (using lib.mkMerge) which allows us to
keep a logical separation between each step and preserve some
optionality in the features.
The result is a generally simplified definition of systemd unit
dependencies and an obvious entrypoint for future setup extensions.
This change is to support LEGO's capability to spawn an external process that
solves the DNS-01 challenge. In particular, this enables a setup where LEGO
runs a shell script that uses nsd-control to add an appropriate zone to a
local NSD instance.
It was being created with the default home permissions of 700, and then
set to 755 at runtime by something either some script or systemd as
part of service startup.
It worked fine without sysusers, but when it's enabed with:
systemd.sysusers.enable = true;
systemd-tmpfiles is resetting permissions on each activation, which
breaks, for example, nginx reload, because it cannot load certificates
anymore, because it doesn't have any access to `/var/lib/acme`.
Fix this by setting `homeMode = "755";` explicitely so that it's set to
the final value from the beginning.
I have no idea what this escape sequence even is, but it breaks the nix parser with cryptic errors if not used in a comment.
A friend let me know MacOS is prone to input weird spaces, not sure if that is the source.
Candidates were located and created with:
chr="$(echo -e '\xc2\xa0')"; rg -F "$chr" -l | xe sd -F "$chr" " "
There are some examples left, most being example output from `tree` in various markdown documents, some patches which we can't really touch, and `pkgs/tools/nix/nixos-render-docs/src/tests/test_commonmark.py` which I'm not sure if should be addressed
The accounts directory is based on the hash of the settings.
https://github.com/NixOS/nixpkgs/pull/270221 changed the default of
security.acme.defaults.server from null to the default letsencrypt URL
however as an unwanted side effect this means the accounts directory
changes and the ACME module will create a new a new account.
This can cause issues with people using CAA records that pin the
account ID or people who have datacenter-scale NixOS deployments
We allow setting this option to null again for people who want
to keep the old account and migrate at their own leisure.
Fixes https://github.com/NixOS/nixpkgs/issues/316608
Co-authored-by: Arian van Putten <arian.vanputten@gmail.com>
these changes were generated with nixq 0.0.2, by running
nixq ">> lib.mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> Inherit >> mdDoc[remove]" --batchmode nixos/**.nix
two mentions of the mdDoc function remain in nixos/, both of which
are inside of comments.
Since lib.mdDoc is already defined as just id, this commit is a no-op as
far as Nix (and the built manual) is concerned.
fixes#232505
Implements the new option `security.acme.maxConcurrentRenewals` to limit
the number of certificate generation (or renewal) jobs that can run in
parallel. This avoids overloading the system resources with many
certificates or running into acme registry rate limits and network
timeouts.
Architecture considerations:
- simplicity, lightweight: Concerns have been voiced about making this
already rather complex module even more convoluted. Additionally,
locking solutions shall not significantly increase performance and
footprint of individual job runs.
To accomodate these concerns, this solution is implemented purely in
Nix, bash, and using the light-weight `flock` util. To reduce
complexity, jobs are already assigned their lockfile slot at system
build time instead of dynamic locking and retrying. This comes at the
cost of not always maxing out the permitted concurrency at runtime.
- no stale locks: Limiting concurrency via locking mechanism is usually
approached with semaphores. Unfortunately, both SysV as well as
POSIX-Semaphores are *not* released when the process currently locking
them is SIGKILLed. This poses the danger of stale locks staying around
and certificate renewal being blocked from running altogether.
`flock` locks though are released when the process holding the file
descriptor of the lock file is KILLed or terminated.
- lockfile generation: Lock files could either be created at build time
in the Nix store or at script runtime in a idempotent manner.
While the latter would be simpler to achieve, we might exceed the number
of permitted concurrent runs during a system switch: Already running
jobs are still locked on the existing lock files, while jobs started
after the system switch will acquire locks on freshly created files,
not being blocked by the still running services.
For this reason, locks are generated and managed at runtime in the
shared state directory `/var/lib/locks/`.
nixos/security/acme: move locks to /run
also, move over permission and directory management to systemd-tmpfiles
nixos/security/acme: fix some linter remarks in my code
there are some remarks left for existing code, not touching that
nixos/security/acme: redesign script locking flow
- get rid of subshell
- provide function for wrapping scripts in a locked environment
nixos/acme: improve visibility of blocking on locks
nixos/acme: add smoke test for concurrency limitation
heavily inspired by m1cr0man
nixos/acme: release notes entry on new concurrency limits
nixos/acme: cleanup, clarifications
this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh
since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).
also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
With Go 1.19 calls to setrlimit are required for lego to run.
While we could allow setrlimit alone, I think it is not unreasonable to
allow @resources in general.
Closes: #197513
Lego has a built-in mechanism for sleeping for a random amount
of time before renewing a certificate. In our environment this
is not only unnecessary (as our systemd timer takes care of it)
but also unwanted since it slows down the execution of the
systemd service encompassing it, thus also slowing down the
start up of any services its depending on.
Also added FixedRandomDelay to the timer for more predictability.
Fixes#190493
Check if an actual key file exists. This does not
completely cover the work accountHash does to ensure
that a new account is registered when account
related options are changed.
Fixes#191794
Lego threw a permission denied error binding to port 80.
AmbientCapabilities with CAP_NET_BIND_SERVICE was required.
Also added a test for this.
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running
nix-doc-munge nixos/**/*.nix
nix-doc-munge --import nixos/**/*.nix
the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
mostly no rendering changes. some lists (like simplelist) don't have an
exact translation to markdown, so we use a comma-separated list of
literals instead.
