It was brought up that the restricted file-system access breaks
tablespaces[1]. I'd argue that this is the desired behavior, the whole
point of the hardening is the lock the service down and I don't consider
tablespaces common enough to elevate privileges again. Especially since
the workaround is trivial as shown in the diff.
For completeness sake, this adds the necessary `ReadWritePaths` change
to the postgresql section of the manual.
This also adds a small correction about the state of
`ensurePermissions`.
[1] https://github.com/NixOS/nixpkgs/pull/344925#issuecomment-2521188907
This allows users to set the Admin Password via secrets. `configFile` must be writable which is why it isn't sufficient.
nixfmt nixos/modules/services/databases/couchdb.nix
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build a08b3a4d19.tar.gz \
--argstr baseRev b32a094368
result/bin/apply-formatting $NIXPKGS_PATH
Commit e14483d6a6 fixed a bug in the `ini`
type with `listsAsDuplicatedKeys = true;`: multiple list declarations
weren't merged, but latter declarations shadowed the former without any
error.
The fix brought another issue to surface however: before, the
`plugin-load-add` declaration in the MySQL test shadowed the
`auth_socket.so` setting in the module. But now the attempt
to merge a list and a single declaration breaks because of
`types.either` seeing a mix of declarations from the left
AND right type.
Turning the `plugin-load-add` in the module into a list triggers the
correct merging behavior and thus fixes the evaluation error (and
merging behavior of `plugin-load-add`)!
This wasn't an issue for mysql itself (empty `plugin-load-add` in the
test) and neither for mariadb (the `auth_socket.so` isn't added for
this).
Commit bfb9d1825d added RFC42 support
which is a good thing in general, but this implementation has the
following flaws:
* `services.pgbouncer.logFile` was not renamed to `[...].log_file`, but
to `[...].logfile`. Also the use of `mkRenamedOptionModule` is
inappropriate here because the two options are not equivalent: the old
option took a path relative to the home directory, the new an absolute
path.
* Using `mkRenamedOptionModule` with options that don't exist (but are
keys in a freeform attr-set or an `attrsOf X`), you get the following
error when referencing an option you didn't declare:
error: evaluation aborted with the following error message: 'Renaming error: option `services.pgbouncer.settings.pgbouncer.listen_port' does not exist.'
This error is pretty bad because it's not actionable for an end-user of
the module. A possible use-case is doing
networking.firewall.allowedTCPPorts = [ config.services.pgbouncer.listenPort ];
without specifying a custom listen port. This is an example of why you
want to keep options, they already contain defaults and you can re-use
those defaults in other parts of your system configuration.
I decided to re-add a bunch of options where I figured that it's
either useful to be able to address those in the NixOS configuration
or having documentation directly in the options' reference in the
NixOS manual.
I didn't add all options, I'll leave that to the maintainers of
pgbouncer.
This will be EOL at the end of November, so there's little reason to
keep it in 24.11[1]. As discussed, we'd like to keep it for as long as
possible to make sure there's a state in nixpkgs that has the latest
minor of postgresql_12 available with the most recent CVEs fixed for
people who cannot upgrade[2].
This aspect has been made explicit in the manual now for the next .11
release.
During the discussions it has been brought up that if people just do
`services.postgresql.enable = true;` and let the code decide the
postgresql version based on `system.stateVersion`, there's a chance that
such EOL dates will be missed. To make this harder, a warning will now
be raised when using the stateVersion-condition and the oldest still
available major is selected.
Additionally regrouped the postgresql things in the release notes to
make sure these are all shown consecutively. Otherwise it's a little
hard to keep track of all the changes made to postgresql in 24.11.
[1] https://endoflife.date/postgresql
[2] https://github.com/NixOS/nixpkgs/pull/353158#issuecomment-2453056692
The test breaks like this otherwise:
machine # WARNING: error during JITing: Permission denied
machine # [ 14.012280] postgres[913]: [913] WARNING: error during JITing: Permission denied
machine # ERROR: failed to look up symbol "evalexpr_0_1": Failed to materialize symbols: { (main, { evalexpr_0_1, evalexpr_0_0 }) }
previously if you set the "user" option and did not create a group
account with the same name the module would create a service that would
fail to start.
with this change:
- the module is more explicit about this behaviour
- you can configure the group directly, so that you're not forced to a
particular user/group structure
- you can read the group name used by the redis service. this is useful
for giving other services permission to use the redis socket.
The issue was that the old test-case used `/tmp` to share data. Using
`JoinsNamespaceOf=` wasn't a real workaround since the private `/tmp` is
recreated when a service gets stopped/started which is the case here, so
the wals were still lost.
To keep the test building with `PrivateTmp=yes`, create a dedicated
directory in `/var/cache` with tmpfiles and allow the hardened
`postgresql.service` to access it via `ReadWritePaths`.
Factor out part of the provisioning script into a
wait-until-service-is-ready script, and put it unconditionally in
front of ExecStartPost=, so that services that depend on influxdb2 are
not started until influxdb2 responds to requests.
Fixes https://github.com/NixOS/nixpkgs/issues/317017 ("Scrutiny tries to start before influxdb has started")
Without the change the doc build fails as:
$ nix build -f nixos config.system.build.manual.manualHTML -L
...
nixos-manual-html> RuntimeError: heading in line 366 does not have an id
This splits a dev output to make the default output not depend on any
build dependencies anymore. This also avoids removing references from
pgxs' Makefile this way, which should, at least theoretically, be good
to build extensions via pgxs, making sure they use the same tooling.
ecpg is the "embedded SQL C preprocessor", which is certainly a dev
tool.
Most important, for closure size anyway, is to move pg_config to the dev
output, since it retains paths to all the other outputs.
The only thing with references to the dev output remaining is then the
postgres binary itself. It contains all the output paths, because it
shows those in the pg_config system view. There is no other way than
to nuke those references to avoid circular dependencies between outputs
- and blowing up closure size again.
This change adds services.pgbouncer.settings option as per [RFC 0042]
and deprecates other options that were previously used to generate
configuration file.
In addition to that, we also place the configuration file under
environment.etc to allow reloading configuration without service
restart.
[RFC 0042]: https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
When mysql starts up for the first time, the binary `mysql_install_db`
will run and try to set up the correct folder structure and permissions.
The very first step is to change the owner and group of the data
directory. This can fail in some cases, for example if `cfg.dataDir` is
something like `/mnt/mysql`:
```
Jul 31 15:24:35 junction systemd[1]: Starting MySQL Server...
Jul 31 15:24:36 junction mysql-pre-start[1346]: chown: changing ownership of '/mnt/mysql': Operation not permitted
Jul 31 15:24:36 junction mysql-pre-start[1309]: Cannot change ownership of the database directories to the 'mysql'
Jul 31 15:24:36 junction mysql-pre-start[1309]: user. Check that you have the necessary permissions and try again.
Jul 31 15:24:36 junction systemd[1]: mysql.service: Control process exited, code=exited, status=1/FAILURE
Jul 31 15:24:36 junction systemd[1]: mysql.service: Failed with result 'exit-code'.
Jul 31 15:24:36 junction systemd[1]: Failed to start MySQL Server.
```
This is because `/mnt` usually is owned by root.
To work around this issue, use `systemd.tmpfiles` to set the owner,
group and permissions. This will always work, and if the permissions are
already set as expected, mysql will not try to alter them again.
Done using `grep -rP 'description\s+=\s+"[^"]+[ ]";' | cut -d ':' -f 1 |
xargs -i nvim {}` and sorting the opened files by hand, avoiding
generated packages list
Upgrade default postgresql for stateVersion >=24.11.
This also rebuilds all packages linking against `libpq.so` to use
postgresql 16.
After re-reading https://www.postgresql.org/docs/16/release-16.html
I don't see any major risks about doing that.
Since https://github.com/redis/redis/pull/4001 included in 6.2.0
transparent hugepages works when being set to madvise which is the NixOS
and upstream recommended default.
> WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition.
> Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328.
> To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
these changes were generated with nixq 0.0.2, by running
nixq ">> lib.mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> Inherit >> mdDoc[remove]" --batchmode nixos/**.nix
two mentions of the mdDoc function remain in nixos/, both of which
are inside of comments.
Since lib.mdDoc is already defined as just id, this commit is a no-op as
far as Nix (and the built manual) is concerned.
The enableJIT = true case was fixed in #221851 or
e2fb651752 respectively.
However this did not take the case into consideration, when doing this:
services.postgresql = {
enable = true;
enableJIT = false;
package = pkgs.postgresql_15_jit;
};
If enableJIT is treated as the source of truth, then this should indeed
cause JIT to be disabled, which this commit does.
The main idea behind that was to be able to do more sophisticated
merging for stuff that goes into `postgresql.conf`:
`shared_preload_libraries` is a comma-separated list in a `types.str`
and thus not mergeable. With this change, the option accepts both a
comma-separated string xor a list of strings.
This can be implemented rather quick using `coercedTo` +
freeform modules. The interface still behaves equally, but it allows to
merge declarations for this option together.
One side-effect was that I had to change the `attrsOf (oneOf ...)` part into
a submodule to allow declaring options for certain things. While at it,
I decided to move `log_line_prefix` and `port` into this structure as
well.
This was proposed by abbradar in #150801, but left out of the follow up PR
#221851 by Ma27 to reduce the size of the diff. Compared to the initial
proposal this includes the callPackage call in the recursion, which avoids
breaking the withJIT/withoutJIT helpers.
In terms of nixpkgs, this is a pure refactor, no derivations change. However,
this makes downstream expressions like the following possible:
(postgresql.override { jitSupport = true; }).pkgs.postgis
This would have not worked before without passing another "this" argument,
which is error prone as can be seen in this example:
https://github.com/PostgREST/postgrest/pull/3222/files
This makes it less error-prone to use the llvm package in extensions, because
it will always match the package used by the postgresql derivation itself.
Previously, you could've accidentally used llvm instead of postgresql.llvm
with a different result.
While `/var/lib/lldap` isn't technically accessible by unprivileged
users thanks to `DynamicUser=true`, a user might prefer and change it to
`DynamicUser=false`.
There is currently also a PR open that intends to make `DynamicUser`
configurable via module option.
As such, `jwt_secret_file`, if bootstrapped by the service start
procedure, might be rendered world-readable due to its permissions
(`0644/-rw-r--r--`) defaulting to the service's umask (`022`) and
`/var/lib/lldap` to `0755/drwxr-xr-x` due to `StateDirectoryMode=0755`.
This would usually be fixed by using `(umask 027; openssl ...)` instead
of just `openssl ...`.
However, it was found that another file (`users.db`), this time
bootstrapped by `lldap` itself, also had insufficient permissions
(`0644/-rw-r--r--`) inherited by the global umask and would be left
world-readable as well.
Due to this, we instead change the service's to `027`.
And to lower the impact for already bootstrapped files on existing
instances like `users.db`, set `StateDirectoryMode=0750`.
...effectively what was planned already in #266270, but it was too late
because the branches were restricted and didn't allow any breaking
changes anymore.
It also suffers from the same issue that we already had when discussing
this the last time[1] when `ensureDBOwnership` was ultimately introduced
as band-aid fix: newly created users don't get CREATE permission on
the `public` schema anymore (since psql 15), even with `ALL PRIVILEGES`.
If one's use-case is more sophisticated than having a single owner, it's
questionable anyways if this module is the correct tool since
permissions aren't dropped on a change to this option or a removal which
is pretty surprising in the context of NixOS.
[1] https://github.com/NixOS/nixpkgs/pull/266270
Since 2c5abd89c7 setting the option `boot.kernel.sysctl."net.core.rmem_max"` no longer has any downsides compared to what was previously used. Since 439350753e the same is also true for `boot.kernel.sysctl."net.core.wmem_max"`.
Do not assume that port and unixSocket are the only options that affect
address families used by Redis. There are other options, e.g. tls-port,
and also clustered setup that are not covered by the declarative
configuration. Instead of trying to selectively restrict unused address
families based on the configuration, limit address families to IP and
Unix sockets and let users lib.mkForce a stricter sandboxing is needed.
See also
https://docs.redis.com/latest/rs/networking/port-configurations/
When `services.firebird.enable` following error is thrown:
error: cannot coerce an integer to a string
After explicitly cast the port (integer) to string the error disappears.
As it is technically a breaking change, we should at least make a strong deprecation
of `ensurePermissions` and leave it in the broken state it is, for out of tree users.
We give them a 6 months notice to migrate away by doing so, which is honest.
In the meantime, we forbid usage of `ensurePermissions` inside of nixpkgs.
Closes#216989
First of all, a bit of context: in PostgreSQL, newly created users don't
have the CREATE privilege on the public schema of a database even with
`ALL PRIVILEGES` granted via `ensurePermissions` which is how most of
the DB users are currently set up "declaratively"[1]. This means e.g. a
freshly deployed Nextcloud service will break early because Nextcloud
itself cannot CREATE any tables in the public schema anymore.
The other issue here is that `ensurePermissions` is a mere hack. It's
effectively a mixture of SQL code (e.g. `DATABASE foo` is relying on how
a value is substituted in a query. You'd have to parse a subset of SQL
to actually know which object are permissions granted to for a user).
After analyzing the existing modules I realized that in every case with
a single exception[2] the UNIX system user is equal to the db user is
equal to the db name and I don't see a compelling reason why people
would change that in 99% of the cases. In fact, some modules would even
break if you'd change that because the declarations of the system user &
the db user are mixed up[3].
So I decided to go with something new which restricts the ways to use
`ensure*` options rather than expanding those[4]. Effectively this means
that
* The DB user _must_ be equal to the DB name.
* Permissions are granted via `ensureDBOwnerhip` for an attribute-set in
`ensureUsers`. That way, the user is actually the owner and can
perform `CREATE`.
* For such a postgres user, a database must be declared in
`ensureDatabases`.
For anything else, a custom state management should be implemented. This
can either be `initialScript`, doing it manual, outside of the module or
by implementing proper state management for postgresql[5], but the
current state of `ensure*` isn't even declarative, but a convergent tool
which is what Nix actually claims to _not_ do.
Regarding existing setups: there are effectively two options:
* Leave everything as-is (assuming that system user == db user == db
name): then the DB user will automatically become the DB owner and
everything else stays the same.
* Drop the `createDatabase = true;` declarations: nothing will change
because a removal of `ensure*` statements is ignored, so it doesn't
matter at all whether this option is kept after the first deploy (and
later on you'd usually restore from backups anyways).
The DB user isn't the owner of the DB then, but for an existing setup
this is irrelevant because CREATE on the public schema isn't revoked
from existing users (only not granted for new users).
[1] not really declarative though because removals of these statements
are simply ignored for instance: https://github.com/NixOS/nixpkgs/issues/206467
[2] `services.invidious`: I removed the `ensure*` part temporarily
because it IMHO falls into the category "manage the state on your
own" (see the commit message). See also
https://github.com/NixOS/nixpkgs/pull/265857
[3] e.g. roundcube had `"DATABASE ${cfg.database.username}" = "ALL PRIVILEGES";`
[4] As opposed to other changes that are considered a potential fix, but
also add more things like collation for DBs or passwords that are
_never_ touched again when changing those.
[5] As suggested in e.g. https://github.com/NixOS/nixpkgs/issues/206467
As described in the release lifecycle docs from postgresql[1], v11 will
stop receiving fixes as of Nov 9 2023. This means it's EOL throughout
the entire lifetime of 23.11, so let's drop it now.
A lot of examples are also referencing postgresql_11. Where it's
sensible, use postgresql_15 as example now to avoid confusion.
This is also handy because the LLVM 16 fix for postgresql is not
available for postgresql 11 ;-)
[1] https://www.postgresql.org/support/versioning/
This was causing the following warning before when building the manual:
warning: literalExample is deprecated, use literalExpression instead, or use literalMD for a non-Nix description.
Rather than using `literalExpression`, nothing is used. This option
expects a string and the example is a string, no special handling
required. Both `literalExample` from the docbook ages and
`literalExpression` now are only required if the example is
a Nix expression rather than a value of the option's type.
