most of these are hidden because they're either part of a submodule that
doesn't have its type rendered (eg because the submodule type is used in
an either type) or because they are explicitly hidden. some of them are
merely hidden from nix-doc-munge by how their option is put together.
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running
nix-doc-munge nixos/**/*.nix
nix-doc-munge --import nixos/**/*.nix
the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
there are sufficiently few variable list around, and they are
sufficiently simple, that it doesn't seem helpful to add another
markdown extension for them. rendering differences are small, except in
the tor module: admonitions inside other blocks cannot be made to work
well with mistune (and likely most other markdown processors), so those
had to be shuffled a bit. we also lose paragraph breaks in the list
items due to how we have to render from markdown to docbook, but once we
remove docbook from the pipeline those paragraph breaks will be restored.
mostly no rendering changes. some lists (like simplelist) don't have an
exact translation to markdown, so we use a comma-separated list of
literals instead.
most of the screen tags used in option docs are actually listings of
some sort. nsd had a notable exception where its screen usage was pretty
much a raw markdown block that made most sense to convert into docbook lists.
#167013 introduced a property conflict with the concurrently-written commit
aea940da63, over property
systemd.services.prosody. Fix this by moving the reload option into the block.
using regular strings works well for docbook because docbook is not as
whitespace-sensitive as markdown. markdown would render all of these as
code blocks when given the chance.
a lot of markdown syntax has already snuck into option docs, many of it
predating the intent to migrate to markdown. we don't convert all of it
here, just that which is accompanied by docbook tags as well. the rest
can be converted by simply adding the mdDoc marker.
this renders the same in the manpage and a little more clearly in the
html manual. in the manpage there continues to be no distinction from
regular text, the html manual gets code-type markup (which was probably
the intention for most of these uses anyway).
Plausible fails on start because clickhouse is not ready,
when clickhouse has low CPU available, eg.
```nix
{systemd.services.clickhouse.serviceConfig.CPUWeight = 20;}
```
Fixed with
```nix
{systemd.services.plausible.after = [ "clickhouse.service" ];}
```
now nix-doc-munge will not introduce whitespace changes when it replaces
manpage references with the MD equivalent.
no change to the manpage, changes to the HTML manual are whitespace only.
make (almost) all links appear on only a single line, with no
unnecessary whitespace, using double quotes for attributes. this lets us
automatically convert them to markdown easily.
the few remaining links are extremely long link in a gnome module, we'll
come back to those at a later date.
markdown can't represent the difference without another extension and
both the html manual and the manpage render them the same, so keeping the
distinction is not very useful on its own. with the distinction removed
we can automatically convert many options that use <code> tags to markdown.
the manpage remains unchanged, html manual does not render
differently (but class names on code tags do change from "code" to "literal").
the conversion procedure is simple:
- find all things that look like options, ie calls to either `mkOption`
or `lib.mkOption` that take an attrset. remember the attrset as the
option
- for all options, find a `description` attribute who's value is not a
call to `mdDoc` or `lib.mdDoc`
- textually convert the entire value of the attribute to MD with a few
simple regexes (the set from mdize-module.sh)
- if the change produced a change in the manual output, discard
- if the change kept the manual unchanged, add some text to the
description to make sure we've actually found an option. if the
manual changes this time, keep the converted description
this procedure converts 80% of nixos options to markdown. around 2000
options remain to be inspected, but most of those fail the "does not
change the manual output check": currently the MD conversion process
does not faithfully convert docbook tags like <code> and <package>, so
any option using such tags will not be converted at all.
The option `services.jira.sso.applicationPassword` has been replaced by
`applicationPasswordFile` that needs to be readable by the `jira`-user
or group.
The new `crowd.properties` is created on startup in `~jira` and the
secret is injected into it using `replace-secret`.
Transform exit handlers of the form
trap cleanup EXIT [INT] [TERM] [QUIT] [HUP] [ERR]
(where cleanup is idempotent)
to
trap cleanup EXIT
This fixes a common bash antipattern.
Each of the above signals causes the script to exit. For each signal,
bash first handles the signal by running `cleanup` and then runs
`cleanup` again when handling EXIT.
(Exception: `vscode/*` prevents the second run of `cleanup` by removing
the trap in cleanup`).
Simplify the cleanup logic by just trapping exit, which is always run
when the script exits due to any of the above signals.
Note: In case of borgbackup, the exit handler is not idempotent, but just
trapping EXIT guarantees that it's only run once.
Commit 8109d8a set the `StateDirectory=` option of the systemd service
configuration to the value of `cfg.workDir` which is wrong, according
to dasJ [1]. This commit resolves this issue by stripping the
`/var/lib/` prefix from `cfg.workDir`.
[1] https://github.com/NixOS/nixpkgs/pull/172824#issuecomment-1130350412
There is a comment above the invocation of 'nextcloud-occ app:enable', stating
that the script should not fail if any of the apps cannot be enabled, but there
is nothing in place to suppress errors. The app:enable command already
continues installing the remaining apps when one fails to install, and we do not
want to suppress errors in the setup script, so this just removes the comment
about not failing.
* Add an option services.nextcloud.nginx.hstsMaxAge for setting the max-age
directive of the Strict-Transport-Security HTTP header.
* Make the Strict-Transport-Security HTTP header in the Nginx virtualhost block
dependant upon the option services.nextcloud.https instead of
services.nextcloud.nginx.recommendedHttpHeaders, as this header makes no sense
when not using HTTPS. (Closes#169465)
Added Nextcloud 23 and set it as the default Nextcloud version for the
NixOS module. Added PHP 8.1 as an option for phpPackage and default for
Nextcloud ≥ 24.
Release notes available at https://www.keycloak.org/docs/latest/release_notes/index.html#keycloak-18-0-0.
The way the database port is configured changed in Keycloak 18 and the
old way of including it in the `db-url-host` setting no longer
works. Use the new `db-url-port` setting instead.
Signed-off-by: Felix Singer <felixsinger@posteo.net>
Signed-off-by: Kim Lindberger <kim.lindberger@gmail.com>
I recently learned that Nextcloud 23's new profile feature — basically a
way for users to share personal contact details — has a problematic
default setting, profile data is shared with **everyone** by default.
This means that an unauthenticated user can access personal information
by accessing `nextcloud.tld/u/user.name`.
The announcement of v23 states[1]:
> We go a step further and introduce a profile page. Here you can put a
> description of yourself, show links to, for example, social media, what
> department you are in and information on how to contact you. All these
> are of course entirely optional and you can choose what is visible to who!
> The profile and user status are accessible also from our mobile and desktop clients.
It's not mentioned that by default you share personal information[3] with
everyone and personally I think that's somewhat problematic.
To work around that, I decided to add an option for the recently added[2]
and even set it to `false` by default to make an explicit opt-in for
that feature.
[1] https://nextcloud.com/blog/nextcloud-hub-2-brings-major-overhaul-introducing-nextcloud-office-p2p-backup-and-more/
[2] https://github.com/nextcloud/server/pull/31624/files
[3] By default, this affects the following properties:
* About
* Full name
* Headline
* Organisation
* Profile picture
* Role
* Twitter
* Website
Phone, Address and Email are not affected and only shown to
authenticated users by default.
With version 17 of Keycloak, the Wildfly based distribution was
deprecated in favor of the one based on Quarkus. The difference in
configuration is massive and to accommodate it, both the package and
module had to be rewritten.
This fixes the following issues with the database provisioning script
included in the services.keycloak module:
- It lacked permission to access the DB password file specified in the
module option 'services.keycloak.database.passwordFile'.
- It prevented Keycloak from starting after the second time if the user
chose MySQL for the database.
If a secret path is a subset of a second secret path, there's a risk
that its secret is substituted for the matching part of the second
path. To prevent this, use the sha256 of the paths as placeholder
string instead.
users.users.*.createHome makes home only owner-readable.
This breaks nginx reading static assets from nextcloud's home,
after a nixos-rebuild that did not restart nextcloud-setup.
Closes#112639
This commit introduces a new option
`services.nextcloud.nginx.recommendedHttpHeaders` that can be used to
optionally disable serving recommended HTTP Response Headers in nginx.
This is especially useful if some headers are already configured
elsewhere to be served in nginx and thus result in duplicate headers.
Resolves#120223
The `extraConfig` parameter only handles text - it doesn't support
arbitrary secrets and, with the way it's processed in the setup
script, it's very easy to accidentally unescape the echoed string and
run shell commands / feed garbage to bash.
To fix this, implement a new option, `config`, which instead takes a
typed attribute set, generates the `.env` file in nix and does
arbitrary secret replacement. This option is then used to provide the
configuration for all other options which change the `.env` file.
When upgrading bookstack, if something in the cache conflicts with the
new installation, the artisan commands might fail. To solve this, make
the cache lifetime bound to the setup service. This also removes the
`cacheDir` option, since the path is now handled automatically by
systemd.
Instead of referencing all library functions through `lib.` and
builtins through `builtins.` at every invocation, inherit them into
the appropriate scope.
Use systemd's LoadCredential mechanism to make the secret files
available to the service.
This gets rid of the privileged part of the ExecPreStart script which
only served to copy these files and assign the correct
permissions. There's been issues with this approach when used in
combination with DynamicUser, where sometimes the user isn't created
before the ExecPreStart script runs, causing the error
install: invalid user ‘keycloak’
This should fix that issue.
Unfortunately, all of the ExecPreStart script had to be moved to
ExecStart, since credentials aren't provided to ExecPreStart. See
https://github.com/systemd/systemd/issues/19604.
This together with extraConfig:
{
"subsystem=undertow"."server=default-server"."http-listener=default"."proxy-address-forwarding" = true;
"subsystem=undertow"."server=default-server"."https-listener=https"."proxy-address-forwarding" = true;
}
Allows to run Keycloak behind a reverse proxy that provides
X-Forwarded-* headers.
Allow update commands in the script to be ordered using `mkOrder`.
If we encounter ordered sub-objects we sort them by priority.
To implement this we now explicitly pass current node in `recurse`,
which also allows us to clean up edge case for top-level node.
Also refactor `recurse` to avoid passing result text argument; we
weren't tail recursive before anyway.
link to search.nixos.org instead of pulling package metadata out of pkgs. this
lets us cache docs of a few more modules and provides easier access to package
info from the HTML manual, but makes the manpage slightly less useful since
package description are no longer rendered.
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
One use case for Mattermost configuration is doing a "mostly
mutable" configuration where NixOS module options take priority
over Mattermost's config JSON.
Add a preferNixConfig option that prefers configured Nix options
over what's configured in Mattermost config if mutableConfig is set.
Remove the reliance on readFile (it's flake incompatible) and use
jq instead.
Merge Mattermost configs together on Mattermost startup, depending
on configured module options.
Write tests for mutable, mostly mutable, and immutable configurations.
- Added defaultText for all inheritable options.
- Add docs on using new defaults option to configure
DNS validation for all domains.
- Update DNS docs to show using a service to configure
rfc2136 instead of manual steps.
- Add the migrations directory to the package
- Add postgres support to the package
- Add a service for powerdns-admin
Co-authored-by: Zhaofeng Li <hello@zhaofeng.li>
some options have default that are best described in prose, such as
defaults that depend on the system stateVersion, defaults that are
derivations specific to the surrounding context, or those where the
expression is much longer and harder to understand than a simple text
snippet.
escape interpolations in descriptions where possible, replace them with
sufficiently descriptive text elsewhere. also expand cfg.* paths in
descriptions.
Instead of patching the path to /public in Discourse's sources, make
the nginx configuration refer to the symlink in the discourse
package which points to the real path.
When there is a mismatch between the path nginx serves and the path
Discourse thinks it serves, we can run into issues like files not
being served - at least when sendfile requests from the ruby app are
processed by nginx. The issue I ran into most recently is that backup
downloads don't work.
Since Discourse refers to the public directory relative to the Rails
root in many places, it's much easier to just sync this path to the
nginx configuration than trying to patch all occurrences in the
sources. This should hopefully mean less potential for breakage in
future Discourse releases, too.
Also, add support for updating plugins which keep gem versions in
files at the root of the repo (discourse-prometheus) and replace the
`up-plugin.sh` script with a README file pointing to the plugin
packaging documentation.
This option enables a jibri service on the same host that is running
jitsi-meet. It was written, along with the jibri module, by @puckipedia
for nixcon-video-infra 2020.
Co-authored-by: Puck Meerburg <puck@puck.moe>
The version 20 of Nextcloud will be EOLed by the end of this month[1].
Since the recommended default (that didn't raise an eval-warning) on
21.05 was Nextcloud 21, this shouldn't affect too many people.
In order to ensure that nobody does a (not working) upgrade across
several major-versions of Nextcloud, I replaced the derivation of
`nextcloud20` with a `throw` that provides instructions how to proceed.
The only case that I consider "risky" is a setup upgraded from 21.05 (or
older) with a `system.stateVersion` <21.11 and with
`services.nextcloud.package` not explicitly declared in its config. To
avoid that, I also left the `else-if` for `stateVersion < 21.03` which
now sets `services.nextcloud.package` to `pkgs.nextcloud20` and thus
leads to an eval-error. This condition can be removed
as soon as 21.05 is EOL because then it's safe to assume that only
21.11. is used as stable release where no Nextcloud <=20 exists that can
lead to such an issue.
It can't be removed earlier because then every `system.stateVersion <
21.11` would lead to `nextcloud21` which is a problem if `nextcloud19`
is still used.
[1] https://docs.nextcloud.com/server/20/admin_manual/release_schedule.html
A few minor changes to get #119638 - nextcloud: add option to set
datadir and extensions - ready:
* `cfg.datadir` now gets `cfg.home` as default to make the type
non-nullable.
* Enhanced the `basic` test to check the behavior with a custom datadir
that's not `/var/lib/nextcloud`.
* Fix hashes for apps in option example.
* Simplify if/else for `appstoreenable` in override config.
* Simplify a few `mapAttrsToList`-expressions in
`nextcloud-setup.service`.
Note the appstoreEnable which will prevent nextcloud form updating
nix-managed apps. This is needed because nextcloud will store an other
version of the app in /var/lib/nextcloud/store-apps and it will
no longer be manageable.
The `$(</path/to/file)`-expansion appears verbatim in the cmdline of
`nextcloud-occ` which means that an unprivileged user could find
sensitive values (i.e. admin password & database password) by monitoring
`/proc/<pid>/cmdline`.
Now, these values don't appear in a command line anymore, but will be
passed as environment variables to `nextcloud-occ`.
* Linkify documentation about objectstore-feature rather than only
mentioning it.
* Use `<literal>` where it makes sense.
* Remove unnecessary `Whether to load` from `enableImagemagick` because
`mkEnableOption` already prepends `Whether to enable` to the given
description.
Previously, the `nix_read_pwd` function was only used for reading the
`dbpassFile`, however it has since been refactored to handle reading
other secret files too. This fixes the message of the exception that is
thrown in the case that the file is not present so that it no longer
refers specifically to the `dbpass` file.
Removes the submodule in favour of using an attrset.
Also:
- Makes better use of nix's laziness in config expansion.
- Makes use of `boolToString` where applicable.
We should discourage users from adding secrets in a way that allows for
them to end up in the globally readable `/nix/store`. Users should use
the `objectstore.s3.secretFile` option instead.
This allows to declaratively configure an S3 class object storage as the
primary storage for the nextcloud service. Previously, this could only
be achieved by manually editing the `config.php`.
I've started testing this today with my own digitalocean nextcloud
instance, which now points to my digitalocean S3-compatible "Space" and
all appears to be working smoothly.
My motivation for this change is my recent discovery of how much cheaper
some S3-compatible object storage options are compared to digitalocean's
"Volume" options.
Implementation follows the "Simple Storage Service" instructions here:
https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html
I have neglected to implement a submodule for the OpenStack Swift
object storage as I don't personally have a use case for it or a method
to test it, however the new `nextcloud.objectstore.s3` submodule should
act as a useful guide for anyone who does wish to implement it.
The MariaDB version 10.6 doesn't seem supported with current Nextcloud
versions and the test fails with the following error[1]:
nextcloud # [ 14.950034] nextcloud-setup-start[1001]: Error while trying to initialise the database: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 4047 InnoDB refuses to write tables with ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE.
According to a support-thread in upstream's Discourse[2] this is because
of a missing support so far.
Considering that we haven't received any bugreports so far - even though
the issue already exists on master - and the workaround[3] appears to
work fine, an evaluation warning for administrators should be
sufficient.
[1] https://hydra.nixos.org/build/155015223
[2] https://help.nextcloud.com/t/update-to-next-cloud-21-0-2-has-get-an-error/117028/15
[3] setting `innodb_read_only_compressed=0`
When restoring a backup, discourse decompresses the backup archive in
the /share/discourse/tmp dir. Before this change, it is linked to /run
which is typically backed by memory, so the backup will fail to
restore if you do not have enough memory on your system to contain the
backup. This has already happened to me on two small forums.
This moves tmp to the StateDirectory /var/lib/discourse/tmp which is
typically backed by disk.
The `services.mastodon` module currently hardcodes sidekiq's concurrency
to 25, but doesn't set a DB pool size, which defaults to 5 or the number
of configured web threads.
(This behaviour is very strange, and arguably a mastodon bug.)
This also makes sidekiq's concurrency configurable, because 25 is a tad
high for the hardware I'm running it on.
This error occurs if `nextcloud-occ maintenance:install` fails and the
`upgrade` command is attempted to be executed afterwards.
Due to the nature of the installer we can't do much about it, so I guess
it makes sense to add some notes about it. The other notes in the
`Pitfalls`-section are semantically a list of different topics, so I
changed that accordingly now.
Closes#111175
This doesn't work anymore and thus breaks the installation leaving a
broken `/var/lib/nextcloud`.
It isn't a big deal since we set this value in the override config
before, so the correct table-prefix is still used. In order to confirm
that, I decided to add a custom prefix to the basic test.
Let the update.py script handle the initial, repetitive task of
packaging new plugins. With this in place, the plugin only needs to be
added to the list in `update-plugins` and most of the work will be
done automatically when the script is run. Metadata still needs to be
filled in manually and some packages may of course require additional
work/patching.
This combines the two functions nix_read_pwd and nix_read_secrets into a
single function nix_read_file, which takes two arguments: the file to be
read, and an error message that should be printed if reading the file
failed.
For plugins to work properly, their assets need to be precompiled
along with the rest of Discourse's assets. This means we need to build
new packages when the list of plugins change.
Since 3edde6562e, we can no longer use
aliases inside the test framework. This has the implication that we can
no longer use aliases in any NixOS modules used by the test framework as
well (which is good), but does mean we need to clean up any instances
where this is the case.
* Most significant is probably the patching necessary to run plausible
with postgres without superuser privilege. This change includes:
* updating ecto_sql to 3.6 where `CREATE DATABASE` is only executed if
it doesn't exist[1].
* patching a migration to only modify the `users.email` column (to use
`citext` rather than creating the extension. `plausible-postgres`
takes care of that).
* Correctly declare dependencies in systemd.
* A few minor fixes.
[1] 051baf669e
A secret key generated by the nixos module was misspelled, which could
possibly impact the security of session cookies.
To recover from this situation we will wipe all security keys that were
previously generated by the NixOS module, when the misspelled one is
found. This will result in all session cookies being invalidated. This
is confirmed by the wordpress documentation:
> You can change these at any point in time to invalidate all existing
> cookies. This does mean that all users will have to login again.
https://wordpress.org/support/article/editing-wp-config-php/#security-keys
Meanwhile this issue shouldn't be too grave, since the salting function
of wordpress will rely on the concatenation of both the user-provided
and automatically generated values, that are stored in the database.
> Secret keys are located in two places: in the database and in the
> wp-config.php file. The secret key in the database is randomly
> generated and will be appended to the secret keys in wp-config.php.
https://developer.wordpress.org/reference/functions/wp_salt/
Fixes: 2adb03fdae ("nixos/wordpress:
generate secrets locally")
Reported-by: Moritz Hedtke <Moritz.Hedtke@t-online.de>
Assert that the PostgreSQL version being deployed is the one used
upstream. Allow the user to override this assertion, since it's not
always possible or preferable to use the recommended one.
Instead of requiring the user to bundle the certificate and private
key into a single file, provide separate options for them. This is
more in line with most other modules.
`install` copies the files before setting their mode, so there could
be a breif window where the secrets are readable by other users
without a strict umask.
Feeding `psql` the password on the command line leaks it through the
`psql` process' `/proc/<pid>/cmdline` file. Using `echo` to put the
command in a file and then feeding `psql` the file should work around
this, since `echo` is a bash builtin and thus shouldn't spawn a new
process.
Using `replace-literal` to insert secrets leaks the secrets through
the `replace-literal` process' `/proc/<pid>/cmdline`
file. `replace-secret` solves this by reading the secret straight from
the file instead, which also simplifies the code a bit.
Using `replace-literal` to insert secrets leaks the secrets through
the `replace-literal` process' `/proc/<pid>/cmdline`
file. `replace-secret` solves this by reading the secret straight from
the file instead, which also simplifies the code a bit.
This reverts commit d9e18f4e7f.
This change is broken, since it doesn't configure the proper database
username in keycloak when provisioning a local database with a custom
username. Its intended behavior is also potentially confusing and
dangerous, so rather than fixing it, let's revert to the old one.
Bash doesn't handle subshell errors properly if the result is used as
input to a command. To cause the services to fail when the files can't
be read, we need to assign the value to a variable, then export it
separately.
As the only consequence of isSystemUser is that if the uid is null then
it's allocated below 500, if a user has uid = something below 500 then
we don't require isSystemUser to be set.
Motivation: https://github.com/NixOS/nixpkgs/issues/112647
This adds two options to the nextcloud module, with the aim of allowing
the entirety of `config.php` to be set declaratively:
1. `services.nextcloud.extraOptions`, which takes free-form options
given as an attribute set, and reads them in via json from the php
side (which lets us prevent syntax errors in php, if not key errors –
given the full length of nextcloud's potential options, I don't think
specifying them all via Nix is a viable option)
2. `services.nextcloud.secretFile`, which takes a path to a json file
specifying options which a user may want to keep secret and not add
to their nix store; this file is read in the same way on startup by
php, and may even overwrite options set in some other way.
This allows for shared hledger installations, where the web interface is
available via network and multiple user share a SSH access to the
hledger user.
Also added `--serve` to the CLI options, as hledger-web tries to open a
webbrowser otherwise:
hledger-web: xdg-open: rawSystem: runInteractiveProcess: exec: does not
exist (No such file or directory)
Co-authored-by: Aaron Andersen <aaron@fosslib.net>
ChangeLog: https://nextcloud.com/changelog/#latest21
* Packaged 21.0.0, test-deployed it to my personal instance and tested
the most basic functionality (`davfs2`-mount, {card,cal}dav sync, file
management).
* Bumped the default version for unstable/21.05 to `nextcloud21`. Since
`nextcloud20` was added after the release of 20.09 (and thus the
default on 20.09 is still `nextcloud19`), it's now needed to upgrade
across two majors.
This is not a problem though since it's possible to upgrade to v20 on
20.09 already and if not, the module will guard the administrator
through the upgrade with eval warnings as it's the case since 20.03.
* Dropped `nextcloud17` attribute and marked `nextcloud18` as EOL.
4255954d97 set the StateDirectory to 0750,
but nginx wasn't in the Mastodon group. This commit also deletes a line,
that probably was intended to serve this purpose, but makes no sense.
Why should the Mastodon user be added as an extraGroup to the nginx
user?
The NixOS 21.03 release has been delayed to 21.05. See NixOS/rfcs#80.
There are two instances of 21.03 which have been left as is, since they
are in stateVersion comparisons. This will ensure that existing user
configurations which refer to 21.03 will continue to work.
* It should be made explicit in the eval-error that the CVE only affects
a component which is turned off by default.
* For more clarity, the default version used by the module is noted in
the manual.
Closes#108419
The service was failing with:
gunicorn[2192104]: [2020-12-31 13:35:28 +0000] [2192104] [ERROR] Exception in worker process
gunicorn[2192104]: Traceback (most recent call last):
gunicorn[2192104]: File "/nix/store/jmc14qf1sfnlhw27xyyj862ghkmdkj5a-python2.7-gunicorn-19.10.0/lib/python2.7/site-packages/gunicorn/arbiter.py", line 586, in spawn_worker
gunicorn[2192104]: worker.init_process()
gunicorn[2192104]: File "/nix/store/jmc14qf1sfnlhw27xyyj862ghkmdkj5a-python2.7-gunicorn-19.10.0/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 196, in init_process
gunicorn[2192104]: self.patch()
gunicorn[2192104]: File "/nix/store/jmc14qf1sfnlhw27xyyj862ghkmdkj5a-python2.7-gunicorn-19.10.0/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 65, in patch
gunicorn[2192104]: monkey.patch_all(subprocess=True)
gunicorn[2192104]: File "/nix/store/fysf67w3i8iv1hfvp536nl8jbzqyk1s7-python-2.7.18-env/lib/python2.7/site-packages/gevent/monkey.py", line 1160, in patch_all
gunicorn[2192104]: from gevent import events
gunicorn[2192104]: File "/nix/store/fysf67w3i8iv1hfvp536nl8jbzqyk1s7-python-2.7.18-env/lib/python2.7/site-packages/gevent/events.py", line 67, in <module>
gunicorn[2192104]: from zope.interface import Interface
gunicorn[2192104]: ImportError: No module named zope.interface
`file_exists` also returns `FALSE` if the file is in a directory that
can't be read by the user. This e.g. happens if permissions for
`nixops(1)`-deployment keys aren't configured correctly.
This patch improves the error message for invalid files to avoid
confusion[1].
[1] https://discourse.nixos.org/t/nixops-deploy-secrets-to-nextcloud/10414/4
This adds a `package` option to allow for easier overriding of the used
CodiMD version and `runCommandLocal` with `nativeBuildInputs` is now
used to pretty print the configuration.
These were broken since 2016:
f0367da7d1
since StartLimitIntervalSec got moved into [Unit] from [Service].
StartLimitBurst has also been moved accordingly, so let's fix that one
too.
NixOS systems have been producing logs such as:
/nix/store/wf98r55aszi1bkmln1lvdbp7znsfr70i-unit-caddy.service/caddy.service:31:
Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring.
I have also removed some unnecessary duplication in units disabling
rate limiting since setting either interval or burst to zero disables it
(ad16158c10/src/basic/ratelimit.c (L16))
Please note that this is only for 21.03 since `nextcloud19` is intended
to be the default for the already feature-frozen 20.09 (the bump itself
is supposed to get backported however).
Both packages will get EOLed within the lifetime of 20.09. `nextcloud17`
can be removed entirely (the attribute-path is kept however to provide
meaningful errors), however `nextcloud18` must be kept as `insecure` to
make sure that users from `nextcloud17` can properly upgrade to
`nextcloud19` on NixOS 20.09.
Secrets are injected from the environment into the rendered
configuration before each startup using envsubst.
The test now makes use of this feature for the db password.
This patch ensures that latest Nextcloud works flawlessly again on our
`nginx`. The new config is mostly based on upstream recommendations
(again)[1]:
* Trying to access internals now results in a 404.
* All `.php`-routes get properly resolved now.
* Removed 404/403 handling from `nginx` as the app itself takes care of
this. Also, this breaks the `/ocs`-API.
* `.woff2?`-files expire later than other assets like images.
Closes#95293
[1] https://docs.nextcloud.com/server/latest/admin_manual/installation/nginx.html
Since systemd 243, docs were already steering users towards using
`journal`:
eedaf7f322
systemd 246 will go one step further, it shows warnings for these units
during bootup, and will [automatically convert these occurences to
`journal`](f3dc6af20f):
> [ 6.955976] systemd[1]: /nix/store/hwyfgbwg804vmr92fxc1vkmqfq2k9s17-unit-display-manager.service/display-manager.service:27: Standard output type syslog is obsolete, automatically updating to journal. Please update│······················
your unit file, and consider removing the setting altogether.
So there's no point of keeping `syslog` here, and it's probably a better
idea to just not set it, due to:
> This setting defaults to the value set with DefaultStandardOutput= in
> systemd-system.conf(5), which defaults to journal.
This breaks the Nextcloud vhost declaration when adding e.g. another
vhost as the `services.nginx.virtualHosts` option has `{ nextcloud =
...; }` as *default* value which will be replaced by another
`virtualHosts`-declaration with a higher (e.g. the default) priority.
The following cases are now supported & covered by the module:
* `nginx` is enabled with `nextcloud` enabled and other vhosts can be
added / other options can be declared without having to care
about the declaration's priority.
* Settings in the `nextcloud`-vhost in `nginx` have to be altered using
`mkForce` as this is the only way how we officially support `nginx`
for `nextcloud` and customizations have to be done explicitly using
`mkForce`.
* `nginx` will be completely omitted if a user enables nextcloud
and disables nginx using `services.nginx.enable = false;`. (because
nginx will be enabled by this module using `mkDefault`).
This reverts commit 128dbb31cc.
Closes#95259
In fd9eb16b24, the option
"services.nextcloud.nginx.enable" has been removed since the module now
exclusively supports nginx only.
Unfortunately, with the option gone from the manual, the link in the
Nextcloud-specific documentation referencing the NixOS option also
became a dead link and thus the manual will no longer build.
I also removed a second reference to this option in the Nextcloud-
specific documentation, which while it doesn't lead to a build error in
the manual is nevertheless a good idea to remove as well to ensure we
don't present outdated information to readers of the manual.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @DavHau, @Ma27
- remove optons cfg.user, cfg.groups
- add option `serverUser` which is required when not using nginx
- add `serverUser` to nextcloud group
- set user/group to "nextcloud" for nextcloud services
- make setup-service non-root
Enhance the heuristics to make sure that a user doesn't accidentally
upgrade across two major versions of Nextcloud (e.g. from v17 to v19).
The original idea/discussion has been documented in the nixpkgs manual[1].
This includes the following changes:
* `nextcloud19` will be selected automatically when having a stateVersion
greater or equal than 20.09. For existing setups, the package has to
be selected manually to avoid accidental upgrades.
* When using `nextcloud18` or older, a warning will be thrown which recommends
upgrading to `nextcloud19`.
* Added a brief paragraph about `nextcloud19` in the NixOS 19.09 release
notes.
* Restart `phpfpm` if the Nextcloud-package (`cfg.package`) changes[2].
[1] https://nixos.org/nixos/manual/index.html#module-services-nextcloud-maintainer-info
[2] https://github.com/NixOS/nixpkgs/pull/89427#issuecomment-638885727
This option exposes the prefconfigured nextcloud-occ
program. nextcloud-occ can then be used in other systemd services or
added in environment.systemPackages.
The nextcloud test shows how it can be add in
environment.systemPackages.
The OC_PASS environment variable can be used to create a user with
`occ user:add --password-from-env`. It is currently not possible to
use the `nextcloud-occ` to "non-interactively" create a user since
this variable is ignored by sudo.
Rework withExtensions / buildEnv to handle currently enabled
extensions better and make them compatible with override. They now
accept a function with the named arguments enabled and all, where
enabled is a list of currently enabled extensions and all is the set
of all extensions. This gives us several nice properties:
- You always get the right version of the list of currently enabled
extensions
- Invocations chain
- It works well with overridden PHP packages - you always get the
correct versions of extensions
As a contrived example of what's possible, you can add ImageMagick,
then override the version and disable fpm, then disable cgi, and
lastly remove the zip extension like this:
{ pkgs ? (import <nixpkgs>) {} }:
with pkgs;
let
phpWithImagick = php74.withExtensions ({ all, enabled }: enabled ++ [ all.imagick ]);
phpWithImagickWithoutFpm743 = phpWithImagick.override {
version = "7.4.3";
sha256 = "wVF7pJV4+y3MZMc6Ptx21PxQfEp6xjmYFYTMfTtMbRQ=";
fpmSupport = false;
};
phpWithImagickWithoutFpmZip743 = phpWithImagickWithoutFpm743.withExtensions (
{ enabled, all }:
lib.filter (e: e != all.zip) enabled);
phpWithImagickWithoutFpmZipCgi743 = phpWithImagickWithoutFpmZip743.override {
cgiSupport = false;
};
in
phpWithImagickWithoutFpmZipCgi743
Use types.str instead of types.path to exclude private information from
the derivation.
Add a warinig about the contents of acl beeing included in the nix
store.
Enables multi-site configurations.
This break compatibility with prior configurations that expect options
for a single dokuwiki instance in `services.dokuwiki`.
The new wording does not assume the user is upgrading.
This is because a user could be setting up a new installation on 20.03
on a server that has a 19.09 or before stateVersion!!
The new wording ensures that confusion is reduced by stating that they
do not have to care about the assumed 16→17 transition.
Then, the wording explains that they should, and how to upgrade to
version 18.
It also reviews the confusing wording about "multiple" upgrades.
* * *
The only thing we cannot really do is stop a fresh install of 17 if
there was no previous install, as it cannot be detected. That makes a
useless upgrade forced for new users with old state versions.
It is also important to state that they must set their package to
Nextcloud 18, as future upgrades to Nextcloud will not allow an uprade
from 17!
I assume future warning messages will exist specifically stating what to
do to go from 18 to 19, then 19 to 20, etc...
So now we have only packages for human interaction in php.packages and
only extensions in php.extensions. With this php.packages.exts have
been merged into the same attribute set as all the other extensions to
make it flat and nice.
The nextcloud module have been updated to reflect this change as well
as the documentation.
Many options define their example to be a Nix value without using
literalExample. This sometimes gets rendered incorrectly in the manual,
causing confusion like in https://github.com/NixOS/nixpkgs/issues/25516
This fixes it by using literalExample for such options. The list of
option to fix was determined with this expression:
let
nixos = import ./nixos { configuration = {}; };
lib = import ./lib;
valid = d: {
# escapeNixIdentifier from https://github.com/NixOS/nixpkgs/pull/82461
set = lib.all (n: lib.strings.escapeNixIdentifier n == n) (lib.attrNames d) && lib.all (v: valid v) (lib.attrValues d);
list = lib.all (v: valid v) d;
}.${builtins.typeOf d} or true;
optionList = lib.optionAttrSetToDocList nixos.options;
in map (opt: {
file = lib.elemAt opt.declarations 0;
loc = lib.options.showOption opt.loc;
}) (lib.filter (opt: if opt ? example then ! valid opt.example else false) optionList)
which when evaluated will output all options that use a Nix identifier
that would need escaping as an attribute name.
It's impossible to move two major-versions forward when upgrading
Nextcloud. This is an issue when comming from 19.09 (using Nextcloud 16)
and trying to upgrade to 20.03 (using Nextcloud 18 by default).
This patch implements the measurements discussed in #82056 and #82353 to
improve the update process and to circumvent similar issues in the
future:
* `pkgs.nextcloud` has been removed in favor of versioned attributes
(currently `pkgs.nextcloud17` and `pkgs.nextcloud18`). With that
approach we can safely backport major-releases in the future to
simplify those upgrade-paths and we can select one of the
major-releases as default depending on the configuration (helpful to
decide whether e.g. `pkgs.nextcloud17` or `pkgs.nextcloud18` should be
used on 20.03 and `master` atm).
* If `system.stateVersion` is older than `20.03`, `nextcloud17` will be
used (which is one major-release behind v16 from 19.09). When using a
package older than the latest major-release available (currently v18),
the evaluation will cause a warning which describes the issue and
suggests next steps.
To make those package-selections easier, a new option to define the
package to be used for the service (namely
`services.nextcloud.package`) was introduced.
* If `pkgs.nextcloud` exists (e.g. due to an overlay which was used to
provide more recent Nextcloud versions on older NixOS-releases), an
evaluation error will be thrown by default: this is to make sure that
`services.nextcloud.package` doesn't use an older version by accident
after checking the state-version. If `pkgs.nextcloud` is added
manually, it needs to be declared explicitly in
`services.nextcloud.package`.
* The `nixos/nextcloud`-documentation contains a
"Maintainer information"-chapter which describes how to roll out new
Nextcloud releases and how to deal with old (and probably unsafe)
versions.
Closes#82056
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
Only use sudo if we are currently not running as the nextcloud user.
This is problematic when occ is called from a systemd service with
NoNewPrivileges=true
This commit adds a Strict-Transport-Security header to
the nginx config file generated by the nextcloud module.
The Strict-Transport-Security header is recommended in
official guide for hardening Nextcloud installations:
https://docs.nextcloud.com/server/16/admin_manual/installation/harden_server.html
Further, if it is not set, we see a warning in the security scan results
in the Nextcloud admin panel:
```
The "Strict-Transport-Security" HTTP header is not set to at least "15552000" seconds. For enhanced security, it is recommended to enable HSTS as described in the security tips
```
IE6 is long gone and this directive is not useful anymore. We can
spare a few CPU cycles (and maybe skip some bugs) by not trying to
disable gzip for MSIE6.
Regression I caused with 3944aa051c, sorry
for this! The Nextcloud installer broke back then because
`trusted_domains` was an empty value by default (a.k.a an empty array)
which seemed to break the config merger of Nextcloud as Nextcloud
doesn't do recursive merging and now no domain was trusted because of
that, hence Nextcloud was unreachable for the `curl` call.
One of the main problems of the Nextcloud module is that it's currently
not possible to alter e.g. database configuration after the initial
setup as it's written by their imperative installer to a file.
After some research[1] it turned out that it's possible to override all values
with an additional config file. The documentation has been
slightly updated to remain up-to-date, but the warnings should
remain there as the imperative configuration is still used and may cause
unwanted side-effects.
Also simplified the postgresql test which uses `ensure{Databases,Users}` to
configure the database.
Fixes#49783
[1] https://github.com/NixOS/nixpkgs/issues/49783#issuecomment-483063922
nixos/nextcloud: Add documentation for nextcloud app installation and updates
nixos/nextcloud: Enable autoUpdateApps in nextcloud test
nixos/nextcloud: Fix typo in nixos/modules/services/web-apps/nextcloud.xml
Co-Authored-By: Florian Klink <flokli@flokli.de>
nixos/nextcloud: Escape html in option description
nixos/nextcloud: Fix autoUpdateApps URL in documentation.
Co-Authored-By: Florian Klink <flokli@flokli.de>
