Using `replace-literal` to insert secrets leaks the secrets through
the `replace-literal` process' `/proc/<pid>/cmdline`
file. `replace-secret` solves this by reading the secret straight from
the file instead, which also simplifies the code a bit.
The upstream recommended minimum length for db_key_base is 30 bytes,
which our option descriptions repeated. Recently, however, upstream
has, in many places, moved to using aes-256-gcm, which requires a key
of exactly 32 bytes. To allow for shorter keys, the upstream code pads
the key in some places. However, in many others, it just truncates the
key if it's too long, leaving it too short if it was to begin
with. This adds a patch that fixes this and updates the descriptions
to recommend a key of at least 32 characters.
See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53602
This should reduce memory fragmentation drastically and is recommended
by both the Puma and the Sidekiq author. It's also the default value
for Ruby deployments on Heroku.
To make it easier to start and stop all GitLab services, introduce
`gitlab.target` which wants all services (meaning they will start with
it) and which all services are part of (meaning they will stop with
it).
Make the config initialization script run in gitlab.service's PreStart
section into two new services, `gitlab-config.service` and
`gitlab-db-config.service`. Other services can then depend on the
config scripts they need instead of unnecessarily depending on
`gitlab.service`. This makes the reason for the configured service
dependencies much clearer and should also reduce the restart time of
the `gitlab` service quite a lot, when triggered manually.
Also, set up stricter service dependencies, using `bindsTo`, to ensure
that if a service fails or is stopped, its dependants are also
stopped. For example, if we're using the `postgresql` service and it's
stopped, `gitlab.service` and `gitlab-sidekiq.service`, which depend on
it to function, should also be stopped.
A function is more appropriate for this use. See
http://mywiki.wooledge.org/BashFAQ/050 for reference.
Also, we don't need to run the service as root: since we essentially
run all commands as `services.postgresql.superUser` anyway, the whole
service can just run as that user instead.
Change the default SMTP port to `25`, to better match the default
address `localhost`. This gets rid of some error outputs in the test,
where it fails to connect to localhost:465.
Also, don't enable postfix by default unless it's actually useful to
us.
This removes all the subdirectories in `config` on start.
From one version of GitLab to the next, the files in the `config`
directory changes. Since we're only overwriting the existing files
with ones from the repo, cruft sometimes gets left behind,
occationally causing issues.
Ideally, all configuration put in the `config` directory is declared
by NixOS options and we could just remove the whole directory on
start, but I'm not sure if that's the case. It would also require a
little bit of additional rework and testing. The subdirectories,
however, should seldom contain user configuration and the ones that
frequently does, `initializers`, is already removed on start.
The comment at the top of git-and-tools/default.nix said:
/* All git-relates tools live here, in a separate attribute set so that users
* can get a fast overview over what's available.
but unfortunately that hasn't actually held up in practice.
Git-related packages have continued to be added to the top level, or
into gitAndTools, or sometimes both, basically at random, so having
gitAndTools is just confusing. In fact, until I looked as part of
working on getting rid of gitAndTools, one program (ydiff) was
packaged twice independently, once in gitAndTools and once at the top
level (I fixed this in 98c3490196).
So I think it's for the best if we move away from gitAndTools, and
just put all the packages it previously contained at the top level.
I've implemented this here by just making gitAndTools an alias for the
top level -- this saves having loads of lines in aliases.nix. This
means that people can keep referring to gitAndTools in their
configuration, but it won't be allowed to be used within Nixpkgs, and
it won't be presented to new users by e.g. nix search.
The only other change here that I'm aware of is that
appendToName "minimal" is not longer called on the default git
package, because doing that would have necessitated having a private
gitBase variable like before. I think it makes more sense not to do
that anyway, and reserve the "minimal" suffix only for gitMinimal.
> WARNING: PID file creation will be removed in Sidekiq 6.0, see #4045.
Please use a proper process supervisor to start and manage your
services
Since NixOS uses a proper process supervisor AND does not use the PID
file anywhere, we can just drop it to be upwards compatible and fix that
warning.
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
Default behavior is to continue executing the script even when one or
multiple steps fail. We want to abort early if any part of the
initialization fails to not run with a partially initialized state.
Default behavior also allows dereferencing non-existent variables,
potentially resulting in hard-to-find bugs.
On start, unicorn, sidekiq and other parts running ruby code emits
quite a few warnings similar to
/var/gitlab/state/config/application.rb:202: warning: already initialized constant Gitlab::Application::LOOSE_EE_APP_ASSETS
/nix/store/ysb0lgbzxp7a9y4yl8d4f9wrrzy9kafc-gitlab-ee-12.3.5/share/gitlab/config/application.rb:202: warning: previous definition of LOOSE_EE_APP_ASSETS was here
/var/gitlab/state/lib/gitlab.rb:38: warning: already initialized constant Gitlab::COM_URL
/nix/store/ysb0lgbzxp7a9y4yl8d4f9wrrzy9kafc-gitlab-ee-12.3.5/share/gitlab/lib/gitlab.rb:38: warning: previous definition of COM_URL was here
This seems to be caused by the same ruby files being evaluated
multiple times due to the paths being different - sometimes they're
loaded using the direct path and sometimes through a symlink, due to
our split between config and package data. To fix this, we make sure
that the offending files in the state directory always reference the
store path, regardless of that being the real file or a symlink.
This reverts commit 2ee14c34ed.
This caused the initializers directory to be cleaned out while gitlab
was running in some instances. We clean out the directory on the
preStart stage already, so ensuring existance and permissions should
suffice.
gitlab:db:configure prints the root user's password to stdout on
successful setup, which means it will be logged to the
journal. Silence this informational output. Errors are printed to
stderr and will thus still be let through.
The initializers directory is populated with files from the gitlab
distribution on start, but old files will be left in the state folder
even if they're removed from the distribution, which can lead to
startup failures. Fix this by always purging the directory on start
before populating it.
Since the preStart script is no longer running in privileged mode, we
reassign the files in the state directory and its config subdirectory
to the user we're running as. This is done by splitting the preStart
script into a privileged and an unprivileged part where the privileged
part does the reassignment.
Also, delete the database.yml symlink if it exists, since we want to
create a real file in its place.
Fixes#68696.
Make sure that we don't create a database if we're not going to
connect to it. Also, fix the assertion that usernames be equal to only
trig when peer authentication is used (databaseHost == "").
config.services.postgresql.package is only defined when the postgresql
service is activated, which means we fail to evaluate when
databaseCreateLocally == false. Fix this by using the default
postgresql package when the postgresql service is disabled.
The state path now, since the transition from initialization in
preStart to using systemd-tmpfiles, has the following restriction: no
parent directory can be owned by any other user than root or the user
specified in services.gitlab.user. This is a potentially breaking
change and the cause of the error isn't immediately obvious, so
document it both in the release notes and statePath description.
Adds the ability to make any parameter specified in extraConfig secret
by defining it an attrset containing the attr _secret, which in turn
is a path to a file containing the actual secret.
Use the postgresql module to provision a local db (if
databaseCreateLocally is true) instead of doing this locally.
Switch to using the local unix socket for db connections by default;
this is needed since dbs created by the postgresql module only support
peer authentication.
Instead of running the rake tasks db:schema:load, db:migrate and
db:seed_fu, run gitlab:db:configure, which in turn runs these tasks
when needed.
Solves issue #53852 for gitlab.
Add support for storing secrets in files outside the nix store, since
files in the nix store are world-readable and secrets therefore can't
be stored safely there.
The old string options are kept, since they can potentially be handy
for testing purposes, but their descriptions now state that they
shouldn't be used in production. The manual section is updated to use
the file options rather than the string options and the tests now test
both.
