- Add HDFS journalnode and ZKFC services
- Test failover of HDFS and YARN master services in full hadoop test
- Check if a minimal HDFS cluster works in the minimal HDFS test
The parentheses prevent the `continue` line from working by running the
enclosed in a subshell -- I noticed that ZFS would start asking me for
my password to encrypted child datasets, even though they were not
specified in `requestEncryptionCredentials`. The following logs would
also be present in the import unit's journal:
Oct 31 22:13:17 host systemd[1]: Starting Import ZFS pool "pool"...
Oct 31 22:13:44 host zfs-import-pool-start[3711]: importing ZFS pool "pool"...
Oct 31 22:13:44 host zfs-import-pool-start[4017]:pool/nix/store/39zij3xcxn4w38v6x8f88bx8y91nv0rm-unit-script-zfs-import-pool-start/bin/zfs-import-pool-start: line 31: continue: only meaningful in a `for', `while', or `until' loop
Oct 31 22:13:44 host zfs-import-pool-start[4020]:pool/nix/store/39zij3xcxn4w38v6x8f88bx8y91nv0rm-unit-script-zfs-import-pool-start/bin/zfs-import-pool-start: line 31: continue: only meaningful in a `for', `while', or `until' loop
Oct 31 22:15:14 host zfs-import-pool-start[4023]: Failed to query password: Timer expired
Oct 31 22:15:14 host zfs-import-pool-start[4024]: Key load error: encryption failure
Oct 31 22:15:14 host systemd[1]: zfs-import-pool.service: Main process exited, code=exited, status=255/EXCEPTION
Oct 31 22:15:14 host systemd[1]: zfs-import-pool.service: Failed with result 'exit-code'.
Oct 31 22:15:14 host systemd[1]: Failed to start Import ZFS pool "pool".
Details on https://github.com/NixOS/nixops/issues/1063#issuecomment-453253666.
`partOf` makes that if `smokeping.service` is stopped, `thttpd.service` will
be stopped as well.
(But not that `thttpd` will be started when `smokeping` is started).
Once `thttpd.service` is stopped that way, `Restart = always` will not apply.
When the smokeping config options are changed, NixOS's `switch-configuration.pl`
will stop `smokeping` (whit shuts down thttpd due to `partOf`), and then restart
smokeping; but this does not start thttpd.
As a result, thttpd will be off after changing the config, which isn't desired.
This commit fixes it by removing the `partOf`, which makes `Restart` work
as expected.
This avoids a common problem:
Until now, port forwarding to multiple hosts running smokeping did not work;
they all show the data of the first smokeping instance.
That ws because the image URLs generated by smokeping are absolute
(`imgurl` setting).
Consequently, if you ran
ssh node-1 -L 8081:localhost:8081
ssh node-2 -L 8081:localhost:8082
ssh node-3 -L 8081:localhost:8083
and try to open http://localhost:8081, http://localhost:8082 and
http://localhost:8083, they all would show the images of node-1!
Using a relative `imgurl` fixes that.
As per smokeping docs on `imgurl`:
> Either an absolute URL to the `imgcache` directory or one relative to the
> directory where you keep the SmokePing cgi.
qemu_kvm is only built for one architecture, so it's smaller and takes
MUCH less time to build if it has to be built from source. And this
module doesn't support running a VM for one architecture from another
architecture, so the one architecture is all we'll need.
Some upstream changes broke the automatic fallback to SwiftShader.
Without this workaround the GPU initialization fails (apparently due to requiring Vulkan):
machine # libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
machine # [1001:1001:1101/104304.000625:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
machine # [1052:1052:1101/104305.060496:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # [1084:1084:1101/104305.165898:ERROR:angle_platform_impl.cc(44)] Display.cpp:894 (initialize): ANGLE Display::initialize error 0: Internal Vulkan error (-3): Initialization of an object could not be completed for implementation-specific reasons, in ../../third_party/angle/src/libANGLE/renderer/vulkan/RendererVk.cpp, initialize:1048.
machine # [1084:1084:1101/104305.171191:ERROR:gl_surface_egl.cc(782)] EGL Driver message (Critical) eglInitialize: Internal Vulkan error (-3): Initialization of an object could not be completed for implementation-specific reasons, in ../../third_party/angle/src/libANGLE/renderer/vulkan/RendererVk.cpp, initialize:1048.
machine # [1084:1084:1101/104305.178707:ERROR:gl_surface_egl.cc(1382)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
machine # [1084:1084:1101/104305.180111:ERROR:gl_ozone_egl.cc(20)] GLSurfaceEGL::InitializeOneOff failed.
machine # [1084:1084:1101/104305.189760:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # [1092:1092:1101/104305.233470:ERROR:gpu_init.cc(457)] Passthrough is not supported, GL is disabled, ANGLE is
This workaround restores the previous result:
machine # [1004:1004:1101/104551.131190:ERROR:gpu_init.cc(457)] Passthrough is not supported, GL is swiftshader, ANGLE is
The workaround is only required for Chromium, not Google Chrome.
This reverts commit e2bea4427b.
While this commit was probably fine, I want to be conservative
with changes right before the release branch-off.
So far the extendModules commits have been adding and refactoring
expressions that did not affect derivation hashes, whereas this
commit changes the module ordering. I will open a separate PR for
it.
The involved test was nixosTests.nextcloud.basic21.
It has a testScript that is strict in nodes.nextcloud.config.system.build.vm,
in assertions about imagemagick being in the system closure.
The recursion was introduced in 329a4461a7 from
https://github.com/NixOS/nixpkgs/pull/140792
Some specialisations (such as those which affect various boot-time
attributes) cannot be switched to at runtime. This allows picking the
specialisation at boot time.
This adds a `wantedBy` clause to the user systemd service for
yubikey-agent, to ensure an enabled agent is started on boot. This
brings the behavior inline with existing documentation.
If the user has selected a graphical pinentry program, then we need to
wait for the graphical environment to exist before starting the
yubikey-agent. I've found that if we start the agent earlier it will
fail when we perform an ssh command later.
Massively reduce the time it takes running the test by building a
proper root disk image and increasing the virtualized core count to
4. This should make it much easier for the tests to pass even on
weaker systems.
With my laptop (AMD Ryzen 7 PRO 2700U) as the reference system, I see
the following test run times:
- No change:
Times out after 28 mins
- Building a root image:
7 mins, 48 secs
- Building a root image and bumping the core count:
7 mins, 17 secs
The times include the time it takes to build the image
(~1 min, 20 secs).
Massively reduce the time it takes running the test by building a
proper root disk image and increasing the virtualized core count to
4. This should make it much easier for the tests to pass even on
weaker systems.
With my laptop (AMD Ryzen 7 PRO 2700U, 32GB RAM) as the reference
system, I see the following test run times:
- No change:
25 mins, 49 secs
- Building a root image:
4 mins, 44 secs
- Building a root image and bumping the core count:
3 mins, 6 secs
The times include the time it takes to build the image (~40 secs).
Make sure the all derivations referenced by the test script are
available on the nodes. Accessing these derivations works just fine
without this change when using 9p to mount the host's store, but when
an image is built (virtualisation.buildRootImage), the dependencies
need to be copied to the image. We don't want to copy the script
itself, though, since that would trigger unnecessary image rebuilds.
pathsInNixDB isn't a very accurate name when a Nix store image is
built (virtualisation.useNixStoreImage); rename it to additionalPaths,
which should be general enough to cover both cases.
Add the `useNixStoreImage` option, allowing a disk image with the
necessary contents from the Nix store to be built using
make-disk-image.nix. The image will be mounted at `/nix/store` and
acts as a drop-in replacement for the usual 9p mounting of the host's
Nix store.
This removes the performance penalty of 9p, drastically improving
execution speed of applications which do lots of reads from the Nix
store. The caveats are increased disk space usage and image build
time.
Add a copyChannel argument which controls whether the current source
tree will be made available as a nix channel in the image or
not. Previously, it always was. Making it available is useful for
interactive use of nix utils, but changes the hash of the image when
the sources are updated.
the `nixos-rebuild` command has multiple subcommands, and not each of
them would fix the problem of a large `/boot` partition, so let’s be
more precise here.
nixos-rebuild test causes pam_mount to prompt for a password when running with
an encrypted home:
building '/nix/store/p6bflh7n5zy2dql8l45mix9qnzq65hbk-nixos-system-mildred-18.09.git.98592c5da79M.drv'...
activating the configuration...
setting up /etc...
reenter password for pam_mount:
(mount.c:68): Messages from underlying mount program:
(mount.c:72): crypt_activate_by_passphrase: File exists
(pam_mount.c:522): mount of /dev/mapper/vg0-lv_home_peter failed
kbuildsycoca5 running...
This change makes pam_mount not prompt. It still tries to remount (and fails in
the process) but that message can be ignored.
Fixes: #44586
This is done as the s3CredentialsFile specifies the environmentFile
for the systemd service, which can be used for more than just s3.
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
This module was written by @puckipedia for nixcon-video-infra 2020.
Minor changes made by @cleeyv for compat with existing jibri package.
Co-authored-by: Puck Meerburg <puck@puck.moe>
This option enables a jibri service on the same host that is running
jitsi-meet. It was written, along with the jibri module, by @puckipedia
for nixcon-video-infra 2020.
Co-authored-by: Puck Meerburg <puck@puck.moe>
* FluidSynth 1.1.11 was kept around as a dependency of some packages
that hadn't yet adjusted to API breakages. All of these packages now
use FluidSynth 2.x, so fluidsynth_1 can be removed. It has been broken
ever since glib was updated to 2.70 and was affected by an unpatched
CVE.
* Refactor expression a bit, use pname instead of name.
* Add changelog entry in case someone was using this downstream
(accidentally?).
Fixes#141508.
Fixes#124624.
The existing tests for HDFS and YARN only check if the services come up and expose their web interfaces.
The new combined hadoop test will also test whether the services and roles work together as intended.
It spin up an HDFS+YARN cluster and submit a demo YARN application that uses the hadoop cluster for
storageand yarn cluster for compute.
The version 20 of Nextcloud will be EOLed by the end of this month[1].
Since the recommended default (that didn't raise an eval-warning) on
21.05 was Nextcloud 21, this shouldn't affect too many people.
In order to ensure that nobody does a (not working) upgrade across
several major-versions of Nextcloud, I replaced the derivation of
`nextcloud20` with a `throw` that provides instructions how to proceed.
The only case that I consider "risky" is a setup upgraded from 21.05 (or
older) with a `system.stateVersion` <21.11 and with
`services.nextcloud.package` not explicitly declared in its config. To
avoid that, I also left the `else-if` for `stateVersion < 21.03` which
now sets `services.nextcloud.package` to `pkgs.nextcloud20` and thus
leads to an eval-error. This condition can be removed
as soon as 21.05 is EOL because then it's safe to assume that only
21.11. is used as stable release where no Nextcloud <=20 exists that can
lead to such an issue.
It can't be removed earlier because then every `system.stateVersion <
21.11` would lead to `nextcloud21` which is a problem if `nextcloud19`
is still used.
[1] https://docs.nextcloud.com/server/20/admin_manual/release_schedule.html
during the rewrite the checkPasswords=false feature of the old module
was lost. restore it, and with it systems that allow any client to use
any username.
borg is able to process stdin during backups when backing up the special path -,
which can be very useful for backing up things that can be streamed (eg database
dumps, zfs snapshots).
mosquitto needs a lot of attention concerning its config because it doesn't
parse it very well, often ignoring trailing parts of lines, duplicated config
keys, or just looking back way further in the file to associated config keys
with previously defined items than might be expected.
this replaces the mosquitto module completely. we now have a hierarchical config
that flattens out to the mosquitto format (hopefully) without introducing spooky
action at a distance.
/etc/crypttab can contain the _netdev option, which adds crypto devices
to the remote-cryptsetup.target.
remote-cryptsetup.target has a dependency on cryptsetup-pre.target. So
let's add both of them.
Currently, one needs to manually ssh in and invoke `systemctl start
systemd-cryptsetup@<name>.service` to unlock volumes.
After this change, systemd will properly add it to the target, and
assuming remote-cryptsetup.target is pulled in somewhere, you can simply
pass the passphrase by invoking `systemd-tty-ask-password-agent` after
ssh-ing in, without having to manually start these services.
Whether remote-cryptsetup.target should be added to multi-user.target
(as it is on other distros) is part of another discussion - right now
the following snippet will do:
```
systemd.targets.multi-user.wants = [ "remote-cryptsetup.target" ];
```
Makes service more customizeable and makes debuggingin easier through
the use of flags like `--log-debug` or `--dump-settings`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
The headless option broke with 7d8b303e3f
because the path /bin/vmware-user-suid-wrapper does not exist in the
headless variant of the open-vm-tools package.
Since the vmblock fuse mount and vmware-user-suid-wrapper seem to only
be used for shared folders and drag and drop, they should not exist in
the vmware-guest module if it is configured as headless.
The current implementation just forks off a thread to read
QEMU's stdout and lets it exist forever. This, however,
makes the interpreter shutdown racy, as the thread could
still be running and writing out buffered stdout when the
main thread exits (and since it's using the low level API,
the worker thread does not get cleaned up by the atexit hooks
installed by `threading`, either). So, instead of doing that,
let's create a real `threading.Thread` object, and also
explicitly `join` it along with the other stuff when cleaning up.
Move all `virtualisation.libvirtd.qemu*` options to a
`virtualisation.libvirtd.qemu` submodule.
Also for consistency, add `virtualisation.libvirtd.qemu.swtpm.package`
(only new option during this refactor.)
