Warnings and descriptions for `virtualisation.docker.enableNvidia` and
`virtualisation.podman.enableNvidia` point erroneously to set
`virtualisation.containers.cdi.dynamic.nvidia.enable`. This NixOS
option has been deprecated and the recommended NixOS option is
`hardware.nvidia-container-toolkit.enable`.
Summary of this change:
- Simplify code.
- Stop a disk image from being cached in the binary cache.
- Make erofs Nix Store image build in an acceptable time outside of
testing environments (like `darwin.builder`).
- Do not regress on performance for tests that use many store paths in
their Nix store image.
- Slightly longer startup time for tests where not many store paths are
included in the image (these probably shouldn't use `useNixStoreImage`
anyways).
- Slightly longer startup time when inputs of VM do not change because
the Nix store image is not cached anymore.
Remove the `storeImage` built with make-disk-image.nix. This produced a
separate derivation which is then cached in the binary cache. These
types of images should be avoided because they gunk up the cache as they
change frequently. Now all Nix store images, whether read-only or
writable are based on the erofs image previously only used for read-only
images.
Additionally, simplify the way the erofs image is built by copying the
paths to include to a separate directory and build the erofs image from
there.
Before this change, the list of Nix store paths to include in the Nix
store image was converted to a complex regex that *excludes* all other
paths from a potentially large Nix store.
This previous approach suffers from two issues:
1. The regex is complex and, as admitted in the source code of the
includes-to-excludes.py script, most likely contains at least one
error. This means that it's unlikely that anyone will touch this
piece of software again.
2. When the Nix store image is built from a large Nix store (like when
you build the VM script to run outside of any testing context) this
regex becomes painfully slow. There is at least one prominent
use-case where this matters: `darwin.builder`.
Benchmarking impressions:
- Building Nix store via make-disk-image.nix takes ~25s
- Building Nix store as an erofs image takes ~4s
- Running nixosTests.qemu-vm-writable-store-image takes ~10s when
building the erofs image with the regex vs ~14s when building by
copying to a temporary directory.
- nixosTests.gitlab which had the biggest gains from the initial erofs
change takes the same time as before.
- On a host with ~140k paths in /nix/store, building the erofs image
with the regex takes 410s as opposed to 6s when copying to a temporary
directory.
When `services.resolved` is enabled, then `resolve [!UNAVAIL=return]`
is added to `system.nssDatabases.hosts` with priority 501,
which prevents lower-priority NSS modules from running
unless systemd-resolved is not available.
Quoting from `man nss-resolve`:
> To activate the NSS module, add "resolve [!UNAVAIL=return]" to the line
> starting with "hosts:" in /etc/nsswitch.conf. Specifically, it is
> recommended to place "resolve" early in /etc/nsswitch.conf's "hosts:"
> line. It should be before the "files" entry, since systemd-resolved
> supports /etc/hosts internally, but with caching. To the contrary, it
> should be after "mymachines", to give hostnames given to local VMs and
> containers precedence over names received over DNS. Finally, we
> recommend placing "dns" somewhere after "resolve", to fall back to
> nss-dns if systemd-resolved.service is not available.
Note that the man page (just) recommends "early" and means with this
"before the 'files' and 'dns' entries". It does not insist on being
first or excluding other modules.
For this reason, libvirt NSS modules should run before the `resolve`
module. They should come right next to `mymachines` because both are
conceptually very similar -- they resolve local VMs/containers.
Since the data source of the libvirt NSS modules are local
plain text files (see source code of the libvirt NSS module),
no performance impact is expected form this raise of priorities.
Other NSS modules in NixOS also explicitly set their priority, which is
why this change increases consistency.
Fixes#322022
The recent move to strip out DSS support from the openssh package
means that older key formats cause the key-printing command to
fail. Rather than causing the entire unit to fail, we should instead
skip those keys - while still letting the error through to the
console - and continue to print other keys the loop may find.
diskSize defaults to the previous hard-coded 8192:
no change for existing users.
Users can set diskSize when building images which require
larger disk space; thus avoiding the error:
ERROR: cptofs failed. diskSize might be too small for closure.
Signed-off-by: Sirio Balmelli <sirio@b-ad.ch>
Co-authored-by: superherointj <5861043+superherointj@users.noreply.github.com>
- rename hardware.opengl to hardware.graphics
- remove hardware.opengl.driSupport, which does nothing
- remove hardware.opengl.setLdLibraryPath, which should never be done
- rename hardware.opengl.driSupport32Bit to hardware.graphics.enable32Bit
- lost of small docs / formatting cleanups
This is a feature supported out of the box by upstream and allows the
incusd service to be restarted without impacting running
instances. While this does give up a bit of reproducibility, qemu and
lxc for example, there are clear benefits in allowing the host to
apply updates without impacting instances.
Modeled after the zabbly implementation: 2a67c3e260/systemd/incus-startup.service
This will now be the default.
This avoids having to use workarounds like the following to retain the
default settings which podman requires to run.
virtualisation.containers.storage.settings = lib.recursiveUpdate options.virtualisation.containers.storage.settings.default {
storage.options.mount_program = lib.getExe pkgs.fuse-overlayfs;
};
I manually audited all `RunCommand` and `exec.LookPath` calls in the incus repo, combined with the following information
/run/wrappers/bin
lxc usable-cub 20240427123718.368 WARN idmap_utils - ../src/lxc/idmap_utils.c:lxc_map_ids:165 - newuidmap binary is missing
iw
lxc 20240427123830.358 ERROR network - ../src/lxc/network.c:lxc_netdev_move_wlan:1679 - Couldn't find the application iw in PATH
minio-client
https://github.com/lxc/incus/pull/777
ceph-client
Added, but could be missing bits to actually work
May need full ceph package for `radosgw-admin` for object storage?
Add the NixOS option `hardware.nvidia-container-toolkit-cdi-generator.enable`.
This enables the ability to expose GPU's in containers for container
runtimes that support the Container Device Interface (CDI)
Remove `cdi.static` and `cdi.dynamic.nvidia.enable` attributes.
Was previously broken due to a missing runtime dependency.
> Error: Failed to start device "vtpm": Failed to validate environment: Required tool 'swtpm' is missing
This reverts parts of commit d87c4e1a72 from @Mic92
After switching from nixos-23.11 to nixos-unstable, I got the following error:
```
$ nixos-rebuild switch --flake nixos/#digitalocean
...
error:
Failed assertions:
- You must set the option ‘boot.loader.grub.devices’ or 'boot.loader.grub.mirroredBoots' to make the system bootable.
```
In my nixos config I don't set boot.loader.grub.devices or boot.loader.grub.mirroredBoots explicitly.
It seems like the values are not really defaulted via mkDefault? I am still relatively new to Nix, not sure how to fix this properly, so I am proposing to revert this change.
these changes were generated with nixq 0.0.2, by running
nixq ">> lib.mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> Inherit >> mdDoc[remove]" --batchmode nixos/**.nix
two mentions of the mdDoc function remain in nixos/, both of which
are inside of comments.
Since lib.mdDoc is already defined as just id, this commit is a no-op as
far as Nix (and the built manual) is concerned.
- use normal VM nodes for target, with some extra trickery
- rename preBootCommands to postBootCommands to match its actual intent
- rename VMs to installer and target, so they're not all called machine
- set platforms on non-UEFI tests properly
- add missing packages for systemd-boot test
- fix initrd secrets leaking into the store and having wrong paths
- Build virtualbox guest additions from source and fix paths
- Install VBoxDRMClient to support resizing
- Support resizing on wayland and x11
- Adding multiple new options
- clipboard
- seamless
- Removing x11 option
- Support linux 6.8
If the user provides a `fileSystems."/".device` option, it should have
higher precedence than the default value.
Same for `loader.grub.devices` (also set by disko internally).
The qemu module shouldn't implicitly (and for all architectures) enable
SSM when enabling Secure Boot.
Additionally, this breaks aarch64 Secure Boot tests because this module
doesn't use the right machine type for anything but X86.
Previously we were doing some parts like activation in the init script,
so linking to that works for non-systemd init
With boot.initrd.systemd.enable we no longer run activation in the init script,
but instead a new script named prepare-root, which is used instead.
this lets us *dis*able filesystem explicitly, as is required by e.g. the
zfs-less installer images. currently that specifically is only easily
possible by adding an overlay that stubs out `zfs`, with the obvious
side-effect of also removing tooling that could run without the kernel
module loaded.
Changed the default security model for shared directories from 'none' to
the 'mapped-xattr'. QEMU recommends using this option for its security and
reliability benefits.
This change replaces the previously hard-coded `/boot` path with a
reference to `efiSysMountPoint` and more importantly this change makes
it possible to override these rules in scenarios in which they are not
desired.
One such scenario would be when `systemd-gpt-auto-generator(8)` is used
to automount the ESP. Consider this section from the mentioned manpage:
> The ESP is mounted to /boot/ if that directory exists and is not used
> for XBOOTLDR, and otherwise to /efi/. Same as for /boot/, an automount
> unit is used. The mount point will be created if necessary.
Prior to this change, the ESP would be automounted under `/efi` on first
boot, then the previous tmpfiles rules caused `/boot` to be created.
Following the quote above, this meant that the ESP is mounted under
`/boot` for each subsequent boot.
Add an option allowing packages containing out-of-tree vhost-user drivers for
QEMU to be specified. The relevant configurations are then linked at runtime
where libvirt expects them.
An example use case for this is virtiofs.
* fixes#250302
* ASHMEM was removed in Linux 5.18 and waydroid can use MEMFD instead.
MEMFD is enabled by default in 4.18 and later kernels while we
already require this version for namespace support.
Add warning to the documentation of
`virtualisation.docker.storageDriver` that changing will cause any
existing containers and images to become inaccessible.
waagent's extension `Microsoft.OSTCExtensions.VMAccessForLinux` requires Python, otherwise it would be failed to install with the following error message in `/var/log/waagent.log`:
```
No Python interpreter found on the box
```
waagent's extension `Microsoft.CPlat.Core.RunCommandLinux` needs lsof, otherwise it would produce the following error message in `/var/log/waagent.log`:
```
/var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.10/bin/custom-script-shim: line 60: lsof: command not found
```
The virtualisation.directBoot.initrd option was added for netboot
images, but the assertion to check directBoot enabled if it was used
caused an infinite recursion if it was. Minimal reproduction:
import nixos/tests/make-test-python.nix ({ pkgs, ... }: {
name = "";
nodes = {
machine = { config, ...}: {
imports = [ nixos/modules/installer/netboot/netboot-minimal.nix ];
virtualisation.directBoot = {
enable = true;
initrd = "${config.system.build.netbootRamdisk}/${config.system.boot.loader.initrdFile}";
};
};
};
testScript = "";
}) {}
The fix is to swap the two conditions, so that cfg.directBoot.enable
is checked first, and the initrd comparision will be short circuited.
This wasn't noticed during review because in earlier versions of the
virtualisation.directBoot patch, the assertion was accidentally in the
conditional above, so wasn't evaluated unless port forwarding was in
use.
The idea is to run an async process waiting for swtpm
and we have to ensure that `FD_CLOEXEC` is cleared on this process'
stdin file descriptor, we use `fdflags` for this, a loadable builtin in
Bash ≥ 5.
The async process when exited will terminate `swtpm`, we bind the
termination of the async process to the termination of QEMU by virtue of
having `qemu` exec in that Bash script.
Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
Co-authored-by: Raito Bezarius <masterancpp@gmail.com>
Recent change to nixos-rebuild (https://github.com/NixOS/nixpkgs/pull/258571)
adds systemd-run, which brings with it a cleaner environment
(ie $PATH not available).
Workaround: use absolute path for ln to avoid command-not-found error
This commit makes auto-restarting declarative containers optional.
This is useful when you don't want changes to your config to automatically
restart the container and thus stop any applications running inside it.
Allow the user to disable overriding the fileSystems option with
virtualisation.fileSystems by setting
`virtualisation.fileSystems = lib.mkForce { };`.
With this change you can use the qemu-vm module to boot from an external
image that was not produced by the qemu-vm module itself. The user can
now re-use the modularly set fileSystems option instead of having to
reproduce it in virtualisation.fileSystems.
A couple notes:
---------------
Adding invalid `console=` parameters is not an issue. Any invalid
console is unused. The kernel will use the "rightmost" (last) valid
`console=` parameter as the default output. Thus the SBBR-mandated AMA0
on A1, and ttyS0 on x86_64 as documented by Oracle.
`nvme_core.shutdown_timeout=10` was added as it was written this way in
the A1 images. Unclear whether `nvme.shutdown_timeout=10` is wrong. At
worst this is a no-op.
This commit makes the OpenSSH option `PermitRootLogin` available to be
configured by other NixOS modules when using the Google Cloud Engine
(GCE) NixOS image builder. Other options like `PasswordAuthentication`
were already configurable, so I think it makes sense to make `PermitRootLogin`
configurable as well is order to disable it completely, for example.
NOTES:
@jakeschurch did not realize that is was already updated on master, but not
backported to 23.05 channel
Signed-off-by: Jake Schurch <jakeschurch@gmail.com>
We were setting the systemd pre-start script through the
systemd.services.<name>.preStart NixOS option. This option uses a
string containing the pre-start script as input.
In some scenarios, you want to extend this script to perform some
additional actions before launching a container.
At the moment, your only option is to mkForce the pre-start string and
rewrite a preStart script from scratch. Potentially vendoring the
Nixpkgs pre-start script in your custom pre-start script. (you can
also create a new service unit in charge of running the custom
pre-start and create a dependency link between the units, but that's
also sub-optimal).
The systemd.services.<name>.serviceConfig.ExecStartPre NixOS option
gives us a better way to extend a pre-start script. Instead of being a
simple script, this option can be a list of scripts. The NixOS module
system then merges the multiple list declarations instead of
overriding them. Meaning that if we use this ExecStartPre option, we
can trivially extend the exec-start script: just add the custom script
in the systemd service override and you're done.
ExecStartPre behaves a tiny bit differently from preStart. Instead of
expecting a string containing a script, it expects a path pointing to
a script. We take advantage of this API change to check the pre-start
script with shellCheck via the pkgs.writeShellApplication function.
This change removes the bespoke logic around identifying block devices.
Instead of trying to find the right device by iterating over
`qemu.drives` and guessing the right partition number (e.g.
/dev/vda{1,2}), devices are now identified by persistent names provided
by udev in /dev/disk/by-*.
Before this change, the root device was formatted on demand in the
initrd. However, this makes it impossible to use filesystem identifiers
to identify devices. Now, the formatting step is performed before the VM
is started. Because some tests, however, rely on this behaviour, a
utility function to replace this behaviour in added in
/nixos/tests/common/auto-format-root-device.nix.
Devices that contain neither a partition table nor a filesystem are
identified by their hardware serial number which is injecetd via QEMU
(and is thus persistent and predictable). PCI paths are not a reliably
way to identify devices because their availability and numbering depends
on the QEMU machine type.
This change makes the module more robust against changes in QEMU and the
kernel (non-persistent device naming) and by decoupling abstractions
(i.e. rootDevice, bootPartition, and bootLoaderDevice) enables further
improvement down the line.
As with many things, we have scenarios where we don't want to boot on a
disk / bootloader and also we don't want to boot directly.
Sometimes, we want to boot through an OptionROM of our NIC, e.g. netboot
scenarios or let the firmware decide something, e.g. UEFI PXE (or even
UEFI OptionROM!).
This is composed of:
- `directBoot.enable`: whether to direct boot or not
- `directBoot.initrd`: enable overriding the
`config.system.build.initialRamdisk` defaults, useful for
netbootRamdisk for example.
This makes it possible.
Adds a new option to the virtualisation modules that enables specifying explicitly named network interfaces in QEMU VMs.
The existing `virtualisation.vlans` option is still supported for cases where the name of the network interface is irrelevant.
Libvirt support calling user defined hooks on certains events.
Documentation can be found https://libvirt.org/hooks.html.
This commit allow specifying these hooks via the
virtualisation.libvirtd.hooks.<name>.* options
Calling `eval-config.nix` without a `system` from a Nix flake fails with
`error: attribute 'currentSystem' missing` since #230523. Setting
`system = null` removes the use of `currentSystem` and instead uses the
value from the `nixpkgs` module.
