This is a feature supported out of the box by upstream and allows the
incusd service to be restarted without impacting running
instances. While this does give up a bit of reproducibility, qemu and
lxc for example, there are clear benefits in allowing the host to
apply updates without impacting instances.
Modeled after the zabbly implementation: 2a67c3e260/systemd/incus-startup.service
This will now be the default.
This avoids having to use workarounds like the following to retain the
default settings which podman requires to run.
virtualisation.containers.storage.settings = lib.recursiveUpdate options.virtualisation.containers.storage.settings.default {
storage.options.mount_program = lib.getExe pkgs.fuse-overlayfs;
};
I manually audited all `RunCommand` and `exec.LookPath` calls in the incus repo, combined with the following information
/run/wrappers/bin
lxc usable-cub 20240427123718.368 WARN idmap_utils - ../src/lxc/idmap_utils.c:lxc_map_ids:165 - newuidmap binary is missing
iw
lxc 20240427123830.358 ERROR network - ../src/lxc/network.c:lxc_netdev_move_wlan:1679 - Couldn't find the application iw in PATH
minio-client
https://github.com/lxc/incus/pull/777
ceph-client
Added, but could be missing bits to actually work
May need full ceph package for `radosgw-admin` for object storage?
Add the NixOS option `hardware.nvidia-container-toolkit-cdi-generator.enable`.
This enables the ability to expose GPU's in containers for container
runtimes that support the Container Device Interface (CDI)
Remove `cdi.static` and `cdi.dynamic.nvidia.enable` attributes.
Was previously broken due to a missing runtime dependency.
> Error: Failed to start device "vtpm": Failed to validate environment: Required tool 'swtpm' is missing
This reverts parts of commit d87c4e1a72 from @Mic92
After switching from nixos-23.11 to nixos-unstable, I got the following error:
```
$ nixos-rebuild switch --flake nixos/#digitalocean
...
error:
Failed assertions:
- You must set the option ‘boot.loader.grub.devices’ or 'boot.loader.grub.mirroredBoots' to make the system bootable.
```
In my nixos config I don't set boot.loader.grub.devices or boot.loader.grub.mirroredBoots explicitly.
It seems like the values are not really defaulted via mkDefault? I am still relatively new to Nix, not sure how to fix this properly, so I am proposing to revert this change.
these changes were generated with nixq 0.0.2, by running
nixq ">> lib.mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> Inherit >> mdDoc[remove]" --batchmode nixos/**.nix
two mentions of the mdDoc function remain in nixos/, both of which
are inside of comments.
Since lib.mdDoc is already defined as just id, this commit is a no-op as
far as Nix (and the built manual) is concerned.
- use normal VM nodes for target, with some extra trickery
- rename preBootCommands to postBootCommands to match its actual intent
- rename VMs to installer and target, so they're not all called machine
- set platforms on non-UEFI tests properly
- add missing packages for systemd-boot test
- fix initrd secrets leaking into the store and having wrong paths
- Build virtualbox guest additions from source and fix paths
- Install VBoxDRMClient to support resizing
- Support resizing on wayland and x11
- Adding multiple new options
- clipboard
- seamless
- Removing x11 option
- Support linux 6.8
If the user provides a `fileSystems."/".device` option, it should have
higher precedence than the default value.
Same for `loader.grub.devices` (also set by disko internally).
The qemu module shouldn't implicitly (and for all architectures) enable
SSM when enabling Secure Boot.
Additionally, this breaks aarch64 Secure Boot tests because this module
doesn't use the right machine type for anything but X86.
Previously we were doing some parts like activation in the init script,
so linking to that works for non-systemd init
With boot.initrd.systemd.enable we no longer run activation in the init script,
but instead a new script named prepare-root, which is used instead.
this lets us *dis*able filesystem explicitly, as is required by e.g. the
zfs-less installer images. currently that specifically is only easily
possible by adding an overlay that stubs out `zfs`, with the obvious
side-effect of also removing tooling that could run without the kernel
module loaded.
Changed the default security model for shared directories from 'none' to
the 'mapped-xattr'. QEMU recommends using this option for its security and
reliability benefits.
This change replaces the previously hard-coded `/boot` path with a
reference to `efiSysMountPoint` and more importantly this change makes
it possible to override these rules in scenarios in which they are not
desired.
One such scenario would be when `systemd-gpt-auto-generator(8)` is used
to automount the ESP. Consider this section from the mentioned manpage:
> The ESP is mounted to /boot/ if that directory exists and is not used
> for XBOOTLDR, and otherwise to /efi/. Same as for /boot/, an automount
> unit is used. The mount point will be created if necessary.
Prior to this change, the ESP would be automounted under `/efi` on first
boot, then the previous tmpfiles rules caused `/boot` to be created.
Following the quote above, this meant that the ESP is mounted under
`/boot` for each subsequent boot.
Add an option allowing packages containing out-of-tree vhost-user drivers for
QEMU to be specified. The relevant configurations are then linked at runtime
where libvirt expects them.
An example use case for this is virtiofs.
* fixes#250302
* ASHMEM was removed in Linux 5.18 and waydroid can use MEMFD instead.
MEMFD is enabled by default in 4.18 and later kernels while we
already require this version for namespace support.
Add warning to the documentation of
`virtualisation.docker.storageDriver` that changing will cause any
existing containers and images to become inaccessible.
waagent's extension `Microsoft.OSTCExtensions.VMAccessForLinux` requires Python, otherwise it would be failed to install with the following error message in `/var/log/waagent.log`:
```
No Python interpreter found on the box
```
waagent's extension `Microsoft.CPlat.Core.RunCommandLinux` needs lsof, otherwise it would produce the following error message in `/var/log/waagent.log`:
```
/var/lib/waagent/Microsoft.Azure.Extensions.CustomScript-2.1.10/bin/custom-script-shim: line 60: lsof: command not found
```
The virtualisation.directBoot.initrd option was added for netboot
images, but the assertion to check directBoot enabled if it was used
caused an infinite recursion if it was. Minimal reproduction:
import nixos/tests/make-test-python.nix ({ pkgs, ... }: {
name = "";
nodes = {
machine = { config, ...}: {
imports = [ nixos/modules/installer/netboot/netboot-minimal.nix ];
virtualisation.directBoot = {
enable = true;
initrd = "${config.system.build.netbootRamdisk}/${config.system.boot.loader.initrdFile}";
};
};
};
testScript = "";
}) {}
The fix is to swap the two conditions, so that cfg.directBoot.enable
is checked first, and the initrd comparision will be short circuited.
This wasn't noticed during review because in earlier versions of the
virtualisation.directBoot patch, the assertion was accidentally in the
conditional above, so wasn't evaluated unless port forwarding was in
use.
The idea is to run an async process waiting for swtpm
and we have to ensure that `FD_CLOEXEC` is cleared on this process'
stdin file descriptor, we use `fdflags` for this, a loadable builtin in
Bash ≥ 5.
The async process when exited will terminate `swtpm`, we bind the
termination of the async process to the termination of QEMU by virtue of
having `qemu` exec in that Bash script.
Signed-off-by: Arthur Gautier <baloo@superbaloo.net>
Co-authored-by: Raito Bezarius <masterancpp@gmail.com>
Recent change to nixos-rebuild (https://github.com/NixOS/nixpkgs/pull/258571)
adds systemd-run, which brings with it a cleaner environment
(ie $PATH not available).
Workaround: use absolute path for ln to avoid command-not-found error
This commit makes auto-restarting declarative containers optional.
This is useful when you don't want changes to your config to automatically
restart the container and thus stop any applications running inside it.
Allow the user to disable overriding the fileSystems option with
virtualisation.fileSystems by setting
`virtualisation.fileSystems = lib.mkForce { };`.
With this change you can use the qemu-vm module to boot from an external
image that was not produced by the qemu-vm module itself. The user can
now re-use the modularly set fileSystems option instead of having to
reproduce it in virtualisation.fileSystems.
A couple notes:
---------------
Adding invalid `console=` parameters is not an issue. Any invalid
console is unused. The kernel will use the "rightmost" (last) valid
`console=` parameter as the default output. Thus the SBBR-mandated AMA0
on A1, and ttyS0 on x86_64 as documented by Oracle.
`nvme_core.shutdown_timeout=10` was added as it was written this way in
the A1 images. Unclear whether `nvme.shutdown_timeout=10` is wrong. At
worst this is a no-op.
This commit makes the OpenSSH option `PermitRootLogin` available to be
configured by other NixOS modules when using the Google Cloud Engine
(GCE) NixOS image builder. Other options like `PasswordAuthentication`
were already configurable, so I think it makes sense to make `PermitRootLogin`
configurable as well is order to disable it completely, for example.
NOTES:
@jakeschurch did not realize that is was already updated on master, but not
backported to 23.05 channel
Signed-off-by: Jake Schurch <jakeschurch@gmail.com>
We were setting the systemd pre-start script through the
systemd.services.<name>.preStart NixOS option. This option uses a
string containing the pre-start script as input.
In some scenarios, you want to extend this script to perform some
additional actions before launching a container.
At the moment, your only option is to mkForce the pre-start string and
rewrite a preStart script from scratch. Potentially vendoring the
Nixpkgs pre-start script in your custom pre-start script. (you can
also create a new service unit in charge of running the custom
pre-start and create a dependency link between the units, but that's
also sub-optimal).
The systemd.services.<name>.serviceConfig.ExecStartPre NixOS option
gives us a better way to extend a pre-start script. Instead of being a
simple script, this option can be a list of scripts. The NixOS module
system then merges the multiple list declarations instead of
overriding them. Meaning that if we use this ExecStartPre option, we
can trivially extend the exec-start script: just add the custom script
in the systemd service override and you're done.
ExecStartPre behaves a tiny bit differently from preStart. Instead of
expecting a string containing a script, it expects a path pointing to
a script. We take advantage of this API change to check the pre-start
script with shellCheck via the pkgs.writeShellApplication function.
This change removes the bespoke logic around identifying block devices.
Instead of trying to find the right device by iterating over
`qemu.drives` and guessing the right partition number (e.g.
/dev/vda{1,2}), devices are now identified by persistent names provided
by udev in /dev/disk/by-*.
Before this change, the root device was formatted on demand in the
initrd. However, this makes it impossible to use filesystem identifiers
to identify devices. Now, the formatting step is performed before the VM
is started. Because some tests, however, rely on this behaviour, a
utility function to replace this behaviour in added in
/nixos/tests/common/auto-format-root-device.nix.
Devices that contain neither a partition table nor a filesystem are
identified by their hardware serial number which is injecetd via QEMU
(and is thus persistent and predictable). PCI paths are not a reliably
way to identify devices because their availability and numbering depends
on the QEMU machine type.
This change makes the module more robust against changes in QEMU and the
kernel (non-persistent device naming) and by decoupling abstractions
(i.e. rootDevice, bootPartition, and bootLoaderDevice) enables further
improvement down the line.
As with many things, we have scenarios where we don't want to boot on a
disk / bootloader and also we don't want to boot directly.
Sometimes, we want to boot through an OptionROM of our NIC, e.g. netboot
scenarios or let the firmware decide something, e.g. UEFI PXE (or even
UEFI OptionROM!).
This is composed of:
- `directBoot.enable`: whether to direct boot or not
- `directBoot.initrd`: enable overriding the
`config.system.build.initialRamdisk` defaults, useful for
netbootRamdisk for example.
This makes it possible.
Adds a new option to the virtualisation modules that enables specifying explicitly named network interfaces in QEMU VMs.
The existing `virtualisation.vlans` option is still supported for cases where the name of the network interface is irrelevant.
Libvirt support calling user defined hooks on certains events.
Documentation can be found https://libvirt.org/hooks.html.
This commit allow specifying these hooks via the
virtualisation.libvirtd.hooks.<name>.* options
Calling `eval-config.nix` without a `system` from a Nix flake fails with
`error: attribute 'currentSystem' missing` since #230523. Setting
`system = null` removes the use of `currentSystem` and instead uses the
value from the `nixpkgs` module.
Context summary:
'vma create' can't otherwise write to tmpfs such as /dev/shm.
This is important when used from non-nixos machines which may
have /build as tmpfs.
VMA is Proxmox's virtual machine image format that wraps QEMU images,
augmenting these with proxmox-specific configuration file.
proxmox-image.nix uses the VMA tool to create vma image files.
The VMA tool exists as a patchset ontop of QEMU.
VMA writes its output with open() and O_DIRECT flag.
O_DIRECT does not work on Linux tmpfs [1]. Thus:
$ vma create ~/output.vma ... # works, assuming home isn't tmpfs.
$ vma create /dev/shm/output.vma ... # fails since /dev/shm is tmpfs
Failure results in assert(*errp == NULL).
O_DIRECT is a cache performance hint.
But it currently blocks our usage of nixos-generate -f proxmox from
Non-NixOS hosts and Docker.
The patch here simply removes O_DIRECT:
vma-writer.c later performs memalign due to O_DIRECT, but this is
safe to do with or without O_DIRECT.
Ideally, this should be fixed in upstream Proxmox: Perhaps by falling
back to open without O_DIRECT.
Another attempt to fix this SIGABRT is [2], which writes the vma file
directory to $out/ folder -- however that may still be tmpfs mounted
which it is in our case.
[1] https://lore.kernel.org/lkml/45A29EC2.8020502@tmr.com/t/
[2] https://github.com/NixOS/nixpkgs/pull/224282
`useEFIBoot` is somewhat misleading, but we should make it possible to
enable UEFI environment / firmware without buying into a bootloader.
This makes it possible.
Previously, it was possible to run with a tmpfs / with
`virtualisation.diskImage = null;`, this was likely broken by my changes
in 4b4e4c3ef9.
It is reintroduced by disabling properly the bootloader for now, as it
is complicated to make it work with.
Now that `useBootLoader` produces a full system image, moving disk
images can be slow because they have a full Nix store in them.
It does not make sense to keep the 9p mountpoint to shadow the
/nix/store of the VM.
We disable it if we have `useBootLoader` and introduce an option for
easy overrides.
This option has been introduced in 678eed323f without realizing there was this
PR inflight, unfortunately, it collide with what this PR does and make
it irrelevant.
Therefore, I remove it here.
trying to get all of the podman functionality to work with the wrapper
is becoming more complicated with each release, it isn't sustainable
removing the wrapper does mean that using extraPackages will need to build from source
- remove unnecessary serviceConfig overrides
- set HELPER_BINARIES_DIR to libexec/podman
- use install.bin target on linux for podman/tmpfiles
- also installs quadlet/rootlessport in libexec
- symlink binaries from helpersBin into HELPER_BINARIES_DIR
- remove unnecessary rootlessport output
- remove unnecessary substituteInPlace
The typo creates an empty directory 0755 in initrd rootfs rather than
create the Nix store directories with mode 0755.
I guess setting the mode is not strictly necessary if it worked before
this change, but I'll leave the `-m 0755` in just in case.
trying to get all of the podman functionality to work with the wrapper
is becoming more complicated with each release, it isn't sustainable
removing the wrapper does mean that using extraPackages will need to build from source
- include pkgs.zfs by default in the wrapped podman used by the module so it is cached
- anyone using zfsUnstable will need to build from source
- remove unnecessary serviceConfig overrides
- set HELPER_BINARIES_DIR during build
- use install.bin target on linux for podman/tmpfiles
- also installs quadlet/rootlessport in libexec
- remove unnecessary rootlessport output
- remove unnecessary substituteInPlace
This is because vSphere version 6.7.0.51000 errors with
Issues detected with selected template. Details: -
78:7:VALUE_ILLEGAL: Value ''3'' of Parent element does not refer
to a ref of type DiskControllerReference.
when using SATA.
...for explicitly named network interfaces
This reverts commit 6ae3e7695e.
(and evaluation fixups 08d26bbb727aed90a969)
Some of the tests fail or time out after the merge.
Adds a new option to the virtualisation modules that enables specifying
explicitly named network interfaces in QEMU VMs. The existing
`virtualisation.vlans` is still supported for cases where the name of
the network interface is irrelevant.
Previously, secrets were named according to the initrd they were
associated with. This created a problem: If secrets were changed whilst
the initrd remained the same, there were two versions of the secrets
with one initrd. The result was that only one version of the secrets would
by recorded into the /boot partition and get used. AFAICT this would
only be the oldest version of the secrets for the given initrd version.
This manifests as #114594, which I found frustrating while trying to use
initrd secrets for the first time. While developing the secrets I found
I could not get new versions of the secrets to take effect.
Additionally, it's a nasty issue to run into if you had cause to change
the initrd secrets for credential rotation, etc, if you change them and
discover you cannot, or alternatively that you can't roll back as you
would expect.
Additional changes in this patch.
* Add a regression test that switching to another grub configuration
with the alternate secrets works. This test relies on the fact that it
is not changing the initrd. I have checked that the test fails if I
undo my change.
* Persist the useBootLoader disk state, similarly to other boot state.
* I had to do this, otherwise I could not find a route to testing the
alternate boot configuration. I did attempt a few different ways of
testing this, including directly running install-grub.pl, but what
I've settled on is most like what a user would do and avoids
depending on lots of internal details.
* Making tests that test the boot are a bit tricky (see hibernate.nix
and installer.nix for inspiration), I found that in addition to
having to copy quite a bit of code I still couldn't get things to
work as desired since the bootloader state was being clobbered.
My change to persist the useBootLoader state could break things,
conceptually. I need some help here discovering if that is the case,
possibly by letting this run through a staging CI if there is one.
Fix#114594.
cc potential reviewers:
@lopsided98 (original implementer) @joachifm (original reviewer),
@wkennington (numerous fixes to grub-install.pl), @lheckemann (wrote
original secrets test).
The aarch64-linux kernel and initrd recently eclipsed 60M, causing the
boot disk image build to run out of space and fail. Double the size of
the image to 120M to fix the issue.
The disk image is stored in expandable qcow2 format, so only the space
actually used by files in the image is consumed. Therefore, other
architectures are not unfairly penalized, and the output size does not
suddenly double.
This also fixes NixOS tests which use this option, like systemd-boot's.
The agent has not been updated for a very long time. In addition to
updating to the newest tagged version the change creates a package for
it.
The existing version has issues with the new python2.7 package not
containing crypt.so file. And the commit
6910a4eea0 I believe introduced
regression that caused the shebang to not be updated.
This adds a new ``parallelShutdown`` option that allows users to control
how many guests can be shut down concurrently. Allowing multiple virtual
machines to be shut down at the same time reduces the amount of time it
takes to reboot the host.
Upstream documentation: https://www.libvirt.org/manpages/libvirt-guests.html#files
This fixes `lxd init`, which previously failed like this:
$ yes "" | lxd init
[...]
Error: Failed to create storage pool "default": Failed to run: losetup --find --nooverlap --direct-io=on --show /var/lib/lxd/disks/default.img: exec: "losetup": executable file not found in $PATH
