Before this commit there was no way to access (boot into) specialisation of previous generations from grub,even tho they are there.
This commit will add grub submenu for each generation if the generation has any specialisation.
Which will allow you to boot into them.
Co-authored-by: Samuel Dionne-Riel <samuel@dionne-riel.com>
There is only other `with` with a somewhat broad scope, `with pkgs`, but
it's used in a place where it would become awkward to change out. And
anyway its scope is rather limited still.
With a limited testing of all packaged GRUB 2 themes (pkgs.nixos-grub2-theme)
this is tested to work.
Without this change, the theme loading will error out (waiting for a key press).
With this change, the theme loads and works as expected.
The intent was to not pass the flag when installing as removable. In
reality there is a third case, where you may not want to touch EFI
variables, and not want to install as removable.
In that case, it would install to the generic \EFI\grub\grubx64.efi,
which is not a good choice in any cases. The operating system should
"own" their path under \EFI\ to be a good citizen [citation needed].
With this change, there can be only two paths GRUB can be installed to:
- \EFI\NixOS-boot\grubx64.efi
- \EFI\BOOT\bootx64.efi
This removes the surprising behaviour where GRUB may be installed to a
different location only because we configured NixOS not to touch EFI
variables.
It may be necessary under some configurations to install GRUB without
touching EFI variables, but to the NixOS-owned location.
The whole option set was recommended against since mid-2019, and never
worked with the Raspberry Pi 4 family of devices.
We should have deprecated it in early 2020 for removal by 2021. At the
time I did not feel confident in making such a decision, and never
ended-up getting around to it.
The ***only*** supported-by-NixOS boot methods for AArch64 are
standards-based boot methods, namely UEFI or the pragmatically
almost-standard extlinux-compatible for U-Boot.
You can quote me on that.
Without this change, GRUB installation on non-PC systems (such as
aarch64-linux) only works if boot.loader.grub.devices is set to exactly
`["nodev"]`. If boot.loader.grub.devices was any other value (including
the default `[]`), users got the error:
Died at /nix/store/an9ngv2vg95bdcy0ifsxlbkasprm4dcw-install-grub.pl line 586.
install-grub.pl verifies that if both $grub and $grubEfi are set, then
$grubTarget (e.g. i386-pc) and $grubTargetEfi (e.g. x86_64-efi) must
both be set, or the script will `die`. On non-PC systems, $grubTarget
is "".
When boot.loader.grub.devices is ["nodev"], $grub is set to null,
disabling non-EFI installation. But if a user has devices set for an
x86_64 config, or is using only mirroredBoots without setting devices,
they will hit this `die`.
This change sets $grub to "" if $grubTarget is "".
In order to fix
https://github.com/NixOS/nixpkgs/issues/114552 (profile name with
special characters), all OSError have been ignored while only the OSError
with errno 22 (invalid argument) could has been ignored.
The drawback of ignoring all OSError is that the "No space left on
device" error is also ignored. When the /boot doesn't have enough
available disk space, the switch-to-configuration script succeeds
while the boot menu has not been updated: the user thinks it's system
has been updated, but on the next reboot it is actually rollbacked.
this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh
since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).
also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
Previously, secrets were named according to the initrd they were
associated with. This created a problem: If secrets were changed whilst
the initrd remained the same, there were two versions of the secrets
with one initrd. The result was that only one version of the secrets would
by recorded into the /boot partition and get used. AFAICT this would
only be the oldest version of the secrets for the given initrd version.
This manifests as #114594, which I found frustrating while trying to use
initrd secrets for the first time. While developing the secrets I found
I could not get new versions of the secrets to take effect.
Additionally, it's a nasty issue to run into if you had cause to change
the initrd secrets for credential rotation, etc, if you change them and
discover you cannot, or alternatively that you can't roll back as you
would expect.
Additional changes in this patch.
* Add a regression test that switching to another grub configuration
with the alternate secrets works. This test relies on the fact that it
is not changing the initrd. I have checked that the test fails if I
undo my change.
* Persist the useBootLoader disk state, similarly to other boot state.
* I had to do this, otherwise I could not find a route to testing the
alternate boot configuration. I did attempt a few different ways of
testing this, including directly running install-grub.pl, but what
I've settled on is most like what a user would do and avoids
depending on lots of internal details.
* Making tests that test the boot are a bit tricky (see hibernate.nix
and installer.nix for inspiration), I found that in addition to
having to copy quite a bit of code I still couldn't get things to
work as desired since the bootloader state was being clobbered.
My change to persist the useBootLoader state could break things,
conceptually. I need some help here discovering if that is the case,
possibly by letting this run through a staging CI if there is one.
Fix#114594.
cc potential reviewers:
@lopsided98 (original implementer) @joachifm (original reviewer),
@wkennington (numerous fixes to grub-install.pl), @lheckemann (wrote
original secrets test).
On some systems, EFI variables are not supported or otherwise wonky.
bootctl attempting to access them causes failures during bootloader
installations and updates. For such systems, NixOS provides the options
`boot.loader.efi.canTouchEfiVariables` and
`boot.loader.systemd-boot.graceful` which pass flags to bootctl that
change whether and how EFI variables are accessed.
Previously, these flags were only passed to bootctl during an install
operation. However, they also apply during an update operation, which
can cause the same sorts of errors. This change passes the flags during
update operations as well to prevent those errors.
Fixes https://github.com/NixOS/nixpkgs/issues/151336
The build of initrd-secrets can routinely fail for old boot entries
if the secrets have been removed or renamed in a later generation.
This always happens for generation 1, because it's built from the
NixOS installer and the paths differs by the mount point (i.e. /mnt).
The error is very confusing because it fails to mention it's about
an older generation and that it's somewhat harmless.
This commit turns the error into a warning for all generations but the
current, adds the name of the failed entry to the message and a note
explaining why it can happen.
apparently pandoc has changed behavior over the past releases, so the
files are no longer in sync. occasionally this requires edits
to the markdown source to not remove an anchor that was there
before (albeit wth a very questionable id), or where things were simply
being misrendered due to syntax errors.
That version has a regression that leaves some machines unbootable.
While we wait for the fix (252.2) to land in master, this is a workaround that
should save people some pain.
Before this patch, the entry match condition always fails, causing all
entries being removed. The error is not noticed because later they are
re-generated.
Before this patch, the gen_number found by regex contains
"-specialisation-foo" if specialisation is used. As a result, applying
int() to gen_number raises ValueError, causing entries containing
a specialisation part not being removed.
mostly no rendering changes. some lists (like simplelist) don't have an
exact translation to markdown, so we use a comma-separated list of
literals instead.
our xslt already replaces double line breaks with a paragraph close and
reopen. not using explicit para tags lets nix-doc-munge convert more
descriptions losslessly.
only whitespace changes to generated documents, except for two
strongswan options gaining paragraph two breaks they arguably should've
had anyway.
the conversion procedure is simple:
- find all things that look like options, ie calls to either `mkOption`
or `lib.mkOption` that take an attrset. remember the attrset as the
option
- for all options, find a `description` attribute who's value is not a
call to `mdDoc` or `lib.mdDoc`
- textually convert the entire value of the attribute to MD with a few
simple regexes (the set from mdize-module.sh)
- if the change produced a change in the manual output, discard
- if the change kept the manual unchanged, add some text to the
description to make sure we've actually found an option. if the
manual changes this time, keep the converted description
this procedure converts 80% of nixos options to markdown. around 2000
options remain to be inspected, but most of those fail the "does not
change the manual output check": currently the MD conversion process
does not faithfully convert docbook tags like <code> and <package>, so
any option using such tags will not be converted at all.
Handling of the string length condition in should_update
was broken, as evident with the log message
> leaving systemd-boot 246 in place (250.4 is not newer)
Discussion with @mweinelt came to the conclusion
that Python's "<" operator already does what we need,
so the should_update function can be dropped.
Fixes a30de3b849
Since, 4ddc78818e systemd-boot-builder
is broken in two ways:
* if no systemd-boot is currently installed *and* the NIXOS_INSTALL_BOOTLOADER
env variable is not set, it will try to run "bootctl update", which will fail
* if the currently installed systemd-boot version is newer than the version
we're about to install, it will also try to run "bootctl update", which will fail
This patch changes the behaviour,
* for the first case to still fail, but not even bother to try running
"bootctl update" and instead erroring out with an exception
* for the second case to leave the newer version in place, restoring
the pre - 4ddc78818e behaviour
To do the proper version check a new "should_update" helper function was introduced,
mimicing the compare_product C function from bootctl. If the following systemd
issue gets resolved, we would have a nice way to get rid of this function:
> https://github.com/systemd/systemd/issues/23450
This change allows to again switch to a different NixOS configuration which contains
an older systemd-boot.
Co-authored-by: Martin Weinelt <mweinelt@users.noreply.github.com>
People running nixos-install in non-NixOS environments
occasionally run into the mktemp builtin not being loaded
into bash (yes, even NixOS' bash). Rather than try and
figure out why exactly that is happening, just use a known
good mktemp from coreutils.
The regexp was only matching numbers and not the '.', so everyone using
systemd-boot would always see `could not find any previously installed
systemd-boot` on a `nixos-rebuild`.
Some specialisations (such as those which affect various boot-time
attributes) cannot be switched to at runtime. This allows picking the
specialisation at boot time.
On some systems bootctl cannot write the `LoaderSystemToken` EFI variable
during installation, which results in a failure to install the boot
loader. Upstream provides a flag (--graceful) to ignore such write failures -
this change exposes it as a configuration option.
As the exact semantics of this option appear to be somewhat volatile it
should be used only if systemd-boot otherwise fails to install.
os.readlink only resolves one layer of symlinks. This script explicitly relies on the real path of a file for deduplication, hence symlink resolution should recurse.
Since 03eaa48 added perl.withPackages, there is a canonical way to
create a perl interpreter from a list of libraries, for use in script
shebangs or generic build inputs. This method is declarative (what we
are doing is clear), produces short shebangs[1] and needs not to wrap
existing scripts.
Unfortunately there are a few exceptions that I've found:
1. Scripts that are calling perl with the -T switch. This makes perl
ignore PERL5LIB, which is what perl.withPackages is using to inform
the interpreter of the library paths.
2. Perl packages that depends on libraries in their own path. This
is not possible because perl.withPackages works at build time. The
workaround is to add `-I $out/${perl.libPrefix}` to the shebang.
In all other cases I propose to switch to perl.withPackages.
[1]: https://lwn.net/Articles/779997/
Catch and ignore errors during writing of the boot entries. These
errors could stem from profile names that are not valid filenames on
vfat filesystems.
fixes#114552
It was introduced in c10fe14 but removed in c4f910f.
It remained such that people with older generations in their boot
entries could still boot those. Given that the parameter hasn't had any
use in quite some years, it seems safe to remove now.
Fixes#60184
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
This is a temporary fix for #97433. A more proper fix has been
implemented upstream in systemd/systemd#17001, however until it gets
backported, we are stuck with ignoring the error.
After the backport lands, this commit should be reverted.
For UEFI setups, "device" will generally be the special value "nodev"
which represents not running grub-install at all. Using "nodev" for
boot mirrors should therefore be allowed.
The commit enforces buildPackages in the builder but neglects
the fact that the builder is intended to run on the target system.
Because of that, the builder will fail when remotely building a
configuration eg. with nixops or nix-copy-closure.
This reverts commit a6ac6d00f9.
Fixes error
Can't use an undefined value as an ARRAY reference at /nix/store/...-install-grub.pl line 642, <FILE> line 5.
with `/boot/grub/state` being:
```
grub
2.04
no
/dev/sda
/boot
```
I am not sure where the trailing empty line can come from; the script does not
seem to write it. In any case, now we handle that situation as well.
Further, ensure that `extraGrubInstallArgs` defaults to the empty array
if its key is not present in the `jsonState`.
For example, turns the error
cannot copy /nix/store/g24xsmmsz46hzi6whv7qwwn17myn3jfq-grub-2.04/share/grub/unicode.pf2 to /boot
into the more useful
cannot copy /nix/store/g24xsmmsz46hzi6whv7qwwn17myn3jfq-grub-2.04/share/grub/unicode.pf2 to /boot: Read-only file system
Useful for when you need to build grub modules into your grub kernel
to get a working boot, as shown in the added example.
To store this new value, we switch to more structural JSON approach.
Using one line per value to store in `/boot/grub/state` gets really messy when
the values are arrays, or even worse, can contain newlines (escaping would be
needed). Further, removing a value from the file would get extra messy
(empty lines we'd have to keep for backwards compatibility).
Thus, from now on we use JSON to store all values we'll need in the future.
This can be used to explicitly specify a specific dtb file, relative to
the dtb base.
Update the generic-extlinux-compatible module to make use of this option.
Some bootloaders might not properly detect the model.
If the specific model is known by configuration, provide a way to
explicitly point to a specific dtb in the extlinux.conf.
This option exposes the builder command used to populate an image,
honoring all options except the -c <path-to-default-configuration>
argument.
Useful to have for sdImage.populateRootCommands.
Special care needs to be taken w.r.t cross - the populate command runs
on the host platform, the activation script on the build platform (so
the builders differ)
The way this ends up being called with the raspberry pi 4 image builder
ends up not using the `-e` from the shebang.
In turn, the builds fails during cross-compilation. The wrong coreutils
ends up being used, but this is not made apparent.
The issue I faced is already fixed on master, but this ensures no one
ends up with a failed build "succeeding".
Memtest86+ doesn't support EFI, so unfree Memtest86 is used when EFI
support is enabled (systemd-boot currently also uses Memtest86 when
memtest is enabled).
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
