The concept of having a "minimum supported Nix version" doesn't work
anymore today, for the following reasons:
- With multiple forks / implementations of Nix available, their feature
sets and versions will differ. We'd need *multiple* minimum versions,
one for each implementation.
- Lix does not expose its real version. It only reports "2.18.3-lix",
even though its real version is in the 2.90+ range.
- A minimum version has the expectation that it could be *raised* in the
future. That's not possible with Lix, because Lix will always and
forever report the above version.
- A minimum version has the expectation that *all* versions bigger than
the minimum are supported. That was already quite a stretch when minver
was 2.3 and none of the Nix versions between 2.4 and 2.23 were packed
anymore. But it's impossible for us to test all these non-LTS versions
anyway: We don't have Nix 2.18, 2.19, 2.20, 2.21, 2.22, 2.23, 2.25, 2.26
and 2.27 available in Nixpkgs at the time of this writing.
With their policy around `builtins.nixVersion`, Lix forces our hand: We
need to replace minver.nix with a "feature detection" mechanism.
This PR introduces the first two features:
- The availability of `builtins.nixVersion`: If this is not available,
the version of Nix is so old, that we surely don't support it anymore.
- The value of `builtins.nixVersion` being greater or equal to 2.18.
Note, that this does **not** imply support for Nix 2.18. Instead,
explicitly supported versions of Lix and Nix are only these that we
actually test against.
If, eventually, we realize that the supported versions have advanced and
Nixpkgs has adopted a feature only available in newer versions, we will
have to add a feature check for this.
Put differently: The list of features in `minfeatures.nix` is not
expected to be complete. It's a list of known-to-be-bad conditions that
will cause problems when evaluating Nixpkgs. Their only purpose is to be
able to show a helpful error message. Some other versions might also not
be supported, but might fail with more subtle errors. That's just
reality and has always been the case previously as well.
This doesn't do the right thing here, because it evaluates
the test with nix that is evaluating the `nixpkgs-lib-tests-nix-${nix.version}`
derivation, not the Nix/Lix under test. This was just really busted for a long
time.
`fromHexString` is backed by `builtins.fromTOML`. Per [the TOML
v1.0.0 specification]:
> Arbitrary 64-bit signed integers (from −2^63 to 2^63−1) should be
> accepted and handled losslessly. If an integer cannot be represented
> losslessly, an error must be thrown.
[the TOML v1.0.0 specification]: <https://toml.io/en/v1.0.0#integer>
The saturating behaviour of the toml11 version currently used
by Nix is not lossless, and is therefore a violation of the TOML
specification. We should not be relying on it. This blocks the update
of toml11, as it became stricter about reporting this condition.
This, yes, is arguably an evaluation compatibility break. However,
integer overflow was recently explicitly defined as an error by
both Nix and Lix, as opposed to the C++ undefined behaviour it was
previously implemented as:
* <https://nix.dev/manual/nix/stable/release-notes/rl-2.25>
* <https://docs.lix.systems/manual/lix/stable/release-notes/rl-2.91.html#fixes>
This included changing `builtins.fromJSON` to explicitly
reject overflowing integer literals. I believe that the case for
`builtins.fromTOML` is comparable, and that we are effectively testing
undefined behaviour in TOML and the Nix language here, in the same way
that we would have been if we had tests relying on overflowing integer
arithmetic. I am not aware of any use of this behaviour outside of
these tests; the reverted toml11 bump in Nix did not break the 23.11
evaluation regression test, for example.
C++ undefined behaviour is not involved here, as toml11 used the C++
formatted input functions that are specified to saturate on invalid
values. But it’s still a violation of the TOML specification caused
by insufficient error checking in the old version of the library,
and inconsistent with the handling of overflowing literals in the
rest of Nix.
Let’s fix this so that Nix implementations can correctly flag up
this error and we can unblock the toml11 update.
This was cherry‐picked from
<https://github.com/NixOS/nixpkgs/pull/266705> and merged as part of
<https://github.com/NixOS/nixpkgs/pull/318712>, despite there being
a blocking review on the former pointing out these kinds of issues.
This documents some of the dodgy behaviour. It also can’t handle
negative literals. It might be worth considering deprecating and
dropping this, by inlining it into `lib.network.ipv6.fromString`,
its only in‐tree user.
This allows individual types to add attributes that would be discarded during normal evaluation.
Some examples:
types.submodule performs a submodule evluation which yields an 'evalModules' result.
It returns '.config' but makes the original result accessible via 'valueMeta' allowing introspection of '.options' and all other kinds of module evaluation results
types.attrsOf returns an attribute set of the nestedType.
It makes each valueMeta available under the corresponding attribute name.
This has been marked insecure a while ago, as some CVEs have not been
backported. Even if *some* CVEs are fixed, we'd need **all** of them to
be, to get it back into the cache.
Not having it in the cache means, we can not test it in CI. This means
we can't make sure to actually support this version to evaluate Nixpkgs.
The concept of this alias becomes questionable once we move past 2.18,
where Lix was forked. We should probably move to a feature-detection
based approach for lib/minver.nix eventually, too.
hsjobeki: Using config in imports is possible in general. But its not possible to do conditional imports where the condition depends on config. Thats two different statements.
Co-authored-by: Johannes Kirschbauer <hsjobeki+github@gmail.com>
refactorial changes:
- move `CPP` into `env`
- remove `sed 's,^as_dummy.*,as_dummy="\$PATH",' -i configure` because
it didn't change anything other than the store paths in the resulting
derivation (using `diff --recursive` (and `xxd` to diff binary files
that changed))
- remove `xorg.buildPackages.libc.static` from `depsBuildBuild` since
`xorg.buildPackages` doesn't exist (thus
`xorg.buildPackages.libc.static or null` evaluating to `null`).
Fixing that to `libc.static` (and moving it to `nativeBuildInputs`
(where non-compiler deps belong)) actually broke the static build in
a weird way i don't understand.
As initially designed, `lib.packagesFromDirectoryRecursive` allowed
passing a string for the `directory` argument. This is necessary for
several reasons:
- `outPath` on derivations and Flake inputs is not a path.
- Derivations can be coerced to their `outPath` in string interpolation,
but that produces strings, not paths.
- `builtins.path`, bizarrely, returns a string instead of a path (not
that the documentation makes this clear).
If a path is used instead of a string here, then Nix will dutifully copy
the entire directory into a new path in the Nix store (ignored as
WONTFIX by Eelco in https://github.com/NixOS/nix/issues/9428). For
industrial use cases, this can result in an extra 10-15 seconds on every
single eval just to copy files from one spot in the Nix store to another
spot in the Nix store.
In #361424, this was changed so that `directory` must be a path,
breaking these use-cases.
I'm not really sure what happened here -- #361424 has very little
justification for why it exists, only a reference to a previous version
of the PR (#359941), which itself had very little justification given.
The description on #359941 explained that it would "Shrink the
function's code by ~2/3rd 🎉", but 60% of the reduction in size was just
deleting comments (!) and bindings like `directoryEntryIsPackage` that
helped clarify the intent of the implementation. As a result, the new
implementation is (to my eyes) more challenging to read and understand.
I think the whole thing was in service of #392800, which adds a
`newScope` argument in order "to create nested scopes for each
(sub)directory (not just the top-level one) when `newScope` is given."
Nobody noticed this regression until after the commit was merged. After
@phanirithvij pointed out the regression, @nbraud said they would
"shortly prepare a PR to fix this" [1] but did not. Later, they would
explain that they were "quite ill the last month(s)" [2], which explains
why this got forgotten about. @nbraud also requested a review from
@Gabriella439 [3], as she had reviewed the original PR adding
`lib.packagesFromDirectoryRecursive`, but not from me, the original
author of that PR. @Gabriella439 did not review the "refactor" PR, and
no attempt to contact her or myself was made after that initial request.
This behavior is admittedly rather subtle, so I'm not sure either
Gabriella or myself would have noticed the change (especially since the
relevant PR restructures the entire implementation).
While I find this a bit frustrating, I should have added a test for this
use-case in my original PR; if there was a test that relied on passing
paths in as a string, perhaps the authors modifying this code would have
noticed that the implementation was not an accident.
[1]: https://github.com/NixOS/nixpkgs/pull/361424#discussion_r1912407693
[2]: https://github.com/NixOS/nixpkgs/pull/359984#issuecomment-2775768808
[3]: https://github.com/NixOS/nixpkgs/pull/361424#issuecomment-2521308983
this architecture mapping is used broadly in the node ecosystem.
an assortment of tools and hooks, like buildNpmPackage or
pnpm.configHook, will benefit from reusing these values. placing them in
stdenv makes sense because (1) several of these tools don't currently
depend on nodejs, and may even be available where nodejs is not and (2)
`stdenv.{build,host,target}Platform` seems to be less error-prone than
`pkgs*.nodejs.{os,arch}` -- especially for setup hooks where the offsets
are shifted.
Tested with:
1. Replace the callPackageWith call by `null`, to simulate an ancient Nix
2. Run the following commands in a terminal in nixpkgs:
$ mkdir test/
$ echo '{ asdfasdfasdf }: null' >test/default.nix
$ nix repl -f .
nix-repl> callPackage ./test { }
error:
… while calling the 'abort' builtin
at /home/user/src/nixpkgs/lib/customisation.nix:312:7:
311| else
312| abort "lib.customisation.callPackageWith: ${error}";
| ^
313|
error: evaluation aborted with the following error message: 'lib.customisation.callPackageWith: Function called without required argument "asdfasdfasdf" at /home/user/src/nixpkgs/test/default.nix'
ELFv1 is the historically better supported one on glibc, ELFv2 seems to have some issues with our toolchain.
Restore the option to pick the ABI with pkgsCross.
Previously we were taking nixVersions and this made external use from
the Lix repo's CI annoying.
We should probably also test other nix versions than stable (i.e. also
latest and Lix), but this involves writing GitHub Actions about it and
maybe not running it on every single PR. Future work.
Before:
$ nix-build . -A opensycl
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
/nix/store/8g0lfv82s0sprmqgfj146ggkb7bn3rm6-adaptivecpp-25.02.0
$ nix-instantiate . -A opensycl
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/b8q5xp94s4n7zfn7as5a9xjvylh6y3pi-adaptivecpp-25.02.0.drv
After:
$ nix-build . -A opensycl
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
/nix/store/8g0lfv82s0sprmqgfj146ggkb7bn3rm6-adaptivecpp-25.02.0
$ nix-instantiate . -A opensycl
trace: evaluation warning: 'opensycl' has been renamed to 'adaptivecpp'
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/b8q5xp94s4n7zfn7as5a9xjvylh6y3pi-adaptivecpp-25.02.0.drv
The vast majority of CI jobs to build the lib tests are caused by
changes in the maintainer list. In this case, we currently run the full
test-suite which takes 3-4 minutes. By moving the maintainers and teams
tests out of the test-with-nix file, we save almost all of that.
Building only those two tests on a change is almost instant. This only
works, because we previously enabled cachix for the workflow.
Note, that these tests are not actually run with both nix versions, even
though they were listed in the "test with specific nix version" file.
That's because we only differ in the nix version run *inside* the
sandbox, but not doing the outer build.
Since this file seems to be re-used by NixOS/nix' CI, this is
technically a small loss in coverage for that repo, but nixpkgs CI
considerations outweigh that. But because of this, I left the other
non-nix-version-specific tests in that file.
This patch adds `lib.repoRevToName` function that generalizes away most of the
code used for derivation name generation by `fetch*` functions (`fetchzip`,
`fetchFromGitHub`, etc, except those which are delayed until latter commits
for mass-rebuild reasons).
It's first argument controls how the resulting name will look (see below).
Since `lib` has no equivalent of Nixpkgs' `config`, this patch adds
`config.fetchedSourceNameDefault` option to Nixpkgs and then re-exposes
`lib.repoRevToName config.fetchedSourceNameDefault` expression as
`pkgs.repoRevToNameMaybe` which is then used in `fetch*` derivations.
The result is that different values of `config.fetchedSourceNameDefault` now
control how the `src` derivations produced by `fetch*` functions are to be
named, e.g.:
- `fetchedSourceNameDefault = "source"` (the default):
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-source.drv
```
- `fetchedSourceNameDefault = "versioned"`:
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-libfuse-2.9.9-source.drv
```
- `fetchedSourceNameDefault = "full"`:
```
$ nix-instantiate -A fuse.src
/nix/store/<hash>-libfuse-2.9.9-github-source.drv
```
See the documentation of `config.fetchedSourceNameDefault` for more info.
I believe this change is wrong both theoretically and practically.
Theoretically, `null` is available on every platform, because
`buildInputs = [ null ];` always succeeds and never throws a platform
availability error. `null` should be handled consistently with packages
that have no explicit list of supported platforms, as it of course
has no such list itself.
Practically, we use `null` to represent libraries that are always
present on a platform and do not require a library (for instance,
because they are part of `libc` or the macOS SDK). This has been
used for a long time by `libintl` (on all non‐glibc platforms),
and is also now used by `libGL` and friends on Darwin. This change
broke the check SDL3 does for OpenGL availability on Darwin, causing
<https://github.com/NixOS/nixpkgs/issues/407056>, which had to be
worked around by <https://github.com/NixOS/nixpkgs/pull/409525>.
Both `libintl` and `libGL` should count as available on platforms
where their functionality is part of the standard build environment,
and a package that is completely unavailable and whose functionality
cannot be expected should not use `null`, as it should result in
errors if used in a dependency list on an unsupported platform.
I accept that overriding with `null` is often a useful way to disable
dependencies that don’t have explicit feature flags, but I do not
think that making it work better with feature flags conditioned on
availability is worth the inconsistency and problems caused by this
change. Packages can instead expose the relevant feature flags as
arguments that default to the `lib.meta.availableOn` check or, if they
want to keep an “override the dependency to `null`” interface,
insert an explicit `pkg != null && …` check.
Additionally, the pull request was merged over a week after all
breaking changes were restricted for the 25.05 release. I believe that
the potential problems of dealing with the effects of this change for
an entire release cycle – the first release cycle where `libGL` is
`null` on Darwin, a change I made before the deadline and before this
change to `lib.meta.availableOn` – offset the risks of backporting
this revert at such a late stage.
It will cause overrides to backwards‐incompatibly revert to the
behaviour they had before the change, but since such overrides were
not possible until a few weeks ago, I hope that is an acceptable risk
compared to the potential issues leaving this in the release can
cause, given that it was merged after the deadline and has already
broken an existing construction in Nixpkgs.
This reverts commit 9338d924db.
The `importApply` docs reference using the `_key` attr along with
`importApply` or `_file`, however the actual attr name used by the
module system is `key`.
The new implementation of `mapAttrsToList` is simpler than the previous one, avoiding an extra string conversion. Benchmarking shows a slight performance improvement. See the discussion here: https://discourse.nixos.org/t/another-implementation-of-mapattrstolist
Additionally, I searched nixpkgs for expressions equivalent to the old `mapAttrsToList` and replaced them with direct calls to the new implementation.
To fix overriding packages that checks for platform compatibility, like pipewire.
`pipewire` contains the following logic to enable support for ldac depending on library platform compatibility:
```nix
ldacbtSupport = lib.meta.availableOn stdenv.hostPlatform ldacbt
```
Which is used later in the expression to create a Meson flag:
```nix
(lib.mesonEnable "bluez5-codec-ldac" (bluezSupport && ldacbtSupport))
```
This means that attempting to build `pipewire` without `ldacbt` like:
```nix
pipewire.override {
ldacbt = null;
}
```
will fail because the the Meson flag indicates the feature should be enabled, but the library is passed to `buildInputs` as `null`.
lib.strings.escapeC produces single‐digit hexadecimal strings for
character values ≤ 15, which results in an ambiguity. If the following
character is a hex digit, it will be interpreted as being part of the
escape sequence.
systemd, which also relies on C‐style escape sequences, does not
decode single‐digit sequences at all, even if unambiguous.
Padding the hexadecimal string with "0" avoids this problem.
qemu architecture names are fixed — we're using uname here just
because it's more likely to be correct than CPU name (see e.g. POWER).
This means that aarch64 is always called aarch64, even on Darwin where
uname reports arm64.
Fixes: 61582c7043 ("lib/systems: use Darwin architecture names for `config` and `uname`")
So we are adding a simplified version that builds a monolithic nix binary to get finished
in time for the release. Afterwards we will switch to the modular build again.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
