`strings.trim` returns a copy of the string with all leading and trailing
whitespace removed.
`strings.trimWith` does the same thing, but calling code can decide
whether to trim the start and/or end of the string.
Those attrs have been renamed and throwing is the best way to show it,
if we only warned then the user would only get an error like this `error: Unsupported sdk: 33`
from `pkgs/top-level/darwin-packages.nix`.
If someone wants to support multiple NixOS versions then they can simply
set both attrs. (`!args ? androidSdkVersion` is for that)
`sdkVer` conflicts with the old `sdkVer`(now `darwinSdkVersion` but that still uses `sdkVer` if set) used by darwin
This shouldn't be an issue but due to `pkgs/development/interpreters/python/cpython/default.nix`
running `lib.filterAttrs (n: v: ! lib.isDerivation v && n != "passthruFun")` on it's inputs (2 of them are darwin only)
the `throw "Unsupported sdk...` in `pkgs/top-level/darwin-packages.nix` will be triggered.
After this change `pkgsCross.armv7a-android-prebuilt.python3.pythonOnBuildForHost` won't fail with
`error: Unsupported sdk: 33`
Issue was bisected to 3cb23cec23
- Bring up to date
- Give meaning to the metavariables
- Use italics for metavariables (just like the Nix manual)
- Don't abbreviate
- No hard feelings, Val.
The old stdenv didn't work, and was also impure. The new one works, and
is pure. Presently, the bootstrap tools are cross compiled into one small
nar and one large tar, which is then unpacked, patched, and split into
smaller derivations. Efforts were made to make the boot process as short
as possible - there are only two clangs built, and as many packages are
propagated between stages as possible while leaving the bootstrap tools
out of the final stdenv's closure.
This is a follow-up to https://github.com/NixOS/nixpkgs/pull/325380 where @emilazy discovered that we have some typos in field names.
This refactoring wraps the implicit lib.licenses schema up in a factory function to ensure consistency.
While a more proper type checker like the one we use for meta checks would be better I didn't want to depend on that in more places.
In fact, we might want to make meta type checks more strict on the license field.
Previously we would fallback to using `kernel` as the `os` which would
result in using the wrong `os` value (`none`) when actually we want
`unknown`. This seems to be a special case for wasm32-unknown-unknown
and wasm64-unknown-unknown so I extended the if statement to support it.
All builtins should be in mirrored in lib, for consistency, as well
as control to let the Nixpkgs pin effect *subtle* improvements in
behavior (such as the foldl' accumulator strictness).
Benefits:
- some lookups happened in the hot path, and will now be slightly faster,
with only a variable lookup and no attribute selection
- it's now harder to accidentally use args.lib aka specialArgs.lib, which
has happened
- shorter
The practical use for this should be very limited because I don't
think anyone should change `lib`, let alone change `lib.functionArgs`,
but, but it would be even stranger to rely on `args.lib` (or really
`specialArgs.lib` for what's clearly a behavior of the current
`evalModules`, which uses its own ambient lib for basically everything.
The shadowing of `lib` by `args.lib` here seems to be a small mistake,
which is easy to make.
Add lib.meta.getLicenseFromSpdxIdOr as a variant of
lib.meta.getLicenseFromSpdxId that explicitly state the default
(fallback) value if there's no license matching the given SPDX ID.
Add a library function to parse and validate an IPv6 address from a
string. It can parse the first two versions of an IPv6 address according
to https://datatracker.ietf.org/doc/html/rfc4291#section-2.2. The third
form "x❌x❌x:x.d.d.d.d" is not yet implemented. Optionally parser can accept prefix length (128 is default).
Add shell script network.sh to test IPv6 parser functionality.
* doc: migrate lib.filesystem to doc-comment format
* defintion list fixes lib/filesystem.nix
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
---------
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
`rustc.config` is called `rust.rustcTarget` now, and
`{rustc -> rust}.platform`.
This is the new way (tm), and is preferred since
https://github.com/NixOS/nixpkgs/pull/271707 -
though the documentation still is outdated, and some expressions in
nixpkgs were using the old interface.
This updates both.
Friend came to me with misunderstanding of how this function works.
After discussion we came to the conclusion having an example would have
prevented this.
* Extend libc
Include non-libc core libraries in the libc package. Many of these
mirror libraries present in glibc on linux, such as libgcc, libraries
used for iconv, and libraries used for reading kernel info (libkvm,
libprocstat, libmemstat).
Without this many packages outside the freebsd tree would need to be
modified to include standard dependencies which would already be on
the system for other packages.
* Mark FreeBSD as using LLVM
* Update default LLVM version FreeBSD
* Use patch monolith
The patchesRoot system combined with the fact that each derivation
will Request specific names of patches makes it very annoying to use
other FreeBSD source trees with nixpkgs. This new system allows
providing one Or more entire trees of patches whose contents will be
dynamically Parsed and only the relevant patches will be applied for
any one Derivation.
With this commit, the following knobs are available for specifying the
FreeBSD source:
- overriding `freebsd.versionInfo`, for picking another official
supported FreeBSD release.
- overriding `freebsd.source` for specifying a specific unpatched
FreeBSD source tree.
- overriding `freebsd.patches`, for specifying the patches to apply.
Co-Authored-by: Audrey Dutcher <audrey@rhelmot.io>
Co-Authored-by: John Ericson <John.Ericson@Obsidian.Systems>
The idea behind that is to enable users and developers of
downstream tools such as home-manager to test Nix master for several
reasons:
* Nix is currently trying to have a `master` branch that's always
releasable[1]. We're still on Nix 2.18 in nixpkgs due to too many
notable regressions. Enabling people to test latest master may help on
that end.
* This uses the most bleeding-edge Nix, but our packaging, so we can
identify issues with our packaging early.
* From what I've seen, most people are using the packages from nixpkgs
anyways instead of the upstream flake, this is far more convenient
anyways.
My plan is to update this once a week. Right now we rely on the
`installCheckPhase` here, but as soon as we have proper regression
testing[2], we may want to add `nixUnstable` there as well (however with
failures being allowed probably).
[1] https://discourse.nixos.org/t/nix-release-schedule-and-roadmap/14204
[2] https://github.com/NixOS/nixpkgs/pull/304332
This is all I could find after co-maintaining lib for a long time.
I've had the fortune of basically not really noticing this file,
because it has had very few interactions until the confusion in
https://github.com/NixOS/nixpkgs/pull/304277
It seems to be a state of limbo, which would be nice to resolve
(with great care), but this is not urgent, and first we should
document its status.
Part 1 of #301908.
This renames the two versions of the Apple Public Source License seen in
nixpkgs; `apsl20` was often confused as being for the widely-used Apache
License 2.0.
I haven't managed to trigger the error, and it turns out that this
method is optional.
Specifically, getSubmodules is unimplemented (and unimplementable),
the tests pass, and we seem to have good location info.
* doc: migrate lib.trivial to use doc-comments
* Apply suggestions from code review
---------
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
This allows refactoring in the file without accidentally modifying the
public interface of the file.
Also, pull in symbols consistently from `lib` instead of `builtins`.
There's a couple of aliases in play:
1. `lib.isList` is an alias for `builtins.isList`.
2. `lib.strings.concatStringsSep` is an alias for `builtins.concatStringsSep`
* doc: add details on `mapAttrsRecursive[Cond]`
from first reading it wasn't clear that `f` also takes the current
attribute path. also the value f receives is tricky due to how the
condition is evaluated.
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
makeOverridable is very careful to ensure the arguments to the
overridden function are the same as the input function. As a result,
the arguments of hello.override are exactly the same as the original
arguments of the hello function that produced the derivation.
However, callPackagesWith calls makeOverridable with a lambda that
does not propagate the arguments. The override function for a package
instantiated with callPackagesWith will not have the original
arguments.
For example:
nix-repl> lib.functionArgs hello.override
{ callPackage = false; fetchurl = false; hello = false; lib = false; nixos = false; stdenv = false; testers = false; }
nix-repl> lib.functionArgs openssl.override
{ }
By copying the arguments onto the inner lambda before passing it to
makeOverridable, we can make callPackage and callPackages behave the
same.
nix-repl> lib.functionArgs openssl.override
{ buildPackages = false; coreutils = false; cryptodev = false; enableSSL2 = true; enableSSL3 = true; fetchurl = false; lib = false; perl = false; removeReferencesTo = false; static = true; stdenv = false; withCryptodev = true; withPerl = true; }
This was found when trying to run the fileset tests on Darwin
(https://github.com/NixOS/nix/pull/9546#issuecomment-1967409445), which mysteriously fail on Darwin:
test case at lib/fileset/tests.sh:342 failed: toSource { root = "/nix/store/foobar"; fileset = ./.; } should have errored with this regex pattern:
lib.fileset.toSource: `root` \(/nix/store/foobar\) is a string-like value, but it should be a path instead.
\s*Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
but this was the actual error:
error: lib.fileset.toSource: `root` (/nix/store/foobar) is a string-like value, but it should be a path instead.
Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
After dissecting this, I find out that apparently \s works on Linux, but not on Darwin for some reason!
From the bash source code, it looks like <regex.h> with `REG_EXTENDED` is used for all platforms the same,
so there's nothing odd there.
It's almost impossible to know where <regex.h> comes from,
but it looks to be a POSIX thing.
So after digging through the almost impossible to find POSIX specifications
(https://pubs.opengroup.org/onlinepubs/007908799/xbd/re.html#tag_007_003_005),
I can indeed confirm that there's no mention of \s or the like!
_However_, there is a mention of `[[:blank:]]`, so we'll use that instead.
The previous code was optimized for the old uniq behavior, which did not
call merge. That's changed, so the legacy path is not a hot path anymore,
and is not worth any tech debt.
The deprecation warnings in lib were wildly inconsistent. Different
formulations were used in different places for the same meaning. Some warnings
used builtins.trace instead of lib.warn, which prevents silencing; one even
only had a comment instead. Make everything more uniform.
This allows for adding new, conditionally set, derivation attributes
to an existing derivation without changing any output paths in the
case where the condition is not met.
The only reason shallow clones are not the default in
`builtins.fetchGit` is that `revCount` can't be provided when cloning a
shallow repository. However, `revCount` isn't used or exposed by
`lib.fileset`. Hence, allowing cloning shallow repositories makes
`gitTracked` more general without any drawbacks.
Co-authored-by: Silvan Mosberger <github@infinisil.com>
It is useful that all (or almost all) .nix files in nixpkgs at least
parse since it allows for checking syntax in the repository
programmatically without evaluating anything.
This doesn't change uniq. Why not?
- In NixOS it seems that uniq is only used with
simple types that are fully checked by t.check.
- It exists for much longer and is used more widely.
- I believe we should deprecate it, because unique was
already better.
- unique can be a proving ground.
According to the WebAssembly design doc, wasm32 is an ILP32 ABI like
x32, mips64n32, and aarch64_ilp32 (Apple Watch). This commits adds
it to the predicate.
1319968ca5/CAndC%2B%2B.md?plain=1#L16
Most of the time when we do a patchelf conditional on
hostPlatform.isLinux, what we really mean is hostPlatform.isElf.
Now that we are starting to support BSDs, this is becoming more important.
Type: either ints.positive (enum ["auto"])
Before: positive integer, meaning >0 or value "auto" (singular enum)
After: positive integer, meaning >0, or value "auto" (singular enum)
Currently, the `lib/tests/modules.sh` test checks the output of
`nix-instantiate --eval` without `--json`, which outputs an unspecified
human-readable format.
This patch modifies `modules.sh` to use the `--json` output instead, to
be robust against future changes to `nix-instantiate` output.
This commit temporarily adds pkgs/test/release to the
lib/tests/release.nix test suite, because ofborg already knows about
that entry point.
We should move the list of test entry points out of ofborg and into
a central place in nixpkgs:
https://github.com/NixOS/nixpkgs/issues/272591
Once we do that we won't need to have this ugly kludge in an
inappropriate place.
* Improves the comments of `lib/flake-version-info.nix` and drops the
`__`-prefix from the filename.
* `lib'` -> `lib0` in `nixpkgs/lib`.
* Drop the declaration of `trivial.version` in the overlay because this
declaration already uses the final expressions of `versionSuffix` and
`release` now.
* No need to fall back to `self.lastModified` anymore, this was a
workaround for pre2.4 Nix.
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: Silvan Mosberger <contact@infinisil.com>
That way each expression uses the final version of other lib.trivial
declarations.
For instance, when replacing `versionSuffix` with the string `"fnord"`
in a lib overlay, `trivial.version` uses `"fnord"` as suffix now rather
than `pre-git`.
A more efficient sort in some cases, and often convenient.
This exposes `lib.lists.sortOn` immediately on `lib`, because it is
a sibling of `sort`, which is already present there.
Omitting it would lead to more confusion, and worse outcomes.
There's no confusion about the types `sort` or `sortOn` operate on.
Haskell agrees about the type for `sortOn`, and it is in its `base`.
This type is necessary to have correct merging behavior for
`allowUnfreePredicate` and `allowInsecurePredicate`
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
For the time being, we're moving towards https://nix.dev/ containing
all tutorials and guides. The Nixpkgs manual is reinforced to be a
_reference_ manual. While it's not just reference for now, that's what
the docs team is working towards.
This commits rewrites the Nixpkgs manual introduction to reflect that
and point to some more useful links. The contribution docs are updated
similarly so it's not missed.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
An important idea around the rust stuff in lib.systems is that it's
elaborated — this means that it should idempotently add to the values
passed in, if any. But we missed that the names used for the
parameter and the elaborated value for "rustcTarget"/"config" didn't
line up. The intention was to use "rustcTarget" everywhere in the new
interface, as a more descriptive name than "config".
This fixes setting the system in NixOS configuration, which results in
an already elaborated system being elaborated again. Before, this
wouldn't produce the correct result:
% nix-instantiate --eval -A stdenv.hostPlatform.rust.rustcTarget --system armv7l-linux
"armv7-unknown-linux-gnueabihf"
% NIX_PATH= nix-instantiate --eval -E '(import nixos/lib/eval-config.nix { system = "armv7l-linux"; modules = []; }).pkgs.stdenv.hostPlatform.rust.rustcTarget'
"arm-unknown-linux-gnueabihf"
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
Fixes: https://github.com/NixOS/nixpkgs/issues/271000
Usually, attributes passed explicitly to elaborate take precedence
over the elaborated ones, but since we also elaborate the nested
"rust" attrset, we need to push that one level down, so the rest of
"rust" is still filled in if you just pass
{ rust = { config = ... } }.
I've had to drop the assertion that checked that at most one of "rust"
and "rustc" was part of the un-elaborated system, because doing this
broke passing an elaborated system in, which should be idempotent.
For the same reason, I've also had to make it possible for
rust.rustcTargetSpec to be passed in. Otherwise, on the second call,
since platform was filled in by the first, the custom target file
would be constructed. The only other way to avoid this would be to
compare the platform attrs to all built in Rust targets to check it
wasn't one of those, and that isn't feasible.
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
- Make fromSource's missing file error message more consistent with others,
and add a test for it
- Indent some function arguments
- Fix an internal type
The aws-sdk-cpp tests are flaky.
Since pull requests to staging cause nix to be rebuilt, this means
that staging PRs end up getting false CI failures due to whatever is
flaky in the AWS SDK tests. Since none of our CI needs to (or
should be able to) contact AWS S3, let's just omit it all. Bonus:
the tests build way faster.
While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
- All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
- This is furthermore pointed out in the new error message when a file set is passed
Along the lines of `assertOneOf`, but expects a list of values to be
compared. This gives a good error message and is useful for lists of
values, like `supportedGhcVersions` in the arguments of
`haskell-language-server`.
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
Just minor changes like:
- Always using "X is a Y, but it should be Z"
- "X is a path that does not exist" rather than "X does not exist"
- Always using multi-line strings for errors
- Always quoting string-like values and not quoting path-like values
- But do quote filesystem roots. Even though they're paths, they might
be very small, good to have quotes to know the start/end
- Capitalise the first word
- Distinguish root vs filesystem root more
We have several cross-compilation bugs that show up if
hostPlatform!=buildPlatform yet
hostPlatform.config==buildPlatform.config.
These bugs have appeared and disappeared as we've fiddled with the
definition of equality for platform objects. This commit adds a
clear-cut case where they are *not* equal and never will be, so we
can test it.
This does decrease performance unfortunately
Benchmarking expression toSource { root = ./.; fileset = ./.; }
Mean CPU time 0.103747 (σ = 0.012415) for 10 runs is 97.32181384964636% (σ = 16.34179537413021%) of the old value 0.106602 (σ = 0.0125571)
Statistic .envs.elements (205920) is 105.5842% (+10891) of the old value 195029
Statistic .gc.totalBytes (20247696) is 101.7495% (+348160) of the old value 19899536
Statistic .nrThunks (134824) is 108.7878% (+10891) of the old value 123933
Statistic .symbols.number (996) is 100.1005% (+1) of the old value 995
Statistic .values.number (275238) is 104.1199% (+10891) of the old value 264347
Currently just throws the Nix error because unknown file types are not
supported by the Nix store, but nothing catches this error earlier (yet,
see next commit)
