As initially designed, `lib.packagesFromDirectoryRecursive` allowed
passing a string for the `directory` argument. This is necessary for
several reasons:
- `outPath` on derivations and Flake inputs is not a path.
- Derivations can be coerced to their `outPath` in string interpolation,
but that produces strings, not paths.
- `builtins.path`, bizarrely, returns a string instead of a path (not
that the documentation makes this clear).
If a path is used instead of a string here, then Nix will dutifully copy
the entire directory into a new path in the Nix store (ignored as
WONTFIX by Eelco in https://github.com/NixOS/nix/issues/9428). For
industrial use cases, this can result in an extra 10-15 seconds on every
single eval just to copy files from one spot in the Nix store to another
spot in the Nix store.
In #361424, this was changed so that `directory` must be a path,
breaking these use-cases.
I'm not really sure what happened here -- #361424 has very little
justification for why it exists, only a reference to a previous version
of the PR (#359941), which itself had very little justification given.
The description on #359941 explained that it would "Shrink the
function's code by ~2/3rd 🎉", but 60% of the reduction in size was just
deleting comments (!) and bindings like `directoryEntryIsPackage` that
helped clarify the intent of the implementation. As a result, the new
implementation is (to my eyes) more challenging to read and understand.
I think the whole thing was in service of #392800, which adds a
`newScope` argument in order "to create nested scopes for each
(sub)directory (not just the top-level one) when `newScope` is given."
Nobody noticed this regression until after the commit was merged. After
@phanirithvij pointed out the regression, @nbraud said they would
"shortly prepare a PR to fix this" [1] but did not. Later, they would
explain that they were "quite ill the last month(s)" [2], which explains
why this got forgotten about. @nbraud also requested a review from
@Gabriella439 [3], as she had reviewed the original PR adding
`lib.packagesFromDirectoryRecursive`, but not from me, the original
author of that PR. @Gabriella439 did not review the "refactor" PR, and
no attempt to contact her or myself was made after that initial request.
This behavior is admittedly rather subtle, so I'm not sure either
Gabriella or myself would have noticed the change (especially since the
relevant PR restructures the entire implementation).
While I find this a bit frustrating, I should have added a test for this
use-case in my original PR; if there was a test that relied on passing
paths in as a string, perhaps the authors modifying this code would have
noticed that the implementation was not an accident.
[1]: https://github.com/NixOS/nixpkgs/pull/361424#discussion_r1912407693
[2]: https://github.com/NixOS/nixpkgs/pull/359984#issuecomment-2775768808
[3]: https://github.com/NixOS/nixpkgs/pull/361424#issuecomment-2521308983
lib.strings.escapeC produces single‐digit hexadecimal strings for
character values ≤ 15, which results in an ambiguity. If the following
character is a hex digit, it will be interpreted as being part of the
escape sequence.
systemd, which also relies on C‐style escape sequences, does not
decode single‐digit sequences at all, even if unambiguous.
Padding the hexadecimal string with "0" avoids this problem.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
Previously if function in defs had set pattern in argument,
this information would be lost.
This keeps functionArgs in set pattern,
by using functor and `__functionArgs`
that is later used by `lib.functionArgs`.
Before this patch, code like this would break generate invalid XML:
lib.generators.toPlist {} "ab<cd"
That's obviously bad, since the call to toPlist often happens through
indirection, as is the case in e.g. the nix-darwin project. A user might
not realize that they have to escape the strings.
This patch adds the argument 'escape' to lib.generators.plist and emits
a warning if it is not set to true. In a future release, this behavior
should become the default.
I have also added a note for future maintainers, in case I forget to
actually remove the deprecated functionality in a future release.
These utilities will now leave the string undisturbed if it doesn't need to be quoted (because it doesn't have any special characters). This can help generate nicer-looking command lines.
This also transitively improves the output of `lib.toGNUCommandLine` which uses `escapeShellArg` internally
`strings.trim` returns a copy of the string with all leading and trailing
whitespace removed.
`strings.trimWith` does the same thing, but calling code can decide
whether to trim the start and/or end of the string.
Add lib.meta.getLicenseFromSpdxIdOr as a variant of
lib.meta.getLicenseFromSpdxId that explicitly state the default
(fallback) value if there's no license matching the given SPDX ID.
makeOverridable is very careful to ensure the arguments to the
overridden function are the same as the input function. As a result,
the arguments of hello.override are exactly the same as the original
arguments of the hello function that produced the derivation.
However, callPackagesWith calls makeOverridable with a lambda that
does not propagate the arguments. The override function for a package
instantiated with callPackagesWith will not have the original
arguments.
For example:
nix-repl> lib.functionArgs hello.override
{ callPackage = false; fetchurl = false; hello = false; lib = false; nixos = false; stdenv = false; testers = false; }
nix-repl> lib.functionArgs openssl.override
{ }
By copying the arguments onto the inner lambda before passing it to
makeOverridable, we can make callPackage and callPackages behave the
same.
nix-repl> lib.functionArgs openssl.override
{ buildPackages = false; coreutils = false; cryptodev = false; enableSSL2 = true; enableSSL3 = true; fetchurl = false; lib = false; perl = false; removeReferencesTo = false; static = true; stdenv = false; withCryptodev = true; withPerl = true; }
This allows for adding new, conditionally set, derivation attributes
to an existing derivation without changing any output paths in the
case where the condition is not met.
Type: either ints.positive (enum ["auto"])
Before: positive integer, meaning >0 or value "auto" (singular enum)
After: positive integer, meaning >0, or value "auto" (singular enum)
