The code of `lib.closePropagation` was internally using a
recursion on the dependencies and returns all the derivation directly or
indirectly referenced by buildInputs.
`lib.closeProgation` is implemented in pure nix and uses an unique
function for list which is quadratic and does "true" equality, which
needs deep set comparison.
Instead, we use the `builtins.genericClosure` which is implemented as a
builtin and uses a more efficient sorting feature.
Note that `genericClosure` needs a `key` to discriminate the values, we
used the `outPath` which is unique and orderable.
On benchmarks, it performs up to 15x time faster on a benchmark related
to haskellPackages.ghcWithPackages.
Personally, I think that warnings such as
warning: The option `services.redis.enable' defined in `/home/ma27/Projects/nixpkgs/test.nix@node-vm' has been renamed to `services.redis.servers..enable'.
are fairly confusing because of the `..` and it's more correct to
actually quote that. With this change the warning now looks like this:
warning: The option `services.redis.enable' defined in `/home/ma27/Projects/nixpkgs/test.nix@node-vm' has been renamed to `services.redis.servers."".enable'.
While implementing that I realized that you'd have
a similar problem whenever you use attribute-names that aren't
identifiers, e.g.
services.nginx.virtualHosts."example.org".locations."/".invalid = 23;
now results in the following error:
error: The option `interactive.nodes.vm.services.nginx.virtualHosts."example.org".locations."/".invalid' does not exist. Definition values:
- In `/home/ma27/Projects/nixpkgs/test.nix@node-vm': 23
Of course there are some corner-cases where this won't work: when
generating the manual, you display submodules like this:
services.nginx.virtualHosts.<name>
Since `<name>` isn't a value, but an indicator for a submodule, it must
not be quoted. This also applies to the following identifiers:
* `*` for `listOf submodule`
* `<function body>` for `functionTo`
This might not be correct if you actually have a submodule with an
attribute name called `<name>`, but I think it's an improvement over the
current situation and for this you'd probably need to make even more
complex changes to the module system.
The motivation is to have a single identifier for that. Useful for the
next commit where I'll try to escape option-parts correctly (options can
be any kind of strings, but unless these are Nix identifiers, they must
be quoted).
Since `<function body>` (or `<name>`/`*`) are special identifiers in
error messages and the manual, we need a unique way to mark an option
part as function call because these are not to be quoted.
Move already implemented functionality to the upper level so
it could be used in a more generic way.
Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko@unikie.com>
This brings two benefits:
1. The complete list of collisions is printed in the whenever any colliding
attribute is accessed.
2. The sets are intersected using a C++ primitive, which runs in O(n) time
(intersecting pre-sorted lists) with small constants rather than interpreted
Nix code.
Thanks to @toonn for prompting this improvement.
```
nix-repl> pkgsCross.arm-embedded.stdenv.hostPlatform.emulatorAvailable pkgsCross.arm-embedded.buildPackages
false
nix-repl> pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulatorAvailable pkgsCross.aarch64-multiplatform.buildPackages
true
```
will be useful for stuff like handling https://github.com/NixOS/nixpkgs/issues/187109
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
The comment in lib/systems/default.nix for uname.processor indicates that it
should match `uname -p`. I tried that command and found that it reports
`unknown` on all of these machines:
- `x86_64-linux`
- `aarch64-linux`
- `mips64el-linux`
- `powerpc64le-linux`
The command `uname -m` reports the expected value on all of the above.
I think the comment is wrong. So I fixed it.
This attr provides the location of each definition.
This is particularly useful for introspecting options of type
`attrsOf`. E.g., it allows finding the location of a systemd
service definition by parsing
`options.systemd.services.definitionsWithLocations`.
This is particularly useful for disabling modules defined in a flake.
Example:
disabledModules = [ "${flake}/modules/mymodule.nix" ];
Previously, absolute string paths were internally prepended with `modulesPath`,
which caused the module filtering to fail.
Recent commit 59356f11c1 ("perlPackages: Ensure all packages have a
license", 2022-08-22) added a license field to Perl packages where the
license was missing. The above mentioned packages got assigned
`unfreeRedistributable` license, which is not precise and makes all
packages depending on them unbuildable without `NIXPKGS_ALLOW_UNFREE`.
The packages actually have a license which SPDX calls
BSD-4-Clause-Shortened (https://spdx.org/licenses/BSD-4-Clause-Shortened.html).
In this commit, we add this license to the list of allowed licenses
and change the license field of the mentioned packages.
Closes#188103
In Nixpkgs, we assume that the "config" field is a canonicalized GNU
triple. I noticed that non-canonical values were being used here,
because the pkgsCross.mips64el-linux-gnu triples did not contain the
vendor field, but the pkgsCross.mips64el-linux-gnu.pkgsStatic did.
Here, I've run all the MIPS triples in lib.systems.examples through
config.sub to canonicalize them. I think this will avoid nasty
surprises in future.
Tested by building Nix and the bootstrap files for
pkgsCross.mips64el-linux-gnu.
More nixpkgs code such as `boot.initrd.systemd.emergencyAccess` defines
options that takes hashed passwords, so move the type definition from
modules/ into lib/.
The type definition itself stays unchanged.
`m` must always be an attrset at this point. It is basically always
evaluated. This will make it throw when any of the attrs is accessed,
rather than just `config`. We assume that this will improve the error
message in more scenarios.
This has been deprecated for a long time, and it's doubtful it had any
users to start with. And having an undisablable warning when
enumarating platforms is not good.
These servers apparently no longer exist, since September 2, 2021[1].
If somebody needs this for non-Scaleway machines, they should suggest
its reintroduction with a different name.
[1]: https://news.ycombinator.com/item?id=27192757
Very confusingly, the `isPowerPC` predicate in
`lib/systems/inspect.nix` does *not* match `powerpc64le`!
This is because `isPowerPC` is defined as
isPowerPC = { cpu = cpuTypes.powerpc; };
Where `cpuTypes.powerpc` is:
{ bits = 32; significantByte = bigEndian; family = "power"; };
This means that the `isPowerPC` predicate actually only matches the
subset of machines marketed under this name which happen to be 32-bit
and running in big-endian mode which is equivalent to:
with stdenv.hostPlatform; isPower && isBigEndian && is32bit
This seems like a sharp edge that people could easily cut themselves
on. In fact, that has already happened: in
`linux/kernel/common-config.nix` there is a test which will always
fail:
(stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit)
A more subtle case of the strict isPowerPC being used instead of the
moreg general isPower accidentally are the GHC expressions:
Update pkgs/development/compilers/ghc/8.10.7.nix
Update pkgs/development/compilers/ghc/8.8.4.nix
Update pkgs/development/compilers/ghc/9.2.2.nix
Update pkgs/development/compilers/ghc/9.0.2.nix
Update pkgs/development/compilers/ghc/head.nix
Since the remaining legitimate use sites of isPowerPC are so few, remove
the isPowerPC predicate completely. The alternative expression above is
noted in the release notes as an alternative.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
canExecute is like isCompatible, but also checks that the Kernels are
_equal_, i.e. that both platforms use the same syscall interface. This
is crucial in order to actually be able to execute binaries for the
other platform.
isCompatible is dropped, since it has changed semantically and there's
no use case left in nixpkgs.
Since we (exclusively) use isCompatible to gauge whether platform a can
execute binaries built for platform b, mode switching CPUs are not to be
considered compatible for our purposes: Switching the mode of a CPU
usually requires a reset. At the very least we can't execute a mix of
executables for the two modes which would usually be the case in nixpkgs
where we may want to execute buildInputs for the hostPlatform in
addition to nativeBuildInputs for the buildPlatform.
Since the list only gates the platforms the nixpkgs flake exposes
packages to build on, the `hydra` label made little sense. It was also
only used for this purpose, so the `tier*` attributes were largely
unnecessary.
To reflect the intention more accurately, we expose
`lib.systems.flakeExposed` and use it to gate flake.nix's system list.
Currently the only way to set the description for a submodule type is to
use `freeformType`. This is not ideal as it requires setting a
freeform type, and evaluates the submodule config unnecessarily.
Instead, add a `description` argument to `submoduleWith`.
This reverts commit PR #167947.
Flakes aren't standardised and the `lib` namespace shouldn't be
polluted with utilities that serve only experimental uses.
This patch causes the autodetection code in lib/systems/platforms.nix
to return {} if it cannot detect the platform and one of the
platform.nix-detection-provided attributes (linux-kernel, gcc, and
rustc) are accessed, rather than silently assuming the "pc" platform
as was previously done.
It is definitely safe to assume that code using these attributes is
prepared to deal with `gcc` and `rustc` not being defined, because
many of the working entries in this file don't define it.
Regarding `linux-kernel` the situation is less certain, but some code
(`lib/systems/default.nix` for example) is already designed to deal
with that attribute being missing. At worst it would result in an
"attribute not found" error.
While adding mips64el bootstrap support to nixpkgs, the silent
assumption that mips64el routers are actually Intel PCs caused
significant frustration. This commit removes that assumption in order
to save people who port nixpkgs to new platforms in the future from
this frustration.
For other platforms like Intel and ARM, we can do
e.g. lib.platforms.aarch64 to get only the 64-bit ARM platorms, but
until now there were no equivalents for RISC-V.
Prior to this commit, nixpkgs would assume that every little-endian
mips32 system was a "fuloong2f_n32".
Not only are there plenty of mips32 chips other than the fuloong, but
the fuloong is actually a mips64 chip! Note that the "n32" ABI is
(confusingly) an ABI for 64-bit mips chips (like the "x32" ABI for
amd64 chips -- both are ABIs which use 32-bit pointers on an
otherwise-64-bit system).
This error causes far-ranging problems. One of them was particularly
difficult to track down: it caused GCC to select 128-bit `long double`
types, which is invalid for the mips32 ABI. This isn't noticed until
you try to build musl-libc, which is careful to check for these things.
Prior to this commit,
nix-build . -A pkgsCross.mipsel-linux-gnu.pkgsStatic.hello
would fail. With this commit and #170736, it succeeds.
There is only one ABI for 32-bit MIPS chips. Before mips64, it didn't
really have a name.
The 64-bit MIPS ABI comes in two flavors, "n64" and "n32". It is
commonplace to refer to the old 32-bit ABI as "o32" (MIPS and SGI
documents do this).
However, when configuring gcc, one must use --with-abi=32, not
--with-abi=o32.
Let's keep GCC happy with this commit.
Probably being the most prominent document demonstrating the problem,
configuration.nix(5) describes various types in plural, e.g.
- ` Type: list of strings`
- ` Type: list of systemd options`
However, there are other cases where appending "s" to the inner type
effectively changes the type, e.g.
- ```
Type: list of string matching the pattern
[a-zA-Z0-9@%:_.\-]+[.](service|socket|device|mount|automount|swap|target|path|timer|scope|slice)s
```
This should've read "list of string[s]..." but instead changes the
regular expression.
Simply drop the best-effort plural in favour of correctness and
simplicity rather than adding more grammar related logic/trying to fix
this.
* lib/strings: optimise hasInfix function
* lib/strings: optimise hasInfix further using regex
* rstudio: call hasInfix with a string
* lib/strings: remove let from hasInfix
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
This commit creates flakes.nix, which is a library containing functions
which relate to interacting with flakes. It also moves related functions
from trivial.nix into it.
This is essentially a copy of the function of the same name, from
flake-compat. callLocklessFlake is useful when trying to utilise a
flake.nix without a lock file, often for when you want to create a
subflake from within a parent flake.
Co-authored-by: Tom Bereknyei <tomberek@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Closes#168327
The issue reported there can be demonstrated with the following
expression:
→ nix-instantiate --eval -E "with import ./. {}; pkgs.lib.options.showDefs [ { file = \"foo\"; value = pkgs.rust.packages.stable.buildRustPackages; } ]"
error: attempt to call something which is not a function but a string
at /home/ma27/Projects/nixpkgs/lib/trivial.nix:442:35:
441| isFunction = f: builtins.isFunction f ||
442| (f ? __functor && isFunction (f.__functor f));
| ^
443|
Basically, if a `__functor` is in an attribute-set at depth-limit,
`__functor` will be set to `"<unevaluated>"`. This however breaks
`lib.isFunction` which checks for a `__functor` by invoking `__functor`
with `f` itself.
The same issue - "magic" attributes being shadowed by `withRecursion` -
also applies to others such as
`__pretty`/`__functionArgs`/`__toString`.
Since these attributes have a low-risk of causing a stack overflow
(because these are flat attr-sets or even functions), ignoring them in
`withRecursion` seems like a valid solution.
This allows other module system consumers to
disable these docs via option merging.
For instance arion uses asciidoc instead of
docbook so that would look awful.
This commit adds an `isPower64` predicate to the two existing
predicates for this architecture (`isPower` and `isPowerPC`).
Note that `isPowerPC` matches only 32-bit machines, whereas `isPower`
matches both 64-bit and 32-bit machines. Prior to this commit there
was no single `isXXX` predicate for `powerpc64le`.
Documents the _module.args option, motivated by many usages in Flakes,
especially with the deprecation of extraArgs
(78ada83361)
The documentation rendering for this option had to be handled a bit
specially, since it's not declared in nixos/modules like all the other
NixOS options.
Co-Authored-By: pennae <github@quasiparticle.net>
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
This uses the levenshtein distance to look through all possible
arguments to find ones that are close to what was requested:
error: Function in /home/infinisil/src/nixpkgs/pkgs/tools/text/ripgrep/default.nix
called without required argument "fetchFromGithub",
did you mean "fetchFromGitHub" or "fetchFromGitLab"?
With https://github.com/NixOS/nix/pull/3468 (in current nixUnstable) the error
message becomes even better, adding line location info
Adds some functions related to string similarity:
- lib.strings.commonPrefixLength
- lib.strings.commonSuffixLength
- lib.strings.levenshtein
- lib.strings.levenshteinAtMost
types.optionSet has been deprecated for almost 10 years now
(0e333688ce)! A removal
was already attempted in 2019
(27982b408e), but it was promptly
reinstantiated since some third-party uses were discovered
(f531ce75e4).
It's finally time to remove it for good :)
The current logic assumes that everything that isn't a derivation is a
store path, but it can also be something that's *coercible* to a store
path, like a flake input.
Unnecessary uses of `lib.toDerivation` result in errors in pure evaluation
mode when `builtins.storePath` is disabled.
Also document what a `package` is.
MIPS has a large space of {architecture,abi,endianness}; this commit
adds all of them to lib/systems/platforms.nix so we can be done with
it.
Currently lib/systems/inspect.nix has a single "isMips" predicate,
which is a bit ambiguous now that we will have both mips32 and mips64
support, with the latter having two ABIs. Let's add four new
predicates (isMips32, isMips64, isMips64n32, and isMips64n64) and
treat the now-ambiguous isMips as deprecated in favor of the
more-specific predicates. These predicates are used mainly for
enabling/disabling target-specific workarounds, and it is extremely
rare that a platform-specific workaround is needed, and both mips32
and mips64 need exactly the same workaround.
The separate predicates (isMips64n32 and isMips64n64) for ABI
distinctions are, unfortunately, useful. Boost's user-scheduled
threading (used by nix) does does not currently supports mips64n32,
which is a very desirable ABI on routers since they rarely have
more than 2**32 bytes of DRAM.
This reverts commit 6b077c47ff.
Thanks Infinisil for discovering this problem:
> After a lot of trial and error, trying to prove why fixupOptionType should
> be used here or not, I figured it out: It's needed for the sake of file
> locations in error messages.
... where a bare submodule is an option that has a type like
`submoduleWith x`, as opposed to `attrsOf (submoduleWith x)`.
This makes migration unnecessary when introducing a freeform type
in an existing option tree.
Closes#146882
This ensures that the module file locations are propagated to the
freeform type, which makes it so that submodules in freeform types now
have their declaration location shown in the manual, fixing
https://github.com/NixOS/nixpkgs/issues/132085.
In addition, this also newly allows freeformTypes to be declared
multiple times and all declarations being merged together according to
normal option merging.
This also removes some awkwardness regarding the type of `freeformType`
This type correctly merges multiple option types together while also
annotating them with file information. In a future commit this will be
used for `_module.freeformType`
Makes all options rendered in the manual throw an error if they don't
have a type specified.
This is a follow-up to #76184
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>
Change comment type so than nixdoc picks them up into Nixpkgs manual.
Also improve phrasing a bit and move stuff around so that it is formatted better.
This is a squashed commit. These are the original commit messages:
lib/option: Improve comment
better comment
Update documentation
Updated nixos/doc/manual/development/options-declarations.md with info on mkEnableOption and mkPackageOption.
Updated the comment on mkEnableOption in lib/options.nix
remove trailing whitespace
nixos/doc/option-declarations: Update IDs & formatting
nixos/docs/option-declarations: Escape angle brackets
Build DB from MD
(Amended) Fix typo
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
(Amended) Build DB from MD (again)
This is a squashed commit. These are the original commit messages:
lib/options: Add mkPackageOption
lib/options: Add missing semicolon
lib/options.nix: Make mkPackageOption more complicated
lib/options: Fix indent. & spacing
lib/options.nix: Remove example and align comment
lib/options: ravenous overuse of arguments
lib/options: Format better
lib/options: Add default examplePath decl
lib/options: Make better mkPackageOption function
lib/options: Remove trailing whitespace
lib/options: Improve mkPackageOptions
lib/options: Remove pkgs prefixing
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
lib/options: Slim down mkPackageOption further
lib/options: mkPackageOption: Add "pkgs." to example
lib/options: mkPackageOption: Make name & pkgs single arguments
lib/options: mkPackageOption: Swap name & pkgs
lib/options: Remove unnecessary import
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
Allow a \n character at the end of the string and remove it during the
merge function.
An option of this type will resolve to the value "foo" whether it is set
to "foo" or "foo\n".
This is useful when using 'builtins.readFile' or ''-strings, which might
add an unintended newline (for example, bash trim the final newline from
a subshell).
`assert` has the annoying property that it dumps a lot of code at the
user without the built in capability to display a nicer message. We have
worked around this using `assertMsg` which would *additionally* display
a nice message. We can do even better: By using `throw` we can make
evaluation fail before assert draws its conclusions and prevent it from
displaying the code making up the assert condition, so we get the nicer
message of `throw` and the syntactical convenience of `assert`.
Before:
nix-repl> python.override { reproducibleBuild = true; stripBytecode = false; }
trace: Deterministic builds require stripping bytecode.
error: assertion (((lib).assertMsg (reproducibleBuild -> stripBytecode)) "Deterministic builds require stripping bytecode.") failed at /home/lukas/src/nix/nixpkgs/pkgs/development/interpreters/python/cpython/2.7/default.nix:45:1
After:
nix-repl> python.override { reproducibleBuild = true; stripBytecode = false; }
error: Deterministic builds require stripping bytecode.
Add a new type, inheriting 'types.str' but checking whether the value
doesn't contain any newline characters.
The motivation comes from a problem with the
'users.users.${u}.openssh.authorizedKeys' option.
It is easy to unintentionally insert a newline character at the end of a
string, or even in the middle, for example:
restricted_ssh_keys = command: keys:
let
prefix = ''
command="${command}",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding
'';
in map (key: "${prefix} ${key}") keys;
The 'prefix' string ends with a newline, which ends up in the middle of
a key entry after a few manipulations.
This is problematic because the key file is built by concatenating all
the keys with 'concatStringsSep "\n"', with result in two entries for
the faulty key:
''
command="...",options...
MY_KEY
''
This is hard to debug and might be dangerous. This is now caught at
build time.
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
- Eta reduce `mapAttrsRecursiveCond`, `foldAttrs`, `getAttrFromPath`.
- Modify `matchAttrs` to use `elemAt` instead of `head (tail xs)` to access
elements.
- Modify `matchAttrs` to use `any id` instead of `foldr and true`.
- Eta reduce formal arguments of `recursiveUpdate'.
- Access elements in `recursiveUpdateUntil` using `elemAt` and `head`
directly instead of `head (tail xs)` which copies a singleton unnecessarily.
(`elemAt` is used instead of `last` to save a primitive call to `length`,
this is possible because the 2-tuple structure is guranteed)
- Use `length` instead of comparison to empty list to save a copy.
the foldl is equivalent to a zip with concat. list concatenation in nix
is an O(n) operation, which makes this operation extremely inefficient
when large numbers of modules are involved.
this change reduces the number of list elements by 7 million on the
system used to write this, total memory spent on lists by 58MB, and
total memory allocated on the GC heap by almost 100MB (with a similar
reduction in GC heap size). it's also slightly faster.
While it is a fact of life that aarch64-darwin is built on Hydra, it has
never formally been elevated from the Tier 7 state it was originally
assigned in RFC 0046. Since platform Tier status is not only
descriptive, but also normative, a consensus to commit to supporting
aarch64-darwin would need to be reached.
`builtins.currentSystem` is not available in pure eval. For this
particular test, we don't really care since it's all about generating
.drv files.
Fixes the following error:
$ nix flake check
warning: unknown flake output 'lib'
error: attribute 'currentSystem' missing
at /nix/store/8wvnlbjxlr90kq2qa6d9zjpj8rqkilr5-source/lib/tests/misc.nix:499:73:
498| let
499| deriv = derivation { name = "test"; builder = "/bin/sh"; system = builtins.currentSystem; };
| ^
500| in {
(use '--show-trace' to show detailed location informat
Move function spdxLicense, internally used in yarn2nix
to lib/meta.nix, and
rename to getLicenseFromSpdxId
A similar function is implemented in poetry2nix,
but the one originally in yarn2nix seems beter.
since it falls back to an license-like attrset
for mismatched case
instead of a plain string
Makes any programming errors more likely to show up early.
Non-obvious changes because of this:
- Ignore the `evalConfig` result in `reportFailure`; we're not checking
it at that point.
- Pre-increment `$fail` and `$pass` to make sure the arithmetic doesn't
result in a zero, which would result in a non-zero exit code for the
expression.
mkDerivedConfig : Option a -> (a -> Definition b) -> Definition b
Create config definitions with the same priority as the definition of another option.
This should be used for option definitions where one option sets the value of another as a convenience.
For instance a config file could be set with a `text` or `source` option, where text translates to a `source`
value using `mkDerivedConfig options.text (pkgs.writeText "filename.conf")`.
It takes care of setting the right priority using `mkOverride`.
recursiveUpdate does not produce an attrset until it has evaluated
both its arguments to weak head normal form.
nix-repl> lib.recursiveUpdate (throw "a") (throw "b")
error: b
nix-repl> lib.recursiveUpdate (throw "a") {}
error: a
The current implementation of `mutuallyExclusive` builds a new list with
length subtracted by one on every recursive call which is expensive. When
b is empty, the function still traverses a in its entirety before returning
a result.
The new implementation uses `any` to check if each element of list b is in
list a using `elem`. This maintains short circuiting when list a or b is empty
and has a worst case time complexity of O(nm).
the fix to extendDerivation in #140051 unwittingly worsened eval performance by
quite a bit. set elements alone needed over 1GB extra after the change, which
seems disproportionate to how small it was. if we flip the logic used to
determine which outputs to install around and keep a "this one exactly" flag in
the specific outputs instead of a "all of them" in the root we can avoid most
of that cost.
if extendDerivation is called on something that already had extendDerivation
called on it (eg a mkDerivation result) the second call will set
outputUnspecified=true on every output by way of propagating attributes of the
full derivation to the individual outputs. this in turn causes buildEnv--and
thus nix-shell and environment.systemPackages--to install every output of such a
derivation even when only a specific output was requested, which renders the
point of multiple outputs moot. this happens in python modules (see #139756),
though it seems that tcl and possibly others should also be affected.
PowerNV was looking for a nonexisting zImage file.
Remove unnecessary .file / .installTarget.
Also add config options needed for default minimal
NixOS config and QEMU VirtIO/VirtFS devices.
I'm working on a project that involves running a virtual machine
monitor program, which creates a control socket in the project
directory (because it doesn't make sense to put it anywhere else).
This obviously isn't part of the source of my program, so I think
cleanSource should filter it out.
Consider a derivation a value to be serialized.
nix-repl> lib.generators.toGitINI { hello = { drv = pkgs.hello; }; }
error: evaluation aborted with the following error message: 'generators.mkValueStringDefault: attrsets not supported: <derivation /nix/store/533q15q67sl6dl0272dyi7m7w5pwkkjh-hello-2.10.drv>'
Fixes#137390
The current implementation of the concatStringsSep fallback references
concatStrings whcih is just a partial application of concatStringsSep,
forming a circular dependency. Although this will almost never be
encountered as (assuming the user does not explicitly trigger it):
1. the or operator will short circuit both in lazy and strict
evaluation
2. this can only occur in Nix versions prior to 1.10
which is not compatible with various nix operations as of 2.3.15
However it is still important if scopedImport is used or the builtins
have been overwritten. lib.foldl' is used instead of builtins.foldl'
as the foldl' primops was introduced in the same release as concatStringsSep.
As suggested in #131205.
Now it's possible to pretty-print a value with `lib.generators` like
this:
with lib.generators;
toPretty { }
(withRecursion { depthLimit = 10; } /* arbitrarily complex value */)
Also, this can be used for any other pretty-printer now if needed.
The message I originally implemented here was to catch a mixup of
`config' and `options' in a `types.submodule'[1]. However it looks
rather weird for a wrongly declared top-level option.
So I decided to throw
error: The option `foo' does not exist. Definition values:
- In `<unknown-file>':
{
bar = {
_type = "option";
type = {
_type = "option-type";
...
It seems as you're trying to declare an option by placing it into `config' rather than `options'!
for an expression like
with import ./lib;
evalModules {
modules = [
{
foo.bar = mkOption {
type = types.str;
};
}
];
}
[1] fa30c9abed
When having a bogus declaration such as
{ lib, ... }:
{
foo.bar = mkOption {
type = types.str;
};
}
the evaluation will terminate with a not-so helpful
error: stack overflow (possible infinite recursion)
To make sure a useful error is still provided, I added a `depthLimit` of
`10` which should be perfectly sufficient to `toPretty` when it's used
in an error-case for `showDefs`.
When having e.g. recursive attr-set, it cannot be printed which is
solved by Nix itself like this:
$ nix-instantiate --eval -E 'let a.b = 1; a.c = a; in builtins.trace a 1'
trace: { b = 1; c = <CYCLE>; }
1
However, `generators.toPretty` tries to evaluate something until it's
done which can result in a spurious `stack-overflow`-error:
$ nix-instantiate --eval -E 'with import <nixpkgs/lib>; generators.toPretty { } (mkOption { type = types.str; })'
error: stack overflow (possible infinite recursion)
Those attr-sets are in fact rather common, one example is shown above, a
`types.<type>`-declaration is such an example. By adding an optional
`depthLimit`-argument, `toPretty` will stop evaluating as soon as the
limit is reached:
$ nix-instantiate --eval -E 'with import ./Projects/nixpkgs-update-int/lib; generators.toPretty { depthLimit = 2; } (mkOption { type = types.str; })' |xargs -0 echo -e
"{
_type = \"option\";
type = {
_type = \"option-type\";
check = <function>;
deprecationMessage = null;
description = \"string\";
emptyValue = { };
functor = {
binOp = <unevaluated>;
name = <unevaluated>;
payload = <unevaluated>;
type = <unevaluated>;
wrapped = <unevaluated>;
};
getSubModules = null;
getSubOptions = <function>;
merge = <function>;
name = \"str\";
nestedTypes = { };
substSubModules = <function>;
typeMerge = <function>;
};
}"
Optionally, it's also possible to let `toPretty` throw an error if the
limit is exceeded.
- Remove inheritance of `lists.fold` as it isn't used anywhere.
- Inherit `foldl'` for consistency as only `cartesianProductOfSets` explicitly
reference lib.
- Inline `foldr` to generate nested attrs instead of using `listToAttrs` and `tail`.
This allows checking e.g. stdenv.hostPlatform.isGnu, just like isMusl
or isUClibc. It was already possible to check for glibc with
stdenv.hostPlatform.libc == "glibc", but when that doesn't line up
with how every other platform check works, this is apparently
sufficiently non-obvious that we've ended up with stuff like adding
glibc.static if !isMusl, which is obviously wrong.
This regressed in 9c213398b3
The recursiveUpdate gave the platform both gcc.cpu and gcc.arch attrs
instead of only gcc.cpu. This is invalid; gcc configuration fails with:
```
Switch "--with-arch" may not be used with switch "--with-cpu"
```
So we revert to using `//` to retain only gcc.cpu
(which is more specific than the processor arch).
Trying to create a simple flake with input:
```
{
inputs.nixpkgs-lib.url = "github:NixOS/nixpkgs?dir=lib";
outputs = {nixpkgs-lib, ...}: { };
}
```
Fails when trying to evaluate the nixpkgs-lib flake with an error like:
```
error: getting status of '/nix/store/xxxx-source/lib/lib': No such file or directory
(use '--show-trace' to show detailed location information)
```
Allows for distinction of licenses that are unfree overall but do grant the
right to redistribute. Defaults to the freeness of the license.
Note: Many unfree but are redistributable licenses aren't marked as such.
I expect that to be fixed in a distributed manner over time.
Closes https://github.com/NixOS/nixpkgs/pull/97789
Previously, if a derivation without a `drvPath` was handled, an error
would be thrown:
nix-repl> lib.generators.toPretty {} { type = "derivation"; }
error: attribute 'drvPath' missing, at /home/infinisil/src/nixpkgs/lib/generators.nix:251:24
With this commit it doesn't anymore:
nix-repl> lib.generators.toPretty {} { type = "derivation"; }
"<derivation ???>"
This matches what `nix repl` outputs:
nix-repl> { type = "derivation"; }
«derivation ???»
For an option definition that uses `lib.options.mergeEqualOption`
underneath, like `types.anything`, an error is thrown when multiple
functions are assigned, indicating that functions can't be compared for
equivalence:
error: The option `test' has conflicting definition values:
- In `xxx': <function>
- In `xxx': <function>
(use '--show-trace' to show detailed location information)
However, the error message didn't use the correct files. While above
error indicates that both definitions are in the xxx file, that's in
fact not true. Above error was generated with
options.test = lib.mkOption {
type = lib.types.anything;
};
imports = [
{
_file = "yyy";
test = y: y;
}
{
_file = "xxx";
test = x: x;
}
];
With this change, the error uses the correct file locations:
error: The option `test' has conflicting definition values:
- In `xxx': <function>
- In `yyy': <function>
(use '--show-trace' to show detailed location information)
so the underlaying use case of the preceding commit is so
generic, that we gain a lot in reasoning to give it an
appropriate name.
As the comment states:
image media needs to override host config short of mkForce
m68k was recently added for Linux and none, but NetBSD also supports
m68k. Nothing will build yet, but I want to make sure we at least
encode the existence of NetBSD support for every applicable
architecture we support for other operating systems.
In Autoconf, some old NetBSD targets like "i686-unknown-netbsd" are
interpreted as a.out, not elf, and virtually nothing supports it. We
need to specify e.g. "i686-unknown-netbsdelf" to get the right
behaviour.
Newer bootloaders for RISC-V (i.e., OpenSBI + U-Boot) support
flat and compressed kernel images but not vmlinux. Therefore,
let's build "Image" like what we do with aarch64.
Also copy DTBs while we are at it.
This will begin the process of breaking up the `useLLVM` monolith. That
is good in general, but I hope will be good for NetBSD and Darwin in
particular.
Co-authored-by: sterni <sternenseemann@systemli.org>
Stating that CPUs and the isCompatible relation forms a category (or
preorder) is correct but overtly technical. We can state it more
clearly for readers unfamiliar with mathematics while retaining some
keywords to be useful to technical readers.
This PR adds a new aarch64 android toolchain, which leverages the
existing crossSystem infrastructure and LLVM builders to generate a
working toolchain with minimal prebuilt components.
The only thing that is prebuilt is the bionic libc. This is because it
is practically impossible to compile bionic outside of an AOSP tree. I
tried and failed, braver souls may prevail. For now I just grab the
relevant binaries from https://android.googlesource.com/.
I also grab the msm kernel sources from there to generate headers. I've
included a minor patch to the existing kernel-headers derivation in
order to expose an internal function.
Everything else, from binutils up, is using stock code. Many thanks to
@Ericson2314 for his help on this, and for building such a powerful
system in the first place!
One motivation for this is to be able to build a toolchain which will
work on an aarch64 linux machine. To my knowledge, there is no existing
toolchain for an aarch64-linux builder and an aarch64-android target.
When a list is passed to isStorePath this is most likely a mistake and
it is therefore better to just return false. There is one case where
this theoretically makes sense (if a list contains a single element for
which isStorePath elem), but since that case is also probably seldomly
intentional, it may save someone from debbuging unclear evaluation
errors.
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.
In particular:
- Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
a proper `callPackage`.
- Adjust Darwin `stdenv`'s overlays keeping things from the previous
stage to not stick around too much.
- Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
closer to a unified LLVM and GCC bootstrap.
Previously, an option of type
attrsOf string
wouldn't throw a deprecation warning, even though the string type is
deprecated. This was because the deprecation warning trigger only looked
at the type of the option itself, not any of its subtypes.
This commit fixes this, causing each of the types deprecationMessages to
trigger for the option. This relies on the subtypes mkOptionType
attribute introduced in 26607a5a2e06653fec453c83d063cdfc4b59185f
In 2d45a62899, the submodule type
description was amended with the freeformType description. This causes
all the modules passed to the submodule to be evaluated once on their
own, without any extra definitions from the config section. This means
that the specified modules need to be valid on their own, without any
undeclared options.
This commit adds a test that evaluates a submodules option description,
which would trigger the above problem for one of the tests, if it were
not fixed by this commit as well.
This is done because the next commit makes option evaluation a bit more
strict, which would also trigger this test failure, even though it's not
related to the change at all.
This will be used to issue deprecation warnings recursively in the next
commit
In addition, this allows easily getting nested types of other options, which
is useful when you want to create an option that aliases a part of
another one.
It's a common pattern in Nixpkgs to want to emit a warning in certain
cases, but not actually change behaviours.
This is often expressed as either
if cond then lib.warn "Don't do that thing" x else x
Or
(if cond then lib.warn "Don't do that thing" else lib.id) x
Neither of which really expresses the intent here, because it looks
like 'x' is being chosen conditionally.
To make this clearer, I introduce a "warnIf" function, which makes it
clear that the only thing being affected by the condition is whether
the warning is generated, not the value being returned.
A subflake that can be indidividually accessed without also providing
an interface to the whole of nixpkgs.
Usage:
inputs.nixpkgs-lib.url = "github:NixOS/nixpkgs?dir=lib"
Since shellPackage actually requires the value to be an attribute set
(i. e. an derivation in this case), we cannot re-use the package.check
type checker since it also allows strings or things that are coercible
to strings as long as they look like store paths.
These are all the architectures supported by Nixpkgs on other
platforms, that are also supported by NetBSD. (So I haven't added
any architectures that are new to Nixpkgs here, even though NetBSD
supports some that we don't have.)
The previous mess was partially grouped by OS, and partially grouped
by architecture, which made it very difficult to know where to add new
entries.
I've chosen to group by OS entirely, because OSes are likely to
maintain exhaustive lists of supported architectures, but it's far
less likely we'd be able to find exhaustive lists of supported OSes
for every architecture.
When in the presence of worktrees, it happens that /commondir has a
trailing slash.
In these circumstances, it can lead to `lib.pathType` being passed paths
like `/foo/bar/.git/`, which in turn lead to
`error: attribute '.git' missing`.
With this change, we now make sure send properly-formatted paths to all
other functions.
This, in particular, fixes running NixOS tests on worktrees created by
libgit2 on my machine. (Worktrees created by git itself appear to not
hit the issue.)
I recently wrote some Nix code where I wrongly set a value to an option
which wasn't an actual option, but an attr-set of options. The mistake I
made can be demonstrated with an expression like this:
{
foo = { lib, pkgs, config, ... }: with lib; {
options.foo.bar.baz = mkOption {
type = types.str;
};
config.foo.bar = 23;
};
}
While it wasn't too hard to find the cause of the mistake for me, it was
necessary to have some practice in reading stack traces from the module
system since the eval-error I got was not very helpful:
error: --- TypeError --------------------------------------------------------- nix-build
at: (323:25) in file: /nix/store/3nm31brdz95pj8gch5gms6xwqh0xx55c-source/lib/modules.nix
322| foldl' (acc: module:
323| acc // (mapAttrs (n: v:
| ^
324| (acc.${n} or []) ++ f module v
value is an integer while a set was expected
(use '--show-trace' to show detailed location information)
I figured that such an error can be fairly confusing for someone who's
new to NixOS, so I decided to catch this case in th `byName` function in
`lib/modules.nix` by checking if the value to map through is an actual
attr-set. If not, a different error will be thrown.
Usually we ensure using the mapAttrs call wrapping the license set that
every license has an associated shortName. A change related to legacy
aliases most likely introduced the removal of the shortName attribute
for all the legacy license names by splitting the set into two sets
connected by a record update operator -- leading to mapAttrs only
affecting the first set.
Since it used to be a valid assumption to have that every license had a
shortName attribute, we reintroduce this attribute for the legacy
aliases as well.
Forcing the module to be builtin breaks 5.10, which wants to compile it as a
module (probably due to dependencies). There doesn't seem to be a need to have
it builtin anymore, so we can just remove the override.
In 9c213398b3 kernelPreferBuiltin was
moved/renamed to linux-kernel.preferBuiltin. However, for
armv7l-hf-multiplatform the new option was written with an uppercase P,
which made the kernel build process ignore it.
This reverts commit d9a7d03da8.
Reason for this is that it actually doesn't migitate the issue on nix
stable for another reason: builtins.tryEval doesn't prevent the error
generated by builtins.functionArgs from halting evaluation:
> builtins.tryEval (builtins.functionArgs builtins.functionArgs)
error: 'functionArgs' requires a function, at (string):1:19
Thus it seems that there is no workaround to make
lib.generators.toPretty work with nix stable and primops since there is
no way to distinguish between primops and lambdas in nix.
An high level example case of this problem occuring can be found below:
nix-repl> lib.generators.toPretty {} (lib.concatStringsSep "\n")
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
However this does not happen on other partially applied functions:
nix-repl> lib.generators.toPretty {} (lib.concatMapStringsSep "\n")
"<function>"
The issue, as it turns out is that while builtins are functions,
builtins.functionArgs throws if is passed a builtin or a partially
applied builtin:
nix-repl> lib.generators.toPretty {} builtins.toString
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
nix-repl> lib.generators.toPretty {} (builtins.foldl' (a: b: a + b))
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
I'm pretty sure this qualifies as a nix bug and should be filed
accordingly, but we can work around it in lib.generators.toPretty by
using tryEval and falling back to {} which functionArgs _should_ return
for builtins.
The nix behavior is inconsistent to say the least:
nix-repl> builtins.functionArgs builtins.functionArgs
error: 'functionArgs' requires a function, at (string):1:1
nix-repl> builtins.typeOf builtins.functionArgs
"lambda"
builtins.functionArgs (a: 1 + a)
{ }
nix-repl> builtins.typeOf (a: 1 + a)
"lambda"
PPC64 supports two ABIs: ELF v1 and v2.
ELFv1 is historically what GCC and most packages expect, but this is
changing because musl outright does not work with ELFv1. So any distro
which uses musl must use ELFv2. Many other platforms are moving to ELFv2
too, such as FreeBSD (as of v13) and Gentoo (as of late 2020).
Since we use musl extensively, let's default to ELFv2.
Nix gives us the power to specify this declaratively for the entire
system, so ELFv1 is not dropped entirely. It can be specified explicitly
in the target config, e.g. "powerpc64-unknown-linux-elfv1". Otherwise the
default is "powerpc64-unknown-linux-elfv2". For musl,
"powerpc64-unknown-linux-musl" must use elfv2 internally to function.
Now type checks the resulting function values and allows mkMerge and co.
Also indicates that the type check is done in the function body
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
Looks like these got left behind in the
kernelArch -> linuxArch migration.
Fixes:
* pkgsCross.powernv.linuxHeaders
* pkgsCross.riscv64.linuxHeaders
* pkgsCross.riscv32.linuxHeaders
and dependees
Immensely helpful when you want to see the changes a function makes to
its value as it passes through.
Example:
```
$ nix-instantiate --strict --eval -E '(with import ./lib; traceFnSeqN 2 "id" (x: x) { a.b.c = 3; })'
trace: {
fn = "id";
from = {
a = {
b = {…};
};
};
to = {
a = {
b = {…};
};
};
}
{ a = { b = { c = 3; }; }; }
```
This reverts commit 4ff1ab5a56.
We need this to type options like:
services.xserver.windowManager.xmonad.extraPackages that specify functions that
take an attribute set containing packages / plugins and return a list containing
a selection of the values in this set.
The reason we need a dedicated type for this is to have the correct merge
behaviour. Without the functionTo type merging multiple function option
definitions results in an evaluation error. The functionTo type merges
definitions by returning a new function that applies the functions of all the
definitions to the given input and merges the result.
(cherry picked from commit 7ed41ff5e7)
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
For renames like
mkAliasOptionModule [ "services" "compton" ] [ "services" "picom" ]
where the target is an option set (like services.picom) instead of a single
option (like services.picom.enable), previously the renamed option type
was unset, leading to it being `types.unspecified`.
This changes it to be `types.submodule {}` instead, which makes more
sense.
Since 40e7be1 all ARM platforms that didn't have a parsed cpu version
(e.g. arm-none-eabi) would be handled as armv7l-hf-multiplatform which
did break building arm-trusted-platform packages for some targets (e.g.
rk3399).
Using pcBase as fallback, instead of armv7l-hf-multiplatform,
corresponds with the behaviour we had before 40e7be1.
We recently switched to more explicit GPL license names in line
with the SPDX change and GNU Foundation recommendations:
https://www.gnu.org/licenses/identify-licenses-clearly.html
This followed up older change to use the recommended SPDX ID
18a5e8c36b
but using the `-only` variant for these deprecated licenses too
makes it harder to check for them automatically.
Let’s switch to the appropriate SPDX ID again.
Previously the .enable option was used to encode the condition as well,
which lead to some oddness:
- In order to encode an assertion, one had to invert it
- To disable a check, one had to mkForce it
By introducing a separate .check option this is solved because:
- It can be used to encode assertions
- Disabling is done separately with .enable option, whose default can be
overridden without a mkForce
Previously this option was thought to be necessary to avoid infinite
recursion, but it actually isn't, since the check evaluation isn't fed
back into the module fixed-point.
This implements assertions/warnings supported by the module system directly,
instead of just being a NixOS option (see
nixos/modules/misc/assertions.nix).
This has the following benefits:
- It allows cleanly redoing the user interface. The new
implementation specifically allows disabling assertions or
converting them to warnings instead.
- Assertions/warnings can now be thrown easily from within
submodules, which previously wasn't possible and needed workarounds.
Initially https://github.com/NixOS/nixpkgs/pull/82897 prevented
non-visible options from being rendered in the manual, but
visible-but-internal options were still being recursed into. This fixes
this, aligning the recurse condition here with the one in
make-options-doc/default.nix
The last use of `kernelMajor` in Nixpkgs was removed in 2018.
Even then, I'm not positive it was actually in an exercised code path.
AFAIUI this is now totally redundant and useless as it really was meant
for the 2.4 -> 2.6 transition.
split comes from builtins, not lib.
error: attribute 'split' missing, at /nix/path/nixpkgs/lib/sources.nix:4:4
(use '--show-trace' to show detailed location information)
I think there was a silent (i.e. semantic) merge conflict between PR #101139 and
PR #100456. This commit should fix the error, which manifests as follows:
error: undefined variable 'boolToString' at /home/kkini/src/nixpkgs/lib/types.nix:552:42
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.
Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.
If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.
Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
Add a friendly function to easily return a flattened list of files
within a directory.
This is useful if you want to easily iterate or concatSep the list of
files all found within a directory.
(i.e. when constructing Java's CLASSPATH)
Style improvements
Co-authored-by: Silvan Mosberger <github@infinisil.com>
Previously if `_file` was specified by a module:
trace: warning: The type `types.string' of option `foo' defined in `/nix/store/yxhm2il5yrb92fldgriw0wyqh2kk9qyc-bug.nix' is deprecated. See https://github.com/NixOS/nixpkgs/pull/66346 for better alternative types.
With this change:
trace: warning: The type `types.string' of option `foo' defined in `/home/infinisil/src/nixpkgs/bug.nix' is deprecated. See https://github.com/NixOS/nixpkgs/pull/66346 for better alternative types.
If multiple definitions are passed, this evaluates them all as if they
were the only one, for a better error message. In particular this won't
show module-internal properties like `_type = "override"` and co.
- These symbols can be confusing for those not familiar with them
- There's no harm in making these more obvious
- Terminals may not print them correctly either
Also changes the function argument printing slightly to be more obvious
This new type has unsurprising merge behavior: Only attribute sets are
merged together (recursively), and only if they don't conflict.
This is in contrast to the existing types:
- types.attrs is problematic because later definitions completely
override attributes of earlier definitions, and it doesn't support
mkIf and co.
- types.unspecified is very similar to types.attrs, but it has smart
merging behavior that often doesn't make sense, and it doesn't support
all types
The vision here is that configuration tools can generate .json or .toml
files, which can be plugged into an existing configuration.
Eg:
{ lib, ... }:
{
imports = [
(lib.modules.importJSON ./hardware-configuration.json)
];
}
Jasper has been marked insecure for a while, and upstream has not
been responsive to CVEs for over a year.
Fixes#55388.
Signed-off-by: David Anderson <dave@natulte.net>
Previously the only way to deprecate a type was using
theType = lib.warn "deprecated" (mkOptionType ...)
This caused the warning to be emitted when the type was evaluated, but
the error didn't include which option actually used that type.
With this commit, types can specify a deprecationMessage, which when
non-null, is printed along with the option that uses the type
> NOTE: This function is not performant and should be avoided.
It's not used at all in-tree now, so we can remove it completely after
any remaining users are given notice.
An easy-to-make mistake when declaring e.g. a submodule is the accidental
confusion of `options` and `config`:
types.submodule {
config = {
foo = mkOption { /* ... */ };
};
}
However the error-message
The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.
is fairly unhelpful because it seems as the options are declared at the
first sight. In fact, it took a colleague and me a while to track down such
a mistake a few days ago and we both agreed that this should be somehow caught
to save the time we spent debugging the module in question.
At first I decided to catch this error in the `submodules`-type directly
by checking whether `options` is undeclared, however this becomes fairly
complicated as soon as a submodule-declaration e.g. depends on existing
`config`-values which would've lead to some ugly `builtins.tryExec`-heuristic.
This patch now simply checks if the option's prefix has any options
defined if a point in evaluation is reached where it's clear that the
option in question doesn't exist. This means that this patch doesn't
change the logic of the module system, it only provides a more detailed
error in certain cases:
The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.
However it seems as there are no options defined in [definition 1-entry 1]. Are you sure you've
declared your options properly? This happens if you e.g. declared your options in `types.submodule'
under `config' rather than `options'.
The refactoring in fd75dc8765
introduced a mistake in the error message that doesn't show the full
context anymore. E.g. with this module:
options.foo.bar = lib.mkOption {
type = lib.types.submodule {
baz = 10;
};
default = {};
};
You'd get the error
The option `baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.
instead of the previous
The option `foo.bar.baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.
This commit undoes this regression
Submodules that have a freeform type set behave as if that was the type
of the option itself (for values that don't have an option). Since the
submodules options are shown as separate entries in the manual, it makes
sense to show the freeform type as the submodule type.
For programs that have a lot of (Nix-representable) configuration options,
a simple way to represent this in a NixOS module is to declare an
option of a type like `attrsOf str`, representing a key-value mapping
which then gets generated into a config file. However with such a type,
there's no way to add type checking for only some key values.
On the other end of the spectrum, one can declare a single separate
option for every key value that the program supports, ending up with a module
with potentially 100s of options. This has the benefit that every value
gets type checked, catching mistakes at evaluation time already. However
the disadvantage is that the module becomes big, becomes coupled to the
program version and takes a lot of effort to write and maintain.
Previously there was a middle ground between these two
extremes: Declare an option of a type like `attrsOf str`, but declare
additional separate options for the values you wish to have type
checked, and assign their values to the `attrsOf str` option. While this
works decently, it has the problem of duplicated options, since now both
the additional options and the `attrsOf str` option can be used to set a
key value. This leads to confusion about what should happen if both are
set, which defaults should apply, and more.
Now with this change, a middle ground becomes available that solves above
problems: The module system now supports setting a freeform type, which
gets used for all definitions that don't have an associated option. This
means that you can now declare all options you wish to have type
checked, while for the rest a freeform type like `attrsOf str` can be
used.
This fundamentally changes how the module evaluation internally
handles definitions without an associated option.
Previously the values of these definitions were discarded and only
the names were propagated. This was fine because that's all that's
needed for optionally checking whether all definitions have an
associated option with _module.check.
However with the upcoming change of supporting freeform modules,
we *do* need the values of these.
With this change, the module evaluation cleanly separates definitions
that match an option, and ones that do not.
`toHex` converts the given positive integer to a string of the hexadecimal
representation of that integer. For example:
```
toHex 0 => "0"
toHex 16 => "10"
toHex 250 => "FA"
```
`toBase base i` converts the positive integer `i` to a list of it
digits in the given `base`. For example:
```
toBase 10 123 => [ 1 2 3 ]
toBase 2 6 => [ 1 1 0 ]
toBase 16 250 => [ 15 10 ]
```
The previous hash was too short and caused evaluation-time errors like:
invalid SRI hash 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
Additionally, since the fact that this is broken implies that nobody
could have been using it, "SRI" is a bit of a vague and obscure term,
`fakeSriHash` would be somewhat of a mouthful, and the relevant fetcher
parameters are just called `hash`, rename it to `fakeHash`.
This is used in in the manual generation for option identifiers that can
be linked. This, unike what the example describes, doesn't preserve
quotes which is needed for these identifiers to be valid.
This reverts commit 124cccbe3b.
124cccbe3b
broke the build of NixOS manual.
It does not make sense to be as strict as with attributes since we
are not limited by the CLI's inability to handle numbers.
Placeholders should not be quoted either as they are not part of Nix
syntax but a meta-level construct.
fix: Adding libtool to allow darwin compiles
Libtool seems to be required for mongodb to compile on darwin.
fix: Marking MongoDB as broken on aarch64
fix: Adding libtools to the pkg imports
Update mongodb to 4.0.4
Currently, not providing `name` to `cleanSourceWith` will use the name
of the imported directory. However, a common case is for this to be the
top level of some repository. In that case, the name will be the name of
the checkout on the current machine, which is not necessarily
reproducible across different settings, and can lead to e.g. cache
misses in CI.
This is documented in the comment on `cleanSourceWith`, but this does
not stop it being a subtle trap for users.
There are different tradeoffs in each case:
1. If `cleanSourceWith` defaults to `"source"`, then we may end up with a
user not knowing what directory a source store path corresponds to.
However, it being called "unnamed" may give them a clue that there is a
way for them to name it, and lead them to the definition of the
function, which has a clear `name` parameter.
2. If `cleanSoureWith` defaults to the directory name, then a user may face
occasional loss of caching, which is hard to notice, and hard to track
down. Tracking it down likely requires use of more advanced tools like
`nix-diff`, and reading the source of a lot of nix code.
I think the downside of the status quo is worse.
This is really another iteration of
https://github.com/NixOS/nix/issues/1305: that led to adding the `name`
argument in the first place, this just makes us use a better default
`name`.
The _module option is added as an internal option set, and it messes up
the results of module evaluations, requiring people to manually filter
_modules out.
If people depend on this, they can still use config._module from inside
the modules, exposing _module as an explicitly declared user option. Or
alternatively with the _module attribute now returned by evalModules.
newlib is the default for most tools when no kernel is provided. Other
exist, but this seems like a safe default.
(cherry picked from commit 8009c20711)
This helps with troubleshooting exceptions in config values, which were hard
to track down for options with many definitions.
The trace will look like:
error: while evaluating the attribute 'config.foo' at undefined position:
[...]
while evaluating the option `foo':
[...]
while evaluating definitions from `/home/user/mymod.nix':
while evaluating 'dischargeProperties' at /home/user/nixpkgs/lib/modules.nix:464:25, called from /home/user/nixpkgs/lib/modules.nix:392:137:
while evaluating the attribute 'value' at /home/user/nixpkgs/lib/modules.nix:277:44:
Value error!
where the `/home/user/mymod.nix` module is
{ lib, ... }: {
options.foo = lib.mkOption {
type = lib.types.lines;
};
config.foo = builtins.throw "Value error!";
}
In 87a19e9048 I merged staging-next into master using the GitHub gui as intended.
In ac241fb7a5 I merged master into staging-next for the next staging cycle, however, I accidentally pushed it to master.
Thinking this may cause trouble, I reverted it in 0be87c7979. This was however wrong, as it "removed" master.
This reverts commit 0be87c7979.
I merged master into staging-next but accidentally pushed it to master.
This should get us back to 87a19e9048.
This reverts commit ac241fb7a5, reversing
changes made to 76a439239e.
Let’s call them by what they are, option names.
`generators.mkValueStringDefault` is a better value string renderer
than plain `toString`.
Also add docs to all options.
The semantic difference between `encode` and `to` is not apparent.
Users are likely to confuse both functions (which leads to unexpected
error messages about the wrong types). Like in `generators.nix`, all
functions should be prefixed by `to`.
Furthermore, converting to a string depends on the target context. In
this case, it’s a POSIX shell, so we should name it that (compare
`escapeShellArg` in `strings.nix`).
We can later add versions that escape for embedding in e.g. python
scripts or similar.
lib/cli is very similar to generators, so it should follow largely the
same interface. Similar to how generators isn’t exported, we should
also namespace cli by default (plus “cli” is only three characters to
type).
Before c9214c394b and
9d396d2e42 if .git is symlink the version
would gracefully default to no git revision. With those changes an
exception is thrown instead.
This introduces a new function `pathIsGitRepo` that checks if
`commitIdFromGitRepo` fails without error so we don't have to
reimplement this logic again and can fail gracefully.
Not all modules use name attribute as the name of the submodule, for example,
environment.etc uses target. We will need to maintain a list of exceptions.
lib.commitIdFromGitRepo now resolves the refs from the
parent repository in case the supplied path is a file
containing the path to said repository. this adds support
for git-worktree and things alike. see gitrepository-layout(5).
this also:
- adds a new boolean function lib.pathIsRegularFile to
check whether a path is a regular file
- patches lib.revisionWithDefault and
the revision and versionSuffix attributes in
config.system.nixos in order to support git-worktrees
The standard attrsOf is strict in its *values*, meaning it's impossible to
access only one attribute value without evaluating all others as well.
lazyAttrsOf is a version that doesn't have that problem, at the expense
of conditional definitions not properly working anymore.
Without this change, accessing `mergedValue` from `mergeDefinitions` in
case there are no definitions will throw an error like
error: evaluation aborted with the following error message: 'This case should never happen.'
This change makes it throw the appropriate error
error: The option `foo' is used but not defined.
This is fully backwards compatible.
Fix the broken test in https://github.com/NixOS/nixpkgs/pull/77416
Apparently hydra uses `nix-build lib/tests/release.nix` to run all
tests, where IFD isn't allowed. Fortunately we can get around this with
builtins.toFile, which doesn't require IFD, but still can test the
properties we want.
This fixes imports from the store not being possible, which was caused by
https://github.com/NixOS/nixpkgs/pull/76857
E.g. such a case:
imports = [ "${home-manager}/nixos" ];
With this change, disabledModules applies recursively, meaning if you
have a module "foo.nix" with
imports = [ ./bar.nix ];
then setting
disabledModules = [ "foo.nix" ];
will disable both "foo.nix" and "bar.nix", whereas previously only
"foo.nix" would be disabled.
This change along with https://github.com/NixOS/nixpkgs/pull/61570 allows
modules to be fully disabled even when they have some `mkRenamedOption`
imports.
Previously when this function was called without a value coercible to a
string it would throw an error instead of returning false. Now it does.
As a result this now allows the use of a type like `either path attrs`
without it erroring out when a definition is an attribute set.
The warning about there not being a isPath primop was removed because
this is not the case anymore, there is builtins.isPath. But also there
always was `builtins.typeOf x == "path"` that could've been used
instead. However the path type now stands for more than just path types,
but absolute paths in general.
If I understand correctly, the problem isn't so much that you're assigning to
that top-level attribute, but that the assignment to the attribute (or any
child of the attribute) introduces the 'config' object and prevents 'lifting'
all settings to a generated 'config' object.
This reverts commit eec83d41e3.
This broke hydra evaluation because with this commit submodule values
are allowed to be paths, however the certmgr module uses `either
(submodule ...) path` in its type, meaning it already used paths for
something else which would now be interpreted as a submodule.
