This moves the parsing of ci/OWNERS into the Nix sandbox. We also get
rid of checking out the nixpkgs repo another time in the reviewers
workflow - we already have everything we need in the eval/compare job.
The creation of owners.txt in this way is only temporary, it should
eventually be moved further, similar to how maintainers.json is
currently migrating to a maintainer map for the whole repo stored on the
target branch as artifact.
(cherry picked from commit 3bef0dcf65)
Instead of setting the maintainer-related labels based entirely on Eval
results, this uses the new maintainer map from the target branch. This
allows labeling PRs correctly, that had been created *before* a
contributor became a maintainer of the respective package.
(cherry picked from commit 6b5e6cbbee)
There is no point in passing these arguments around between all the
different parts of the eval chain - these global settings should surely
not be modified between different steps.
This also makes it simpler to add new arguments like extra nixpkgs
configuration.
(cherry picked from commit 11749394df)
This creates a simple mapping of all packages to github ids of all their
maintainers. This is uploaded as an artifact and is then available for
download on the latest commit of each branch with a merge queue. This
will allow scheduled jobs to use this information for setting
maintainer-related labels, to request reviewers and to implement the
merge-bot.
The advantage over querying this information directly via Nix in each
case: The scheduled job does not need to install Nix and does not need
to checkout the target branch.
Compared to obtaining the maintainer information just for a single PR
during Eval, this will allow retroactively changing maintainers for a
package: For example, it allows to request a new maintainer as reviewer
for a PR that was created before they became maintainer, but is still
open - and similarly for maintainer labels and merge-bot rights.
None of these extensions are implemented by this PR, yet.
(cherry picked from commit 498574f238)
Currently the `diff-<system>` artifacts are 6-7 MB in size - and almost
all of that is the `paths.json` file, which is only used to generate the
diff itself. This had been stored in the artifact previously for
debugging purposes. Ever since we moved to Cachix this is not required
anymore, since it's possible to run the same eval locally and thus fetch
the `eval.singleSystem` result, including `paths.json`, from Cachix.
This will be even more helpful when the next step adds `meta.json` -
which is magnitudes bigger than `paths.json`.
(cherry picked from commit 350a469f73)
In a recent change, the path matching was simplified in maintainers.nix.
This revealed a pre-existing logic bug: Packages without `meta.position`
would get an empty string as their file name. The change would then
cause this empty string to always be matched, which lead to maintainer
pings for these packages in seemingly random PRs, when some of their
dependencies were changed.
(cherry picked from commit 6d564102b7)
We should never try to ping maintainers through package aliases, this
can only lead to errors. One example case is, where an attribute is a
throw alias, but then re-introduced in a PR. This would trigger the
throw. By disabling aliases, we can fallback gracefully.
(cherry picked from commit 6a26c66335)
This allows requesting reviewers for pure refactor PRs, which don't
cause a rebuild of the package. This is only possible for by-name,
because only here the package names can be inferred from the filenames.
(cherry picked from commit f33892fcc9)
This adds support to ping maintainers when arbitrary files in by-name
are changed, as long as they still cause a rebuild. For example, this is
the case when changing .json files with version metadata. These were
previously not detected as belonging to the package, and didn't cause
maintainer pings.
(cherry picked from commit 0a5ffcdf31)
The only reason for the additional `lib.hasSuffix` check was, that the
`lib.removePrefix` was broken - it was never adjusted when porting this
from ofborg, so the relative path was wrong and no prefix ever removed,
since no packages are in `ci/`.
This additionally strips the leading `/`, so that `relevantFilenames`
will then have paths starting with `pkgs/...`, similar to how git
reports those paths in the `changedpathsjson` file. This allows simple
equality comparison.
(cherry picked from commit 7c865ddd46)
Removes unused code and comments, uses shorter lib over builtins and
switches to `lib.importJSON`.
Readability is key.
(cherry picked from commit 79029e39a4)
This has been broken for months, since all the interface to `ci/eval`
changed. Nobody noticed, so nobody is using it.
(cherry picked from commit 44e4c1ba9f)
This indicates that the NixOS test-driver changed and all NixOS tests
have to be rebuilt. It can be used to either re-target to staging or to
batch this with other similar changes, at least.
(cherry picked from commit 63e6fc4c43)
We can't eval all nixos tests, this would be way too expensive. But we
can evaluate `nixosTests.simple` as an indication whether the nixos test
driver has changed. If that's the case, this means that *all* NixOS
tests need to be rebuilt.
(cherry picked from commit e3f583c5bf)
The change to use `builtins.storePath` was good - for when the store
path *is* already part of the nix store. In all my tests so far, that
was already the case, because I was iterating on the solution and the
Eval results stayed the same.
But when this is run on a entirely new commit, these the values for
`afterDir` and `combinedDir` are *not* in the store, yet. As part of
running `eval.full` on a new commit they will be created. `eval.full` is
linked up, so that the values passed around there will actually be
derivations, which might not be realized, yet.
Checking whether the input is a path or not fixes this for both cases.
(cherry picked from commit b9d4098221)
Due to how we pass in existing store paths via CLI arguments for the
diff and combine scripts, Nix didn't register a dependency on the store
paths properly. This meant that some of the derivations that were built,
didn't have the right store paths made available in the sandbox -
leading to all kinds of "not found" errors.
We worked around this in CI by resolving the symlinks to the nix store
beforehand. We tried to work around this locally by storing the nix
store path in BASELINE, but this didn't fully work. By explicitly
registering these store paths as dependencies, this should work across
the board - without any magic required by the caller.
(cherry picked from commit 45a765a282)
Update the README.md to document what gets returned.
We might in the future split these up into other attrsets but I don't see a usecase for that at the moment.
(cherry picked from commit 07916fc3fd)
This attribute was supposed to be set on derivations, to make the
release tools recurse into them. The remaining uses were all on regular
attrsets, though, so this is safe to remove.
(cherry picked from commit d0c0b875f7)
The following changes were made:
- Using `lib.` instead of `builtins.`
- Using `mapAttrsToList` instead of `mapAttrs` + `attrValues`
- Joining two of the if conditions with the same return value
- Using `traceIf` instead of `if` / `else`
- Using `showAttrPath` instead of `concatStringsSep`
(cherry picked from commit 04fcbb45e1)
This condition doesn't make a difference anymore, ever since we removed
the tryEval code from this file and had already enabled unfree packages
earlier anyway.
(cherry picked from commit 9524a21fe0)
By now, these files have been changed enough to not need the "vendored
from" notes anymore. These links would still be there when going through
the history of the file, but today GHA CI has not many similarities
anymore to what ofborg did, so these are not really helpful.
(cherry picked from commit 2aae142529)
These files are tightly coupled with the code in ci/eval and not used
anywhere else. They are subject to the same backporting requirements as
the remaining CI code. They are better placed next here.
(cherry picked from commit 0d51e920d3)
Not all packages that are reported as changed will actually exist on the
platform that the maintainers are colleted on.
This is the case for some attributes that are only available on Darwin
or explicitly set to `null` on Linux. By filtering out packages without
maintainers, these are ignored - and we should potentially get a small
performance improvement as well.
(cherry picked from commit f2ca5796de)
