Commit graph

9 commits

Author SHA1 Message Date
Wolfgang Walther 6c105942ca ci/eval: fix local comparison with baseline
Due to how we pass in existing store paths via CLI arguments for the
diff and combine scripts, Nix didn't register a dependency on the store
paths properly. This meant that some of the derivations that were built,
didn't have the right store paths made available in the sandbox -
leading to all kinds of "not found" errors.

We worked around this in CI by resolving the symlinks to the nix store
beforehand. We tried to work around this locally by storing the nix
store path in BASELINE, but this didn't fully work. By explicitly
registering these store paths as dependencies, this should work across
the board - without any magic required by the caller.

(cherry picked from commit 45a765a282)
2025-09-10 17:29:07 +00:00
Ben Siraphob 8bb3b72d53
ci/eval/README.md: adjust wording
(cherry picked from commit 807ce4b7b3)
2025-09-09 10:06:12 +02:00
Philip Taron 2fbedb9402 ci: have eval.full return the report as displayed in CI
Update the README.md to document what gets returned.
We might in the future split these up into other attrsets but I don't see a usecase for that at the moment.

(cherry picked from commit 07916fc3fd)
2025-09-08 14:09:00 +00:00
Wolfgang Walther 579c4e6a60 ci/eval.full: allow local comparison with rebuilds
This allows running a full comparison between two commits locally.

What was previously `eval.full` is now called `eval.all`. The new
`eval.full` takes a `baseline` argument for the comparison.

(cherry picked from commit ccc12c839b)
2025-08-28 16:20:15 +00:00
Wolfgang Walther d9a6405eab ci/eval: clarify README with current defaults and memory requirements
We had set a default of 5000 for local evaluation earlier for
`singleSystem`, it makes sense to also use that for `full`.

The README is also a bit outdated, because Nix 2.30 significantly
changed the memory requirements. Rewriting the README to also show the
ability to directly evaluate the current system only.

(cherry picked from commit 0e07097947)
2025-08-28 16:20:15 +00:00
Wolfgang Walther 51f994d5b5
ci/eval/README.md: one sentence per line
(cherry picked from commit 60e35e4ded)
2025-06-29 21:35:58 +02:00
Wolfgang Walther 456a4697b1
workflows/eval: load supportedSystems from JSON file
This is a refactor to prepare the next commit. It doesn't do much on its
own, but is separated for ease of review.
2025-05-11 22:22:45 +02:00
Gaetan Lepage 6eadbf9c97 ci/eval: allow precisely choosing which systems to evaluate for (evalSystem -> evalSystems) 2024-12-14 21:42:36 +01:00
Silvan Mosberger fbbe972898 Parallel GH actions workflow for Nixpkgs eval
Motivated by ofborg struggling [1] and its evaluations taking too long,
inspired by Jörg's initial PR [2]
and Adam's previous attempt to parallelise Nixpkgs evaluation [3],
this PR contains initial work to relief ofborg from its evaluation duty
by using GitHub Actions to evaluate Nixpkgs.

For now this doesn't take care of all of what ofborg does, such as
requesting appropriate reviewers or labeling mass rebuilds, but this can
be follow-up work.

[1]: https://discourse.nixos.org/t/infrastructure-announcement-the-future-of-ofborg-your-help-needed/56025?u=infinisil
[2]: https://github.com/NixOS/nixpkgs/pull/352808
[3]: https://github.com/NixOS/nixpkgs/pull/269403

Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
Co-Authored-By: Adam Joseph <adam@westernsemico.com>
2024-11-20 10:35:56 +01:00