Commit graph

135 commits

Author SHA1 Message Date
Wolfgang Walther b5af85e10e
Revert "First-class GitHub team reviews" 2025-10-28 07:38:20 +00:00
Silvan Mosberger e6c822f51f workflows/reviewers: First-class team package maintainer review requests 2025-10-28 01:35:25 +01:00
Matt Sturgeon 9ea88ca354
ci/eval/compare/maintainers: simplify meta.position predicate 2025-10-22 21:11:19 +01:00
Wolfgang Walther 6d564102b7
ci/eval/compare/maintainers: fix maintainer pings without meta.position
In a recent change, the path matching was simplified in maintainers.nix.
This revealed a pre-existing logic bug: Packages without `meta.position`
would get an empty string as their file name. The change would then
cause this empty string to always be matched, which lead to maintainer
pings for these packages in seemingly random PRs, when some of their
dependencies were changed.
2025-10-22 21:06:53 +02:00
Wolfgang Walther 6a26c66335
ci/eval/compare/maintainers: disable aliases
We should never try to ping maintainers through package aliases, this
can only lead to errors. One example case is, where an attribute is a
throw alias, but then re-introduced in a PR. This would trigger the
throw. By disabling aliases, we can fallback gracefully.
2025-10-22 13:33:41 +02:00
Wolfgang Walther 2b7ea9379a
ci/eval: disable GC warning
This came up more often recently, and breaks Eval because it prints on
stderr.
2025-10-18 14:06:09 +02:00
Wolfgang Walther f33892fcc9
ci/eval/compare/maintainers: ping by-name reviewers for pure refactors
This allows requesting reviewers for pure refactor PRs, which don't
cause a rebuild of the package. This is only possible for by-name,
because only here the package names can be inferred from the filenames.
2025-10-17 16:37:14 +02:00
Wolfgang Walther 0a5ffcdf31
ci/eval/compare/maintainers: ping by-name reviewers for all rebuilds
This adds support to ping maintainers when arbitrary files in by-name
are changed, as long as they still cause a rebuild. For example, this is
the case when changing .json files with version metadata. These were
previously not detected as belonging to the package, and didn't cause
maintainer pings.
2025-10-17 16:37:09 +02:00
Wolfgang Walther 7c865ddd46
ci/eval/compare/maintainers: refactor path matching
The only reason for the additional `lib.hasSuffix` check was, that the
`lib.removePrefix` was broken - it was never adjusted when porting this
from ofborg, so the relative path was wrong and no prefix ever removed,
since no packages are in `ci/`.

This additionally strips the leading `/`, so that `relevantFilenames`
will then have paths starting with `pkgs/...`, similar to how git
reports those paths in the `changedpathsjson` file. This allows simple
equality comparison.
2025-10-17 16:37:04 +02:00
Wolfgang Walther 79029e39a4
ci/eval/compare/maintainers: refactor using lib
Removes unused code and comments, uses shorter lib over builtins and
switches to `lib.importJSON`.

Readability is key.
2025-10-17 16:37:00 +02:00
Wolfgang Walther 44e4c1ba9f
maintainers/scripts/get-maintainer-pings-between: drop
This has been broken for months, since all the interface to `ci/eval`
changed. Nobody noticed, so nobody is using it.
2025-10-16 13:36:42 +02:00
Vladimír Čunát dd51f936da
Merge branch 'master' into staging-next
lixPackageSets.git conflicted via 9c29a340be + eabfa160aa
I think, hopefully good now.
2025-10-05 08:18:28 +02:00
NAHO dec908b169
treewide: remove optional builtins prefixes from prelude functions
Remove optional builtins prefixes from prelude functions by running:

    builtins=(
      abort
      baseNameOf
      break
      derivation
      derivationStrict
      dirOf
      false
      fetchGit
      fetchMercurial
      fetchTarball
      fetchTree
      fromTOML
      import
      isNull
      map
      null
      placeholder
      removeAttrs
      scopedImport
      throw
      toString
      true
    )

    fd --type file --exec-batch sed --in-place --regexp-extended "
      s/\<builtins\.($(
        printf '%s\n' "${builtins[@]}" |
          paste --delimiter '|' --serial -
      ))\>/\1/g
    "

    nix fmt
2025-09-30 09:05:08 +02:00
nixpkgs-ci[bot] 15409f9947
Merge staging-next into staging 2025-09-18 18:06:41 +00:00
Philip Taron 3d0e2841b8
ci.eval.compare: keep warnings as warnings rather than raising them as errors
This reverts commit f8210561f3 (ci.eval.compare: turn warnings into errors, 2025-09-16).

It turns out that there are normal math warnings and we don't want to block CI on the math coming out wrong.
2025-09-18 08:37:36 -07:00
Philip Taron d80d4a77b7
ci.eval.compare: split out equivalent values into their own table 2025-09-17 14:04:28 -07:00
Philip Taron fb1647ec6e
ci.eval.compare: explain the various metrics under the --explain flag 2025-09-17 14:04:28 -07:00
Philip Taron c9860ef95c
ci.eval.compare: remove the duplicate cpuTime key 2025-09-17 14:04:28 -07:00
Philip Taron 2817f79649
ci.eval.compare: put things with counts together 2025-09-17 14:04:28 -07:00
Philip Taron 210e3e1151
ci.eval.compare: put things with bytes together 2025-09-17 14:04:28 -07:00
Philip Taron 3edc1e204e
ci.eval.compare: make the table format using tabulate not manually 2025-09-17 14:04:28 -07:00
Philip Taron 4eaa094f20
ci.eval.compare: sort time metrics first, then GC metrics, then everything else 2025-09-17 14:04:28 -07:00
Philip Taron e83e900874
ci.eval.compare: assert types in flatten_data 2025-09-17 14:04:28 -07:00
Philip Taron 9959a4e507
ci.eval.compare: delete unreferenced global 2025-09-17 14:04:28 -07:00
Philip Taron e4101ea3a9
ci.eval.compare: instead of manually tabulating, use tabulate 2025-09-17 14:04:28 -07:00
Philip Taron 7818a245f7
ci.eval.compare: support passing single files to cmp-stats 2025-09-17 14:04:27 -07:00
Philip Taron 4bc54e7a3a
ci.eval.compare: allow before_vals == 1 but avoid the t-test 2025-09-17 14:04:27 -07:00
Philip Taron f8210561f3
ci.eval.compare: turn warnings into errors
This helps detect my math errors
2025-09-17 14:04:27 -07:00
Philip Taron 4c7ec9bf20
ci.eval.compare: require the directories to exist (they always should) 2025-09-17 14:04:27 -07:00
Philip Taron 241bb94b64
ci.eval.compare: use argument parsing instead of environment variables to pass before/after to cmp-stats.py 2025-09-17 14:04:27 -07:00
Philip Taron 2fe7b1cec2
ci.eval.compare: extract a derivation for cmp-stats
It's not very useful yet.
2025-09-17 14:04:27 -07:00
Philip Taron 4c2c6d9c43
ci.eval.compare: extract main function in cmp-stats.py 2025-09-17 14:04:27 -07:00
Philip Taron 6900cf62ee
ci.eval.compare: sort imports in cmp-stats.py 2025-09-17 14:04:27 -07:00
Philip Taron a3c01782d3
ci.eval.compare: format cmp-stats.py with ruff 2025-09-17 14:04:27 -07:00
nixpkgs-ci[bot] 6ec57b45c7
Merge staging-next into staging 2025-09-15 16:09:34 +00:00
Wolfgang Walther e5ce5c2500
ci/eval/compare: add 10.rebuild-nixos-tests label (#439255) 2025-09-15 14:29:47 +00:00
nixpkgs-ci[bot] 96acd252c7
Merge staging-next into staging 2025-09-12 00:16:59 +00:00
Wolfgang Walther b9d4098221
ci/eval: fix local full eval
The change to use `builtins.storePath` was good - for when the store
path *is* already part of the nix store. In all my tests so far, that
was already the case, because I was iterating on the solution and the
Eval results stayed the same.

But when this is run on a entirely new commit, these the values for
`afterDir` and `combinedDir` are *not* in the store, yet. As part of
running `eval.full` on a new commit they will be created. `eval.full` is
linked up, so that the values passed around there will actually be
derivations, which might not be realized, yet.

Checking whether the input is a path or not fixes this for both cases.
2025-09-11 20:18:48 +02:00
nixpkgs-ci[bot] 3e0c2daff8
Merge staging-next into staging 2025-09-10 18:06:27 +00:00
Wolfgang Walther 45a765a282
ci/eval: fix local comparison with baseline
Due to how we pass in existing store paths via CLI arguments for the
diff and combine scripts, Nix didn't register a dependency on the store
paths properly. This meant that some of the derivations that were built,
didn't have the right store paths made available in the sandbox -
leading to all kinds of "not found" errors.

We worked around this in CI by resolving the symlinks to the nix store
beforehand. We tried to work around this locally by storing the nix
store path in BASELINE, but this didn't fully work. By explicitly
registering these store paths as dependencies, this should work across
the board - without any magic required by the caller.
2025-09-10 14:35:46 +02:00
Wolfgang Walther b9ec8bed14
treewide: fix typos in md files (#439306) 2025-09-09 08:03:31 +00:00
Ben Siraphob 807ce4b7b3
ci/eval/README.md: adjust wording 2025-09-09 13:14:36 +07:00
Philip Taron 07916fc3fd
ci: have eval.full return the report as displayed in CI
Update the README.md to document what gets returned.
We might in the future split these up into other attrsets but I don't see a usecase for that at the moment.
2025-09-08 06:58:24 -07:00
Wolfgang Walther 63e6fc4c43
ci/eval/compare: add 10.rebuild-nixos-tests label
This indicates that the NixOS test-driver changed and all NixOS tests
have to be rebuilt. It can be used to either re-target to staging or to
batch this with other similar changes, at least.
2025-09-04 21:27:30 +02:00
Wolfgang Walther e3f583c5bf
ci/eval: eval nixosTests.simple
We can't eval all nixos tests, this would be way too expensive. But we
can evaluate `nixosTests.simple` as an indication whether the nixos test
driver has changed. If that's the case, this means that *all* NixOS
tests need to be rebuilt.
2025-09-03 18:35:01 +02:00
Wolfgang Walther 5c4608656e
top-level/release-outpaths: move to ci/eval & refactor (#439221) 2025-09-03 16:27:37 +00:00
Wolfgang Walther f2ca5796de
ci/eval/compare: handle missing packages
Not all packages that are reported as changed will actually exist on the
platform that the maintainers are colleted on.

This is the case for some attributes that are only available on Darwin
or explicitly set to `null` on Linux. By filtering out packages without
maintainers, these are ignored - and we should potentially get a small
performance improvement as well.
2025-09-01 15:50:05 +02:00
Wolfgang Walther 8104100101
ci/eval/attrpaths: update cross stdenvs
`pkgsArocc` and `pkgsZig` had been added to `release.nix`, so should be
listed here as well.
2025-08-31 21:09:31 +02:00
Wolfgang Walther d0c0b875f7
treewide: remove __recurseIntoDerivationForReleaseJobs
This attribute was supposed to be set on derivations, to make the
release tools recurse into them. The remaining uses were all on regular
attrsets, though, so this is safe to remove.
2025-08-31 20:54:49 +02:00
Wolfgang Walther b627d181e9
ci/eval: remove unused checkMeta argument
This should always be set anyways.
2025-08-31 12:16:28 +02:00