Philip Taron
3d0e2841b8
ci.eval.compare: keep warnings as warnings rather than raising them as errors
...
This reverts commit f8210561f3 (ci.eval.compare: turn warnings into errors, 2025-09-16).
It turns out that there are normal math warnings and we don't want to block CI on the math coming out wrong.
2025-09-18 08:37:36 -07:00
Philip Taron
d80d4a77b7
ci.eval.compare: split out equivalent values into their own table
2025-09-17 14:04:28 -07:00
Philip Taron
fb1647ec6e
ci.eval.compare: explain the various metrics under the --explain flag
2025-09-17 14:04:28 -07:00
Philip Taron
c9860ef95c
ci.eval.compare: remove the duplicate cpuTime key
2025-09-17 14:04:28 -07:00
Philip Taron
2817f79649
ci.eval.compare: put things with counts together
2025-09-17 14:04:28 -07:00
Philip Taron
210e3e1151
ci.eval.compare: put things with bytes together
2025-09-17 14:04:28 -07:00
Philip Taron
3edc1e204e
ci.eval.compare: make the table format using tabulate not manually
2025-09-17 14:04:28 -07:00
Philip Taron
4eaa094f20
ci.eval.compare: sort time metrics first, then GC metrics, then everything else
2025-09-17 14:04:28 -07:00
Philip Taron
e83e900874
ci.eval.compare: assert types in flatten_data
2025-09-17 14:04:28 -07:00
Philip Taron
9959a4e507
ci.eval.compare: delete unreferenced global
2025-09-17 14:04:28 -07:00
Philip Taron
e4101ea3a9
ci.eval.compare: instead of manually tabulating, use tabulate
2025-09-17 14:04:28 -07:00
Philip Taron
7818a245f7
ci.eval.compare: support passing single files to cmp-stats
2025-09-17 14:04:27 -07:00
Philip Taron
4bc54e7a3a
ci.eval.compare: allow before_vals == 1 but avoid the t-test
2025-09-17 14:04:27 -07:00
Philip Taron
f8210561f3
ci.eval.compare: turn warnings into errors
...
This helps detect my math errors
2025-09-17 14:04:27 -07:00
Philip Taron
4c7ec9bf20
ci.eval.compare: require the directories to exist (they always should)
2025-09-17 14:04:27 -07:00
Philip Taron
241bb94b64
ci.eval.compare: use argument parsing instead of environment variables to pass before/after to cmp-stats.py
2025-09-17 14:04:27 -07:00
Philip Taron
4c2c6d9c43
ci.eval.compare: extract main function in cmp-stats.py
2025-09-17 14:04:27 -07:00
Philip Taron
6900cf62ee
ci.eval.compare: sort imports in cmp-stats.py
2025-09-17 14:04:27 -07:00
Philip Taron
a3c01782d3
ci.eval.compare: format cmp-stats.py with ruff
2025-09-17 14:04:27 -07:00
Johannes Kirschbauer
4f64ebd02b
ci/eval-stats: sort output table by metric name
2025-05-07 18:05:07 +02:00
Johannes Kirschbauer
0d584f7c8f
ci/compare: nix stats comparison
...
Displays stats table in the step-summary if there are no added/removed packages
2025-05-06 21:49:18 +02:00
Johannes Kirschbauer
a3520e9577
Revert "ci/compare: Bring back nix stats comparison"
2025-05-01 23:02:38 +02:00
Johannes Kirschbauer
78e8992345
ci/compare: nix stats comparison
...
Displays stats table in the step-summary if there are no added/removed packages
2025-05-01 22:41:12 +02:00