This change adds a flag --slow to hydra-report.sh get-report which
causes it to fetch the cheap evaluation overview endpoint (which only
contains build ids and meta data). The gathered information is then used
to request each build's status individually instead of in bulk which is
very slow, but useful as a last resort if the bulk endpoint times out.
Since every failure in the jobset means one request to get the log when
generating the list of newly broken packages, we need to add an option
to disable log requesting in case a lot of new breakage needs to be
entered.
If we want to push only one branch, we'll have to specify branch and
remote explicitly. Pushing to origin doesn't work for everyone, since
some of us have a origin remote that can't be pushed to. Using plain
`git push` has the problem that it'll try pushing all checked out
branchs which fails e.g. if some branches (staging, staging-next, …) are
behind their remote counterparts.
The solution is to require everyone to configure a per branch pushRemote
for haskell-updates which will then be used by merge-and-open-pr.sh.
cabal-install 3.10 has some quirky new logic for config, cache, …
directory discovery. We reimplement this in this simple bash script,
additionally respecting the CABAL_DIR environment variable.
This update adds support for $CABAL_DIR as well as the new
$XDG_CACHE_HOME location of the hackage db.
Since we maintain hackage-db, having the latest version always is nice
even though it has more reverse dependencies than the other libraries we
maintain.
Add a newtype for a package name and a package set. This is less for
correctness, and more just to make the code a little easier to read
through without having to keep in mind what each Text refers to.
nixpkgs:trunk also builds aarch64-darwin these days, so this forces our
hand a little bit. We can still refuse to care about failures _too_
much, but at least we will stop merging as big a rebuilds as we are
currently.
`ghc-pkg list` tells us everything hackage2nix needs to know. In the
past the core-packages list and compiler setting in hackage2nix was
maintained manually which inevitably leads to it being forgot once in a
while – this will then mess with flag resolution when generating the
package set in some cases. Luckily, we can just let a simple derivation
do this for us.
Resolves#202621.
Since we now have a versioned configuration-ghc-*.nix file for GHC HEAD,
we don't need to add a super special case to the package set logic in
test-configurations.nix anymore. We can just create a versioned
attribute for the ghcHEAD package set (which is not exposed) and keep
using the normal discovery logic.
The only tricky bit is that GHC HEAD's configuration file is named after
the GHC release that will be branched off from it, so a little bit of
arithmetic is involved.
The Haskell Hydra report generator
(`maintainers/scripts/haskell/hydra-report.hs`) uses this
`maintainer-handles.nix` script for generating a mapping of email
addresses to GitHub handles.
This `maintainer-handles.nix` script is necessary because the Haskell
Hydra report generator gets Hydra job status info as input, but needs to
ping users on GitHub. Hydra job status info only contains user emails (not
GitHub handles). So the `maintainer-handles.nix` script is necessary
for looking up GitHub handles from email addresses.
This commit fixes the `maintainers-handles.nix` code to ignore
maintainers that don't have email addresses. The code was originally
assuming that all maintainers have email addresses, but there was
recently a maintainer added without an email address.
- use `restrict-eval` so that we're not affected by the user's environment
- use jq instead of the horrible echo+sed hack
The second point also fixes the indentation before each line to be two
spaces instead of one, so I set it back to one space to avoid a diff.
Accept either Nightly or LTS as the solver configuration variable in the
script. The Stackage version is now considered a tuple of solver and
version, allowing the script to handle updates and switches between
solvers gracefully.
Tested updating Nightly and updating from Nightly to LTS.
We have generally shipped the latest cabal-install version. Stackage has
re-added cabal-install recently which caused cabal-install to get
downgraded to 3.4 to match the Cabal version shipped by GHC 9.0.2. This
commit reverts that change.
configuration-nix.nix uses builtins.intersectAttrs to not any overrides
for packages not present in `super` (presumably for use outside of
nixpkgs?). To accomodate it, we pass an attribute set with every
attribute of haskellPackages, but set to `null` as `super`, and — while
we're at it — a fix point as `self`.
While being able to test them is neat (on x86_64-linux they work very
well, actually), we usually don't want to do this, since the set is
only (recommended to be) used to bootstrap GHC. Consequently there is
almost no binary cache and testing them mostly leads to unenlightening
and seemingly endless compilation.
The added nix expression allows maintainers to check for regressions in
the configuration overlays employed by haskellPackages and friends. The
reasoning behind this is that, if we add an override for something, it
should also build. To test this fact, we extract all attributes touched
by a configuration and obtain all relevant derivations corresponding to
it which can then be thrown into nix-build --keep-going.
I've been using this expression to verify configuration-ghc-9.2.x.nix
for a week or so which works quite well. The amount of stale overrides
in other configuration makes it a bit more painful for other use cases
at the moment.
Allow setting the HACKAGE2NIX environment variable to change the
hackage2nix the script uses to an absolute path to a local build. Useful
to test local changes in a realistic environment.
We split configuration-hackage2nix.yaml into multiple files. We bump
cabal2nix-unstable to get support for multiple config files in
hackage2nix.
* The file main.yaml is only supposed to be edited by humans.
* The file stackage.yaml is only supposed to be updated by the
update-stackage.sh
* The file broken.yaml can be edited by humans, but probably future
helpers will want to insert broken packages into this file based on
hydra reports.
* The file transitive-broken.yaml is newly introduced to be generated
by regenerate-transitive-broken-packages.sh
regenerate-transitive-broken-packages.sh makes a nix query (in
transitive-broken-packages.nix) which evaluates all haskellPackages
once with and once without "allowBroken" this way it get's a list of
packages which are broken by some transitive dependency, but does not
disable packages which have eval errors not caused by a broken package.
This is the preferred format for things fetched from git or similar that
are not proper releases: https://nixos.org/manual/nixpkgs/stable/#sec-package-naming
Also we should think about making name and attribute name more
consistent: cabal2nix-latest -> cabal2nix-unstable
Introduces a script that can be used to update the Nix expressions for
the Haskell package set. In service of that, also
- introduces cabal2nix-latest, which pins the hackage2nix version used
- changes all-cabal-hashes to use fetchFromGitHub
- adds update-hackage.sh & update-cabal2nix-latest.sh & update-stackage.sh maintainer scripts
