These dependencies need to be pulled from the same unstable commit that
cabal2nix is pulled from, otherwise changes to any of them will not be
in effect and possibly break the build.
The contribution guidelines require an unstable version to be leading
with a real version, or if none is available with `0-`.
This is because certain Nix operations split a package's full name into
name and version component starting with the first section starting with
a digit.
Hydra unilaterally removed hydra-eval-jobs, so we need to figure out how
to migrate to nix-eval-jobs. I can't shake the feeling that it's slower.
Maybe we need to increase the resource limitations for nix-eval-jobs.
nix-eval-jobs no longer produces a big JSON object, but instead one
object per line (one for each job). This is supported in a simple way by
readJSONLinesProcess. It'd be possible to implement this without
presupposing that there's one object per line, however, it is not an
usecase exactly intended by aeson, it seems.
nix-eval-jobs makes our job easier in some ways, e.g. jobs have a proper
meta set now, so we no longer need to cross reference a mail address to
github handle map. There is even room for further improvement, e.g.
attribute paths can just be queried instead of generating them using
Text.splitOn.
See also https://github.com/NixOS/hydra/pull/1421.
cabal2nix-unstable is mostly used for regenerating the Haskell package
set. Thus we should aim to make it quick to rebuild in case its hash
changes because of Haskell related changes.
- cabal2nix is not fussy about the Nix version it uses for nix-env(1)
and we can just assume it is already in PATH like we do for
maintainers/scripts/haskell/*.
- nix-prefetch-scripts causes the most trouble since its python
dependencies depend on pandoc, so many Haskell changes require
an additional Python rebuild when building cabal2nix-unstable.
nix-prefetch-scripts is most likely installed and not necessary in
many cases, e.g. hackage2nix doesn't need them which is the main use
we have for cabal2nix-unstable. For the update scripts that do need
them, we add them to the used nix-shell explicitly.
Based on the Nixpkgs used and the version of nixfmt-rfc-style in that
version, it's likely that not the correct version is used.
Update scripts should instead run within a Nixpkgs development shell
(`nix-shell`/`nix develop`/`direnv`), where the correct version of
`nixfmt` (although `treefmt` should be preferred) is always available.
Currently, every package set consists of three commits, generated by
update-hackage.sh, update-stackage.sh and
regenerate-hackage-packages.sh, respectively. This is suboptimal, as it
necessarly causes intermediate states of Nixpkgs where the generated
hackage-packages.nix and all-cabal-hasehs and/or the hackage2nix
configuration files are out of sync. Ideally, running
regenerate-hackage-packages.sh is a no-op for every Nixpkgs revision.
This is achieved by adding a wrapper script, update-package-set.sh,
which runs the individual moving parts and commits the result.
Calling git commit with --edit here, allows the user to rephrase the
very nondescript default message, in doing so e.g. clarifying why the
regeneration was necessary etc. This should hopefully encourage better
commit mesages.
The “This commit was generated by …” message is very wordy and often
exceeds 72 characters which is a good (loose) target for wrapping lines
in commit messages.
This allows other scripts to detect whether anything changed without
resorting to git-diff(1): If nothing changed, stdout will be empty (i.e.
we now enforce that other messages go to stderr). To make sure that this
behavior is retained in the future, the scripts' behavior is briefly
documented in the files' header.
Since the shebang calls nix-shell, we can safely assume that Nix (Lix,
C++ Nix) is installed. Our scripts should support a wide enough range of
Nix versions so that using the “impure” version of the tool is not a
problem.
This works around #400784. My theory is that the Nix frontend commands
no longer work with older versions of the Nix daemon nor the Lix daemon
in our workloads.
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build a08b3a4d19.tar.gz \
--argstr baseRev 78e9caf153
result/bin/apply-formatting $NIXPKGS_PATH
With cabal-install >= 3.12, we need to adjust our cabal-install overlay
once again.
- Due to the new dependency semaphore-compat, which appears to require
unix >= 2.8 it is very difficult to get to work for GHC < 9.6 (but
probably possible). Technically, using pkgs.cabal-install should
always be fine, so there's no strict need for cabal-install to work
with every GHC. Let's drop support for GHC < 9.6 for now.
- Make sure that cabal-install-solver also uses the latest version
always.
- Due to Cabal 3.12, we need to deviate from Stackage for
hackage-security. cabal-install does support the standard version of
resolv now, though.
This commit has been generated by maintainers/scripts/haskell/update-stackage.sh
and maintainers/scripts/haskell/regenerate-hackage-packages.sh.
Add capability to update to an out of date solver in update-stackage.sh.
In practice, almost all requests to Hydra take longer than the default
timeout of 30 seconds.
This commit bumps all requests to the max timeout of 15 minutes. This
should hopefully make the hdyra-report.hs script more reliable and fail
less.
This change adds a flag --slow to hydra-report.sh get-report which
causes it to fetch the cheap evaluation overview endpoint (which only
contains build ids and meta data). The gathered information is then used
to request each build's status individually instead of in bulk which is
very slow, but useful as a last resort if the bulk endpoint times out.
Since every failure in the jobset means one request to get the log when
generating the list of newly broken packages, we need to add an option
to disable log requesting in case a lot of new breakage needs to be
entered.
If we want to push only one branch, we'll have to specify branch and
remote explicitly. Pushing to origin doesn't work for everyone, since
some of us have a origin remote that can't be pushed to. Using plain
`git push` has the problem that it'll try pushing all checked out
branchs which fails e.g. if some branches (staging, staging-next, …) are
behind their remote counterparts.
The solution is to require everyone to configure a per branch pushRemote
for haskell-updates which will then be used by merge-and-open-pr.sh.
cabal-install 3.10 has some quirky new logic for config, cache, …
directory discovery. We reimplement this in this simple bash script,
additionally respecting the CABAL_DIR environment variable.
