This hopefully makes it easier to re-write this script in a language
that people understand. Because it's shelling out, it's likely slower
but hopefully still fast enough for our purposes.
These dependencies need to be pulled from the same unstable commit that
cabal2nix is pulled from, otherwise changes to any of them will not be
in effect and possibly break the build.
(cherry picked from commit 631661e66c)
The contribution guidelines require an unstable version to be leading
with a real version, or if none is available with `0-`.
This is because certain Nix operations split a package's full name into
name and version component starting with the first section starting with
a digit.
(cherry picked from commit 453d0f8eab)
This is a new flag introduced in `nix-eval-jobs`, which reduces runtime
for the `ping-maintainers` subcommand from 12 minutes to 3 minutes for
me.
The `--constituents` flag is not supported anymore with
`--no-instantiate`, but it doesn't seem to make a difference anyway.
Same for `--max-jobs` - I don't get a difference with or without it.
Removing the `foldl'` import because it is warned to be redundant after
switching to import the current nixpkgs instance, likely due to a newer
GHC version (?).
Also, rearrange the GHC‐related release notes to be in order of
most likely to matter to anyone.
Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
These dependencies need to be pulled from the same unstable commit that
cabal2nix is pulled from, otherwise changes to any of them will not be
in effect and possibly break the build.
Stackage LTS 24 contains git-annex (contrary to LTS 23). I believe this
resulted in a downgrade of git-annex when switching to LTS 23. We'll
want to continue using the latest version of git-annex. At the very
least, this will fix the trouble we're having with its test suite on
haskell-updates.
The contribution guidelines require an unstable version to be leading
with a real version, or if none is available with `0-`.
This is because certain Nix operations split a package's full name into
name and version component starting with the first section starting with
a digit.
Hydra unilaterally removed hydra-eval-jobs, so we need to figure out how
to migrate to nix-eval-jobs. I can't shake the feeling that it's slower.
Maybe we need to increase the resource limitations for nix-eval-jobs.
nix-eval-jobs no longer produces a big JSON object, but instead one
object per line (one for each job). This is supported in a simple way by
readJSONLinesProcess. It'd be possible to implement this without
presupposing that there's one object per line, however, it is not an
usecase exactly intended by aeson, it seems.
nix-eval-jobs makes our job easier in some ways, e.g. jobs have a proper
meta set now, so we no longer need to cross reference a mail address to
github handle map. There is even room for further improvement, e.g.
attribute paths can just be queried instead of generating them using
Text.splitOn.
See also https://github.com/NixOS/hydra/pull/1421.
cabal2nix-unstable is mostly used for regenerating the Haskell package
set. Thus we should aim to make it quick to rebuild in case its hash
changes because of Haskell related changes.
- cabal2nix is not fussy about the Nix version it uses for nix-env(1)
and we can just assume it is already in PATH like we do for
maintainers/scripts/haskell/*.
- nix-prefetch-scripts causes the most trouble since its python
dependencies depend on pandoc, so many Haskell changes require
an additional Python rebuild when building cabal2nix-unstable.
nix-prefetch-scripts is most likely installed and not necessary in
many cases, e.g. hackage2nix doesn't need them which is the main use
we have for cabal2nix-unstable. For the update scripts that do need
them, we add them to the used nix-shell explicitly.
Update scripts aren't always run in the development shell by default,
especially not in the regular automated updates.
The parent commit makes the update scripts depend on running in the
development shell, so let's make sure it's always done.
Based on the Nixpkgs used and the version of nixfmt-rfc-style in that
version, it's likely that not the correct version is used.
Update scripts should instead run within a Nixpkgs development shell
(`nix-shell`/`nix develop`/`direnv`), where the correct version of
`nixfmt` (although `treefmt` should be preferred) is always available.
Except:
- Instances in documentation, because people in older versions
can't switch to nixfmt yet due to it having pointed to nixfmt-classic
before
- In code that runs based on a CI Nixpkgs version, which is also a bit
older still
- In update script shebangs, because many of them don't pin Nixpkgs, and run
with whatever is in NIX_PATH (and it's not easy to fix this, see
https://github.com/NixOS/nixpkgs/issues/425551)
Currently, every package set consists of three commits, generated by
update-hackage.sh, update-stackage.sh and
regenerate-hackage-packages.sh, respectively. This is suboptimal, as it
necessarly causes intermediate states of Nixpkgs where the generated
hackage-packages.nix and all-cabal-hasehs and/or the hackage2nix
configuration files are out of sync. Ideally, running
regenerate-hackage-packages.sh is a no-op for every Nixpkgs revision.
This is achieved by adding a wrapper script, update-package-set.sh,
which runs the individual moving parts and commits the result.
Calling git commit with --edit here, allows the user to rephrase the
very nondescript default message, in doing so e.g. clarifying why the
regeneration was necessary etc. This should hopefully encourage better
commit mesages.
The “This commit was generated by …” message is very wordy and often
exceeds 72 characters which is a good (loose) target for wrapping lines
in commit messages.
This allows other scripts to detect whether anything changed without
resorting to git-diff(1): If nothing changed, stdout will be empty (i.e.
we now enforce that other messages go to stderr). To make sure that this
behavior is retained in the future, the scripts' behavior is briefly
documented in the files' header.
Since the shebang calls nix-shell, we can safely assume that Nix (Lix,
C++ Nix) is installed. Our scripts should support a wide enough range of
Nix versions so that using the “impure” version of the tool is not a
problem.
This works around #400784. My theory is that the Nix frontend commands
no longer work with older versions of the Nix daemon nor the Lix daemon
in our workloads.
