Based on the Nixpkgs used and the version of nixfmt-rfc-style in that
version, it's likely that not the correct version is used.
Update scripts should instead run within a Nixpkgs development shell
(`nix-shell`/`nix develop`/`direnv`), where the correct version of
`nixfmt` (although `treefmt` should be preferred) is always available.
Except:
- Instances in documentation, because people in older versions
can't switch to nixfmt yet due to it having pointed to nixfmt-classic
before
- In code that runs based on a CI Nixpkgs version, which is also a bit
older still
- In update script shebangs, because many of them don't pin Nixpkgs, and run
with whatever is in NIX_PATH (and it's not easy to fix this, see
https://github.com/NixOS/nixpkgs/issues/425551)
Currently, every package set consists of three commits, generated by
update-hackage.sh, update-stackage.sh and
regenerate-hackage-packages.sh, respectively. This is suboptimal, as it
necessarly causes intermediate states of Nixpkgs where the generated
hackage-packages.nix and all-cabal-hasehs and/or the hackage2nix
configuration files are out of sync. Ideally, running
regenerate-hackage-packages.sh is a no-op for every Nixpkgs revision.
This is achieved by adding a wrapper script, update-package-set.sh,
which runs the individual moving parts and commits the result.
Calling git commit with --edit here, allows the user to rephrase the
very nondescript default message, in doing so e.g. clarifying why the
regeneration was necessary etc. This should hopefully encourage better
commit mesages.
The “This commit was generated by …” message is very wordy and often
exceeds 72 characters which is a good (loose) target for wrapping lines
in commit messages.
This allows other scripts to detect whether anything changed without
resorting to git-diff(1): If nothing changed, stdout will be empty (i.e.
we now enforce that other messages go to stderr). To make sure that this
behavior is retained in the future, the scripts' behavior is briefly
documented in the files' header.
Since the shebang calls nix-shell, we can safely assume that Nix (Lix,
C++ Nix) is installed. Our scripts should support a wide enough range of
Nix versions so that using the “impure” version of the tool is not a
problem.
This works around #400784. My theory is that the Nix frontend commands
no longer work with older versions of the Nix daemon nor the Lix daemon
in our workloads.
The CI job calls this as trusted/maintainers/..., i.e. with a working
directory outside the checkout. The git commands inside the script
assume to be inside the checkout, though, so let's force that.
Bash will not propagate the exit code from a subshell within a
herestring, so the script silently passes when git throws an error
there. Re-arranging things a bit and an error will now be thrown.
Fix an issue in the vim-plugins-updater where it was trying to access the
'normalized_name' attribute on an HTTPResponse object, causing the updater
to crash with:
AttributeError: 'HTTPResponse' object has no attribute 'normalized_name'
The fix adds type checking to ensure we only access normalized_name on
Plugin objects, and properly handle other types like Exceptions and
HTTPResponse objects.
Signed-off-by: Austin Horstman <khaneliman12@gmail.com>
When updating GNOME packages in reverse topological order using the following command:
nix-shell maintainers/scripts/update.nix --arg predicate '(path: pkg: builtins.any (team: team.shortName == "GNOME") pkg.meta.teams or [])' --argstr order reverse-topological
the `update.py` script would crash with `ValueError` on calling `ordered.index("adwaita-fonts")`.
This happened because the `adwaita-fonts` does not depend on any other GNOME package, nor it is depended upon by one.
As a result, it had been discarded by the `reverse_edges` function.
Co-authored-by: Philip Taron <philip.taron@gmail.com>
We already have a generic keep-sorted workflow, so we don't need to roll
our own for the maintainers list.
This will result in a slightly different order, because the nix script
ignored "-" and "_", while keep-sorted sorts them first. But, the order
is consistent and that's what matters.
I've added the tests because I like the ability to check the behavior of
the plugin when run in the sandbox without my crazy config.
It also helps documenting how to check the plugin when running updates.
Previously, when updating multiple packages, we just updated them in arbitrary order. However, when some of those packages depended on each other, it could happen that some of the intermediary commits would not build because of version constraints on dependencies.
If we want each commit in the history to build when feasible, we need to consider four different scenarios:
1. Updated dependant is compatible with both the old and the new version of the dependency. Order of commits does not matter. But updating dependents first (i.e. reverse topological order) is useful since it allows building each package on the commit that updates it with minimal rebuilds.
2. Updated dependant raises the minimal dependency version. Dependency needs to be updated first (i.e. topological order).
3. Old dependant sets the maximal dependency version. Dependant needs to be updated first (i.e. reverse topological order).
4. Updated dependant depends on exact version of dependency and they are expected to be updated in lockstep. The earlier commit will be broken no matter the order.
This change allows selecting the order of updates to facilitate the first three scenarios. Since most package sets only have loose version constraints, the reverse topological order will generally be the most convenient. In major package set updates like bumping GNOME release, there will be exceptions (e.g. libadwaita typically requires GTK 4 from the same release) but those were probably in broken order before as well.
The downside of this feature is that it is quite slow – it requires instantiating each package and then querying Nix store for requisites.
It may also fail to detect dependency if there are multiple variants of the package and dependant uses a different one than the canonical one.
Testing with:
env GNOME_UPDATE_STABILITY=unstable NIX_PATH=nixpkgs=$HOME/Projects/nixpkgs nix-shell maintainers/scripts/update.nix --arg predicate '(path: pkg: path == ["gnome-shell"] || path == ["mutter"] || path == ["glib"] || path == ["gtk3"] || path == ["pango"] || path == ["gnome-text-editor"])' --argstr order reverse-topological --argstr commit true --argstr max-workers 4
