Skip to content

Move supported toplevel toolchains from eb_hooks.py into a separate JSON for easier define-once-and-reuse - #312

Open
casparvl wants to merge 3 commits into
EESSI:mainfrom
casparvl:separate-supported-toolchains
Open

casparvl wants to merge 3 commits into
EESSI:mainfrom
casparvl:separate-supported-toolchains

Conversation

@casparvl

@casparvl casparvl commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

The toplevel toolchains supported per EESSI version (EESSI_SUPPORTED_TOP_LEVEL_TOOLCHAINS) are defined inside eb_hooks.py. Other scripts that need this list (e.g. the native compiler flags check in #311 ) would have to parse or import the hooks file, and importing it requires EasyBuild. This PR moves the list into eessi_supported_toolchains.json, so the list stays in one place and any script can read it.

  • eessi_supported_toolchains.json sits next to eb_hooks.py, and install_scripts.sh installs it next to it in <prefix>/init/easybuild/. eb_hooks.py locates it relative to its own file. This works regardless of whether the eb_hooks.py file is used from it's installed location in the CVMFS repo, or from a clone of the software-layer-scripts repository, since every EasyBuild version in use loads the hooks via importlib's spec_from_file_location, which sets __file__. If the file can't be found or parsed, an EasyBuildError names the expected location.
  • Toolchains that need a minimum EasyBuild version (lfoss/2025b from 5.2.0, rompi/2025a from 5.3.1) have a min_easybuild_version field instead of being appended conditionally in code.
  • CI: test-eb-hooks.yml also checks that the deployed eessi_supported_toolchains.json matches the repo, like the existing check for eb_hooks.py.

Testing done (by AI). Loaded through EasyBuild's own hooks loader, EESSI_SUPPORTED_TOP_LEVEL_TOOLCHAINS is identical before and after this change under EasyBuild 4.9.4, 5.2.1 and 5.4.0, which covers both version guards. With eb --stop fetch in EESSI 2025.06 (EasyBuild 5.4.0), the toolchain check still accepts M4-1.4.19-GCCcore-14.2.0 and CDO-2.5.3-lompi-2025b, and rejects M4-1.4.19-GCCcore-12.3.0. That holds both for the repo's eb_hooks.py and for a copy installed with install_scripts.sh into a scratch prefix.

Note: anyone pointing EASYBUILD_HOOKS at their own copy of eb_hooks.py now needs eessi_supported_toolchains.json next to it.

Deploy required

Note that since this PR changes eb_hooks.py, it needs to be deployed (only once per EESSI version, I believe, since it's installed into <EESSI_VERSION>/init/easybuild).

AI disclosure

This change was made with an AI coding assistant (Claude, via Claude Code), as a spin-off of the native compiler flags check PR (#311). I asked for the supported-toolchains list to be moved out of eb_hooks.py into an easily parseable file used by both eb_hooks.py and the new check, with the requirement that it works both from a clone of this repository and from the copy installed in CVMFS.

The assistant:

  • checked in the EasyBuild source how the hooks file is loaded;
  • implemented the change;
  • verified it as described above, recording the old behaviour before changing anything.

At my request it then split this off from the larger PR into this preparatory one. I reviewed the result before opening this PR.

🤖 Generated with Claude Code

…SON file

The supported toplevel toolchains per EESSI version are now defined in
eessi_supported_toolchains.json rather than in eb_hooks.py itself, so that
they can also be used by other scripts without having to parse (or import)
the hooks file.

- eessi_supported_toolchains.json is located next to eb_hooks.py, both in
  the repository and when installed in <prefix>/init/easybuild/ by
  install_scripts.sh. eb_hooks.py locates it relative to its own location.
- Toolchains that can only be installed with a recent enough EasyBuild
  version (lfoss/2025b, rompi/2025a) now specify 'min_easybuild_version'
  instead of being appended conditionally in eb_hooks.py.
- CI also checks that the deployed eessi_supported_toolchains.json is
  up-to-date, like is done for eb_hooks.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@trz42 trz42 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small and simple change.

Find it a little unsatisfactory for being generated by an AI-assistant:

  • duplicated code in the test (and name of test file is a bit off now)
  • it could have added some unit tests for the function and JSON format
  • would have been nice with some specific instructions on how this was tested (command log)
  • PR title is repeated in the PR description

Comment thread install_scripts.sh
Comment thread .github/workflows/test-eb-hooks.yml Outdated
Comment thread eb_hooks.py Outdated
Comment thread eb_hooks.py Outdated
Comment thread eb_hooks.py Outdated
- Allow setting the location of the supported toolchains file through
  EESSI_SUPPORTED_TOOLCHAINS_FILE (default: next to eb_hooks.py), with
  specific errors for a missing file and for invalid JSON
- Document return value of load_supported_top_level_toolchains()
- Rename hook_files to easybuild_init_files in install_scripts.sh
- Rename test-eb-hooks.yml to test-eb-init-files.yml and check all init
  files in a single loop instead of duplicating the step
- Add unit tests for the JSON format and loader, run in CI only when
  relevant files change

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@casparvl

casparvl commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

PR title is repeated in the PR description

I haven't linked my GH to Claude, so I'm still manually creating the PRs, and the opening post was a copy-paste. So this is really my fault, not Claude's ;-)

Only the structure of the shipped eessi_supported_toolchains.json is
tested now; the behaviour of load_supported_top_level_toolchains() is
tested using JSON files created on the fly, so the toolchain content is
not duplicated in the tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@casparvl

casparvl commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

it could have added some unit tests for the function and JSON format

Unit tests were added. Some may be a bit overkill, but at least they check things like: does the load-function in eb_hooks work, is the syntax of the json correct, can we succesfully override the location with the env var, etc. It even tests that some of the error messages contain certain information, probably because I asked the AI quite explicitely to include that information in the return message. I'm not sure if we should have actual tests for that - but it doesn't hurt either.

@casparvl
casparvl requested a review from trz42 October 5, 2026 15:18

@trz42 trz42 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants