Skip to content

Add WRF-Chem Apptainer container definition files#95

Merged
georgemccabe merged 8 commits into
mainfrom
feature_76_wrf_chem_container
May 1, 2026
Merged

Add WRF-Chem Apptainer container definition files#95
georgemccabe merged 8 commits into
mainfrom
feature_76_wrf_chem_container

Conversation

@vweeks

@vweeks vweeks commented Feb 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

  • Adds Apptainer .def files for building WRF-Chem 4.7.1 / WPS 4.6.0 containers on AlmaLinux 9, supporting both single-layer (distribution) and multi-layer (development) builds
  • Includes all WRF-Chem preprocessing tools (mozbc, anthro_emis, fire_emis, megan_bio_emiss, wesely, exo_coldens)
  • Adds Dockerfile equivalent of the full single-layer Apptainer build using multi-stage Docker build
  • Adds README with build instructions, HPC (Derecho) usage examples, container layout, and known issues (including WRF 4.7.1 KPP compile bug patches)
  • Adds GitHub Actions workflow (.github/workflows/build-wrf-chem-docker.yml) that builds the Docker image on ubuntu-latest and publishes it to GHCR on merges to main and on wrf-chem-v* tags. PRs that touch docker/wrf-chem/** build-only to validate the Dockerfile without publishing.

Closes #76

Docker build workflow

Because macOS (Apple Silicon) can't realistically produce an amd64 WRF-Chem image locally, the Docker build is delegated to GitHub Actions:

  • Runner: ubuntu-latest (4 vCPU / 16 GB RAM / ~14 GB free disk).
  • Disk prep: an inline cleanup step removes preinstalled Android SDK, .NET, Haskell, CodeQL, etc. to reclaim ~30 GB — the full build otherwise exceeds the default free space.
  • Build driver: docker buildx with a docker-container builder (required for advanced cache backends).
  • Caching: registry-backed — layer cache is stored as a sibling OCI tag ghcr.io/ncar/i-wrf/wrf-chem:buildcache (type=registry,mode=max). First build is cold; subsequent builds skip cached stages.
  • Auth: uses the built-in GITHUB_TOKEN with packages: write; no separate secrets to manage.
  • Tag derivation (inline shell, since docker/metadata-action is blocked by NCAR's third-party-action policy):
    • push to main → :main, :latest, :sha-<short>
    • tag wrf-chem-vX.Y.Z → :X.Y.Z, :latest
    • workflow_dispatch → :manual-<short>
    • PR (build-only, not pushed) → :pr-<num>
  • Policy compliance: uses only actions/checkout@v4 (GitHub-owned). All other steps are raw shell / docker CLI to satisfy NCAR's "actions must be from a repository owned by NCAR or created by GitHub" restriction.
  • Result: first cold build of the full multi-stage image (AlmaLinux 9 → Dev Tools + NetCDF/HDF5 → WRF-Chem 4.7.1 + WPS 4.6.0 + preprocessing tools) completes well under the 6-hour job limit.

Test plan

  • Build multi-layer containers (deps → preproc → wrf-chem) and verify each layer succeeds
  • Verify WRF-Chem executables are present and functional inside container (wrf.exe, real.exe, preprocessing tools)
  • Test running on Derecho with bind mounts and MPI
  • Build full single-layer Apptainer container: apptainer build --fakeroot wrf-chem_full_almalinux9.sif wrf-chem_full_almalinux9.def
  • Build Docker image via GitHub Actions workflow on PR (build-only, no push)
  • Post-merge: confirm GitHub Actions run on main pushes image to ghcr.io/ncar/i-wrf/wrf-chem:main / :latest and seeds :buildcache
  • Tag wrf-chem-v4.7.1 to publish a versioned image ghcr.io/ncar/i-wrf/wrf-chem:4.7.1

🤖 Generated with Claude Code

@vweeks @claude
Add WRF-Chem Apptainer container definition files (#76)
2555877 
Add Apptainer .def files for building WRF-Chem containers on AlmaLinux 9,
including dependency, preprocessing, WRF-Chem, and full build definitions,
along with a README documenting usage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vweeks vweeks requested a review from georgemccabe February 20, 2026 19:47
@vweeks vweeks self-assigned this Feb 20, 2026
vweeks and others added 6 commits February 20, 2026 13:18
@vweeks @claude
Add Dockerfile equivalent of WRF-Chem full Apptainer definition
549a514 
Multi-stage Docker build translating wrf-chem_full_almalinux9.def for
environments where Docker is preferred over Apptainer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@georgemccabe
add --allowerasing argument to dnf install to prevent error with libc…
9a3abe0 
…url and libcurl-minimal conflict
@vweeks @claude
Fix WRF-Chem Dockerfile so WPS can find WRF build artifacts
62e6529 
Build WRF directly in /opt/WRF instead of /tmp/build with a symlink,
so WPS configure/link finds external/io_netcdf/libwrfio_nf.a under
the WRF_DIR path it expects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vweeks @claude
Add GitHub Actions workflow to build WRF-Chem Docker image
9c87079 
Builds the docker/wrf-chem image on ubuntu-latest and pushes to
ghcr.io/ncar/i-wrf/wrf-chem on pushes to main and on wrf-chem-v* tags.
PRs that touch docker/wrf-chem build-only to validate the Dockerfile
without publishing a package version. Uses BuildKit GHA layer cache
(mode=max) so incremental builds skip the expensive WRF/WPS compiles,
and runs a disk-cleanup step up front to stay under the runner's
~14 GB free-space ceiling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vweeks @claude
Rewrite WRF-Chem build workflow using only actions/checkout
6103c00 
NCAR's org policy restricts third-party actions, so replace
docker/setup-buildx-action, docker/login-action, docker/metadata-action,
docker/build-push-action, and jlumbroso/free-disk-space with inline
shell equivalents. Layer cache moves from type=gha to type=registry
(sibling :buildcache tag in GHCR), which needs no runtime tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vweeks @claude
Lowercase image name and opt into Node 24 for WRF-Chem build
f787b7d 
OCI registries reject uppercase characters in repository paths, so the
NCAR owner segment must be lowercased before use. GitHub Actions
expressions have no lowercase function, so derive IMAGE via bash
parameter expansion in a dedicated step. Also silence the Node 20
deprecation warning by opting into Node 24 for JavaScript actions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vweeks

vweeks commented Apr 23, 2026

Copy link
Copy Markdown
Collaborator Author

@georgemccabe The docker build as a GitHub Action was successful. Once this is merged then the full workflow will publish it to ghcr.io/ncar/i-wrf/wrf-chem

georgemccabe
georgemccabe reviewed Apr 27, 2026
View reviewed changes
Comment thread .github/workflows/build-wrf-chem-docker.yml Outdated
georgemccabe
georgemccabe reviewed Apr 27, 2026
View reviewed changes
Comment thread .github/workflows/build-wrf-chem-docker.yml
cache_from="--cache-from=type=registry,ref=${cache_ref}"
cache_to=""
push_flag=""
if [[ "$DO_PUSH" == "true" ]]; then

@georgemccabe georgemccabe Apr 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be cleaner to build the image in 1 step, then push the image in another step. This would make the logs easier to navigate, as the build log is very long. You would also be able to easily see from the workflow job summary page if the push was performed or not, as the step that does the push would be clearly skipped.

georgemccabe
georgemccabe previously approved these changes Apr 27, 2026
View reviewed changes

@georgemccabe georgemccabe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made some optional suggestions in-line. This is great functionality to add. It would be nice to eventually add a way to push an image to DockerHub instead of GHCR, where we host the the stable images used in the I-WRF User's Guide instructions. We should discuss the criteria for when we make updates to those images, as we don't want to clutter the tags with development versions that need to be tested.

@georgemccabe georgemccabe merged commit 4e89cb8 into main May 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@georgemccabe georgemccabe georgemccabe left review comments

Assignees

@vweeks vweeks

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

WRF-Chem Container

2 participants

@vweeks @georgemccabe