Skip to content

Replace Vagrant with Cloud-Init and QEMU for VM image builds - #97

Open
ralphlange wants to merge 15 commits into
epics-training:mainfrom
ralphlange:cloud-init
Open

Replace Vagrant with Cloud-Init and QEMU for VM image builds#97
ralphlange wants to merge 15 commits into
epics-training:mainfrom
ralphlange:cloud-init

Conversation

@ralphlange

@ralphlange ralphlange commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Use cloud-init under QEMU instead of Vagrant for building VM images.

  • add scripts that run QEMU/cloud-init to initialize a qcow2 image and run the initial_setup and m_base roles
  • rewire the GitHub Actions CI jobs to use the 'original' qcow2 images instead of 'very similar' containers
  • bump the versions of many components and modules
  • fix libxml2 issues with areadetector
  • fix C23/gcc15 issues with many modules
  • replace the brittle "add Firefox bookmark" task with a more robust and portable "assemble a homepage and drop a desktop icon"
  • add proper support for ARM (aarch64). Doesn't work on GHA yet, as their aarch64 runners do not provide KVM access
  • update documentation.

(fixes #94)

ralphlange and others added 15 commits August 1, 2026 19:46
- remove Vagrant-related files and documentation
- implement cloud-init/create_vm.sh for VM image generation using QEMU
- add -c <ca_cert> command line option to add trusted CA certificate
- add cloud-init/provisioning.sh to handle in-VM provisioning with Ansible
- make graphical subsystem installation configurable via install_graphics
- do all file system resizing in teh cloud-init part
- support generation of both qcow2 and VDI images for all flavors
- update documentation to reflect the new workflow

- fixes epics-training#94, fixes epics-training#93, fixes epics-training#85, fixes epics-training#82, fixes epics-training#81, fixes epics-training#69,
  fixes epics-training#43, fixes epics-training#39,

Co-authored-by: google-labs-jules[bot]
- use Wayland instead of Xorg
- use Ansible command for 'set-default graphical.target'
- add check to make this step idempotent
- don't activate EPEL (Rocky)
- don't fix Rocky mirrors (Rocky)
- use @"Workstation" for GNOME (RedHat)
- use task-gnome-desktop for GNOME (Debian)
- improve update/upgrade (Debian)
- don't install recommends or suggests (Debian)
- only install ansible-core for Ansible
Cherry-picked-from: 6178e70
Co-authored-by: Wayne Lewis
(closes epics-training#83)
- Rocky stays on 9 for a few more cycles,
  as Rocky 10 is still pretty new
Replace the container-based two-stage build mechanism
with new cloud-init-based build as stage 1,
then booting the qcow2 artifact under QEMU
and running Ansible inside it for the stage 2 jobs

- Add cloud-init/run_ansible_test.sh: boots a qcow2 with -snapshot,
  scp's the current checkout's ansible/ into the guest,
  and ssh-runs ansible-playbook for the given vars/ci-*.yml.
- Replace _wait_for_workflow.yml with _resolve_build_run.yml:
  resolves to the same-commit stage-1 build if it succeeded,
  then an older build on the same branch,
  then the latest successful build on main,
  and hard-fails if neither exists.
- Rewrite test-roles-{modules,phoebus-aa,oac-tree}.yml to resolve and
  download a qcow2 artifact and call run_ansible_test.sh.
- Flip in_container: false in ci-modules/ci-phoebus-aa/ci-oac-tree.yml
  now that stage 2 runs in real VMs with real systemd.
- Update ansible/vars/README.md for the new pipeline.
- Add CLAUDE.md to .gitignore.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- p4p: bump version to 4.2.2
- asyn: bump version to 4.46
- stream: cherry-pick commit e87e093 for current ASYN
- remove all remaining skips to see where builds break
m_pvaPy was previously listed as an unconditional dependency,
but nothing in m_areadetector's own modules
(busy, ADCore, ADSimDetector, pvaDriver) actually references PVAPY.
- also run the headless firefox with a timeout
- cherry-pick 'fix gcc-15 problem' from upstream
- suppress open62541 warning that breaks on newer GCC

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- patch deprecated xmlGetGlobalState() issue

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Function pointers declared the old K&R way are illegal
with C23 (default with GCC 15 under new distros).
Solve by setting "-std=gnu17".

Applies to:
- busy
- seq/lemon
- sscan
- calc
- stream
- adding a Firefox bookmark (scripted, headless) is a super-heavy task
  which is becoming impossible under Fedora 44 (others will follow)
- replace this wormhole with a plain-and-simple approach:
  Assemble a Training-VM homepage (so that roles can drop lines
  with links) and put a link on the desktop
- fixes epics-training#95

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Make the cloud-init image build and the role tests work
for both x86_64 and aarch64, natively as well as cross-built.

- playbook.yml now loads vars/arch/<architecture>.yml, mirroring the existing
  distro detection. It provides the arch dict (epics_host_arch, deb, rpm,
  multiarch_tuple, java) plus is_x86_64/is_aarch64, replacing all
  hardcoded or open-coded architecture names in the roles.

- Two upstream artifacts are only published for x86_64 and are now skipped
  elsewhere:
  - UaExpert (closed-source, its vendor ships x86_64 Linux only)
  - the Raven repo serving oac-tree's cosmetic adwaita-qt6 has no aarch64 tree.

- cloud-init/qemu_arch.sh is a new shared helper mapping a target
  architecture to emulator binary, machine type, CPU model and - for
  aarch64, which is UEFI-only - the pflash firmware pair.
  create_vm.sh gets -a (default: host architecture) and writes
  <flavor>-<arch>.qcow2/.vdi;
  run_ansible_test.sh gets -a and otherwise infers the architecture
  from the image filename. A target differing from the host needs no extra flag:
  it switches to TCG emulation with -cpu max and warns about the slowdown.

- All CI workflows are matrixed over both architectures on native runners
  (ubuntu-latest, ubuntu-24.04-arm). Note that GitHub's arm64 hosted
  runners do not expose /dev/kvm; since accel=kvm:tcg falls back
  to TCG emulation (taking hours), theses jobs are opt-in (manual start).
- The molecule converge playbooks load the arch vars as well, since they
  run the roles directly and bypass playbook.yml's pre_tasks.

- Rename the stage 2 jobs, removing the misleading "Test " prefix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ralphlange
ralphlange requested review from a team, gilesknap, shroffk and simon-ess August 3, 2026 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drop Vagrant/initial_setup for cloud-init bootstrapping process

1 participant