Skip to content

[DRAFT] Pass cloud init to shim containers#6038

Draft
zedi-pramodh wants to merge 1 commit into
lf-edge:masterfrom
zedi-pramodh:fix-container-ci
Draft

[DRAFT] Pass cloud init to shim containers#6038
zedi-pramodh wants to merge 1 commit into
lf-edge:masterfrom
zedi-pramodh:fix-container-ci

Conversation

@zedi-pramodh

Copy link
Copy Markdown

For shim containers on eve-k we are not passing in the cloud init. This fix addressed that.

PR dependencies

List all dependencies of this PR (when applicable, otherwise remove this
section).

How to test and validate this PR

Please describe how the changes in this PR can be validated or verified. For
example:

  • If your PR fixes a bug, outline the steps to confirm the issue is resolved.
  • If your PR introduces a new feature, explain how to test and validate it.

This will be used

  1. to provide test scenarios for the QA team
  2. by a reviewer to validate the changes in this PR.

The first is especially important, so, please make sure to provide as much
detail as possible.

If it's covered by an automated test, please mention it here.

Changelog notes

Text in this section will be used to generate the changelog entry for
release notes. The consumers of this are end users, not developers.
So, provide a clear and short description of what is changed in the PR from
the end user perspective. If it changes only tooling or some internal
implementation, put a note like "No user-facing changes" or "None".

PR Backports

For all current LTS branches, please state explicitly if this PR should be
backported or not. This section is used by our scripts to track the backports,
so, please, do not omit it.

Here is the list of current LTS branches (it should be always up to date):

  • 16.0-stable
  • 14.5-stable
  • 13.4-stable

For example, if this PR fixes a bug in a feature that was introduced in 14.5,
you can write:

- 16.0-stable: To be backported.
- 14.5-stable: No, as the feature is not available there.
- 13.4-stable: No, as the feature is not available there.

Also, to the PRs that should be backported into any stable branch, please
add a label stable.

Checklist

  • I've provided a proper description
  • I've added the proper documentation
  • I've tested my PR on amd64 device
  • I've tested my PR on arm64 device
  • I've written the test verification instructions
  • I've set the proper labels to this PR

For backport PRs (remove it if it's not a backport):

  • I've added a reference link to the original PR
  • PR's title follows the template

And the last but not least:

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

Please, check the boxes above after submitting the PR in interactive mode.

@zedi-pramodh zedi-pramodh marked this pull request as draft June 11, 2026 19:53
@github-actions github-actions Bot requested review from rene, rucoder and shjala June 11, 2026 23:39
@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 21.11%. Comparing base (53975ca) to head (38acc3d).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
pkg/pillar/cmd/domainmgr/domainmgr.go 0.00% 34 Missing ⚠️
pkg/pillar/hypervisor/kubevirt.go 0.00% 15 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6038      +/-   ##
==========================================
+ Coverage   20.72%   21.11%   +0.38%     
==========================================
  Files         504      514      +10     
  Lines       92760    94500    +1740     
==========================================
+ Hits        19222    19951     +729     
- Misses      71917    72745     +828     
- Partials     1621     1804     +183     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rene rene added the next-17.0.0-rc PR must be present in the next 17.0.0-lts release label Jun 12, 2026
@rene

rene commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

@zedi-pramodh , pls, don't forget to fix the commit message (blank) and the PR description.... the changes looks good for me...

@zedi-pramodh zedi-pramodh force-pushed the fix-container-ci branch 3 times, most recently from b008fd3 to c2f9fd5 Compare June 12, 2026 23:47
Container apps on eve-k run as kubevirt shim VMIs (VirtualizationMode != NOHYPER)
launched from eve-external-boot-image. Two HV=k-only gaps relative to HV=kvm
are addressed:

1. Cloud-init env vars never reached the guest. On HV=kvm/xen the snapshot-
   mount path projects env vars into /environment on the rootfs via the OCI
   spec. On HV=k the container rootfs is a read-only PVC at /dev/vda, so that
   path is not available.

2. Customer-attached volumes were visible in the guest but never mounted, so
   the container ran without its persistent data.

Pillar side:
- pkg/pillar/cmd/domainmgr/domainmgr.go: when hvTypeKube &&
  VirtualizationMode != NOHYPER, emit a synthetic DiskStatus with
  Devtype="cloudinit-nocloud" and the env map rendered as sorted shell
  KEY="value" lines (escaping " and \). Refactor the VM-app cloud-init ISO
  path into appendCloudInitDisk so its behavior is unchanged; the new
  branch lives alongside it.
- pkg/pillar/hypervisor/kubevirt.go: translate the cloudinit-nocloud
  DiskStatus into v1.Volume{CloudInitNoCloud: {UserDataBase64: ...}} attached
  as a read-only sata cdrom. Base64 keeps the payload out of plaintext
  `kubectl describe vmi`.

Initrd side (pkg/xen-tools/initrd):
- init-initrd: for root != 9p (i.e. HV=k path, not HV=kvm/xen which already
  projects env vars via OCI), probe for a cdrom with LABEL=cidata, mount it
  iso9660, and source user-data lines as `export KEY=value` into
  /mnt/environment. Parse raw `blkid` output by hand — busybox's
  `blkid -L cidata` returns empty in the eve-external-boot-image build even
  when raw `blkid` enumerates the device with the correct LABEL field. No
  modprobe is needed; iso9660 and the sata/sr_mod stack are built into the
  eve-external-boot-image kernel.
- mount_disk.sh: skip the boot block device. On HV=k the container PVC is
  itself the boot block device, already mounted at /mnt. Without the skip
  it would be treated as the first extra volume — either remounting the
  rootfs on top of itself, or consuming mountPoints line 1 and shifting
  every subsequent device-to-path mapping down by one, leaving the
  customer's actual extra volume (/dev/vdb) either unreached or mapped to
  the wrong target. On HV=kvm/xen root=9p has no block device at /mnt so
  the guard is a no-op there.
- mount_disk.sh: exit cleanly when /mnt/mountPoints is absent (image
  declared no VOLUMEs — writeKubevirtMountpointsFile in
  pkg/pillar/cas/containerd.go gates the write on len(mountpoints) > 0),
  instead of erroring on every enumerated block device.

Build system (Makefile):
- Set `pkg/external-boot-image: FORCE_BUILD := --force`. The package tag
  is hashed only from git-tracked files (Dockerfile.in, build.yml), so it
  never changes when xen-tools changes. Without --force, `linuxkit pkg build`
  short-circuits on cache-hit and keeps serving stale runx-initrd
  indefinitely, regardless of init-initrd / mount_disk.sh edits.
- Set `pkg/kube: FORCE_BUILD := --force` for the same reason one level up:
  pkg/kube bundles pkg/kube/external-boot-image.tar which is gitignored, so
  changes to it don't change pkg/kube's hash. Without --force, the eve-kube
  image in cache keeps the old tar baked in, and the rootfs ships stale
  content to the device.

Signed-off-by: Pramodh Pallapothu <pramodh@zededa.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

next-17.0.0-rc PR must be present in the next 17.0.0-lts release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants