Skip to content

3.1.2 - #161

Merged
jefflester merged 10 commits into
masterfrom
3.1.2
Sep 30, 2026
Merged

3.1.2#161
jefflester merged 10 commits into
masterfrom
3.1.2

Conversation

@jefflester

@jefflester jefflester commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Summary

Dependency Updates

Library: cache-service Example MV Memory

  • The example MV (LIMIT 500 per UNION ALL branch, partitioned by orderdate) opened an ORC writer for nearly every row; its refresh peaked at ~483 MB of the 614 MB per-node limit locally and exceeded it in CI, deterministically failing the Starburst cache-service lib test
  • Reduced to LIMIT 50 per branch (still partitioned by orderdate); peak refresh memory ~57 MB
  • Ruled out the dependency bumps via a local A/B: identical memory config under click 8.5.0 / docker 7.2.0, refresh succeeds

CI/CD: Smoke Test False Passes

  • The provision step captured tee's exit code, so it always passed on both OSes
  • This hid real macOS failures: Homebrew's docker formula lacks the Compose v2 and Buildx plugins
  • Fix: PIPESTATUS[0]; install and register docker-compose and docker-buildx on macOS; verify docker info, docker compose version, and docker buildx version before provisioning

CI/CD: Reusable Smoke Test Workflow

  • Moved into smoke-test.yaml, called from release.yaml as the release gate
  • Also runnable via workflow_dispatch (any published version) and on PRs that modify smoke-test.yaml (latest published version), so smoke test changes are validated before they gate a release
  • Colima startup bounded to 8 min per attempt with one retry from a clean VM; step (20 min) and job (35 min) timeouts so a hung runner fails fast
  • Version input validated as X.Y.Z; token permissions restricted to contents: read

CI/CD: Build Test Images When Fallback Images Are Missing

  • Unchanged-image PRs pull latest-{trino,starburst} from GHCR, but no workflow publishes those tags, so every test job failed in setup with manifest unknown
  • Change detection now builds test images whenever the fallback images can't be resolved

Docs

  • Release workflow docs updated to reflect the actual four-stage flow and smoke test behavior

Test plan

  • Unit tests pass locally with click 8.5.0 / docker 7.2.0 (414 passed)
  • pre-commit (incl. actionlint, shellcheck) passes
  • CI: all CLI and Trino/Starburst lib test jobs pass (24/24), including Starburst cache-service with 0 memory-limit errors
  • Smoke test against 3.1.1 via PR trigger: Ubuntu and macOS provision genuinely start (killed by 30s timeout), not a masked pass
  • Local: cache-service example MV refresh peak memory 483 MB → 57 MB

Follow-ups (not in this PR)

  • Publish latest-* test images from master so unchanged-image PRs skip builds (requires cleanup-test-images.yaml to not delete shared versions)
  • Migrate CommandLineInterface from click.MultiCommand to click.Group before any Click 9.x bump
  • Plan for the Aug 2027 retirement of macos-15-intel (last x64 macOS runner)

jefflester and others added 10 commits September 29, 2026 21:31
…ates (#159)

Bumps the github-actions group with 4 updates in the / directory: [actions/checkout](https://github.com/actions/checkout), [actions/setup-java](https://github.com/actions/setup-java), [actions/cache](https://github.com/actions/cache) and [actions/setup-python](https://github.com/actions/setup-python).


Updates `actions/checkout` from 6 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v6...v7)

Updates `actions/setup-java` from 4 to 5
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](actions/setup-java@v4...v5)

Updates `actions/cache` from 4 to 6
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v4...v6)

Updates `actions/setup-python` from 6 to 7
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/setup-java
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/setup-python
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker](https://github.com/docker/docker-py) from 7.1.0 to 7.2.0.
- [Release notes](https://github.com/docker/docker-py/releases)
- [Commits](docker/docker-py@7.1.0...7.2.0)

---
updated-dependencies:
- dependency-name: docker
  dependency-version: 7.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Applied manually from #157; Dependabot's rebase reset that PR's base to
master, where it conflicts with the docker bump (#158) on adjacent lines.
The provision check captured tee's exit code instead of minitrino's, so it
always passed. On macOS this hid a real failure: Homebrew's docker formula
lacks the Compose v2 plugin, so provision failed with 'unknown shorthand
flag: f'.

- Move the smoke test to smoke-test.yaml (workflow_call from release.yaml,
  plus workflow_dispatch and a pull_request trigger for changes to the
  workflow itself, testing the latest published version)
- Use PIPESTATUS to capture the provision exit code
- Install and register the docker-compose plugin on macOS, and verify
  Docker and Compose before provisioning
- Validate the version input and restrict token permissions to read
- Update workflow docs to reflect the actual four-stage release flow
With Compose fixed, provision on macOS got further and failed building the
image: Homebrew's docker formula also lacks the Buildx plugin, so Compose
fell back to the classic builder, which doesn't support additional build
contexts. Install and register docker-buildx and verify it before
provisioning.
When a PR doesn't change image files, CI pulls latest-trino/latest-starburst
from GHCR instead of building. No workflow publishes those tags, so they were
missing and every test job failed in setup with 'manifest unknown'. Change
detection now builds test images whenever the fallback images can't be
resolved.
The example MV selected 500 rows per UNION ALL branch while partitioning by
orderdate, opening an ORC writer for nearly every row. Its refresh peaked at
~483 MB of the 614 MB per-node query limit locally and exceeded it in CI,
failing the Starburst cache-service lib test. Selecting 50 rows per branch
keeps the partitioned MV demo intact and drops peak refresh memory to
~57 MB.
A macOS smoke run hung in colima start for ~46 minutes until the hosted
runner lost communication. Bound each start attempt to 8 minutes (observed
normal starts: 4-6 minutes), retry once from a clean VM, and add step (20m)
and job (35m) timeouts so a hung runner fails the release gate quickly.
@jefflester
jefflester merged commit 715a85e into master Sep 30, 2026
27 checks passed
@jefflester
jefflester deleted the 3.1.2 branch September 30, 2026 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant