• gardener/controlplane Helm chart is deprecated and scheduled for final deletion end of 12/2025
• We could use this opportunity to restructure our e2e test jobs and setups
• The gardener-operator-based setup will become the only option for local development
• Provider Extension setup must also be reworked a little bit

Target Setups

• Single-node kind cluster w/ operator (make operator-up) with and without registering this kind as a seed
• Multi-node kind cluster w/ operator w/ node high-availability (make operator-ha-single-zone-up) with and without registering this kind as a seed
• Multi-node kind cluster w/ operator w/ zone high-availability (make operator-ha-multi-zone-up) with and without registering this kind as a seed
• Single-node kind2 cluster w/ gardenlet (make kind2-up) to test control plane migration or deployment of gardenlets into remote clusters
• Multi-node kind2 cluster w/ gardenlet w/ node high-availability (make kind2-ha-single-zone-up) to test control plane migration or deployment of gardenlets into remote clusters
• Provider Extensions Setup
• Dual-Stack and IPv6-only single-node kind cluster w/ operator

Target Makefile Rules

• make kind-up [--ha-single-zone|--ha-multi-zone|--cpm|--cpm-ha-single-zone|--dual|--ipv6] (starts a kind cluster)
• make operator-up (deployment of gardener-operator, no Garden or Gardenlet resource or anything)
• make garden-up [--extensions] (deployment of Garden resource, detects automatically which setup (single-node/multi-zone/...))
• make seed-up [--cpm|--dual|--ipv6] (deployment of garden config and Gardenlet resources, detects automatically which setup (single-node/multi-zone/...) except CPM/DUAL/IPv6 because they need to be in a separate kind cluster)
• make gardener-up [--dual|--ipv6] (everything combined except kind, detects automatically which setup (single-node/multi-zone/...) except DUAL/IPv6 because they need to be in a separate kind cluster)
• make gardenadm-up [--high-touch|--medium-touch]

Target File System Structure

./
./dev-setup
./dev-setup/kind
./dev-setup/kind/{calico,cluster,metrics-server,registry*} (everything related to preparing the kind cluster)
./dev-setup/garden
./dev-setup/garden/... (`Garden` manifests for preparing the garden cluster, i.e., no longer use `example/operator/20-garden.yaml` for e2e tests)
./dev-setup/extensions
./dev-setup/extensions/... (`Extension` manifests like `provider-local`, `networking-{calico,cilium}`)
./dev-setup/gardenconfig
./dev-setup/gardenconfig/... (`CloudProfile`, `Project`, `Secrets` and `*Binding`s, etc.)
./dev-setup/gardenlet
./dev-setup/gardenlet/... (`Gardenlet` manifests and patches for setting up `gardenlet`s)
./dev-setup/gardenlet/kind.yaml
./dev-setup/gardenlet/kind2.yaml
./dev-setup/gardenlet/kind2-ha-single-zone.yaml
./dev-setup/gardenadm
./dev-setup/gardenadm/... (see current `example/gardenadm-local` folder)

# Thought: Maybe we could establish a scenarios structure to make it easy find all configuration related to a scenario (e.g., everything related to IPv6 tests)
./dev-setup/scenarios
./dev-setup/scenarios/...

General Considerations

• Keep operator-up and operator-seed-up two separate scenario, but drastically simplify kustomizations in example/provider-local folder
• The following folders in example/gardener-local can be dropped and don't need to be moved to ./dev-setup:
  • controlplane (move service-account-issuer secret into kustomizations)
  • etcd
  • kube-apiserver
• The Helm chart values in example/gardener-local/gardenlet must be translated into Gardenlet resources (or rather kustomize patches if possible) like already done in example/gardener-local/gardenlet/operator

Concrete Work Streams

• Prerequisites/can happen in parallel/upfront:
  • Make gardener-operator and Garden scenario work on single-node kind cluster #12138
  • Speed-up bring-up of Gardener and a seed using gardener-operator-based setup #12156
• Switch existing e2e tests/scenarios to operator-based setup (no longer run gardener*-up)
  • Rework Garden, extensions, garden config, and Gardenlet kustomizations (move to /dev-setup) #12213
  • Rework gardenadm kustomizations (move to /dev-setup) #12331
  • Rework ha-single-zone kustomizations (move to /dev-setup) #12345
    • Rename ha-single-zone to ha-multi-node #12395
  • make {operator,garden,seed,gardenadm}-{up,dev,debug,down}, support single-node kind cluster w/ gardener-operator #12439
  • Support different Garden profiles for the various scenarios (single-node (not yet supported, see above), multi-node (ha-single-zone), multi-zone (ha-multi-zone), dualstack, ipv6)
  • Support different Gardenlet profiles for the various scenarios (see above + cpm, cpm-multi-node)
  • Switch local IPv6 setup to gardener-operator #12709
  • Switch local dual-stack setup to gardener-operator #12986
  • Drop test-e2e-local-operator-seed #13042 and Cleanup gardener/gardener-e2e-kind-operator-seed job ci-infra#4558
  • Switch Makefile rules and hack/ci-e2e-*.sh scripts to the new profiles
• Move kind setup into ./dev-setup/kind and implement/adapt make kind-up
• Move gardener-operator deployment into ./dev-setup/operator and implement/adapt make operator-up
• Define Garden manifests and put them into ./dev-setup/garden and implement/adapt make garden-up
  • Generate Extension and Controller{Deployment,Registration} manifests for provider-local into ./dev-setup/extensions/provider-local and augment make garden-up
  • Move garden config resources like CloudProfile, Project, etc. into ./dev-setup/gardenconfig and augment make garden-up
• Translate gardenlet Helm chart values into Gardenlet resources (or kustomize patches) and put them into ./dev-setup/gardenlet and implement/adapt make seed-up
• Adapt provider extensions setup
• Later:
  • Simplify kind Cluster Helm chart (drop complexity for values no longer needed/configured)
  • Change local IP addresses (*.1 should be virtual-garden-istio-ingressgateway, *.10 should be the default istio-ingressgateway, *.1{1,2,3} should be the zonal istios )
  • Consider using the provider-local CoreDNS like in the e2e tests also locally such that we can get rid of this
  • Deploy operator.gardener.cloud/v1alpha1.Extensions for networking-{calico,cilium} instead of Controller{Registration,Deployment}
  • Eliminate istio webhook workaround in provider-local introduced with Rework ha-single-zone kustomizations (move to /dev-setup) #12345