Skip to content

Give new GCP Postgres servers the dual-NIC layout under a flag - #6651

Draft
furkansahin wants to merge 3 commits into
gcp-postgres-dual-nic-vmfrom
gcp-postgres-dual-nic-wiring
Draft

furkansahin wants to merge 3 commits into
gcp-postgres-dual-nic-vmfrom
gcp-postgres-dual-nic-wiring

Conversation

@furkansahin

Copy link
Copy Markdown
Member

Fourth PR of the GCP Postgres dual-NIC fencing series. Stacked on #6636 (which is stacked on #6623), so its diff shows only this PR's commits.

It lets new GCP Postgres servers get the dual-NIC layout from #6636: control plane SSH on a management nic0, and customer traffic on the Dynamic NIC nic0.2. Everything stays off until an operator turns on a flag. The delete_nic lockout mechanism that uses this layout comes in a later PR.

Commits

  1. Reach GCE management NICs over IPv6 without an EIP. Launch GCP VMs with a management NIC and a Dynamic user NIC #6636 had only the IPv4 management mode. A management NIC assembled with use_eip: false has no static IPv4, but the launch code still asked for an IPv4 NAT on nic0 and gave it no IPv6.
    • Such a nic0 is now dual-stack with an external IPv6 address and no IPv4 NAT, and that address becomes the SSH host. The management VPC already allows SSH from the IPv6 control plane CIDRs.
    • In the guest, router advertisements are off on nic0, because they would put a second IPv6 default route into the main table. Its address and gateway come from the metadata server, and its default route is only in the management table.
    • The spec now also runs the fill script against fake metadata in both modes.
  2. Request management NICs for GCP Postgres under a flag.
    • The server assemble asks the resource, through provider dispatch, whether to use a separate management NIC and whether it gets an EIP. AWS answers as before, and metal never uses one.
    • On GCP, a new server gets a management NIC when the gcp_postgres_dual_nic Config or the postgres_gcp_dual_nic project flag is on and the location's management subnet is ready.
    • Once a server of a resource has a management NIC, every later server gets one too, also with both flags off.
    • The postgres_gcp_ssh_ipv6 project flag selects the IPv6 mode, as postgres_aws_ssh_ipv6 does on AWS.
  3. Leave port 22 out of dual-NIC GCP Postgres firewalls. The internal firewall's tag rules reach only the customer VPC, so on a dual-NIC server port 22 there opens nothing the control plane uses.
    • mgmt_ssh_via_user_security_group? is now dispatched per provider. The AWS rule moves into its provider module unchanged.
    • A new dual-NIC GCP resource gets no port 22. Existing resources keep it.

Tested live

I ran a local control plane (respirate and monitor) from this branch against a GCP dev project, with the management network bootstrapped in gcp-us-central1 and gcp-us-east4.

  • IPv4 management mode, with GCP_POSTGRES_DUAL_NIC=true: postgres_standard, postgres_ubuntu_2604, postgres_ha, postgres_upgrade, postgres_unarchive and postgres_firewall all passed.
    • Every server, also the new ones from the 26.04 migration, the upgrade and the unarchive, had nic0 in the management VPC and nic0.2 in the customer VPC.
    • Every internal firewall had only 5432 and 6432.
    • The HA failover ran its lockout over the management NIC.
    • postgres_firewall passed, so the customer firewall rules work on the Dynamic NIC.
  • IPv6 management mode, in a project with postgres_gcp_ssh_ipv6: postgres_standard passed on postgres-ubuntu-2604.
    • nic0 was IPV4_IPV6 with an external IPv6 address and no IPv4 NAT, and SSH used that address.
    • In the guest, both main-table default routes use user-nic. nic0's IPv6 default route is only in table 100, with source rules for both management addresses.
  • Not tested: the IPv6 mode on postgres-ubuntu-2204.

🤖 Generated with Claude Code

furkansahin and others added 3 commits October 9, 2026 14:06
A management NIC assembled with use_eip false has no static IPv4,
because Vnet::Gcp::NicNexus skips the address. The dual-NIC launch code
still asked for an IPv4 NAT on nic0, which then got an ephemeral
address, and gave nic0 no IPv6. This is the GCP side of the IPv6
management mode that AWS has with postgres_aws_ssh_ipv6, for control
planes whose SSH leaves over IPv6.

Such a nic0 is now dual-stack with an external IPv6 address and no IPv4
NAT, and that address becomes the SSH host. The management VPC already
allows SSH from the IPv6 control plane CIDRs.

GCE sends router advertisements on an IPv6-enabled nic0, which would
put a second IPv6 default route into the main table, next to the user
NIC's. So the netplan turns them off on nic0 and sets its address and
gateway statically from the metadata server, as for the user VLAN, with
the default route only in the management table and a source rule for
SSH replies.

Nothing assembles a GCP management NIC without an EIP yet; a later
commit adds the Postgres flag for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the management network from
#6623 and the dual-NIC GCE
instances from #6636, a GCP
Postgres server can keep control plane SSH on a management nic0 and its
customer traffic on a Dynamic NIC. A later lockout mechanism deletes
that Dynamic NIC to fence a primary, as detach_nic does on AWS.

The server assemble now asks the resource, through provider dispatch,
whether to use a separate management NIC and whether that NIC gets an
EIP, instead of checking location.aws? and aws_ssh_ipv6? itself. AWS
answers as before, and metal never uses one.

On GCP, a new server gets a management NIC when the
gcp_postgres_dual_nic Config or the postgres_gcp_dual_nic project flag
is on and the location's management subnet is ready, so a location that
is not bootstrapped keeps single-NIC servers. Once a server of a
resource has a management NIC, every later server gets one too, also
with both flags off. Such a resource must not mix layouts, because the
next commit drops port 22 from its internal firewall. If the management
subnet is gone by then, the VM assemble fails instead of creating a
server the control plane cannot reach.

The postgres_gcp_ssh_ipv6 project flag gives the management NIC no EIP,
so the control plane reaches it over IPv6, as postgres_aws_ssh_ipv6
does on AWS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The internal firewall of a Postgres resource opens port 22 to the
control plane CIDRs while some server needs control plane SSH on its
user NIC. On GCP, its tag rules reach only the NICs in the customer
VPC, so on a dual-NIC server they open 22 on the Dynamic NIC, where no
control plane SSH arrives. That SSH goes to the management nic0, which
the management VPC policy opens to the same CIDRs.

So the decision is now dispatched per provider. GCP keeps port 22 while
new servers get no management NIC or some server has none, and leaves
it out for a new dual-NIC resource, whose rules are written at assemble
before its first server exists. The previous commit keeps such a
resource dual-NIC. The AWS rule moves into its provider module
unchanged, and metal keeps port 22.

Existing GCP resources keep port 22.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant