Skip to content

Conversation

@flotter
Copy link
Contributor

@flotter flotter commented Sep 30, 2025

The daemon HTTPS server (when enabled) uses TLS v1.3 for the client server handshake.

The traditional TLS handshake does not provide us with a way to exchange identity certificates between client and server, since only the server certificate is sent to the client. This leaves the current TLS connection relying on basic authentication headers for passing a client identity to the server.

This patch migrates the HTTPS server over from basic authentication to using mutual TLS. The client (see #715) now has to supply a suitable client identity certificate that will be used for verifying the identity during normal API usage, although this requirement is not enforced for end-points with open access.

In #702 we added a pairing manager, which integrates with the daemon API permissions system to facilitate client server pairing during a carefully controlled pairing window on a dedicated API end-point v1/pairing only.

A followup task will provide an official event binding mechanism (trigger manager) by which the pairing window can be enabled when certain events are configured. For now, this cannot be enabled and therefore pairing is not possible. The the patch below hooking in the pairing window enable for testing purposes.

State of HTTPS (mTLS) after this patch

  • When Pebble enables the HTTPS server only mTLS connections will be allowed (basic authorization is no longer available on the HTTPS transport, but can still be used with HTTP)
  • The Pebble client package adds (see feat(client): mTLS support [2b/2] #715):
    • The ability to supply the client identity certificate, but this is not yet used by the CLI and therefore Pebble cannot use HTTPS today.
    • The ability to obtain the server identity certificate by making an insecure HTTPS connection. Can be used to implement an SSH-like workflow to implement a CLI warning asking the user to trust the server identity for future connections.
    • The ability to pair with the server by supplying the server identity fingerprint. This workflow requires obtaining the fingerprint by other means, such as DNSSD or via a display on the server side.
  • The Pairing Manager provides the EnablePairing method, but this is not yet available, and requires future work to support binding events to actions such as pairing enable.

Testing

The following test procedure can offer a demonstration how this works, but the process requires a small temporary patch to enable pairing:

  1. Apply the following patch as a temporary step to enable the pairing on startup.
diff --git a/internals/overlord/pairingstate/manager.go b/internals/overlord/pairingstate/manager.go
index 833b979..c933ef9 100644
--- a/internals/overlord/pairingstate/manager.go
+++ b/internals/overlord/pairingstate/manager.go
@@ -131,6 +131,12 @@ func (m *PairingManager) Stop() {
 	m.stopTimer()
 }
 
+// TESTING HACK: Open the pairing window on startup.
+func (m *PairingManager) StartUp() error {
+	fmt.Printf("Pairing Enabled ...")
+	return m.EnablePairing(60 * time.Second)
+}
+
 // PairMTLS adds a client identity with admin permissions to the identity
 // subsystem. A pairing request always closes the pairing window.
 func (m *PairingManager) PairMTLS(clientCert *x509.Certificate) error {
  1. Generate the client private key and certificate
~/> openssl genpkey -algorithm ed25519 -out client-key.pem

~/> openssl req -x509 -new -nodes -key client-key.pem -days 365 -out client-cert.pem -subj "/O=Test/CN=client" -addext "keyUsage=critical,digitalSignature" -addext "extendedKeyUsage=clientAuth" -addext "basicConstraints=critical,CA:FALSE"
  1. Build pebble, run:
~/pebble> go build ./cmd/pebble
~/pebble> mkdir layers
~/pebble> echo -e "pairing:\n  mode: multiple" > layers/001-default.yaml
~/pebble> PEBBLE=$(pwd) ./pebble run --create-dirs --https=:8443
  1. Test HTTPS (within the first 60 seconds of starting pebble):
# Not yet paired.
~/> curl -X GET --cert ./client-cert.pem --key ./client-key.pem --insecure https://localhost:8443/v1/plan?format=yaml
{"type":"error","status-code":401,"status":"Unauthorized","result":{"message":"access denied","kind":"login-required"}}

# Pair during open pairing window (will close window).
~/> curl -X POST -H "Content-Type: application/json" -d '{"action": "pair"}' --cert ./client-cert.pem --key ./client-key.pem --insecure https://localhost:8443/v1/pairing

{"type":"sync","status-code":200,"status":"OK"}

# Test HTTPS with mTLS enabled.
~/> curl -X GET --cert ./client-cert.pem --key ./client-key.pem --insecure https://localhost:8443/v1/plan?format=yaml

{"type":"sync","status-code":200,"status":"OK","result":"pairing:\n    mode: multiple\n"}

@flotter flotter force-pushed the feature/mtls-integration branch 4 times, most recently from 59eea14 to a29dd07 Compare September 30, 2025 16:44
@flotter flotter force-pushed the feature/mtls-integration branch 7 times, most recently from 2e731d8 to ca29abb Compare October 6, 2025 16:16
@flotter flotter force-pushed the feature/mtls-integration branch 2 times, most recently from 2e62737 to ba52ae9 Compare October 16, 2025 06:36
flotter added a commit that referenced this pull request Oct 16, 2025
The pairing manager enables and processes incoming mTLS client pairing
requests on the ```/v1/pairing``` HTTP end-point.

The pairing manager does not by itself decide when the pairing window
should be enabled. The ```EnablePairing``` call will be made by another
manager (called the trigger manager) that deals with event bindings,
such as mapping a button press to an enable pairing request. The trigger
manager will be added at a later stage.

This pull-request focuses only on the Pairing Manager itself,
implementing the following functionality (KO093, KO088):

```go
// PairingEnabled returns whether pairing is currently enabled.
func (m *PairingManager) PairingEnabled() bool {...}
```

```go
// EnablePairing requests the pairing manager to enable the pairing window.
func (m *PairingManager) EnablePairing(timeout time.Duration) error {...}
```

```go
// PairMTLS adds a certificate based client identity with admin permissions to
// the identity subsystem. A pairing request always closes the pairing window.
func (m *PairingManager) PairMTLS(clientCert *x509.Certificate) error {...}
```

Integration of the pairing manager into the HTTPS server and access
checks will follow in #708
@flotter flotter force-pushed the feature/mtls-integration branch from ba52ae9 to 56bccf1 Compare October 16, 2025 06:44
@flotter flotter marked this pull request as ready for review October 16, 2025 06:45
@flotter flotter requested review from benhoyt and hpidcock October 16, 2025 07:15
Copy link
Contributor

@benhoyt benhoyt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good overall, however, see the note about the missing UserState.UID -- I might be missing something, but I believe this is a bug that will cause UID-based notices to not work.

@benhoyt
Copy link
Contributor

benhoyt commented Oct 19, 2025

Oh, one other thought: it would make for a smaller PR to review if the client changes were moved to another PR, as the changes here don't depend on that (there's no CLI changes).

Copy link

@ghislainbourgeois ghislainbourgeois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a small suggestion, and commented on Ben's comment touching on crypto code. I am happy with the mTLS implementation here.

@flotter flotter changed the title feat(pairingstate): mTLS support with client pairing [2/2] feat(daemon): mTLS support [2a/2] Oct 21, 2025
@flotter
Copy link
Contributor Author

flotter commented Oct 21, 2025

The client code is now split into a separate PR (#715). Note that the end-to-end test proposal will only work with both PRs in place.

@flotter flotter requested a review from benhoyt October 21, 2025 20:24
Copy link
Contributor

@benhoyt benhoyt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good, thanks for the updates.

@flotter flotter force-pushed the feature/mtls-integration branch from 275179f to 6d79697 Compare October 22, 2025 18:25
@flotter
Copy link
Contributor Author

flotter commented Oct 22, 2025

Final approval happened for 6d79697

I added another commit on top of this, and then reverted it back to this commit (I will move the unit test enhancements into a separate PR).

@flotter
Copy link
Contributor Author

flotter commented Oct 22, 2025

Performed curl based final check to prove server and pairing works with mTLS.

@flotter flotter merged commit 3289f77 into canonical:master Oct 22, 2025
39 checks passed
flotter added a commit that referenced this pull request Oct 22, 2025
The client side mTLS code was split from #708.

For the complete description of the work, please refer to the
description in #708.

Pebble client changes adds:
- The ability to supply the client identity certificate, but this is not
yet used by the CLI and therefore Pebble cannot use HTTPS today.
- The ability to obtain the server identity certificate by making an
insecure HTTPS connection. Can be used to implement an SSH-like workflow
to implement a CLI warning asking the user to trust the server identity
for future connections.
- The ability to pair with the server by supplying the server identity
fingerprint. This workflow requires obtaining the fingerprint by other
means, such as DNSSD or via a display on the server side.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants