Skip to content

foundationdb: build 7.3.81 against OpenSSL 4, drop openssl-3.5 - #1580

Open
bryan-minimal wants to merge 2 commits into
mainfrom
foundationdb-openssl4
Open

bryan-minimal wants to merge 2 commits into
mainfrom
foundationdb-openssl4

Conversation

@bryan-minimal

Copy link
Copy Markdown
Member

This PR moves foundationdb 7.3.81 from the openssl-3.5 compat package to openssl (4.x). It is a draft because it touches TLS: Bryan reviews it.

openssl-3.5 has three users left. This PR removes foundationdb. #1579 handles actions-runner, and the Rust stage0 rung is handled separately.

What broke and why

FDB 7.3 fails to build against OpenSSL 4 for two reasons:

  • OpenSSL 4.0 made the X509 accessors return const: X509_get_subject_name(), X509_get_issuer_name(), X509_NAME_get_entry() and X509_NAME_ENTRY_get_data().
  • It also made ASN1_STRING opaque.

Upstream has no fix. apple/foundationdb#14348 pins OpenSSL 3.5 instead; it is open and its CI fails.

I compared FDB's OpenSSL use against the 4.1 headers, ossl-removed-api(7) and ossl-guide-migration(7), and found four breakages:

# Where Error on 4.x Fix
1 flow/MkCert.cpp makeCertNative() It adds entries to the subject name the certificate returns, which is now const Build a fresh X509_NAME, then X509_set_subject_name(). The certificate encodes the same.
2 flow/TLSConfig.actor.cpp match_criteria_entry / match_name_criteria Const X509_NAME_ENTRY / ASN1_STRING assigned to non-const Add const
3 flow/TLSConfig.actor.cpp check_verify / match_criteria Const X509_NAME (subject/issuer) assigned to non-const Add const
4 Bundled Boost 1.78: asio rfc2818_verification.ipp, pulled in by <boost/asio/ssl.hpp> Reads ASN1_STRING members (->data, ->length) directly Add -DBOOST_ASIO_SSL_RFC2818_VERIFICATION_HPP to CXXFLAGS. FDB never uses that deprecated class. Boost 1.87 removed the file, but FDB 7.3 needs Boost 1.86 or older for io_service.

Everything else FDB uses compiles against 4.x:

  • Removed APIs: FDB uses none of the APIs 4.0 removed (fixed TLS-version methods, ENGINE, ERR_remove_*, ASN1_STRING_data).
  • Deprecated APIs: the deprecated 3.0 calls it does use (RSA_*, EC_KEY_*, HMAC_*, RAND_set_rand_method) are still in 4.1 and only warn. USE_WERROR is off.
  • asio: the rest of asio 1.78's SSL code takes the OpenSSL ≥ 1.1 paths, so it builds too.

Nothing was compiled locally. Build-bot's run of this PR is the first real compile.

The patch (packages/foundationdb/foundationdb-7.3.81-openssl4.patch, 18 changed lines)

  • Verification path: the TLSConfig change is const qualifiers only. No comparison, branch or return value in the peer-rule matcher changes, so the review is about 7 lines.
  • MkCert: it only generates certificates for tests and tools (mkcert, unit tests). It is not on the verification path.
  • Compatibility: the patch also compiles against OpenSSL 3.x.
  • How build.sh applies it: patch -Np1 -F0, with sha256 checks of both files before and after (the same pattern as llama.cpp). A different upstream file, or a partial apply, fails the build.
  • When to drop it: the header comments in the patch and build.sh say to drop the patch and the Boost define once upstream supports OpenSSL 4.

Recipe

  • build.ncl:
    • openssl-3-5 is replaced by openssl in build_deps and runtime_deps;
    • patch is added to build_deps;
    • the patch is a Local dep.
  • build.sh:
    • drops the openssl-3.5 -I and CMake OPENSSL_* paths, so CMake finds the system OpenSSL 4;
    • flips the link check: fdbserver must NEED libssl.so.4, and nothing shipped may NEED libssl.so.3 or libcrypto.so.3.
  • AGENTS.md: the OpenSSL section no longer lists foundationdb as a package that can't build on 4.x.
  • Not edited: the comment in packages/openssl-3.5/build.ncl that still lists foundationdb as a user. I left it so this PR doesn't touch (and possibly rebuild) openssl-3.5. It can go when the last user does.

Tests

The existing server_smoke test never exercised TLS. Two new standalone tests do.

crypto_unittests runs FDB's own unit tests under libcrypto.so.4:

  • fdbserver -r unittests -f /fdbrpc/JsonWebKeySet covers PEM/DER key round trips through EVP and OSSL_PARAM, using EC keys from MkCert's generator plus RSA keys.
  • -f /fdbrpc/TokenSign/JWT covers 100 ES256 JWT sign and verify iterations.
  • Each test requires Found N tests with N ≥ 1 and 1 tests passed; 0 tests failed. The unittests role also counts a pattern that matches nothing as a failure, so neither test can pass vacuously.

tls_loopback runs on 127.0.0.1 only and needs no network:

  • It generates a CA and P-256 certificates (with an IP SAN) using openssl.
  • It starts fdbserver on 127.0.0.1:4520:tls with --tls-verify-peers Check.Valid=1,S.CN=fdb-test.
  • Positive case: a client from the trusted CA with CN=fdb-test configures the database. It must see The database is available. and set and get a key.
  • Negative 1, the patched matcher: a client from the same CA with CN=fdb-intruder must be refused. The server's trace log must show TLSPolicyFailure Reason="Cert subject match failure". This is the path the const change touches.
  • Negative 2, chain verification: a client from an untrusted CA (same CA name, different key) must be refused with Reason="preverification failed".
  • Risk: this is the first TLS test for this package. If it can't run in the standalone sandbox, I'll report that here and keep the unit tests.

Note for the later 8.0 bump

🤖 Generated with Claude Code

FDB 7.3 fails to build against OpenSSL 4 in four places: three because
the X509 accessors now return const, one because ASN1_STRING is opaque.
- Carry foundationdb-7.3.81-openssl4.patch for flow/MkCert.cpp and
  flow/TLSConfig.actor.cpp. MkCert now builds the subject X509_NAME
  separately; TLSConfig's peer-rule matcher only gains `const`.
  build.sh applies the patch with `patch -F0` and checks sha256 of both
  files before and after.
- Pre-define BOOST_ASIO_SSL_RFC2818_VERIFICATION_HPP, so the bundled
  Boost 1.78 skips asio's deprecated rfc2818_verification.ipp. That
  header reads ASN1_STRING members directly, and FDB never uses it.

Build and run against openssl (4.x) in place of openssl-3-5. The link
check is flipped: fdbserver must NEED libssl.so.4, and nothing shipped
may NEED libssl.so.3 or libcrypto.so.3.

New tests:
- crypto_unittests: fdbserver -r unittests for /fdbrpc/JsonWebKeySet
  and /fdbrpc/TokenSign/JWT.
- tls_loopback: an OpenSSL-generated CA and certs on 127.0.0.1. A
  matching client must be accepted. A same-CA client with the wrong CN
  and a client from an untrusted CA must be refused, each with its
  reason in the server's trace log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 4 billable files and costs up to $1.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 58 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 67 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 9cebbef and 7bf54ae.

Files selected for processing: 4

  • AGENTS.md
  • packages/foundationdb/build.ncl
  • packages/foundationdb/build.sh
  • packages/foundationdb/foundationdb-7.3.81-openssl4.patch
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@bryan-minimal
bryan-minimal marked this pull request as ready for review November 19, 2026 00:47
@bryan-minimal
bryan-minimal requested a review from a team as a code owner November 19, 2026 00:47
The first run on build-bot passed the positive case and the intruder
refusal on both arches. The rogue (untrusted-CA) client never reached
verification: no TLSPolicyFailure was logged and the TLSPolicyFailures
counter stayed 0. After one failed handshake, fdbserver refuses the
same peer IP for 9s without a handshake
(TLS_SERVER_CONNECTION_THROTTLE_ATTEMPTS=1, _TIMEOUT=9, Net2.actor.cpp).
Every client here is 127.0.0.1, so the rogue client was dropped
unverified right after the intruder.

Start the server with
--knob_tls_server_connection_throttle_attempts 1000000, so each
refusal comes from real verification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant