Repository navigation
foundationdb: build 7.3.81 against OpenSSL 4, drop openssl-3.5 - #1580
Open
bryan-minimal wants to merge 2 commits into
Open
bryan-minimal wants to merge 2 commits into
bryan-minimal wants to merge 2 commits into
Conversation
FDB 7.3 fails to build against OpenSSL 4 in four places: three because the X509 accessors now return const, one because ASN1_STRING is opaque. - Carry foundationdb-7.3.81-openssl4.patch for flow/MkCert.cpp and flow/TLSConfig.actor.cpp. MkCert now builds the subject X509_NAME separately; TLSConfig's peer-rule matcher only gains `const`. build.sh applies the patch with `patch -F0` and checks sha256 of both files before and after. - Pre-define BOOST_ASIO_SSL_RFC2818_VERIFICATION_HPP, so the bundled Boost 1.78 skips asio's deprecated rfc2818_verification.ipp. That header reads ASN1_STRING members directly, and FDB never uses it. Build and run against openssl (4.x) in place of openssl-3-5. The link check is flipped: fdbserver must NEED libssl.so.4, and nothing shipped may NEED libssl.so.3 or libcrypto.so.3. New tests: - crypto_unittests: fdbserver -r unittests for /fdbrpc/JsonWebKeySet and /fdbrpc/TokenSign/JWT. - tls_loopback: an OpenSSL-generated CA and certs on 127.0.0.1. A matching client must be accepted. A same-CA client with the wrong CN and a client from an untrusted CA must be refused, each with its reason in the server's trace log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Warning Review limit reached
This review includes 4 billable files and costs up to $1.00.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Or wait 58 minutes for your next included review. View limit details
|
The first run on build-bot passed the positive case and the intruder refusal on both arches. The rogue (untrusted-CA) client never reached verification: no TLSPolicyFailure was logged and the TLSPolicyFailures counter stayed 0. After one failed handshake, fdbserver refuses the same peer IP for 9s without a handshake (TLS_SERVER_CONNECTION_THROTTLE_ATTEMPTS=1, _TIMEOUT=9, Net2.actor.cpp). Every client here is 127.0.0.1, so the rogue client was dropped unverified right after the intruder. Start the server with --knob_tls_server_connection_throttle_attempts 1000000, so each refusal comes from real verification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR moves foundationdb 7.3.81 from the
openssl-3.5compat package toopenssl(4.x). It is a draft because it touches TLS: Bryan reviews it.openssl-3.5has three users left. This PR removes foundationdb. #1579 handles actions-runner, and the Rust stage0 rung is handled separately.What broke and why
FDB 7.3 fails to build against OpenSSL 4 for two reasons:
X509_get_subject_name(),X509_get_issuer_name(),X509_NAME_get_entry()andX509_NAME_ENTRY_get_data().ASN1_STRINGopaque.Upstream has no fix. apple/foundationdb#14348 pins OpenSSL 3.5 instead; it is open and its CI fails.
I compared FDB's OpenSSL use against the 4.1 headers,
ossl-removed-api(7)andossl-guide-migration(7), and found four breakages:flow/MkCert.cppmakeCertNative()X509_NAME, thenX509_set_subject_name(). The certificate encodes the same.flow/TLSConfig.actor.cppmatch_criteria_entry/match_name_criteriaX509_NAME_ENTRY/ASN1_STRINGassigned to non-constconstflow/TLSConfig.actor.cppcheck_verify/match_criteriaX509_NAME(subject/issuer) assigned to non-constconstrfc2818_verification.ipp, pulled in by<boost/asio/ssl.hpp>ASN1_STRINGmembers (->data,->length) directly-DBOOST_ASIO_SSL_RFC2818_VERIFICATION_HPPto CXXFLAGS. FDB never uses that deprecated class. Boost 1.87 removed the file, but FDB 7.3 needs Boost 1.86 or older forio_service.Everything else FDB uses compiles against 4.x:
ERR_remove_*,ASN1_STRING_data).RSA_*,EC_KEY_*,HMAC_*,RAND_set_rand_method) are still in 4.1 and only warn.USE_WERRORis off.Nothing was compiled locally. Build-bot's run of this PR is the first real compile.
The patch (
packages/foundationdb/foundationdb-7.3.81-openssl4.patch, 18 changed lines)constqualifiers only. No comparison, branch or return value in the peer-rule matcher changes, so the review is about 7 lines.mkcert, unit tests). It is not on the verification path.patch -Np1 -F0, with sha256 checks of both files before and after (the same pattern as llama.cpp). A different upstream file, or a partial apply, fails the build.Recipe
openssl-3-5is replaced byopensslin build_deps and runtime_deps;patchis added to build_deps;Localdep.-Iand CMakeOPENSSL_*paths, so CMake finds the system OpenSSL 4;fdbservermust NEEDlibssl.so.4, and nothing shipped may NEEDlibssl.so.3orlibcrypto.so.3.packages/openssl-3.5/build.nclthat still lists foundationdb as a user. I left it so this PR doesn't touch (and possibly rebuild) openssl-3.5. It can go when the last user does.Tests
The existing
server_smoketest never exercised TLS. Two new standalone tests do.crypto_unittestsruns FDB's own unit tests under libcrypto.so.4:fdbserver -r unittests -f /fdbrpc/JsonWebKeySetcovers PEM/DER key round trips through EVP and OSSL_PARAM, using EC keys from MkCert's generator plus RSA keys.-f /fdbrpc/TokenSign/JWTcovers 100 ES256 JWT sign and verify iterations.Found N testswith N ≥ 1 and1 tests passed; 0 tests failed.The unittests role also counts a pattern that matches nothing as a failure, so neither test can pass vacuously.tls_loopbackruns on 127.0.0.1 only and needs no network:openssl.127.0.0.1:4520:tlswith--tls-verify-peers Check.Valid=1,S.CN=fdb-test.CN=fdb-testconfigures the database. It must seeThe database is available.and set and get a key.CN=fdb-intrudermust be refused. The server's trace log must showTLSPolicyFailure Reason="Cert subject match failure". This is the path theconstchange touches.Reason="preverification failed".Note for the later 8.0 bump
TLSConfig.cppneeds about 6 moreconstedits there. 8.0 bundles Boost 1.89, so it doesn't need the Boost define.SSL_set1_host, Authenticate external TLS servers by CA chain and hostname, not peer rules apple/foundationdb#14449) for connections made by hostname.SSL_set1_hostis also deprecated in 4.0. That is only a warning for us, but upstream's-WerrorCI will trip on it.🤖 Generated with Claude Code