Add telemetry around signature validation
This is a cherry pick from #3410 to dev8x
Original PR's description
This pull request introduces telemetry tracking for JWT and SAML/2 signature validation in the JsonWebTokenHandler and related classes. It refactors several methods to non-static to support instance-level telemetry. The changes enable detailed telemetry reporting on signature validation outcomes, such as missing keys, unsupported algorithms, and verification failures.
Telemetry integration for JWT and SAML signature validation:
- Added telemetry reporting to all major signature validation paths in
JsonWebTokenHandler, SamlSecurityTokenHandler, and Saml2SecurityTokenHandler, capturing events like signing key not found, algorithm not supported, signature provider creation failure, and signature verification failure or success. This is accomplished by calling the new RecordSignatureValidationTelemetry helper at appropriate points.
- Refactored
ValidateSignature, ValidateSignatureUsingAllKeys, and ValidateSignatureWithKey methods from static to instance methods, allowing access to the instance telemetry client (_telemetryClient).
- Updated the signature validation logic to pass the telemetry client through the call stack, ensuring telemetry is recorded for all signature validation attempts, including when using all keys or a specific key.
These changes lay the groundwork for comprehensive telemetry on signature validation, improving observability and diagnostics for token validation scenarios.
New Telemetry Counter
Signature Validation Counter
The counter tracks 5 dimensions (tags):
| Tag Name |
Tag Key |
Purpose |
Value Source |
| Library Version |
IdentityModelVersion |
Library version tracking |
IdentityModelTelemetryUtil.ClientVer |
| Algorithm |
Algorithm |
Signature algorithm from token header |
Token alg claim |
| Key Algorithm |
KeyAlgorithm |
Key type and size identifier |
CryptoTelemetry.GetKeyAlgorithmId(SecurityKey) |
| Issuer |
Issuer |
Token issuer (allowlisted hosts only) |
CryptoTelemetry.GetTrackedIssuerOrOther(issuer) |
| Error |
Error |
Error type or success indicator |
TelemetryConstants.SignatureValidationErrors.* |
Tag Cardinality Breakdown
1. IdentityModelVersion
- Cardinality: Low (~5-10 active versions)
- Values: Semantic versions (e.g.,
"7.3.1", "8.0.0")
- Typical Production: 1-3 versions during rollouts
- Lifecycle: Reduces over time as deployments converge
2. Algorithm
- Cardinality: ~12-15
- Possible Values (from JWT/SAML specs):
| Algorithm Family |
Algorithms |
Count |
| RSA (PKCS#1 v1.5) |
RS256, RS384, RS512 |
3 |
| RSA-PSS |
PS256, PS384, PS512 |
3 |
| ECDSA |
ES256, ES384, ES512, ES256K |
4 |
| HMAC |
HS256, HS384, HS512 |
3 |
| Legacy/Rare |
none, custom algorithms |
~2-5 |
- Typical Production: 2-5 algorithms (commonly
RS256, ES256, PS256)
- Bounded: JWT/JOSE specs define a finite set of algorithms
3. KeyAlgorithm
- Cardinality: ~11-13
- Implementation:
CryptoTelemetry.GetKeyAlgorithmId(SecurityKey) returns predefined constants
- Possible Values:
| Key Type |
Key Algorithm IDs |
Description |
| RSA |
RSA-2048, RSA-3072, RSA-4096, RSA-UNKNOWN |
RSA public/private keys |
| ECDSA |
ECDSA-P256, ECDSA-P384, ECDSA-P521, ECDSA-UNKNOWN |
Elliptic curve keys |
| Symmetric |
SYM-128, SYM-192, SYM-256, SYM-384, SYM-512, SYM-UNKNOWN |
HMAC shared secrets |
| Special |
NO-KEY, UNKNOWN |
Missing key or unsupported key type |
- Typical Production: 3-6 key types (commonly
RSA-2048, RSA-4096, ECDSA-P256)
- Bounded: Fixed set of industry-standard key sizes
4. Issuer
- Cardinality: Low (~5-20 tracked hosts)
- Implementation:
CryptoTelemetry.GetTrackedIssuerOrOther(issuer) with allowlist-based filtering
- Behavior:
- Extracts host from issuer URI (e.g.,
https://login.microsoftonline.com/tenant/ → login.microsoftonline.com)
- Returns host if in
CryptoTelemetry.TrackedIssuers allowlist
- Returns
"other" for all non-allowlisted issuers
- Typical Values:
login.microsoftonline.com (Microsoft Entra ID)
accounts.google.com (Google)
appleid.apple.com (Apple)
other (catch-all for non-tracked issuers)
- Cardinality Control: Allowlist prevents unbounded growth from arbitrary issuers
- Default: Empty allowlist (all issuers reported as
"other")
5. Error
- Cardinality: 6 fixed values
- Implementation:
TelemetryConstants.SignatureValidationErrors.*
- Possible Values:
| Error Type |
Constant |
Meaning |
| Success |
None |
Signature validation succeeded |
| Verification Failed |
SignatureVerificationFailed |
Signature invalid (key present, crypto works, but signature doesn't match) |
| Algorithm Not Supported |
AlgorithmNotSupported |
Algorithm not supported by key or crypto provider |
| Provider Creation Failed |
SignatureProviderCreationFailed |
Crypto provider could not create signature provider |
| Signing Key Not Found |
SigningKeyNotFound |
No signing key was found or resolved |
| Other |
Other |
Other errors not covered by specific categories |
- Typical Production: 2-4 error types observed (commonly
None, SignatureVerificationFailed, SigningKeyNotFound)
- Bounded: Fixed set of error constants to prevent cardinality explosion
Total Cardinality Calculation
Total Combinations = IdentityModelVersion × Algorithm × KeyAlgorithm × Issuer × Error
= 5 × 15 × 13 × 20 × 6
= 117,000 theoretical maximum (with full issuer allowlist)
Production Reality (Empty Issuer Allowlist - Default):
Active Versions × Active Algorithms × Active Key Types × Issuer ("other" only) × Active Errors
= 2 × 4 × 4 × 1 × 3
= 96 typical active time series
Production Reality (With 5 Tracked Issuers):
Active Versions × Active Algorithms × Active Key Types × (Tracked Issuers + "other") × Active Errors
= 2 × 4 × 4 × 6 × 3
= 576 typical active time series
Upper Bound Estimate:
- Default (no issuer tracking): ~150-250 time series
- With issuer tracking (5-10 issuers): ~1,000-2,000 time series
Note: The issuer dimension is strictly controlled via the CryptoTelemetry.TrackedIssuers allowlist, preventing unbounded cardinality growth.
Cardinality Assessment
✅ Low-Medium Cardinality - Safe for production at scale
Rationale:
- ✅ Issuer dimension is strictly allowlist-controlled (default: all issuers →
"other")
- ✅ Error dimension is a fixed enumeration (6 values)
- ✅ Algorithm set is finite and standardized
- ✅ Key sizes are industry-standard values (not arbitrary)
- ✅ Library versions naturally consolidate over time
Benchmarks
For loop 100_000 calls + 10 OperationsPerInvoke
| Method |
Mean |
Error |
StdDev |
Median |
P90 |
P95 |
P100 |
Ratio |
RatioSD |
Gen0 |
Allocated |
Alloc Ratio |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryDisabled |
180.9 ms |
1.81 ms |
3.77 ms |
179.5 ms |
185.9 ms |
188.3 ms |
193.1 ms |
1.00 |
0.03 |
4300.0000 |
38.68 MB |
1.00 |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryEnabledNoTracking |
179.5 ms |
4.20 ms |
9.13 ms |
175.2 ms |
190.0 ms |
195.0 ms |
225.0 ms |
0.99 |
0.05 |
4300.0000 |
38.68 MB |
1.00 |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryEnabledWithTracking |
178.2 ms |
2.31 ms |
5.12 ms |
176.4 ms |
185.9 ms |
188.2 ms |
191.7 ms |
0.99 |
0.03 |
4300.0000 |
38.68 MB |
1.00 |
No for loop, 100_000 OperationsPerInvoke
| Method |
Mean |
Error |
StdDev |
P90 |
P95 |
P100 |
Ratio |
RatioSD |
Allocated |
Alloc Ratio |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryDisabled |
0.8680 ns |
0.0362 ns |
0.0779 ns |
0.9705 ns |
1.000 ns |
1.064 ns |
1.01 |
0.13 |
- |
NA |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryEnabledNoTracking |
1.0351 ns |
0.1411 ns |
0.2882 ns |
1.3120 ns |
1.621 ns |
2.207 ns |
1.20 |
0.35 |
- |
NA |
| JsonWebTokenHandler_ValidateTokenAsync_TelemetryEnabledWithTracking |
1.0730 ns |
0.1187 ns |
0.2556 ns |
1.4330 ns |
1.605 ns |
1.770 ns |
1.25 |
0.31 |
- |
NA |
- Remove TelemetryConfiguration enum from benchmarks; update benchmark attributes and tracked issuer values for consistency.
- Refactor and expand CryptoTelemetry tests: remove host extraction tests, consolidate key algorithm ID tests, and add more scenarios for tracked issuer matching and allowlist filtering.
- Update API documentation to reflect field renaming.
- Overall, unify telemetry client usage and streamline issuer tracking logic, with tests updated to match new behavior.
- Removed case insensitive comparison for telemetry issuer extraction based on PR feedback
(cherry picked from commit 61449b853c3ce153aa2a21ed75d5a9bc27af37cf)
Originally posted by @iNinja in AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet#3415
Add telemetry around signature validation
This is a cherry pick from #3410 to dev8x
Original PR's description
This pull request introduces telemetry tracking for JWT and SAML/2 signature validation in the
JsonWebTokenHandlerand related classes. It refactors several methods to non-static to support instance-level telemetry. The changes enable detailed telemetry reporting on signature validation outcomes, such as missing keys, unsupported algorithms, and verification failures.Telemetry integration for JWT and SAML signature validation:
JsonWebTokenHandler,SamlSecurityTokenHandler, andSaml2SecurityTokenHandler, capturing events like signing key not found, algorithm not supported, signature provider creation failure, and signature verification failure or success. This is accomplished by calling the newRecordSignatureValidationTelemetryhelper at appropriate points.ValidateSignature,ValidateSignatureUsingAllKeys, andValidateSignatureWithKeymethods from static to instance methods, allowing access to the instance telemetry client (_telemetryClient).These changes lay the groundwork for comprehensive telemetry on signature validation, improving observability and diagnostics for token validation scenarios.
New Telemetry Counter
Signature Validation Counter
The counter tracks 5 dimensions (tags):
IdentityModelVersionIdentityModelTelemetryUtil.ClientVerAlgorithmalgclaimKeyAlgorithmCryptoTelemetry.GetKeyAlgorithmId(SecurityKey)IssuerCryptoTelemetry.GetTrackedIssuerOrOther(issuer)ErrorTelemetryConstants.SignatureValidationErrors.*Tag Cardinality Breakdown
1. IdentityModelVersion
"7.3.1","8.0.0")2. Algorithm
RS256,RS384,RS512PS256,PS384,PS512ES256,ES384,ES512,ES256KHS256,HS384,HS512none, custom algorithmsRS256,ES256,PS256)3. KeyAlgorithm
CryptoTelemetry.GetKeyAlgorithmId(SecurityKey)returns predefined constantsRSA-2048,RSA-3072,RSA-4096,RSA-UNKNOWNECDSA-P256,ECDSA-P384,ECDSA-P521,ECDSA-UNKNOWNSYM-128,SYM-192,SYM-256,SYM-384,SYM-512,SYM-UNKNOWNNO-KEY,UNKNOWNRSA-2048,RSA-4096,ECDSA-P256)4. Issuer
CryptoTelemetry.GetTrackedIssuerOrOther(issuer)with allowlist-based filteringhttps://login.microsoftonline.com/tenant/→login.microsoftonline.com)CryptoTelemetry.TrackedIssuersallowlist"other"for all non-allowlisted issuerslogin.microsoftonline.com(Microsoft Entra ID)accounts.google.com(Google)appleid.apple.com(Apple)other(catch-all for non-tracked issuers)"other")5. Error
TelemetryConstants.SignatureValidationErrors.*NoneSignatureVerificationFailedAlgorithmNotSupportedSignatureProviderCreationFailedSigningKeyNotFoundOtherNone,SignatureVerificationFailed,SigningKeyNotFound)Total Cardinality Calculation
Production Reality (Empty Issuer Allowlist - Default):
Production Reality (With 5 Tracked Issuers):
Upper Bound Estimate:
Note: The issuer dimension is strictly controlled via the
CryptoTelemetry.TrackedIssuersallowlist, preventing unbounded cardinality growth.Cardinality Assessment
✅ Low-Medium Cardinality - Safe for production at scale
Rationale:
"other")Benchmarks
For loop 100_000 calls + 10 OperationsPerInvoke
No for loop, 100_000 OperationsPerInvoke
(cherry picked from commit 61449b853c3ce153aa2a21ed75d5a9bc27af37cf)
Originally posted by @iNinja in AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet#3415