Repository navigation
Bound numeric string length before parsing - #154689
Conversation
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
|
Hi @reugn, I've created a changelog YAML for you. |
🔍 Preview links for changed docs⏳ Building and deploying preview... View progress This comment will be updated with preview links when the build is complete. |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
| if (token == Token.VALUE_STRING) { | ||
| checkCoerceString(coerce, Short.class); | ||
|
|
||
| checkNumericStringLength(text()); |
There was a problem hiding this comment.
optimizedText() would allow to check the length without materializing the string
|
|
||
| // Numeric strings longer than this are rejected before coercion, whose cost grows with the digit count; | ||
| // matches the unquoted JSON number-token limit. Mirrored by AbstractXContentParser#MAX_NUMERIC_STRING_LENGTH. Keep in sync. | ||
| public static final int MAX_NUMERIC_STRING_LENGTH = 1000; |
There was a problem hiding this comment.
I don't see why we need support for nearly this many digits. Elasticsearch only supports indexing up to unsigned longs, not arbitrary length numbers.
There was a problem hiding this comment.
The limit matches the unquoted JSON number-token limit (Jackson's maxNumberLength default of 1000). Parsing a 1000-digit number is trivially fast.
There was a problem hiding this comment.
Parsing 1000 digits implies that ES will actually use it, but it won't. I don't think we need to match Jackson's default.
There was a problem hiding this comment.
Max unsigned long requires 20 chars, double in scientific notation needs 24. But plain double literals can take hundreds of digits, while still being valid, so 1000 gives safe headroom.
| public void testNumericCoercionRejectsOversizedString() throws IOException { | ||
| // Every numeric accessor bounds the length of a quoted string before coercing it, avoiding unbounded | ||
| // (and, for long, super-linear) parsing work on an attacker-sized value. | ||
| String oversized = "1" + "0".repeat(5000); |
There was a problem hiding this comment.
I think we want to structure this test to ensure that strings at MAX_NUMERIC_STRING_LENGTH parse but MAX_NUMERIC_STRING_LENGTH+1 throw an exception like the other tests below.
There was a problem hiding this comment.
Required making MAX_NUMERIC_STRING_LENGTH public, but agreed, it's better. Refactored the test.
chrisparrinello
left a comment
There was a problem hiding this comment.
LGTM! Thanks for taking care of this!
💔 Backport failed
You can use sqren/backport to manually backport by running |
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected.
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected. (cherry picked from commit f407f9a)
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected. (cherry picked from commit f407f9a)
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected. (cherry picked from commit f407f9a)
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected.
Coercing a numeric value supplied as a string goes through new BigDecimal(String) or new BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it. This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via Double.parseDouble or Float.parseFloat and not only the BigDecimal/BigInteger sites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected.
Coercing a numeric value supplied as a string goes through
new BigDecimal(String)ornew BigInteger(String), whose parsing cost grows with the number of digits (super-linearly for the BigDecimal and BigInteger radix-10 constructors). The JSON parser bounds the length of an unquoted number token, but a number supplied as a quoted string is not bounded, so a single small request carrying a long digit string can consume a large amount of CPU on the thread that parses it.This change bounds the length of a numeric string before it is parsed, using the same limit that already applies to an unquoted JSON number token. The check covers every path that coerces a user supplied numeric string to a number, including the accessors that parse via
Double.parseDoubleorFloat.parseFloatand not only theBigDecimal/BigIntegersites, so the bound is uniform for all numeric types: the streaming parser used at index time (every integer and floating point accessor), the number and unsigned_long field mappers, and the ES|QL and SQL numeric literal and cast parsers. A value longer than the limit is rejected before the costly numeric parsing and conversion, and the limit is far above any real numeric value so valid input is unaffected.The same string to number coercion is implemented independently in several layers, so the guard is added in each. The ES|QL and SQL/EQL libraries depend on server and reuse
Numbers.MAX_NUMERIC_STRING_LENGTHdirectly.libs/x-contentsits below server and cannot depend on it, so the same limit is duplicated inAbstractXContentParser, with matching values and a cross reference comment to keep the two in sync.