Repository navigation
Conversation
…ition. (#792) One problem with this highlighting approach is that it sometimes doesn't disappear even after the cursor is no longer over the rule's name. The second problem is that when a relatively large rule is highlighted, it can be quite annoying. Instead of highlighting the entire rule definition, I suggest highlighting only the name, if that is acceptable. It is what I've changed in this PR. --------- Co-authored-by: Victor M. Alvarez <vmalvarez@virustotal.com>
The text explaining how to install `yara-x` using `brew` in macOS was in the wrong section of the documentation.
Also generalize the extraction of header constraints from integer reads so that it supports reads of any width, including 8-byte reads. Part of #26.
Integers are still stored as `i64`, but `TypeValue::Integer` now has an `unsigned` flag indicating that the value must be interpreted as the bit pattern of an `u64`. Signed and unsigned integers share `Type::Integer`, so existing type checks accept both, and only operations where signedness matters need to take it into account. Nothing produces unsigned integers yet, so there are no user-visible changes. The serialization version is bumped because `TypeValue` is part of the serialized globals. Part of #26.
Integer literals that don't fit in a signed 64-bit integer but fit in an unsigned one (e.g: `0xFFFFFFFFFFFFFFFF`) are now unsigned integers. * Comparisons between signed and unsigned integers are mathematically exact (e.g: `0xFFFFFFFFFFFFFFFF > -1` is always true). * Arithmetic and bitwise operations produce an unsigned result only when every operand is either unsigned or a non-negative constant, otherwise the result is signed. Division and modulo use unsigned instructions for unsigned results. * Shifts take the signedness of the left operand. Right shifts of unsigned integers are logical. * Unary minus always produces a signed result, bitwise not preserves signedness. Both emit the new `unsigned_unary_op` warning when applied to unsigned integers. * Constant folding of integer arithmetic is now exact (previously it was performed using floats, losing precision for large values), and results that don't fit in the result type are reported as errors. Part of #26.
These functions read a 64-bit unsigned integer from the scanned data, in little-endian and big-endian respectively. The result is an unsigned integer, so values greater than `0x7FFFFFFFFFFFFFFF` are never negative. Functions exported to WASM can now return `u64`, which is represented as `i:U` in mangled names. Header constraints are also extracted from comparisons involving `uint64` and `uint64be`. Part of #26.
Module fields of type `uint64` are now unsigned integers, instead of being reinterpreted as signed integers. This means that values greater than `0x7FFFFFFFFFFFFFFF` are no longer negative, and that comparisons and arithmetic follow the rules for unsigned integers. Repeated `uint64` fields are represented by the new `Array::UnsignedIntegers` variant. Map keys of type `uint64` still use signed integers, but lookups with unsigned keys work as expected because keys are compared by their bit patterns. Part of #26.
Instead of an `i64` value plus an `unsigned` flag, `LiteralInteger::value` is now an `IntegerValue` enum with `Signed(i64)` and `Unsigned(u64)` variants. Part of #26.
They are now `TypeValue::const_signed_integer_from` and `TypeValue::const_unsigned_integer_from`, for coherency. Part of #26.
They are now `TypeValue::unknown_signed_integer` and `TypeValue::unknown_unsigned_integer`, for coherency. Part of #26.
…eValue`. `unknown_integer_with_constraints`, `var_integer_from`, `var_unsigned_from` and `try_as_unsigned` are now `unknown_signed_integer_with_constraints`, `var_signed_integer_from`, `var_unsigned_integer_from` and `try_as_unsigned_integer`, for coherency. Part of #26.
…flag. Unsigned integers are now represented by their own `TypeValue` variant, holding a `Value<u64>`, instead of a `TypeValue::Integer` with an `unsigned` flag and the `i64` bit pattern of the value. Both variants share the same `Type::Integer` type. Code that needs the raw 64 bits of an integer, regardless of its signedness (e.g: WASM lookups and constant folding of bitwise operations), uses the new `try_as_integer_bits` and `as_integer_bits` methods. Part of #26.
Also move `TypeValue::UnsignedInteger` next to it. This changes the discriminants of the variants that follow, and therefore the hashes in the IR golden files. Part of #26.
`IntegerConstraint::Range` now uses `i128` bounds, which can represent ranges for both signed and unsigned integers, and `TypeValue::UnsignedInteger` has a `constraints` field, like `TypeValue::SignedInteger`. Constraints are checked when a constant integer, signed or unsigned, is compared with a constrained integer of any signedness. Mangled names can also combine the `U` and `R` constraints (e.g: `i:U:R0:255`). Part of #26.
Having two `TypeValue` variants (`SignedInteger` and `UnsignedInteger`) with the largest size prevented the compiler from storing the enum discriminant in a niche, growing `Expr` from 24 to 28 bytes in 32-bit Linux. Integer constraints are now stored as `Option<Box<[_]>>` instead of `Option<Vec<_>>`, which also reduces `Expr` from 48 to 40 bytes in 64-bit platforms. Part of #26.
Renames `IntegerValue` enum variants from `Signed` to `I64` and `Unsigned` to `U64`. This explicitly indicates the underlying `i64` and `u64` primitive types, improving clarity and consistency in integer handling throughout the compiler's AST and IR stages.
…guard runtime bounds
…it integer comparisons
…d LiteralInteger. Remove TypeValue::SignedInteger, TypeValue::UnsignedInteger, and ast::IntegerValue. Both TypeValue::Integer and ast::LiteralInteger now store the integer bit pattern as an i64 along with an is_unsigned boolean flag.
…signed integer keys.
Ensure negative `i64` start values are explicitly clamped to `0` before conversion to `usize`. This clarifies the intent for handling out-of-bounds negative indices and improves robustness.
Rewrite the documentation section on signed and unsigned integers to provide a more detailed explanation of their behavior, especially when mixed in operations, and include illustrative examples.
Adds comprehensive documentation explaining how integer signedness is handled in YARA-X. Specifically clarifies why `uint8`, `uint16`, and `uint32` functions produce signed 64-bit integers with range constraints, rather than unsigned integers. This design prevents unexpected wrap-around on underflow and simplifies comparisons with other signed integers. Updates the `wasm_export` macro to consistently represent `RangedInteger` as a signed type with a range, aligning with the detailed explanation provided in the documentation.
…th non-negative ranged integers
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #796 +/- ##
==========================================
+ Coverage 93.25% 93.40% +0.14%
==========================================
Files 153 153
Lines 44178 44647 +469
==========================================
+ Hits 41200 41701 +501
+ Misses 2978 2946 -32 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This change aligns the internal representation and arithmetic behavior of protobuf `uint32` fields with the documented design for `uint8`, `uint16`, and `uint32` functions. These types are now treated as signed 64-bit integers with a value range constraint of `[0, u32::MAX]`, preventing unexpected wrap-around on underflow and simplifying comparisons with other signed integers.
plusvic
enabled auto-merge (squash)
October 9, 2026 10:05
plusvic
disabled auto-merge
October 9, 2026 10:08
The runtime guard added for range bounds treated any negative bound as an empty range. That is only correct for unsigned bounds greater than i64::MAX, which have a negative bit pattern when interpreted as i64. Signed lower bounds can legitimately be negative at runtime (e.g: `filesize - 10` or `@a[1] - 4`), and on main the loop iterated over the negative values. Restrict the check to unsigned bounds, while keeping the explicit `upper_bound < lower_bound` check, which prevents the number of iterations from wrapping around into a positive value.
The number of iterations required by a percentage quantifier is computed as `ceil(n * percentage / 100)` in floating point and converted back to an integer with `i64.trunc_f64_s`, which traps if the value doesn't fit in an i64. Percentages are guaranteed to be in [0, 100] only when they are constant, but with functions like `int64` and `uint64` a percentage computed from the scanned data can easily overflow, and the trap made the scanner panic. Use the saturating conversion `i64.trunc_sat_f64_s` instead, which produces i64::MIN or i64::MAX, values that are never reached. Also convert unsigned percentages to float as unsigned integers, so that values greater than i64::MAX are not treated as negative.
…d dotnet Protobuf `uint64` fields are unsigned integers in YARA rules, which means that subtractions that underflow wrap around to huge positive numbers (e.g: `pe.number_of_exports - 1 < 10` is false when there are no exports). This is a problem for fields that are counts, which never exceed i64::MAX, and are frequently used in arithmetic expressions. Count fields (`number_of_*`, `*_entries`) are now `uint32`, which in YARA rules are signed integers constrained to the range [0, 4294967295]. These values never wrap around, but they are still known to be non-negative, so combining them with an unsigned `uint64` value produces an unsigned result. Fields that can legitimately exceed i64::MAX, like addresses, offsets and sizes, remain `uint64`. The JSON/YAML output of module dumps is not affected, but the Rust types of these fields in `yara_x::mods` change from `u64` to `u32`.
The [0, u32::MAX] constraint for protobuf `uint32` fields was attached only when the field's value was unknown. Fields in proto3 modules always have a value (they default to 0, even at compile time), so they lost the constraint, and therefore weren't treated as non-negative integers. Attach the constraint regardless of the field's value.
The `*.no-folding.ir` golden files were outdated, and the ones for inputs 13 to 16 didn't exist. The error in `errors/161.in` and two assertions in `unsigned_integers` depend on constant folding, so they are now skipped when the feature is disabled.
When the whole workspace is tested, Cargo unifies the features of `yara-x` across all workspace members. Some of them depend on `yara-x` through the `[workspace.dependencies]` entry, which enables `constant-folding`, `exact-atoms`, `fast-regexp` and `generate-proto-code`, so the `no-default-features` job never tested `yara-x` without those features.
This reverts commit 736b06c, which changed these fields to `uint32`. That changed the type of the fields in the public `yara_x::mods` API from `u64` to `u32`, which is a breaking change. As a consequence, these fields are unsigned integers in YARA rules again.
…isfied. Negative quantifiers computed at runtime can't be satisfied, but small negative percentages were an exception. When `n * percentage` is in the range (-100, 0), `max_count` was rounded to 0, which means `none`. For instance, `for int8(0) % i in (0..9) : (false)` was true when `int8(0)` was -1. Negative percentages are now replaced with i64::MAX, which produces a `max_count` that is never reached.
… unsigned result. The signedness of arithmetic and bitwise operations depended on the values of the operands. A signed operand that was a non-negative constant adopted the signedness of the unsigned operands, while any other signed operand made the result signed. As the value of an expression like `4 \ 2` is known only when constant folding is enabled, `(4 \ 2) * (0xFFFFFFFFFFFFFFFF \ 2)` was unsigned with constant folding, and signed without it. Something similar happened with loop variables: `i * x` was signed even if `i` could only take the value 2, while `2 * x` was unsigned. Now the result is unsigned if at least one of the operands is unsigned, regardless of the remaining operands, which are interpreted as unsigned, like in C. This means that negative values wrap around (e.g: `uint64(0) + (-1)` is equivalent to `uint64(0) - 1`). The signedness depends only on the types of the operands, so it doesn't change with constant folding.
…tegers. The documentation said that the values returned by `uint8`, `uint16` and `uint32` are signed integers, which looks inconsistent with `uint64` being unsigned. These values are never negative, but they are promoted to signed 64-bit integers because all of them fit in one, like C promotes the integer types narrower than `int` to `int`. This also keeps them compatible with YARA. `uint64` values can't be promoted, which is why they are the only unsigned integers. The documentation now uses that explanation, and shows the difference between `uint32(0) - 1 < 0` and `uint64(0) - 1 < 0`.
Since unsigned literals were introduced, a minus sign followed by an unsigned literal other than 9223372036854775808 produced a unary minus applied to an unsigned integer, so literals like `-9223372036854775809` or `-18446744073709551615` were accepted and wrapped around (e.g: `-18446744073709551615 == 1` was true), instead of being rejected as before. Now these literals are invalid integers again. Additionally, constant folding of unary minus wrapped around silently, so `-(-9223372036854775808)` was equal to -9223372036854775808, while `0 - (-9223372036854775808)` was an error. Now unary minus is folded using exact arithmetic, like the other arithmetic operations, and produces an error when the result doesn't fit in a signed integer. This also applies to unsigned constants, like in `-(0xFFFFFFFFFFFFFFFF)`. When the operand is not known at compile time the result still wraps around.
…bool` consistent. Both functions now receive the compile context, the `ExprId` of the expression being checked and the span where the warning is reported, and they obtain the expression's type by themselves. `warn_if_unsigned` additionally receives the unary operator (`-` or `~`), and the note shown in the warning is now chosen from the operator instead of being passed by every caller. `warn_if_not_bool` was only used in `ast2ir.rs`, so it is now private.
`TypeValue::integer_range` returned `None` for constants, which is what prevented `eq_check` from checking a constant against the range of the other constant's type in comparisons like `0xFFFFFFFFFFFFFFFF == -1`. Now `integer_range` depends only on the type, and `eq_check` performs the range check only when one operand is constant and the other is not. The new `test_17` in `warnings/no_warnings.in` makes sure that comparing constants with different signedness doesn't raise `unsatisfiable_expr`.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for 64-bit integer reading functions (
int64,int64be,uint64,uint64be) and introduces 64-bit unsigned integer semantics across the parser, type system, IR, and WASM code generator.Key Changes
Built-in 64-bit read functions
int64(offset),int64be(offset),uint64(offset), anduint64be(offset)for reading 64-bit signed and unsigned integers (little-endian and big-endian) from scanned data.Unsigned 64-bit integer support
i64::MAX(0x7FFFFFFFFFFFFFFF) up tou64::MAX(0xFFFFFFFFFFFFFFFF) are now parsed and represented as 64-bit unsigned integers instead of being rejected as out of range.==,!=,<,<=,>,>=): Comparisons between signed and unsigned integers are mathematically exact at both compile time and runtime (e.g.,0xFFFFFFFFFFFFFFFF > -1istrue, and an unsigned integer is never equal to a negative signed integer).+,-,*,\,%,&,|,^):\) and remainder (%), and guarded against WASM runtime traps when dividingi64::MINby-1(evaluating toundefinedinstead).<<,>>): Preserve the signedness of the left operand, using logical right shifts (i64.shr_u) for unsigned integers and arithmetic right shifts (i64.shr_s) for signed integers.-,~):-) always produces a signed integer; bitwise NOT (~) preserves the operand's signedness.unsigned_unary_opcompiler warning when-or~is applied to an unsigned integer.IntegerConstraint::Rangetoi128bounds so both signed (i64) and unsigned (u64) ranges can be represented and checked forunsatisfiable_expr/invariant_exprwarnings.i64::MAXin range bounds and quantifiers where negative or out-of-rangei64values are invalid.