Repository navigation
fix(0.x): harden decoders and speed up hot paths (Viem audit) - #688
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Contributor
|
Published pnpm add ox@pr-688
pnpm add ox@0.0.0-pr-688-024b0ed |
Backports main's strict decoding. `Rlp.toBytes`/`Rlp.toHex`, and so every `TxEnvelope*.deserialize`, now throw `TrailingBytesError` when bytes follow the item, `ListBoundaryExceededError` when a child overruns its list, and `DepthLimitExceededError` past 1024 levels of nesting instead of overflowing the stack. The decoder now reads from a plain position over the bytes instead of a cursor, and the encoder measures then writes in two passes without closures. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
Backports main's check. `Hex.toNumber` and `Bytes.toNumber` throw `Hex.IntegerOutOfRangeError` instead of returning a rounded number (for example for `0x20000000000001`) or `NaN` (for `0x`). The ABI decoder reads offsets and lengths with its own lossy helper, so huge offsets still throw `PositionOutOfBoundsError` and AbiEvent still maps them to `DataMismatchError`. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
…pic size `AbiParameters.decode` throws `BytesSizeMismatchError` when the padding of a `bytesN` word is not zero, and `AbiEvent.decode` throws the new `TopicSizeMismatchError` when a topic for a statically typed indexed parameter is not 32 bytes. With topics known to be one word, canonical `address`, `uint256` and `bytes32` topics are read directly instead of through `AbiParameters.decode`. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
`Siwe.generateNonce` used `uid(96)`, which read a `Math.random` buffer through a window that slid one character per call, so consecutive nonces shared 95 of 96 characters. It now returns `Hex.random(48)` without the prefix. `internal/uid.ts` had no other users and is removed. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
`Transaction`, `TransactionReceipt` and `TransactionRequest` (core and Tempo) map RPC `type` strings through plain objects, so a type of `"constructor"` resolved to the `Object` function. `TypedData` did the same with struct and field type names, which broke `hashStruct`. These lookups now check `Object.hasOwn` first. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
`Cbor.decode`, which parses WebAuthn attestations, recursed without a limit. It now throws `Cbor.DepthLimitExceededError` when arrays, maps or tags nest deeper than 1024 instead of overflowing the stack. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
`Json.parse` threw on untrusted text such as `{"a":"x#__bigint"}`. A
string that ends in the bigint marker but is not a bigint is now kept as
a string.
Without a reviver, text that cannot contain the marker (literally or
through a `\u` escape) now parses without the reviver wrapper.
Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2
Co-authored-by: jxom <jake@tempo.xyz>
…ase key Backports main's `internal/address.ts` classifier. `Address.assert` and `Address.validate` now check the shape and casing in one pass without building and catching errors, and reject values that are not strings, which `strict: false` used to coerce. `Address.checksum` caches only the lowercase address, so every casing of an address shares one entry instead of each taking a slot. `LruMap.set` now refreshes an existing key's recency, and an empty-string key can be evicted. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
`HttpError` passed the request body, which is usually already JSON, through `JSON.stringify` again, so the error showed it escaped and quoted. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
- `Bytes.toBigInt` reads a full 32-byte word as four 64-bit halves, and `Bytes.toNumber` builds values of up to 6 significant bytes as numbers. - `Hex.fromNumber` encodes non-negative bigints that fit `size` without the range check, and keeps that check in a separate function so that `fromNumber` stays small enough to inline. Signed `Hex.toBigInt` uses `BigInt.asIntN`. - `AbiParameters.encode` joins parts once instead of spreading them into `Hex.concat`, which overflowed the stack for a `uint256[]` of 200k elements, and caches integer bounds per type. - Cursors create their `DataView` on first use. - `withTimeout` builds its `TimeoutError` only once a request times out. Co-authored-by: jxom <jake@tempo.xyz>
…ders once `TransactionRequest.toRpc` leaves out `data`, `input`, `to` and `value` for Tempo requests instead of deleting them afterwards, and `Transaction.fromRpc` leaves out `aaAuthorizationList` the same way. `TxEnvelopeTempo.deserialize` checks that the fee payer field is a string before validating it as an address, once, so a fee payer signature tuple no longer goes through `Address.validate` twice. Amp-Thread-ID: https://ampcode.com/threads/T-01a1e175-cf94-74ff-8a7c-572b7cb2b7d2 Co-authored-by: jxom <jake@tempo.xyz>
…d map only own keys in receipts
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the findings of a Viem audit onto
0.x, the line Viem pins (0.14.x). Wheremainalready had a fix, this PR backports it.This branch is rebased onto #687, which landed several of the same perf items first: native
Hex.fromBytes, the nibble-tableBytes.fromHex, the cursor read bitset,Hex.concat/trim/pad,Hex.validate,decodeAddress/getArrayComponents, and a global type-hash cache inTypedData. Those parts are #687's code, and the numbers below are measured againstorigin/0.xwith #687 included.Commits
Each commit typechecks and passes the tests it touches on its own.
fix(Rlp): strict decoding, plus a cursor-free decoder and a two-pass encoder.fix(Hex):toNumberthrows when the value is not a safe integer.fix(AbiParameters, AbiEvent): canonicalbytesNpadding and topic size, plus a direct path for canonical topics.fix(Siwe): random nonces.fix: own-key lookups for untrusted type names.fix(Cbor): depth limit.fix(Json): marker strings that are not bigints, and skip the reviver when the text cannot contain the marker.fix(Address): one-pass classifier, rejection of non-strings, lowercase-only checksum cache, andLruMapfixes.fix(RpcTransport):HttpErrorbody.perf(core):Bytes.toNumber/toBigInt, bigintHex.fromNumber, signedHex.toBigInt,AbiParameters.encodejoining and integer bounds, a lazy cursorDataView, and a lazywithTimeouterror.perf(tempo): formatters built withoutdelete, and a single sender check inTxEnvelopeTempo.deserialize.Behaviour changes
Each of these has a changeset.
RLP strict decoding (from
main):Rlp.toBytes/Rlp.toHex, and so everyTxEnvelope*.deserializeincluding Tempo, throw:Rlp.TrailingBytesErrorfor bytes after the item, such asRlp.toHex('0x01ff')ordeserialize(tx + 'ff').Rlp.ListBoundaryExceededErrorwhen a child overruns its list, such as'0xc18180'.Rlp.DepthLimitExceededErrorpast 1024 levels of nesting, which used to overflow the stack.As a result,
AccountSimulation.fromRpcwith trailing config bytes now throwsRlp.TrailingBytesError, where it used to throwInvalidSimulationError: noncanonical config encoding.SignaturePreAuthorized.unwrapshows the RLP message in its details.Hex.toNumberandBytes.toNumber(frommain) throwHex.IntegerOutOfRangeErrorinstead of returning a rounded number ('0x20000000000001') orNaN('0x'). The ABI decoder reads offsets and lengths with a local lossy helper, so huge offsets still throwPositionOutOfBoundsErrorand AbiEvent'sDataMismatchErrormapping is unchanged.Siwe.generateNoncedraws each nonce fromcrypto.getRandomValues(Hex.random(48).slice(2)). Before, consecutive nonces shared 95 of 96 characters.internal/uid.tshad no other users and is removed.Canonical ABI data:
AbiParameters.decodethrowsBytesSizeMismatchErrorwhenbytesNpadding is not zero.AbiEvent.decodethrows the newAbiEvent.TopicSizeMismatchErrorwhen a topic for a statically typed indexed parameter is not exactly 32 bytes.Untrusted type names such as
"constructor"are looked up withObject.hasOwninTransaction,TransactionReceiptandTransactionRequest(core and Tempo), and inTypedData. Before, an RPCtype: "constructor"turnedtx.typeinto theObjectfunction, and a field type of"constructor"brokehashStruct.Cbor.decodethrowsCbor.DepthLimitExceededErrorpast 1024 levels of nesting.Json.parsekeeps a string that ends in#__bigintbut is not a bigint as a string, where it used to throw (Json.parse('{"a":"x#__bigint"}')).Address.assertandAddress.validatereject values that are not strings. Before,strict: falsecoerced them to strings, soAddress.validate({ toString: () => address }, { strict: false })returnedtrue. In strict modeassertthrew aTypeError. Both now reject such values withInvalidAddressError/false.Address.checksumcaches only the lowercase address. Every casing of an address shares one cache entry, where each casing used to take its own slot.LruMap.setnow refreshes an existing key's recency, and an empty-string key can be evicted.RpcTransport.HttpErrorshows a request body that is already a string as is, instead of encoding it twice.Accepted defaults
These were reviewed and accepted as they are:
Address.checksumor strictAddress.validatemisses the direct lookup, so it pays for a shape check and a lowercase lookup instead of oneget. Lowercase and invalid input get faster. See theAddress.validaterows below.Address.validaterejects non-strings without coercing them (above).32 * (len + 8192)).mainuses the same budget, and flat strings from nativeHex.fromBytes(perf(0.x): hot-path speed and memory improvements, and Blobs.to multi-blob fix #687) already remove most of the memory amplification.Json.stringifydoes not try plainJSON.stringifyfirst: that is about 1.9× faster without bigints but 0.73× with them, and it would run getters andtoJSONtwice.TypedData: perf(0.x): hot-path speed and memory improvements, and Blobs.to multi-blob fix #687's global type-hash cache already covers that item. A per-call map on top measured within noise (0.99–1.09×), so it was dropped.Performance
These were measured on Node 26 / V8 in an orb, against
origin/0.x(with #687). Each run interleaves0.xand this branch seven times in one process, and each library builds its own fixtures. The table shows the median of three runs, with the range of their speedups. The orb's CPU speed varied between runs, so the absolute times move around, but the interleaved ratios are stable. The control rows, whose code this PR doesn't touch, stay within ±5% of 1.00×.0.xControls: paths that use #687's code unchanged (expected ≈1.00×)
0.xNotes:
0.x. Most of the 1.5–2.1× measured before perf(0.x): hot-path speed and memory improvements, and Blobs.to multi-blob fix #687 came from the hex and byte conversions that perf(0.x): hot-path speed and memory improvements, and Blobs.to multi-blob fix #687 landed first.TypedData.getSignPayloadis about 4% slower, from theObject.hasOwnguards on type lookups.Address.validateis 0.59×, the accepted cost of the lowercase-only cache (above).Hex.fromNumberkeeps its range check in a separate function. Adding the bigint path inline pushedfromNumberpast V8's inlining budget, which made the number path about 30% slower. Confirmed with--max-inlined-bytecode-size.Verification
vitest --project core --project tempo-unit: 3384 passed and 2 failed. Both failures arebehavior: networkenvelope tests, which fail on untouched0.xtoo and passed in CI on the previous push.tsc --noEmit, biome and the tsdoc lint are clean, and every commit typechecks on its own.origin/0.xran 316,657 checks with random and edge-case inputs across Hex, Bytes, Rlp, ABI encode/decode, AbiEvent, TypedData, Json, the envelopes and the Tempo formatters. It found no differences beyond the intended changes above.