Skip to content

Fetch assets for multiple identities in one pipelined batch - #59

Open
0xluk wants to merge 3 commits into
mainfrom
feature/58-pipelined-batch-assets
Open

Fetch assets for multiple identities in one pipelined batch#59
0xluk wants to merge 3 commits into
mainfrom
feature/58-pipelined-batch-assets

Conversation

@0xluk

@0xluk 0xluk commented Aug 11, 2026

Copy link
Copy Markdown

Closes #58.

Adds POST /assets, which returns the owned and possessed assets of several identities in one call, using the request pipelining added in go-node-connector v2.3.0.

Callers that needed assets for many identities previously had to fan out N*2 requests, one per identity per asset kind, which saturated the connection pool. PrefetchOwnedAndPossessedAssets writes all 2N requests back to back on a single connection and demultiplexes the responses by DejaVu nonce, so a batch costs roughly one round trip for the whole set.

API

curl -X POST localhost:8000/assets \
  -H 'Content-Type: application/json' \
  -d '{"identities":["IGJQ...MKBC","AFZP...PCVJ"]}'

Returns {identity, ownedAssets[], possessedAssets[]} per identity, in request order. Every requested identity appears in the response, with empty lists if it holds nothing.

A few decisions worth flagging for review:

  • One combined RPC rather than separate owned/possessed batch endpoints. The connector always fetches both in a single batch, so splitting them would make the node do both halves of the work and discard one.
  • POST rather than GET. An identity is 60 characters, so a batch of any useful size would push a GET URL past common proxy limits. Follows the existing body: "*" precedent from BroadcastTransaction and QuerySmartContract.
  • Response reuses the existing OwnedAsset / PossessedAsset messages, so a client that already parses /assets/{identity}/owned needs no new parsing code.

Validation

All validation runs before a pooled connection is taken, so malformed input never costs one. Empty lists, batches over the limit, malformed identities and duplicates are rejected with INVALID_ARGUMENT. The limit defaults to 15 and is configurable via QUBIC_API_SIDECAR_SERVER_MAX_BATCH_IDENTITIES.

A failed batch leaves the connection byte-unaligned, so the handler closes it rather than returning it to the pool. There is a test pinning that specifically, since putting a dead connection back would poison the pool for later requests.

⚠️ Behavior change to an existing endpoint

The nested owned_asset of /assets/{identity}/possessed was built from the possession record's own fields rather than the ownership record it wraps. Only ownerIdentity was correct. It reported:

field before after
type 3 (possession) 2 (ownership)
padding, managingContractIndex, issuanceIndex, numberOfUnits the possession record's the ownership record's

These are two independently unmarshalled structs with identical field names, which is how they got mixed up. That the issuanceIndex slot is really an ownership index is visible in the connector's parallel struct, which names it OwnershipIndex.

Extracting the duplicated conversion into shared converters fixes this by construction, since the converter reads only the ownership record it is handed. TestAssetConverter_convertPossessedAsset gives the possession and its nested ownership record deliberately different values so this cannot silently regress — I verified it fails against the old behavior. README samples updated to match; the other nested sample values are unchanged real-node output I could not re-verify, so a second look from someone with a node handy is welcome.

/assets/{identity}/owned and /assets/{identity}/issued output is unchanged.

Testability

The issue asks for unit-testable code. Server.qPool was the concrete *qubic.Pool, whose Get returns a client holding a real socket, so no handler could be tested. Handlers now reach the pool through a nodePool / nodeClient seam.

The seam covers every client method the handlers already use, which is why no existing handler body changed — only the field type and one function parameter. var _ nodeClient = (*qubic.Client)(nil) keeps the interface honest as the client evolves.

Tests

20 new cases, none requiring a live node:

  • Converters: owned, possessed (the regression test above), issued, order preservation, non-nil empty slices, and the batch mapping.
  • Handler: happy path, identity holding no assets, all four rejection paths, pool-get failure, and prefetch failure closing rather than returning the connection.

Verification

go build, go vet and go test ./... -race all pass, and each of the three commits builds and tests green on its own. I also ran the service and exercised the real HTTP route: all four validation errors return before touching the pool, a valid request reaches the node layer, and the existing /assets/... routes are not shadowed by the new POST /assets.

Incidental

protobuff/Makefile did not pass the protobuf include path to its openapi-v3 target, only to the Go codegen target, so on macOS make all always failed to find the well-known types and never regenerated the OpenAPI document. One-line fix, included so ./make.sh works.

Note this touches protobuff/*.proto, so merging will dispatch doc rebuilds to qubic/integration and qubic/docs.

🤖 Generated with Claude Code

0xluk and others added 3 commits August 11, 2026 11:57
v2.3.0 adds PrefetchOwnedAndPossessedAssets, which pipelines the owned and
possessed asset requests for several identities over a single connection. The
only changed file upstream is qubic.go; NewClient gained variadic options and
stays source compatible, so nothing here needed adapting.

Also pass the protobuf include path to the openapi-v3 target. It was only
applied to the Go codegen target, so on macOS `make all` always failed to find
the well known types and never regenerated the OpenAPI document.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds GetAssetsForIdentities, bound to POST /assets, returning the owned and
possessed assets of each requested identity.

The request takes a list rather than a path parameter, and is a POST so the
identities travel in the body: an identity is 60 characters, so a batch of any
useful size would push a GET URL past common proxy limits. This follows the
existing body: "*" precedent set by BroadcastTransaction and QuerySmartContract.

The response reuses the existing OwnedAsset and PossessedAsset messages, so a
client that already parses /assets/{identity}/owned needs no new parsing code.

Generated files are produced by protobuff/make.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Implements GetAssetsForIdentities on top of PrefetchOwnedAndPossessedAssets, so
fetching assets for N identities costs roughly one round trip instead of 2N.
Callers previously had to fan out N*2 requests, which saturated the connection
pool.

Identities are validated before a connection is taken, so malformed input never
costs a pooled connection. Empty lists, batches over the configured limit,
malformed identities and duplicates are all rejected with InvalidArgument. The
limit defaults to 15 and is configurable via
QUBIC_API_SIDECAR_SERVER_MAX_BATCH_IDENTITIES.

A failed batch leaves the connection byte unaligned, so the handler closes it
rather than returning it to the pool, and a test pins that.

To make the handler testable, the pool is reached through a nodePool/nodeClient
seam instead of the concrete *qubic.Pool, whose Get returns a client holding a
real socket. The seam covers every client method the handlers already use, so no
existing handler body changed, and a compile time assertion keeps the interface
honest as the client evolves.

BEHAVIOR CHANGE: the nested owned_asset of /assets/{identity}/possessed was
built from the possession record's own fields instead of the ownership record it
wraps, so it reported the possession type (3) rather than the ownership type (2),
and the possession's index and unit count. Extracting the duplicated conversion
into shared converters fixes this by construction, since the converter reads only
the ownership record it is given. README samples updated to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement pipeline requests for asset request endpoint

1 participant