Verification of ONNX cached queries generated for RAG 25

Following https://github.com/castorini/anserini/pull/3194, I verified the correctness of cached queries generated by ONNX via `QueryEncoder`.

👍 @UShivani3 

Running SPLADEv3 w/ ONNX:

```bash
# Running condition "splade-v3.onnx": SPLADE-v3: ONNX 

- topic_key: rag24.test

bin/run.sh io.anserini.search.SearchCollection -threads 16 -index msmarco-v2.1-doc-segmented.splade-v3 -topics rag24.test -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt -impact -pretokenized -removeQuery -hits 1000 -encoder SpladeV3

bin/run.sh trec_eval -c -m ndcg_cut.20 rag24.test-umbrela-all runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt
bin/run.sh trec_eval -c -m ndcg_cut.100 rag24.test-umbrela-all runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt
bin/run.sh trec_eval -c -m recall.100 rag24.test-umbrela-all runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt

     nDCG@20: 0.5167    [OK] 
    nDCG@100: 0.4587    [OK] 
       R@100: 0.2437    [OK] 

bin/run.sh trec_eval -c -m ndcg_cut.20 rag24.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt
bin/run.sh trec_eval -c -m ndcg_cut.100 rag24.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt
bin/run.sh trec_eval -c -m recall.100 rag24.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt

     nDCG@20: 0.4642    [OK] 
    nDCG@100: 0.4349    [OK] 
       R@100: 0.3198    [OK] 

- topic_key: rag25.test

bin/run.sh io.anserini.search.SearchCollection -threads 16 -index msmarco-v2.1-doc-segmented.splade-v3 -topics rag25.test -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt -impact -pretokenized -removeQuery -hits 1000 -encoder SpladeV3

bin/run.sh trec_eval -c -m ndcg_cut.30 rag25.test-umbrela2 runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt
bin/run.sh trec_eval -c -m ndcg_cut.100 rag25.test-umbrela2 runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt
bin/run.sh trec_eval -c -m recall.100 rag25.test-umbrela2 runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt

     nDCG@30: 0.5838    [OK] 
    nDCG@100: 0.5372    [OK] 
       R@100: 0.1992    [OK] 

bin/run.sh trec_eval -c -m ndcg_cut.30 rag25.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt
bin/run.sh trec_eval -c -m ndcg_cut.100 rag25.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt
bin/run.sh trec_eval -c -m recall.100 rag25.test runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt

     nDCG@30: 0.5957    [OK] 
    nDCG@100: 0.5387    [OK] 
       R@100: 0.2103    [OK] 
```

Using cached queries, same effectiveness:

```bash
bin/run.sh io.anserini.search.SearchCollection -index msmarco-v2.1-doc-segmented.splade-v3 -topics tools/topics-and-qrels/topics.rag24.test.splade-v3.tsv.gz -topicReader TsvString \
  -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt -impact -pretokenized -removeQuery -hits 1000 &

bin/run.sh io.anserini.search.SearchCollection -index msmarco-v2.1-doc-segmented.splade-v3 -topics tools/topics-and-qrels/topics.rag25.test.splade-v3.jsonl.gz -topicReader JsonString \
  -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt -impact -pretokenized -removeQuery -hits 1000 &
```

Creating cached queries from scratch:

```bash
bin/run.sh io.anserini.encoder.EncodeQuery -encoder SpladeV3 -queries tools/topics-and-qrels/topics.rag24.test.txt -topicReader TsvString -output encoded-topics.rag24.test.splade-v3.tsv.gz -compress
bin/run.sh io.anserini.encoder.EncodeQuery -encoder SpladeV3 -queries tools/topics-and-qrels/topics.rag25.test.jsonl -topicReader JsonString -output encoded-topics.rag25.test.splade-v3.jsonl.gz -compress
```

Running with newly created queries:

```bash
bin/run.sh io.anserini.search.SearchCollection -index msmarco-v2.1-doc-segmented.splade-v3 -topics encoded-topics.rag24.test.splade-v3.tsv.gz -topicReader TsvString \
  -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag24.test.txt -impact -pretokenized -removeQuery -hits 1000 &

bin/run.sh io.anserini.search.SearchCollection -index msmarco-v2.1-doc-segmented.splade-v3 -topics encoded-topics.rag25.test.splade-v3.jsonl.gz -topicReader JsonString \
  -output runs/run.msmarco-v2.1-doc-segmented.core.splade-v3.onnx.rag25.test.txt -impact -pretokenized -removeQuery -hits 1000 &
```

Effectiveness from running newly created cached queries match effectiveness from ONNX.


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Verification of ONNX cached queries generated for RAG 25 #3211

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Verification of ONNX cached queries generated for RAG 25 #3211

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions