[webshop] fix: use selected dataset when building search indexes - #270

What does this PR do?

Fixes #252: setup.sh -d all downloads full product data but the converter still reads the default 1k product/attribute files. Pass the selected paths explicitly so the main search collection and its index correspond to the requested dataset.

Changes

• Add optional --filepath / --attrpath arguments to the converter, retaining the existing defaults for standalone no-argument calls.
• Wire both setup.sh and setup_arm.sh to select matching product and attribute files for small and all.
• Stop setup when conversion fails, before indexing potentially stale resources.
• Document selection, standalone rebuild commands, and the distinction from runtime environment defaults (which this PR does not change).
• Add eight offline regression tests covering both scripts and modes, failure propagation, default conversion, explicit full-file conversion, and the existing subset limits. The explicit-file fixture has 1,005 products, verifying that the main collection is not truncated at 1k.

Tests

python -m pytest tests/test_webshop_index_setup.py -q
ruff check tests/test_webshop_index_setup.py
ruff format --check tests/test_webshop_index_setup.py
bash -n agent_system/environments/env_package/webshop/webshop/setup.sh
bash -n agent_system/environments/env_package/webshop/webshop/setup_arm.sh
git diff --check

• Base 20bd331bdbc9026a5668e11362178e10ab7400c8: 7 failed, 1 passed with these tests.
• Patched: 8 passed.
• Tests execute the actual shell scripts with fake install/download/Python commands, checking paths forwarded to conversion and all four existing indexing invocations. Converter tests execute the actual converter with a lightweight replacement for the environment loader, reading fixture JSON and verifying generated JSONL contents/counts.
• These are offline wiring/serialization checks, not a full dataset download, Lucene index build, or WebShop runtime benchmark. Existing production-file style has been retained to avoid unrelated formatting changes; full-repository pre-commit and remote CI are not claimed to pass.

Additional information

• No training/inference backend changes; this is dataset preparation only.
• Duplicate check: setup.sh -d all does not build full-size search index (still uses 1k data source) #252 was open with no comments or linked PR; searching PRs for setup.sh -d all does not build full-size search index (still uses 1k data source) #252 and inspecting open PRs found no existing fix before submission.
• Prepared and reviewed with OpenAI Codex assistance, submitted with the account owner's authorization. Tests were actually run; independent human line-by-line review is not claimed.

Checklist

• Search for similar PRs.
• Read the contribution guide linked by the repository template.
• Add regression tests and documentation.
• Run targeted tests and syntax checks.
• Full-repository pre-commit and remote CI.