[diffusion, trainer, tests] feat: strip responses from worker batches - #430

What does this PR do?

Implements Phase 1 of #386 by excluding driver-only responses pixels from diffusion actor, reference, teacher, and old-log-prob worker payloads while retaining the original DataProto for reward scoring, evaluation, and logging.

• Adds one shallow DataProto → TensorDict projection that removes only responses without copying tensor storage.
• Applies the projection to the v0 policy-gradient/direct-preference paths and the equivalent v1 trainer paths.
• Adds CPU contracts for field removal, driver ownership, shared tensor storage, metadata preservation, and every affected production entry point.

This complements #373, which implemented the separate uint8 response representation phase. AI assistance was used for this change, and the final diff received human review before publication.

Checklist Before Starting

• Searched for similar PRs: open PRs referencing #386 and exact head WenZheWang:wenzhe/strip-unused-responses; no duplicate found.
• Formatted the PR title as [{modules}] {type}: {description} and validated it with tests/special_sanity/check_pr_title.py.

Test

pytest -q tests/trainer/diffusion
175 passed, 4 warnings

pre-commit run --files \
  verl_omni/trainer/diffusion/ray_diffusion_trainer.py \
  verl_omni/trainer/diffusion/v1/trainer_base.py \
  tests/trainer/diffusion/test_worker_batch_projection_on_cpu.py
all hooks passed

The CPU suite ran on macOS with an import-only stub for the optional vLLM rollout package because vLLM is unavailable on macOS; the production trainer modules and methods were exercised.

Hardware validation on exact head ba3918cfb4e2a2bca9120ded699b2e52ddf9ba50:

• CPU Ray transport benchmark (torch 2.6.0+cu124, Ray 2.53.0): batch 8, 192 frames, 512×512 RGB uint8 responses, 2 warmups and 8 measured repetitions per arm. Tensor payload fell from 1,224,737,280 to 16,777,728 bytes (98.63% reduction); median end-to-end transfer time fell from 1.64636 s to 0.03249 s (98.03% reduction). The driver retained responses, the projected worker payload excluded it, and all predeclared gates passed.
• One-A800 deterministic GPU smoke (NVIDIA A800 80GB PCIe, CUDA 12.4, torch 2.6.0+cu124, Ray 2.53.0): seed 42, six optimizer steps per arm. Tensor payload fell from 402,784,288 to 131,104 bytes (99.967% reduction); baseline and projected loss sequences were identical, maximum absolute loss and parameter differences were both 0.0, and final parameter SHA256 values matched. All eight predeclared correctness gates passed.

The hardware evidence is intentionally scoped to response-payload transport and deterministic optimizer equivalence. It does not claim end-to-end diffusion quality, full-model production throughput, or multi-GPU semantics.

API and Usage Example

No public API, configuration, or user workflow changes. Diffusion trainer call sites automatically omit responses only when preparing actor/ref worker payloads.

Design & Code Changes

1. DataProto.to_tensordict() creates the worker-side container while preserving references to existing tensor storage.
2. The projection removes responses from that container only; the driver-side DataProto remains unchanged.
3. All v0/v1 actor update, reference/teacher inference, and old-log-prob entry points reuse the same projection before RPC dispatch.
4. Tests assert the worker payload excludes pixels while all other tensor, non-tensor, and metadata contracts remain intact.

Checklist Before Submitting

• Read the Contribute Guide and repository agent instructions.
• Applied pre-commit checks; all hooks passed for the changed files.
• Documentation is unchanged because this is an internal transport optimization with no API, configuration, or usage change.
• Added test_worker_batch_projection_on_cpu.py; existing CPU workflow discovery covers tests/trainer/diffusion/*_on_cpu.py, so no workflow edit is required.