Complete discrete DQN and DDPG solvers - #57

What changed

• complete the discrete DQN and Double DQN solvers with persistent replay, masked Bellman targets, and target-network updates
• implement the Virne-specific discrete DDPG adaptation with separate actor/critic optimizers and soft target updates
• persist and restore algorithm-specific checkpoint state
• add off-policy configuration, solver registration, action policy construction, documentation, and focused regression tests

Why

The previous DQN/DDPG classes were incomplete placeholders. They reused rollout-style behavior, did not retain a bounded replay memory correctly, and did not preserve the target/critic state required for reliable off-policy training. DDPG also needed an explicit adaptation because Virne exposes a discrete physical-node action space rather than continuous actions.

Impact

dqn_mlp+, double_dqn_mlp+, and ddpg_mlp+ can now train, save, reload, and run inference. Invalid physical-node actions remain masked during exploration and target calculation. Existing on-policy solvers retain their previous rollout behavior.

Validation

• .venv/bin/pytest -q: 277 passed
• DQN: 10 x 250 training requests, 8,880 finite updates
• discrete DDPG: 10 x 250 training requests, 8,893 finite updates
• held-out evaluation over five seeds (100 requests each): DQN acceptance improved from 33.8% to 48.2%; DDPG improved from 33.8% to 46.4%
• checkpoint save/load and inference covered by regression tests and real smoke training

ddpg_mlp+ is intentionally documented as a discrete-action DDPG-style adaptation, not canonical continuous-action DDPG.