[Bugfix] Fix TENT sync transfer status handling - #4264

Description

Fix TENT sync-transfer handling for terminal states and peer failures. #3995

Implementation

• Fix TENT retry and terminal-state handling.
• Gracefully handle transfer status errors.
• Evict stale Segments and release failed batches.
• Update batch retry hints.
• Add regression coverage.

Validation

Before:

• Regression test: 2 failed.
• RDMA peer-kill （1 GiB in-flight transfer）returned -1, but reused the stale Segment.

After:

• Regression test: 2 passed.
• RDMA peer-kill (1 GiB in-flight transfer): returned -1 without hanging.
• Subsequent request logged cache not found.

Module

• Transfer Engine (mooncake-transfer-engine)
• Mooncake Store (mooncake-store)
• Mooncake Conductor (mooncake-conductor)
• Reshard (mooncake-reshard)
• Mooncake EP (mooncake-ep)
• Mooncake PG (mooncake-pg)
• Integration (mooncake-integration)
• P2P Store (mooncake-p2p-store)
• Python Wheel (mooncake-wheel)
• Common (mooncake-common)
• Mooncake RL (mooncake-rl)
• CI/CD
• Docs
• Other

Type of Change

• Bug fix
• New feature
• Refactor
• Breaking change
• Documentation update
• Performance improvement
• Other

How Has This Been Tested?

Test commands:

# Example: bash scripts/run_ci_test.sh

Test results:

• Unit tests pass
• Integration tests pass (if applicable)
• Manual testing done (describe below)

Checklist

• I have performed a self-review of my own code
• I have formatted my code using ./scripts/code_format.sh
• I have run pre-commit on the files changed in this PR and all hooks pass
• I have updated the documentation (if applicable)
• I have added tests to prove my changes are effective
• For changes >500 LOC: I have filed an RFC issue

AI Assistance Disclosure

• No AI tools were used
• AI tools were used (specify below)