Skip to content

GPRO: use_liger_loss + zero3 error #3368

@paul777chen

Description

@paul777chen

Reproduction

✅ use_liger_loss=true + zero2
✅ use_liger_loss=false + zero3
❌ use_liger_loss=true + zero3

error:size mismatch

System Info

pytorch=2.6
deepspeed=0.16.4

Checklist

  • I have checked that my issue isn't already filed (see open issues)
  • I have included my system information
  • Any code provided is minimal, complete, and reproducible (more on MREs)
  • Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
  • Any traceback provided is complete

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions