feat: add HF safetensors export for qwen3.5_moe - #2954
mrinalghoshh wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds support for exporting Qwen 3.5 MoE models to Hugging Face Transformers format, introducing configuration mapping, weight conversion utilities, end-to-end tests, and a verification script. Feedback on the changes points out potential AttributeErrors in the configuration exporter due to missing backbone attributes, and suggests using the mean logits difference instead of the maximum absolute difference for numerical verification to comply with repository guidelines.
| "decoder_sparse_step": backbone.decoder_sparse_step, | ||
| "moe_intermediate_size": backbone.moe_intermediate_dim, | ||
| "shared_expert_intermediate_size": backbone.intermediate_dim, | ||
| "num_experts_per_tok": backbone.top_k, | ||
| "num_experts": backbone.num_experts, | ||
| "norm_top_k_prob": backbone.norm_top_k_prob, |
There was a problem hiding this comment.
The attributes decoder_sparse_step, norm_top_k_prob, and intermediate_dim (accessed as backbone.intermediate_dim) do not exist on Qwen3_5MoeBackbone. Accessing them directly will raise an AttributeError during export.
intermediate_dimshould beshared_expert_intermediate_size.decoder_sparse_stepandnorm_top_k_probshould be safely retrieved usinggetattrwith appropriate defaults.
| "decoder_sparse_step": backbone.decoder_sparse_step, | |
| "moe_intermediate_size": backbone.moe_intermediate_dim, | |
| "shared_expert_intermediate_size": backbone.intermediate_dim, | |
| "num_experts_per_tok": backbone.top_k, | |
| "num_experts": backbone.num_experts, | |
| "norm_top_k_prob": backbone.norm_top_k_prob, | |
| "decoder_sparse_step": getattr(backbone, "decoder_sparse_step", 1), | |
| "moe_intermediate_size": backbone.moe_intermediate_dim, | |
| "shared_expert_intermediate_size": backbone.shared_expert_intermediate_size, | |
| "num_experts_per_tok": backbone.top_k, | |
| "num_experts": backbone.num_experts, | |
| "norm_top_k_prob": getattr(backbone, "norm_top_k_prob", True), |
|
|
||
| keras_outputs = keras_model.predict(input_ids.numpy(), verbose=0) | ||
|
|
||
| diff = np.max(np.abs(hf_outputs.numpy() - keras_outputs)) |
There was a problem hiding this comment.
According to the repository's general rules, when performing numerical verification for converted model checkpoints, we should validate using the mean logits difference tolerance rather than the maximum absolute difference.
| diff = np.max(np.abs(hf_outputs.numpy() - keras_outputs)) | |
| diff = np.mean(np.abs(hf_outputs.numpy() - keras_outputs)) |
References
- When performing numerical verification for converted model checkpoints, validate using the mean logits difference tolerance rather than the maximum absolute difference.
…sors-checkpoint-conversion
|
❌ Approved issue check failed. This PR stays in draft until it links an approved issue that is assigned to you. To fix this:
The PR will be marked Ready for review automatically once the check passes. No issue reference was found in the description. |
Description of the change
Reference
Colab Notebook
https://colab.research.google.com/gist/mrinalghoshh/2457e93a4c4ffc437893f3af06db7dbd/qwen3_5_moe_export_verification.ipynb
Checklist