Search results

what is the initial learning rate

Status: Open.

#74 In jiaweizzhao/GaLore;

· Wenhua-Hu opened on Jul 20, 2025

GLUE finetuning: reproductions for LoRA

Status: Open.

#69 In jiaweizzhao/GaLore;

· nikitaved opened on Feb 26, 2025

Table 2 hyperparameters

Status: Open.

#68 In jiaweizzhao/GaLore;

· athglentis opened on Feb 25, 2025

Question on the estimated memory of GaLore

Status: Open.

#67 In jiaweizzhao/GaLore;

· zqOuO opened on Dec 14, 2024

Question on Convergence and Grad Norm Behavior During Training with GaLore

Status: Open.

#66 In jiaweizzhao/GaLore;

· chelouche9 opened on Nov 9, 2024

loss figure data

Status: Open.

#61 In jiaweizzhao/GaLore;

· BaohaoLiao opened on Sep 2, 2024

ValueError: can't optimize a non-leaf Tensor (param.is_leaf=False,param.retains_grad=False)

Status: Open.

#60 In jiaweizzhao/GaLore;

· liveck opened on Aug 21, 2024

Results vs FP32

Status: Open.

#59 In jiaweizzhao/GaLore;

· tsengalb99 opened on Jul 30, 2024

Zero Loss: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated singular values

Status: Open.

#58 In jiaweizzhao/GaLore;

· akjindal53244 opened on Jul 23, 2024

Figure 1 clarification on batch size and sequence length

Status: Open.

#57 In jiaweizzhao/GaLore;

· psandovalsegura opened on Jul 17, 2024

Questions about glue task report scores

Status: Open.

#56 In jiaweizzhao/GaLore;

· MYT677 opened on Jul 10, 2024

Support for DDP with multi-gpus

Status: Open.

#55 In jiaweizzhao/GaLore;

· seongjunyun opened on Jul 8, 2024