Understanding Dim and Alpha: Key Concepts for LoRA Training
Introduction Different Names, Same Concepts What Is Dim? What Is Alpha? Balancing Dim and Alpha for Effective Training Dim = Alpha (e.g., 32 and 32) Alpha Twice the Size of Dim (e.g., Dim = 32, Alpha = 64) Alpha < Dim How These Parameters Affect Training Conclusion Have Questions or Want to Learn More?