Understanding LoRA from a Gradient Perspective
©PaperWeekly Original · Author | Su Jianlin Affiliation | Zhuiyi Technology Research Area | NLP, Neural Networks With the popularity of ChatGPT and its alternatives, various parameter-efficient fine-tuning methods have also gained traction, among which one of the most popular is the focus of this article, LoRA, originating from the paper “LoRA: Low-Rank Adaptation of … Read more