Towards High-Rank LoRA: Fewer Parameters, Higher Rank
This is a very impressive paper. The MeLoRA algorithm proposed in the paper not only achieves a rank increase but also shows certain improvements in computational efficiency compared to vanilla LoRA. Although the theory in this paper is relatively simple and there are not many mathematical formulas, the specific methods are quite enlightening. Article Title: … Read more