↑↑↑ Follow and Star Kaggle Competition Guide Kaggle Competition Guide Author: Elvin Loves to Ask, excerpted from Zhai Ma LoRA+MoE: A Historical Interpretation of the Combination of Low-Rank Matrices and Multi-Task Learning This article introduces some works that combine LoRA and MoE, hoping to be helpful to everyone. 1. MoV and MoLoRA Paper: 2023 | … Read more
Author | Sanhe Factory Girl Source | See “Read the Original” at the end Concept When optimizing more than one objective function in a single task, it is referred to as multi-task learning. Some Exceptions “Multi-task of a single objective function”: In many tasks, the losses are combined and backpropagated, effectively optimizing a single objective … Read more
MLNLPThe MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP graduate students, university professors, and corporate researchers.The vision of the communityis to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners.Source | Machine HeartEditor | … Read more
Reported by Machine Heart Editor: Zhang Qian How much parameter redundancy exists in LoRA? This innovative research introduces the LoRI technology, which demonstrates that even significantly reducing the trainable parameters of LoRA can still maintain strong model performance. The research team tested LoRI on mathematical reasoning, code generation, safety alignment, and eight natural language understanding … Read more
Follow the WeChat public account “ML_NLP“ and set it as a “starred“, delivering substantial content to you in real-time! This article is authorized to be transferred from the Zhihu author Anticoder, https://zhuanlan.zhihu.com/p/59413549. Unauthorized reproduction is prohibited. Background: Focusing solely on a single model may overlook potential information that could enhance the target task from related … Read more
Author: Anticoder Column: Optimazer’s Garden https://zhuanlan.zhihu.com/p/59413549 Background: Focusing solely on a single model may overlook potential information that could enhance the target task from related tasks. By sharing parameters to some extent between different tasks, the original task may generalize better. Broadly speaking, as long as there are multiple losses, it counts as MTL, with … Read more
In the field of large language models, the pre-training + fine-tuning paradigm has become an important foundation for deploying various downstream applications. Within this framework, the use of low-rank adaptation (LoRA) methods for efficient fine-tuning of large model parameters (PEFT) has resulted in a large number of reusable LoRA adapters tailored for specific tasks. However, … Read more
The AIxiv column is a section published by Machine Heart that features academic and technical content. Over the past few years, the AIxiv column has reported on more than 2000 pieces of content, covering top laboratories from major universities and companies worldwide, effectively promoting academic exchange and dissemination. If you have excellent work to share, … Read more