Summary of Multi-task Learning Methods

Summary of Multi-task Learning Methods

Follow the WeChat public account “ML_NLP“ and set it as a “starred“, delivering substantial content to you in real-time! This article is authorized to be transferred from the Zhihu author Anticoder, https://zhuanlan.zhihu.com/p/59413549. Unauthorized reproduction is prohibited. Background: Focusing solely on a single model may overlook potential information that could enhance the target task from related … Read more

Overview of Multi-task Learning

Overview of Multi-task Learning

Author: Anticoder Column: Optimazer’s Garden https://zhuanlan.zhihu.com/p/59413549 Background: Focusing solely on a single model may overlook potential information that could enhance the target task from related tasks. By sharing parameters to some extent between different tasks, the original task may generalize better. Broadly speaking, as long as there are multiple losses, it counts as MTL, with … Read more

Solving Composite Problems in One Inference: The MeteoRA Architecture for Scalable Integration of Knowledge Modules in Large Language Models Based on MoE

Solving Composite Problems in One Inference: The MeteoRA Architecture for Scalable Integration of Knowledge Modules in Large Language Models Based on MoE

In the field of large language models, the pre-training + fine-tuning paradigm has become an important foundation for deploying various downstream applications. Within this framework, the use of low-rank adaptation (LoRA) methods for efficient fine-tuning of large model parameters (PEFT) has resulted in a large number of reusable LoRA adapters tailored for specific tasks. However, … Read more

Solving Composite Problems in One Inference: The MeteoRA Architecture for Scalable Integration of Knowledge Modules in MoE-based Large Language Models

Solving Composite Problems in One Inference: The MeteoRA Architecture for Scalable Integration of Knowledge Modules in MoE-based Large Language Models

The AIxiv column is a section published by Machine Heart that features academic and technical content. Over the past few years, the AIxiv column has reported on more than 2000 pieces of content, covering top laboratories from major universities and companies worldwide, effectively promoting academic exchange and dissemination. If you have excellent work to share, … Read more