Model Adaptation

Understanding Large Model Fine-Tuning: What Are the Differences Between SFT and LoRA?

by
Understanding Large Model Fine-Tuning: What Are the Differences Between SFT and LoRA?

In the application of large models, “fine-tuning” is a crucial step to adapt general models to specific scenarios — for example, enabling ChatGPT to write product copy or allowing LLaMA to answer industry-specific questions, both of which rely on fine-tuning. However, many people confuse SFT and LoRA, two core technologies: both are based on optimizing … Read more

Differences Between LoRA and Full Fine-Tuning Explained in MIT Paper

by
Differences Between LoRA and Full Fine-Tuning Explained in MIT Paper

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP graduate students, university teachers, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial circles of natural language processing and machine learning, especially for beginners. Reprinted from | … Read more