NLP

P-Tuning and LoRA Fine-Tuning for ChatGLM3-6B

by
P-Tuning and LoRA Fine-Tuning for ChatGLM3-6B

1. Environment Preparation Install Dependencies conda create -n glm3-pfinetune python=3.10 -y conda activate glm3-pfinetune pip install torch torchvision torchaudio –index-url https://download.pytorch.org/whl/cu118 pip install transformers==4.36.2 pip install datasets accelerate peft bitsandbytes pip install tqdm Download the GLM3-6B Model Model address (the open-source model is released by Tsinghua KEG):👉 <span>THUDM/glm-3-6b</span> (available on Hugging Face, requires <span>transformers>=4.34</span>) 2. … Read more

From BM25 to Multi-Vector: Six Evolutionary Paths of Embedding

by
From BM25 to Multi-Vector: Six Evolutionary Paths of Embedding

This public account mainly focuses on cutting-edge AI technologies such as NLP, CV, LLM, RAG, and Agent, sharing practical industry cases and courses for free, helping you fully embrace AIGC. 1. Sparse Embedding (Keyword-based Sparse Vectors) Dimension Description Typical Implementations TF-IDF, BM25, SPLADE Vector Shape 50,000+ dimensions, >95% of positions are 0 Similarity Calculation Cosine … Read more

LoRA: Unlocking Low-Rank Adaptation Optimization

by
LoRA: Unlocking Low-Rank Adaptation Optimization

Click the blue text to follow us 01 Introduction In the fields of machine learning and deep learning, optimizing model parameters is key to improving model performance. Traditional optimization methods, such as Stochastic Gradient Descent (SGD), are effective but may encounter limitations in computational resources and time when dealing with large-scale models. To address these … Read more

Meta AI Open Sources 175B Parameter Model – OPT, One-Click Call by FlagAI!

by
Meta AI Open Sources 175B Parameter Model - OPT, One-Click Call by FlagAI!

MLNLP ( Machine Learning Algorithms and Natural Language Processing ) community is a well-known natural language processing community both domestically and internationally, covering NLP master’s and doctoral students, university professors, and corporate researchers.The vision of the community is to promote communication between the academic and industrial circles of natural language processing and machine learning, as … Read more

NeurlPS 2022 | Classification of NLP Related Papers

by

MLNLP community is a well-known machine learning and natural language processing community in China and abroad, covering NLP master’s and doctoral students, university teachers, and industry researchers. The vision of the community is to promote communication and progress between the academic and industrial fields of natural language processing and machine learning, especially for beginners. Reprinted … Read more

Using SpaCy for Multi-Language Support

by
Using SpaCy for Multi-Language Support

Handling multi-language text has always been a headache. However, with the powerful natural language processing tool SpaCy, analyzing multi-language text becomes super simple. It supports over 50 languages, including major languages like Chinese, English, and Japanese, and can handle various complex linguistic phenomena. Installation Made Easy Installing SpaCy is particularly simple, just one line of … Read more

Introduction to Core Technologies in Neural Networks

by
Introduction to Core Technologies in Neural Networks

In recent years, artificial intelligence technology has been changing our lives at an astonishing pace. From intelligent voice assistants to autonomous driving, from medical image analysis to personalized recommendations, AI technology has permeated all aspects. Are you curious about what supports these magical functions? Today, we will unveil the technical core of the AI world … Read more

Improving Model Performance on Multi-Typo Correction

by
Improving Model Performance on Multi-Typo Correction

Daily delivery of NLP technology insights! From: Lanzhou Technology Current Contributor: Gan Zifa Algorithm intern at Lanzhou Technology, second-year master’s student at the Natural Language Processing Laboratory of Zhengzhou University (ZZUNLP). Currently researching text correction. E-mail: [email protected] Introduction Natural language texts often contain spelling errors (typos), which are known as incorrect characters in Chinese. Chinese … Read more

Understanding Multi-Head Attention in NLP

by
Understanding Multi-Head Attention in NLP

1. Multi-Head Attention Multi-Head Attention is a widely adopted extension of the attention mechanism in the Transformer model. It captures different attention distributions in various subspaces of the input sequence by running multiple independent attention mechanisms in parallel, thereby comprehensively capturing the various semantic associations present in the sequence. In Multi-Head Attention, the input sequence … Read more

Understanding LoRA: A Cost-Effective Method for AI Adaptation in Just Ten Minutes

by
Understanding LoRA: A Cost-Effective Method for AI Adaptation in Just Ten Minutes

It’s not difficult at all; a good paper will make you soar. I will use many examples to help you understand it instantly, so don’t be afraid! Here’s a tip: You can skip the parts you don’t want to read, take what you need, and finishing reading is more important than understanding everything. If you … Read more