P-Tuning and LoRA Fine-Tuning for ChatGLM3-6B
1. Environment Preparation Install Dependencies conda create -n glm3-pfinetune python=3.10 -y conda activate glm3-pfinetune pip install torch torchvision torchaudio –index-url https://download.pytorch.org/whl/cu118 pip install transformers==4.36.2 pip install datasets accelerate peft bitsandbytes pip install tqdm Download the GLM3-6B Model Model address (the open-source model is released by Tsinghua KEG):👉 <span>THUDM/glm-3-6b</span> (available on Hugging Face, requires <span>transformers>=4.34</span>) 2. … Read more
