ChatGLM2-6B

Exploring the New ChatGLM Language Model

by

I have grown tired of the stable diffusion I was using before, and I am now a well-known artist with thousands of followers on Pixiv. Today, I picked up a new toy, ChatGLM! This is a large-scale bilingual language model with question-and-answer and dialogue capabilities, optimized for Chinese, and is currently in an invitation-only beta … Read more

Fine-tuning CPU Lora ChatGLM2-6B

by
Fine-tuning CPU Lora ChatGLM2-6B

The open-source dataset found contains less than 50,000 Q&A pairs, and it is recommended to have over 200G of memory. My local setup with 60G of memory cannot run it. The lora uses Hugging Face’s peft: https://github.com/huggingface/peft Two versions of the training part were written: One references the peft example: https://github.com/huggingface/peft/tree/main/examples. With 60G memory and … Read more