LoRA Fine-Tuning for Flan T5 Small Model “Train your own AI model using just a Mac, without a GPU or cloud services.” 1. Why Choose LoRA? Lightweight: Only trains 0.1% to 1% of new parameters, fast speed, and low memory usage. Pluggable: Multiple LoRA adapters can be switched at any time, allowing for versatile use. … Read more
Word count: 740, reading time approximately 4 minutes After installing the device system, configure libraries such as CUDA, cuDNN, OpenCV, and TensorRT through JetPack. 1. Install Conda Miniconda download link: <span>https://repo.anaconda.com/miniconda/</span>, choose the appropriate version to download, for example, <span>Miniconda3-py38_23.11.0-2-Linux-aarch64.sh</span> 1. Installation command cd /root wget -q https://repo.anaconda.com/miniconda/Miniconda3-py38_23.5.2-0-Linux-aarch64.sh bash ./Miniconda3-py38_23.11.0-2-Linux-aarch64.sh -b -f -p /root/miniconda3 rm … Read more
Focus on Model Compression, Low-Bit Quantization, Mobile Inference Acceleration Optimization, and Deployment Introduction: This issue contains 15 items. 【News】Shanghai Zhangjiang – News from several GPGPU companies: BoHan released cloud AI inference chip performance exceeding T4 with mass production expected in Q4 this year; Suipian released the largest AI chip in China, Birun’s first 7nm GPU … Read more
Overview In this project, we encountered a need for Optical Character Recognition (OCR). However, the best-performing <span>PaddleOCR</span> can only run on Baidu’s <span>PaddlePaddle</span> framework. In common projects, the more widely used Pytorch framework is often preferred. Installing PaddlePaddle separately not only makes the project overly bulky but may also lead to conflicts. In the previous … Read more
In May 2022, PyTorch officially announced support for model acceleration on Mac with M1 chip. Official comparison data shows that the training speed on M1 can be accelerated by an average of 7 times compared to CPU. Wow, without needing a separate GPU, I couldn’t wait to get my hands on an M1 MacBook to … Read more
Artificial intelligence has fundamentally transformed various fields from software development to content creation. However, faced with a plethora of AI tools on the market, developers often find themselves in a dilemma: either pay for a subscription to commercial platforms or waste time on inefficient tools. In fact, the open-source community has long nurtured a batch … Read more
This article evaluates the performance of MindSpore and PyTorch based on Huawei’s Ascend Neural Processing Unit (Ascend NPU). Huawei’s Ascend NPU isa chip designed specifically for accelerating neural network computations, which not onlysupports the self-developed framework MindSpore but alsohas been adapted for PyTorch. However, there is currentlya lack of comparative evaluation studies on MindSpore and … Read more
Overview The official PyTorch plugin provided by Huawei: <span>torch-npu</span> enables running PyTorch code on Huawei Ascend servers, facilitating AI development—training and inference—based on the open-source PyTorch ecosystem. Although Huawei has its own machine learning development framework: MindSpore, similar frameworks exist from companies like Baidu and Alibaba, such as Baidu’s PaddlePaddle. Essentially, these frameworks are directly … Read more
FBGEMM (Facebook General Matrix Multiplication) is a C++ library developed by Meta (Facebook) that is primarily used for low-precision, high-performance matrix multiplication and convolution operations in server-side inference. It is designed for small batch data and can significantly improve inference efficiency while supporting various techniques to reduce precision loss, such as row-wise quantization and outlier-aware … Read more
Machine Heart ReportEditor: Xiao Zhou Karpathy: for fun. A few days ago, Andrej Karpathy, former head of Tesla Autopilot and OpenAI scientist, released a project called “llm.c” that implements GPT-2 training in just 1000 lines of code on CPU/fp32. llm.c aims to simplify large model (LM) training — using pure C language / CUDA, without … Read more