Exploration Robotics

Latest Advances in LoRA: A Comprehensive Review

by
Latest Advances in LoRA: A Comprehensive Review

Abstract——The rapid development of foundational models——large-scale neural networks trained on diverse and extensive datasets——has revolutionized artificial intelligence, driving unprecedented advancements in fields such as natural language processing, computer vision, and scientific discovery. However, the enormous parameter counts of these models, often reaching billions or even trillions, pose significant challenges in adapting them to specific downstream … Read more

Comprehensive Analysis of LoRA, QLoRA, RLHF, PPO, DPO, and Flash Attention

by
Comprehensive Analysis of LoRA, QLoRA, RLHF, PPO, DPO, and Flash Attention

With the rapid development of large models, there has been significant technological iteration and updates in just a year, from LoRA, QLoRA, AdaLoRa, ZeroQuant, Flash Attention, KTO, distillation techniques to model incremental learning, data processing, and understanding new open-source models, almost every day brings new developments. As algorithm engineers, do you feel like your learning … Read more

ReLoRA: Efficient Large Model Training Through Low-Rank Updates

by
ReLoRA: Efficient Large Model Training Through Low-Rank Updates

This article focuses on reducing the training costs of large Transformer language models. The author introduces a low-rank update-based method called ReLoRA. A core principle in the development of deep learning over the past decade has been to “stack more layers,” and the author aims to explore whether stacking can similarly enhance training efficiency for … Read more

New PiSSA Method From Peking University Enhances Fine-Tuning

by
New PiSSA Method From Peking University Enhances Fine-Tuning

Machine Heart Column Machine Heart Editorial Team As the parameter size of large models continues to grow, the cost of fine-tuning the entire model has gradually become unacceptable. To address this, a research team from Peking University proposed a parameter-efficient fine-tuning method called PiSSA, which outperforms the widely used LoRA in fine-tuning effectiveness on mainstream … Read more

Step 3 of AI Painting: Create Realistic Characters with Lora

by
Step 3 of AI Painting: Create Realistic Characters with Lora

No matter how prosperous the virtual world is, real things have an irresistible charm, and AI painting is no exception. Today, let’s talk about how to use ChilloutMix and Lora to create particularly “realistic” characters, pursuing realism in the virtual world, which is actually the core goal of our series. What is ChilloutMix? Essentially, it … Read more

NB-IoT vs. LoRa: Meeting Basic IoT Needs

by
NB-IoT vs. LoRa: Meeting Basic IoT Needs

Recently, I was fortunate to have a friend help me connect with a renowned electronic product OEM giant in Shenzhen to discuss our solutions. The discussion was with the head of a subsidiary of this company, which handles its global supply chain logistics and also provides B2B logistics services to other factories. Industry insiders may … Read more

S-LoRA: Enabling Thousands of Large Models on a GPU

by
S-LoRA: Enabling Thousands of Large Models on a GPU

Machine Heart reports Editor: Danjiang Generally, the deployment of large language models adopts a “pre-training – then fine-tuning” approach. However, when fine-tuning the base model for numerous tasks (such as personalized assistants), the training and service costs can become extremely high. Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning method, typically used to adapt the base … Read more

LoRA: The Nitrogen Accelerator for Large Models

by
LoRA: The Nitrogen Accelerator for Large Models

Selected from Raphael G’s blog Translated by Machine Heart Author: Raphael G Editor: Big Chicken Using LoRA to build faster AI models. AI models are becoming increasingly powerful and complex, and their speed has become one of the standards for measuring advancement. If AI is a luxury sports car, then the LoRA fine-tuning technology is … Read more

Essential Jetson Development Tools for NVIDIA Developers

by
Essential Jetson Development Tools for NVIDIA Developers

The NVIDIA Jetson platform provides extensive access to edge AI and robotics application development through the world’s most comprehensive AI software stack and ecosystem. It integrates scalable platform software, modern AI stacks, flexible microservices and APIs, ROS packages, and AI workflows tailored for specific applications. In this talk, you will learn the skills to accelerate … Read more

How to Calculate Wireless Module Communication Time?

by
How to Calculate Wireless Module Communication Time?

In industrial communication scenarios, both LoRa and ZigBee wireless communication technologies are widely used. However, during the application process, we often find ourselves confused about the calculation of their communication time. Today, let’s understand how their communication time is calculated together. Product Prototype GxCOM-NET is an intelligent LoRa | ZigBee data communication box independently developed … Read more