Language Modeling Articles

What is the World’s First Large Language Model for Semiconductors?

2026-07-15 by boardor

This account has repeatedly expressed concerns: Western countries led by the United States will accelerate the intelligent upgrade of the semiconductor industry to target the current confidentiality policies implemented within the domestic semiconductor industry (even between corporate departments). Whether this policy truly confuses the adversary is currently unassessable (after all, foreign equipment still dominates), but … Read more

Meta’s New Multi-Token Attention!

2026-07-02 by boardor

Title: Multi-Token Attention Paper Link: https://arxiv.org/pdf/2504.00927 Innovations This paper presents a new attention method—Multi-Token Attention (MTA), which allows the model to adjust attention weights based on multiple query and key vectors simultaneously. By applying convolution operations on queries, keys, and attention heads, MTA enables nearby queries and keys to influence each other, resulting in more … Read more

Exploring the New ChatGLM Language Model

2026-06-20 by boardor

I have grown tired of the stable diffusion I was using before, and I am now a well-known artist with thousands of followers on Pixiv. Today, I picked up a new toy, ChatGLM! This is a large-scale bilingual language model with question-and-answer and dialogue capabilities, optimized for Chinese, and is currently in an invitation-only beta … Read more

Saturn Cloud Edge Computing Microserver SE110S-WA32 Supports DeepSeek for Local Deployment of Private Inference Models

2026-04-30 by boardor

Model IntroductionDeepSeek-R1-Distill-Qwen-7B is a high-performance language model that leverages the inference capabilities of DeepSeek-R1. By using distillation technology, it transfers the inference mode to a smaller Qwen model, significantly reducing resource consumption while maintaining high performance, making it more suitable for deployment in resource-constrained environments.This model excels in various fields such as mathematics, programming, and … Read more

Mastering Lora Training: A Step-by-Step Guide

2026-04-28 by boardor

Have you ever wondered why some people seem completely clueless about AI, yet confidently discuss its various benefits? Do they have a secret manual on Lora training hidden in their minds? Today, let me take you into this magical world and show you how we can gradually become little experts in AI! Understanding the Past … Read more

TinyLlama-1.1B: Training a Compact Language Model from Scratch

2026-01-26 by boardor

©PaperWeekly Original · Author | StatNLP Institution | Singapore University of Technology and Design The TinyLlama project aims to pre-train on 3 trillion tokens, building a Llama model with 1.1 billion parameters. With careful optimization, we only need 16 A100-40G GPUs to complete this task in 90 days🚀🚀. The model training started on 2023-09-01 and … Read more

OlaGPT: A Cognitive Framework Enhancing Language Models by 85%

2025-07-07 by boardor

New Intelligence Report Editor: LRS [New Intelligence Guide] Enabling language models to think like humans. When ChatGPT was first released, it shocked us with its performance in conversations, so much so that it created the illusion that language models possess “thinking abilities”. However, as researchers delved deeper into language models, they gradually discovered that there … Read more

DeepMind Introduces Relational RNN: A Tool for Reinforcement Learning

2025-05-07 by boardor

New Intelligence Compilation Source: arxiv Editor: Xiao Qin 【New Intelligence Guide】Traditional memory architectures struggle with relational reasoning. This paper from DeepMind and University College London proposes the Relational Memory Core (RMC), which can perform relational reasoning within sequential information, achieving state-of-the-art performance on the WikiText-103, Project Gutenberg, and GigaWord datasets. Paper: https://arxiv.org/pdf/1806.01822v1.pdf Memory-based neural networks … Read more