This public account mainly focuses on cutting-edge AI technologies such as NLP, CV, LLM, RAG, and Agent, sharing practical industry cases and courses for free, helping you fully embrace AIGC.
1. Sparse Embedding (Keyword-based Sparse Vectors)
| Dimension | Description |
|---|---|
| Typical Implementations | TF-IDF, BM25, SPLADE |
| Vector Shape | 50,000+ dimensions, >95% of positions are 0 |
| Similarity Calculation | Cosine or dot product, only activated dimensions participate in calculations |
Pain Points
- Only performs “exact keyword matching”; synonyms/phrase variations become ineffective
- High-dimensional sparsity leads to storage and indexing bloat
Effectiveness
- Extremely high accuracy when keywords match, strong interpretability (can directly see “which word” scored)
Case StudyNews copyright deduplication: Editors use 5 core entity keywords from the original text as queries, retrieving suspected plagiarized articles within 10 ms with an accuracy rate of 98% using BM25.
One-sentence Selection“As long as keywords can solve the problem, don’t use neural networks.”
2. Dense Embedding (Semantic-level Dense Vectors)
| Dimension | Description |
|---|---|
| Typical Models | text-embedding-3-large, BGE, E5-mistral |
| Vector Shape | 256~1536 dimensions, all non-zero |
| Similarity Calculation | Cosine distance |
Pain Points
- Requires GPU/CPU for inference, single query takes 10~100 ms
- Only shows its power for semantically similar queries that do not match literally
Effectiveness
- Captures synonyms, hypernyms, and cross-language semantics
- Lower dimensions lead to more “squeezed” information, requiring a trade-off between accuracy and storage
Case StudySaaS customer service FAQ retrieval: A user asks in a colloquial manner, “What should I do if I forget my password?” The dense vector matches “How to reset the login password,” with the top 1 match rate increasing from 62% with keywords to 89%.
One-sentence Selection“As long as users can ask in ‘human language’, use Dense.”
3. Quantized Embedding (Compressed Dense Version)
| Dimension | Description |
|---|---|
| Compression Method | Convert float32 to int8 / uint8 |
| Compression Rate | 75% volume reduction, recall drop <1% (experiment) |
Pain Points
- Must retrain or perform Post-Training Quantization
- Extremely low bits (4 bits) can lead to significant “value range truncation” errors
Effectiveness
- Memory *4↓, Disk *4↓, Vector retrieval QPS *2↑
- Suitable for ANN engines (FAISS-IVF, Milvus) that can be loaded into memory at once
Case StudyFor an e-commerce platform with 200 million product vectors originally occupying 2.4 TB, quantization reduced it to 600 GB, with in-memory retrieval latency dropping from 18 ms to 9 ms.
One-sentence Selection“Memory is money; quantization saves costs.”
4. Binary Embedding (Extreme 0/1 Compression)
| Dimension | Description |
|---|---|
| Encoding Method | Binary quantization of float vectors using sign() or ITQ rotation |
| Storage | 1 bit × dim, saving 32× compared to float32 |
Pain Points
- Hamming distance can only sort, cannot obtain true similarity scores
- Average accuracy drops by 5~15%
Effectiveness
- Calculations change to XOR + popcount, with a single CPU core capable of over 100 million operations per second
- Preferred for offline retrieval on mobile devices
Case StudyAndroid photo gallery “Duplicate Photo Cleaner”: After binarizing 256-dimensional CNN vectors, it found similarities among 30,000 photos in 80 ms, consuming <1% battery.
One-sentence Selection“For device-side, offline, large-scale, and speed-focused applications—go with Binary.”
5. Matryoshka (Variable Dimension) Embedding
| Dimension | Description |
|---|---|
| Training Technique | Hierarchically weighted 1024-dimensional vectors, with the first 64 dimensions containing 80% of the information |
| Invocation Method | Same file, truncate as needed to dim=64/128/256… |
Pain Points
- Must use the official “MRL training” model; ordinary models will crash with hard truncation
- When early dimensions are too short, different semantics can easily “squeeze” together
Effectiveness
- One storage solution supports a “performance ↔ accuracy” slider
- Low-dimensional phase QPS increases by 3~5 times, while high-dimensional phase accuracy approaches that of the complete vector
Case StudyA startup first created a POC with 1 million documents using 64 dimensions for the demo; after signing the client, they switched to 512 dimensions without needing to re-export data.
One-sentence Selection“The boss needs a quick demo now and precise deployment later—Matryoshka prevents requirement cuts.”
6. Multi-Vector (ColBERT-style ‘Post-Interaction’)
| Dimension | Description |
|---|---|
| Representation Method | A piece of text → 128 vectors of 128 dimensions (one for each token) |
| Similarity Calculation | First calculate the maximum cosine similarity between each query token and document token, then sum (MaxSim) |
Pain Points
- Index size increases by 100×, requiring specialized compression (ColBERTv2 residual + quantization)
- Computational load during retrieval is much higher than with single vectors
Effectiveness
- Achieves fine-grained “word-to-word” matching, improving long document recall by 10~20%
- Interpretability: can highlight “which sentence” was matched
Case StudyA law firm searched through 500,000 judgment documents: After using ColBERT, when a lawyer inputs “How to calculate overtime pay for employees,” it returns paragraph-level hits, reducing document reading time from 15 minutes to 3 minutes.
One-sentence Selection“For long texts, specialized fields, and precise paragraph localization—Multi-Vector is the way to go.”
7. How to Choose the Right Solution?
| Scenario Keywords | Preferred Solution | Alternative |
|---|---|---|
| Exact Keyword Matching | Sparse | — |
| General Semantic Search | Dense | Matryoshka |
| Memory/Disk Crisis | Quantized | Binary |
| Fast Offline Search on Device | Binary | — |
| Long Document Paragraph Localization | Multi-Vector | — |
| Frequent Requirement Changes from Boss | Matryoshka | — |
8. Steps for Solution Selection
You can choose available embedding solutions based on the five elements of “data scale, latency, memory, accuracy, and interpretability” according to different scenarios.
- First use Dense to establish a baseline, then decide “whether to go more complex”.
- Before quantization/binarization, be sure to test the “recall@K drop” curve on business data, not just look at the compression rate.
- Multi-Vector index size is large; using ColBERTv2’s “residual + quantization” can compress 100 GB to 8 GB while maintaining 95% accuracy.
- When going live, combining “Sparse + Dense” for Hybrid Ranking is often the most reliable: Sparse ensures accuracy, Dense ensures recall, with adjustable weights.
Recommended Reading
-
Break the Illusion! Qwen’s latest OCR boosts recognition accuracy for seals, forms, and formulas by 45%
-
Eliminate 90% of BI Engineers: Automatically generate SQL, insights, predictions, and even draw charts for you
-
16K Star Zep outperforms many Graph RAGs with a single “Dynamic Memory Map”
-
[10,000-word article] Anthropic releases a new guide on Prompt Engineering
-
Tencent open-sources WeKnora knowledge base, seamlessly linking to the WeChat ecosystem