Multi-Token Prediction (MTP) in V3: Inference Acceleration
Last time we discussed the structure of the Deepseek model. Today, I will share the inference acceleration tool in V3, the multi-token prediction (MTP) module, which is somewhat similar to speculative decoding, but the specific code has not been open-sourced. Speculative decoding was introduced in a series of articles on inference optimization, which essentially changes … Read more