L-RoPE

MultiTalk: A State-of-the-Art Model for Multi-Character Dialogue Generation with 98.7% Speech-Visual Alignment Accuracy!

by
MultiTalk: A State-of-the-Art Model for Multi-Character Dialogue Generation with 98.7% Speech-Visual Alignment Accuracy!

Developed by Sun Yat-sen University, Meituan, and Hong Kong University of Science and Technology, MultiTalk enables the generation of multi-character dialogue videos. It achieves state-of-the-art performance in synchronizing speech with lip movements and supports interactions between characters, objects, and scenes through prompts. Related Links Homepage: https://meigen-ai.github.io/multi-talk/ Code: https://github.com/MeiGen-AI/MultiTalk Paper: https://arxiv.org/abs/2505.22647 Paper Introduction In recent years, … Read more

MeiGen-MultiTalk: Enabling Multi-Person Interactive Video Generation from a Single Photo

by
MeiGen-MultiTalk: Enabling Multi-Person Interactive Video Generation from a Single Photo

Reposted from Big Company Talk Recently, Meituan launched the audio-driven multi-person dialogue video generation framework MultiTalk, which has been open-sourced on GitHub. It introduces the innovative L-RoPE binding technology, which accurately addresses the challenges of multi-audio streams and character misalignment through label rotation positional encoding. This framework innovatively employs local parameter training and multi-task learning … Read more