Human Motion Diffusion Model
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or and Amit H. Bermano Tel Aviv University, Israel
Abstract
Natural and expressive human motion generation is the holy grail of computer animation. It is a challenging task, due to the diversity of possible motion, human perceptual sensitivity to it, and the difficulty of accurately describing it. Therefore, current generative solutions are either low-quality or limited in expressiveness. Diffusion models, which have already shown remarkable generative capabilities in other domains, are promising candidates for human motion due to their many-to-many nature, but they tend to be resource hungry and hard to control. In this paper, we introduce Motion Diffusion Model (MDM), a carefully adapted classifier-free diffusion-based generative model for the human motion domain. MDM is transformer-based, combining insights from motion generation literature. A notable design-choice is the prediction of the sample, rather than the noise, in each diffusion step. This facilitates the use of established geometric losses on the locations and velocities of the motion, such as the foot contact loss. As we demonstrate, MDM is a generic approach, enabling different modes of conditioning, and different generation tasks. We show that our model is trained with light
中文速览
自然逼真的人体动作生成一直是计算机动画领域的核心难题——动作种类繁多、人类感知极为敏感，而现有方法要么质量不高、要么表达能力受限。研究团队提出了运动扩散模型（Motion Diffusion Model，MDM），将无分类器引导的扩散模型（classifier-free diffusion）与运动领域已有知识相结合：采用轻量级Transformer编码器作为骨干网络，并在每个扩散步骤中直接预测干净样本而非噪声，从而能够方便地引入位置、速度、脚部接触等几何损失，有效消除动作抖动和脚部滑动等失真问题。实验表明，MDM在文本驱动动作生成（text-to-motion）和动作类别驱动生成（action-to-motion）的主流基准上均达到了业界最优水平，用户研究中人类评估者有42%的情况更偏好MDM的生成结果而非真实动作，且整个模型只需在一块中端GPU上训练约三天。这项工作的重要意义在于：它以极低的计算成本将扩散模型的强大生成能力引入人体动作领域，同时支持文本、动作类别等多种条件控制以及动作补全和局部编辑，为动画、游戏和机器人等应用提供了一个通用、高效的生成框架。
原文 arXiv:2209.14916;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2209.14916v2