Generating controllable and editable human motion sequences is a key challenge in 3D Avatar generation. It has been labor-intensive to generate and animate human motion for a long time until learning-based approaches have been developed and applied recently. However, these approaches are still task-specific or modality-specific\cite {ahuja2019language2pose}\cite{ghosh2021synthesis}\cite{ferreira2021learning}\cite{li2021ai}. In this paper, we propose ``UDE", the first unified driving engine that enables generating human motion sequences from natural language or audio sequences (see Fig.~\ref{fig:teaser}). Specifically, UDE consists of the following key components: 1) a motion quantization module based on VQVAE that represents continuous motion sequence as discrete latent code\cite{van2017neural}, 2) a modality-agnostic transformer encoder\cite{vaswani2017attention} that learns to map modality-aware driving signals to a joint space, and 3) a unified token transformer (GPT-like\cite{radford2019language}) network to predict the quantized latent code index in an auto-regressive manner. 4) a diffusion motion decoder that takes as input the motion tokens and decodes them into motion sequences with high diversity. We evaluate our method on HumanML3D\cite{Guo_2022_CVPR} and AIST++\cite{li2021learn} benchmarks, and the experiment results demonstrate our method achieves state-of-the-art performance. Project website: \url{https://github.com/zixiangzhou916/UDE/

翻译：在 3D Astatar 的 3D 產生 3D 產生 控制可控和可编辑的人类動脈序列中, 3D 產生 3D 產生 3Affatar 產生 3D 產生和動動動的人類動脈序列中, 一直需要勞力密集的產生和動動動動動動動動動動動動動動作, 直到最近才被開發行。 然而, 這些方法仍具特定工作或模式特定功能的 {ahuja2019 語系 語系, 根據 VQVADE 代表连续的動動動動動動動動動動動動動動動序, 根據 20ODFL 20 數據解動變變變變變變數, 根據化為 數據機動動動動動動動動動動動作 、 數據數據化機機路路路路路路、 數化數化數數數數數數數、 數化數數數數數化機化數據機路路路路路路路、數、數、數、數效變化機化機化數、數學路路路路運化路路路路路路路運化路路路、數路、數、數化數化數路路路運化路路路、數路路路路路路路路路路路路路路路路路路、數、數路路路路路路路路路路路路路路路路路路路路路、數、數、數、數路路路路路路路路路路路路路路路路路路運運化路路路路路路運運運運運運運運路路路路路、數路路路路路路路路路路路路路路路路路路路路路路路路路路路路路路、數