Learning an animatable and clothed human avatar model with vivid dynamics and photorealistic appearance from multi-view videos is an important foundational research problem in computer graphics and vision. Fueled by recent advances in implicit representations, the quality of the animatable avatars has achieved an unprecedented level by attaching the implicit representation to drivable human template meshes. However, they usually fail to preserve the highest level of detail, particularly apparent when the virtual camera is zoomed in and when rendering at 4K resolution and higher. We argue that this limitation stems from inaccurate surface tracking, specifically, depth misalignment and surface drift between character geometry and the ground truth surface, which forces the detailed appearance model to compensate for geometric errors. To address this, we propose a latent deformation model and supervising the 3D deformation of the animatable character using guidance from foundational 2D video point trackers, which offer improved robustness to shading and surface variations, and are less prone to local minima than differentiable rendering. To mitigate the drift over time and lack of 3D awareness of 2D point trackers, we introduce a cascaded training strategy that generates consistent 3D point tracks by anchoring point tracks to the rendered avatar, which ultimately supervises our avatar at the vertex and texel level. To validate the effectiveness of our approach, we introduce a novel dataset comprising five multi-view video sequences, each over 10 minutes in duration, captured using 40 calibrated 6K-resolution cameras, featuring subjects dressed in clothing with challenging texture patterns and wrinkle deformations. Our approach demonstrates significantly improved performance in rendering quality and geometric accuracy over the prior state of the art.

翻译：从多视角视频中学习具有生动动态和逼真外观的可动画化、着衣人体化身模型，是计算机图形学与视觉领域一个重要的基础研究问题。得益于隐式表示的最新进展，通过将隐式表示附着于可驱动的人体模板网格，可动画化化身的质量已达到前所未有的水平。然而，这些方法通常难以保留最高级别的细节，在虚拟相机放大以及以4K及以上分辨率渲染时尤为明显。我们认为，这一局限源于不准确的表面追踪，具体表现为角色几何与真实表面之间的深度失准和表面漂移，这迫使细节外观模型去补偿几何误差。为解决此问题，我们提出了一种潜在变形模型，并利用基础性2D视频点追踪器的引导来监督可动画角色的3D变形；这些追踪器对光照和表面变化具有更强的鲁棒性，且比可微分渲染更不易陷入局部极小值。为缓解2D点追踪器随时间产生的漂移及其缺乏3D感知的问题，我们引入了一种级联训练策略，通过将点轨迹锚定到渲染的化身来生成一致的3D点轨迹，最终在顶点和纹理像素级别监督我们的化身。为验证方法的有效性，我们提出了一个新颖的数据集，包含五个多视角视频序列，每个序列时长超过10分钟，使用40台经过校准的6K分辨率相机拍摄，拍摄对象穿着具有挑战性纹理图案和褶皱变形的服装。与现有最先进技术相比，我们的方法在渲染质量和几何精度方面均展现出显著提升的性能。