All hand-object interaction is controlled by forces that the two bodies exert on each other, but little work has been done in modeling these underlying forces when doing pose and contact estimation from RGB/RGB-D data. Given the pose of the hand and object from any pose estimation system, we propose an end-to-end differentiable model that refines pose estimates by learning the forces experienced by the object at each vertex in its mesh. By matching the learned net force to an estimate of net force based on finite differences of position, this model is able to find forces that accurately describe the movement of the object, while resolving issues like mesh interpenetration and lack of contact. Evaluating on the ContactPose dataset, we show this model successfully corrects poses and finds contact maps that better match the ground truth, despite not using any RGB or depth image data.

翻译：所有人工物体的相互作用都由两机构相互施加的力量所控制,但是在根据RGB/RGB-D数据进行显示和接触估计时,在模拟这些基本力量时没有做多少工作。鉴于手和物体来自任何构成估计系统,我们提议了一个端到端的可区分模型,通过了解物体在网格中每个顶部所经历的力来改进估计。通过将所学净力与基于位置差异有限的净力估计数相匹配,这一模型能够找到精确描述物体移动情况的力量,同时解决网状内穿透和缺乏接触等问题。在ContelectPose数据集上,我们成功地纠正了该模型所显示的方位,并找到更符合地面真相的接触图,尽管没有使用任何RGB或深度图像数据。