The strength of the human hand lies in its ability to manipulate small objects precisely and robustly. In contrast, simple robotic grippers have low dexterity and fail to handle small objects effectively. This is why many automation tasks remain unsolved by robots. This paper presents an optimization-based framework for in-hand manipulation with a robotic hand equipped with compact Magnetic Tactile Sensors (MTSs). The small form factor of the robotic hand from Shadow Robot introduces challenges in estimating the state of the object while satisfying contact constraints. To address this, we formulate a trajectory optimization problem using Nonlinear Programming (NLP) for finger movements while ensuring contact points to change along the geometry of the fingers. Using the optimized trajectory from the solver, we implement and test an open-loop controller for rolling motion. To further enhance robustness and accuracy, we introduce a force controller for the fingers and a state estimator for the object utilizing MTSs. The proposed framework is validated through comparative experiments, showing that incorporating the force control with compliance consideration improves the accuracy and robustness of the rolling motion. Rolling an object with the force controller is 30\% more likely to succeed than running an open-loop controller. The demonstration video is available at https://youtu.be/6J_muL_AyE8.

翻译：人类手部的优势在于其能够精确且鲁棒地操控小型物体。相比之下，简单的机器人夹爪灵活性较低，难以有效处理小型物体。这正是许多自动化任务仍无法由机器人完成的原因。本文提出了一种基于优化的框架，用于配备紧凑型磁触觉传感器（MTSs）的机器人手进行手内操作。Shadow Robot公司设计的机器人手因其小型化结构，在满足接触约束的同时估计物体状态方面带来了挑战。为解决这一问题，我们采用非线性规划（NLP）为手指运动构建轨迹优化问题，同时确保接触点沿手指几何形状变化。利用求解器得到的优化轨迹，我们实现并测试了用于滚动运动的开环控制器。为进一步增强鲁棒性和精度，我们引入了基于MTSs的手指力控制器和物体状态估计器。通过对比实验验证了所提框架的有效性，结果表明结合柔顺性考虑的力控制提高了滚动运动的精度和鲁棒性。使用力控制器滚动物体的成功率比运行开环控制器高出30%。演示视频可在 https://youtu.be/6J_muL_AyE8 查看。