As the demand for immersive 3D content grows, the need for intuitive and efficient interaction methods becomes paramount. Current techniques for physically manipulating 3D content within Virtual Reality (VR) often face significant limitations, including reliance on engineering-intensive processes and simplified geometric representations, such as tetrahedral cages, which can compromise visual fidelity and physical accuracy. In this paper, we introduce GS-Verse (Gaussian Splatting for Virtual Environment Rendering and Scene Editing), a novel method designed to overcome these challenges by directly integrating an object's mesh with a Gaussian Splatting (GS) representation. Our approach enables more precise surface approximation, leading to highly realistic deformations and interactions. By leveraging existing 3D mesh assets, GS-Verse facilitates seamless content reuse and simplifies the development workflow. Moreover, our system is designed to be physics-engine-agnostic, granting developers robust deployment flexibility. This versatile architecture delivers a highly realistic, adaptable, and intuitive approach to interactive 3D manipulation. We rigorously validate our method against the current state-of-the-art technique that couples VR with GS in a comparative user study involving 18 participants. Specifically, we demonstrate that our approach is statistically significantly better for physics-aware stretching manipulation and is also more consistent in other physics-based manipulations like twisting and shaking. Further evaluation across various interactions and scenes confirms that our method consistently delivers high and reliable performance, showing its potential as a plausible alternative to existing methods.

翻译：随着沉浸式3D内容需求的增长，对直观高效交互方法的需求变得至关重要。当前在虚拟现实（VR）中物理操控3D内容的技术常面临显著局限，包括依赖工程密集型流程及简化的几何表示（如四面体笼），这可能损害视觉保真度与物理准确性。本文提出GS-Verse（用于虚拟环境渲染与场景编辑的高斯溅射），一种通过将物体网格直接与高斯溅射（GS）表示相集成来克服这些挑战的新方法。我们的方法实现了更精确的表面近似，从而产生高度真实的形变与交互。通过利用现有3D网格资产，GS-Verse促进了无缝内容复用并简化了开发工作流。此外，我们的系统设计为物理引擎无关，为开发者提供了强大的部署灵活性。这种多功能架构为交互式3D操控提供了高度真实、适应性强且直观的途径。我们通过一项涉及18名参与者的对比用户研究，严格验证了我们的方法相对于当前将VR与GS耦合的最先进技术。具体而言，我们证明该方法在物理感知拉伸操控方面统计显著更优，且在扭转、摇晃等其他基于物理的操控中也更具一致性。跨多种交互与场景的进一步评估证实，我们的方法始终提供高且可靠的性能，显示出其作为现有方法可行替代方案的潜力。