Recent advancements in image editing have utilized large-scale multimodal models to enable intuitive, natural instruction-driven interactions. However, conventional methods still face significant challenges, particularly in spatial reasoning, precise region segmentation, and maintaining semantic consistency, especially in complex scenes. To overcome these challenges, we introduce SmartFreeEdit, a novel end-to-end framework that integrates a multimodal large language model (MLLM) with a hypergraph-enhanced inpainting architecture, enabling precise, mask-free image editing guided exclusively by natural language instructions. The key innovations of SmartFreeEdit include:(1)the introduction of region aware tokens and a mask embedding paradigm that enhance the spatial understanding of complex scenes;(2) a reasoning segmentation pipeline designed to optimize the generation of editing masks based on natural language instructions;and (3) a hypergraph-augmented inpainting module that ensures the preservation of both structural integrity and semantic coherence during complex edits, overcoming the limitations of local-based image generation. Extensive experiments on the Reason-Edit benchmark demonstrate that SmartFreeEdit surpasses current state-of-the-art methods across multiple evaluation metrics, including segmentation accuracy, instruction adherence, and visual quality preservation, while addressing the issue of local information focus and improving global consistency in the edited image. Our project will be available at https://github.com/smileformylove/SmartFreeEdit.

翻译：近年来，图像编辑领域利用大规模多模态模型实现了直观、自然的指令驱动交互。然而，传统方法仍面临显著挑战，尤其是在空间推理、精确区域分割以及保持语义一致性方面，在复杂场景中尤为突出。为克服这些挑战，我们提出了SmartFreeEdit，一种新颖的端到端框架，该框架将多模态大语言模型（MLLM）与超图增强的修复架构相结合，实现了仅通过自然语言指令引导的精确、无需掩码的图像编辑。SmartFreeEdit的核心创新包括：（1）引入区域感知令牌和掩码嵌入范式，以增强对复杂场景的空间理解；（2）设计了一个推理分割流程，旨在基于自然语言指令优化编辑掩码的生成；（3）一个超图增强的修复模块，确保在复杂编辑过程中同时保持结构完整性和语义连贯性，克服了基于局部图像生成的局限性。在Reason-Edit基准上进行的大量实验表明，SmartFreeEdit在多个评估指标上超越了当前最先进的方法，包括分割准确性、指令遵循度和视觉质量保持度，同时解决了局部信息聚焦问题，并提升了编辑图像的全局一致性。我们的项目将在https://github.com/smileformylove/SmartFreeEdit上公开。