Joint multilingual instruction tuning is a widely adopted approach to improve the multilingual instruction-following ability and downstream performance of large language models (LLMs), but the resulting multilingual capability remains highly sensitive to the composition and selection of the training data. Existing selection methods, often based on features like text quality, diversity, or task relevance, typically overlook the intrinsic linguistic structure of multilingual data. In this paper, we propose LangGPS, a lightweight two-stage pre-selection framework guided by language separability which quantifies how well samples in different languages can be distinguished in the model's representation space. LangGPS first filters training data based on separability scores and then refines the subset using existing selection methods. Extensive experiments across six benchmarks and 22 languages demonstrate that applying LangGPS on top of existing selection methods improves their effectiveness and generalizability in multilingual training, especially for understanding tasks and low-resource languages. Further analysis reveals that highly separable samples facilitate the formation of clearer language boundaries and support faster adaptation, while low-separability samples tend to function as bridges for cross-lingual alignment. Besides, we also find that language separability can serve as an effective signal for multilingual curriculum learning, where interleaving samples with diverse separability levels yields stable and generalizable gains. Together, we hope our work offers a new perspective on data utility in multilingual contexts and support the development of more linguistically informed LLMs.

翻译：联合多语言指令微调是一种广泛采用的方法，旨在提升大语言模型的多语言指令遵循能力及下游任务性能，但其最终的多语言能力仍对训练数据的构成与选择高度敏感。现有的选择方法通常基于文本质量、多样性或任务相关性等特征，往往忽视了多语言数据的内在语言结构。本文提出LangGPS，一种轻量级的两阶段预选框架，以语言可分离性为指导——该指标量化了不同语言样本在模型表示空间中的可区分程度。LangGPS首先依据可分离性分数筛选训练数据，随后利用现有选择方法对子集进行精炼。在六个基准测试和22种语言上的大量实验表明，在现有选择方法基础上应用LangGPS，能够提升其在多语言训练中的有效性与泛化能力，尤其对于理解任务和低资源语言效果显著。进一步分析揭示，高可分离性样本有助于形成更清晰的语言边界并支持更快的适应，而低可分离性样本则倾向于作为跨语言对齐的桥梁。此外，我们还发现语言可分离性可作为多语言课程学习的有效信号，其中穿插不同可分离性水平的样本能够带来稳定且可泛化的性能提升。综上，我们希望本研究为多语言语境下的数据效用提供新的视角，并支持开发更具语言感知能力的大语言模型。