Fairness has become a trending topic in natural language processing (NLP), which addresses biases targeting certain social groups such as genders and religions. However, regional bias in language models (LMs), a long-standing global discrimination problem, still remains unexplored. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models that are broadly used in NLP tasks. In addition to verifying the existence of regional bias in LMs, we find that the biases on regional groups can be strongly influenced by the geographical clustering of the groups. We accordingly propose a HiErarchical Regional Bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with respect to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB.

翻译：公平已成为自然语言处理(NLP)中一个趋势性议题,它涉及针对性别和宗教等某些社会群体的偏见,然而,长期存在的全球歧视问题,即语言模式方面的区域偏见仍未得到解决。本文通过分析在自然语言处理任务中广泛使用的经过培训的语言模式所学到的区域偏见,弥补了差距。除了核实语言处理中存在区域偏见之外,我们发现对区域集团的偏见可能受到这些群体地理分组的强烈影响。因此,我们提议采用高等级区域双向评估方法(HERB),利用次区域分组的信息量化预先培训的LMS中的偏见。实验表明,我们的等级指标能够有效地评估区域在综合专题上的偏见,并衡量可传播到下游任务的潜在区域偏见。我们的代码可在https://github.com/Bernard-Yang/HERB查阅。