Mixture-of-Experts (MoE) enables efficient scaling of large language models (LLMs) with sparsely activated experts during inference. To effectively deploy large MoE models on memory-constrained devices, many systems introduce *expert offloading* that caches a subset of experts in fast memory, leaving others on slow memory to run on CPU or load on demand. While some research has exploited the locality of expert activations, where consecutive tokens activate similar experts, the degree of this **local routing consistency** varies across models and remains understudied. In this paper, we propose two metrics to measure local routing consistency of MoE models: (1) **Segment Routing Best Performance (SRP)**, which evaluates how well a fixed group of experts can cover the needs of a segment of tokens, and (2) **Segment Cache Best Hit Rate (SCH)**, which measures the hit rate of an expert cache utilizing a length of future information under a cache limit. We analyze 20 MoE LLMs with diverse sizes and architectures and use toy models to verify key factors related to local routing consistency. We find a strong trade-off between local routing consistency and *local* load balance, while showing that *global* load balance can coexist with local routing consistency. Meanwhile, settings like shared experts that decrease expert combination space can lead to low local routing consistency. We further reveal that domain-specialized experts contribute more to routing consistency than vocabulary-specialized ones, and that most models balance between cache effectiveness and efficiency with cache sizes approximately twice the active experts. These findings pave the way for memory-efficient MoE design and deployment without compromising inference speed. We publish the code for replicating experiments at https://github.com/ljcleo/moe-lrc .

翻译：混合专家（MoE）模型通过稀疏激活专家实现了大语言模型（LLM）的高效扩展推理。为了在内存受限的设备上有效部署大型MoE模型，许多系统引入了*专家卸载*机制，将部分专家缓存于高速内存中，其余专家则保留在低速内存中由CPU执行或按需加载。尽管已有研究利用专家激活的局部性（即连续令牌倾向于激活相似的专家），但这种**局部路由一致性**的程度因模型而异且尚未得到充分研究。本文提出两种度量MoE模型局部路由一致性的指标：（1）**分段路由最优性能（SRP）**，用于评估固定专家组对一段令牌需求的覆盖能力；（2）**分段缓存最优命中率（SCH）**，用于衡量在缓存容量限制下利用未来信息长度的专家缓存命中率。我们分析了20个不同规模和架构的MoE大语言模型，并通过理论模型验证了影响局部路由一致性的关键因素。研究发现局部路由一致性与*局部*负载均衡存在显著权衡，而*全局*负载均衡可与局部路由一致性共存。同时，共享专家等减少专家组合空间的设置可能导致较低的局部路由一致性。进一步研究表明，领域专业化专家比词汇专业化专家对路由一致性的贡献更大，且多数模型在缓存容量约为激活专家数两倍时能平衡缓存效果与效率。这些发现为在不影响推理速度的前提下实现内存高效的MoE设计与部署奠定了基础。实验复现代码发布于 https://github.com/ljcleo/moe-lrc 。