Network Intrusion Detection Systems (NIDS) developed using publicly available datasets predominantly focus on enterprise environments, raising concerns about their effectiveness for converged Information Technology (IT) and Operational Technology (OT) in energy infrastructures. This study evaluates the representativeness of five widely used datasets: CIC-IDS2017, SWaT, WADI, Sherlock, and CIC-Modbus2023 against network-detectable MITRE ATT&CK techniques extracted from documented energy sector incidents. Using a structured five-step analytical approach, this article successfully developed and performed a gap analysis that identified 94 network observable techniques from an initial pool of 274 ATT&CK techniques. Sherlock dataset exhibited the highest mean coverage (0.56), followed closely by CIC-IDS2017 (0.55), while SWaT and WADI recorded the lowest scores (0.38). Combining CIC-IDS2017, Sherlock, and CIC-Modbus2023 achieved an aggregate coverage of 92%, highlighting their complementary strengths. The analysis identifies critical gaps, particularly in lateral movement and industrial protocol manipulation, providing a clear pathway for dataset enhancement and more robust NIDS evaluation in hybrid IT/OT energy environments.
翻译:基于公开数据集开发的网络入侵检测系统(NIDS)主要聚焦于企业环境,这引发了对其在能源基础设施中融合信息技术(IT)与运营技术(OT)环境下有效性的担忧。本研究评估了五个广泛使用的数据集——CIC-IDS2017、SWaT、WADI、Sherlock和CIC-Modbus2023——相对于从已记录的能源行业事件中提取的网络可检测MITRE ATT&CK技术的代表性。通过采用结构化的五步分析方法,本文成功开发并执行了差距分析,从初始的274项ATT&CK技术中识别出94项网络可观测技术。Sherlock数据集表现出最高的平均覆盖率(0.56),紧随其后的是CIC-IDS2017(0.55),而SWaT和WADI的得分最低(0.38)。结合CIC-IDS2017、Sherlock和CIC-Modbus2023可实现92%的总体覆盖率,突显了它们的互补优势。该分析揭示了关键差距,特别是在横向移动和工业协议操纵方面,为混合IT/OT能源环境下的数据集增强和更稳健的NIDS评估提供了清晰路径。