While web data quality is crucial for large language models, most curation efforts focus on filtering and deduplication,treating HTML-to-text extraction as a fixed pre-processing step. Existing web corpora rely on heuristic-based extractors like Trafilatura, which struggle to preserve document structure and frequently corrupt structured elements such as formulas, codes, and tables. We hypothesize that improving extraction quality can be as impactful as aggressive filtering strategies for downstream performance. We introduce MinerU-HTML, a novel extraction pipeline that reformulates content extraction as a sequence labeling problem solved by a 0.6B-parameter language model. Unlike text-density heuristics, MinerU-HTML leverages semantic understanding and employs a two-stage formatting pipeline that explicitly categorizes semantic elements before converting to Markdown. Crucially, its model-based approach is inherently scalable, whereas heuristic methods offer limited improvement pathways. On MainWebBench, our benchmark of 7,887 annotated web pages, MinerU-HTML achieves 81.8\% ROUGE-N F1 compared to Trafilatura's 63.6\%, with exceptional structured element preservation (90.9\% for code blocks, 94.0\% for formulas). Using MinerU-HTML, we construct AICC (AI-ready Common Crawl), a 7.3-trillion token multilingual corpus from two Common Crawl snapshots. In controlled pretraining experiments where AICC and Trafilatura-extracted TfCC undergo identical filtering, models trained on AICC (62B tokens) achieve 50.8\% average accuracy across 13 benchmarks, outperforming TfCC by 1.08pp-providing direct evidence that extraction quality significantly impacts model capabilities. AICC also surpasses RefinedWeb and FineWeb on key benchmarks. We publicly release MainWebBench, MinerU-HTML, and AICC, demonstrating that HTML extraction is a critical, often underestimated component of web corpus construction.

翻译：尽管网络数据质量对大型语言模型至关重要，但多数数据筛选工作集中于过滤与去重，将HTML到文本的提取视为固定的预处理步骤。现有网络语料库依赖基于启发式规则的提取器（如Trafilatura），这些方法难以保持文档结构，并经常破坏公式、代码和表格等结构化元素。我们假设，提升提取质量对下游任务性能的影响可能与激进过滤策略同样显著。本文提出MinerU-HTML——一种新颖的提取流程，将内容提取重新定义为由6亿参数语言模型解决的序列标注问题。与基于文本密度的启发式方法不同，MinerU-HTML利用语义理解能力，采用两阶段格式化流程：先显式分类语义元素，再转换为Markdown格式。关键在于，其基于模型的方法具有内在可扩展性，而启发式方法的改进路径有限。在包含7,887个标注网页的基准测试集MainWebBench上，MinerU-HTML的ROUGE-N F1值达到81.8%（Trafilatura为63.6%），并在结构化元素保留方面表现卓越（代码块90.9%，公式94.0%）。基于MinerU-HTML，我们构建了AICC（AI就绪通用爬虫语料库）——一个从两个Common Crawl快照中提取的、包含7.3万亿token的多语言语料库。在对照预训练实验中，对AICC与Trafilatura提取的TfCC语料进行相同过滤后，使用AICC（620亿token）训练的模型在13个基准测试中平均准确率达到50.8%，较TfCC提升1.08个百分点，直接证明提取质量显著影响模型能力。AICC在关键基准测试中也超越了RefinedWeb和FineWeb。我们公开发布了MainWebBench、MinerU-HTML和AICC，表明HTML提取是网络语料库构建中关键且常被低估的环节。