Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents

The paper identifies a gap in LLM evaluation: existing benchmarks focus on domain knowledge and QA, overlooking intrinsic quality review of professional documents. National standards like China's GB/T documents are lengthy, structured, and governed by explicit rules for scope, terminology, and cross-section consistency, making them a representative testbed.

To address this, the authors introduce GB/T-Bench, a benchmark with a hierarchical GB/T Review Taxonomy covering structure, scope alignment, normative modality, terminology consistency, and normative references, comprising 25 error types. They generate 7,306 traceable error instances from 488 documents using a controllable counterexample mechanism combining deterministic rules and constrained LLM rewriting.

A diagnosis-oriented evaluation protocol requires exact matches on error location, dimension, and type, plus document-level coverage metrics. They also propose GB/T-Reviewer, a multi-agent framework that coordinates global inspection, targeted diagnosis, rule scanning, and result verification via specialized skills.

Experiments with 14 mainstream LLMs show a substantial human-LLM gap: the best model achieves 0.3280 CMCS versus 0.6640 for experts. GB/T-Reviewer raises the best score to 0.5094, demonstrating the value of structured skill coordination for rule-intensive review. The work aims to enable trustworthy AI in standardization and other high-stakes document domains.