CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physics
A fine-grained benchmark for calculation-intensive reasoning in condensed matter physics.
Citation
Weida Wang*, Dongchen Huang*, Jiatong Li*, Tengchao Yang, Ziyang Zheng, Di Zhang, Dong Han et al. (2026). CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physics. ICLR 2026. (Co-First Author)
About this work
CMPhysBench contains more than 520 graduate-level calculation questions in condensed matter physics, covering topics such as magnetism, superconductivity, and strongly correlated systems.
The benchmark uses Scalable Expression Edit Distance (SEED) to award partial credit by comparing tree representations of mathematical expressions. In the reported evaluation, the best-performing model, Grok-4, reaches an average SEED score of 36 and 28% accuracy.