TeXFix-Bench: Benchmark for LLM-Based Document Source Repair

Researchers have introduced TeXFix-Bench, a multi-format benchmark designed to evaluate large language models (LLMs) on the task of repairing faulty document source code. The benchmark is grounded in an empirically mined fault taxonomy, addressing the limitations of previous evaluations that used ad-hoc fault injections. The study, available on arXiv (2608.07617), involved a Grounded-Theory analysis of 168 verified hard-crash LaTeX faults sourced from TeX Stack Exchange, GitHub commits, and package documentation. Through dual open coding with a kappa of 0.34, the researchers developed an 18-category taxonomy, which was instantiated as DocMut, a set of 48 AST-aware mutation operators applicable across three formats: LaTeX, Typst, and Markdown. In cross-benchmark tests with three models, DocMut faults proved 5.6 to 9.2 percentage points harder to repair than pattern-based mutations on the same seeds. A real-error case study, involving 88 mined human crashes, achieved a 67.0% repair success rate, bracketing the performance of synthetic faults. This work provides a more realistic and challenging benchmark for LLM-based document repair, with implications for improving the reliability of scientific and technical writing tools.