LLM Benchmarking on NRC Reactor Operator Exam

A recent study is looking into the Gemma 4 31B-IT, which is a multimodal model boasting 31 billion parameters, in connection with the Reactor Operator licensing exam conducted by the U.S. Nuclear Regulatory Commission (NRC). Researchers are comparing eight different configurations of this model against 14 Generic Fundamentals Examinations (GFE) that took place in March from 2015 to 2021. These include seven exams each for both pressurized and boiling water reactors, with an 80% passing rate, the same as for humans. The analysis examines the base model and those enhanced with supervised fine-tuning, retrieval-augmented generation, and retrieval-augmented fine-tuning. The goal is to understand how large language models can be utilized in the nuclear energy field.