Machine Psychometrics: Measuring Latent Structure in AI Behavior

A recent publication on arXiv introduces Machine Psychometrics, a new measurement science aimed at assessing artificial intelligence by utilizing principles from mathematical psychology to analyze the underlying psychological structures in AI behavior, rather than focusing solely on performance metrics. The authors contend that existing evaluation methods favor capability scores, resulting in two related errors: Artificial Mind Blindness, which overlooks psychological organization in non-biological entities, and Artificial Mind Projection, which assumes human-like mental states based on fluent actions. They propose that this deadlock can be resolved by implementing a structured measurement framework beneath the consciousness debate. The research references Michael Levin's continuum perspective on cognition and employs various mathematical psychology techniques, including Item Response Theory and Bayesian cognitive modeling. The paper can be found on arXiv under ID 2605.23952.