Measuring the Machine: Evaluating Generative AI as Pluralist Sociotechnical Systems
A new thesis argues that generative AI evaluation must shift from functionalist benchmarks to a descriptive, sociotechnical framework. The author proposes Machine-Society-Human (MaSH) Loops to trace how models, users, and institutions co-construct meaning and values. The work reframes evaluation as examining enacted values rather than judging outputs.
Key facts
- Thesis argues generative AI must be evaluated as a pluralist sociotechnical system
- Proposes Machine-Society-Human (MaSH) Loops framework
- Critiques functionalist and prescriptive evaluation approaches
- Evaluation shifts from judging outputs to examining enacted values
- Three contributions: conceptual, methodological, and empirical
Entities
Institutions
- arXiv