LLM-Based Chatbots vs Dashboards in Manufacturing Decision Support
A recent study available on arXiv investigates the performance of LLM-based conversational agents (CAs) against conventional dashboards in manufacturing decision support. The research involved a mixed factorial experiment with 134 industrial decision-makers, who were tasked with completing three progressively complex activities using either a conversational user interface (CUI) or a dashboard. The study assessed factors such as perceived mental workload (MWL), accuracy of decisions, time taken to complete tasks, and intended reliance on the tools. Findings suggest that the effectiveness of LLM-based CAs varies with task complexity, offering some benefits for direct data access, yet not significantly replacing dashboards. The study can be found under arXiv:2605.31287.
Key facts
- Study compares LLM-based conversational agents with dashboards in manufacturing decision support.
- Mixed factorial experiment with 2x3 design involving 134 industrial decision-makers.
- Tasks varied in complexity to test interface effectiveness.
- Measured mental workload, decision accuracy, completion time, and intended reliance.
- LLM-based CAs may provide more direct access to operational data.
- Dashboards remain dominant in industrial contexts.
- Effectiveness of CAs depends on information-processing demands of the task.
- Paper published on arXiv with ID 2605.31287.
Entities
Institutions
- arXiv