CheckSupport: Local LLM Tool Automates Manuscript Checklist Selection and Completion
CheckSupport, an open-source system that can be deployed locally, has been created by researchers to automate the suggestion and fulfillment of reporting checklists for scientific papers using large language models. This system utilizes a staged prompting method to break down reporting processes into specific inference tasks, focusing on accurate extraction rather than generating text. All inference tasks are conducted locally with instruction-tuned models, ensuring data privacy and facilitating reproducible, auditable workflows. When tested on a set of peer-reviewed manuscripts, CheckSupport demonstrated a 90% accuracy rate for checklist recommendations and 88% for item-level completions. The tool seeks to improve compliance with reporting guidelines, which often suffer from manual inconsistencies.
Key facts
- CheckSupport is an open-source, locally deployable system.
- It uses large language models to automate checklist recommendation and completion.
- It employs a staged prompting strategy for constrained inference tasks.
- All inference is performed locally to preserve data privacy.
- Achieved 90% accuracy for checklist recommendations.
- Achieved 88% accuracy for item-level completions.
- Evaluated on a corpus of peer-reviewed manuscripts.
- Aims to improve adherence to reporting guidelines.
Entities
—