LLARS Platform Bridges Domain Experts and Developers for LLM Systems
The LLARS (LLM Assisted Research System) is an open-source initiative aimed at enhancing collaboration between experts in specific fields and developers for the creation of LLM-driven systems. It consists of three key components: Collaborative Prompt Engineering, which allows for simultaneous co-authoring with version control and immediate LLM testing; Batch Generation, enabling customizable output creation based on user-defined prompts, models, and data while managing costs; and Hybrid Evaluation, where both human and LLM evaluators analyze outputs using various techniques, offering real-time agreement metrics and provenance insights to pinpoint the best model-prompt pairings. Newly created prompts and models are readily accessible for batch generation, and finished batches can easily be transformed into evaluation scenarios with just one click. The platform's effectiveness was confirmed through discussions with six domain experts and three developers in the field of online counseling.
Key facts
- LLARS is an open-source platform for LLM-based systems.
- It bridges domain experts and developers.
- Three modules: Collaborative Prompt Engineering, Batch Generation, Hybrid Evaluation.
- Collaborative Prompt Engineering enables real-time co-authoring with version control.
- Batch Generation allows configurable output across prompts, models, and data.
- Hybrid Evaluation uses human and LLM evaluators with live agreement metrics.
- New prompts and models auto-available for batch generation.
- Validated with six domain experts and three developers in online counseling.
Entities
—