Chat2Workflow Benchmark Introduced for Generating Visual Workflows with Natural Language
A new benchmark called Chat2Workflow has been developed by researchers to assess the capability of large language models in automating the generation of executable visual workflows from natural language inputs. Traditionally, the creation of these workflows—widely used in industrial settings for their reliability and control—requires manual effort. Developers face the challenge of designing workflows, crafting prompts for each phase, and adjusting logic as needed, leading to a process that is expensive, lengthy, and error-prone. This benchmark is derived from an extensive array of real-world business workflows, allowing generated workflows to be easily adapted for practical platforms. To tackle frequent execution issues, the researchers suggest a strong agentic framework. This research, found in arXiv:2604.19667v1, seeks to alleviate the manual workload by investigating automation via natural language.
Key facts
- Chat2Workflow is a benchmark for generating executable visual workflows from natural language.
- Executable visual workflows are a mainstream paradigm in real-world industrial deployments.
- Current workflows are constructed manually, making development costly, time-consuming, and error-prone.
- The benchmark is built from a large collection of real-world business workflows.
- Generated workflows can be transformed and directly deployed to practical workflow platforms.
- A robust agentic framework is proposed to mitigate recurrent execution errors.
- The study explores whether large language models can automate the multi-round interaction process.
- The research is documented in arXiv:2604.19667v1.
Entities
—