LeGo-Code Research Explores Modular Curriculum Learning for Complex Code Generation in Text-to-SQL
A recent study explores the potential of curriculum learning to improve the efficacy of code-focused large language models in Text-to-SQL tasks. This application is crucial for enabling non-experts to interact with relational databases using natural language. Despite progress, leading models struggle with intricate logic, such as deeply nested queries with multiple joins and conditions. Additionally, real-world database schemas that are disorganized or poorly designed pose further challenges. The research utilizes benchmarks like Spider and BIRD to refine models through various curriculum strategies. Findings indicate that a simplistic curriculum method, which organizes training samples by complexity within a single epoch, does not surpass conventional fine-tuning due to issues like catastrophic forgetting. This research seeks to enhance the ability to generate complex code, addressing the shortcomings of current LLMs in handling sophisticated database queries.
Key facts
- Research explores curriculum learning for code-oriented LLMs on Text-to-SQL tasks
- Text-to-SQL enables natural language interaction with relational databases
- Models struggle with complex logic like nested statements and multiple joins
- Real-world database schemas can be noisy or poorly structured
- Benchmarks include Spider and BIRD
- Naive curriculum ordering by complexity fails due to catastrophic forgetting
- Study investigates modular curriculum learning strategies
- Paper is available on arXiv with identifier 2604.18254v1
Entities
Institutions
- arXiv