CYK Algorithm Injected into Neural Network Architecture
Researchers propose CYKNN, a recurrent neural network that directly encodes the Cocke-Younger-Kasami (CYK) algorithm for context-free grammar parsing. The architecture uses trainable matrix-vector operations to implement the algorithm, outperforming LLMs with over 20 billion parameters in in-context learning settings and smaller Qwen models fine-tuned with LoRA on a simple grammar with four variations. This work demonstrates a novel neuro-symbolic approach by injecting algorithmic structure into neural networks.
Key facts
- CYKNN is a recurrent neural network architecture encoding the CYK algorithm.
- The CYK algorithm parses context-free grammars in Chomsky Normal Form.
- CYKNN uses trainable matrix-vector operations.
- Experiments used a simple grammar with four variations.
- CYKNN outperformed LLMs with over 20 billion parameters in in-context learning.
- CYKNN outperformed smaller Qwen models fine-tuned with LoRA.
- The approach injects algorithms directly into neural network architecture.
- The paper is on arXiv under Computer Science > Computation and Language.
Entities
Institutions
- arXiv