Algebraic Ontology Projection Controls LLM Logical Collapse
A recent study presents Algebraic Ontology Projection (AOP), a technique that maps hidden states from large language models into the Galois Field F2, adhering to Liskov Substitution Principle constraints. By utilizing merely 42 relational pairs as algebraic keys, AOP achieves a remarkable 93.33% zero-shot inclusion accuracy on novel concept pairs with Gemma-2 Instruct and optimized prompts, maintaining a steady 86.67% accuracy across various model families—achieved solely through prompts without any model adjustments. The algebraic framework is notably dependent on layers. Additionally, the authors propose Semantic Crystallisation (SC), a metric that measures F2 constraint satisfaction against a random baseline and forecasts zero-shot accuracy without the need for held-out data. System prompts serve as algebraic boundary conditions; their integration with instruction tuning is essential to avert Late-layer Collapse, a decline in logical reasoning. This research is available on arXiv under ID 2605.12968.
Key facts
- Method called Algebraic Ontology Projection (AOP) projects LLM hidden states into Galois Field F2.
- Uses Liskov Substitution Principle constraints.
- Only 42 relational pairs are used as algebraic keys.
- Achieves up to 93.33% zero-shot inclusion accuracy on unseen concept pairs.
- Tested on Gemma-2 Instruct with optimized prompt.
- Consistent 86.67% accuracy observed across multiple model families.
- No model tuning required; only prompt engineering.
- Algebraic structure is strongly layer-dependent.
- Semantic Crystallisation (SC) metric quantifies F2 constraint satisfaction.
- SC predicts zero-shot accuracy without held-out data.
- System prompts act as algebraic boundary conditions.
- Combination of system prompts and instruction tuning prevents Late-layer Collapse.
- Paper published on arXiv (ID: 2605.12968).
Entities
Institutions
- arXiv