Open Ontologies: LLM-Driven Ontology Engineering with Stable Matching
Open Ontologies, a Rust-based open-source ontology engineering framework, combines LLM-driven development with formal OWL reasoning and ontology alignment through the Model Context Protocol. A significant discovery indicates that stable 1-to-1 matching is crucial for alignment quality: on the OAEI Anatomy track, it records an F1 score of 0.832 (P = 0.963, R = 0.733), rivaling top-tier systems and surpassing all others in precision. Testing across five weight configurations reveals that signal weights are insignificant when stable matching is utilized (F1 fluctuates by less than 0.004), whereas omitting stable matching reduces F1 to 0.728. On the Conference track, the same approach yields an F1 score of 0.438. In tool-augmented ontology interaction, an LLM analyzing a raw OWL file (F1 = 0.323) underperforms compared to the same LLM without a file (F1 = 0.431), while structured MCP tool access results in F1 = 0.
Key facts
- Open Ontologies is an open-source ontology engineering system implemented in Rust.
- It integrates LLM-driven construction with formal OWL reasoning and ontology alignment via the Model Context Protocol.
- Stable 1-to-1 matching is the dominant factor in ontology alignment quality.
- On the OAEI Anatomy track, it achieves F1 = 0.832 (P = 0.963, R = 0.733).
- It exceeds all state-of-the-art systems in precision on the Anatomy track.
- Ablation shows signal weights are irrelevant when stable matching is applied (F1 varies by less than 0.004).
- Removing stable matching drops F1 to 0.728 on the Anatomy track.
- On the Conference track, the same method achieves F1 = 0.438.
- An LLM reading a raw OWL file (F1 = 0.323) performs worse than the same LLM with no file (F1 = 0.431).
- Structured MCP tool access achieves F1 = 0.
Entities
—