SliceGraph: Mapping Reasoning Process Isomers in Multi-Run Chain-of-Thought
The innovative approach known as SliceGraph creates a graphical representation of the geometric framework involved in multi-run chain-of-thought reasoning by utilizing intermediate computation states. It employs mutual-kNN based on Jaccard similarity derived from sparse activation keys to pinpoint shared reasoning-state units (biconnected components) and process families (strategy-coherent route units). In 85.5% of the 954 problem-model cells examined, correct chain-of-thoughts yielding the same answer were divided into various process families; for cells with a minimum of two such runs, an average of 76.6% of run pairs were cross-family. This indicates that while correct paths may differ in their processes, they can still arrive at the same conclusion, challenging the notion of collapsed final-answer aggregates. The research involved three 4B/8B models tested against math and science benchmarks.
Key facts
- SliceGraph is a post-hoc problem-model-cell graph built by mutual-kNN over sparse activation-key Jaccard similarity between CoT slices.
- It treats the graph as a measurement object for process geometry, not a decoding program.
- Three primary 4B/8B models were used on math and science benchmarks.
- Blinded annotation supports SliceGraph biconnected components as shared reasoning-state units.
- Process families are within-family strategy-coherent route units.
- In 85.5% of 954 problem-model cells, correct CoTs sharing the same normalized answer split into multiple process families.
- Among cells with at least two such runs, 76.6% of run pairs are cross-family on average.
- The paper calls these same-answer, family-divergent correct trajectories 'process isomers'.
Entities
Institutions
- arXiv