Unifying Decision Trees and Diffusion Models via Global Trajectory Score Matching
A recent publication on arXiv integrates decision trees with diffusion models by demonstrating a mathematical link between hierarchical decision trees and diffusion processes in their limiting cases. This integration uncovers a common optimization principle known as Global Trajectory Score Matching (GTSM), for which an idealized version of gradient boosting proves asymptotically optimal. The paper introduces two practical implementations: TreeFlow, which offers competitive generation quality for tabular data with enhanced fidelity and a 2× increase in computational efficiency, and DSMTree, an innovative distillation technique that conveys hierarchical decision logic into neural networks, achieving teacher performance within 2% across various benchmarks. This work falls under Computer Science > Machine Learning and was submitted on May 1, 2025.
Key facts
- Decision trees and diffusion models are unified via a crisp mathematical correspondence.
- The unification reveals a shared optimization principle: Global Trajectory Score Matching (GTSM).
- Gradient boosting (in an idealized version) is asymptotically optimal for GTSM.
- TreeFlow achieves competitive generation quality on tabular data with higher fidelity and 2× speedup.
- DSMTree is a distillation method that transfers hierarchical decision logic into neural networks.
- DSMTree matches teacher performance within 2% on many benchmarks.
- The paper is classified under Computer Science > Machine Learning.
- The paper was submitted on May 1, 2025.
Entities
Institutions
- arXiv