DynaTab: Dynamic Feature Ordering for High-Dimensional Tabular Data
DynaTab, an innovative deep learning technique, tackles the issue of high-dimensional tabular data that lacks a natural order of features. This method employs a dynamic feature ordering system inspired by the concept of neural rewiring. A lightweight criterion assesses the advantages of feature permutation for a dataset by measuring its intrinsic complexity. Features are reordered using a neural rewiring algorithm and are processed through a compact combination of positional embeddings, importance-based gating, and masked attention layers. The model is trained end-to-end, incorporating custom dynamic feature ordering and dispersion losses. It demonstrates statistically significant improvements, especially with high-dimensional datasets, and was evaluated against 45 advanced baselines across 36 real-world tabular datasets.
Key facts
- DynaTab is a dynamic feature ordering-enabled architecture for high-dimensional tabular data.
- It is inspired by neural rewiring.
- A lightweight criterion predicts when feature permutation benefits a dataset.
- DynaTab uses a neural rewiring algorithm to dynamically reorder features.
- It combines positional embeddings, importance-based gating, and masked attention layers.
- The model is trained end-to-end with dynamic feature ordering and dispersion losses.
- It achieves statistically significant gains on high-dimensional datasets.
- Benchmarked against 45 baselines across 36 real-world tabular datasets.
Entities
—