ZAYAN: Self-Supervised Contrastive Transformer for Tabular Remote Sensing Data
ZAYAN is a cutting-edge framework designed for self-supervised learning, specifically tailored for tabular data in remote sensing and environmental studies. It addresses challenges like feature redundancy, limited labeled data, and heterogeneity. By analyzing features rather than samples, it eliminates the need for selecting anchors or class labels, creating a cleaner embedding space. The framework has two parts: ZAYAN-CL, which pretrains feature embeddings using a zero-anchor contrastive method with dynamic changes and masking, and ZAYAN-T, a Transformer that applies these embeddings for classification tasks. This method was tested on eight datasets, including six standard datasets for remote-sensing tabular data and two flood-prediction datasets from satellite and GIS information.
Key facts
- ZAYAN stands for Zero-Anchor dYnamic feAture eNcoding.
- It is a self-supervised contrastive framework for tabular data.
- Contrastive learning is performed at the feature level, not sample level.
- No explicit anchor selection or class labels are required.
- The framework encourages a redundancy-minimized, disentangled embedding space.
- ZAYAN-CL pretrains feature embeddings via zero-anchor contrastive objective with dynamic perturbations and masking.
- ZAYAN-T is a Transformer that conditions on these embeddings for classification.
- Evaluated on eight datasets: six remote-sensing tabular benchmarks and two flood-prediction tables.
Entities
Institutions
- arXiv