PilotWiMAE: Self-Supervised Learning for Wireless Channels
PilotWiMAE is an innovative self-supervised framework designed for channel foundation models, capable of directly handling noisy pilot observations without relying on the unrealistic premise of fully observed channels during deployment. Its encoder employs a factorization of attention across temporal and joint space-frequency dimensions, drawing inspiration from channel physics. This approach significantly diminishes the observation space by as much as two orders of magnitude and reduces latency. The factorized architecture fosters strong representations, achieving a pretraining mask ratio of 99%. Additionally, it incorporates patch-normalized reconstruction to address small-scale fading and utilizes an auxiliary scale loss for large-scale fading, along with an AWGN curriculum to align pilot noise in both pretraining and deployment phases.
Key facts
- PilotWiMAE is a self-supervised framework for channel foundation models.
- It processes noisy pilot observations directly, removing the assumption of fully observed channels.
- The encoder factorizes attention along temporal and joint space-frequency axes.
- Observation space is reduced by up to two orders of magnitude.
- Lower latency is achieved compared to full-CSI methods.
- Pretraining mask ratio of 99% is possible due to factorized design.
- Patch-normalized reconstruction captures small-scale fading.
- Auxiliary scale loss recovers large-scale fading features.
- AWGN curriculum matches pilot noise between pretraining and deployment.
Entities
Institutions
- arXiv