Partial Channel Dependence via Channel Masks in Transformers for Time Series
A new approach introduces partial channel dependence (PCD) for multivariate time series modeling in Transformers. The method uses channel masks (CMs) integrated into attention matrices via element-wise multiplication. CMs consist of a similarity matrix capturing relationships, enabling dataset-specific refinement of channel dependency. This addresses the limitation of prior attention-based methods that neglect dataset-specific characteristics. The work is published on arXiv (2410.23222v3) and focuses on improving channel dependency capture in foundation models for time series.
Key facts
- Introduces partial channel dependence (PCD) for multivariate time series
- Proposes channel masks (CMs) integrated into Transformer attention matrices
- CMs use element-wise multiplication with attention matrices
- CMs consist of a similarity matrix capturing relationships
- Addresses neglect of dataset-specific characteristics in prior methods
- Published on arXiv with ID 2410.23222v3
- Focuses on improving channel dependency in foundation models
- Applies to multivariate time series domain
Entities
Institutions
- arXiv