EVA-0: Two-Forward-Pass Test-Time Model Evolution
Researchers have rolled out a new framework known as EVA-0, aimed at improving zeroth-order model evolution during testing. It operates within a two forward pass limit, which eliminates the need for backpropagation. This makes it suitable for edge devices, quantized models, specialized accelerators, and even black-box models. EVA-0 addresses three key challenges in zeroth-order optimization: the potential for shortcut solutions, the problem of weight drift, and the difficulty of accurately estimating update directions. To tackle these, it uses scale-invariant loss to prevent shortcuts, anchor-guided optimization to manage weight drift, and sample-wise symmetric two-sided perturbation for better gradient estimation. This research can be found on arXiv under ID 2605.18867.
Key facts
- EVA-0 requires only two forward passes per sample for test-time adaptation.
- It eliminates backpropagation, reducing memory overhead.
- Suitable for edge devices, quantized models, specialized accelerators, and black-box models.
- Addresses shortcut solutions via scale-invariant loss.
- Uses anchor-guided optimization to prevent weight drift.
- Employs sample-wise symmetric two-sided perturbation for gradient estimation.
- Published on arXiv with ID 2605.18867.
- The paper is categorized as a cross submission.
Entities
Institutions
- arXiv