Aftab: Benchmarking CNN Encoders and Advanced Value Functions in Parallelized Q-Networks
A recent preprint available on arXiv (2608.07335) introduces Aftab, a detailed benchmark focused on Convolutional Neural Network (CNN) encoders and sophisticated value functions within the Parallelized Q-Network (PQN) framework. This research thoroughly examines the architectural design space, assessing eight unique CNN architectures for their sample efficiency while adhering to stringent parameter limitations. Additionally, it incorporates the Hadamax encoding method along with advanced Q-learning techniques, such as distributional, ensemble, and dueling heads. Experiments utilize the Atari-57 benchmark. This study highlights the often-overlooked representational capacity and parameter efficiency of visual encoders in buffer-free environments, aiming to enhance the development of effective encoders for PQN and related algorithms.
Key facts
- The paper is a preprint on arXiv with identifier 2608.07335.
- It focuses on the Parallelized Q-Network (PQN) algorithm.
- PQN achieves stable off-policy learning without replay buffers or target networks.
- Eight distinct CNN topologies are designed and evaluated.
- The study optimizes for sample efficiency under strict parameter constraints.
- The Hadamax encoding paradigm is integrated.
- Advanced Q-learning extensions include distributional, ensemble, and dueling heads.
- Experiments are conducted on the Atari-57 benchmark.
Entities
Institutions
- arXiv