Uncertainty's dual effect on exploration: volatility vs stochasticity

A recent study published on arXiv (2605.19215) indicates that various uncertainty types influence exploration in adaptive decision-making in contrasting ways. Specifically, while both volatility (the gradual shift of latent reward states) and stochasticity (inconsistent observations) heighten posterior uncertainty, volatility actually boosts optimal exploration, whereas stochasticity diminishes it. The authors expand the Gittins index framework to accommodate Gaussian state-space bandits with latent dynamics, introducing CAUSE (Cause-Aware Uncertainty-Sensitive Exploration), a closed-form exploration bonus derived from control-as-inference that maintains similar monotonic properties. This research challenges previous beliefs that all forms of uncertainty uniformly encourage exploration, underscoring the importance of distinguishing between different uncertainty types in both biological and artificial intelligence contexts.