PAC-Bayes Beyond Parameter Space: Behavioral Equivalence and Complexity Decomposition

A recent theoretical study published on arXiv (2608.11465) introduces a framework aimed at breaking down PAC-Bayes generalization bounds beyond the parameter space. The researchers contend that the predictive risk is solely influenced by a hypothesis's predictive behavior rather than its internal realization. They establish behavioral equivalence through a measurable behavior map and apply measure disintegration to separate probability measures into distributions over behaviors and conditional distributions for equivalent configurations. This results in a precise structural decomposition of traditional PAC-Bayes complexity, differentiating uncertainty in predictive behavior from variation in behaviorally equivalent realizations. The paper, which focuses on over-parameterized systems with multiple configurations yielding the same predictions, offers a more nuanced understanding of generalization. It is classified as a cross announcement and can be accessed via the provided URL.