Neural Network Verification Errors Grow Exponentially with Depth in Convex Relaxations

A study published on arXiv (ID: 2604.18728v1) examines the trade-offs in neural network verification systems that use convex relaxations to improve performance. These systems often represent neural networks as constraint programs, with sound and complete representations requiring integer constraints to simulate activations. Recent approaches convexly relax these integer constraints, which enhances computational efficiency but sacrifices soundness by considering outputs unreachable by the original network. The research investigates both qualitative and quantitative aspects of the worst-case divergence between original networks and their convex relaxations. The relaxation space forms a lattice where the top element corresponds to a full relaxation with every neuron linearized, while the bottom element represents the original network. Analytical upper and lower bounds for the ℓ∞-distance between fully relaxed and original outputs reveal that this distance grows exponentially with network depth and linearly with input radius. This exponential growth highlights significant verification errors in deep networks when using convex relaxations, impacting the reliability of neural network verification methods that prioritize performance over accuracy.