Fractional Optimizers and Fractal Activations: A Unified Study

An empirical study has been released as a new arXiv preprint (2608.14636) that explores how fractional optimization techniques interact with fractal activation functions during neural network training. This research, highlighted as a cross-listing, delves into two distinct paths for enhancement: fractional optimizers that utilize fractional derivatives and memory effects to expand first-order optimization, and fractal activations that offer multi-scale nonlinear representations through self-similar Weierstrass- and Blancmange-type functions. The evaluation of fractional optimizer families occurs on Ackley and Himmelblau benchmark surfaces, both in their standard forms and with added Weierstrass-type perturbations. These optimizers are then tested on feed-forward neural networks employing both traditional and fractal activations across ten classification datasets. The analysis includes comparisons with standard methods, regularization-style optimizers, and various memory-based fractional optimizers. Results indicate that combining fractional optimization with fractal activations can enhance training dynamics and overall performance. The paper can be accessed on arXiv with the identifier 2608.14636.