HYDRA: A Hyperbolic Extension of Kolmogorov-Arnold Networks

A new architecture named HYDRA (Hyperbolic Dynamic Representation Architecture) has been developed by researchers as a parameter-efficient enhancement of Kolmogorov-Arnold Networks (KANs) to minimize parameter redundancy. While KANs enhance the approximation of nonlinear functions by substituting scalar weights with learnable univariate functions, the assignment of an independent function for each connection results in significant redundancy, hindering scalability and efficiency. HYDRA integrates spline-based functional learning with representations within the Poincaré ball, converting vector-valued inputs into a constrained hyperbolic latent space. It executes KAN-style updates in tangent space and utilizes a low-rank prototype block to disseminate functional transformations across hidden dimensions. The structured radial coordinate from hyperbolic representations aids interpretation, and radius control enhances training stability by avoiding boundary complications. The research paper can be found on arXiv with the identifier 2608.12194.