Human-Centered Learning Mechanics framework for entropy-regulated representation learning
A novel theoretical framework known as Human-Centered Learning Mechanics (HCLM) presents a dynamic and information-theoretic perspective on deep learning, viewing training as an open and regulated system instead of a closed optimization task. This framework tackles practical issues such as uncertainty, limitations in resources, shifts in distribution, risks in downstream decisions, and human feedback. A key element of HCLM is the notion of effective entropy, which posits that entropy regularization is advantageous only if the selected entropy surrogate produces a non-degenerate information force along the optimization path. If not, entropy terms may lead to weak, unstable, or misaligned gradients, resulting in dynamics that revert to standard loss minimization. The research introduces manageable geometric entropy surrogates, such as variance-based and log-determinant covariance measures. The paper can be found on arXiv with reference 2605.22940.
Key facts
- HCLM is a dynamical and information-theoretic framework for open and controlled learning systems.
- The framework treats deep learning as a dynamical process in parameter space.
- Entropy regularization is useful only when the entropy surrogate generates a non-degenerate information force.
- Weak entropy surrogates may cause dynamics to collapse toward ordinary loss minimization.
- Effective entropy is a key concept introduced in the paper.
- Tractable geometric entropy surrogates include variance-based and log-determinant covariance measures.
- The paper addresses real-world challenges: uncertainty, resource constraints, distribution shift, decision risks, and human feedback.
- The paper is available on arXiv with ID 2605.22940.
Entities
Institutions
- arXiv