Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints
Zilin Kang, Chonghua Liao, Tingqiang Xu, Huazhe Xu
2025-10-10
Summary
This paper introduces a new method called ERA that aims to make AI models more reliable and perform better by controlling how much 'randomness' they exhibit in their outputs.
What's the problem?
AI models, especially powerful ones like large language models, can sometimes be unpredictable. They might give wildly different answers to the same question, or in reinforcement learning, an agent might make erratic decisions. This unpredictability, measured by something called 'entropy', can hurt performance and make the models less trustworthy. The goal is to reduce this harmful randomness without sacrificing the model's ability to learn and adapt.
What's the solution?
The researchers developed ERA, which works by adding special components, called 'activations', to the very end of the model's processing. These activations act like a filter, limiting how much the model can vary its outputs. Essentially, ERA encourages the model to be more confident and consistent in its predictions. They tested this on three different types of AI tasks: answering math problems, controlling a virtual humanoid robot, and classifying images. In each case, ERA improved performance.
Why it matters?
This work is significant because it shows a simple yet effective way to improve the reliability and performance of AI models across various applications. The computational cost of using ERA is relatively low, meaning it can be easily added to existing models. It suggests that controlling the 'randomness' of a model's output is a promising area for future research, potentially leading to more robust and dependable AI systems.
Abstract
We propose ERA, a new paradigm that constrains the sampling entropy above given thresholds by applying specially designed activations to the outputs of models. Our approach demonstrates broad effectiveness across different domains: 1) for large language models(LLMs), boosting the AIME 2025 score for Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning agents, improving performance by more than 30% over strong baselines such as SAC on the challenging HumanoidBench; 3) for image classification, enhancing ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a computational overhead of less than 7%. Our work validates output activation as a powerful tool for entropy control, opening a new direction for designing simpler and more robust algorithms.