Don't Waste It: Guiding Generative Recommenders with Structured Human Priors via Multi-head Decoding
Yunkai Zhang, Qiang Zhang, Feng, Lin, Ruizhong Qiu, Hanchao Yu, Jason Liu, Yinglong Xia, Zhuoran Yu, Zeyu Zheng, Diji Yang
2025-11-17
Summary
This paper focuses on improving recommendation systems, not just by making them accurate, but also by making them more diverse, surprising, and tailored to individual users. It's about making sure people stay happy with recommendations over the long haul.
What's the problem?
Currently, recommendation systems often use knowledge experts have about the items being recommended – things like categories or popular times to use them – but this knowledge is added *after* the main recommendation model is built. This is a problem because newer, more powerful recommendation systems are 'end-to-end,' meaning they learn everything at once, and don't easily incorporate this valuable expert knowledge. Also, many attempts to improve diversity or personalization throw away this expert knowledge and let the system learn everything from scratch, which isn't ideal.
What's the solution?
The researchers created a new method that smoothly integrates expert knowledge directly into the training process of these advanced recommendation systems. They use small 'adapter heads' – think of them as little add-ons – that guide the system to understand what users want in a way that makes sense to humans, like whether they're looking for something for a specific activity or a long-term interest. They also figured out a way to combine different types of expert knowledge to make the recommendations even better.
Why it matters?
This work is important because it shows how to build recommendation systems that are both accurate *and* satisfying in other ways, like offering variety and personalization. It also allows these systems to take advantage of the years of experience and understanding that experts have about the items being recommended, and it helps them work better with larger amounts of data and more complex models.
Abstract
Optimizing recommender systems for objectives beyond accuracy, such as diversity, novelty, and personalization, is crucial for long-term user satisfaction. To this end, industrial practitioners have accumulated vast amounts of structured domain knowledge, which we term human priors (e.g., item taxonomies, temporal patterns). This knowledge is typically applied through post-hoc adjustments during ranking or post-ranking. However, this approach remains decoupled from the core model learning, which is particularly undesirable as the industry shifts to end-to-end generative recommendation foundation models. On the other hand, many methods targeting these beyond-accuracy objectives often require architecture-specific modifications and discard these valuable human priors by learning user intent in a fully unsupervised manner. Instead of discarding the human priors accumulated over years of practice, we introduce a backbone-agnostic framework that seamlessly integrates these human priors directly into the end-to-end training of generative recommenders. With lightweight, prior-conditioned adapter heads inspired by efficient LLM decoding strategies, our approach guides the model to disentangle user intent along human-understandable axes (e.g., interaction types, long- vs. short-term interests). We also introduce a hierarchical composition strategy for modeling complex interactions across different prior types. Extensive experiments on three large-scale datasets demonstrate that our method significantly enhances both accuracy and beyond-accuracy objectives. We also show that human priors allow the backbone model to more effectively leverage longer context lengths and larger model sizes.