ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation
Yupeng Hou, Jianmo Ni, Zhankui He, Noveen Sachdeva, Wang-Cheng Kang, Ed H. Chi, Julian McAuley, Derek Zhiyuan Cheng
2025-02-20
Summary
This paper talks about ActionPiece, a new way to improve how AI recommender systems understand and predict user actions. It's like teaching a computer to read between the lines of what users do online, not just looking at each action separately.
What's the problem?
Current AI recommendation systems look at each user action (like clicking on a product) as a separate thing, without considering how it relates to other actions. This is like trying to understand a story by looking at each word alone, without thinking about how the words fit together. This approach can miss important patterns in user behavior, leading to less accurate recommendations.
What's the solution?
The researchers created ActionPiece, which looks at user actions in context, considering how they relate to each other. It breaks down each action into smaller parts (called features) and then looks at how these features appear together across different user actions. It's like learning to understand phrases and sentences instead of just individual words. ActionPiece also shuffles these features around to help the AI understand them better, kind of like rearranging words in a sentence to see if it still makes sense.
Why it matters?
This matters because it could make AI recommendations much more accurate and useful. Better recommendations can help users find products they like more easily, help businesses sell more effectively, and improve the overall online experience. In tests, ActionPiece improved recommendation accuracy by 6% to nearly 13%, which is a big deal in the world of AI. This could lead to smarter online shopping experiences, better content recommendations on streaming services, and more personalized digital experiences across the internet.
Abstract
Generative recommendation (GR) is an emerging paradigm where user actions are tokenized into discrete token patterns and autoregressively generated as predictions. However, existing GR models tokenize each action independently, assigning the same fixed tokens to identical actions across all sequences without considering contextual relationships. This lack of context-awareness can lead to suboptimal performance, as the same action may hold different meanings depending on its surrounding context. To address this issue, we propose ActionPiece to explicitly incorporate context when tokenizing action sequences. In ActionPiece, each action is represented as a set of item features, which serve as the initial tokens. Given the action sequence corpora, we construct the vocabulary by merging feature patterns as new tokens, based on their co-occurrence frequency both within individual sets and across adjacent sets. Considering the unordered nature of feature sets, we further introduce set permutation regularization, which produces multiple segmentations of action sequences with the same semantics. Experiments on public datasets demonstrate that ActionPiece consistently outperforms existing action tokenization methods, improving NDCG@10 by 6.00% to 12.82%.