SIGIR 2025 - Padova, Italy
Are we really making much progress? An analysis of intent-aware recommendation models
Additional Information: Source Code, Optimized Hyper-Parameters and Additional Result Tables
Lately, we have observed a growing interest in intent-aware recommender systems (IARS). The promise of such systems is that they are capable of generating better recommendations by predicting and considering the underlying motivations and short-term goals of consumers. From a technical perspective, various sophisticated neural models were recently proposed in this emerging and promising area. In the broader context of complex neural recommendation models, a growing number of research works unfortunately indicates that the true benefits of such models may be limited in reality, e.g., because the reported improvements were obtained through comparisons with untuned or weak baselines. In this work, we investigate if recent research in IARS is similarly affected by such problems. Specifically, we reproduced five contemporary IARS models that were published in top-level outlets, and we benchmarked them against a number of traditional non-neural recommendation models. Worryingly, we find that all examined IARS approaches are almost consistently outperformed by traditional models. These results point to enduring methodological issues in the field and to a pressing need for more rigorous scholarly practices.
Source Code and Datasets
The full source code of the framework and utilized dataset can be found here:
https://github.com/Faisalse/IntentAware_topn