Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 174 tok/s
Gemini 2.5 Pro 51 tok/s Pro
GPT-5 Medium 38 tok/s Pro
GPT-5 High 34 tok/s Pro
GPT-4o 91 tok/s Pro
Kimi K2 205 tok/s Pro
GPT OSS 120B 438 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

Cognitive Evolutionary Learning to Select Feature Interactions for Recommender Systems (2405.18708v1)

Published 29 May 2024 in cs.AI, cs.IR, and cs.NE

Abstract: Feature interaction selection is a fundamental problem in commercial recommender systems. Most approaches equally enumerate all features and interactions by the same pre-defined operation under expert guidance. Their recommendation is unsatisfactory sometimes due to the following issues: (1)~They cannot ensure the learning abilities of models because their architectures are poorly adaptable to tasks and data; (2)~Useless features and interactions can bring unnecessary noise and complicate the training process. In this paper, we aim to adaptively evolve the model to select appropriate operations, features, and interactions under task guidance. Inspired by the evolution and functioning of natural organisms, we propose a novel \textsl{Cognitive EvoLutionary Learning (CELL)} framework, where cognitive ability refers to a property of organisms that allows them to react and survive in diverse environments. It consists of three stages, i.e., DNA search, genome search, and model functioning. Specifically, if we regard the relationship between models and tasks as the relationship between organisms and natural environments, interactions of feature pairs can be analogous to double-stranded DNA, of which relevant features and interactions can be analogous to genomes. Along this line, we diagnose the fitness of the model on operations, features, and interactions to simulate the survival rates of organisms for natural selection. We show that CELL can adaptively evolve into different models for different tasks and data, which enables practitioners to access off-the-shelf models. Extensive experiments on four real-world datasets demonstrate that CELL significantly outperforms state-of-the-art baselines. Also, we conduct synthetic experiments to ascertain that CELL can consistently discover the pre-defined interaction patterns for feature pairs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. W. Guo et al., “Dual graph enhanced embedding neural network for ctr prediction,” in KDD, 2021, pp. 496–504.
  2. P. Li et al., “Dual attentive sequential learning for cross-domain click-through rate prediction,” in KDD, 2021, pp. 3172–3180.
  3. S.-T. Shi, W. Zheng, J. Tang, Q.-G. Chen, Y. Hu, J. Zhu, and M. Li, “Deep time-stream framework for click-through rate prediction by tracking interest evolution,” in AAAI, 2020, pp. 5726–5733.
  4. Z. Lyu, Y. Dong, C. Huo, and W. Ren, “Deep match to rank model for personalized click-through rate prediction,” in AAAI, vol. 34, no. 01, 2020, pp. 156–163.
  5. Q. Shao et al., “Toward intelligent financial advisors for identifying potential clients: A multitask perspective,” Big Data Mining and Analytics, vol. 5, no. 1, pp. 64–78, 2021.
  6. R. Yu et al., “Xcrossnet: Feature structure-oriented learning for click-through rate prediction,” in PAKDD, 2021, pp. 436–447.
  7. M. Dash and H. Liu, “Feature selection for classification,” Intelligent data analysis, vol. 1, no. 1-4, pp. 131–156, 1997.
  8. X. Li, Y. Wang, and R. Ruiz, “A survey on sparse learning models for feature selection,” IEEE transactions on Cybernetics, pp. 1642–1660, 2022.
  9. S. Rendle, “Factorization machines,” in ICDM.   IEEE, 2010, pp. 995–1000.
  10. X. He and T.-S. Chua, “Neural factorization machines for sparse predictive analytics,” in SIGIR, 2017, pp. 355–364.
  11. J. Xiao et al., “Attentional factorization machines: learning the weight of feature interactions via attention networks,” in IJCAI, 2017, pp. 3119–3125.
  12. H. Guo et al., “Deepfm: A factorization-machine based neural network for CTR prediction,” in IJCAI, 2017, pp. 1725–1731.
  13. B. Xue, M. Zhang, W. N. Browne, and X. Yao, “A survey on evolutionary computation approaches to feature selection,” IEEE Transactions on Evolutionary Computation, vol. 20, no. 4, pp. 606–626, 2015.
  14. R. Yu, X. Xu, Y. Ye, Q. Liu, and E. Chen, “Cognitive evolutionary search to select feature interactions for click-through rate prediction,” in KDD, 2023, pp. 3151–3161.
  15. A. Telikani, A. Tahmassebi, W. Banzhaf, and A. H. Gandomi, “Evolutionary machine learning: A survey,” ACM Computing Surveys (CSUR), vol. 54, no. 8, pp. 1–35, 2021.
  16. B. Tran, B. Xue, and M. Zhang, “A new representation in pso for discretization-based feature selection,” IEEE Transactions on Cybernetics, vol. 48, no. 6, pp. 1733–1746, 2017.
  17. F. Cheng, F. Chu, Y. Xu, and L. Zhang, “A steering-matrix-based multiobjective evolutionary algorithm for high-dimensional feature selection,” IEEE Transactions on Cybernetics, vol. 52, no. 9, pp. 9695–9708, 2022.
  18. G. Zhou, N. Mou, Y. Fan, Q. Pi, W. Bian, C. Zhou, X. Zhu, and K. Gai, “Deep interest evolution network for click-through rate prediction,” in AAAI, 2019, pp. 5941–5948.
  19. Y. Xie et al., “Fives: Feature interaction via edge search for large-scale tabular data,” in KDD, 2021, pp. 3795–3805.
  20. H. Guo, B. Chen et al., “An embedding learning framework for numerical features in ctr prediction,” in KDD, 2021, pp. 2910–2918.
  21. Y. Juan, Y. Zhuang, W.-S. Chin, and C.-J. Lin, “Field-aware factorization machines for ctr prediction,” in RecSys, 2016, pp. 43–50.
  22. W. Zhang, T. Du, and J. Wang, “Deep learning over multi-field categorical data,” in European Conference on Information Retrieval.   Springer, 2016, pp. 45–57.
  23. Y. Qu et al., “Product-based neural networks for user response prediction,” in ICDM.   IEEE, 2016, pp. 1149–1154.
  24. Y. Qu, B. Fang et al., “Product-based neural networks for user response prediction over multi-field categorical data,” ACM Transactions on Information Systems (TOIS), vol. 37, no. 1, pp. 1–35, 2018.
  25. J. Lian et al., “xdeepfm: Combining explicit and implicit feature interactions for recommender systems,” in KDD, 2018, pp. 1754–1763.
  26. H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked et al., “Wide & deep learning for recommender systems,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016, pp. 7–10.
  27. R. Wang, B. Fu, G. Fu, and M. Wang, “Deep & cross network for ad click predictions,” in ADKDD, 2017, pp. 1–7.
  28. C.-F. Juang, C.-Y. Chou, and C.-T. Lin, “Navigation of a fuzzy-controlled wheeled robot through the combination of expert knowledge and data-driven multiobjective evolutionary learning,” IEEE Transactions on Cybernetics, vol. 52, no. 8, pp. 7388–7401, 2022.
  29. X. Zheng, W. Wu, W. Deng, C. Yang, and K. Huang, “Reconstruction of tree network via evolutionary game data analysis,” IEEE Transactions on Cybernetics, vol. 52, no. 7, pp. 6083–6094, 2022.
  30. K. Chen, B. Xue, M. Zhang, and F. Zhou, “An evolutionary multitasking-based feature selection method for high-dimensional classification,” IEEE Transactions on Cybernetics, vol. 52, no. 7, pp. 7172–7186, 2022.
  31. X.-F. Song, Y. Zhang, D.-W. Gong, and X.-Z. Gao, “A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high-dimensional data,” IEEE Transactions on Cybernetics, vol. 52, no. 9, pp. 9573–9586, 2022.
  32. F. Khawar et al., “Autofeature: Searching for feature interactions and their architectures for click-through rate prediction,” in CIKM, 2020, pp. 625–634.
  33. Q. Song, D. Cheng et al., “Towards automated neural interaction discovery for click-through rate prediction,” in KDD, 2020, pp. 945–955.
  34. B. Liu et al., “Autogroup: Automatic feature grouping for modelling explicit high-order feature interactions in ctr prediction,” in SIGIR, 2020, pp. 199–208.
  35. L. V. DiBello, L. A. Roussos, and W. Stout, “A review of cognitively diagnostic assessment and a summary of psychometric models,” Handbook of statistics, vol. 26, pp. 979–1030, 2006.
  36. J. De La Torre, “The generalized dina model framework,” Psychometrika, vol. 76, no. 2, pp. 179–199, 2011.
  37. S. Tong et al., “Item response ranking for cognitive diagnosis,” in IJCAI, 2021, pp. 1750–1756.
  38. L. Yue et al., “Circumstances enhanced criminal court view generation,” in SIGIR, 2021, pp. 1855–1859.
  39. Y. Gu et al., “Neuralac: Learning cooperation and competition effects for match outcome prediction,” in AAAI, vol. 35, no. 5, 2021, pp. 4072–4080.
  40. W. Tao, Y. Li, L. Li, Z. Chen, H. Wen, P. Chen, T. Liang, and Q. Lu, “Sminet: State-aware multi-aspect interests representation network for cold-start users recommendation,” in AAAI, vol. 36, no. 8, 2022, pp. 8476–8484.
  41. H. Liu, K. Simonyan, and Y. Yang, “Darts: Differentiable architecture search,” in ICLR, 2019.
  42. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in ICML.   PMLR, 2017, pp. 1126–1135.
  43. J. Luketina, M. Berglund, K. Greff, and T. Raiko, “Scalable gradient-based tuning of continuous regularization hyperparameters,” in ICML.   PMLR, 2016, pp. 2952–2960.
  44. L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein, “Unrolled generative adversarial networks,” arXiv preprint arXiv:1611.02163, 2016.
  45. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  46. L. Xiao, “Dual averaging method for regularized stochastic learning and online optimization,” NeurIPS, vol. 22, pp. 2116–2124, 2009.
  47. S.-K. Chao and G. Cheng, “A generalization of regularized dual averaging and its dynamics,” arXiv preprint arXiv:1909.10072, 2019.
  48. B. Liu et al., “Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction,” in KDD, 2020, pp. 2636–2645.
  49. M. Richardson et al., “Predicting clicks: estimating the click-through rate for new ads,” in WWW, 2007, pp. 521–530.
  50. W. Song et al., “Autoint: Automatic feature interaction learning via self-attentive neural networks,” in CIKM.   ACM, 2019, pp. 1161–1170.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 tweet and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial