A Generic Coordinate Descent Framework for Learning from Implicit Feedback

Abstract: In recent years, interest in recommender research has shifted from explicit feedback towards implicit feedback data. A diversity of complex models has been proposed for a wide variety of applications. Despite this, learning from implicit feedback is still computationally challenging. So far, most work relies on stochastic gradient descent (SGD) solvers which are easy to derive, but in practice challenging to apply, especially for tasks with many items. For the simple matrix factorization model, an efficient coordinate descent (CD) solver has been previously proposed. However, efficient CD approaches have not been derived for more complex models. In this paper, we provide a new framework for deriving efficient CD algorithms for complex recommender models. We identify and introduce the property of k-separable models. We show that k-separability is a sufficient property to allow efficient optimization of implicit recommender problems with CD. We illustrate this framework on a variety of state-of-the-art models including factorization machines and Tucker decomposition. To summarize, our work provides the theory and building blocks to derive efficient implicit CD algorithms for complex recommender models.

• The paper introduces the k-separability property, enabling efficient computation of implicit regularizers without iterating over all context-item pairs.
• It develops a generic coordinate descent framework applicable to models such as Matrix Factorization, Factorization Machines, and tensor factorizations.
• Numerical results demonstrate substantial runtime improvements in handling large-scale implicit feedback, paving the way for more efficient recommender systems.

A Generic Coordinate Descent Framework for Learning from Implicit Feedback

The paper presents a novel framework for efficient optimization of complex recommender systems from implicit feedback using coordinate descent (CD). The authors address inherent challenges in learning from implicit feedback, highlighting limitations in current stochastic gradient descent (SGD) methods, especially for large item spaces. The proposed framework expands the applicability of CD beyond simple matrix factorization models, previously the main beneficiary of efficient CD solvers.

Key Contributions

1. $k$-separability Property: The paper introduces a new concept, '$k$-separability,' essential for optimizing implicit feedback problems. A $k$-separable model allows for efficient computation of the implicit regularizer without iterating over all context-item pairs. This property reduces the computational complexity from the order of the product of the number of contexts and items to a sum of their orders.
2. Generic CD Framework: The authors develop a general framework for deriving CD algorithms for $k$-separable models. This framework is versatile and applicable to numerous state-of-the-art models, ensuring optimization efficiency while handling implicit feedback data.
3. Applications to State-of-the-Art Models: The framework's efficacy is illustrated through its application to prominent models such as Matrix Factorization (MF), Feature-based Factorization Machines (FM), and Tensor Factorizations like PARAFAC and Tucker Decomposition. For each of these models, the authors derive efficient implicit CD algorithms demonstrating both applicability and efficiency.

Numerical Results and Implications

The numerical results demonstrate substantial efficiency improvements in solving implicit feedback problems using the proposed framework. The framework allows the efficient computation of gradients and updates for $k$-separable models - drastically reducing runtime in large-scale applications. The results offer practical insights into implementing efficient recommender systems in industrial-scale environments, where processing implicit feedback is often computationally prohibitive.

Implications for Future Research

The introduction of $k$-separability opens new research avenues in model design for recommender systems. Models traditionally thought non-amenable to CD approaches due to their complexity might be revisited and potentially optimized within this framework. The paper also suggests that the contrast between BPR and CD optimization strategies might shift in favor of CD under specific scenarios, particularly given the framework's generic nature.

Conclusion

This paper contributes significantly to the domain of recommendation systems by broadening the capacity to learn from implicit feedback through an innovative framework. By providing both the conceptual foundation and practical algorithms, it extends the frontier of what's feasible in recommender system research and application. Consequently, researchers and practitioners now have a robust tool to explore more sophisticated models, ultimately enhancing system performance in user experience and engagement through targeted product recommendations.