A Best-of-Both Approach to Improve Match Predictions and Reciprocal Recommendations for Job Search

Abstract: Matching users with mutual preferences is a critical aspect of services driven by reciprocal recommendations, such as job search. To produce recommendations in such scenarios, one can predict match probabilities and construct rankings based on these predictions. However, this direct match prediction approach often underperforms due to the extreme sparsity of match labels. Therefore, most existing methods predict preferences separately for each direction (e.g., job seeker to employer and employer to job seeker) and then aggregate the predictions to generate overall matching scores and produce recommendations. However, this typical approach often leads to practical issues, such as biased error propagation between the two models. This paper introduces and demonstrates a novel and practical solution to improve reciprocal recommendations in production by leveraging pseudo-match scores. Specifically, our approach generates dense and more directly relevant pseudo-match scores by combining the true match labels, which are accurate but sparse, with relatively inaccurate but dense match predictions. We then train a meta-model to output the final match predictions by minimizing the prediction loss against the pseudo-match scores. Our method can be seen as a best-of-both (BoB) approach, as it combines the high-level ideas of both direct match prediction and the two separate models approach. It also allows for user-specific weights to construct personalized pseudo-match scores, achieving even better matching performance through appropriate tuning of the weights. Offline experiments on real-world job search data demonstrate the superior performance of our BoB method, particularly with personalized pseudo-match scores, compared to existing approaches in terms of finding potential matches.

• The paper introduces the Best-of-Both method, which integrates sparse true labels with dense match predictions to generate pseudo-match scores.
• It employs a meta-model with personalized weighting to optimize match predictions and boost NDCG@10 performance across diverse user segments.
• Experimental results on real-world job search datasets demonstrate enhanced matching accuracy, particularly benefiting less active users.

A Best-of-Both Approach to Improve Match Predictions and Reciprocal Recommendations for Job Search

This paper introduces an innovative method designed to enhance the efficacy of reciprocal recommendations in job search systems. Reciprocal recommendation systems are critical in applications where mutual user interest is crucial, such as job search platforms, which require recommendations of job seekers to employers and vice versa. Traditional methods often deploy separate preference predictions for each user type and then aggregate these predictions to generate recommendations. However, this predict-then-aggregate approach can propagate errors from each model, limiting accuracy and performance.

The proposed "Best-of-Both" (BoB) approach provides a novel solution by integrating sparse true match labels with dense match predictions to create pseudo-match scores. True match labels are highly accurate but sparse, whereas the match predictions, while denser, are not as accurate. The pivotal innovation of the BoB method lies in its ability to blend these two data sources using weighted averages, thereby crafting more relevant pseudo-match scores. Subsequently, a meta-model is trained to optimize match prediction by minimizing prediction loss against these pseudo-match scores.

The BoB approach stands out because it effectively combines the direct accuracy strengths of true match labels with the density advantages of match predictions. Also, it introduces a mechanism for user-specific weighting, allowing personalized tuning of pseudo-match scores. This tailoring can distinguish patterns among diverse user segments, enabling finer recommendation granularity.

Experimental results on a real-world job search dataset highlight the potential of the BoB approach. Offline tests demonstrated that the BoB approach outperformed existing methods in finding potential matches, especially for smaller user segments such as less active users. These findings are supported by numerical outcomes indicating higher NDCG@10 scores — a metric for ranking quality. The study finds that the optimal balance between true labels and predictions varies across user segments, indicating the value of personalized weighting.

The practical implications of this research are manifold. In job matching, improving the precision of reciprocal recommendations directly enhances user satisfaction and platform engagement. Theoretically, the introduction of pseudo-match scores that leverage combined data streams may inspire further research into hybrid models that address similar sparsity-dense tradeoffs in other domains.

Future research could explore extending the BoB approach to other reciprocal recommendation applications, such as online dating. Additionally, further exploration into adaptive weighting strategies might yield more adaptive models that continuously refine predictions based on evolving user behavior and feedback. As AI and machine learning techniques progress, integrating sophisticated models with this approach could further refine the accuracy and relevance of recommendations in reciprocal systems.