Learn to Match: Two-Sided Matching with Temporally Extended Feedback

Abstract: Two-sided matching markets often involve information that unfolds over time through interviews, repeated interaction, learning, and separation. Existing matching models typically reduce this process to immediate sub-Gaussian feedback about fixed preferences, missing settings where payoff-relevant information is revealed gradually and changes future matching decisions. We introduce a framework with temporally extended feedback, that formulates two-sided matching as a partially observable Markov game with costly pre-match screening, noisy post-match observations, evolving latent profiles, and endogenous continuation or dissolution. We instantiate this framework in Learn2Match, a multi-agent reinforcement-learning benchmark for dynamic matching markets. Learn2Match supports decentralized decision making over whom to interview, whom to match with, and when to dissolve a match, while evaluating policies using regret, social welfare, and an information-friction loss that measures the welfare gap caused by incomplete revelation of latent preferences. We find that independent PPO achieves higher cumulative social welfare and lower cumulative regret than the bandit-style CA-ETC baseline under temporally extended feedback, demonstrating the promise of MARL for dynamic matching markets. However, PPO still incurs higher information-friction loss, revealing that end-to-end MARL does not yet provide the coordinated exploration structure of matching-bandit methods. These results position Learn2Match as a benchmark for developing the next generation of matching-market algorithms: methods that are adaptive like RL agents, statistically disciplined like bandit algorithms, and structurally aware like stable-matching mechanisms. Please refer to https://sites.google.com/view/learn-to-match/home for the official website and the code link.

• The paper introduces Learn2Match, a benchmark modeling evolving preferences with temporally extended feedback to better capture real-world matching markets.
• It formalizes matching dynamics as a partially observable Markov game, comparing MARL-based PPO with bandit-style CA-ETC under varying information frictions.
• Empirical results show that PPO significantly reduces cumulative regret and boosts social welfare, highlighting the need for hybrid exploration and adaptive strategies.

Two-Sided Matching with Temporally Extended Feedback: The Learn2Match Benchmark

Introduction and Motivation

Traditional models of two-sided matching, such as those based on Gale–Shapley stable matching and bandit-based online learning, have foundational relevance in labor markets, school choice, online platforms, and similar domains. However, these models universally adopt the simplifying assumption of immediate, one-shot feedback—where a matching generates an instantaneous, noisy but complete, signal about agents’ preferences. This abstraction fails to represent a crucial feature of most real-world markets: payoff-relevant information is revealed only over time, through preliminary screening, interaction, on-the-job learning, and endogenous dissolution or continuation decisions.

This paper introduces Learn2Match, a new formal and experimental framework that explicitly models temporally extended feedback in two-sided matching markets (2606.06744). By framing matching dynamics as a partially observable Markov game with pre- and post-match information frictions, evolving agent profiles, and strategic search/dissolution decisions, the paper aims to bridge the gap between classical matching theory and realistic, temporally-structured environments.

Framework and Problem Setting

Learn2Match models a dynamic two-sided market with $N^{\mathcal W}$ agents (e.g., workers) and $N^{\mathcal F}$ agents (e.g., firms), where agents have latent profiles—real-valued vectors $x_i, y_j$—that determine preferences but are unobservable. Each agent has local, noisy beliefs about its potential matches, updated through a sequential information acquisition process: screening interviews, matching, and tenure. The key innovation is that observed information about match quality emerges only gradually, both before and after matches are formed, resulting in partial observability at each stage.

The environment is formalized as a partially observable Markov game. Agents choose actions (interview proposals, match proposals, responses, and dissolutions), observe noisy signals on counterparts’ latent vectors, and receive rewards contingent on how closely matches align with true but hidden preferences. The state consists of current matches, revealed agent beliefs, and market history. A central property—asymptotic revelation—ensures that latent profiles become observable after sufficient repeated interaction.

Temporal extensions are operationalized as follows:

• Multi-stage Screening: Agents actively allocate search/interview effort before matching; outcomes are noisy and only partially unfold over several periods.
• Post-match Learning: After matching, further information about match quality is revealed as a function of tenure, with observed profiles converging sigmoidal to true latent vectors.
• Costly Dissolution: Matches can be dissolved endogenously, but with potential costs and future opportunities sacrificed.

This structure captures career path dependence and persistent information frictions documented in labor economics, as matching choices affect both information sets and future market possibilities.

Benchmark Instantiation and Experimental Design

Learn2Match is instantiated as a MARL-compatible environment, supporting decentralized agent policies that choose whom to screen, whom to match with, when to stay, and when to dissolve. The reward structure is linear in (latent, estimated) profiles, which flexibly encodes both cardinal and ordinal preference structures.

Two algorithmic approaches are compared:

• Independent PPO (Proximal Policy Optimization): Decentralized multi-agent RL, where each agent maximizes its own expected long-run payoff based on local observations/history using recurrent neural policies.
• CA-ETC (Collision-Avoiding Explore-then-Commit): A bandit-style baseline, adapted to temporally extended feedback, that alternates between structured exploration rounds (with round-robin interviewing and empirical preference estimation) and exploitation via Gale–Shapley matching.

Experimental setups examine varying market sizes and two core regimes:

1. Low-noise/nearly static: Immediate and almost noiseless feedback, favoring bandit approaches.
2. Temporally extended feedback: Realistic, noisy, and slow information revelation—interviews and post-match updates are both far from the latent profile, converging only with tenure.

Evaluations use four metrics:

• Worker regret and firm regret: Cumulative loss relative to worker-optimal stable matching.
• Social welfare: Aggregate realized rewards over all periods.
• Information-friction loss: The welfare gap due to incomplete revelation of preferences, measured as the difference between acting on beliefs vs. on perfectly revealed preferences.

Empirical Findings and Analysis

In the low-noise setting, PPO policies outperform CA-ETC in both social welfare and regret, despite the latter’s near-zero information-friction loss. Strong numerical results indicate that PPO roughly halves cumulative firm and worker regret compared to CA-ETC in finite horizons, confirming advantages for MARL-type temporal planning even in favorable conditions for the bandit baseline.

In the temporally extended feedback regime, the gap widens: PPO continues to achieve both lower regret and higher welfare, particularly in large markets and over longer horizons. However, neither approach drives information-friction loss to zero—CA-ETC maintains lower friction loss due to explicit, structurally coordinated exploration, while PPO policies tend to commit to persistent matches after initial exploration, echoing real labor-market persistence and under-exploration documented in economic studies.

Notably, the paper claims (and demonstrates empirically):

• End-to-end MARL (PPO) does not inherently implement the coordinated exploration of bandit algorithms, and leaves information-friction loss substantially above theoretical minimums.
• Bandit-style methods reduce friction loss but sacrifice long-run welfare by failing to exploit sequential structure beyond the explore-then-commit horizon.
• The design space for matching-market algorithms must extend beyond classical RL or bandits toward methods that simultaneously coordinate structured exploration, adaptivity, and stable-matching constraints.

Implications and Future Directions

This work demonstrates that temporally extended feedback is fundamental in practical matching markets, and the static-revelation, immediate-feedback abstractions are limiting for both algorithm design and theoretical analysis. The proposed Learn2Match benchmark provides an extensible platform for developing and stress-testing new algorithms that can coordinate exploration, leverage sequential learning, and respect the combinatorial constraints intrinsic to matching.

Practically, this opens the door for robust automated matching mechanisms in labor markets, online platforms, ride-sharing, and decentralized assignment settings, where both sides are adaptive and where misallocation caused by information frictions has significant economic impact.

Theoretically, Learn2Match motivates new work at the intersection of MARL, multi-armed bandits, and matching theory. The results suggest that hybrid approaches—incorporating the exploration discipline of bandits, the adaptivity of RL, and the structural guarantees of stable matching—are required for optimal market efficiency. Future research directions include:

• Algorithmic development for jointly adaptive and statistically disciplined exploration processes in partially observable Markov games.
• Analytical characterization of the price of information-friction under various market parameters.
• Generalizations to many-to-many, large-scale, and heterogeneously structured markets.
• Validation and calibration using real-world datasets from platforms and administrative labor records.

Conclusion

Learn2Match introduces a rigorous environment for studying dynamic, two-sided matching with temporally extended feedback. Empirical and theoretical insights indicate that classical models are insufficient under realistic feedback structures and that new hybrid algorithmic paradigms are required. This benchmark establishes an agenda for research at the interface of RL, matching theory, and applied market design, with implications for both social welfare and the microstructure of information in dynamic assignment markets (2606.06744).