Optimal Rebate Design: Incentives, Competition and Efficiency in Auction Markets

Abstract: This study explores the design of an efficient rebate policy in auction markets, focusing on a continuous-time setting with competition among market participants. In this model, a stock exchange collects transaction fees from auction investors executing block trades to buy or sell a risky asset, then redistributes these fees as rebates to competing market makers submitting limit orders. Market makers influence both the price at which the asset trades and their arrival intensity in the auction. We frame this problem as a principal-multi-agent problem and provide necessary and sufficient conditions to characterize the Nash equilibrium among market makers. The exchange's optimization problem is formulated as a high-dimensional Hamilton-Jacobi-Bellman equation with Poisson jump processes, which is solved using a verification result. To numerically compute the optimal rebate and transaction fee policies, we apply the Deep BSDE method. Our results show that optimal transaction fees and rebate structures improve market efficiency by narrowing the spread between the auction clearing price and the asset's fundamental value, while ensuring a minimal gain for both market makers indexed on the price of the asset on a coexisting limit order book.