Pong: A Multidisciplinary Research Benchmark

Updated 18 April 2026

Pong is a foundational table-tennis–inspired video game that serves as a benchmark for reinforcement learning, robotics, and algebraic topology research.
Deep reinforcement learning approaches use policy gradients, DQN, and actor-critic models with Pong to achieve near-perfect play through careful preprocessing and adversarial training.
Research on Pong also spans control theory and hybrid architectures, informing advanced studies in vision-based tracking, probabilistic grasping, and symplectic topology.

Pong is a foundational table-tennis–inspired two-player video game, and serves as a canonical testbed for research in reinforcement learning, vision-based tracking, robotics, algebraic topology, and probabilistic modeling. Although named for its minimalistic virtual implementation, "Pong" and its physical analogs (e.g., ping pong/table tennis) are central to multiple research domains, ranging from real-time control policies and learning architectures to geometric and homological methods in mathematics.

1. Historical Context and Game Environment

Pong originated as an early video game emulating table tennis, but in the research literature, it became a primary benchmark for sequential decision making from high-dimensional sensory observations. The standard Pong environment used in deep RL builds on the Atari "Pong" console interface, providing a discrete two-player competitive scenario with a vertically symmetric board, simple physics, and a discrete set of player actions. State observations typically are sequences of low-resolution images or pixel arrays, and the environment is episodic, terminating when one player reaches a score threshold (typically 21) (Neuwirth et al., 2021, Phon-Amnuaisuk, 2018, He et al., 2022).

Customized Pong environments have been implemented to allow for systematic experimental control. For example, Neuwirth & Riley constructed a fully configurable engine in Python (using NumPy-based pixel buffers) offering higher control over ball/paddle/arena parameters, action/reward assignment, frame-skipping, and simultaneous human-AI play (Neuwirth et al., 2021). Core preprocessing steps include cropping, downsampling (e.g., to 80×96 or 80×80), pixel binarization, and frame-differencing to encode motion.

2. Deep Reinforcement Learning Methodologies

Policy Optimization and Network Architectures

The Pong task is typically formalized as a Markov Decision Process (MDP) $(S, A, P, r, \gamma)$ , where states $s_t$ are frame-difference pixel arrays, actions $a_t\in\{\text{up},\text{down},\text{none}\}$ or one-hot extended action sets, and rewards are sparse ( $+1$ for scoring, $-1$ for conceding, $0$ otherwise). The objective is to find a parameterized policy $\pi_\theta(a|s)$ that maximizes expected discounted reward (Phon-Amnuaisuk, 2018, Neuwirth et al., 2021). The most prevalent training algorithms include Monte Carlo policy gradient (REINFORCE) and value-based methods such as DQN.

Policy Networks: Early policy gradient models follow the "Karpathy-style" feedforward architecture: input is a flattened $80\times80$ or $80\times96$ image, processed through 1–4 fully connected ReLU layers (best: 200→200→100), with softmax output probabilities (Neuwirth et al., 2021, Phon-Amnuaisuk, 2018). Loss is standard cross-entropy, with REINFORCE gradients computed using discounted returns.
DQN/Value Networks: DQN models for Pong adopt convolutional front-ends (as in Mnih et al. 2013) followed by a 512-unit dense layer mapping to Q-values for all action choices. Inputs are stacks of four consecutive grayscale frames resized to $84\times84$ (He et al., 2022). Exploration is mediated by ε-greedy policies.
Asynchronous Actor-Critic (A3C): Multi-threaded A3C accelerates convergence by employing parallel actor-learners that asynchronously update shared policy and value parameters. The typical architecture includes convolutional or dense actor-critic heads with entropy regularization for robust exploration (Phon-Amnuaisuk, 2018, Mittel et al., 2018).

Training Protocols and Empirical Performance

Training typically proceeds with episodic batch updates, discount factor $s_t$ 0 in the range $s_t$ 1– $s_t$ 2, learning rates between $s_t$ 3– $s_t$ 4, and batch-sizes from 1 up to 25 episodes per parameter update. Adversarial self-play schemes—such as "Chainer" (progressive self-conquest) and "Pool" (training against a pool of former selves)—improve policy robustness and generalization, yielding agents significantly more robust to exploitation than those trained versus static opponents (He et al., 2022).

Small, deep dense networks (e.g., 200→200→100) trained with careful input preprocessing (frame-differencing, ball-size tuning, opponent variety) reliably achieve perfect or near-perfect Pong play (21–0 against both scripted and human opponents) (Neuwirth et al., 2021, Phon-Amnuaisuk, 2018). Adversarial curricula produce richer internal state representations, as measured by linear probes for ball-landing prediction (He et al., 2022).

3. Control Theory and Non-Learning (PCT) Approaches

Perceptual control theory (PCT) offers a structured, closed-loop alternative to data-driven RL for Pong. PCT-based controllers (PCTagent) operate without parameter learning or reward maximization. Instead, perceptual variables (e.g., ball–paddle distance, paddle centroid) are extracted from the image by binarization and contour detection, and hierarchical error-driven controllers generate motor commands to minimize the perceptual discrepancy at each level.

Reference signals at each layer propagate dynamic setpoints down the hierarchy; the residual error at the lowest level determines button-press actions, achieving continuous closed-loop control. PCTagent achieves near-perfect Atari Pong performance (mean score margin 5–6, no training frames), and functional operation is immediate given the architectural specification (Gulrez et al., 2021).

4. Certainty-Driven and Hybrid Architectures

Certainty-based reward-modulated architectures integrate multiple neural networks (Prediction, Positive/Negative Reward, Intuition) to assess the agent's confidence in returning the ball, modulating exploration accordingly. When certainty exceeds specified thresholds, the prediction network's output is trusted; for mid-level certainty, an Intuition network is used; otherwise, actions are random. This gating expedites reward-based learning and improves sample efficiency, surpassing simple architectures in learning rate and ultimate score margins (Oberdorfer et al., 2016).

Quantum-classical hybrid agents, such as those combining parametrized quantum circuits (PQCs) with classical feature extractors and output layers, can achieve mean perfect scores in Pong, demonstrating performance on par with classical deep RL models under architectural constraints. Notably, tuning the quantum layer's learning rate is critical to accommodate the trigonometric landscape of the PQC Q-values. The hybrid agent can solve Pong in as few as 400–600k environment steps, outperforming comparably constrained classical references (Freinberger et al., 2024).

5. Vision, Tracking, and Physical Table Tennis

Pong-inspired research extends to real-world ping pong tracking, action recognition, and robotics. The SPIN dataset offers high-resolution, high-frame-rate stereo video (150 Hz, 1024×1280 pixels) of extensive table tennis rallies, annotated for ball position, human pose, and spin. Baseline models employ lightweight CNNs and conv-recurrent units (e.g. gated recurrent, Conv-LSTM) to achieve high accuracy: ball tracking AUC@2 px up to 88.5%, spin cluster accuracy up to 72.8%, and pose PCK@16 px up to 87.4% (Schwarcz et al., 2019).

Physical insights from SPIN reveal that spin can be discretized into three clusters (no spin, light topspin, heavy topspin), and that professional players more frequently induce high-spin trajectories while non-professionals usually generate the "no-spin" regime. Multi-task learning reveals that adding pose estimation to spin classification improves performance, indicating that body pose dynamics are predictive of imparted spin.

Event-based vision (“neuromorphic” vision) using wearable sensors further augments trajectory prediction capabilities, enabling sub-5 ms perception latencies and <15 cm endpoint errors in egocentric table tennis prediction. Eye-gaze–driven foveated processing dramatically reduces computational load (10.81× reduction) and maintains high detection rates (92.6% for <5px error) (Alberico et al., 9 Jun 2025).

6. Probabilistic Grasping: The “PONG” Metric in Robotics

“PONG” (Probabilistic Object Normals for Grasping) refers to an analytic metric for assessing the probability of force-closure under surface-normal uncertainty in robotic precision grasping. Contact-point normals are modeled as Gaussian random variables in the tangent plane; a conservative lower bound on the true force-closure probability is computed via a convex inner approximation (polygonal in the tangent plane) and line-integral formulation for efficient, differentiable grasp optimization.

Empirical results show that grasp synthesis optimizing the PONG metric yields higher empirical success rates under both simulation and real-world experiments, especially for objects exhibiting substantial geometric uncertainty as in NeRF-reconstructed models. PONG's analytic, uncertainty-aware metric enables rapid and robust grasp evaluation and optimization in dexterous manipulation scenarios (Li et al., 2023).

7. Algebraic and Topological Foundations: The Pong Algebra

In symplectic topology, the "pong algebra" is a differential graded (DG) algebra defined over a polynomial ring $s_t$ 5, generated by lifted partial permutations with a bigrading given by Maslov and monomial gradings. The multiplication and differential mirror geometric operations (crossing resolution, composition) in Heegaard Floer theory.

Ozsváth–Szabó identified the pong algebra $s_t$ 6 with the endomorphism algebra in the wrapped Fukaya category of the symmetric product of a punctured disk, linking combinatorial algebraic constructions to holomorphic curve counts in symplectic geometry. This correspondence provides a computable model for endomorphism algebras underlying bordered and knot Floer homologies, uniting the combinatorics of "game moves" (strands, partial permutations) with wrapped Fukaya categories and symplectic topology (Ozsvath et al., 2022).