Codeforces-Based Elo Ratings

• Codeforces-based Elo ratings are rating constructions derived from contest data that use contest-rank inversion, predictive forecasting, or difficulty calibration to estimate performance.
• They enable reconstruction of contest standings and robust prediction of participants’ future ratings using historical contest outcomes and deep learning methodologies.
• The approach is sensitive to contest protocols and highlights the importance of benchmarking choices in interpreting both participant skill and problem difficulty.

Codeforces-based Elo ratings are rating constructions whose evidence, calibration, or interpretation is anchored to the Codeforces ecosystem rather than to an abstract paired-game ladder. The literature uses the phrase for several non-equivalent objects: contest-level Elo estimates reconstructed from historical Codeforces standings, forecasts of a contestant’s future official rating, and problem-difficulty scales that use Codeforces ratings as benchmark strata. This terminological spread matters because some studies are genuinely rating systems, whereas others merely treat official Codeforces ratings as labels or use problem Elo only as an exogenous difficulty axis (Quan et al., 2 Jan 2025, Rahman et al., 2023, Souza et al., 9 Apr 2025).

1. Scope and terminological distinctions

A precise account begins by separating three meanings that are often conflated. First, a Codeforces-based Elo can be a contest-rank reconstruction: a model or agent is inserted into a real Codeforces contest standing, and the rating that would make its observed rank expected against the human field is inferred. Second, it can be a predictive surrogate: official Codeforces ratings are used as supervised targets, and a model forecasts the next official rating from prior contest history and auxiliary activity. Third, it can be a difficulty label: Codeforces problem ratings organize benchmark instances by hardness without rating participants at all (Quan et al., 2 Jan 2025, Rahman et al., 2023, Souza et al., 9 Apr 2025).

Use in the literature	Rated object	Core evidence source
Contest-rank reconstruction	Model or agent	Real Codeforces contest standings
Rating forecasting	Human contestant’s next official rating	Prior contests and practice activity
Difficulty calibration	Problems	Codeforces problem rating buckets

Two negative clarifications follow. The forecasting study on contestant performance is explicitly not a reconstruction of the official Codeforces update rule; it predicts the next timestamp’s rating from a sequence model over historical features (Rahman et al., 2023). Conversely, benchmark papers that report Codeforces-based Elo for LLMs are not reproducing a user’s native platform history; they are constructing a benchmark-induced performance rating from contest placement relative to historical human fields (Quan et al., 2 Jan 2025).

A related distinction is between problem rating and participant rating. In benchmark design, Codeforces problem Elo is a property of the task set, used to stratify difficulty. In rating reconstruction, Elo is a property of a contestant or model, inferred from performance against a field. Treating one as a substitute for the other is not supported by the surveyed sources (Quan et al., 2 Jan 2025, Souza et al., 9 Apr 2025).

2. Contest-rank reconstruction against historical human fields

The most explicit Codeforces-based Elo construction appears in "CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings" (Quan et al., 2 Jan 2025). The benchmark uses 54 contests comprising 398 problems from May 4, 2024 through November 4, 2024. The retained set includes 8 Div. 1+2 contests, 33 Div. 2 contests, 10 Div. 3 contests, and 3 Div. 4 contests. Solutions are submitted directly to the official Codeforces platform by an automatic submission bot, and only “Accepted” counts as success. The protocol allows up to eight attempts per problem; no elapsed-time penalty is applied because all model submissions are assumed to occur within the first minute, but failed attempts before success still contribute penalty in the contest score-and-penalty framework (Quan et al., 2 Jan 2025).

The core inversion step is contest-based rather than pairwise-online. For a contest with $n$ human participants of ratings $r_{(i)}$, and model rank $m$, the model’s contest-level expected rating $r$ is defined implicitly by

$$m = \sum\limits_{i=1}^{n} \frac{1}{1 + 10^{(r - r_{(i)})/400}}.$$

Because the right-hand side is monotone, $r$ is solved by binary search. This yields a performance rating for that contest. The benchmark then averages contest-level inferred ratings across contests. In the variance analysis reported in the paper, if the per-contest inferred rating has variance $\mathrm{Var}(r)$, then the average over $k$ independent contests has variance $\mathrm{Var}(r)/k$, whereas the official-style recursive comparison point is written as

$r_i = \frac{r_{i-1} + E(r_i)}{2}.$

The benchmark therefore presents itself as aligned with the platform’s judging and ranking semantics while using a lower-variance estimator than a naïve online historical update.

Platform fidelity is a central design claim. Problems are judged by the official Codeforces environment, so hidden tests, runtime constraints, and special judges are inherited directly from the platform. The paper reports that about 30% of problems in a random sample of 100 required a special judge, a failure mode that offline benchmarks generally do not support. This makes the inferred Elo more interpretable than offline pass rates on public samples, but it also makes the score tightly coupled to the submission protocol.

The resulting numbers are benchmark-induced rather than native account ratings. Under this protocol, o1-mini reaches Elo 1578 and percentile 89.2 among human participants, while QwQ-32B-Preview reaches Elo 1261 and percentile 63.6. The same study reports that when language is left unspecified, models choose Python more than 95% of the time, whereas forcing C++ yields higher Elo for all tested models. This directly indicates that Codeforces-based Elo in benchmark settings is partly a property of the elicitation protocol, not only of underlying reasoning ability.

3. Forecasting future official ratings rather than deriving a rating rule

"Predicting Participants' Performance in Programming Contests using Deep Learning Techniques" formulates a different problem: given a contestant’s recent Codeforces history and coarse inter-contest practice activity, predict the rating at the next contest timestamp (Rahman et al., 2023). The dataset is collected from the public Codeforces API for 100 contestants. The raw fields are contestant’s ratings, competition ranks, problem submissions, and submission verdicts. Each contest is treated as a timestamp with four feature groups: rating, rank, solve rating, and practice features.

The practice features are aggregated between consecutive contests. Accepted (AC) is the number of problems solved between the previous contest and the current contest, and Wrong Answer (WA) is the number of unsuccessfully attempted problems in the same interval. Sequence construction is explicit: each continuous 16 timestamps of a user are considered as a sequence, with the first 15 timestamps as input and the 16th timestamp as the target. The split is 80% train and 20% test (Rahman et al., 2023).

Although the paper’s prose sometimes describes performance from two perspectives—direction of rating change and rating value—the implemented task is regression on the next rating. The reported metrics are MAE, RMSE, MSE, and $r_{(i)}$0, with $r_{(i)}$1 and $r_{(i)}$2 defined as the actual and predicted ratings. The models are sequence architectures: LSTM, Bi-LSTM, GRU, and LSTM + Attention. Shared implementation details include four recurrent layers with 256 neurons, dropout 0.5 after each layer, a dense layer of 100 neurons with Relu, MAE loss, Adam optimizer, 1000 epochs, and batch size 256.

The key empirical result is that LSTM + Attention performs best in both feature settings. With practice features included, its reported next-rating performance is RMSE 51.325, MSE 3243.234, MAE 39.217, and $r_{(i)}$3. The paper states that adding practice features improves LSTM+Attention by about 26% in RMSE, 33% in MSE, 24% in MAE, and 10% in $r_{(i)}$4. This makes inter-contest activity a strong predictive signal for future official rating.

What the study does not provide is equally important. It does not report an Elo expectation formula, a Codeforces expected-rank formula, seed computation, performance rating, or delta equation. It therefore cannot serve as a source for official Codeforces mechanics. Its contribution is evidentiary: prior rating trajectory, prior ranks, contest-score proxy, and coarse practice volume carry predictive information about the next official rating. A plausible implication is that any Codeforces-like Elo system augmented with activity variables would track short-horizon future performance differently from a pure contest-only scalar rating.

4. Multiplayer contest Elo formulations and empirical tuning

A direct analogue for programming contests is developed in "An Elo-based rating system for TopCoder SRM" (Batty et al., 2019). The paper starts from the fact that standard Elo is designed for two-player outcomes and adapts it to many-participant rank-only contests by defining a logarithmic rank-performance signal. The rank performance function is

$r_{(i)}$5

motivated by an elimination-tournament interpretation. Pairwise expected superiority still uses standard Elo odds,

$r_{(i)}$6

but these pairwise expectations are aggregated into an expected rank $r_{(i)}$7. Relative contest performance is then

$r_{(i)}$8

This replaces naïve all-pairs independence with a contest-level performance discrepancy.

The update rule is progressively regularized. A direct Elo-scale conversion uses

$r_{(i)}$9

Empirical improvements then introduce an experience factor $m$0, an informativeness term $m$1, and a saturation transform

$m$2

leading to

$m$3

The paper reports successive improvement over official SRM ratings, with overall error moving from 0.8301 for SRM to 0.7513 for the main Elo variant and 0.7497 for the final Elo2 variant. It also states that the method outperforms SRM on the error metric in 89.2% of rounds, with the advantage increasing sharply in larger contests (Batty et al., 2019).

For Codeforces, this is not a drop-in formula, but it is the closest surveyed multiplayer Elo-style construction for programming contests. A plausible implication is that a Codeforces adaptation would benefit from treating rank discrepancy on a logarithmic scale as the contest-performance signal rather than converting one contest into a large set of independent pairwise wins.

A complementary line of work addresses not the contest model but the parameterization problem. "Empirical parameterization of the Elo Rating System" tunes the parameter set $m$4 by maximizing predictive usefulness on historical gameplay data rather than inheriting conventional settings (Maitra et al., 19 Dec 2025). The expected score and update rule remain standard,

$m$5

with a piecewise experience-dependent $m$6-schedule. The paper explicitly says that the methodology is generalizable to multiplayer setups through suitable modification of the parameter space. This suggests a Codeforces implementation should treat $m$7-schedules, cutoffs, and perhaps the scaling constant $m$8 as empirical hyperparameters rather than as fixed conventions.

5. Theoretical foundations and structural extensions

General Elo theory provides a useful baseline for interpreting Codeforces-based constructions, even when the target system is not literally pairwise. "An Analysis of Elo Rating Systems via Markov Chains" studies standard online logistic Elo under the Bradley–Terry–Luce model, viewing Elo as stochastic gradient ascent on the pairwise log-likelihood and as a Markov chain in rating space (Olesker-Taylor et al., 2024). Its central quantitative control variable is the spectral gap $m$9 of the comparison graph. The paper shows that time-averaged Elo ratings approximate latent skills at rates competitive with strong offline estimators, and emphasizes that sparse or bottlenecked comparison structure slows learning. For contest platforms, this implies that fragmented participation patterns can matter as much as the update rule itself.

"Convergence and stationary distribution of Elo rating systems" studies a finite-player zero-sum Elo process and proves existence of a unique stationary distribution under $r$0, along with finite exponential moments, full support, and $r$1 convergence of stationary ratings to true skills as $r$2 (Cortez et al., 2024). The paper’s message is that even with fixed latent skill, Elo ratings fluctuate in the long run rather than settling to a deterministic vector. For Codeforces-like systems, this supports a probabilistic interpretation of rating volatility rather than a point-identification reading.

A different structural remedy is proposed in "The self-justifying Elo rating system" (Langholf, 2018). Instead of updating ratings from expectations computed under old ratings, the paper defines ratings as the fixed point

$r$3

so that the final ratings justify the observed results under their own expectations. The paper argues that this removes path dependence, improves monotonicity, and reduces overcompensation for large result batches. This suggests a batch-consistent contest rating philosophy: when a whole contest yields one dense block of comparative evidence, a fixed-point construction may be more coherent than a single one-step update from prior ratings.

When one scalar is not enough, the relevant extensions are not straightforwardly Codeforces-specific but clarify boundary conditions. "Ordinal Potential-based Player Rating" argues that Elo may fail to preserve transitive relations unless ratings are computed in a suitable transformed space, and decomposes arbitrary games into transitive and cyclic components (Vadori et al., 2023). "Online Learning of Counter Categories and Ratings in PvP Games" proposes Elo-RCC, which augments scalar Elo with latent counter categories and an online residual table, but explicitly notes that the method targets two-player zero-sum PvP games and is only conceptually relevant, not directly operational, for Codeforces (Lin et al., 6 Feb 2025). The common lesson is that extra structure is justified when systematic residual interactions exist beyond a transitive scalar ordering.

If multiple Elo-like subratings are maintained, "Aggregating Elo Ratings: An Axiomatization" proves that the unique scalar aggregate satisfying same-scale normalization, recursive consistency, and marginal Elo-strength consistency is

$r$4

that is, convert each component to Elo strength, take a weighted arithmetic mean of strengths, and convert back (Seven, 9 May 2026). For any prospective Codeforces system with multiple mode-specific ratings, direct averaging on the rating scale would therefore not preserve Elo semantics.

6. Problem-difficulty Elo, benchmark interpretation, and protocol sensitivity

Codeforces-based Elo is also used as a difficulty axis rather than as a participant rating. "Code Generation with Small LLMs: A Deep Evaluation on Codeforces" builds a benchmark of 280 Codeforces problems stratified by problem rating from 800 to 2100, with 20 problems for each rating level and 36 topics overall (Souza et al., 9 Apr 2025). Here Elo is the backbone of benchmark design, not the output metric. The paper’s strongest open model, Phi-4 14B, is nearly perfect through much of 800–1500 in Python and declines sharply beyond 1500; standalone Python reaches pass@3 = 63.6%, while combined Python and C++ reaches pass@3 = 73.6%. Of the 28 problems uniquely solved in C++, 25 were rated above 1500. The paper therefore treats higher Codeforces Elo bands as a composite challenge axis involving harder reasoning, more intricate implementation, and tighter efficiency constraints.

This use of Elo should not be confused with model or user rating. Problem Elo is an external calibration variable. Participant Elo is an inferred skill estimate. The distinction matters because benchmark papers often discuss performance “across Elo ratings” even when no contestant-rating mechanism is being run.

The strongest methodological caution appears in "When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of LLMs" (Zheng et al., 5 Feb 2026). Using a local benchmark of 37 complete contests, 260 problems, and 13,691 test cases, the paper shows that reconstructed Codeforces-based Elo is highly sensitive to protocol choices. Varying submission order changes Elo by up to 394 points. Contest selection changes it by up to 1,122 points for the same model. Run-to-run stochastic variability changes mean scores by up to 349 points, with a reported 1,348-point swing on a single contest. The motivating external discrepancy is that the “same model version,” DeepSeek-R1, was reported once at 2,029 Elo and later at 1,530.

The paper’s conclusion is not that Elo is unusable, but that benchmark-induced Codeforces Elo is protocol-dependent. Exact contest IDs, division mix, number of generated outputs, submission-order policy, local test construction, multi-solution verification, and repeated-run statistics materially determine the reported number. A plausible implication is that Codeforces-based Elo in model evaluation should be read as a controlled experimental estimate rather than as an intrinsic property of a system in isolation.

Taken together, the literature supports a narrow but technically clear synthesis. Codeforces-based Elo ratings are most rigorous when they specify whether Elo refers to contest-rank inversion, future-rating prediction, or problem-difficulty calibration; when they state the transformation from contest outcomes to rating evidence; and when they separate platform-faithful judging from protocol-specific benchmark choices. The concept is therefore best understood as a family of rating and calibration procedures built on Codeforces data, not as a single canonical algorithm.