---
title: Codeforces-Based Elo Ratings
url: https://www.emergentmind.com/topics/codeforces-based-elo-ratings
type: topic
---

# Codeforces-Based Elo Ratings

Codeforces-based Elo ratings are rating constructions whose evidence, calibration, or interpretation is anchored to the Codeforces ecosystem rather than to an abstract paired-game ladder. The literature uses the phrase for several non-equivalent objects: contest-level Elo estimates reconstructed from historical Codeforces standings, forecasts of a contestant’s future official rating, and problem-difficulty scales that use Codeforces ratings as benchmark strata. This terminological spread matters because some studies are genuinely rating systems, whereas others merely treat official Codeforces ratings as labels or use problem Elo only as an exogenous difficulty axis [2501.01257][2302.05602][2504.07343].

## 1. Scope and terminological distinctions

A precise account begins by separating three meanings that are often conflated. First, a Codeforces-based Elo can be a **contest-rank reconstruction**: a model or agent is inserted into a real Codeforces contest standing, and the rating that would make its observed rank expected against the human field is inferred. Second, it can be a **predictive surrogate**: official Codeforces ratings are used as supervised targets, and a model forecasts the next official rating from prior contest history and auxiliary activity. Third, it can be a **difficulty label**: Codeforces problem ratings organize benchmark instances by hardness without rating participants at all [2501.01257][2302.05602][2504.07343].

| Use in the literature | Rated object | Core evidence source |
|---|---|---|
| Contest-rank reconstruction | Model or agent | Real Codeforces contest standings |
| Rating forecasting | Human contestant’s next official rating | Prior contests and practice activity |
| Difficulty calibration | Problems | Codeforces problem rating buckets |

Two negative clarifications follow. The forecasting study on contestant performance is explicitly **not** a reconstruction of the official Codeforces update rule; it predicts the next timestamp’s rating from a sequence model over historical features [2302.05602]. Conversely, benchmark papers that report Codeforces-based Elo for LLMs are not reproducing a user’s native platform history; they are constructing a benchmark-induced performance rating from contest placement relative to historical human fields [2501.01257].

A related distinction is between **problem rating** and **participant rating**. In benchmark design, Codeforces problem Elo is a property of the task set, used to stratify difficulty. In rating reconstruction, Elo is a property of a contestant or model, inferred from performance against a field. Treating one as a substitute for the other is not supported by the surveyed sources [2501.01257][2504.07343].

## 2. Contest-rank reconstruction against historical human fields

The most explicit Codeforces-based Elo construction appears in "CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings" [2501.01257]. The benchmark uses **54 contests comprising 398 problems** from **May 4, 2024 through November 4, 2024**. The retained set includes **8 Div. 1+2 contests, 33 Div. 2 contests, 10 Div. 3 contests, and 3 Div. 4 contests**. Solutions are submitted directly to the official Codeforces platform by an automatic submission bot, and only **“Accepted”** counts as success. The protocol allows **up to eight attempts per problem**; no elapsed-time penalty is applied because all model submissions are assumed to occur within the first minute, but failed attempts before success still contribute penalty in the contest score-and-penalty framework [2501.01257].

The core inversion step is contest-based rather than pairwise-online. For a contest with \(n\) human participants of ratings \(r_{(i)}\), and model rank \(m\), the model’s contest-level expected rating \(r\) is defined implicitly by

\[
m = \sum\limits_{i=1}^{n} \frac{1}{1 + 10^{(r - r_{(i)})/400}}.
\]

Because the right-hand side is monotone, \(r\) is solved by binary search. This yields a **performance rating for that contest**. The benchmark then averages contest-level inferred ratings across contests. In the variance analysis reported in the paper, if the per-contest inferred rating has variance \(\mathrm{Var}(r)\), then the average over \(k\) independent contests has variance \(\mathrm{Var}(r)/k\), whereas the official-style recursive comparison point is written as

\[
r_i = \frac{r_{i-1} + E(r_i)}{2}.
\]

The benchmark therefore presents itself as aligned with the platform’s judging and ranking semantics while using a lower-variance estimator than a naïve online historical update.

Platform fidelity is a central design claim. Problems are judged by the official Codeforces environment, so hidden tests, runtime constraints, and special judges are inherited directly from the platform. The paper reports that about **30%** of problems in a random sample of 100 required a special judge, a failure mode that offline benchmarks generally do not support. This makes the inferred Elo more interpretable than offline pass rates on public samples, but it also makes the score tightly coupled to the submission protocol.

The resulting numbers are benchmark-induced rather than native account ratings. Under this protocol, **o1-mini** reaches **Elo 1578** and percentile **89.2** among human participants, while **QwQ-32B-Preview** reaches **Elo 1261** and percentile **63.6**. The same study reports that when language is left unspecified, models choose Python more than **95%** of the time, whereas forcing **C++** yields higher Elo for all tested models. This directly indicates that Codeforces-based Elo in benchmark settings is partly a property of the elicitation protocol, not only of underlying reasoning ability.

## 3. Forecasting future official ratings rather than deriving a rating rule

"Predicting Participants' Performance in Programming Contests using Deep Learning Techniques" formulates a different problem: given a contestant’s recent Codeforces history and coarse inter-contest practice activity, predict the rating at the next contest timestamp [2302.05602]. The dataset is collected from the public Codeforces API for **100 contestants**. The raw fields are **contestant’s ratings, competition ranks, problem submissions, and submission verdicts**. Each contest is treated as a timestamp with four feature groups: **rating**, **rank**, **solve rating**, and **practice features**.

The practice features are aggregated between consecutive contests. **Accepted (AC)** is the number of problems solved between the previous contest and the current contest, and **Wrong Answer (WA)** is the number of unsuccessfully attempted problems in the same interval. Sequence construction is explicit: **each continuous 16 timestamps of a user are considered as a sequence**, with the **first 15 timestamps** as input and the **16th timestamp** as the target. The split is **80%** train and **20%** test [2302.05602].

Although the paper’s prose sometimes describes performance from two perspectives—direction of rating change and rating value—the implemented task is regression on the next rating. The reported metrics are **MAE**, **RMSE**, **MSE**, and \(R^2\), with \(y_k\) and \(y'_k\) defined as the actual and predicted ratings. The models are sequence architectures: **LSTM**, **Bi-LSTM**, **GRU**, and **LSTM + Attention**. Shared implementation details include four recurrent layers with **256 neurons**, dropout **0.5** after each layer, a dense layer of **100** neurons with **Relu**, **MAE** loss, **Adam** optimizer, **1000 epochs**, and batch size **256**.

The key empirical result is that **LSTM + Attention** performs best in both feature settings. With practice features included, its reported next-rating performance is **RMSE 51.325**, **MSE 3243.234**, **MAE 39.217**, and \(R^2 = 0.97\). The paper states that adding practice features improves LSTM+Attention by about **26%** in RMSE, **33%** in MSE, **24%** in MAE, and **10%** in \(R^2\). This makes inter-contest activity a strong predictive signal for future official rating.

What the study does **not** provide is equally important. It does not report an Elo expectation formula, a Codeforces expected-rank formula, seed computation, performance rating, or delta equation. It therefore cannot serve as a source for official Codeforces mechanics. Its contribution is evidentiary: prior rating trajectory, prior ranks, contest-score proxy, and coarse practice volume carry predictive information about the next official rating. A plausible implication is that any Codeforces-like Elo system augmented with activity variables would track short-horizon future performance differently from a pure contest-only scalar rating.

## 4. Multiplayer contest Elo formulations and empirical tuning

A direct analogue for programming contests is developed in "An Elo-based rating system for TopCoder SRM" [1905.00961]. The paper starts from the fact that standard Elo is designed for two-player outcomes and adapts it to many-participant rank-only contests by defining a logarithmic rank-performance signal. The rank performance function is

\[
\mathit{RP}(n,r)=\log_2 n-\log_2 r,
\]

motivated by an elimination-tournament interpretation. Pairwise expected superiority still uses standard Elo odds,

\[
\mathit{WP}(R_i,R_j)=\frac{1}{1+10^{(R_j-R_i)/400}},
\]

but these pairwise expectations are aggregated into an expected rank \(\hat r\). Relative contest performance is then

\[
P=\mathit{RP}(n,r)-\mathit{RP}(n,\hat r)=\log_2 \hat r-\log_2 r.
\]

This replaces naïve all-pairs independence with a contest-level performance discrepancy.

The update rule is progressively regularized. A direct Elo-scale conversion uses

\[
\Delta R = K_0 \cdot P, \qquad K_0 = \frac{400 \ln 2}{\ln 10}.
\]

Empirical improvements then introduce an experience factor \(W=\sqrt{\mathit{NR}}\), an informativeness term \(V=1+C\cdot P'\), and a saturation transform

\[
\mathit{PA} = \frac{P}{1+\frac{|P|}{M}},
\]

leading to

\[
\Delta R = \frac{K}{V}\cdot\frac{\mathit{PA}}{W}.
\]

The paper reports successive improvement over official SRM ratings, with overall error moving from **0.8301** for SRM to **0.7513** for the main Elo variant and **0.7497** for the final **Elo2** variant. It also states that the method outperforms SRM on the error metric in **89.2%** of rounds, with the advantage increasing sharply in larger contests [1905.00961].

For Codeforces, this is not a drop-in formula, but it is the closest surveyed multiplayer Elo-style construction for programming contests. A plausible implication is that a Codeforces adaptation would benefit from treating **rank discrepancy on a logarithmic scale** as the contest-performance signal rather than converting one contest into a large set of independent pairwise wins.

A complementary line of work addresses not the contest model but the **parameterization problem**. "Empirical parameterization of the Elo Rating System" tunes the parameter set \(\{K_a,K_b,K_c,n_{c_1},n_{c_2}\}\) by maximizing predictive usefulness on historical gameplay data rather than inheriting conventional settings [2512.18013]. The expected score and update rule remain standard,

\[
E_i = \frac{1}{1 + 10^{\frac{R_j - R_i}{D}}}, \qquad
R_i' = R_i + K_i(S_i - E_i),
\]

with a piecewise experience-dependent \(K\)-schedule. The paper explicitly says that the methodology is generalizable to multiplayer setups through suitable modification of the parameter space. This suggests a Codeforces implementation should treat \(K\)-schedules, cutoffs, and perhaps the scaling constant \(D\) as empirical hyperparameters rather than as fixed conventions.

## 5. Theoretical foundations and structural extensions

General Elo theory provides a useful baseline for interpreting Codeforces-based constructions, even when the target system is not literally pairwise. "An Analysis of Elo Rating Systems via Markov Chains" studies standard online logistic Elo under the Bradley–Terry–Luce model, viewing Elo as stochastic gradient ascent on the pairwise log-likelihood and as a Markov chain in rating space [2406.05869]. Its central quantitative control variable is the spectral gap \(\lambda_q\) of the comparison graph. The paper shows that time-averaged Elo ratings approximate latent skills at rates competitive with strong offline estimators, and emphasizes that sparse or bottlenecked comparison structure slows learning. For contest platforms, this implies that fragmented participation patterns can matter as much as the update rule itself.

"Convergence and stationary distribution of Elo rating systems" studies a finite-player zero-sum Elo process and proves existence of a unique stationary distribution under \(KL<1\), along with finite exponential moments, full support, and \(O(\sqrt K)\) convergence of stationary ratings to true skills as \(K\to 0\) [2410.09180]. The paper’s message is that even with fixed latent skill, Elo ratings fluctuate in the long run rather than settling to a deterministic vector. For Codeforces-like systems, this supports a probabilistic interpretation of rating volatility rather than a point-identification reading.

A different structural remedy is proposed in "The self-justifying Elo rating system" [1801.05002]. Instead of updating ratings from expectations computed under old ratings, the paper defines ratings as the fixed point

\[
x = \operatorname{elo}^{\mathrm{cl}_k(x,p),
\]

so that the final ratings justify the observed results under their own expectations. The paper argues that this removes path dependence, improves monotonicity, and reduces overcompensation for large result batches. This suggests a batch-consistent contest rating philosophy: when a whole contest yields one dense block of comparative evidence, a fixed-point construction may be more coherent than a single one-step update from prior ratings.

When one scalar is not enough, the relevant extensions are not straightforwardly Codeforces-specific but clarify boundary conditions. "Ordinal Potential-based Player Rating" argues that Elo may fail to preserve transitive relations unless ratings are computed in a suitable transformed space, and decomposes arbitrary games into transitive and cyclic components [2306.05366]. "Online Learning of Counter Categories and Ratings in PvP Games" proposes Elo-RCC, which augments scalar Elo with latent counter categories and an online residual table, but explicitly notes that the method targets **two-player zero-sum PvP games** and is only conceptually relevant, not directly operational, for Codeforces [2502.03998]. The common lesson is that extra structure is justified when systematic residual interactions exist beyond a transitive scalar ordering.

If multiple Elo-like subratings are maintained, "Aggregating Elo Ratings: An Axiomatization" proves that the unique scalar aggregate satisfying same-scale normalization, recursive consistency, and marginal Elo-strength consistency is

\[
C_\lambda(R) = 400\log_{10}\left( \frac{\sum_{i=1}^n\lambda_i10^{R_i/400}}{\sum_{i=1}^n\lambda_i} \right),
\]

that is, convert each component to Elo strength, take a weighted arithmetic mean of strengths, and convert back [2605.08989]. For any prospective Codeforces system with multiple mode-specific ratings, direct averaging on the rating scale would therefore not preserve Elo semantics.

## 6. Problem-difficulty Elo, benchmark interpretation, and protocol sensitivity

Codeforces-based Elo is also used as a **difficulty axis** rather than as a participant rating. "Code Generation with Small Language Models: A Deep Evaluation on Codeforces" builds a benchmark of **280 Codeforces problems** stratified by problem rating from **800** to **2100**, with **20 problems for each rating level** and **36 topics** overall [2504.07343]. Here Elo is the backbone of benchmark design, not the output metric. The paper’s strongest open model, **Phi-4 14B**, is nearly perfect through much of **800–1500** in Python and declines sharply beyond **1500**; standalone Python reaches **pass@3 = 63.6%**, while combined Python and C++ reaches **pass@3 = 73.6%**. Of the **28 problems uniquely solved in C++**, **25 were rated above 1500**. The paper therefore treats higher Codeforces Elo bands as a composite challenge axis involving harder reasoning, more intricate implementation, and tighter efficiency constraints.

This use of Elo should not be confused with model or user rating. Problem Elo is an external calibration variable. Participant Elo is an inferred skill estimate. The distinction matters because benchmark papers often discuss performance “across Elo ratings” even when no contestant-rating mechanism is being run.

The strongest methodological caution appears in "When Elo Lies: Hidden Biases in Codeforces-Based Evaluation of Large Language Models" [2602.05891]. Using a local benchmark of **37 complete contests**, **260 problems**, and **13,691 test cases**, the paper shows that reconstructed Codeforces-based Elo is highly sensitive to protocol choices. Varying **submission order** changes Elo by up to **394 points**. **Contest selection** changes it by up to **1,122 points** for the same model. **Run-to-run stochastic variability** changes mean scores by up to **349 points**, with a reported **1,348-point** swing on a single contest. The motivating external discrepancy is that the “same model version,” DeepSeek-R1, was reported once at **2,029** Elo and later at **1,530**.

The paper’s conclusion is not that Elo is unusable, but that benchmark-induced Codeforces Elo is **protocol-dependent**. Exact contest IDs, division mix, number of generated outputs, submission-order policy, local test construction, multi-solution verification, and repeated-run statistics materially determine the reported number. A plausible implication is that Codeforces-based Elo in model evaluation should be read as a controlled experimental estimate rather than as an intrinsic property of a system in isolation.

Taken together, the literature supports a narrow but technically clear synthesis. Codeforces-based Elo ratings are most rigorous when they specify whether Elo refers to **contest-rank inversion**, **future-rating prediction**, or **problem-difficulty calibration**; when they state the transformation from contest outcomes to rating evidence; and when they separate platform-faithful judging from protocol-specific benchmark choices. The concept is therefore best understood as a family of rating and calibration procedures built on Codeforces data, not as a single canonical algorithm.

Source: https://www.emergentmind.com/topics/codeforces-based-elo-ratings