---
title: 'Q-jets Formalism: Stochastic Jet Clustering'
url: https://www.emergentmind.com/topics/q-jets-formalism
type: topic
---

# Q-jets Formalism: Stochastic Jet Clustering

The Q-jets formalism is a non-deterministic approach to tree-based jet substructure, introduced to enhance the stability and discrimination power of jet observables by sampling an ensemble of clustering trees rather than producing a single deterministic history. Classical jet clustering algorithms such as $k_T$ or Cambridge/Aachen yield a unique tree for each jet by greedily merging pairs based on a distance metric. Q-jets replaces this deterministic sequence with a stochastic process: at each clustering step, all potential pseudojet pairs are assigned weights and one pair is selected probabilistically, controlled by a rigidity parameter $\alpha$. This procedure is repeated multiple times on the same jet to build an ensemble of possible clustering histories. Observables are then calculated as distributions over this ensemble, enabling improved statistical robustness and new discriminants, such as volatility, that are not accessible via single-tree approaches [1201.1914, 1409.6785].

## 1. Deterministic Versus Non-Deterministic Jet Clustering

Standard tree-based jet algorithms such as $k_T$ and Cambridge/Aachen reconstruct jet substructure by iteratively clustering pairs of proto-jets according to a metric
\[
d_{ij} = 
\begin{cases}
\min(p_{Ti}^2, p_{Tj}^2)\,\Delta R_{ij}^2 & (k_T)\\
\Delta R_{ij}^2 & (\text{Cambridge/Aachen})
\end{cases}, \quad 
\Delta R_{ij}^2 = (\Delta y_{ij})^2 + (\Delta\phi_{ij})^2.
\]
At each step, the pair with the smallest $d_{ij}$ is merged, resulting in a unique, deterministic clustering tree. This uniqueness neglects the inherent ambiguity in the clustering sequence due to soft and collinear emissions, detector effects, and hadronization.

The Q-jets formalism introduces a stochastic element: for each pair $(i,j)$, a weight
\[
\omega_{ij}^{(\alpha)} = \exp\left[-\alpha\,\frac{d_{ij}-d_{\min}}{d_{\min}}\right],
\]
with $d_{\min} = \min_{k<\ell} d_{k\ell}$, is assigned. The probability to select a given pair is then
\[
w_{ij} = \frac{\omega_{ij}^{(\alpha)}}{\sum_{k<\ell} \omega_{k\ell}^{(\alpha)}}.
\]
A random selection according to $w_{ij}$ is made at each step. In the limit $\alpha \to \infty$ one recovers deterministic clustering; for $\alpha \to 0$, the selection is almost uniform. This framework defines a controlled interpolation between deterministic and fully probabilistic clustering [1201.1914].

## 2. Q-jets Algorithm and Implementation

The Q-jets clustering algorithm proceeds as follows:

1. **Initialization:** Given a list of jet constituents, rigidity parameter $\alpha$, and $N_{\rm trees}$ (number of clustering histories to generate).
2. **Tree Sampling:** For each $t=1,\dots,N_{\rm trees}$, perform:
   - Start with the initial four-vectors.
   - While more than one pseudojet exists:
     - Compute all $d_{ij}$. Identify $d_{\min}$.
     - For each pair, compute $\omega_{ij}$ as above.
     - Select a pair probabilistically according to $w_{ij}$.
     - Merge or prune according to chosen grooming procedures (e.g., standard pruning via $z<z_{\text{cut}}$, $\Delta R>D_{\text{cut}}$).
     - Update the list of proto-jets.
   - Store the full clustering history for the current tree.
3. **Observable Construction:** After collecting the ensemble of clustering trees, compute observables (e.g., pruned jet mass) for each tree instance.

A concise pseudocode sketch:

```python
for t in range(N_trees):
    P = initial_particles
    H = []
    while len(P) > 1:
        compute all d_ij, find d_min
        compute omega_ij = exp(-alpha * (d_ij - d_min) / d_min)
        select (i,j) with probability proportional to omega_ij
        optionally apply pruning
        merge (i,j)
        record (i,j) in H
    store H
```
[1201.1914, 1409.6785].

## 3. Statistical Ensemble Observables

Q-jets enables the calculation of distributions for jet observables over the ensemble of trees. Principal quantities include:

- **Mean pruned mass:**
  \[
  \langle m \rangle = \frac{1}{N_{\rm trees}}\sum_{t=1}^{N_{\rm trees}} m^{(t)}
  \]
- **Mass variance (width):**
  \[
  \sigma_m^2 = \langle m^2 \rangle - \langle m \rangle^2 = \frac{1}{N_{\rm trees}} \sum_{t=1}^{N_{\rm trees}} (m^{(t)})^2 - \langle m \rangle^2
  \]
- **Volatility:**
  \[
  \Gamma = \sqrt{\sigma_m^2},\qquad \mathcal{V} = \frac{\Gamma}{\langle m \rangle}
  \]
  Lower volatility typically signals jets containing an intrinsic mass scale (e.g., $W$ decay), while QCD jets show higher volatility.
- **Fractional tagging probability:**
  \[
  \tau = \frac{1}{N_{\rm trees}} \sum_{t=1}^{N_{\rm trees}} \mathbf{1}\bigl[m^{(t)} \in [m_{\rm low}, m_{\rm high}]\bigr]
  \]
  This continuous variable supersedes the classical binary tag, providing enhanced information for statistical analysis.

For weights per clustering history, the ensemble can be represented more generally as
\[
\langle O \rangle_j = \sum_k w_k O^{(k)}, \quad {\rm Var}_j(O) = \sum_k w_k (O^{(k)} - \langle O \rangle_j)^2,
\]
where $w_k$ is the path probability for the $k$th tree [1201.1914, 1409.6785].

## 4. Statistical Advantages and Significance Gains

Q-jets reduces statistical fluctuations in jet measurements by replacing binary event tagging with continuous weights $\tau_j \in [0,1]$. For a sample of $N$ jets, the fractional uncertainty on the number of tagged jets is reduced:
\[
\frac{\delta N_T}{N_T} = \frac{1}{\sqrt{N}} \cdot \frac{1}{\sqrt{1 + \frac{\sigma_\tau^2}{\langle \tau \rangle^2}}} \le \frac{1}{\sqrt{\langle\tau\rangle\,N}} \le \frac{1}{\sqrt{\epsilon N}}
\]
where $\epsilon$ is the tagging efficiency for classical methods. Numerical studies show up to 39% effective luminosity gain for signal significance ($S/\delta B$) when using Q-jets (for pruning + Cambridge/Aachen, $\alpha = 0.1$), and a reduction by a factor of two in required luminosity when volatility is also employed as a cut ($\mathcal{V} \le \mathcal{V}_{\rm cut}$, optimal at $\alpha, \mathcal{V}_{\rm cut} \approx 0.01, 0.03$) [1201.1914].

Q-jets also offers improved mass resolution, with uncertainties on $\langle m \rangle$ reduced by up to 30% relative to deterministic algorithms, and up to 40% reduction in cross-section error as seen for optimal tuning of $\alpha$ ($0.1 \gtrsim \alpha \gtrsim 0.01$) [1409.6785].

## 5. Control Parameters and Optimization

The rigidity parameter $\alpha$ regulates how deterministic the clustering sequence is:

- $\alpha \to \infty$: deterministic clustering (classical limit).
- $0 < \alpha \lesssim 0.1$: optimal regime for statistical stabilization and discrimination.
- $\alpha \to 0$: broad sampling of histories, maximizing statistical ensemble but risking inclusion of non-perturbative/unphysical clusterings.

Parameter tuning considerations:

| Parameter           | Optimal Range              | Effect                                               |
|---------------------|---------------------------|------------------------------------------------------|
| $\alpha$ (rigidity) | $0.01 \lesssim \alpha \lesssim 0.1$ | Minimizes cross-section and mass uncertainty; interpolates between deterministic and fully stochastic regimes |
| $z_\text{cut}$ (pruning)| Around optimized value ($\sim 0.10$) | Fine-tuning further modestly enhances performance    |
| $N_{\rm iter}$      | $200$–$300$ or more       | Stabilizes empirical distributions; CPU cost is linear in $N_{\rm iter}$ |

Q-jets is most effective when the classical tagger is ambiguous or many jets are near selection thresholds, where the continuous tagger smooths sample-to-sample fluctuations [1409.6785].

## 6. Observable Generalization and Theoretical Properties

The Q-jets methodology generalizes directly to any tree-based jet observable: mass drop, $N$-subjettiness, pull, and others may be promoted to their Q-jets versions by measuring distributions over the sampled tree ensemble. The approach may be interpreted as a Monte Carlo integration over the space of all possible clustering histories, with importance sampling governed by $\alpha$.

The formalism is infrared and collinear safe and regularizes statistical distributions without altering the underlying physical biases from fragmentation, pileup, or detector effects. Any grooming strategy (pruning, trimming, filtering) can be applied within the Q-jets algorithm by modifying the merge/prune decisions at each clustering step [1201.1914].

## 7. Limitations, Caveats, and Computational Considerations

Q-jets does not address intrinsic biases in jet reconstruction from hadronization or pileup, nor does it “fix” the physics underlying event structure. Its efficacy is reduced when the deterministic algorithm already provides clear separation; with very high or very low $\alpha$ the statistical gains diminish or may be offset by variance from unphysical histories. The requirement of $N_{\rm iter} \gtrsim 200$–$300$ for stable distributions increases computational cost linearly.

Overall, Q-jets replaces deterministic jet reconstruction with a stochastic ensemble approach, reducing the sampling variance of both signal and background and enabling new, discriminating observables such as volatility, thereby improving sensitivity to new physics and mass resolution at a controlled computational cost [1201.1914, 1409.6785].

Source: https://www.emergentmind.com/topics/q-jets-formalism