Soft Context Formation Coder Overview

• Soft Context Formation Coder is a lossless screen-content coding framework that forms adaptive context by merging statistics from similar causal pixel neighborhoods.
• It employs a three-stage architecture combining context modeling, a global color palette, and residual coding using arithmetic coding to achieve bitrate reductions.
• Recent extensions adapt SCF to YCbCr 4:2:0 and integrate it with VVC, yielding measurable gains in compression efficiency and performance.

Searching arXiv for the cited papers and nearby work on Soft Context Formation / SCF. Soft Context Formation Coder (SCF) most directly denotes a pixel-wise lossless screen-content coding framework in which probabilities are estimated by collecting statistics from previously coded patterns that are similar, rather than relying only on exact hard context matches. In the literature represented here, its canonical form is a screen-content image coder that operates in raster order and combines a context-model based stage, a color palette stage, and a residual coding stage under arithmetic coding (Och et al., 2023). Subsequent work extends this family to improved cross-stage probability pruning, YCbCr 4:2:0 coding, and hybrid integration with VVC for compound screen-content material containing both synthetic and natural regions (Och et al., 26 Aug 2025).

1. Canonical meaning and domain of use

In its established image-coding sense, SCF is a lossless method for screen content images such as desktop screenshots, webpages, text-and-graphics images, and smartphone displays. Its suitability follows from two regularities of such material: repeated local patterns and limited reuse of colors. Rather than coding channels independently in its original RGB 4:4:4 setting, SCF codes the entire RGB triplet at once in its first two stages, which is effective when exact colors recur frequently (Och et al., 2023).

The “soft” in Soft Context Formation refers to how local context is formed. For a current pixel $X$, SCF uses six causal neighboring pixels and gathers statistics from similar previously seen patterns, not just one exact matched neighborhood. The resulting merged histogram defines a probability distribution over candidate colors for the current position. This distinguishes SCF from hard partitioning schemes in which one exact context bin alone determines the symbol model (Och et al., 2023).

A later extension shows that the same coding philosophy can be adapted to YCbCr 4:2:0, even though 4:2:0 does not provide a full three-component color vector at every luma sample. That work preserves the “whole-symbol where possible” principle by coding the $Y$ plane separately and the $CbCr$ pair jointly, rather than abandoning the probabilistic SCF framework (Och et al., 26 Aug 2025).

2. Three-stage coding architecture

SCF processes the image pixel by pixel in raster scan order. Its context formation is based on six causal neighboring pixels, denoted in the paper by a template around the current position $X$. For each distinct pattern encountered during coding, SCF maintains a distribution of associated colors. For the current pixel, the coder retrieves the distributions of similar patterns and merges them into a Stage-1 distribution over candidate RGB colors $\{x_i \mid i \in \mathcal{S}\}$. If the current color is present in that distribution, it is encoded directly by arithmetic coding (Och et al., 2023).

Stage 1 is therefore a context-model based stage. If the current color is absent from the merged histogram, the coder emits an escape signal and proceeds to Stage 2. Stage 2 uses a global color palette $\{x_i \mid i \in \mathcal{P}\}$ containing all previously seen colors together with counts. In the baseline formulation, if the current color is already in the palette, the entire RGB triplet is coded arithmetically using palette statistics. Prior SCF work already sharpened this stage by splitting the palette around a predictor $\hat{x}$ into colors inside a radius and colors outside it, then signaling which sub-palette applies before coding the color (Och et al., 2023).

If the current color is not yet in the palette, the coder falls through to Stage 3. This final stage uses an enhanced median adaptive predictor and codes prediction errors channel by channel using histograms of previously observed residuals. Thus SCF combines three symbol domains within one pipeline: full RGB colors under context-conditioned histograms, full RGB colors under a global palette model, and finally per-channel residuals for genuinely new colors (Och et al., 2023).

Arithmetic coding is the entropy-coding engine in all three stages. The paper states the ideal coding cost in the usual self-information form,

$I(x_i) = -\log_2 p(x_i).$

Because SCF collects statistics during coding and updates its models causally after each pixel, probability estimation is online and image-adaptive rather than fixed in advance (Och et al., 2023).

3. Cross-stage refinements and 4:2:0 generalization

A major refinement of canonical SCF is the explicit removal of impossible symbols from later stages once earlier stages have failed. If Stage 2 is invoked, then none of the colors already present in the Stage-1 support set $\mathcal{S}$ can be the current color. The enhanced palette model therefore removes those colors from the Stage-2 palette, reducing the total palette mass from $N_{\mathrm p}$ to

$Y$0

which raises the probabilities of all remaining feasible colors. The same logic is extended to Stage 3 by pruning residual symbols corresponding to known palette colors from the final-channel residual histogram. The same paper also eliminates some explicitly signaled stage-decision flags when those decisions are already implied by decoder state (Och et al., 2023).

These changes are strictly redundancy-removing rather than architectural replacements. On 173 RGB screen-content images, they reduce average bitrate from $Y$1 bpp for SCF Base to $Y$2 bpp for SCF Base + PRF, corresponding to a $Y$3 average bit-rate decrease. On that corpus, the refined SCF also remains clearly ahead of the cited VVC and HEVC lossless anchors, requiring roughly $Y$4 bpp less than VVC and $Y$5 bpp less than HEVC on average (Och et al., 2023).

The 4:2:0 extension adapts SCF to subsampled chroma by coding $Y$6 first and then coding $Y$7 and $Y$8 jointly as a $Y$9 symbol. This coding order is justified by normalized mutual information analysis: in YCbCr 4:2:0, $CbCr$0-$CbCr$1 dependency remains relatively high, especially for screen content, while luma can later guide chroma prediction. The extension introduces a luma-guided modified predictor (LMAP), which selects top or left chroma when the corresponding downsampled luma matches and otherwise falls back to the modified median adaptive predictor. It also introduces Chroma Range Coding (CRC), a side-information mechanism that transmits occurring luma-chroma combinations so that impossible chroma values can be pruned from Stage-1 histograms, Stage-2 palette candidates, and Stage-3 residual histograms (Och et al., 26 Aug 2025).

In the reported ablation, SCF 420 achieves $CbCr$2 bpp on average, compared with $CbCr$3 bpp without CRC and $CbCr$4 bpp without both CRC and LMAP. Against codec anchors, average bitrates are $CbCr$5 bpp for VTM 17.2, $CbCr$6 bpp for HM-16.21 + SCM-8.8, and $CbCr$7 bpp for SCF 420; equivalently, VTM needs $CbCr$8 more bitrate and HEVC-SCC needs $CbCr$9 more bitrate on the evaluated screen-content datasets (Och et al., 26 Aug 2025).

4. Hybrid coding with VVC

A different line of work treats SCF not as a whole-image replacement for VVC but as a selective coding layer for the synthetic parts of compound screen-content images. The image is divided into $X$0 CTUs, and a learned block classifier predicts whether SCF requires fewer bits than VVC at the current QP. The target condition is whether $X$1, so the classifier is rate-oriented rather than purely semantic. Four block features are used: normalized number of colors, normalized number of unique simplified patterns, entropy of colors, and conditional entropy of pixel color given simplified context. One k-nearest neighbor classifier is trained per QP in $X$2, with 10-fold cross-validation accuracies of $X$3, $X$4, $X$5, and $X$6, respectively (Och et al., 2023).

The hybrid encoder then codes non-SCF blocks with VVC and SCF-favorable blocks with lossless SCF. To do this, all SCF CTUs are set to black before VVC coding, while the SCF coder is modified to skip non-SCF pixels. Crucially, SCF is also conditioned on the already decoded VVC layer. First, decoded VVC pixels are inserted at all non-SCF positions, so Stage-1 contexts and Stage-3 cMAP prediction can use VVC-side reconstructed neighbors at SCF/VVC boundaries. Second, a partial color palette is initialized from the decoded VVC layer; the encoder tests palette prefixes from the full palette down to $X$7 of it and signals a 3-bit parameter $X$8 selecting the best prefix, unless the overlap with SCF-layer colors is too low, in which case no VVC palette initialization is used (Och et al., 2023).

This selective hybrid design yields average BD-rate gains of $X$9 in PSNR, $\{x_i \mid i \in \mathcal{S}\}$0 in SSIM, and $\{x_i \mid i \in \mathcal{S}\}$1 in GFM relative to VTM 17.2. Gains are strongest on more strongly synthetic data sets such as SC-Text and HEVC-CTC, and weakest on SC-Mixed. The method also reports that the SCF share of pixels decreases as QP increases, from $\{x_i \mid i \in \mathcal{S}\}$2 at QP 22 to $\{x_i \mid i \in \mathcal{S}\}$3 at QP 37, which explains why the hybrid gain is larger at lower QP (Och et al., 2023).

Work	Main extension	Reported effect
(Och et al., 2023)	Palette and residual pruning across SCF stages	$\{x_i \mid i \in \mathcal{S}\}$4 average bit-rate decrease over SCF Base
(Och et al., 26 Aug 2025)	Y-first, joint $\{x_i \mid i \in \mathcal{S}\}$5, LMAP, CRC for 4:2:0	HEVC-SCC needs $\{x_i \mid i \in \mathcal{S}\}$6 more bitrate
(Och et al., 2023)	CTU-wise SCF/VVC selection with VVC-guided SCF	$\{x_i \mid i \in \mathcal{S}\}$7 average BD-rate gain vs VVC

5. Broader interpretations beyond screen-content coding

Although SCF has a precise canonical meaning in screen-content lossless coding, several later works use “soft context formation” only by analogy. In octree-based point cloud compression, ECM-OPCC uses learned embeddings and masked transformer attention over ancestor and sibling tokens instead of a handcrafted discrete context index table. Context is therefore formed by weighted combinations of valid tokens under hard autoregressive masks, which is a learned soft context mechanism rather than classic SCF in the screen-content sense (Jin et al., 2022).

A similar analogical extension appears in learned image compression. Contextformer replaces fixed masked-convolution neighborhoods with masked spatio-channel attention over previously decoded latent tokens. The context for each latent symbol is thus a weighted sum over causal spatial and channel predecessors, yielding an adaptive entropy model that the paper itself presents as a stronger form of context adaptivity (Koyuncu et al., 2022).

Outside image and point-cloud compression, the phrase broadens further. Soft-BCT for real-valued time series replaces deterministic context-tree splits with probabilistic routing, so a sample can have nonzero probability for multiple context branches (Saito et al., 16 Jan 2026). ContextCodec for ultra-low bitrate speech transmits a dedicated content-focused context latent aligned with phoneme indices and injects that context at each decoding stage (Liang et al., 9 Jun 2026). In dense retrieval, CODER does not use the phrase explicitly, but its “ranking context” consists of a large query-specific candidate set scored jointly under a softmax-normalized list-wise loss, which the paper interprets as soft contextual scoring rather than explicit context aggregation (Zerveas et al., 2021). In long-context language-model compression, “Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio” uses continuous latent context tokens but executes compression at one of a discrete set of ratios, explicitly to avoid instability from fully continuous structural hyperparameters (Yu et al., 26 Mar 2026). A different direction, Context Codec, treats conversation context as typed semantic commitments rather than as tokens and makes compression verifiable through commitment-level recall and round-trip recoverability (Trukhina et al., 17 May 2026).

These works do not redefine the canonical screen-content SCF algorithm. They instead show that “soft context formation” has become a transferable conceptual label for probabilistic, weighted, learned, or commitment-preserving context construction in compression and retrieval systems.

6. Misconceptions, limitations, and directions

A common misconception is that SCF is inherently a neural or attention-based method. In its canonical form, it is a pixel-wise arithmetic coder that learns image statistics online from similar causal patterns and a palette, not a transformer or latent-variable model (Och et al., 2023). Conversely, another misconception is that any probabilistic or attention-based context model is simply SCF; the broader papers discussed above are better understood as analogical extensions, because several explicitly do not use the phrase in the paper text itself (Zerveas et al., 2021).

Runtime remains a recurring limitation of canonical SCF systems. The enhanced RGB SCF implementation is explicitly described as not optimized for speed, with complexity rising mainly due to extra palette search and residual-stage checks; across 173 images, SCF Base + PRF averages $\{x_i \mid i \in \mathcal{S}\}$8 s encoding and $\{x_i \mid i \in \mathcal{S}\}$9 s decoding on the reported CPU setup, while VTM 17.2 averages $\{x_i \mid i \in \mathcal{P}\}$0 s encoding and $\{x_i \mid i \in \mathcal{P}\}$1 s decoding (Och et al., 2023). The 4:2:0 extension is likewise not optimized, relying on sequential search for palette and context-based coding; it reports $\{x_i \mid i \in \mathcal{P}\}$2 s encoding and $\{x_i \mid i \in \mathcal{P}\}$3 s decoding on average, compared with $\{x_i \mid i \in \mathcal{P}\}$4 s and $\{x_i \mid i \in \mathcal{P}\}$5 s for HM-16.21 + SCM-8.8 (Och et al., 26 Aug 2025). The VVC-hybrid system improves encoding slightly on SCID, $\{x_i \mid i \in \mathcal{P}\}$6 s versus $\{x_i \mid i \in \mathcal{P}\}$7 s for VVC, but decoding remains much slower, $\{x_i \mid i \in \mathcal{P}\}$8 s versus $\{x_i \mid i \in \mathcal{P}\}$9 s (Och et al., 2023).

A second limitation is structural granularity. In the VVC-hybrid work, CTU labels are predicted independently even though both SCF and VVC efficiency depend on causal neighboring CTUs. The paper explicitly identifies this as suboptimal and suggests incorporating already encoded neighboring CTUs into the decision process (Och et al., 2023). In the standalone SCF refinements, the main future opportunity identified is more systematic propagation of impossible-symbol information across stages, together with more intelligent palette search and data structures (Och et al., 2023). For SCF 420, the reported forward path includes dedicated binary range-image coding for CRC and more sophisticated search algorithms as natural optimization targets (Och et al., 26 Aug 2025).

Taken together, these works suggest two complementary futures for the field. One is internal refinement of the canonical screen-content SCF family: stronger cross-stage pruning, better signaling elimination, broader chroma-format support, and faster implementations. The other is external generalization of the core idea: context formation as a soft, adaptive, and causally valid construction of the symbol model, whether through pattern similarity, masked attention, probabilistic routing, or commitment-preserving semantic compression.