Factions Within, Uncertain Across: Within-Document Reader Sub-Groups in Social Highlighting

Abstract: When many people highlight the same document, is the crowd a single consensus, or is it internally structured into reader sub-groups that mark different things -- and is that structure a stable property of a reader or of the document? Building on prior work showing an individual's within-document highlighting signal is a whisper while individuality lives in selection, we ask the group-level question on a co-readership platform using a margin-preserving curveball null. Experiment 1: within a document, readers form strong sub-groups -- pairs agree far beyond what shared salience, mark density, and sentence popularity predict (nearest-neighbour agreement z=+6.3, significant in 88% of documents). Under an eight-block region-preserving null, shared engagement with the same coarse regions of the document accounts for about 40% of this excess; the majority survives as finer reader-specific agreement (z=+3.6, 77% significant). So the within-document crowd is, in a descriptive sense, factional. Experiment 2: is that grouping a stable reader trait? Here we are honest about power. The cross-document split-half reproducibility of a pair's agreement is near zero pooled (+0.078 and 0.000 in two separately drawn samples), and a power calibration shows the test is informative only for pairs that co-read many documents. In the only informative high-overlap subset (k>=4), point estimates are positive but small-sample, imprecise across the separately drawn samples, never significant, and attenuate under the region-preserving null. We therefore leave cross-document stability unresolved: the data is consistent with anything from situational grouping to a weak-to-moderate stable reader trait. The crowd is factional within a document; whether its factions follow the reader across documents is, honestly, beyond our reach.

• The paper demonstrates that readers form robust, fine-grained factions within documents (z = +6.3, 88% significance).
• It employs innovative margin- and region-preserving permutation tests to isolate genuine subgroup signals beyond shared regional focus.
• Despite strong within-document factional behavior, the study finds unresolved cross-document stability, limiting history-based personalization.

Sub-Group Structure and Stability in Social Highlighting: A Technical Review

Introduction and Context

The paper "Factions Within, Uncertain Across: Within-Document Reader Sub-Groups in Social Highlighting" (2606.11613) systematically investigates the aggregate structure of reader highlighting behavior using data from a large-scale social highlighting platform. The motivating question is whether social signals like "popular highlights" on documents represent a true consensus, or instead are composed of internally-structured reader sub-groups ("factions") that consistently attend to different textual regions. The study further queries whether such sub-group structure is specific to documents, or whether similar factional alignments recur across multiple documents—an issue central to the design of personalization methods based on user history.

This work extends prior analyses of individual-level highlighting behavior ("Personal Salience" (Nakayashiki et al., 8 Jun 2026), "Selection, Not Salience" (Nakayashiki et al., 9 Jun 2026)), reframing the problem at the group level. Key technical innovations include the deployment of margin- and region-preserving curveball permutation nulls to rigorously quantify sub-group agreement, alongside a power-calibrated cross-document stability test.

Within-Document Sub-Groups: Strong, Fine-Grained Structure

The core empirical result (Experiment 1) demonstrates that, within a given document, readers form robust sub-groups whose patterns of sentence-level agreement significantly exceed what is expected under null models that control for shared salience, highlight density, and generic popularity effects. The primary metric—nearest-neighbor agreement—is $z = +6.3$ (95% CI $[5.4, 7.3]$), with $88\%$ of documents individually significant (Figure 1).

Figure 1: Per-document reader-agreement structure shows a right-shifted distribution for the effect (green) relative to robust nulls, indicating strong sub-group structure within documents.

Further decomposition reveals that only $\sim 40\%$ of this agreement excess is attributable to shared engagement with the same document regions—quantified via an 8-block region-preserving null that locks each reader's markings into coarse sections. The remaining $\sim 60\%$ persists even after matching on region, signaling finer, reader-specific agreement (mean $z = 3.6$, $77\%$ of documents significant).

This analytic framework robustly rules out artifacts from data sparsity, duplicate readers, or document-shortness, and is validated by synthetic controls, ensuring the observed sub-group structure represents genuine heterogeneity in reader interests.

Cross-Document Stability: Unresolved Under Current Data Density

Experiment 2 interrogates whether the observed within-document group structure reflects stable reader traits—that is, whether the same reader pairs consistently form sub-groups across multiple co-read documents. This property is critical for enabling history-based personalization: without cross-document consistency, inferences from past behavior would yield little predictive power for new documents.

The statistical test here employs a split-half correlation of excess pairwise agreement across all shared documents ($k \geq 2$) for each reader pair, stratified by number of shared documents to guard against power dilution. Calibration on synthetic datasets establishes that this approach is only powered for pairs with $k \geq 4$ shared documents. Most pairs, however, share only two, providing effectively no signal.

Results for the informative $k \geq 4$ subset yield positive but highly imprecise and non-significant estimates (e.g., $[5.4, 7.3]$0 sample 1, $[5.4, 7.3]$1 sample 2, $[5.4, 7.3]$2 under the region-preserving null; all confidence intervals span zero). No stratum achieves statistical significance (Figure 2).

Figure 2: Cross-document stability estimates by shared-document count show positive but inconclusive and variable estimates at high overlap, all CIs crossing zero.

Pooling across all pairs (including those with no statistical power) would misleadingly suggest a null effect. The study cautions that with the current data's co-readership density, neither the existence nor strength of a stable cross-document sub-grouping can be discerned—the question remains open (Figure 3).

Figure 3: Visual synthesis illustrates clear within-document sub-groups (left), but ambiguity regarding their cross-document stability (right).

Implications and Methodological Contributions

Theoretical and Practical Relevance

The unambiguous presence of strong within-document sub-group structure challenges the consensus interpretation often attached to aggregated highlight signals. This directly impacts interface and algorithm design for social reading platforms. Instead of surfacing a single highlights map, it is optimal to acknowledge local diversity and consider exposing multiple "readers marked this way" views for each document.

However, the unresolved status of cross-document stability has important limiting consequences for personalized recommendation or auto-summary systems that seek to leverage historic user data to segment or anticipate preferences for unseen documents. The lack of evidence for persistent, document-independent reader factions precludes such methods from reliably inferring future highlight patterns based solely on history.

Methodological Rigor

Key methodological contributions of the paper include:

• Deploying margin-preserving (curveball) permutation nulls to rigorously control for shared salience and density, isolating true sub-group signals.
• Introducing region-preserving permutation nulls to further differentiate between regional focus and finer reader-specific agreement.
• Power-calibrated, $[5.4, 7.3]$3-stratified split-half correlation tests, with honest quantification of their applicability and limitations.
• Use of robust synthetic controls and bootstrapped confidence intervals to ensure the validity and interpretability of all statistical claims.

These approaches provide a template for future research seeking to decompose subgroup signals in behavioral or crowdsourced annotation data.

Future Directions

Further progress in resolving cross-document sub-group stability will require substantial increases in both the number of readers per document and the overlap in document co-readership, or alternative data sources such as longitudinal tracking in more specialized domains. New experimental designs could also address annotation structure at higher levels of selection (e.g., topic, genre) where stable individual differences are more likely to manifest.

Advanced modeling frameworks—potentially using Bayesian or deep generative approaches capable of jointly modeling document- and reader-level heterogeneity—could offer greater sensitivity, provided sufficient data density.

The question of whether stable, generalizable reader factions exist remains critical for the development of adaptive, user-centered reading and annotation technologies.

Conclusion

This study delivers strong evidence for the presence of robust, fine-grained sub-group structure in collective reader highlighting behavior within documents: the "crowd" is factional rather than unitary, and this structure cannot be reduced to shared regional focus alone. However, the stability of these factions across documents—the necessary condition for portable personalization—remains unresolved, due to power constraints inherent in current data densities. These findings reframe crowd-consensus assumptions in social annotation and set clear boundaries for the design of personalization systems relying on highlight aggregation. Methodologically, the paper establishes rigorous statistical and calibration protocols for subgroup analysis in collaborative behaviors, with wide applicability to future behavioral and annotation-based research.