Engagement Metrics: Definitions and Applications

• Engagement metrics are quantitative measures that capture user responses across domains by analyzing temporal persistence, interaction counts, and latent behavioral signals.
• They utilize robust normalization and calibration methods to adjust for duration, context, and platform biases, ensuring accurate cross-domain comparisons.
• These metrics are applied for both retrospective analysis and real-time predictive modeling in settings such as online video, education, social media, and human–robot interaction.

Searching arXiv for recent and foundational work on engagement metrics across interactive media, social platforms, education, and HRI. arxiv_search(query="engagement metric interactive media user engagement arXiv", max_results=10, sort_by="relevance") arxiv_search(query="site:arxiv.org engagement metric user engagement", max_results=10, sort_by="relevance") An engagement metric is a formal quantitative measure intended to capture how strongly a user, audience, or interacting agent responds to content, a platform, or an interaction episode. In the literature considered here, engagement is operationalized through temporal persistence, interaction counts, watch behavior, sentiment-bearing feedback, structural position in an interaction network, or learned latent scores, depending on domain and data availability (Carlton et al., 2021, Wu et al., 2017, Kowalczyk et al., 2019, Johnston et al., 16 Jul 2025). These metrics are used both retrospectively, to analyze audience response after an experience, and operationally, to support real-time adaptation, recommendation, early-warning systems, or interaction control (Carlton et al., 2021, Johnston et al., 16 Jul 2025, Ghazarian et al., 2019).

1. Measurement problem and scope

A recurring motivation in this literature is that naive proxies such as raw views, total turns, or simple activity counts are often insufficient. In interactive media, engagement is “typically measured by asking samples of users to self-report,” but this is “time consuming and expensive,” which motivates inference from interaction logs (Carlton et al., 2021). In online video, view count is treated as inadequate because time spent watching is “a more appropriate measure for resource allocation problems in attention, networking, and promotion activities” (Wu et al., 2017). In dialogue evaluation, conversation-level heuristics such as turn counts and total duration are criticized because they can only be computed post hoc, conflate length with interest, and are brittle (Ghazarian et al., 2019). In online education, simple count indicators and retrospective predictive methods are presented as problematic for real-time engagement monitoring (Johnston et al., 16 Jul 2025).

The measurement target is also domain-dependent. Some works quantify user attention to media objects, such as videos, tweets, or interactive stories (Wu et al., 2017, Kowalczyk et al., 2019, Carlton et al., 2021). Others quantify engagement with learning platforms through Moodle or VLE logs (Johnston et al., 16 Jul 2025, Johnston et al., 2024). Still others define engagement over interpersonal or agent interactions: utterance-level dialogue turns (Ghazarian et al., 2019), human–robot interaction video streams (Duchetto et al., 2020), or conflict escalation in social robot navigation (Probst et al., 16 Sep 2025). This suggests that “engagement metric” is not a single canonical observable but a family of operationalizations tied to a task, modality, and decision context.

2. Principal mathematical forms

The literature contains several recurrent formulations.

Metric family	Representative definition	Example domains
Temporal/behavioral	$T_{\mathrm{comp}}$, $\Delta t_i$ bins, NAWP, ECR	interactive media, short videos
Relative or normalized interaction rate	$C/V$, $(P+N)/V$, follower-based ratios	YouTube, conference X accounts
Composite latent score	PCA-derived $E_1$, summed IDF scores, metadata-plus-sentiment sums	Twitter, VLEs, e-learning
Learned continuous score	$e_i=f_\theta(q_i,r_i)$, $y_t\in[0,1]$	dialogue, HRI

In interactive media, one formulation begins from a sequence of timestamped events $\{t_i\}_{i=1}^{M}$ and derives active session length, active time to completion, inter-event intervals, pause bins, and counts of narrative events and control actions (Carlton et al., 2021). The active session length is

$$T_{\mathrm{session}}=(t_M-t_1)-T_{\mathrm{hidden}},$$

and active time to completion is

$$T_{\mathrm{comp}}=(t_{\mathrm{end\_node}}-t_1)-T_{\mathrm{hidden}}.$$

Inter-event intervals $\Delta t_i$0 are further classified into short, medium, long, and very long pauses, yielding both counts and relative frequencies (Carlton et al., 2021).

In online video, absolute watch time, average watch time, and average watch percentage are distinguished, and a further calibration step produces relative engagement. If $\Delta t_i$1 is video duration, then

$\Delta t_i$2

and relative engagement $\Delta t_i$3 is defined as the percentile of $\Delta t_i$4 among videos of similar length (Wu et al., 2017). A related short-video formulation introduces NAWP and ECR. NAWP linearly normalizes average watch time against a duration-dependent upper envelope $\Delta t_i$5, while ECR is the fraction of initial viewers who watch more than 5 seconds (Li et al., 2024).

In social media, a prominent composite formulation treats retweets, replies, and favorites as a joint latent signal. After the variance-stabilizing transform $\Delta t_i$6, the one-dimensional compound score is the first principal component,

$\Delta t_i$7

with reported 2017 weights $\Delta t_i$8 and means $\Delta t_i$9 (Kowalczyk et al., 2019). In e-learning, LLM-SEM defines a per-video score

$C/V$0

where $C/V$1 and $C/V$2 are min–max-normalized views and likes, and $C/V$3 is average sentiment polarity from comment-level scores (Hamdi et al., 2024).

Other formulations remain explicitly ratio-based. For YouTube, commenting frequency is $C/V$4, voting frequency is $C/V$5, and voting balance is $C/V$6, with per-thousand-view variants $C/V$7 and $C/V$8 (Liikkanen, 2013). For conference accounts on X, follower/following ratio, repost-and-mention ratio, and interactor ratio are used as account-level engagement or influence descriptors (Ortiz et al., 2 Mar 2025). In disinformation analysis, a weighted interaction score is normalized by campaign volume:

$C/V$9

with weights fixed in the paper at $(P+N)/V$0 for views, $(P+N)/V$1 for likes, $(P+N)/V$2 for comments, and $(P+N)/V$3 for shares (Rushing et al., 17 Oct 2025).

3. Normalization, calibration, and aggregation

Normalization is central because raw engagement observables are often incomparable across durations, chapters, users, or platforms. Several works apply min–max scaling to each metric before modeling. In interactive media, each feature $(P+N)/V$4 is scaled to $(P+N)/V$5 by

$(P+N)/V$6

and any feature with more than 50% zero values is binarized (Carlton et al., 2021). In chapter-based VLE engagement, frequency, immediacy, and diversity are each min–max scaled across students for chapter $(P+N)/V$7 at week $(P+N)/V$8, then summed as

$(P+N)/V$9

and accumulated as

$E_1$0

with $E_1$1 in that study (Johnston et al., 16 Jul 2025).

Other normalization schemes correct specific biases. Relative engagement in online video explicitly removes the length dependence of average watch percentage by ranking a video against length peers (Wu et al., 2017). NAWP similarly rescales average watch time against a duration-dependent ceiling, while ECR isolates early retention and is reported to be nearly independent of duration (Li et al., 2024). On Twitter, log transformation plus PCA is used to stabilize heavy-tailed counts and extract a single latent factor (Kowalczyk et al., 2019).

Aggregation strategy strongly affects interpretation. In dialogue, utterance-level scores $E_1$2 are aggregated by $E_1$3, $E_1$4, or mean, with mean performing best against human conversation judgments (Ghazarian et al., 2019). In HRI, engagement is continuous at frame level, with $E_1$5 at 10 fps (Duchetto et al., 2020). In social robot navigation, engagement has a different meaning: it is the normalized share of conflict escalation attributable to an agent,

$E_1$6

with $E_1$7 (Probst et al., 16 Sep 2025). A plausible implication is that “engagement” may denote either user involvement or interaction intensification, and the surrounding formalism determines which interpretation applies.

4. Predictive modeling and real-time estimation

Engagement metrics are frequently used as targets in predictive models. In interactive media, engagement is posed as binary classification using L$E_1$8-penalized logistic regression, with the label defined by a median split on User Engagement Scale score. The model achieves AUC $E_1$9 on a held-out test set of $e_i=f_\theta(q_i,r_i)$0, with Precision $e_i=f_\theta(q_i,r_i)$1, Recall $e_i=f_\theta(q_i,r_i)$2, and $e_i=f_\theta(q_i,r_i)$3 (Carlton et al., 2021). Session length and time to completion correlate positively with engagement, and SHAP analysis identifies high $e_i=f_\theta(q_i,r_i)$4 as a robust driver of predicted high engagement, while high $e_i=f_\theta(q_i,r_i)$5 pushes prediction toward low engagement (Carlton et al., 2021).

On Twitter, the compound score $e_i=f_\theta(q_i,r_i)$6 is predicted using LightGBM with 31 early-available features. On the multilingual 2017 dataset, the compound target reaches $e_i=f_\theta(q_i,r_i)$7, Spearman $e_i=f_\theta(q_i,r_i)$8, and RMSE $e_i=f_\theta(q_i,r_i)$9, outperforming the single metrics retweets, replies, and favorites (Kowalczyk et al., 2019). Horn’s Parallel Analysis shows that only the first empirical eigenvalue exceeds its 95% null quantile, and the first principal component explains 72% of the variance in 2017 data, 77% in 2018, and 83% in the 2016 image-only dataset (Kowalczyk et al., 2019).

Cold-start video prediction produces similarly strong results. For YouTube, ridge regression using context, topics, and channel reputation predicts relative engagement directly with $y_t\in[0,1]$0 and average watch percentage via the inverse engagement map with $y_t\in[0,1]$1 (Wu et al., 2017). For short videos, a multimodal model over 90,000 Snapchat videos predicts NAWP with SRCC $y_t\in[0,1]$2, PLCC $y_t\in[0,1]$3, and RMSE $y_t\in[0,1]$4, and predicts ECR with SRCC $y_t\in[0,1]$5, PLCC $y_t\in[0,1]$6, and RMSE $y_t\in[0,1]$7 (Li et al., 2024).

Real-time or near-real-time estimation is explicit in several domains. The chapter-based VLE metric is cumulative and training-free, supporting weekly updates and identifying students from week 2 or 3 onward (Johnston et al., 16 Jul 2025). In dialogue, utterance-level predictive engagement is generated by a BERT-based scorer and aggregated in real time, with Pearson correlation $y_t\in[0,1]$8 between human conversation scores and mean-aggregated human utterance scores (Ghazarian et al., 2019). In HRI, a ResNeXt-50 plus one-layer LSTM regresses frame-level engagement with test MSE $y_t\in[0,1]$9 and transfers to UE-HRI with AUC $\{t_i\}_{i=1}^{M}$0 after thresholding the continuous output (Duchetto et al., 2020).

5. Domain-specific instantiations

The same label is attached to markedly different observables across domains. In education, cumulative IDF-based metrics use study-session frequency, immediacy, and diversity to monitor behavioral engagement from Moodle or VLE logs (Johnston et al., 16 Jul 2025, Johnston et al., 2024). In the chapter-based study, Foundations and Stochastic modules achieve alignment with the course-wide metric above $\{t_i\}_{i=1}^{M}$1 by weeks 3–4, while Programming shows substantially weaker signal (Johnston et al., 16 Jul 2025). In a separate Moodle study, the reduced GAM reaches RMSE $\{t_i\}_{i=1}^{M}$2 and $\{t_i\}_{i=1}^{M}$3 when predicting the aggregate engagement metric from weekly resource interactions (Johnston et al., 2024).

In media and platform analytics, engagement may refer to watch persistence, explicit reactions, or compound audience response. YouTube studies use commenting frequency, voting frequency, voting balance, average watch percentage, and relative engagement (Liikkanen, 2013, Wu et al., 2017). Conference analysis on X reports that likes are the most common form of interaction, with sample median $\{t_i\}_{i=1}^{M}$4 and mean $\{t_i\}_{i=1}^{M}$5, whereas replies are least common, with sample median $\{t_i\}_{i=1}^{M}$6 and mean $\{t_i\}_{i=1}^{M}$7 (Ortiz et al., 2 Mar 2025). Related-video recommendation treats offline engagement as Recall@K over held-out co-engagement pairs and reports a two-week live A/B result of $\{t_i\}_{i=1}^{M}$8 on a proprietary topline engagement metric while increasing topic match rate from 51% to 63% (Jaspal et al., 12 Jul 2025).

In networked communities, engagement can be proxied structurally. Stack Exchange work models users as nodes and question–answer interactions as directed weighted edges with weight $\{t_i\}_{i=1}^{M}$9, then analyzes Degree Centrality, Betweenness Centrality, and PageRank as engagement descriptors (Saha et al., 2024). Smaller communities such as Generative AI show higher mean Degree Centrality and PageRank with much larger standard deviations, whereas larger sites exhibit more distributed engagement (Saha et al., 2024).

In conflict-oriented interaction settings, the meaning shifts again. Social robot navigation defines engagement as an agent’s share of conflict intensification rather than audience attention (Probst et al., 16 Sep 2025). In disinformation campaigns, engagement effectiveness is defined as weighted interactions per attacker-generated transmission (Rushing et al., 17 Oct 2025). These cases broaden the term’s technical scope but also make cross-domain comparison non-trivial.

6. Interpretation, misconceptions, and limitations

A common misconception is that raw exposure metrics are sufficient. The cited work repeatedly argues otherwise: views do not adequately capture watch persistence (Wu et al., 2017); average watch time is biased by duration (Li et al., 2024); likes are only one component of a broader response structure (Kowalczyk et al., 2019, Ortiz et al., 2 Mar 2025); and turn count or conversation duration can misclassify confused or stalled dialogue as engaging (Ghazarian et al., 2019). Another misconception is that engagement is always multidimensional. On Twitter, Horn’s Parallel Analysis provides strong evidence for a one-dimensional engagement signal built from retweets, replies, and favorites (Kowalczyk et al., 2019). A plausible implication is that dimensionality is empirical and domain-specific rather than definitional.

The literature also documents substantial limitations. Several educational metrics capture only online behavioral engagement and omit emotional or cognitive dimensions; off-platform work, such as local IDE activity, may be invisible (Johnston et al., 16 Jul 2025). LLM-SEM reports no external validation against survey or learning-outcome data and is limited to Arabic YouTube comments and metadata (Hamdi et al., 2024). Weighted interaction schemes for disinformation are theory-driven rather than fitted to cognitive outcomes, do not deduplicate users, and assume equal importance of transmissions (Rushing et al., 17 Oct 2025). Ratio-based media metrics can be distorted by interface changes, bot activity, or inconsistencies in how views are counted (Liikkanen, 2013). HRI and dialogue scorers are subject to domain shift, lighting, occlusion, viewpoint change, and the limitations of text-only labeling (Duchetto et al., 2020, Ghazarian et al., 2019).

Taken together, these studies present engagement metrics not as interchangeable scores but as calibrated operational definitions. Their validity depends on the target phenomenon, the available signals, the normalization scheme, and the downstream decision problem. The literature therefore favors metrics that are explicit about what is being measured, how nuisance factors are removed, and whether the resulting score is intended for descriptive analysis, prediction, ranking, or real-time intervention (Carlton et al., 2021, Wu et al., 2017, Johnston et al., 16 Jul 2025).