Individual Tendency Learning (ITL)

• Individual Tendency Learning (ITL) is a modeling paradigm that treats individual behavioral regularities as structured signals instead of noise.
• In social dilemmas, ITL employs a trial-and-reflection framework with payoff-dependent exploration to enhance cooperative dynamics on networks.
• In multi-annotator learning, ITL captures annotator-specific labeling tendencies, enabling personalized predictions and richer, behavior-aligned explanations.

Individual Tendency Learning (ITL) is a research term that has acquired at least two distinct technical meanings in recent arXiv literature. In one line of work, ITL denotes a framework for modeling human exploration in social dilemmas as a two-stage process of trial and reflection that coevolves with social imitation; this formulation is used to explain asymmetric exploration and its consequences for cooperation on networks (Hou et al., 11 May 2025). In another line of work, ITL denotes a paradigm in multi-annotator learning that models annotator-specific labeling behavior rather than collapsing disagreement into a single consensus label, and is accompanied by evaluation criteria designed to test whether a model truly captures individual tendencies and their explanations (Zhang et al., 14 Aug 2025). A related but terminologically distinct usage appears in cooperative multi-agent reinforcement learning, where “action tendency” refers to agent-specific policy-value structure and is regularized toward consistency through intrinsic rewards (Zhang et al., 2024). Across these settings, the common theme is that individual-level behavioral regularities are treated as structured signals rather than as noise.

1. Terminological scope and conceptual core

Recent work uses “Individual Tendency Learning” to designate methods that explicitly preserve and model heterogeneity at the level of the individual. In the multi-annotator setting, the contrast is drawn directly against Consensus-oriented Learning (CoL), whose objective is to aggregate multiple annotators’ labels into a single “ground-truth” label per instance, treating all disagreements as noise; typical CoL methods listed in this context include majority voting, probabilistic EM models (Dawid–Skene), confusion-matrix models (MaDL), and Gaussian fitting (PADL) (Zhang et al., 14 Aug 2025). ITL, by contrast, explicitly models each annotator’s unique labeling “tendency”—the systematic pattern of how and when they agree or disagree—and produces annotator-conditional predictions and often annotator-specific explanations (Zhang et al., 14 Aug 2025).

In the social-dilemma literature, ITL is defined differently but retains the same emphasis on individual-level structure. There, the framework developed in Hou et al. models human “exploration” as neither pure random noise nor a fixed mutation rate, but as a multi-step cognitive process of trial and reflection, embedded alongside social imitation (Hou et al., 11 May 2025). The central claim is that exploration intensity can coevolve with social learning, and that this interdependence materially affects the emergence of cooperation.

A useful synthesis is that both usages reject the treatment of deviations from aggregate behavior as unstructured perturbations. In one case, disagreement among annotators is informative about background, bias, expertise, or subjective interpretation; in the other, exploratory deviations in strategic behavior are informative about payoff-sensitive trial-and-error and satisficing (Zhang et al., 14 Aug 2025, Hou et al., 11 May 2025). This suggests that ITL is best understood not as a single method, but as a family of modeling commitments centered on individual-level regularity.

2. ITL in social dilemmas: trial, reflection, and imitation

In the ITL model for cooperation on networks, each update event selects a focal player who, with probability $p_{IL}$, performs individual learning and, with complementary probability $1-p_{IL}$, updates by social imitation (Hou et al., 11 May 2025). Individual learning is formalized as a two-stage process.

Suppose that at time $t$ the focal player has current strategy $s(t)\in\{C,D\}$ with immediate payoff $\pi(t)$. Under individual learning, the player first reverses the current strategy to $1-s(t)$, then engages in $u$ further rounds of play, receiving $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$. The player computes an “experiential cognition”

$$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$

where $(\lambda_1,\lambda_2,\dots,\lambda_u)$ is a weighting kernel over future payoffs (Hou et al., 11 May 2025). If $1-p_{IL}$0, the player retains the reversed strategy; otherwise the player reverts to the original one. Two special cases are identified: Terminal-payoff (TP) focus, where $1-p_{IL}$1 so that $1-p_{IL}$2, and Equal-payoff (EP) focus, where $1-p_{IL}$3 for all $1-p_{IL}$4 (Hou et al., 11 May 2025).

If individual learning is not selected, the player updates by the standard pairwise-comparison rule. The realized payoff $1-p_{IL}$5 is first rescaled into a social value

$1-p_{IL}$6

and a random neighbor $1-p_{IL}$7 is sampled; the focal player adopts $1-p_{IL}$8’s strategy with probability

$1-p_{IL}$9

This embeds ITL within weak-selection imitation dynamics while inserting, with probability $t$0, a multi-step reversal-and-reflection procedure (Hou et al., 11 May 2025).

The resulting framework is explicitly coupled. Individual learning alters the local composition of cooperators around the focal player, which modifies imitative weight under pairwise comparison; social imitation, conversely, shapes the payoffs that enter future trials. Under weak selection, the average change in cooperator density $t$1 is written as

$t$2

where $t$3 is the probability that $t$4 is imitated, $t$5 is the net bias in the pairwise-comparison rule, and $t$6 is the probability that a focal player with strategy $t$7 ends reflection in $t$8 (Hou et al., 11 May 2025). Under TP focus, the transition probabilities are further expressed in matrix form:

$t$9

This formalism makes the feedback between individual and social learning explicit (Hou et al., 11 May 2025).

3. Asymmetric exploration and the promotion of cooperation

A central contribution of the social-dilemma ITL framework is its treatment of exploration-probability as payoff-dependent. To capture the empirical observation that “satisfied” players explore less, the individual-learning probability is written as

$s(t)\in\{C,D\}$0

In the simplest simulation implementation,

$s(t)\in\{C,D\}$1

where $s(t)\in\{C,D\}$2 is the number of cooperator-neighbors (Hou et al., 11 May 2025). Because payoff grows with $s(t)\in\{C,D\}$3, this implements $s(t)\in\{C,D\}$4, so highly paid individuals are less likely to engage in trial-and-error. This negative payoff–exploration correlation produces asymmetric exploration rates $s(t)\in\{C,D\}$5 and is linked in the paper to observations reported by Traulsen et al. and Rand et al. (Hou et al., 11 May 2025).

The analytical and simulation findings distinguish between constant and adaptive exploration. When $s(t)\in\{C,D\}$6 is constant, increasing reflection depth $s(t)\in\{C,D\}$7 causes the critical benefit-to-cost ratio for cooperation, $s(t)\in\{C,D\}$8, to decrease monotonically and approach the ideal network-reciprocity threshold $s(t)\in\{C,D\}$9 (Hou et al., 11 May 2025). Under TP focus, the paper reports an approximate power-law decay

$\pi(t)$0

Deeper reflection therefore makes cooperation easier, but any nonzero constant exploration still raises $\pi(t)$1 above $\pi(t)$2 (Hou et al., 11 May 2025).

The adaptive case changes the qualitative conclusion. When $\pi(t)$3 is negatively correlated with payoff, the individual-learning term in the weak-selection decomposition no longer uniformly penalizes cooperation. For $\pi(t)$4 and $\pi(t)$5, the paper reports $\pi(t)$6 dropping even below $\pi(t)$7, which is interpreted as stable cooperation emerging in very harsh environments (Hou et al., 11 May 2025). The proposed intuition is asymmetric: defectors in low-cooperator clusters, having higher payoffs, explore less, whereas cooperators in low-payoff contexts explore more vigorously, which breaks up defector clusters and seeds new cooperator domains.

The broader significance is that exploration is modeled as an active, payoff-sensitive process rather than blind mutation. The framework thereby addresses mixed evidence for network reciprocity, moody conditional cooperation, and payoff-dependent mutation rates through a unified mechanism in which trial outcomes alter imitation bias and imitation, in turn, shapes future trial payoffs (Hou et al., 11 May 2025).

4. ITL in multi-annotator learning: modeling annotator-specific behavior

In multi-annotator learning, ITL is defined as an alternative to consensus-oriented aggregation. Rather than producing one unified prediction, ITL methods model each annotator’s unique labeling tendency and output annotator-conditional models $\pi(t)$8, one per annotator $\pi(t)$9, often together with annotator-specific explanations such as attention over frames or image regions (Zhang et al., 14 Aug 2025). The motivating claim is that disagreements encode valuable information about personal background, bias, expertise, or subjective interpretation, and that collapsing them into consensus can obscure behavioral structure (Zhang et al., 14 Aug 2025).

The paper "A Unified Evaluation Framework for Multi-Annotator Tendency Learning" introduces evaluation metrics intended to test whether ITL methods actually recover such structure (Zhang et al., 14 Aug 2025). For annotators indexed by $1-s(t)$0, with true labels $1-s(t)$1 and overlap sets $1-s(t)$2 satisfying $1-s(t)$3, the ground-truth inter-annotator consistency matrix $1-s(t)$4 is defined by

$1-s(t)$5

where $1-s(t)$6 is Cohen’s kappa (Zhang et al., 14 Aug 2025). Given model predictions $1-s(t)$7, the predicted consistency matrix $1-s(t)$8 is

$1-s(t)$9

The Difference of Inter-annotator Consistency (DIC) metric is then

$u$0

with lower values indicating better preservation of the true consistency structure (Zhang et al., 14 Aug 2025).

The second metric, Behavior Alignment Explainability (BAE), evaluates whether explanations reflect true behavioral similarity. Ground-truth behavioral similarity is defined by kappa:

$u$1

Model-derived similarity can be computed at the feature level using average feature embeddings

$u$2

with

$u$3

or at the region level for attention-based models using average attention maps

$u$4

with

$u$5

For either model-derived similarity,

$u$6

so higher BAE indicates better alignment (Zhang et al., 14 Aug 2025). Multidimensional Scaling (MDS) is then used to project distances $u$7 into 2D for visual inspection of annotator clustering (Zhang et al., 14 Aug 2025).

This formulation makes ITL in annotation settings a two-part problem: preserving the relational structure of annotator behaviors in predictions, and preserving that same structure in explanations. The explicit separation between tendency capture and explanation faithfulness is one of the paper’s main conceptual contributions (Zhang et al., 14 Aug 2025).

5. Benchmarks, baselines, and empirical findings in multi-annotator ITL

The unified evaluation framework is tested on two datasets: STREET, consisting of urban scene images labeled by 10 annotators across five “impression” dimensions—Happiness, Healthiness, Safety, Liveliness, and Orderliness—and AMER, a video-based emotion recognition dataset with 13 annotators and temporal emotion labels (Zhang et al., 14 Aug 2025). Representative ITL baselines are D-LEMA, PADL, MaDL, and QuMAB. Input preprocessing uses resize to $u$8 and normalization; training follows original protocols per method with the same hyperparameters where applicable; hardware is listed as $u$9 NVIDIA V100 GPUs (Zhang et al., 14 Aug 2025).

The main reported quantitative results show that DIC and BAE differentiate methods more sharply than conventional metrics. On DIC, QuMAB obtains the lowest values across all listed settings, including STREET-Safety at $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$0 and AMER at $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$1, compared with D-LEMA at $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$2 and $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$3, PADL at $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$4 and $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$5, and MaDL at $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$6 and $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$7 (Zhang et al., 14 Aug 2025). The paper further states that, on Street (Safety) and AMER, DIC exhibits a larger discriminative range than Accuracy, Fleiss’ $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$8, and Pearson Corr (Zhang et al., 14 Aug 2025).

On BAE, QuMAB again scores highest in the tabulated results, with feature-level BAE of $\pi(t+1),\pi(t+2),\dots,\pi(t+u)$9 on STREET-Safety and $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$0 on AMER; its region-level values are $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$1 and $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$2, respectively (Zhang et al., 14 Aug 2025). The paper also compares BAE with alternative explainability metrics on Safety and AMER, stating that BAE shows greater variance, with standard deviation approximately $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$3–$$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$4, than Cosine or Gradient (Zhang et al., 14 Aug 2025).

Component	Definition or result	Source
DIC	$$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$5; lower better	(Zhang et al., 14 Aug 2025)
BAE	$$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$6; higher better	(Zhang et al., 14 Aug 2025)
Qualitative finding	QuMAB heatmaps and MDS clusters more closely match ground truth	(Zhang et al., 14 Aug 2025)

Qualitatively, the paper reports that Figure 1 heatmaps for STREET Safety show QuMAB’s predicted $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$7-matrix closely resembling ground truth, while D-LEMA exhibits large structural distortions; Figure 2 MDS projections show QuMAB’s feature-level and region-level embeddings forming clusters that match true high-agreement groupings more closely than competing methods (Zhang et al., 14 Aug 2025). The paper presents these visual analyses as confirmation that DIC and BAE are measuring meaningful structural properties rather than only pointwise predictive accuracy.

The stated limitations are also important for delimiting the applicability of the framework. DIC requires sufficient overlap $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$8 between annotators; very sparse labels may weaken reliability. Region-level BAE is only applicable to attention-based models and yields modest gains. MDS-based interpretability depends on similarity-to-distance conversion choices (Zhang et al., 14 Aug 2025). Proposed improvements include incorporating human-derived signals such as eye-tracking or cursor paths, exploring alignment methods beyond MDS such as Procrustes analysis, and extending the framework to streaming or evolving annotation scenarios with dynamic tendencies (Zhang et al., 14 Aug 2025).

6. Relation to “action tendency” in cooperative multi-agent reinforcement learning

A related but separate research thread studies “action tendency” in cooperative multi-agent reinforcement learning. In "Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning," the tendency of agent $$E(t,\mu)=\sum_{\ell=1}^{\mu}\lambda_\ell \pi(t+\ell),$$9 at time $(\lambda_1,\lambda_2,\dots,\lambda_u)$0 is its vector of Q-values,

$(\lambda_1,\lambda_2,\dots,\lambda_u)$1

and divergent action tendencies among agents are identified as an obstacle to the training efficiency of CTDE-based value-decomposition methods such as VDN and QMIX (Zhang et al., 2024).

The proposed mechanism is not labeled ITL, but it shares the emphasis on modeling individual-level tendency explicitly. Each agent $(\lambda_1,\lambda_2,\dots,\lambda_u)$2 maintains an action model

$(\lambda_1,\lambda_2,\dots,\lambda_u)$3

where $(\lambda_1,\lambda_2,\dots,\lambda_u)$4 is the imagined observation of $(\lambda_1,\lambda_2,\dots,\lambda_u)$5 from neighbor $(\lambda_1,\lambda_2,\dots,\lambda_u)$6’s perspective (Zhang et al., 2024). The action model is trained by supervised regression to the agent’s own Q-values, and its prediction error is converted into a cooperative intrinsic reward

$(\lambda_1,\lambda_2,\dots,\lambda_u)$7

This intrinsic reward penalizes disagreement between an agent’s actual action tendency and what its neighbors predict it will do (Zhang et al., 2024).

To integrate these rewards, the paper introduces Reward-Additive CTDE (RA-CTDE), a factorization of the global TD-loss into per-agent losses and proves gradient-level equivalence to standard CTDE. The augmented reward for agent $(\lambda_1,\lambda_2,\dots,\lambda_u)$8 becomes

$(\lambda_1,\lambda_2,\dots,\lambda_u)$9

and the resulting IAM loss is

$1-p_{IL}$00

(Zhang et al., 2024).

Empirically, the paper reports that embedding IAM into QMIX and VDN yields large gains on SMAC, Google Research Football Academy, and Multi-Agent Particle Environment. The examples given include median win-rate increases on $1-p_{IL}$01 from approximately $1-p_{IL}$02 to above $1-p_{IL}$03, on $1-p_{IL}$04 from approximately $1-p_{IL}$05 to approximately $1-p_{IL}$06, average return on GRF CPT from approximately $1-p_{IL}$07 to $1-p_{IL}$08, and occupancy rate in MPE from approximately $1-p_{IL}$09 to $1-p_{IL}$10 (Zhang et al., 2024). Although this work does not define ITL as such, it shows that the broader research program of explicit tendency modeling extends beyond annotation and social-dilemma settings.

A plausible implication is that the term “tendency” is serving a unifying methodological role across different areas: it names latent, individual-specific behavioral regularities that can be modeled directly, aligned across agents or annotators, and evaluated structurally rather than only through aggregate task reward or consensus accuracy.

7. Interpretive synthesis, misconceptions, and open directions

The most immediate misconception is that ITL denotes a single established framework. Recent arXiv usage does not support that reading. Instead, the term appears in at least two different technical senses: a social-dilemma framework for trial-and-reflection learning coupled to imitation (Hou et al., 11 May 2025), and a multi-annotator learning paradigm centered on annotator-specific prediction and explanation (Zhang et al., 14 Aug 2025). Related work on action tendency consistency in MARL strengthens the case that the underlying idea is broader than any one formalization, but it does not by itself make the terminology uniform (Zhang et al., 2024).

A second misconception is that individual tendencies are merely nuisance variation. All three cited lines of work reject that premise. In the annotation setting, disagreement is explicitly treated as informative about bias, expertise, and interpretation rather than as noise to be removed (Zhang et al., 14 Aug 2025). In social dilemmas, exploratory deviations are modeled as payoff-sensitive, cognitively grounded behavior rather than random mutation (Hou et al., 11 May 2025). In MARL, divergence in action tendencies is treated as a coordination bottleneck that can be regularized through intrinsic rewards (Zhang et al., 2024).

The main methodological commonality is structural evaluation. In the cooperation setting, the relevant structure is the coupled dynamic between imitation bias and trial outcomes. In multi-annotator ITL, it is the inter-annotator consistency matrix and the alignment between behavioral similarity and explanations. In MARL, it is the relation between an agent’s actual Q-vector and neighbor-conditioned predictions of that vector (Hou et al., 11 May 2025, Zhang et al., 14 Aug 2025, Zhang et al., 2024). This suggests that “learning tendencies” is less about fitting isolated labels or actions than about preserving relational organization.

Several open directions are already stated in the underlying works. In multi-annotator ITL, these include richer behavioral ground truth such as eye-tracking and cursor paths, alternative manifold-alignment methods beyond MDS, and dynamic extensions for streaming or evolving tendencies (Zhang et al., 14 Aug 2025). In the social-dilemma setting, the reported dependence on reflection depth $1-p_{IL}$11, payoff-adaptive exploration, and kernel choice indicates that further study of attention span, satisficing, and environment-dependent exploration remains central (Hou et al., 11 May 2025). Taken together, the literature portrays ITL as an emerging family of approaches in which individual-specific regularities are formal model objects, explanatory targets, and, increasingly, primary evaluation units.