Profile Distance Between Taskifications

• Profile distance between taskifications is a structural, model-independent metric that compares task-level distributions from different temporal partitions.
• It leverages empirical plasticity and stability profiles, using measures like the first-order Wasserstein distance to capture local shifts and long-range recurrence.
• Empirical results demonstrate that taskification choices significantly influence continual learning performance through variations in model adaptation and evaluation stability.

Profile distance between taskifications is a structural, model-independent measure for comparing two ways of partitioning the same temporally ordered data stream into tasks. In streaming continual learning, a temporal taskification induces a specific sequence of task-level distributions, and therefore a specific continual-learning regime. The framework introduced in "Temporal Taskification in Streaming Continual Learning: A Source of Evaluation Instability" formalizes this idea through plasticity and stability profiles, then defines a profile distance between taskifications as a weighted combination of distances between those profile distributions (Filat et al., 23 Apr 2026).

1. Temporal taskification as a formal object

A temporal taskification is an ordered partition of a fixed time horizon $[0,T]$ into contiguous intervals,

$$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$

with task $k$ given by

$I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$

Each interval induces a task-level distribution

$P_k^\tau,$

so a taskification corresponds to the sequence

$(P_1^\tau,\dots,P_{K_\tau}^\tau).$

Different valid taskifications of the same stream can differ in the number of tasks $K_\tau$, in the placement of boundaries relative to drifts, events, or cycles, and in the induced pattern of adjacent and long-range distribution changes (Filat et al., 23 Apr 2026).

This formulation treats temporal partitioning not as a neutral preprocessing step but as a structural evaluation choice. The same underlying stream can therefore yield different continual-learning regimes solely because its boundaries are placed differently. A central implication is that benchmark conclusions can vary even when the stream, model, and training budget are held fixed.

2. Plasticity and stability profiles

To compare taskifications with different task counts, the framework does not retain the induced task sequence as a fixed-length object. Instead, it builds two empirical distributions that summarize the regime induced by a taskification.

For a taskification $\tau$, let $d(\cdot,\cdot)$ be a distance or divergence between task distributions. The plasticity profile is constructed from the discrepancies between consecutive tasks,

$\{ d(P_k^\tau,P_{k+1}^\tau) \}_{k=1}^{K_\tau-1},$

treated as samples from an empirical distribution $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$0. The stability profile is constructed from longer-range relations,

$$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$1

treated as samples from an empirical distribution $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$2, where $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$3 excludes immediate neighbors so that the profile reflects longer-range recurrence rather than local transitions (Filat et al., 23 Apr 2026).

The plasticity profile summarizes how often adjacent tasks are small, medium, or large shifts. The stability profile summarizes how similar or dissimilar distant parts of the stream are under the same taskification. Because both are empirical distributions over induced discrepancies rather than task-indexed vectors, the representation is task-count-invariant. This makes it possible to compare, for example, 9-day, 30-day, and 44-day windowings without forcing a one-to-one alignment of tasks.

In the continual-learning interpretation used in the paper, plasticity profiles characterize local adaptation demands, while stability profiles characterize long-range recurrence and variation. Large adjacent-task discrepancies correspond to regimes with stronger plasticity requirements and potentially higher forgetting risk. Long-range recurrence patterns characterize whether later tasks revisit earlier distributions and therefore whether positive backward transfer is structurally plausible.

3. Definition of profile distance and Boundary-Profile Sensitivity

Let $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$4 and $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$5 be two taskifications. Let $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$6 denote a distribution distance between their plasticity profiles, and let $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$7 denote the corresponding distance between their stability profiles. The overall profile distance is defined as

$$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$8

with scaling coefficients $$\tau = (t_0,\dots,t_K), \qquad 0=t_0<t_1<\dots<t_K=T,$$9 (Filat et al., 23 Apr 2026).

In the experiments, $k$0 and $k$1 are assigned equal weight,

$k$2

The paper leaves $k$3 and $k$4 abstract as distribution distances. For the underlying discrepancy $k$5 between task-level distributions, the implemented experiments use the first-order Wasserstein distance.

The quantity $k$6 is small when two taskifications induce similar patterns of adjacent-task change and similar patterns of longer-range recurrence, and large when they induce substantially different continual-learning regimes. Its scope is explicitly structural: it compares regime statistics rather than task identities.

Boundary-Profile Sensitivity (BPS) extends the same formalism from comparison across taskifications to local perturbations of a single taskification. For a taskification $k$7 and perturbation scale $k$8, let $k$9 denote the set of valid taskifications obtained by perturbing each internal boundary of $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$0 by at most $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$1 in time. Then

$I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$2

In implementation, the boundary neighborhood is generated by random perturbation of each boundary by up to $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$3 day (Filat et al., 23 Apr 2026).

Low BPS indicates that small boundary perturbations leave the induced plasticity and stability profiles nearly unchanged. High BPS indicates structural fragility: small perturbations can move the benchmark into a materially different continual-learning regime. The paper illustrates this with three synthetic cases: an abrupt changepoint, a narrow transient, and phase-sensitive recurrence. In each case, small shifts in task boundaries can change whether important structure appears within tasks or across tasks, thereby altering the induced profiles before any continual-learning model is trained.

4. Empirical behavior in streaming continual learning

The framework is evaluated on network traffic forecasting with CESNET-Timeseries24 while keeping the stream, model, and training budget fixed and varying only the temporal taskification. The study compares 9-day, 30-day, and 44-day splits and evaluates continual finetuning, Experience Replay, Elastic Weight Consolidation, and Learning without Forgetting (Filat et al., 23 Apr 2026).

At the distributional level, 9-day windows yield noisy and irregular task-to-task first-order Wasserstein patterns, while 30-day and 44-day windows yield smoother patterns. This is reflected in the profile distances between taskifications. The reported pairwise $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$4 values, averaged over 100 IPs, are:

• 9 days vs 30 days: $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$5,
• 9 days vs 44 days: $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$6,
• 30 days vs 44 days: $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$7.

These values indicate that the 9-day and 44-day taskifications induce the most different regimes, while the 30-day and 44-day taskifications are structurally more similar.

The same pattern appears in Boundary-Profile Sensitivity. Under the standard alignment, the reported BPS values are $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$8 for 9-day splits, $I_k^\tau := [t_{k-1},t_k), \quad k=1,\dots,K.$9 for 30-day splits, and $P_k^\tau,$0 for 44-day splits. After shifting all boundaries forward by 2 days, the values remain ordered in the same way: $P_k^\tau,$1, $P_k^\tau,$2, and $P_k^\tau,$3, respectively. The ordering $P_k^\tau,$4 is therefore robust across alignment choices.

Downstream continual-learning metrics change materially with taskification alone. The 30-day split yields the lowest average MSE for all four methods. The 44-day split is often much worse; for Experience Replay, the reported average MSE changes from $P_k^\tau,$5 on the 30-day split to $P_k^\tau,$6 on the 44-day split. For continual finetuning on the 44-day split, the paper reports $P_k^\tau,$7 and Forgetting $P_k^\tau,$8 in scaled units. The study therefore links structural diagnostics and downstream behavior: shorter taskifications induce noisier distribution-level patterns, larger structural distances, and higher BPS, and taskification alone can materially affect continual-learning evaluation.

5. Interpretation, methodological role, and limitations

The main methodological claim is that temporal taskification is a first-class evaluation variable. Profile distance and BPS are designed to characterize this variable before any continual-learning model is trained, and thus to separate structural benchmark effects from learner-specific effects (Filat et al., 23 Apr 2026).

A common misconception is to treat different valid temporal splits of the same stream as interchangeable. The framework rejects that interpretation. Two taskifications may be applied to the same underlying data stream and still induce different local adaptation demands, different long-range recurrence patterns, and different benchmark conclusions. Profile distance is intended to detect such differences without aligning tasks one by one and without fitting any continual-learning model.

The framework is diagnostic rather than prescriptive. It does not propose an automatic optimal-taskification search, and it does not claim invariance of any learning algorithm to taskification. Its assumptions are also explicit. Temporal ordering must be meaningful; the stream is non-i.i.d. with non-stationarity and possible recurrence; and within each task window, samples are treated as i.i.d. from $P_k^\tau,$9 for the purpose of estimating Wasserstein distances. The empirical scope is limited to CESNET-Timeseries24 and to four continual-learning strategies. The paper also studies fixed-length windows and small random shifts, noting that adaptive or distribution-informed taskifications may require separate analysis.

Within those limits, the proposed workflow is straightforward: estimate task-level distributions for a candidate taskification, compute task-pair discrepancies, build plasticity and stability profiles, compare candidate taskifications through $(P_1^\tau,\dots,P_{K_\tau}^\tau).$0, and assess local fragility through BPS. This suggests a principled way to screen candidate benchmark definitions before full continual-learning experimentation.

6. Relation to other profile-based distance notions

Profile distance between taskifications belongs to a broader pattern in which tasks or task assignments are represented by induced profiles and then compared through distances, although the objects and geometries differ across domains.

In "Fisher Task Distance and Its Application in Neural Architecture Search," a task is represented by Fisher Information Matrices of trained networks, and the Fisher Task Distance compares the Fisher profile of a source task evaluated on target data with the target task’s own Fisher profile. That distance is asymmetric and is interpreted as a proxy for transfer complexity between tasks (Le et al., 2021). The temporal-taskification formulation differs in that it compares partitions of a single stream through distributions of induced inter-task discrepancies rather than through parameter-space curvature.

In "Statistical Deficiency for Task Inclusion Estimation," tasks are treated as joint distributions $(P_1^\tau,\dots,P_{K_\tau}^\tau).$1, and task relations are studied through statistical deficiency and information sufficiency between task-specific embedding channels. The paper further shows how directed inclusion matrices can induce dissimilarities between larger task structures, which provides another route to a profile distance between taskifications, but in an information-theoretic rather than temporal-regime sense (Fosse et al., 7 Mar 2025).

In "Fine-Grained User Profiling for Personalized Task Matching in Mobile Crowdsensing," user-task interactions are represented through hybrid preference scores, category-level reliability profiles, and latent factor embeddings. Those profile vectors can be used to define distances between users, task categories, user-task pairs, and complete task assignments, again illustrating how profile representations induce distances over higher-level configurations (Yang et al., 2018).

Across these settings, the shared pattern is not a single universal metric but a common methodological move: define a profile that captures the relevant structure of tasks or assignments, then compare those profiles at the level of the induced object of interest. In streaming continual learning, the distinctive contribution is to elevate the temporal partition itself to that object of interest and to measure how different taskifications reshape the regime under which continual-learning systems are evaluated.