Category-wise Influence Functions

• Category-wise influence functions are a vectorial method that measures the effect of each training sample on specific classes or categories.
• They enable targeted data cleaning and balanced optimization by disentangling impacts, facilitating Pareto improvements in multi-objective models.
• Applications include robust model diagnostics, efficient reweighting via linear programming, and performance ceiling estimation across different data subpopulations.

Category-wise influence functions provide a vectorial or partitioned quantification of the effect of individual training samples or groups on specific, often disjoint, subcomponents of a learning task—typically corresponding to distinct classes, categories, groups, or features of the underlying data. Unlike classical influence functions, which yield a single scalar effect—usually on the aggregate loss or prediction—category-wise influence functions disentangle and decompose the impact across multiple objectives, categories, or subpopulations, and thereby underpin refined data-centric optimization, robust model selection, and performance diagnostics.

1. Foundations and Mathematical Formulation

A standard influence function quantifies the infinitesimal effect that reweighting, removing, or perturbing a training example $z$ would have on a fixed functional $T(P)$ of the underlying data distribution $P$. The classical formulation (using the Gateaux derivative) is: $$\psi(z; P) = \left. \frac{d}{dt} T((1 - t)P + t\delta_z) \right|_{t=0}$$ or, in empirical risk minimization and parametric models, as the Hessian-inverse–scaled gradient: $$I(z) = - H(\hat\theta)^{-1} \nabla_\theta \ell(z, \hat\theta)$$ where $H$ is the Hessian of the loss at the optimum $\hat\theta$.

Category-wise influence functions extend this by mapping each training sample $z$ to a vector or structured object $P(z) \in \mathbb{R}^K$ (for $K$ categories/classes), where the $k$th component $P^k(z)$ represents the effect of $z$ on the performance (loss, accuracy, or other metric) for category $k$: $P^k(z) = \text{IF}(z, S^k)$ with $S^k$ the subset of samples (or loss terms) associated with category $k$ (Nahin et al., 4 Oct 2025). This enables the explicit assessment of the heterogeneous role of $z$ across categories.

For group influence (e.g., cohorts, batches, or categories), category-wise influence is computed by aggregating (usually summing) per-sample influences over all samples in the group (Koh et al., 2019, Fisher et al., 2022): $\text{GroupIF}(G) = \sum_{z \in G} I(z)$ and for partitioned test sets or category-specific losses, vector-valued influence functions are constructed accordingly.

2. Pareto Frontier and Performance Ceiling Analysis

A critical contribution of the category-wise influence function paradigm is its capacity to identify the Pareto frontier of classifier performance over multiple objectives—typically, class-wise (per-category) accuracies (Nahin et al., 4 Oct 2025).

Given that traditional data cleaning or selection with influence functions can trade off accuracy between classes (improving one at the cost of another), the category-wise approach uses the influence vector $P(z)$ to analyze the possibility of Pareto improvements. Specifically:

• If there exist training samples $z$ with $P^k(z) < 0$ for all $k$ (jointly detrimental), removing or downweighting such samples can increase performance across all categories.
• If all samples have influence vectors displaying only tradeoff directions (positive for one category, negative for another), the model has likely reached its Pareto performance ceiling.

This is formalized by plotting or analyzing the collection of influence vectors. The existence of samples in the joint positive or joint negative orthant of influence space signals whether further simultaneous improvement is possible.

A linear programming (LP) framework (Nahin et al., 4 Oct 2025) is introduced to find a weighted combination of samples that, when reweighted, can move the classifier closer to the Pareto frontier, i.e., achieving improvements in all (or a chosen subset of) categories: $$\max_{w} \sum_{k \in \text{target}} \sum_{i} w_i P^k(z_i) \quad \text{s.t.} \quad \sum_{i} w_i P^k(z_i) \geq \alpha_k \sum_{i} P^k(z_i), \forall k$$ where $w_i$ are sample weights and $\alpha_k$ are thresholding slack variables, often optimized via a genetic algorithm.

3. Applications and Empirical Evaluation

Clustering, Classification, and Statistical Inference

Category-wise influence functions underpin several applications:

• Category-specific divergence estimation: By estimating densities and functionals (entropy, divergence) for each category and applying influence corrections per group, robust between-group metrics are obtained (Kandasamy et al., 2014).
• Robustness diagnostics in multi-class settings: In Linear Discriminant Analysis (LDA), influence functions for group means and discriminant directions quantify the effect of individual or grouped outliers on the separation between groups (Prendergast et al., 2019).
• Efficient inference and bias correction: In settings with partial observation, missing data, or high-dimensional covariates, partitioning the covariate space into interpretable categories enables the computation of group-wise parameters, pseudo-outcomes, and confidence intervals via IF-based estimators (Curth et al., 2020).

Data Selection and Performance Improvement

• Pruning and data cleaning: Harmful or uninformative training examples are identified using their negative impact on category-specific metrics (e.g., class-wise accuracy), allowing targeted sample removal while maintaining or improving multi-class performance (Fein et al., 18 Jul 2025, Nahin et al., 4 Oct 2025).
• Sample reweighting for balanced improvement: The LP-based reweighting methodology ensures Pareto improvement, preventing the common pitfall of aggregate-accuracy optimization that sacrifices minority classes (Nahin et al., 4 Oct 2025).

Empirical validation on synthetic data, vision, and textual benchmarks highlights that category-wise influence vectors are reliable predictors of per-class performance changes. Notably, removing samples with negative influence on all classes systematically yields improvements, whereas in settings where influence vectors only manifest tradeoffs, the classifier is at its performance ceiling.

4. Connections to Group and Channel-wise Influence

The conceptual framework of category-wise influence generalizes to arbitrary data groupings or data components:

• Group-wise influence: Summed or aggregate influence across predefined groups, batches, or data sources; used to diagnose batch effects and attribute performance changes to data sources (Koh et al., 2019, Fisher et al., 2022).
• Channel-wise or feature-wise influence: Applied in multivariate time series or structured data, where the influence matrix encodes the impact of perturbing specific features/channels on performance; self-influence on diagonals is used for anomaly detection and pruning (Wang et al., 27 Aug 2024).
• Graph element influence: In graph neural networks, edge- or node-wise influence functions are similarly derived to analyze the sensitivity of model predictions to local graph modifications (Chen et al., 2022).

These generalizations rely on the same principle: decomposing the influence function with respect to interpretable, structured subgroups of the data.

5. Methodological Extensions and Challenges

• Computation: Estimating influence functions precisely in high dimensions demands scalable approximations, such as efficient Hessian-vector products, parameter-efficient restrictions (e.g., LoRA in LLMs), or tractable group-wise approximations (Fein et al., 18 Jul 2025, Fisher et al., 2022).
• Limitations: Linear approximations may exhibit absolute error when large groups or highly nonlinear effects are present, but group or category-wise sums remain strongly correlated with true effects under mild regularity and redundancy conditions (Koh et al., 2019).
• Statistical guarantees: Finite-sample error bounds have been developed for per-sample and group influence function estimation, scaling with the effective dimension and inverse spectral gap of the Hessian (Fisher et al., 2022).

6. Implications and Future Directions

Category-wise influence functions serve as both diagnostic tools and as operational mechanisms for principled, fair, and balanced model improvement. Their use informs:

• Automated pruning or reweighting pipelines that avoid overfitting to majority classes or degrading performance on minority classes, which is essential in imbalanced and safety-critical applications.
• Performance ceiling measurement: The established methodology allows practitioners to not only optimize but also assess when models are saturated in terms of data utility, providing guidance for further data collection or model innovation (Nahin et al., 4 Oct 2025).
• Interpretable data attribution: By aligning complex, high-dimensional data attribution tasks with categorical or group structure in the data, influence vectors facilitate interpretability, regulatory compliance, and targeted data curation.

Open questions remain regarding efficient cross-group influence estimation in deep, non-convex models; further theoretical work to characterize limitations; and practical pipelines integrating influence-based selection with automated model retraining.

7. Summary Table: Conventional vs. Category-wise Influence Functions

Aspect	Conventional IF	Category-wise IF
Output	Scalar per sample	Vector per sample (per category)
Captures	Aggregate loss/effect	Effect on each class/group
Model improvement	Aggregate-accuracy gains	Pareto, class-wise gains
Pruning/reweighting	Global effect	Fine-grained, targeted by class
Applications	Robustness, debugging	Balanced improvement, fairness

Category-wise influence functions thus provide a structured, multi-objective approach to influence diagnostics and data-centric model optimization, enabling balanced, interpretable, and principled improvements in machine learning systems.