Directed Feature-Dependency Matrix

• Directed Feature-Dependency Matrix (DFDM) is a construct that quantifies asymmetric dependencies among features, modules, or representations.
• It is applied in software clustering, fault localization, and deep model interpretability using statistical tests, cosine similarity, and entropy measures.
• DFDMs facilitate model reduction and explainability by uncovering directional influences that inform feature selection and system behavior analysis.

A Directed Feature-Dependency Matrix (DFDM) is a mathematical construct that encodes directed, potentially weighted, relationships of dependency or influence among a set of features, variables, modules, or learned representations in a system. Such matrices are deployed to uncover, quantify, and exploit asymmetric dependencies for the purposes of model reduction, explanatory analysis, software clustering, mechanistic interpretation of deep models, and causal or information-flow analysis. The structure of a DFDM is context-dependent, arising from diverse analytic and algorithmic frameworks in software engineering, data science, machine learning, and interpretability studies.

1. Formal Definition and General Construction

A DFDM is an $n \times n$ (or in some contexts, $n \times m$) matrix $D$, where each entry $D_{ij}$ quantifies the existence and/or strength of a directed dependency "from $i$ to $j$". The interpretation of $D_{ij}$ is domain-specific:

• In software clustering, $D_{ij}$ may weight the extent to which module $i$ depends on module $j$ by normalizing for shared ("omnivorous") modules using Dedication scores (Kobayashi et al., 2013).
• In statistical feature analysis, $D_{ij}=1$ may signal that $x_j$ is a (possibly nonlinear) function of $x_i$, discovered by dissection of dependency graphs via pairwise statistical independence tests (Breitenbach et al., 2021).
• In model interpretability, $D_{i\to j}$ may quantify the degree to which the influence of feature $i$ is explained through synergy or redundancy with $j$ using SHAP vector decomposition (Ittner et al., 2021).
• In neural net interpretability, $D^{(\ell\to\ell+1)}_{a,b}$ encapsulates the cosine similarity between features (e.g., sparse autoencoder directions) at consecutive model layers, indicating "feature flow" (Laptev et al., 5 Feb 2025).
• In information-flows across response and covariate blocks, $D_{(i,a),(j,b)}$ represents the normalized conditional entropy quantifying directed knowledge flow between response-side and covariate-side clusters (Fushing et al., 2017).

Common to all instantiations is the asymmetric—often directional—character of dependency encoded in $D$.

2. Methodologies for Constructing Directed Feature-Dependency Matrices

2.1 Software Systems: Dedication-Based Adjacency

The SArF algorithm for software clustering constructs a DFDM $A$ by:

1. Extracting all static dependencies (method calls, field accesses, inheritance, type references) at the class or member level from compiled binaries using bytecode analysis.
2. Assigning Dedication weights:
   • Class-level: $D(A,B) = \frac{1}{\text{fanin}(B)}$ where $\text{fanin}(B)$ is the number of distinct modules depending on $B$.
   • Member-level: More precisely, $D_M(A,B)$ aggregates inverse fan-in for each called member of $B$ from $A$.
3. Forming the weighted, directed matrix $A_{ij}=D(i,j)$, sparsely populated where dependencies exist, and employing no further normalization beyond the deduction in the Dedication formula.
4. Feeding $A$ into a directed modularity maximization clustering, yielding feature-coherent clusters (Kobayashi et al., 2013).

2.2 Fault Localization: Transitive Closure of Activity Dependencies

For module-fault localization, the approach is:

1. Model modules/activities as nodes $F = \lbrace f_1, \dots, f_n \rbrace$, with edge $(f_i \to f_j)$ for direct dependency.
2. Construct the binary adjacency matrix $A$.
3. Compute the reachability matrix $D$—the transitive closure—via Boolean matrix algebra or Warshall’s algorithm:

$D = I \vee A \vee A^2 \vee \dots \vee A^{n-1}$

with $D_{ij}=1$ if $f_j$ is reachable from $f_i$.

1. Use $D$ to backtrack from observed faults to upstream root causes (Anand et al., 2014).

2.3 Principal Feature Analysis: Nonlinear Statistical Dependence

Given data $x_1, ..., x_n$:

1. Conduct pairwise independence tests (e.g., $\chi^2$, HSIC, mutual information) between all $(x_i, x_j)$, building the symmetric dependency matrix $M_{ij}$.
2. Build the dependency graph $G$ with edges where $M_{ij}=1$.
3. Iteratively identify and remove minimal node-cuts whose removal splits $G$ into independent subgraphs, recording for each cut-node $k$ the set of directed edges $j \to k$ for every neighbor $j$ at the cut instant.
4. Assemble $D_{j,k}=1$ to record that $x_k$ is functionally dependent on $x_j$ (possibly via a nonlinear relationship) (Breitenbach et al., 2021).

2.4 Inter-layer Neural Feature Flow

For a multi-layer model (e.g., transformer with sparse autoencoder-based features):

1. For each layer $\ell$ and position $P$ (typically residual), extract decoder matrices $W^{(\ell),P}_{dec}$ consisting of $F$ learned features.
2. Compute $$D^{(\ell\to\ell+1)}_{a,b} = \text{cosine similarity}\left(w^{(\ell),P}_a, w^{(\ell+1),P}_b\right)$$ for all pairs $a, b$.
3. Optionally sparsify by top-$k$ or threshold, and normalize rows for probabilistic feature-flow interpretations.
4. Concatenate resulting $F \times F$ matrices for each layer (or store per-layer as block lists) (Laptev et al., 5 Feb 2025).

2.5 SHAP S-R-I Decomposition

Given a model with features, for each feature $i$ and pair $(i,j)$:

1. Compute SHAP main and interaction vectors: $\vec{\phi}_i$ and $\vec{\phi}_{ij}$.
2. Decompose $\vec{\phi}_i$ into synergy ($\vec{s}_{i|j}$), redundancy ($\vec{r}_{i|j}$), and independence ($\vec{u}_{i|j}$) components; corresponding scalar quantities $S_{ij}$, $R_{ij}$, $I_{ij}$ sum to unity.
3. Set $D_{i\to j}=S_{ij}+R_{ij}$, quantifying how much $i$’s contribution depends on $j$ (via both synergy and redundancy).
4. Normalize and threshold as appropriate (Ittner et al., 2021).

2.6 Multiscale Block-Entropy via Data Mechanics

Given data with response features and covariate features:

1. Re-normalize each feature to discrete bins.
2. Compute mutual-conditional-entropy matrices $\Xi_r$ and $\Xi_c$.
3. Using Data Cloud Geometry (DCG), build ultrametric clustering trees to define synergistic feature groups.
4. Run Data Mechanics for fine-scale block decomposition along subjects, forming submatrices corresponding to feature clusters.
5. For each response block and covariate block at chosen tree levels, compute normalized conditional entropy $D_{(i,a),(j,b)} = H(Y^{(i)}|X^{(j)}) / H(Y^{(i)})$ to quantify directed information flow (Fushing et al., 2017).

3. Applications and Analytical Uses

Directed Feature-Dependency Matrices have broad applicability:

• Software Feature Clustering: DFDMs based on Dedication and modularity maximize the probability that class clusters correspond to meaningful features, suppressing the disruptive influence of shared utility modules. Automated clustering can be directly achieved via modularity maximization on $A$ as a weighted digraph (Kobayashi et al., 2013).
• Fault Localization: The reachability DFDM encodes all direct and transitive influences in a module graph, permitting backtracking from observed faults to likely root causes and providing rapid candidate ranking (Anand et al., 2014).
• Feature Selection and Reduction: Nonlinear statistical DFDMs identify principal features, permitting exact model reduction by representing redundant (function) features solely as deterministic or stochastic functions of retained "source" features (Breitenbach et al., 2021).
• Interpretability of Deep Models: In neural architectures, cross-layer DFDMs derived from feature cosine similarity trace the birth, persistence, mutation, or disappearance of interpretable directions, supporting mechanistic analysis and targeted intervention (Laptev et al., 5 Feb 2025).
• Global Model Explanation: S-R-I decomposed DFDMs using SHAP values quantitatively resolve synergy, redundancy, and independence between inputs, leading to fine-grained global explanations of feature interactions (Ittner et al., 2021).
• Data-Driven Knowledge Discovery: Block-entropy DFDMs articulate visible, cluster-mediated information flows between heterogeneous responses and covariates, supporting multiscale, assumption-free causal pattern mapping (Fushing et al., 2017).

4. Interpretive Properties and Theoretical Insights

Key properties of DFDMs, as reported in the literature, include:

• Sparsity: Most empirical DFDMs are inherently sparse, as strongly dedicated or statistically significant dependencies are a small subset of all possible pairs (Kobayashi et al., 2013, Laptev et al., 5 Feb 2025, Breitenbach et al., 2021).
• Directionality: Unlike undirected correlation or co-occurrence matrices, DFDMs encode asymmetry, critical for capturing causality, redundancy, or compositionality (e.g., $x_k$ as a function of $x_j$).
• Robustness to Linear Transformations: Cosine-similarity-based DFDMs remain invariant under orthonormal reparameterization of embedding spaces (Laptev et al., 5 Feb 2025).
• Additive Decomposition: SHAP S-R-I DFDMs formally decompose each input’s model contribution into orthogonal components, summing to the total, with exact closure properties (Ittner et al., 2021).
• Entropy-Minimizing Interpretation: In block-entropy-based DFDMs, the magnitude of $D_{(i,a),(j,b)}$ (normalized conditional entropy) provides a "visible" quantification of how well knowledge in a block on one side predicts outcomes on the other (Fushing et al., 2017).

5. Limitations and Caveats

• Dependence on Statistical Tests and Data Proprieties: Statistical DFDMs rely on the power and appropriateness of the independence tests, as well as binning choices for discretization, which can affect graph structure (Breitenbach et al., 2021).
• Propagation of Errors in Feature Matching: In high-dimensional learned representations, data-free matching (e.g., cosine similarity) may introduce false dependencies or miss semantically meaningful ones across layers (Laptev et al., 5 Feb 2025).
• Combinatorial Complexity: Block-based DFDMs may grow prohibitively large when feature groupings are fine-grained; Warshall's closure used for reachability is $O(n^3)$ (Anand et al., 2014).
• Assumptions of Acyclicity: Some frameworks (e.g., backtracking in fault localization) presume underlying directed acyclic graphs (DAGs); violation leads to ambiguous or degenerate dependency structures (Anand et al., 2014).

6. Illustrative Examples

Table: Selected Applications and Their DFDM Constructions

Application Domain	Matrix Construction	Reference
Software clustering	Weighted adjacency via Dedication & class graph; modularity maximization	(Kobayashi et al., 2013)
Fault localization	Binary transitive closure for root cause tracing	(Anand et al., 2014)
Feature reduction	Pairwise independence graph, iterated node-cut, functional directionality	(Breitenbach et al., 2021)
LLM interpretability	Cross-layer SAE feature cosine similarity, sparse block matrix	(Laptev et al., 5 Feb 2025)
SHAP-based explanation	S-R-I decomposition, synergy + redundancy metrics, per-feature direction	(Ittner et al., 2021)
Information flow	Conditional entropy between response/covariate blocks, Data Mechanics	(Fushing et al., 2017)

For example, training a neural net on principal features derived via DFDMs in data center monitoring achieves the same error rate as using all features, with dimensionality reduction from 2154 to 140, evidencing the matrix’s capacity to identify sufficiency in reduced representations (Breitenbach et al., 2021). In LLMs, tracing a high-coherence feature through the DFDM recapitulates its human-aligned conceptual drift from basic to composite forms across model layers (Laptev et al., 5 Feb 2025).

7. Impact and Research Directions

DFDMs enable systematic discovery, quantification, and exploitation of directed dependencies in complex systems. Their increasing adoption across software engineering, explainable AI, machine learning, and data science reflects their ability to automate, clarify, and operationalize dependency analysis without ad hoc manual heuristics. Current challenges involve improving statistical power in high-dimensional settings, mitigating geometric mismatches in deep model representations, and scaling entropy-based matrices for extremely large feature sets.

Active research continues in areas including block-structured directed dependency modeling, cross-modal and heterogeneous data integration, and real-time dynamic DFDM updates under streaming and distributed computational regimes.