ADAR-ANFIS: Adaptive Pruning in Fuzzy Systems

• ADAR-ANFIS is a dual-stage adaptive pruning framework that optimizes ANFIS architectures by dynamically weighting and reducing redundant attributes and rules.
• It integrates automated pruning, rule growth, and spectral techniques like PCA and BPSO to manage complexity and maintain fuzzy model interpretability.
• Empirical evaluations reveal significant rule reductions and up to 15% RMSE improvement across regression tasks, ensuring efficient and transparent inference.

Attribute and Rule Pruning (ADAR-ANFIS) is a dual-stage adaptive pruning methodology for optimizing fuzzy inference systems, specifically Adaptive Neuro-Fuzzy Inference System (ANFIS) architectures, in both high- and low-dimensional data contexts. The ADAR-ANFIS framework integrates learnable dual weighting mechanisms, automated pruning and growth strategies, and data-driven optimization algorithms to address the challenges of rule-explosion and feature irrelevance, while preserving or enhancing interpretability and predictive accuracy (Liu et al., 27 Apr 2025, Al-Ali et al., 6 Feb 2025).

1. Motivation and Overview

Traditional ANFIS approaches, especially those based on grid partitioning, suffer from exponential growth in rule base size as input dimensionality increases, leading to inefficiency, loss of interpretability, and susceptibility to overfitting. In high-dimensional regimes, many attributes are irrelevant to most rules; in low-dimensional settings, excess rules emerge due to redundancy in the rule-partition space. ADAR-ANFIS addresses these drawbacks by:

• Integrating adaptive attribute and rule weighting to induce structured sparsity.
• Dynamically pruning unnecessary features at the rule level (Attribute Pruning, AP) and discarding or growing rules (Rule Growing and Pruning, RG&RP) based on learned importance.
• Utilizing optimization-based and spectral techniques (PCA, BPSO) for compact rule selection.
• Supporting model transparency, as all active rule and attribute weights are directly inspectable.

The approach is agnostic to problem domain and supports both regression and classification tasks across varied dataset structures (Liu et al., 27 Apr 2025, Al-Ali et al., 6 Feb 2025).

2. ADAR-ANFIS Architecture and Algorithmic Workflow

Architecture

Given a normalized $D$-dimensional input $x=[x_1, ..., x_D]^\top$, the model channels $x$ in parallel to $L$ fuzzy rules, where each rule $l$ is parameterized by:

• Gaussian MF centers $\mu_{l,i}$ and widths $\sigma_{l,i}$ for each feature $i$.
• Attribute-weight logit $w^a_{l,i}$, yielding the post-sigmoid weight $a_{l,i}=\sigma(w^a_{l,i})m_{l,i}$.
• Rule-weight logit $w^r_l$, corresponding to $\beta_l=\sigma(w^r_l)$ after sigmoid.
• Consequent coefficients $c_{l,i}$ for active features.

Forward Pass

The forward computation involves:

1. Attribute weights: $a_{l,i} = \sigma(w^a_{l,i}) m_{l,i}$
2. Gaussian membership: $$\mu_{l,i}(x_i) = \exp\left(-\frac{(x_i-\mu_{l,i})^2}{2\sigma_{l,i}^2}\right)$$
3. Weighted product: $\omega_l = \prod_{i=1}^D [\mu_{l,i}(x_i)a_{l,i}]$
4. Rule weighting: $f_l=\beta_l \omega_l$
5. Normalized firing: $$\hat f_l = f_l / \left(\sum_{k=1}^L f_k + \epsilon\right)$$
6. Consequent output: $y_l = \sum_{i\in A_l} c_{l,i} x_i$
7. Final output: $\hat y = \sum_{l=1}^L \hat f_l y_l$

Learning and Adaptation

Training minimizes mean squared error (MSE), regularized with L1 penalties on both attribute and rule weights:

$$\mathcal{L} = \frac{1}{N}\sum_{n=1}^N (\hat y^{(n)}-y^{(n)})^2 + \lambda_a \sum_{l,i}|w^a_{l,i}| + \lambda_r \sum_l |w^r_l|$$

Parameters are optimized via SGD or Adam. Dynamic structural updates—attribute pruning at interval $P_\mathrm{attr}$ and rule growing/pruning at $P_\mathrm{rule}$—are applied, with optimizer momentum reset after structural changes (Liu et al., 27 Apr 2025).

3. Dual Weighting and Automated Pruning Mechanisms

Attribute and Rule Weighting

• Attribute weights are updated by backpropagation and L1-regularized to promote sparsity, with masking $m_{l,i}$ disabling insignificant weights.
• Rule weights $\beta_l$ are similarly L1-regularized, reflecting a rule’s global importance.
• Both weighting processes use the sigmoid function for smooth pruning and consistent differentiability.

Automated Growth and Pruning

• Attribute Pruning (AP): When $a_{l,i} < \theta_\mathrm{attr}$ for $T_\mathrm{attr}$ consecutive epochs, set $m_{l,i}=0$, permanently disabling feature $i$ for rule $l$.
• Rule Pruning: When $\beta_l < \theta_r$ for $T_\mathrm{rule}$ epochs, rule $l$ is deleted.
• Rule Growing: Triggered if validation loss stagnates for $p$ epochs and $L < L_\mathrm{max}$. High-error samples are clustered, and new rules are created via K-means (Liu et al., 27 Apr 2025).

4. Alternative: PCA and Optimization-Driven Rule Pruning

An alternative realization employs Principal Component Analysis (PCA) on the normalized firing-strength matrix to reveal low-rank structure in rule activations, followed by rule selection using Binary Particle Swarm Optimization (BPSO) (Al-Ali et al., 6 Feb 2025):

• PCA-Based Attribute Pruning: Projects the firing-strength space onto top variance components (typically retaining $>95\%$ variance), implicitly discarding redundant input dimensions.
• BPSO-Based Rule Pruning: Each particle represents a binary mask over rules. The Particle Swarm is evolved to minimize loss plus a sparsity-promoting regularization term, thus efficiently identifying an optimal sparse rule subset.
• After pruning, both consequence and antecedent (MF) parameters are retrained within the reduced architecture.

The PCA+BPSO approach delivers rule counts reduced by up to two orders of magnitude and reduces training time from thousands of seconds to tens of seconds, with overall accuracy preserved or modestly improved (Al-Ali et al., 6 Feb 2025).

5. Empirical Evaluation and Trade-offs

Experimental Results

ADAR-ANFIS has been evaluated on four regression datasets: Auto MPG ($D=7$), Beijing PM2.5 ($D=10$), Boston Housing ($D=13$), and Appliances Energy Consumption ($D=27$). Across these, ADAR-ANFIS consistently outperforms fixed-structure ANFIS, as quantified by RMSE and rule base reduction.

Selected Performance Metrics:

Dataset	Baseline ANFIS (RMSE)	ADAR-ANFIS (RMSE)	Rules Retained
Beijing PM₂.₅	67.77 ± 5.40	57.32 ± 0.59	9
Appliances Energy	95.99 ± 3.25	83.25 ± 0.69	9

• ADAR-ANFIS achieves $\approx$15% RMSE reduction on Beijing PM2.5 and $\approx$13% on Appliances Energy datasets, with an order-of-magnitude rule reduction (Liu et al., 27 Apr 2025).
• Ablation demonstrates that both attribute- and rule-level pruning are necessary for minimal rule overlap and improved fuzzy set placement (Iov reduced from 24.3 to 0.950).

Efficiency and Accuracy Trade-offs

• Rule count and training time are dramatically decreased (examples: Iris dataset, from 81 to 2 rules and from 3530 to 13.1 seconds; Airfoil, from 32 to 2.2 rules and 2810 to 23.4 seconds) (Al-Ali et al., 6 Feb 2025).
• For most benchmarks, RMSE or accuracy is maintained or slightly improved; rare small drops (≈1–2%) occur, reflecting classic bias-variance considerations.

6. Theoretical Properties and Practical Significance

ADAR-ANFIS advances ANFIS-based fuzzy modeling by:

• Mitigating the curse of dimensionality through data-dependent feature selection.
• Yielding adaptable model complexity, with rule base size and feature utilization governed by data regularities rather than fixed design.
• Providing direct interpretability: surviving attribute weights $a_{l,i}$ and rule weights $\beta_l$ are readily inspected.
• Enhancing separation and reducing fuzzy rule overlap, thereby supporting clearer, more parsimonious inference paths (Liu et al., 27 Apr 2025).

A plausible implication is that improved model parsimony without degradation in predictive metrics suggests better generalization and enhanced explainability, especially critical in domains requiring transparent and auditable AI.

7. Related Methods and Extensions

While ADAR-ANFIS dynamically manages both attribute and rule dimensions based on data-driven weighting and pruning, alternative methods deploy rule reduction strategies such as heuristic selection, evolutionary optimization, or grid pruning. The PCA+BPSO alternative illustrated in (Al-Ali et al., 6 Feb 2025) exemplifies a spectral–combinatorial pruning workflow particularly effective in low-dimensional settings. This suggests the potential for further hybridization, combining structural learning, principal component analysis, and evolutionary selection to maximize the interpretability-efficiency-accuracy trade space.

Empirical evidence confirms that the ADAR-ANFIS approach is well-suited for both high- and low-dimensional applications, with scalability, transparency, and performance observed across a variety of conventional regression and classification benchmarks (Liu et al., 27 Apr 2025, Al-Ali et al., 6 Feb 2025).