Fuzzy Logistic Ridge Estimators (FLRE)

• FLRE is a fuzzy logistic regression approach that models both responses and coefficients as triangular fuzzy numbers, integrating classical ridge penalization.
• It stabilizes parameter estimates under severe multicollinearity by adding an ℓ2 penalty to the fuzzy log-likelihood, effectively reducing variance inflation.
• Empirical studies show FLRE reduces mean squared error compared to FMLE, with further improvements by Liu-type extensions in practical applications.

Fuzzy Logistic Ridge Estimators (FLRE) are regularization-based estimators specifically developed for fuzzy logistic regression models exhibiting significant multicollinearity. They extend classical ridge penalization into the fuzzy setting, in which both the response variable and the parameter vector are represented as triangular fuzzy numbers. FLREs aim to stabilize parameter estimation by counteracting variance inflation induced by collinear predictors, while incorporating fuzziness intrinsic to both outcomes and coefficients, as formulated in recent work by Shemail et al. (Shemail et al., 27 Dec 2025).

1. Fuzzy Logistic Regression Model Formulation

In the fuzzy logistic regression framework, every observation and regression coefficient is modeled as a triangular fuzzy number:

• Each response, $\tilde y_i = (\ell_{y_i}, m_{y_i}, u_{y_i})$, possesses a membership function

$$\mu_{\tilde y_i}(y) = \begin{cases} \frac{y - \ell_{y_i}}{m_{y_i} - \ell_{y_i}}, & \ell_{y_i} \leq y \leq m_{y_i} \ \frac{u_{y_i} - y}{u_{y_i} - m_{y_i}}, & m_{y_i} < y \leq u_{y_i} \ 0, & \text{otherwise} \end{cases}$$

• The parameter vector is also fuzzy: $\tilde\beta = (\ell_\beta, m_\beta, u_\beta)$, each entry being triangular.

The success probability for observation $i$ is given by the fuzzy logistic function,

$$\tilde\pi_i = \Pr(\tilde y_i=1|x_i) = \frac{\exp(\tilde\beta^{\top}x_i)}{1+\exp(\tilde\beta^{\top}x_i)},$$

with the logit transform,

$$\operatorname{logit}(\tilde\pi_i) = x_i^\top\tilde\beta.$$

While covariates $x_i\in\mathbb{R}^p$ are crisp, separate equations for lower, center, and upper levels of fuzziness are applied throughout.

2. Construction of the Fuzzy Logistic Ridge Estimator

FLREs are derived by augmenting the fuzzy log-likelihood with a ridge penalty. The log-likelihood for the crisp case is adapted to the fuzzy setting—for each $\alpha$-cut, the form persists:

$$\ell(\beta) = \sum_{i=1}^n \left[ y_i\ln\pi_i + (1 - y_i)\ln(1 - \pi_i) \right], \qquad \pi_i = \frac{e^{x_i^\top\beta}}{1+e^{x_i^\top\beta}}.$$

The penalized criterion is

$$\ell_R(\beta; k) = \ell(\beta) - k\,\beta^\top\beta, \qquad k > 0.$$

The estimating equations become:

$U_R(\beta) = X^\top(y-\pi) - 2k\beta = 0,$

where $X$ is the model matrix and $\pi$ is the vector of fitted probabilities for each cut.

A closed-form FLRE solution, akin to a one-step Newton–Raphson update around the Fuzzy Maximum Likelihood Estimator (FMLE), is:

$$\hat\beta_{FLRE} = (X^\top W X + k I_p)^{-1} X^\top W z$$

with

$$W = \operatorname{diag}(\pi_i(1 - \pi_i)),\quad z = X\beta_{FMLE} + W^{-1}(y - \pi).$$

A compact “shrinkage-follow-MLE” representation (Eq. 13 of (Shemail et al., 27 Dec 2025)):

$$\hat\beta_{FLRE} = (X^\top W X + k I_p)^{-1} X^\top W X \beta_{FMLE}.$$

3. Theoretical Properties under Multicollinearity

Let $S = X^\top W X$. The theoretical risk properties under multicollinearity are characterized as follows:

• Bias:

$$\mathrm{Bias}(\hat\beta_{FLRE}) = - k (S + k I)^{-1} \beta$$

• Variance:

$$\mathrm{Var}(\hat\beta_{FLRE}) = (S + k I)^{-1} S (S + k I)^{-1}$$

• Mean Squared Error (MSE) matrix:

$$\mathrm{MSE}(\hat\beta_{FLRE}) = \mathrm{Var}(\hat\beta_{FLRE}) + \mathrm{Bias}(\hat\beta_{FLRE}) \mathrm{Bias}(\hat\beta_{FLRE})^\top$$

In scalar form,

$$\mathrm{MSE} = k^2 \beta^\top (S + k I)^{-2} \beta + \mathrm{tr}\big[(S + k I)^{-1} S (S + k I)^{-1}\big]$$

• Asymptotic Properties: As $n \to \infty$ and $\frac{k}{n} \to 0$, bias vanishes and variance approaches the fuzzy Fisher information bound.

This framework controls the inflation of estimator variance associated with severe collinearity while introducing a bias that can be analytically characterized and, if $k$ is suitably chosen, minimized in predictive risk.

4. Choice of Ridge Penalty Parameter

Several methods are established for selecting the optimal ridge penalty $k$ in the fuzzy context:

• Cross-Validation minimizes prediction error on held-out folds.
• Generalized Cross-Validation (GCV) provides a computationally efficient approximation to leave-one-out error.
• Information Criteria: $k$ can be chosen by minimizing $$\mathrm{AIC}_k = -2\ell(\hat\beta_{FLRE}) + 2\,df(k)$$, where the degrees-of-freedom depend on the effective number of parameters.
• Analytical Shrinkage Formula: Asar & Genç (2016) propose using the eigen-decomposition $S = Y\Lambda Y^\top$ with $\lambda_j$ eigenvalues and $\alpha_j = y_j^\top\beta$, yielding a median-based shrinkage factor

$$d_{\mathrm{med}} = \mathrm{median} \left\{ \frac{\lambda_j \alpha_j^2}{1 + \lambda_j \alpha_j^2} \right\},$$

and penalty

$$k = \frac{\sum_j \lambda_j\alpha_j^2(1 - d_{\mathrm{med}})}{d_{\mathrm{med}}}$$

These approaches adapt classical criterion to the fuzzy parameter setting, allowing $k$ to be robustly tuned to the magnitude and structure of multicollinearity.

5. Empirical Performance: Simulation Results

Shemail et al. report simulation studies comparing FMLE, FLRE, FLLE, FLLTE, and FLLTPE at high collinearity ($p = 0.99$), $n = 25$, with $k$ determined via median-shrinkage:

Estimator	Goodness-of-Fit $S$	MSE
FMLE	1.9456	2.6909
FLRE	1.5345	2.2798
FLLE	1.1345	1.8798
FLLTE	0.5345	1.2798
FLLTPE	0.4123	1.1576

The FLRE outperforms the fuzzy maximum likelihood estimator (FMLE) in both mean squared error and goodness-of-fit, but FLLTE and FLLTPE show further marked improvements in both measures. This suggests that FLRE's penalization is effective in regularizing fuzzy logistic regression estimates under collinearity, with greater gains realized through Liu-type regularization strategies.

6. Real Data Evaluation: Kidney Failure Study

FLRE was evaluated on a real dataset comprising $n=35$ kidney failure patients with fuzzy outcome labels ("renal failure rate") and crisp covariates: blood sugar ($X_1$), hemoglobin ($X_2$), and age ($X_3$). Severe multicollinearity was diagnosed (condition number $>30$).

Estimator performance is as follows (from Table 9 of (Shemail et al., 27 Dec 2025)):

Method	$\hat\beta_0$	$\hat\beta_1$	$\hat\beta_2$	MSE	$S$
FMLE	3.2632	4.5128	0.4649	1.8073	0.5422
FLRE	2.0395	2.8205	0.2906	1.4152	0.4846
FLLE	2.2661	3.1339	0.3228	1.2882	0.3865
FLLTE	1.8129	2.5071	0.2583	0.5672	0.1702
FLLTPE	1.5863	2.1937	0.2260	0.5342	0.1603

FLRE reduced MSE by approximately 22% relative to FMLE. FLLTE and FLLTPE provided further reductions, nearly halving MSE and doubling the goodness-of-fit compared to ridge penalization alone. This confirms the robust bias–variance trade-off offered by ridge-type and related Liu-type penalizations in fuzzy models under multicollinearity.

7. Summary and Comparative Assessment

FLREs generalize classical ridge regularization to the fuzzy logistic regression domain by incorporating an $\ell_2$ penalty into the fuzzy log-likelihood, offering closed-form shrinkage estimators centered at the FMLE. Explicit bias, variance, and risk expressions can be derived, and $k$ selection is adaptable to the fuzzy setting using cross-validation, information criteria, or median-based shrinkage approaches. Empirical evaluations, both in simulation and application to kidney failure data, indicate substantive improvements over naive fuzzy maximum likelihood estimation. However, Liu-type extensions (FLLTE, FLLTPE) further lower MSE by introducing additional bias structure, suggesting their superiority in scenarios where two-parameter regularization is warranted (Shemail et al., 27 Dec 2025).