Fuzzy Logistic Liu-type Estimators (FLLTE)

• The paper introduces a two-parameter penalization that improves bias–variance performance and numerical stability in fuzzy logistic regression.
• FLLTE represents both responses and parameters as fuzzy triangular numbers, extending the framework to manage uncertainty in biomedical data.
• Empirical studies confirm that FLLTE outperforms standard estimators by achieving lower MSE and superior goodness-of-fit in multicollinear settings.

Fuzzy Logistic Liu-type Estimators (FLLTE) are a class of penalized estimators designed for fuzzy logistic regression under severe multicollinearity. Both the response and model parameters are represented by fuzzy triangular numbers, extending the classical logistic regression framework to address uncertainty and vagueness in both inputs and outputs. The FLLTE introduces a two-parameter shrinkage, yielding improved bias–variance properties and numerical stability, which is empirically validated by simulation and application to high-multicollinearity biomedical data (Shemail et al., 27 Dec 2025).

1. Statistical Model and Notation

In the fuzzy logistic regression context, each observation $i=1,\dots,n$ has a fuzzy binary response $\tilde{y}_i = (l_i, y_i, u_i)$, represented as a triangular fuzzy number with lower, modal, and upper bounds. The parameter vector $\beta = (\beta_0, \beta_1, \ldots, \beta_p)^T$ is also represented as fuzzy triangular numbers: $$\tilde{\beta}_j = (l_{\beta_j}, m_{\beta_j}, u_{\beta_j})$$. The design matrix $X$ is $n \times (p+1)$ with a leading column of ones for the intercept.

The fuzzy predicted probability of success at observation $i$ is

$$\tilde{\pi}_i = \frac{\exp(\tilde{\eta}_i)}{1 + \exp(\tilde{\eta}_i)},\qquad \tilde{\eta}_i = X_i \tilde{\beta}.$$

For estimation, the logit link is applied to the midpoints:

$\logit(\pi_i) = X_i \beta,$

where $\pi_i$ is the midpoint of $\tilde{\pi}_i$. The weight matrix for Fisher scoring is $W = \operatorname{diag}(\pi_i(1-\pi_i))$.

2. Multicollinearity and Standard Fuzzy Estimators

When columns of $X$ display strong linear dependence, the Fisher-information matrix $I(\beta) = X^T W X$ becomes ill-conditioned. The standard Fuzzy Maximum Likelihood Estimator (FMLE),

$\hat{\beta}_{FMLE} = (X^T W X)^{-1} X^T W Z,$

where $Z$ is the working response, exhibits high variance and instability under such conditions. This behavior motivates penalized alternatives to address the inflated variance characteristic of multicollinear settings.

3. Formulation of the Fuzzy Logistic Liu-type Estimator

The FLLTE is derived by augmenting the Newton–Raphson scoring equation with a generalized Liu-type two-parameter penalty:

$U(\beta) - (X^T W X + kI)\beta + dI\,\beta = 0$

for penalty parameters $k>0$ and $0 \leq d < 1$. The resulting estimator is

$$\hat{\beta}_{FLLTE} = (X^T W X + kI)^{-1} (X^T W X + dI)\,\hat{\beta}_{FMLE} = (X^T W X + kI)^{-1}(X^T W Z + d\,\hat{\beta}_{FMLE}).$$

In practice, this is written as

$$\hat{\beta}_{FLLTE} = (X^T W X + kI)^{-1}(X^T W X + dI)(X^T W X)^{-1}X^T W Z.$$

The parameters $k$ and $d$ are selected to balance sparsity, stability, and bias–variance trade-off.

4. Statistical Properties

The FLLTE displays analytically tractable bias and variance:

• Bias: For the true parameter $\beta$,

$$E[\hat{\beta}_{FLLTE}] \approx (X^T W X + kI)^{-1}(X^T W X + dI)\,\beta$$

so the bias is

$B = [(X^T W X + kI)^{-1}(X^T W X + dI) - I]\beta.$

• Variance:

$$\operatorname{Var}(\hat{\beta}_{FLLTE}) \approx (X^T W X + kI)^{-1} (X^T W X) (X^T W X + kI)^{-1}.$$

• Mean Square Error (MSE):

$$MSE(\hat{\beta}_{FLLTE}) = \operatorname{tr}\{ B\,B^T \} + \operatorname{tr}\{ \operatorname{Var}(\hat{\beta}_{FLLTE}) \}.$$

• Asymptotics: As $n \to \infty$, the penalization vanishes and $\hat{\beta}_{FLLTE} \to \hat{\beta}_{FMLE}$ in distribution, guaranteeing consistency.

5. Comparison with FMLE, FLRE, FLLE, and FLLTPE

Several estimators are contrasted: | Estimator | Formula | Penalty Parameters | |------------------------------|---------------------------------------------------------------------|--------------------| | FMLE | $(X^T W X)^{-1} X^T W Z$ | None | | FLRE | $(X^T W X + K I)^{-1} X^T W Z$ | Ridge ($K$) | | FLLE | $$(X^T W X + I)^{-1} (X^T W X + dI)(X^T W X)^{-1} X^T W Z$$ | Liu ($d$) | | FLLTE | $$(X^T W X + kI)^{-1} (X^T W X + dI)(X^T W X)^{-1} X^T W Z$$ | Liu-type ($k, d$) | | FLLTPE | $$(X^T W X + kI)^{-1} (X^T W X + k d I)(X^T W X)^{-1} X^T W Z$$ | Liu-type param. |

Simulation and real-data analyses demonstrate the hierarchy:

$$\text{FLLTPE} \approx \text{FLLTE} > \text{FLLE} > \text{FLRE} > \text{FMLE}$$

in terms of lower MSE and improved goodness-of-fit, especially under strong predictor correlation (Shemail et al., 27 Dec 2025).

6. Choice of Shrinkage Parameters

Shrinkage parameters ($k$, $d$) are selected through data-driven formulas developed by Asar & Genç (2016):

• Compute eigenvalues/vectors $(\lambda_j, \alpha_j)$ of $X^T W X$.
• Set

$$d = \operatorname{median}_j \left\{ \frac{ \lambda_j \alpha_j^2 }{ 1 + \lambda_j \alpha_j^2} \right\},$$

$$k = \frac{ \lambda_j \alpha_j^2 (1 - d) - d }{ d },$$

ensuring $k > 0$. In empirical analysis, medians or means across the components are used for parameter stability.

7. Empirical Evaluation and Application

Simulation studies were conducted for $n \in \{25, 50, 100, 150\}$ and $p \in \{3, 5, 7\}$, with inter-predictor correlations of $0.70$, $0.90$, and $0.99$. The true fuzzy coefficients were set as triangular numbers around predefined values. Over $100$ Monte Carlo repetitions, FLLTE—and particularly FLLTPE—attained the smallest MSE, remaining robust even as correlation approached $0.99$, whereas FMLE demonstrated rapid degradation.

A real-data application used $n=35$ patient observations for modeling the fuzzy “renal failure rate” (triangular-coded as Low $(0,0.25,0.44)$, Medium $(0.35,0.55,0.74)$, High $(0.65,0.85,1)$) as a function of blood sugar, hemoglobin, and age. The condition number of $X$ was approximately $45$, indicating severe multicollinearity. Results are summarized as follows:

Estimator	MSE	Goodness-of-Fit
FMLE	1.81	0.54
FLRE	1.42	0.48
FLLE	1.29	0.39
FLLTE	0.57	0.17
FLLTPE	0.53	0.16

FLLTE and FLLTPE yielded the lowest MSE and best model fit.

8. Implications and Recommendations

The FLLTE addresses instability and inflated variance in fuzzy logistic regression when multicollinearity is present by introducing a flexible two-parameter penalization. Its closed-form expression admits analytic bias–variance trade-offs and preserves asymptotic consistency. Simulation and real-world results confirm notable improvements over FMLE, FLRE, and FLLE with respect to MSE and model fit. Median-based eigen-decomposition parameter selection is recommended to promote stability across settings (Shemail et al., 27 Dec 2025).