---
title: Modified Information Criteria (MIC) Overview
url: https://www.emergentmind.com/topics/modified-information-criteria-mic
type: topic
---

# Modified Information Criteria (MIC) Overview

Modified Information Criteria (MIC) denotes no single standardized criterion in the arXiv literature. Instead, the phrase is used for several families of model-selection criteria that alter classical information criteria such as AIC or BIC, or replace likelihood-based summaries by alternative loss functionals, in order to address sparse high-dimensional regression, mixed and hierarchical models, Bayesian marginal-likelihood scoring, generalized linear model sparsification, or vector autoregressive order selection [1411.4138], [1503.07102], [2601.01190], [1607.05169], [2511.19761]. At the same time, the acronym “MIC” is heavily overloaded: in a different literature it denotes the maximal information coefficient, a dependence measure rather than a model-selection criterion [1301.7745], [1408.4908]. This terminological split is central to understanding the topic.

## 1. Nomenclature and scope

The phrase “modified information criterion” appears most directly in work on sparse linear regression, where “modified versions of Bayesian Information Criterion, like mBIC or mBIC2” are introduced to handle the regime in which the number of predictors \(p\) is much larger than the sample size \(n\) [1411.4138]. In other papers, the same broad idea appears under domain-specific names: a “variant of AIC based on the Bayesian marginal likelihood” in linear regression [1503.07102], a “modified Bayesian Information Criterion” denoted \(\mathrm{BIC\_HES}\) for mixture, mixed, and hierarchical settings [2601.01190], the “Mean Square Information Criterion” for VAR order selection [2511.19761], and “Minimum approximated Information Criterion” for sparse estimation in fixed-dimensional GLMs [1607.05169].

A separate line of work uses MIC to mean the maximal information coefficient. That MIC is a dependence statistic based on mutual information and adaptive grids, not an AIC/BIC-style model-selection rule [1301.7745], [1505.02213]. The same non-model-selection usage also appears in work on biological annotation analysis and gravitational-wave time-series analysis [1403.3495], [2107.03516]. This suggests that “MIC” is best treated as an overloaded acronym rather than a stable technical name.

| Usage | Expansion | Domain |
|---|---|---|
| mBIC, mBIC2 | modified versions of Bayesian Information Criterion | sparse linear regression |
| BIC\_HES | modified Bayesian Information Criterion | mixed, mixture, hierarchical models |
| MIC | Mean Square Information Criterion | VAR order selection |
| MIC | Minimum approximated Information Criterion | sparse GLMs |
| MIC | maximal information coefficient | dependence measurement |

## 2. Sparse high-dimensional regression: mBIC and mBIC2

In sparse fixed-design linear regression, the model is
\[
Y = X\beta + \varepsilon,
\]
with true support
\[
s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},
\]
and candidate models \(s \subseteq \{1,\dots,p\}\). The classical BIC is
\[
\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,
\]
where \(k(s)=|s|\) and \(\mathrm{RSS}(s)\) is the residual sum of squares. In the high-dimensional regime \(p\gg n\), this penalty is too weak; the modified criteria add an explicit multiplicity term depending on \(p\) [1411.4138].

The simplified asymptotic forms are
\[
\mathrm{mBIC}(s) = n\ln\mathrm{RSS}(s) + k(s)\ln n + 2k(s)\ln p,
\]
and
\[
\mathrm{mBIC2}(s) = n\ln \mathrm{RSS}(s) + k(s)\ln n + 2k(s)\ln p -2\ln(k(s)!).
\]
The paper also studies strengthened \(\gamma\)-versions,
\[
\mathrm{mBIC}_{\gamma}(s) = n\ln\mathrm{RSS}(s) + k(s)\ln n + 2\gamma k(s)\ln p,
\]
\[
\mathrm{mBIC2}_{\gamma}(s) = n\ln\mathrm{RSS}(s) + k(s)\ln n + 2\gamma k(s)\ln p -2\ln(k(s)!).
\]

Consistency is defined as exact support recovery over models of bounded size:
\[
P\left(\forall s:\ k(s)\le K_n,\ s\neq s_0,\ C(s)>C(s_0)\right)\xrightarrow[n\to\infty]{}1.
\]
The core identifiability quantity is
\[
\Delta(s)=\|\mu-H(s)\mu\|^2,
\]
with \(\mu = X(s_0)\beta(s_0)\) and \(H(s)\) the projection onto the span of \(X(s)\). The main separation condition is
\[
\lim_{n\rightarrow \infty}\min\left\{\frac{\Delta(s)}{p_0\ln p}\colon s_0 \nsubseteq s,\ k(s)\leq K\right\}=\infty.
\]
Under Gaussian errors and appropriate growth conditions, mBIC and mBIC2 are consistent; under subgaussian errors, consistency is retained for \(\mathrm{mBIC}_{\gamma}\) and \(\mathrm{mBIC2}_{\gamma}\) when \(\gamma\) is calibrated to the subgaussian parameter through conditions involving \(b^2/\sigma^2\) [1411.4138].

The conceptual role of these criteria is precise. They are sparsity-aware BIC modifications whose additional \(2k\ln p\)-type term corrects for the size of the model space. mBIC is more conservative, while mBIC2 weakens the penalty by subtracting \(2\ln(k!)\), which the paper associates with false-discovery-rate-oriented behavior rather than family-wise-error-style conservatism [1411.4138].

## 3. Bayesian marginal-likelihood variants of AIC

A different modification begins with the frequentist Kullback–Leibler risk of a predictive density \(\hat f(\widetilde y; y)\),
\[
R(\omega; \hat f) = \int \left[ \int \log\left\{ \frac{f(\widetilde y \mid \omega)}{\hat f(\widetilde y; y)} \right\} f(\widetilde y \mid \omega)\, d\widetilde y \right] f(y \mid \omega)\, dy,
\]
and estimates the associated information target
\[
I(\omega; \hat f)=E_\omega\!\left[-2\log \hat f(\widetilde y; y)\right].
\]
The distinctive step is to choose \(\hat f\) as a Bayesian marginal likelihood while evaluating its performance from a frequentist viewpoint [1503.07102].

For linear regression with normal prior
\[
\beta \mid \sigma^2 \sim \mathcal N_p\!\left(0,\sigma^2 W\right),
\]
the marginal likelihood is
\[
f_\pi(y\mid \sigma^2) = (2\pi\sigma^2)^{-n/2} |V|^{-1/2} |W X^\top V^{-1}X + I_p|^{-1/2}
\exp\!\left\{-\frac{1}{2\sigma^2}y^\top A y\right\},
\]
with
\[
A = V^{-1} - V^{-1}X(X^\top V^{-1}X + W^{-1})^{-1}X^\top V^{-1},
\qquad
\hat\sigma^2 = \frac{y^\top A y}{n}.
\]
The resulting criterion is
\[
\mathrm{IC}_{\pi,1} = -2\log f_\pi(y\mid \hat\sigma^2) + \frac{2n}{n-p-2}.
\]
An asymptotic approximation replaces the determinant term by \(p\log n\), yielding
\[
\mathrm{IC}_{\pi,1}^* = n\log(2\pi \hat\sigma^2) +\log|V| +p\log n +2 +\frac{y^\top A y}{\hat\sigma^2}.
\]
The paper also gives
\[
\mathrm{IC}_{\pi,2} = n\log(2\pi\hat\sigma^2) +\log|V| +p\log n +p,
\]
an asymptotically unbiased prior-averaged analogue [1503.07102].

Under an improper uniform prior on \(\beta\), the marginal likelihood becomes the residual likelihood
\[
f_r(y\mid \sigma^2) = (2\pi\sigma^2)^{-(n-p)/2} |V|^{-1/2} |X^\top V^{-1}X|^{-1/2}
\exp\!\left\{-\frac{1}{2\sigma^2}y^\top P y\right\},
\]
with
\[
P = V^{-1}-V^{-1}X(X^\top V^{-1}X)^{-1}X^\top V^{-1},
\qquad
\hat\sigma^2=\frac{y^\top P y}{n-p}.
\]
The corresponding criterion is
\[
\mathrm{IC}_r = -2\log f_r(y\mid \hat\sigma^2) +\frac{2(n-p)}{n-p-2},
\]
and its asymptotic form
\[
\mathrm{IC}_r^* = (n-p)\log(2\pi\hat\sigma^2) +\log|V| +p\log n +\frac{(n-p)^2}{n-p-2}
\]
is equivalent to the residual information criterion (RIC) up to an additive model-independent constant [1503.07102].

These constructions modify AIC in a specific sense: they keep an unbiased- or asymptotically-unbiased-risk-estimation logic but replace the ordinary fitted likelihood by a Bayesian marginal likelihood. The paper’s central claim is that this compromises between Bayesian and frequentist standpoints and yields consistency for selecting the true model in the studied regression setting [1503.07102].

## 4. Curvature-aware modified BIC in mixed and hierarchical frameworks

A further modification augments BIC by explicit curvature information from the observed Fisher information or Hessian. The proposed criterion is
\[
\mathrm{BIC\_HES} = -2\log L(\hat\theta)+p\log n+\log|J(\hat\theta)|,
\]
where
\[
J(\hat\theta) = -\nabla^2 \ell(\hat\theta).
\]
The added term \(\log|J(\hat\theta)|\) is motivated by a Laplace approximation to the marginal likelihood and is interpreted as incorporating the local geometry of the likelihood surface [2601.01190].

The derivation begins from
\[
p(y\mid H_i) = \int_{\Theta_i} f_i(y\mid \theta)\, g_i(\theta)\, d\theta,
\]
followed by a second-order expansion of
\[
Q(\theta) = \log\big(f_i(y\mid \theta)g_i(\theta)\big)
\]
around its maximizer. This yields the approximation
\[
p(y\mid H_i) \approx f_i(y\mid \tilde\theta_i)\, g_i(\tilde\theta_i)\, (2\pi)^{p_i/2} |P_i|^{-1/2},
\]
and thereby the determinant correction in the criterion [2601.01190].

For the hierarchical normal example
\[
X_{il}=\mu_i+\varepsilon_{il},
\qquad
\varepsilon_{il}\sim N(0,\sigma^2),
\]
the paper computes
\[
|I(\hat\theta)| = \left(\frac{r}{\hat\sigma^2}\right)^p \cdot \frac{pr}{2(\hat\sigma^2)^2},
\]
hence
\[
\log|I(\hat\theta)| = (p+1)\log r+\log p-\log 2 -(p+2)\log(\hat\sigma^2).
\]
This makes explicit that the modified penalty depends not only on parameter count but also on group structure and residual variance [2601.01190].

The asymptotic consistency statement is regular and nested. For a true model \(M_0\) nested in an overfitted model \(M_1\), with
\[
\Delta = \mathrm{BIC\_HES}(M_1)-\mathrm{BIC\_HES}(M_0),
\]
the paper claims
\[
\lim_{n\to\infty}\Pr(\Delta>0)=1.
\]
At the same time, it explicitly notes that the proof is regular and nested and does not address the singular or nonregular behavior typical of true finite mixture models [2601.01190].

This criterion therefore represents a geometry-aware BIC modification. Its stated empirical advantages are strongest for mixed and hierarchical models, especially in small samples and with noise covariates, while its relevance to mixture models is described more as asserted applicability than as a fully developed singular-learning-theory result [2601.01190].

## 5. Domain-specific MICs: VAR order selection and sparse GLM estimation

The acronym MIC is also used for two domain-specific criteria that are not simple penalty tweaks of BIC.

In vector autoregression, the Mean Square Information Criterion is based on the observation that the profiled mean squared prediction loss is flat once the fitted order reaches or exceeds the true order. For a \(k\)-dimensional VAR,
\[
Z_t = \sum_{i=1}^{p} A_i Z_{t-i} + \epsilon_t,
\]
the profiled population loss is
\[
\mathcal{L}(p) = \Tr(\Gamma_0) - \Tr\!\left( \begin{bmatrix} \Gamma_1 & \dots & \Gamma_p \end{bmatrix}
\begin{bmatrix} \Gamma_0 & \dots & \Gamma_{p-1}\\ \vdots & & \vdots\\ \Gamma_{p-1}^T & \dots & \Gamma_0 \end{bmatrix}^{-1}
\begin{bmatrix} \Gamma_1^T\\ \vdots\\ \Gamma_p^T \end{bmatrix} \right),
\]
and the flat-loss theorem states
\[
\begin{cases}
\mathcal{L}(p) < \mathcal{L}(p-1), & \text{if } p \le p_0,\\[3pt]
\mathcal{L}(p) = \Tr(\Sigma_\epsilon), & \text{if } p \ge p_0.
\end{cases}
\]
The sample criterion uses
\[
L(p) = \frac{1}{n-p}\Tr\left((Y_p-\hat A_pX_p)^T(Y_p-\hat A_pX_p)\right),
\]
and the practical score is written as
\[
\mathrm{MIC}(p) = \Tr(\hat{\Sigma}_p) + \mathrm{MD}\sqrt{\frac{n}{k^2\log(n)}\, p.
\]
This MIC is explicitly likelihood-free and uses \(\Tr(\hat\Sigma_p)\) rather than \(\log |\hat\Sigma_p|\), in contrast to AIC, BIC, and HQ [2511.19761].

In fixed-dimensional generalized linear models, MIC means Minimum approximated Information Criterion. The starting point is the \(\ell_0\)-based information-criterion objective
\[
\min_{\beta\in\Omega}\; -2\,L(\beta)+\lambda_0\|\beta\|_0,
\qquad
\lambda_0=\ln n \text{ for BIC}.
\]
The paper replaces the indicator by a continuous unit dent function,
\[
w(\beta)=\tanh(a\beta^2)
=\frac{\exp(2a\beta^2)-1}{\exp(2a\beta^2)+1},
\]
and then introduces the reparameterization
\[
\beta_j=\gamma_j\, w(\gamma_j),
\qquad
\beta=\mathbf W\gamma,
\]
with \(\mathbf W=\operatorname{diag}(w_1,\dots,w_p)\). The implemented MIC criterion becomes
\[
\min_{\gamma}\; -2\,L(\mathbf W\gamma)+\log(n)\,\operatorname{tr}(\mathbf W).
\]
The method is designed to approximate BIC subset selection while maintaining smooth optimization in \(\gamma\)-space; its theory is developed for fixed \(p\), with oracle-type properties for \(\widetilde\beta\) and asymptotic normality for \(\widetilde\gamma\) [1607.05169].

These two constructions share only a family resemblance. The VAR MIC is a trace-of-residual-covariance plus penalty rule derived from flat mean squared loss, while the GLM MIC is a smooth approximation to an \(\ell_0\)-penalized information criterion with a reparameterization that restores sparsity [2511.19761], [1607.05169].

## 6. Recurring principles, limitations, and the acronym problem

Across these formulations, the modifications are heterogeneous but structurally legible. In sparse regression, the principal alteration is multiplicity-aware penalization through \(2k\ln p\)-type terms [1411.4138]. In Bayesian-marginal-likelihood criteria, the modification lies in replacing the fitted likelihood by a marginal likelihood while preserving a frequentist KL-risk target [1503.07102]. In \(\mathrm{BIC\_HES}\), the modification is curvature-aware penalization through \(\log|J(\hat\theta)|\) [2601.01190]. In the VAR criterion, the modification is a shift from likelihood to mean squared prediction loss and from determinant to trace [2511.19761]. In the GLM construction, the modification is an approximation of the \(\ell_0\)-based information criterion itself, together with the smooth map \(\beta_j=\gamma_j w(\gamma_j)\) [1607.05169].

The limitations are equally specific. The mBIC and mBIC2 theory is for sparse linear regression over models of size at most \(K=kp_0\), under explicit identifiability and growth conditions [1411.4138]. The Bayesian-marginal-likelihood criteria are developed in normal linear regression and depend on the corresponding residual and marginal likelihood calculations [1503.07102]. The \(\mathrm{BIC\_HES}\) proof is regular and nested, and the paper states that it does not address the singular asymptotics of true finite mixtures [2601.01190]. The Mean Square Information Criterion is proved for correctly specified stable VAR(\(p_0\)) processes with white-noise innovations, while its misspecification claims are empirical [2511.19761]. The Minimum approximated Information Criterion is restricted to fixed-dimensional GLMs and leaves diverging-dimension extensions to future work [1607.05169].

A final source of confusion is terminological rather than mathematical. In several widely cited papers, MIC does not mean a modified information criterion at all but the maximal information coefficient, defined for dependence measurement through grid-optimized normalized mutual information [1301.7745], [1505.02213]. In that literature, the central issues are equitability, mutual information, characteristic matrices, and dependence testing, not model selection. This suggests that any use of “MIC” without expansion is ambiguous and must be resolved from context.

Source: https://www.emergentmind.com/topics/modified-information-criteria-mic