Modified Information Criteria (MIC) Overview

• Modified Information Criteria (MIC) are a family of model selection methods that adjust classical criteria like AIC and BIC to better handle high-dimensional, sparse, and complex model settings.
• Variants such as mBIC, Bayesian-marginal-likelihood criteria, curvature-aware BIC, VAR order selection MIC, and GLM approximations each tailor the penalty to the specific properties of the model domain.
• The terminological overlap with the maximal information coefficient highlights the importance of context, as MIC can refer either to modified model selection rules or to a measure of dependence.

Modified Information Criteria (MIC) denotes no single standardized criterion in the arXiv literature. Instead, the phrase is used for several families of model-selection criteria that alter classical information criteria such as AIC or BIC, or replace likelihood-based summaries by alternative loss functionals, in order to address sparse high-dimensional regression, mixed and hierarchical models, Bayesian marginal-likelihood scoring, generalized linear model sparsification, or vector autoregressive order selection (Szulc, 2014, Kawakubo et al., 2015, Ramirez et al., 3 Jan 2026, Su et al., 2016, Hellstern et al., 24 Nov 2025). At the same time, the acronym “MIC” is heavily overloaded: in a different literature it denotes the maximal information coefficient, a dependence measure rather than a model-selection criterion (Kinney et al., 2013, Reshef et al., 2014). This terminological split is central to understanding the topic.

1. Nomenclature and scope

The phrase “modified information criterion” appears most directly in work on sparse linear regression, where “modified versions of Bayesian Information Criterion, like mBIC or mBIC2” are introduced to handle the regime in which the number of predictors $p$ is much larger than the sample size $n$ (Szulc, 2014). In other papers, the same broad idea appears under domain-specific names: a “variant of AIC based on the Bayesian marginal likelihood” in linear regression (Kawakubo et al., 2015), a “modified Bayesian Information Criterion” denoted $\mathrm{BIC\_HES}$ for mixture, mixed, and hierarchical settings (Ramirez et al., 3 Jan 2026), the “Mean Square Information Criterion” for VAR order selection (Hellstern et al., 24 Nov 2025), and “Minimum approximated Information Criterion” for sparse estimation in fixed-dimensional GLMs (Su et al., 2016).

A separate line of work uses MIC to mean the maximal information coefficient. That MIC is a dependence statistic based on mutual information and adaptive grids, not an AIC/BIC-style model-selection rule (Kinney et al., 2013, Reshef et al., 2015). The same non-model-selection usage also appears in work on biological annotation analysis and gravitational-wave time-series analysis (Wang et al., 2014, Jung et al., 2021). This suggests that “MIC” is best treated as an overloaded acronym rather than a stable technical name.

Usage	Expansion	Domain
mBIC, mBIC2	modified versions of Bayesian Information Criterion	sparse linear regression
BIC_HES	modified Bayesian Information Criterion	mixed, mixture, hierarchical models
MIC	Mean Square Information Criterion	VAR order selection
MIC	Minimum approximated Information Criterion	sparse GLMs
MIC	maximal information coefficient	dependence measurement

2. Sparse high-dimensional regression: mBIC and mBIC2

In sparse fixed-design linear regression, the model is

$Y = X\beta + \varepsilon,$

with true support

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$

and candidate models $s \subseteq \{1,\dots,p\}$. The classical BIC is

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$

where $k(s)=|s|$ and $\mathrm{RSS}(s)$ is the residual sum of squares. In the high-dimensional regime $p\gg n$, this penalty is too weak; the modified criteria add an explicit multiplicity term depending on $n$0 (Szulc, 2014).

The simplified asymptotic forms are

$n$1

and

$n$2

The paper also studies strengthened $n$3-versions,

$n$4

$n$5

Consistency is defined as exact support recovery over models of bounded size: $n$6 The core identifiability quantity is

$n$7

with $n$8 and $n$9 the projection onto the span of $\mathrm{BIC\_HES}$0. The main separation condition is

$\mathrm{BIC\_HES}$1

Under Gaussian errors and appropriate growth conditions, mBIC and mBIC2 are consistent; under subgaussian errors, consistency is retained for $\mathrm{BIC\_HES}$2 and $\mathrm{BIC\_HES}$3 when $\mathrm{BIC\_HES}$4 is calibrated to the subgaussian parameter through conditions involving $\mathrm{BIC\_HES}$5 (Szulc, 2014).

The conceptual role of these criteria is precise. They are sparsity-aware BIC modifications whose additional $\mathrm{BIC\_HES}$6-type term corrects for the size of the model space. mBIC is more conservative, while mBIC2 weakens the penalty by subtracting $\mathrm{BIC\_HES}$7, which the paper associates with false-discovery-rate-oriented behavior rather than family-wise-error-style conservatism (Szulc, 2014).

3. Bayesian marginal-likelihood variants of AIC

A different modification begins with the frequentist Kullback–Leibler risk of a predictive density $\mathrm{BIC\_HES}$8,

$\mathrm{BIC\_HES}$9

and estimates the associated information target

$Y = X\beta + \varepsilon,$0

The distinctive step is to choose $Y = X\beta + \varepsilon,$1 as a Bayesian marginal likelihood while evaluating its performance from a frequentist viewpoint (Kawakubo et al., 2015).

For linear regression with normal prior

$Y = X\beta + \varepsilon,$2

the marginal likelihood is

$Y = X\beta + \varepsilon,$3

with

$Y = X\beta + \varepsilon,$4

The resulting criterion is

$Y = X\beta + \varepsilon,$5

An asymptotic approximation replaces the determinant term by $Y = X\beta + \varepsilon,$6, yielding

$Y = X\beta + \varepsilon,$7

The paper also gives

$Y = X\beta + \varepsilon,$8

an asymptotically unbiased prior-averaged analogue (Kawakubo et al., 2015).

Under an improper uniform prior on $Y = X\beta + \varepsilon,$9, the marginal likelihood becomes the residual likelihood

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$0

with

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$1

The corresponding criterion is

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$2

and its asymptotic form

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$3

is equivalent to the residual information criterion (RIC) up to an additive model-independent constant (Kawakubo et al., 2015).

These constructions modify AIC in a specific sense: they keep an unbiased- or asymptotically-unbiased-risk-estimation logic but replace the ordinary fitted likelihood by a Bayesian marginal likelihood. The paper’s central claim is that this compromises between Bayesian and frequentist standpoints and yields consistency for selecting the true model in the studied regression setting (Kawakubo et al., 2015).

4. Curvature-aware modified BIC in mixed and hierarchical frameworks

A further modification augments BIC by explicit curvature information from the observed Fisher information or Hessian. The proposed criterion is

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$4

where

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$5

The added term $s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$6 is motivated by a Laplace approximation to the marginal likelihood and is interpreted as incorporating the local geometry of the likelihood surface (Ramirez et al., 3 Jan 2026).

The derivation begins from

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$7

followed by a second-order expansion of

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$8

around its maximizer. This yields the approximation

$s_0 = \{j:\beta_j\neq 0,\ j\in\{1,\dots,p\}\},$9

and thereby the determinant correction in the criterion (Ramirez et al., 3 Jan 2026).

For the hierarchical normal example

$s \subseteq \{1,\dots,p\}$0

the paper computes

$s \subseteq \{1,\dots,p\}$1

hence

$s \subseteq \{1,\dots,p\}$2

This makes explicit that the modified penalty depends not only on parameter count but also on group structure and residual variance (Ramirez et al., 3 Jan 2026).

The asymptotic consistency statement is regular and nested. For a true model $s \subseteq \{1,\dots,p\}$3 nested in an overfitted model $s \subseteq \{1,\dots,p\}$4, with

$s \subseteq \{1,\dots,p\}$5

the paper claims

$s \subseteq \{1,\dots,p\}$6

At the same time, it explicitly notes that the proof is regular and nested and does not address the singular or nonregular behavior typical of true finite mixture models (Ramirez et al., 3 Jan 2026).

This criterion therefore represents a geometry-aware BIC modification. Its stated empirical advantages are strongest for mixed and hierarchical models, especially in small samples and with noise covariates, while its relevance to mixture models is described more as asserted applicability than as a fully developed singular-learning-theory result (Ramirez et al., 3 Jan 2026).

5. Domain-specific MICs: VAR order selection and sparse GLM estimation

The acronym MIC is also used for two domain-specific criteria that are not simple penalty tweaks of BIC.

In vector autoregression, the Mean Square Information Criterion is based on the observation that the profiled mean squared prediction loss is flat once the fitted order reaches or exceeds the true order. For a $s \subseteq \{1,\dots,p\}$7-dimensional VAR,

$s \subseteq \{1,\dots,p\}$8

the profiled population loss is

$s \subseteq \{1,\dots,p\}$9

and the flat-loss theorem states

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$0

The sample criterion uses

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$1

and the practical score is written as

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$2

This MIC is explicitly likelihood-free and uses $\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$3 rather than $\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$4, in contrast to AIC, BIC, and HQ (Hellstern et al., 24 Nov 2025).

In fixed-dimensional generalized linear models, MIC means Minimum approximated Information Criterion. The starting point is the $\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$5-based information-criterion objective

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$6

The paper replaces the indicator by a continuous unit dent function,

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$7

and then introduces the reparameterization

$\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$8

with $\mathrm{BIC}(s)= n\ln\mathrm{RSS}(s)+k(s)\ln n,$9. The implemented MIC criterion becomes

$k(s)=|s|$0

The method is designed to approximate BIC subset selection while maintaining smooth optimization in $k(s)=|s|$1-space; its theory is developed for fixed $k(s)=|s|$2, with oracle-type properties for $k(s)=|s|$3 and asymptotic normality for $k(s)=|s|$4 (Su et al., 2016).

These two constructions share only a family resemblance. The VAR MIC is a trace-of-residual-covariance plus penalty rule derived from flat mean squared loss, while the GLM MIC is a smooth approximation to an $k(s)=|s|$5-penalized information criterion with a reparameterization that restores sparsity (Hellstern et al., 24 Nov 2025, Su et al., 2016).

6. Recurring principles, limitations, and the acronym problem

Across these formulations, the modifications are heterogeneous but structurally legible. In sparse regression, the principal alteration is multiplicity-aware penalization through $k(s)=|s|$6-type terms (Szulc, 2014). In Bayesian-marginal-likelihood criteria, the modification lies in replacing the fitted likelihood by a marginal likelihood while preserving a frequentist KL-risk target (Kawakubo et al., 2015). In $k(s)=|s|$7, the modification is curvature-aware penalization through $k(s)=|s|$8 (Ramirez et al., 3 Jan 2026). In the VAR criterion, the modification is a shift from likelihood to mean squared prediction loss and from determinant to trace (Hellstern et al., 24 Nov 2025). In the GLM construction, the modification is an approximation of the $k(s)=|s|$9-based information criterion itself, together with the smooth map $\mathrm{RSS}(s)$0 (Su et al., 2016).

The limitations are equally specific. The mBIC and mBIC2 theory is for sparse linear regression over models of size at most $\mathrm{RSS}(s)$1, under explicit identifiability and growth conditions (Szulc, 2014). The Bayesian-marginal-likelihood criteria are developed in normal linear regression and depend on the corresponding residual and marginal likelihood calculations (Kawakubo et al., 2015). The $\mathrm{RSS}(s)$2 proof is regular and nested, and the paper states that it does not address the singular asymptotics of true finite mixtures (Ramirez et al., 3 Jan 2026). The Mean Square Information Criterion is proved for correctly specified stable VAR($\mathrm{RSS}(s)$3) processes with white-noise innovations, while its misspecification claims are empirical (Hellstern et al., 24 Nov 2025). The Minimum approximated Information Criterion is restricted to fixed-dimensional GLMs and leaves diverging-dimension extensions to future work (Su et al., 2016).

A final source of confusion is terminological rather than mathematical. In several widely cited papers, MIC does not mean a modified information criterion at all but the maximal information coefficient, defined for dependence measurement through grid-optimized normalized mutual information (Kinney et al., 2013, Reshef et al., 2015). In that literature, the central issues are equitability, mutual information, characteristic matrices, and dependence testing, not model selection. This suggests that any use of “MIC” without expansion is ambiguous and must be resolved from context.