Ordered Lasso for Network Inference

• The paper introduces TL-O-Lasso as a novel regularized regression approach that integrates an ℓ1 penalty with temporal monotonicity constraints to recover gene regulatory networks.
• The method employs a lag-stacked design matrix and automatic lag selection, outperforming traditional Granger causality and unconstrained Lasso, especially under sparse and short time-series conditions.
• The semi-supervised extension incorporates prior network knowledge by applying differentiated penalties, enhancing the identification of both known and novel regulatory edges in complex biological systems.

Ordered Lasso is a class of regularized regression methods that enforce a monotonicity structure on regression coefficients. In the context of network inference, particularly for gene regulatory networks (GRNs) from time-course gene expression data, the time-lagged Ordered Lasso (TL-O-Lasso) provides a principled framework for reconstructing network structure while imposing biologically motivated temporal constraints. TL-O-Lasso integrates an $\ell_1$ (Lasso) sparsity-inducing penalty and temporal monotonicity constraints, yielding interpretable models that are robust in high-dimensional, low-sample-size regimes (Nguyen et al., 2018). The approach has been shown to address limitations of unconstrained lasso and Granger-causal methods in sparse or short time-series, and it naturally facilitates semi-supervised incorporation of prior network knowledge.

1. Mathematical Principles of Time-lagged Ordered Lasso

The central innovation of TL-O-Lasso lies in its formulation of the network inference problem as a sequence of regularized linear regressions with monotonicity constraints on lagged influence. For a target gene $i$, its expression $x_i(t)$ at time $t$ is modeled as a function of the past $L$ time points of all $p$ possible regulators: $$x_i(t) = \sum_{j=1}^p \sum_{k=1}^L \beta_{ij,k} x_j(t - k\,\Delta t) + r_i(t)$$ where $\beta_{ij,k}$ denotes the effect size of gene $j$ at lag $k$ on target $i$0, and $i$1 is the residual. The optimization objective is: $i$2 subject to monotonicity on each regulator’s coefficients: $i$3 The $i$4 penalty induces sparsity, while the monotonicity constraint enforces that the influence of past expressions cannot increase with lag, aligning with many biological systems where recent states are most predictive (Nguyen et al., 2018).

2. Computational Implementation and Parameter Tuning

Implementation involves constructing a lag-stacked design matrix that includes up to $i$5 lags for all $i$6 potential regulators. The practitioner selects the maximum lag $i$7 (typically chosen as large as the sampling and dataset permit), but need not tune $i$8 precisely; the monotonicity constraint inherently “turns off” higher-order lags if not supported by data. The only penalty parameter in the basic (de novo) model, $i$9, is varied across a regularization path. Network-level performance is assessed by the order in which edges (nonzero $x_i(t)$0) appear as $x_i(t)$1 decreases (Nguyen et al., 2018).

3. Semi-supervised Extension Incorporating Prior Networks

The semi-supervised TL-O-Lasso extends the penalty structure to differentiate edges that are present in a prior network $x_i(t)$2 from those that are not. The objective becomes

$x_i(t)$3

with monotonicity constraints maintained. Edges in the prior network incur a lower penalty ($x_i(t)$4), making them more likely to be retained, while novel edges face a higher threshold ($x_i(t)$5) and only persist when strongly supported by data. This scheme allows simultaneous identification of “novel” regulatory links and possible anomalous removals of putative known edges (Nguyen et al., 2018).

4. Empirical Evaluation and Benchmarking

TL-O-Lasso has been evaluated on synthetic and real gene expression datasets, including:

• Repressilator simulation: A three-gene synthetic oscillator, varying sampling densities and durations as well as model orders $x_i(t)$6.
• DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenges: DREAM2 (50 genes), DREAM3 (10, 50, 100 genes), and DREAM4 (10, 100 genes), incorporating both prokaryotic and eukaryotic regulatory networks.
• HeLa cell-cycle subnetwork: Real data for nine genes across 47 time points, with prior GRN structure from Sambo et al. (2008) and an updated BioGRID network.

Performance metrics follow accepted standards:

• ROC-AUC as $x_i(t)$7 or $x_i(t)$8 varies, quantifying the ranking of true vs. false positive edges
• Precision, recall, F1 score at fixed penalty
• Novel-edge AUC for ranking new predictions (semi-supervised setting)

In the repressilator, TL-O-Lasso attains near-perfect AUC for moderate sampling densities and durations, robustly outperforms Granger causality and unconstrained Lasso in undersampled settings, and requires no delicate tuning of $x_i(t)$9. On large DREAM networks, it achieves competitive AUC that does not degrade for larger lags, unlike comparator methods. On HeLa data, TL-O-Lasso yields the highest ROC-AUC and F1 scores against updated ground truths, and successfully recovers novel regulatory connections (Nguyen et al., 2018).

5. Interpretation, Advantages, and Limitations

Key advantages include:

• Automatic lag selection: The monotonicity constraint selects the most predictive lag per regulator, eliminating the need for manual maximal lag search.
• Sparsity and interpretability: The $t$0 penalty restricts the solution to a parsimonious set of edges, critical in regimes with $t$1.
• Flexible incorporation of prior information: The semi-supervised penalty allows seamless integration of existing biological knowledge and provides a systematic method for ranking novel and anomalous edges.
• Practical scalability: Since the problem decomposes into $t$2 separate convex programs (one per gene), TL-O-Lasso is computationally tractable even for genome-wide datasets.

Limitations are also explicit:

• Assumed linearity: The method presumes a linear response, which may not hold for real GRNs characterized by nonlinear, threshold, or combinatorial interactions.
• Fixed monotonicity: Regulatory effects that peak at an intermediate lag (e.g., delayed or biphasic responses) may be missed.
• Noise sensitivity: High noise or extremely short time series still reduce performance; the method partially alleviates, but does not eliminate, small-sample risks.
• Penalty selection: Cross-validation or information criteria are recommended for $t$3 tuning, though theoretical guarantees in the semi-supervised case are heuristic (Nguyen et al., 2018).

6. Broader Context and Related Methods

Ordered Lasso approaches connect to a wider literature on regularized regression for variable selection and structure learning. While classical Lasso targets sparsity, the addition of monotonicity constraints is motivated by temporal dynamics, especially in biological systems. TL-O-Lasso distinguishes itself from pairwise Granger causality and unconstrained Lasso–Granger models by systematically limiting spurious long-lag effects and enabling high-resolution recovery of dynamic networks as validated on DREAM and empirical datasets.

For ordinal or zero-inflated response settings, other penalized models such as polytomous ordinal logistic regression with Lasso and stability selection mechanisms or knockoff-based importance ordering have also been proposed for network inference, further illustrating the breadth of Lasso-based inference frameworks (Deveau et al., 2018). This suggests a unifying role for structured penalties in extracting interpretable, data-driven networks from high-throughput, noisy, or non-Gaussian measurements.

---

Key References:

• "Time-lagged Ordered Lasso for network inference" (Nguyen et al., 2018)
• "Penalized polytomous ordinal logistic regression using cumulative logits. Application to network inference of zero-inflated variables" (Deveau et al., 2018)