---
title: ForLion Algorithm for Optimal Designs
url: https://www.emergentmind.com/topics/forlion-algorithm
type: topic
---

# ForLion Algorithm for Optimal Designs

Searching arXiv for recent papers on ForLion and related optimal design extensions.
ForLion is a deterministic algorithmic framework for constructing optimal approximate experimental designs in design spaces containing continuous factors, discrete factors, or both. In the literature, it is introduced primarily for locally D-optimal designs under general parametric statistical models with mixed factors, with later extensions to EW D-optimality, A-optimality, and an R implementation. Its defining structure is an iterative alternation among support-point merging, weight optimization by a lift-one procedure, sensitivity-function search for new design points, and an optimality certificate derived from the general equivalence theorem [2309.09367].

## 1. Definition and scope

ForLion was introduced as “a new algorithm for D-optimal designs under general parametric statistical models with mixed factors” [2309.09367]. The design region may contain only discrete factors, only continuous factors, or a mixed-factor product space. One formulation writes the model as
\[
M(\mathbf x; \boldsymbol \theta), \qquad \boldsymbol \theta \in \boldsymbol{\Theta}\subseteq \mathbb{R}^p, \qquad \mathbf x \in \mathcal X \subset \mathbb{R}^d,
\]
with the mixed-factor region
\[
\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,
\]
where the first \(k\) factors are continuous and the remaining \(d-k\) factors are discrete [2507.00923].

A design is represented as
\[
\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},
\]
with support points \(\mathbf x_i\), approximate weights \(w_i\ge 0\), and \(\sum_{i=1}^m w_i=1\) [2507.00923]. Under regularity conditions, its Fisher information matrix is
\[
\mathbf F(\boldsymbol{\xi},\boldsymbol{\theta}) =\sum_{i=1}^m w_i \mathbf F(\mathbf x_i,\boldsymbol{\theta}).
\]
The original ForLion objective is D-optimality, namely maximizing
\[
|\mathbf F(\boldsymbol{\xi},\boldsymbol{\theta})|
\]
for a fixed parameter vector \(\boldsymbol\theta\), which yields a locally D-optimal approximate design [2507.00923].

The later literature treats ForLion as a family rather than a single static algorithm. The 2025 package paper states that the ForLion package implements the ForLion algorithm for locally D-optimal designs and the EW ForLion algorithm for robust EW D-optimal designs under LM, GLM, and MLM settings, and supports both approximate and exact designs [2507.00923]. A separate 2025 paper develops an A-optimal analogue for GLMs with continuous or mixed factors and describes it explicitly as “the A-optimal analogue of the earlier ForLion algorithm for D-optimality” [2507.23240].

## 2. Core algorithmic mechanism

The original ForLion algorithm is described as an iterative procedure with a fixed logical cycle. It starts from an initial design \(\boldsymbol{\xi}_0\) with positive determinant and separated support points, repeatedly merges points that are very close, optimizes the current weights by lift-one, removes zero-weight points, searches for a new candidate point through the sensitivity function, and stops when the equivalence-theorem optimality condition is met [2507.00923].

For D-optimality, the sensitivity function is
\[
d(\mathbf x,\boldsymbol{\xi}_t)=\operatorname{tr}\!\big(\mathbf F(\boldsymbol{\xi}_t)^{-1}\mathbf F_{\mathbf x}\big).
\]
If the maximizer \(\mathbf x^*\) satisfies
\[
d(\mathbf x^*,\boldsymbol{\xi}_t)\le p,
\]
then the current design is D-optimal; otherwise \(\mathbf x^*\) is added and the cycle repeats [2507.00923].

The 2023 ForLion paper emphasizes three ingredients: first-order search for a new point, lift-one optimization of weights on the current support, and a merging step for nearly identical design points [2309.09367]. In its mixed-factor implementation, continuous coordinates are searched by L-BFGS-B, while discrete configurations are handled by enumerating discrete settings and optimizing the continuous block conditionally [2309.09367]. This direct search over the mixed design space is one of the central distinctions of the method: it avoids discretizing the continuous factors into a prespecified candidate grid [2507.00923].

The merging rule is operationally important. If two support points are sufficiently close, they are collapsed into a single point with combined weight. The literature presents this as a mechanism for reducing the number of distinct experimental settings while preserving efficiency [2309.09367]. This suggests that ForLion targets not only statistical optimality but also experimental compactness, a theme made explicit in both the original paper and the package paper [2309.09367].

## 3. Theoretical basis and optimality certification

The algorithm’s optimality guarantee is based on the general equivalence theorem. In the D-optimal setting, the 2023 paper states that a design \(\boldsymbol\xi\) with \(f(\boldsymbol\xi)>0\) is D-optimal if and only if
\[
\max_{\mathbf x\in{\mathcal X}} d(\mathbf x,\boldsymbol\xi)\le p
\]
[2309.09367]. The 2025 package paper reiterates that ForLion is designed to produce a design that is guaranteed to be locally D-optimal when it converges, in contrast to stochastic metaheuristics such as particle swarm optimization [2507.00923].

The same theoretical pattern carries over to later variants. In the EW D-optimal extension, the local information matrix is replaced by its expected or sample-averaged version, and the sensitivity function becomes
\[
d(\mathbf{x},\boldsymbol{\xi})=\operatorname{tr}\!\left({\mathbf F}(\boldsymbol{\xi})^{-1}{\mathbf F}_{\mathbf x}\right),
\]
where \({\mathbf F}(\boldsymbol{\xi})\) is either \(E\{\mathbf F(\boldsymbol{\xi},\boldsymbol{\Theta})\}\) or \(\hat E\{\mathbf F(\boldsymbol{\xi},\boldsymbol{\Theta})\}\) [2505.00629]. The stopping rule remains
\[
d(\mathbf{x}^*,\boldsymbol{\xi}_t)\le p,
\]
and the paper states that the design obtained by the EW ForLion algorithm must be EW D-optimal under the stated assumptions [2505.00629].

For A-optimality, the criterion changes to
\[
h(\boldsymbol\xi) = \left[\operatorname{tr}\big(\mathbf F(\boldsymbol\xi)^{-1}\big)\right]^{-1},
\]
equivalently minimizing \(\operatorname{tr}(\mathbf F^{-1})\), and the sensitivity function becomes
\[
\varphi(\mathbf x,\boldsymbol\xi)
=
\nu(\boldsymbol\beta^T\mathbf q(\mathbf x))
\,\mathbf q(\mathbf x)^T
\mathbf F(\boldsymbol\xi)^{-2}
\mathbf q(\mathbf x).
\]
The A-optimal ForLion reports a design if and only if it is A-optimal among all feasible designs on the compact design region [2507.23240].

A further recurrent theorem across the ForLion literature is sparsity existence: there exists an optimal design with at most
\[
\frac{p(p+1)}{2}
\]
support points in the relevant compact setting [2309.09367]. This upper bound is used to justify the expectation that the returned designs remain sparse.

## 4. Model classes and criterion-specific variants

The ForLion framework is presented as model-agnostic at the level of its outer logic, while relying on model-specific Fisher information and sensitivity calculations. The package paper states that the implementation supports LM, GLM, and MLM [2507.00923].

For GLMs, the literature uses
\[
\mathbf F(\boldsymbol\xi)=\mathbf X_{\boldsymbol\xi}^T\mathbf W_{\boldsymbol\xi}\mathbf X_{\boldsymbol\xi}
=\sum_{i=1}^m w_i\,\nu(\boldsymbol\beta^T\mathbf q(\mathbf x_i))\,\mathbf q(\mathbf x_i)\mathbf q(\mathbf x_i)^T
\]
for A-optimality [2507.23240], and
\[
d({\mathbf x},\boldsymbol\xi) = \nu({\boldsymbol\beta}^T{\mathbf h}({\mathbf x})) \cdot {\mathbf h}({\mathbf x})^T ({\mathbf X}_\xi^T{\mathbf W}_\xi{\mathbf X}_\xi)^{-1} {\mathbf h}({\mathbf x})
\]
for D-optimality [2309.09367]. The package paper lists supported GLM links as logit, probit, cloglog, loglog, cauchit, log, and identity [2507.00923].

For MLMs, the package requires the number of response categories \(J\), a design-matrix function \(hfunc\), its derivative \(h.prime\), the local parameter vector `bvec`, and a function computing Fisher information [2507.00923]. The original ForLion paper treats baseline-category, cumulative, adjacent-categories, and continuation-ratio formulations within a unified framework [2309.09367].

Three principal criterion-specific variants appear in the literature:

| Variant | Objective | Parameter treatment |
|---|---|---|
| ForLion | \(|\mathbf F(\boldsymbol{\xi},\boldsymbol{\theta})|\) | Fixed \(\boldsymbol{\theta}\) |
| EW ForLion | \(\left|E\{\mathbf F(\boldsymbol{\xi},\boldsymbol{\Theta})\}\right|\) or sample analogue | Unknown \(\boldsymbol{\theta}\) averaged over uncertainty |
| A-optimal ForLion | \(\left[\operatorname{tr}(\mathbf F(\boldsymbol\xi)^{-1})\right]^{-1}\) | Local GLM setting |

The EW extension is specifically positioned as a robustness device. It optimizes the determinant of the expected Fisher information rather than the information at a single nominal parameter value and is described as computationally easier than fully Bayesian D-optimal designs while still being robust [2507.00923]. The A-optimal extension keeps the same add-point, merge, and weight-improvement structure but replaces the D-criterion machinery by an A-optimal equivalence theorem and analytic one-dimensional updates [2507.23240].

## 5. Computational features, exact designs, and software realization

A persistent claim across the literature is that ForLion works directly in mixed continuous/discrete spaces rather than first discretizing continuous variables into a candidate set [2507.00923]. This is significant computationally because coarse discretization may miss the optimum, whereas fine discretization may become expensive or unstable [2507.23240]. The A-optimal paper contrasts ForLion with grid-based methods such as REX and reports a grid-resolution tradeoff for discretized continuous factors, whereas ForLion avoids grid selection and produces sparser designs in the example discussed այնտեղ [2507.23240].

The lift-one algorithm functions as the weight-updating engine. For a fixed support set, the \(i\)th weight is changed to \(z\in[0,1]\), with all other weights rescaled proportionally:
\[
\mathbf w_i(z)= \left( \frac{1-z}{1-w_i}w_1,\ldots, \frac{1-z}{1-w_i}w_{i-1}, z, \frac{1-z}{1-w_i}w_{i+1},\ldots, \frac{1-z}{1-w_i}w_m \right)^\top.
\]
The package paper describes this as reducing a multidimensional constrained optimization problem to a sequence of one-dimensional optimizations and notes that it is much faster than many generic optimization methods [2507.00923]. The A-optimal extension goes further by deriving analytic maximizers for the one-dimensional weight update and for the weight assigned to a newly added point [2507.23240].

ForLion first produces approximate designs, but practical experimentation often requires exact designs with integer-valued allocations. The package paper therefore includes a rounding algorithm that merges very close support points using a threshold \(\delta_2\), rounds continuous coordinates to a grid of spacing \(L\), initializes
\[
n_i=\lfloor Nw_i\rfloor,
\]
and allocates remaining units one at a time to maximize the criterion [2507.00923]. The EW D-optimal paper presents a closely related exact-design conversion with user-specified grid levels and total sample size [2505.00629].

The 2025 package paper names the principal exported functions as `ForLion_MLM_Optimal`, `ForLion_GLM_Optimal`, `EW_ForLion_MLM_Optimal`, `EW_ForLion_GLM_Optimal`, `MLM_Exact_Design`, and `GLM_Exact_Design` [2507.00923]. It also documents arguments such as `n.factor`, `factor.level`, `hfunc`, `h.prime`, `bvec`, `bvec_matrix`, `link`, `delta0`, `delta`, `delta2`, `L`, `N`, and `Integral_based` [2507.00923]. This package formalizes ForLion as reusable software rather than only a methodological proposal.

## 6. Empirical behavior, applications, and practical interpretation

The original ForLion paper presents the method as motivated by experimental settings in which cost depends not only on total sample size but also on the number of distinct settings or runs [2309.09367]. Its simulation studies report that ForLion could reduce the number of experimental settings by 25% or improve the relative efficiency of the designs by 17.5% on average [2309.09367]. The package paper likewise emphasizes that direct mixed-space search often produces designs with fewer distinct support points, reducing cost and operational complexity [2507.00923].

Several examples recur in the literature. In the house flies MLM example, the package paper states that ForLion finds a locally D-optimal approximate design with 3 support points for gamma radiation levels, then rounds it to an exact design with near-perfect efficiency [2507.00923]. In the electrostatic discharge GLM example, ForLion finds a 15-point locally D-optimal design for a mixed-factor logistic model, and the exact rounded design with \(L=0.1\) and \(N=500\) retains essentially the same efficiency [2507.00923].

The EW extension is evaluated as a robust alternative to locally D-optimal design. For the paper feeder experiment, the EW ForLion design is reported to have the best determinant value and 100% relative efficiency by definition, while rounded exact designs remain almost as efficient and practical to implement [2505.00629]. In the electrostatic discharge example under parameter uncertainty, the EW ForLion exact design has the highest mean and median determinant-based objective values among the compared methods [2505.00629]. These results are presented as evidence that EW ForLion is more robust against parameter misspecification than locally D-optimal alternatives.

For A-optimality, the 2025 paper reports that in a three-continuous-factor logistic example, ForLion produced an A-optimal design with 8 support points in about 200.02 seconds, whereas REX exhibited a grid-resolution tradeoff and, at one resolution, failed [2507.23240]. The paper’s broader conclusion is that its algorithms can find highly efficient designs with reduced numbers of distinct experimental settings, which may save both experimental time and cost significantly [2507.23240].

A plausible implication of these examples is that ForLion is best understood not simply as an optimizer of a matrix criterion, but as a design-search framework engineered around a joint statistical and operational objective: high criterion efficiency, direct handling of mixed-factor geometry, and support sparsity.

## 7. Nomenclature, relation to other methods, and limitations

The name “ForLion” is specific to optimal design methodology and should be distinguished from the unrelated Lion optimizer literature in machine learning. One 2025 optimization paper explicitly states that it does not define an algorithm named “ForLion”; instead it interprets Lion and Muon as instances of stochastic Frank-Wolfe [2506.04192]. In the experimental-design literature, by contrast, ForLion denotes the mixed-factor design algorithm originating in D-optimal design and then extended to EW D-optimality and A-optimality [2309.09367].

Within optimal design, ForLion is typically contrasted with three classes of alternatives. First, compared with discretize-then-optimize methods, it searches continuous and mixed spaces directly, which avoids dependence on grid resolution [2507.23240]. Second, compared with stochastic metaheuristics such as PSO, it offers a deterministic optimality certificate through the equivalence theorem [2507.00923]. Third, compared with Fedorov-Wynn plus lift-one style methods, it adds an explicit merging step to prevent proliferation of near-duplicate support points [2309.09367].

The literature also makes clear that ForLion is local unless the EW extension is used. The locally D-optimal and A-optimal variants depend on a prespecified parameter vector, so if those values are wrong, the design may be less efficient [2507.00923]. EW ForLion addresses this by averaging the Fisher information under either a prior distribution or a sample distribution from bootstrap or pilot data [2507.00923]. For MLMs, sample-based EW is presented as especially practical because integral-based computation can be difficult when the feasible parameter space is not rectangular [2507.00923].

In summary, ForLion denotes a theoretically certified, sparsity-promoting, mixed-factor design-search framework whose original role was the construction of locally D-optimal approximate designs, and whose later developments extend that framework to robust EW D-optimality, A-optimality, exact-design rounding, and practical software deployment [2507.00923].

Source: https://www.emergentmind.com/topics/forlion-algorithm