ForLion Algorithm for Optimal Designs

• ForLion is a deterministic algorithmic framework for constructing locally D-optimal approximate designs in mixed continuous and discrete spaces.
• It iteratively merges nearby support points, optimizes weights using a lift-one procedure, and employs sensitivity function search to add new design points.
• Extensions to EW D-optimality and A-optimality enhance robustness and reduce experimental cost by ensuring sparser, efficient designs.

Searching arXiv for papers on ForLion and related optimal design extensions. ForLion is a deterministic algorithmic framework for constructing optimal approximate experimental designs in design spaces containing continuous factors, discrete factors, or both. In the literature, it is introduced primarily for locally D-optimal designs under general parametric statistical models with mixed factors, with later extensions to EW D-optimality, A-optimality, and an R implementation. Its defining structure is an iterative alternation among support-point merging, weight optimization by a lift-one procedure, sensitivity-function search for new design points, and an optimality certificate derived from the general equivalence theorem (Huang et al., 2023).

1. Definition and scope

ForLion was introduced as “a new algorithm for D-optimal designs under general parametric statistical models with mixed factors” (Huang et al., 2023). The design region may contain only discrete factors, only continuous factors, or a mixed-factor product space. One formulation writes the model as

$$M(\mathbf x; \boldsymbol \theta), \qquad \boldsymbol \theta \in \boldsymbol{\Theta}\subseteq \mathbb{R}^p, \qquad \mathbf x \in \mathcal X \subset \mathbb{R}^d,$$

with the mixed-factor region

$\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$

where the first $k$ factors are continuous and the remaining $d-k$ factors are discrete (Lin et al., 1 Jul 2025).

A design is represented as

$$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$

with support points $\mathbf x_i$, approximate weights $w_i\ge 0$, and $\sum_{i=1}^m w_i=1$ (Lin et al., 1 Jul 2025). Under regularity conditions, its Fisher information matrix is

$$\mathbf F(\boldsymbol{\xi},\boldsymbol{\theta}) =\sum_{i=1}^m w_i \mathbf F(\mathbf x_i,\boldsymbol{\theta}).$$

The original ForLion objective is D-optimality, namely maximizing

$|\mathbf F(\boldsymbol{\xi},\boldsymbol{\theta})|$

for a fixed parameter vector $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$0, which yields a locally D-optimal approximate design (Lin et al., 1 Jul 2025).

The later literature treats ForLion as a family rather than a single static algorithm. The 2025 package paper states that the ForLion package implements the ForLion algorithm for locally D-optimal designs and the EW ForLion algorithm for robust EW D-optimal designs under LM, GLM, and MLM settings, and supports both approximate and exact designs (Lin et al., 1 Jul 2025). A separate 2025 paper develops an A-optimal analogue for GLMs with continuous or mixed factors and describes it explicitly as “the A-optimal analogue of the earlier ForLion algorithm for D-optimality” (Yang et al., 31 Jul 2025).

2. Core algorithmic mechanism

The original ForLion algorithm is described as an iterative procedure with a fixed logical cycle. It starts from an initial design $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$1 with positive determinant and separated support points, repeatedly merges points that are very close, optimizes the current weights by lift-one, removes zero-weight points, searches for a new candidate point through the sensitivity function, and stops when the equivalence-theorem optimality condition is met (Lin et al., 1 Jul 2025).

For D-optimality, the sensitivity function is

$\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$2

If the maximizer $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$3 satisfies

$\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$4

then the current design is D-optimal; otherwise $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$5 is added and the cycle repeats (Lin et al., 1 Jul 2025).

The 2023 ForLion paper emphasizes three ingredients: first-order search for a new point, lift-one optimization of weights on the current support, and a merging step for nearly identical design points (Huang et al., 2023). In its mixed-factor implementation, continuous coordinates are searched by L-BFGS-B, while discrete configurations are handled by enumerating discrete settings and optimizing the continuous block conditionally (Huang et al., 2023). This direct search over the mixed design space is one of the central distinctions of the method: it avoids discretizing the continuous factors into a prespecified candidate grid (Lin et al., 1 Jul 2025).

The merging rule is operationally important. If two support points are sufficiently close, they are collapsed into a single point with combined weight. The literature presents this as a mechanism for reducing the number of distinct experimental settings while preserving efficiency (Huang et al., 2023). This suggests that ForLion targets not only statistical optimality but also experimental compactness, a theme made explicit in both the original paper and the package paper (Huang et al., 2023).

3. Theoretical basis and optimality certification

The algorithm’s optimality guarantee is based on the general equivalence theorem. In the D-optimal setting, the 2023 paper states that a design $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$6 with $\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$7 is D-optimal if and only if

$\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$8

(Huang et al., 2023). The 2025 package paper reiterates that ForLion is designed to produce a design that is guaranteed to be locally D-optimal when it converges, in contrast to stochastic metaheuristics such as particle swarm optimization (Lin et al., 1 Jul 2025).

The same theoretical pattern carries over to later variants. In the EW D-optimal extension, the local information matrix is replaced by its expected or sample-averaged version, and the sensitivity function becomes

$\mathcal X=\prod_{j=1}^k I_j \times \mathcal D,$9

where $k$0 is either $k$1 or $k$2 (Lin et al., 1 May 2025). The stopping rule remains

$k$3

and the paper states that the design obtained by the EW ForLion algorithm must be EW D-optimal under the stated assumptions (Lin et al., 1 May 2025).

For A-optimality, the criterion changes to

$k$4

equivalently minimizing $k$5, and the sensitivity function becomes

$k$6

The A-optimal ForLion reports a design if and only if it is A-optimal among all feasible designs on the compact design region (Yang et al., 31 Jul 2025).

A further recurrent theorem across the ForLion literature is sparsity existence: there exists an optimal design with at most

$k$7

support points in the relevant compact setting (Huang et al., 2023). This upper bound is used to justify the expectation that the returned designs remain sparse.

4. Model classes and criterion-specific variants

The ForLion framework is presented as model-agnostic at the level of its outer logic, while relying on model-specific Fisher information and sensitivity calculations. The package paper states that the implementation supports LM, GLM, and MLM (Lin et al., 1 Jul 2025).

For GLMs, the literature uses

$k$8

for A-optimality (Yang et al., 31 Jul 2025), and

$k$9

for D-optimality (Huang et al., 2023). The package paper lists supported GLM links as logit, probit, cloglog, loglog, cauchit, log, and identity (Lin et al., 1 Jul 2025).

For MLMs, the package requires the number of response categories $d-k$0, a design-matrix function $d-k$1, its derivative $d-k$2, the local parameter vector bvec, and a function computing Fisher information (Lin et al., 1 Jul 2025). The original ForLion paper treats baseline-category, cumulative, adjacent-categories, and continuation-ratio formulations within a unified framework (Huang et al., 2023).

Three principal criterion-specific variants appear in the literature:

Variant	Objective	Parameter treatment
ForLion	$d-k$3	Fixed $d-k$4
EW ForLion	$d-k$5 or sample analogue	Unknown $d-k$6 averaged over uncertainty
A-optimal ForLion	$d-k$7	Local GLM setting

The EW extension is specifically positioned as a robustness device. It optimizes the determinant of the expected Fisher information rather than the information at a single nominal parameter value and is described as computationally easier than fully Bayesian D-optimal designs while still being robust (Lin et al., 1 Jul 2025). The A-optimal extension keeps the same add-point, merge, and weight-improvement structure but replaces the D-criterion machinery by an A-optimal equivalence theorem and analytic one-dimensional updates (Yang et al., 31 Jul 2025).

5. Computational features, exact designs, and software realization

A persistent claim across the literature is that ForLion works directly in mixed continuous/discrete spaces rather than first discretizing continuous variables into a candidate set (Lin et al., 1 Jul 2025). This is significant computationally because coarse discretization may miss the optimum, whereas fine discretization may become expensive or unstable (Yang et al., 31 Jul 2025). The A-optimal paper contrasts ForLion with grid-based methods such as REX and reports a grid-resolution tradeoff for discretized continuous factors, whereas ForLion avoids grid selection and produces sparser designs in the example discussed այնտեղ (Yang et al., 31 Jul 2025).

The lift-one algorithm functions as the weight-updating engine. For a fixed support set, the $d-k$8th weight is changed to $d-k$9, with all other weights rescaled proportionally: $$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$0 The package paper describes this as reducing a multidimensional constrained optimization problem to a sequence of one-dimensional optimizations and notes that it is much faster than many generic optimization methods (Lin et al., 1 Jul 2025). The A-optimal extension goes further by deriving analytic maximizers for the one-dimensional weight update and for the weight assigned to a newly added point (Yang et al., 31 Jul 2025).

ForLion first produces approximate designs, but practical experimentation often requires exact designs with integer-valued allocations. The package paper therefore includes a rounding algorithm that merges very close support points using a threshold $$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$1, rounds continuous coordinates to a grid of spacing $$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$2, initializes

$$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$3

and allocates remaining units one at a time to maximize the criterion (Lin et al., 1 Jul 2025). The EW D-optimal paper presents a closely related exact-design conversion with user-specified grid levels and total sample size (Lin et al., 1 May 2025).

The 2025 package paper names the principal exported functions as ForLion_MLM_Optimal, ForLion_GLM_Optimal, EW_ForLion_MLM_Optimal, EW_ForLion_GLM_Optimal, MLM_Exact_Design, and GLM_Exact_Design (Lin et al., 1 Jul 2025). It also documents arguments such as n.factor, factor.level, hfunc, h.prime, bvec, bvec_matrix, link, delta0, delta, delta2, L, N, and Integral_based (Lin et al., 1 Jul 2025). This package formalizes ForLion as reusable software rather than only a methodological proposal.

6. Empirical behavior, applications, and practical interpretation

The original ForLion paper presents the method as motivated by experimental settings in which cost depends not only on total sample size but also on the number of distinct settings or runs (Huang et al., 2023). Its simulation studies report that ForLion could reduce the number of experimental settings by 25% or improve the relative efficiency of the designs by 17.5% on average (Huang et al., 2023). The package paper likewise emphasizes that direct mixed-space search often produces designs with fewer distinct support points, reducing cost and operational complexity (Lin et al., 1 Jul 2025).

Several examples recur in the literature. In the house flies MLM example, the package paper states that ForLion finds a locally D-optimal approximate design with 3 support points for gamma radiation levels, then rounds it to an exact design with near-perfect efficiency (Lin et al., 1 Jul 2025). In the electrostatic discharge GLM example, ForLion finds a 15-point locally D-optimal design for a mixed-factor logistic model, and the exact rounded design with $$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$4 and $$\boldsymbol{\xi}=\{(\mathbf x_i,w_i),\, i=1,\ldots,m\},$$5 retains essentially the same efficiency (Lin et al., 1 Jul 2025).

The EW extension is evaluated as a robust alternative to locally D-optimal design. For the paper feeder experiment, the EW ForLion design is reported to have the best determinant value and 100% relative efficiency by definition, while rounded exact designs remain almost as efficient and practical to implement (Lin et al., 1 May 2025). In the electrostatic discharge example under parameter uncertainty, the EW ForLion exact design has the highest mean and median determinant-based objective values among the compared methods (Lin et al., 1 May 2025). These results are presented as evidence that EW ForLion is more robust against parameter misspecification than locally D-optimal alternatives.

For A-optimality, the 2025 paper reports that in a three-continuous-factor logistic example, ForLion produced an A-optimal design with 8 support points in about 200.02 seconds, whereas REX exhibited a grid-resolution tradeoff and, at one resolution, failed (Yang et al., 31 Jul 2025). The paper’s broader conclusion is that its algorithms can find highly efficient designs with reduced numbers of distinct experimental settings, which may save both experimental time and cost significantly (Yang et al., 31 Jul 2025).

A plausible implication of these examples is that ForLion is best understood not simply as an optimizer of a matrix criterion, but as a design-search framework engineered around a joint statistical and operational objective: high criterion efficiency, direct handling of mixed-factor geometry, and support sparsity.

7. Nomenclature, relation to other methods, and limitations

The name “ForLion” is specific to optimal design methodology and should be distinguished from the unrelated Lion optimizer literature in machine learning. One 2025 optimization paper explicitly states that it does not define an algorithm named “ForLion”; instead it interprets Lion and Muon as instances of stochastic Frank-Wolfe (Sfyraki et al., 4 Jun 2025). In the experimental-design literature, by contrast, ForLion denotes the mixed-factor design algorithm originating in D-optimal design and then extended to EW D-optimality and A-optimality (Huang et al., 2023).

Within optimal design, ForLion is typically contrasted with three classes of alternatives. First, compared with discretize-then-optimize methods, it searches continuous and mixed spaces directly, which avoids dependence on grid resolution (Yang et al., 31 Jul 2025). Second, compared with stochastic metaheuristics such as PSO, it offers a deterministic optimality certificate through the equivalence theorem (Lin et al., 1 Jul 2025). Third, compared with Fedorov-Wynn plus lift-one style methods, it adds an explicit merging step to prevent proliferation of near-duplicate support points (Huang et al., 2023).

The literature also makes clear that ForLion is local unless the EW extension is used. The locally D-optimal and A-optimal variants depend on a prespecified parameter vector, so if those values are wrong, the design may be less efficient (Lin et al., 1 Jul 2025). EW ForLion addresses this by averaging the Fisher information under either a prior distribution or a sample distribution from bootstrap or pilot data (Lin et al., 1 Jul 2025). For MLMs, sample-based EW is presented as especially practical because integral-based computation can be difficult when the feasible parameter space is not rectangular (Lin et al., 1 Jul 2025).

In summary, ForLion denotes a theoretically certified, sparsity-promoting, mixed-factor design-search framework whose original role was the construction of locally D-optimal approximate designs, and whose later developments extend that framework to robust EW D-optimality, A-optimality, exact-design rounding, and practical software deployment (Lin et al., 1 Jul 2025).