Nonlinear Dose-Response Curves

• Non-linear dose-response curves are quantitative models that capture complex, non-proportional relationships between dose and effect, exhibiting shapes such as Emax, sigmoidal, and hormetic profiles.
• They leverage advanced methodologies like hierarchical non-linear mixed-effects models, Bayesian design, and nonparametric splines to enhance estimation and experimental design in clinical and toxicological research.
• Accurate inference under non-linearity requires addressing challenges like model uncertainty, parameter identifiability, and integration of biological mechanisms to optimize dose-selection.

A non-linear dose-response curve models the functional relationship between a quantitative input (dose) and its effect (response) when that relationship cannot be adequately described by a linear relation. Such curves are ubiquitous across pharmacology, toxicology, epidemiology, and statistical causal inference, arising both from fundamental biological mechanisms (e.g., receptor binding kinetics, radiobiology) and from pragmatic trial design (e.g., optimal dose-finding). Non-linearity encompasses a wide variety of shapes: saturating (Emax), sigmoidal (Hill, logistic), U-shaped (hormetic), biphasic, and curves exhibiting convex or concave regions with implications for clinical decision-making. Rigorous estimation, experimental design, and inference in the presence of non-linearity have become central to both applied and methodological research.

1. Canonical Non-linear Dose-Response Models

Several parametric models form the foundation for non-linear dose-response analysis:

• Emax model:

$$f_{\rm Emax}(d;\theta_0,\theta_1,\theta_2) = \theta_0 + \frac{\theta_1\,d}{\theta_2 + d}$$

with $\theta_0$ as baseline (placebo), $\theta_1$ the maximum incremental effect, and $\theta_2$ the half-maximal dose (Schorning et al., 2017).

• Exponential model:

$$f_{\exp}(d;\theta_0,\theta_1,\theta_2) = \theta_0 + \theta_1\bigl(e^{d/\theta_2} - 1\bigr)$$

• Linear-in-log:

$$f_{\log}(d;\theta_0,\theta_1,\theta_2) = \theta_0 + \theta_1 \ln\bigl(1 + \frac{d}{\theta_2}\bigr)$$

• 4-parameter log-logistic (widely used in toxicology and plant sciences):

$$f(d;A,B,C,D) = A + \frac{B-A}{1 + \exp\left[-C(\ln d - \ln D)\right]}$$

where $A$ and $B$ are lower and upper asymptotes, $C$ the slope, and $D$ the ED$_{50}$ (Gerhard et al., 2017).

• Sigmoid Emax/Hill model:

$$f(d;E_0,E_{\max},ED_{50},h) = E_0 + \frac{E_{\max}\,d^h}{ED_{50}^h + d^h}$$

These curves capture saturating behavior, inflection points, and nonlinear sensitivity to dose and enable estimation of effective dose metrics (e.g., ED$_{x}$). For biphasic or hormetic relationships, specialized forms such as the Brain–Cousens and Cedergreen models are required to describe inverted-U effects (Abbaraju et al., 2023).

2. Hierarchical and Mixed-Effects Modeling of Non-linearity

Real-world data are often clustered (e.g., by subject, assay, center), necessitating hierarchical non-linear mixed-effects (NLME) models:

$$\begin{align*} y_{ij} &= f(d_{ij}; \boldsymbol\theta_i) + \epsilon_{ij},\quad \epsilon_{ij} \sim N(0, \sigma^2) \ \boldsymbol\theta_i &= \boldsymbol\beta + \mathbf b_i,\quad \mathbf b_i \sim N(\mathbf{0}, \mathbf G) \end{align*}$$

The marginal (population-average) curve, generally not coincident with the conditional (subject-specific) curve due to the non-linear $f$, is:

$$\bar f(d) = \int f(d; \boldsymbol\beta + \mathbf b) \phi(\mathbf b; 0, G) \, d\mathbf b$$

This integral is typically computed by Gaussian-Hermite quadrature (Gerhard et al., 2017). Marginalization is essential for unbiased inference on population-level parameters, especially for dose-reference metrics such as ED$_{x}$.

3. Bayesian and Model-Free Approaches

Bayesian methodologies provide robustness to parameter and model uncertainty, especially for non-linear dose-response:

• Bayesian D-optimal design: Computes optimal dose allocations that maximize the expected information (determinant of Fisher information averaged over a prior) (Schorning et al., 2017).
• MAP-curvature: Penalizes total curvature of the dose-response function via a prior, with SEMAP-curvature using a sigmoid Emax as the default, thus encouraging pharmacologically plausible non-linear shapes (Han et al., 28 Sep 2025).
• Bayesian AM-spline: A nonparametric spline basis (mixtures of monotone kernels) with stick-breaking priors for weights, automatically enforcing monotonicity, positivity, and capturing arbitrary smooth non-linearity (Alamri et al., 2019).
• Rolling-pin copula methods: Flexible approaches for non-monotonic (biphasic/hormetic) curves, modeling the joint density via a monotonization transform and a copula, outperforming classic parametric hormesis fits in RMSE and parameter economy (Tahrir, 2017).

Bayesian and nonparametric methods are particularly advantageous in settings with model uncertainty or complex non-linear patterns not captured by standard forms.

4. Inference, Testing, and Model Selection under Non-linearity

Statistical inference for non-linear dose-response is complicated by boundary, identifiability, and shape constraints:

• Likelihood ratio tests: When testing for a dose effect across a candidate set of (nonlinear) models, classical asymptotics break down due to non-identifiability of curvature parameters under the null. The tube-based LR test controls type I error and power by geometric integration on the sphere of model fits (Gutjahr et al., 2015).
• Multiple-comparison procedures (MCPMod): Compares candidate shapes by optimal contrast tests and refits the best models to the data using two-stage GLS. Model selection or averaging uses information criteria (gAIC, BIC) (Pinheiro et al., 2013).
• Causal inference with continuous treatments: Kernel ridge regression (RKHS estimators), DR-learners, and higher-order influence function methods allow nonparametric estimation of $\mu(a)=\mathbb{E}[Y^a]$ for non-linearly identified dose–response functions, providing closed-form solutions, minimax rates, and sensitivity analysis under unmeasured confounding (Singh et al., 2020, Bonvini et al., 2022, Zhang et al., 15 May 2024).

Methods accounting for model flexibility, non-identifiability, and geometric complexity are critical for valid inference and power estimation in the presence of non-linear dose-response.

5. Biological and Clinical Mechanisms Driving Non-linearity

Mechanistic origins of non-linearity include:

• Receptor kinetics and occupancy: Saturating (Emax) and sigmoidal (Hill) shapes reflect binding, occupancy, and signaling thresholds.
• Radiobiological response: The linear-quadratic (LQ) model describes cell survival curves under ionizing radiation; deviations at very high doses require overdispersed (negative binomial) models or LQ-linear (LQL) forms to capture indirect effects and saturation (Loan et al., 2021, Gago-Arias et al., 2022).
• Antifragility and clinical optimization: The convexity or concavity of a non-linear response governs the effect of dose variability (via Jensen's inequality). Convex ("antifragile") regions benefit from higher variance dosing schedules, whereas concave ("fragile") regions are harmed—this leads to dynamic optimization of dose scheduling in oncology and other applications (Taleb et al., 2022).
• Hormetic response: Inverted-U relationships, characteristic of hormesis, arise from adaptive cellular responses, requiring dedicated models for accurate quantification (Abbaraju et al., 2023, Tahrir, 2017).

Understanding the mechanistic basis of non-linearity is essential for both model specification and interpretation of fitted curves.

6. Design, Power, and Practical Implementation

Optimal experimental design under non-linearity emphasizes robustness and efficiency:

• Saturated Bayesian D-optimal designs: In trial settings with common parameters (e.g., multi-regimen studies), analytical characterizations yield saturated designs with minimal dose points but near-optimal information, simplifying trial logistics (Schorning et al., 2017).
• Model selection in the presence of heterogeneity: Hierarchical Bayesian models, such as cubic spline meta-analysis, accommodate heterogeneity in shape across studies, exposures, or clusters, yielding robust estimation with explicit uncertainty quantification (Hamza et al., 2020).
• Historical borrowing: Bayesian frameworks integrate historical trial data to stabilize estimates and increase power, with quantified gains depending on the overlap of dose levels and steepness region coverage (Han et al., 28 Sep 2025).

Practical recommendations include choosing flexible non-linear models or nonparametric bases whenever prior knowledge of the curve shape is limited or when mechanistic non-linearity is expected.

7. Emerging Methodologies and Open Challenges

Advanced research directions and challenges include:

• Nonparametric identification without positivity: Recent work demonstrates nonparametric identification and inference on dose–response curves without the positivity condition, via integration of derivative estimates (Zhang et al., 15 May 2024).
• Estimation in high-dimensional causal settings: Kernel methods and sequential kernel embedding handle mediation, time-varying treatments, and feedback, providing finite-sample convergence guarantees even with multiple continuous doses and high-dimensional covariates (Singh et al., 2021).
• Robustness to overdispersion and biological variability: Incorporation of overdispersion (e.g., negative-binomial lesion counts) and hierarchical modeling strengthens robustness and aligns statistical fits with biological processes (Loan et al., 2021, Gerhard et al., 2017).

Despite these advances, open problems remain in the empirical discrimination among competing non-linear models especially under small sample sizes, in integrating mechanistic priors into flexible (e.g., Gaussian process) modeling, and in ensuring interpretability and transportability of estimates across settings.

---

Research on non-linear dose–response curves continues to play a foundational role in biostatistics, pharmacometrics, and clinical trial methodology, with ongoing innovation in model specification, inference, optimal design, and integration of mechanistic and empirical knowledge. The interplay of Bayesian, nonparametric, and mechanistic approaches enables increasingly robust and interpretable characterizations of complex biological dose–response relationships.