Time-Dependent Utility Models

• Time-dependent utility is defined as a framework where the value of outcomes varies with time, incorporating discount functions like exponential, hyperbolic, and quasi-hyperbolic models.
• Formal models use dynamic programming and variational discounting to capture phenomena such as present bias, time inconsistency, and risk neutrality in non-ergodic contexts.
• Applications span portfolio selection, medical decision support, and empirical identification in behavioral economics, linking stochastic dynamics with optimal time-sensitive decision-making.

Time-dependent utility refers to formal models in which the utility derived from outcomes, actions, or consumption explicitly varies as a function of time, temporal context, or the history of choices. It arises in intertemporal choice theory, dynamic programming, behavioral economics, control, and computational decision theory. Time dependence can originate from discounting of delayed rewards, stochastic timing, endogenous wealth or consumption dynamics, state- or history-dependent preference updating, or exogenous features such as resource deadlines and time pressure.

1. Formal Models of Time-Dependent Utility

The canonical framework posits utility functions parameterized by time, either directly or through discount functions. For a stream $x=(x_0, x_1, \dots, x_T)$, total utility is frequently modeled as

$U(x) = \sum_{t=0}^{T} D(t) \, u(x_t),$

where $u(\cdot)$ is a per-period von Neumann–Morgenstern utility and $D(t)$ is a time-dependent, typically non-increasing, discount factor.

Prominent specifications include:

• Exponential Discounting: $D(t) = \delta^t$ with $\delta \in (0,1)$, yielding time-consistent preferences and a unique present-value operator.
• Hyperbolic Discounting: $D(t) = 1/(1 + k t)$, exhibiting diminishing impatience and capturing empirical deviations from the exponential model.
• Quasi-Hyperbolic ($\beta$–$\delta$) Discounting: $D(0)=1,\ D(t)=\beta\delta^{t-1}$ for $t \geq 1$, introducing present bias.

Beyond fixed forms, polynomial-parameterized discount families allow $D(t) = Q_t(\theta)$ where $Q_t$ are polynomials and $\theta$ is an unknown parameter, efficiently subsuming standard models and enabling learnability results with low VC dimension (Chase et al., 2018).

Time dependence also arises in the time-dependent utility of actions, as in resource-bounded reasoning. Here, utility $u(A_i,H_j,t)$ assigned to action $A_i$ under hypothesis $H_j$ is an explicit function of elapsed time $t$ (or deadline), often decaying exponentially $u(A_i,H_j,t) = u(A_i,H_j,t_0)\exp(-k_{ij}(t-t_0))$ or linearly (Horvitz et al., 2013).

2. Discounting, Dynamic Consistency, and Variational Frameworks

Discounting future utility is standard in intertemporal models, but the choice of discount function has significant normative and empirical implications. The assumption of a universal discount rate is contested.

A unifying generalization is the variational discounting criterion (Dong-Xuan et al., 2024):

$$I(x) = \min_{\delta \in [0,1)}\{ D_\delta(x) + c(\delta) \}, \quad D_\delta(x) = (1-\delta) \sum_{t=0}^\infty \delta^t x_t,$$

where $c(\delta)$ is a convex penalty encoding plausibility or confidence in $\delta$. As special cases, single $\delta$ recovers exponential, finite sets $E$ recover max–min robust approaches, and mixtures induce hyperbolic schedules as

$$h(t) = \int_0^1 (1 - \delta) \delta^t \gamma(d\delta).$$

Time inconsistency arises when $D(t,s)$ is non-exponential (e.g., $\beta(t)$ time-varying), breaking dynamic consistency. This necessitates state-augmentation for dynamic programming, as in models for inferring utility and discount rates from observed sequential policies (Cao et al., 2024).

Variational discounting framework resolves conflicts in social discount rates by aggregating over expert-recommended rates and enables equal weighting of distant future via Banach–Mazur–type limits.

3. Time-Dependent Utility in Stochastic Dynamics and Ergodicity

A major theme is the distinction between maximizing expected utility (ensemble average) and maximizing time-average growth—particularly when observables are non-ergodic.

The ergodicity economics program posits that for non-ergodic processes (such as multiplicative wealth dynamics), only time-average growth rates capture experienced returns. For additive processes ($x_{t+1} = x_t + \Delta x$), linear utility ($u(x)=x$) is optimal; for multiplicative processes ($x_{t+1} = x_t(1 + r)$), logarithmic utility ($u(x) = \ln x$) is optimal (Peters et al., 2018, Meder et al., 2019). Growth-optimality implies that the appropriate utility function is determined by the underlying wealth evolution SDE, with the ergodicity transformation

$$u(x) = \int^x \frac{dz}{b(z)} \exp\left( -\int^z \frac{2a(y)}{b^2(y)} dy \right ),$$

where $a(x), b(x)$ are drift and diffusion (Peters et al., 2018).

Applied to preferences over lotteries with timing uncertainty, time-average growth yields risk neutrality (RNTL), contrasting with ensemble-based expected discounted utility theory (EDUT), which predicts persistent risk seeking (RSTL) for temporally risky alternatives. Empirical studies systematically falsify EDUT in favor of the time-average growth model: subjects are nearly risk-neutral when ensemble–time average gaps are small, but exhibit increasing aversion as this gap widens (Berman et al., 2021, Meder et al., 2019).

4. Learning, Identification, and Robust Optimization with Time-Dependent Utility

The learnability and inference of models incorporating time-dependent utility is addressed by reducing sample complexity for polynomial discount parameterizations, enabling VC dimension logarithmic in the planning horizon (Chase et al., 2018). Discrete-time and continuous-time frameworks with non-exponential/variable discounting allow the unique identification of both utility curvature and time-preference rates from observed Markov policies, utilizing dynamic programming and Legendre duality (Cao et al., 2024).

Robust multistage preference models must accommodate ambiguity in stage-dependent utility. When per-stage utility $u_t$ can depend on history and the ambiguity set is rectangular (state/history adapted), the optimal policy can be solved recursively via stagewise dynamic programming, preserving time consistency. In contrast, forced state independence of utility generates time inconsistency (Liu et al., 2021).

Piecewise-linear approximations—supported by grid discretization and with provable error bounds—are practical for computationally tractable preference robust optimization under time-dependent utilities.

5. Application Domains: Bounded Reasoning, Portfolio Choice, and Empirical Identification

Time-dependent utility is central in the control of inference under time pressure, exemplified by systems like Protos for medical decision support. Here, the value of continued computation is weighed against decaying utility of delayed action. Action selection is determined by explicit time-dependent utility values parametrized by individualized risk functions (criticality models), enabling trade-offs between belief precision and timeliness of intervention (Horvitz et al., 2013).

In continuous-time finance, portfolio selection under mean–variance–utility with time- and state-dependent risk aversion exhibits closed-form policies where optimal allocations and consumption are driven by coupled Riccati-type ODEs (Yang et al., 2020). This generalizes the Merton problem, generating dynamic asset allocations and consumption rules sensitive to wealth trajectories and labor income, further linking time-dependent risk preferences to model primitives.

Finally, empirical analysis of consumption-dependent (history-dependent) random utility exposes generic misspecification of classical random utility models unless time (or consumption) dependence is properly accounted for. The stationary distribution over preferences is menu-dependent, and ignoring this leads to bias in both welfare estimation and choice probability predictions (Turansick, 2023).

6. Open Problems and Future Directions

Several directions remain open:

• Empirical discrimination between time-average and ensemble-average approaches in lotteries with asymmetric or non-identical safe/risky timing (Berman et al., 2021).
• Extensions to incorporate observed finite horizons, borrowing constraints, or probability weighting, to explain residual behavioral deviations.
• Full identification of utility and discount rates from finite, noisy data (especially under time-varying and possibly non-Markov valuations) (Cao et al., 2024).
• Robust dynamic programming methods for multistage decisions with non-rectangular (e.g., correlated or model-based) utility ambiguity sets.
• Generalizations of variational discounting beyond exponential mixtures, including finitely additive discount kernels, and their impact on social planning (e.g., intergenerational equity) (Dong-Xuan et al., 2024).

7. Comparative Summary of Major Formalisms

Formalism	Temporal structure	Core result	Key reference
Exponential EDU	$D(t)=\delta^t$	Dynamic consistency, unique rate	(Chase et al., 2018)
Hyperbolic/Quasi	$D(t)=1/(1+kt)$	Present bias, time inconsistency	(Dong-Xuan et al., 2024)
Variational Disc.	Min over $\delta$, penalized	Resolves discount rate debate	(Dong-Xuan et al., 2024)
Time-Average Growth	$g_{\text{time}} = \Delta x/\bar t$	Risk neutrality over time lotteries	(Berman et al., 2021)
Ensemble Avg.	$g_{\text{ens}}$	Predicts risk-seeking over time	(Berman et al., 2021)
History-Dependent Utility	$u_t(x;h_{t-1})$	Time-consistent multistage DP	(Liu et al., 2021)

These frameworks collectively clarify that time-dependent utility is not a mere technical extension but a central ingredient for accurately modeling, predicting, and optimizing dynamic decision-making under realistic temporal, stochastic, and informational constraints.