MCSC: Markov Chain Sparse Control

• MCSC is a control-theoretic framework that discretizes continuous health data into finite states and models transitions using a Markov chain.
• It uses sparse intervention strategies to modify select transition probabilities, aiming to suppress shifts toward disease-like states.
• Demonstrated through simulations and real-data applications, MCSC identifies critical transition bottlenecks to guide targeted pre-disease treatment.

Searching arXiv for the specified paper to ground the article in the source record. Markov Chain Sparse Control (MCSC) is a control-theoretic framework for pre-disease treatment that models the time evolution of a probability distribution on a Markov chain as a discrete-time linear system and then designs a sparse controller to identify a small number of candidate states or transitions for intervention. It is proposed to address a practical gap between increasingly mature methods for pre-disease detection and the lack of equally systematic methods for pre-disease treatment. In this formulation, longitudinal health data or simulated dynamics are discretized into finitely many health states, a Markov transition matrix is estimated over those states, and interventions are represented as sparse modifications of that transition structure so as to make disease-like or otherwise undesirable states less likely (Oku, 24 Jul 2025).

1. Conceptual framing and relation to pre-disease dynamics

MCSC is motivated by the distinction between pre-disease detection and pre-disease treatment. The underlying paper states that many tools already exist for detecting pre-disease states, including dynamical network biomarkers, early warning signals, and machine learning risk scores, whereas a systematic framework for treating or controlling pre-disease states has not been equally well established. MCSC is introduced as such a framework.

Within dynamical network biomarker theory, a pre-disease state is described as an intermediate dynamical state between a healthy attractor and a disease attractor, often represented by a double-well potential. It is associated with critical slowing down and large fluctuations as a bifurcation is approached, and detection relies on changes in variability and correlation in subsets of variables. The scope of MCSC is broader: pre-disease states are treated as “clinically significant states where risk is sufficiently high and intervention is justified,” with metabolic syndrome given as an example.

The framework adopts a state-space picture in which healthy basins and disease basins coexist, and the pre-disease regime corresponds to being near a saddle or branching zone where the probability of falling into the disease basin increases. Rather than attempting to mechanistically model detailed nonlinear dynamics, MCSC discretizes the observed state space into a finite set of regions, estimates a Markov transition matrix among them, and controls the probability distribution over those regions. This places the emphasis on probability flow among macrostates rather than on direct manipulation of mechanistic variables.

A central conceptual distinction is that the state of the control system is not the raw health vector itself, but the probability vector over discrete states. In this sense, MCSC operates at the distribution level. A plausible implication is that the method is intended for settings in which interpretable intervention targets are sought even when the underlying biological system is nonlinear, high-dimensional, or only partially observed.

2. State-space discretization and Markov chain construction

The starting point is a continuous health vector for individual $m$ at time $t$,

$$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$

Instead of fitting a nonlinear dynamical system of the form $x(m,t+1)=f(x(m,t))$, the framework partitions $\mathbb{R}^N$ into $K$ disjoint subsets,

$$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$

and maps each observation to a discrete state index,

$$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$

Two discretization strategies are specified. One is axis-wise binning, implemented as a Cartesian product of intervals, as in energy landscape analysis where each variable is binarized or ternarized. The other is a Voronoi partition based on representative points $\{x^{(1)},\ldots,x^{(K)}\}$, typically obtained by $K$-means clustering, with assignment

$t$0

The result is a symbolic time series of discrete states.

The Markov chain is built from transition events

$t$1

with event set

$t$2

The transition matrix $t$3 has entries

$t$4

so $t$5 is column-stochastic and each column lies in

$t$6

The basic estimator is the relative frequency estimator,

$t$7

Several estimation variants are included. Damping, described as PageRank-style, is used to enforce ergodicity: $t$8 or equivalently

$t$9

where $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$0 is the all-ones matrix. Uneven sampling across individuals can be handled by weighting,

$$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$1

followed by a weighted transition estimate. Smoothing across nearby states can be performed with Nadaraya–Watson regression using a kernel $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$2 based on a distance matrix $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$3.

The induced distribution dynamics are linear: $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$4 If $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$5 is ergodic, then $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$6 as $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$7, where the stationary distribution satisfies

$$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$8

For settings without repeated measures of the same individual, such as snapshot single-cell RNA sequencing, the paper describes an alternative estimation route based on optimal transport. Consecutive empirical distributions $$x(m,t) = (x_1(m,t),\ldots,x_N(m,t))^{\top} \in \mathbb{R}^N.$$9 and $x(m,t+1)=f(x(m,t))$0 are coupled by an optimal transport plan $x(m,t+1)=f(x(m,t))$1, the plans are averaged,

$x(m,t+1)=f(x(m,t))$2

and the columns of $x(m,t+1)=f(x(m,t))$3 are normalized to obtain a column-stochastic $x(m,t+1)=f(x(m,t))$4. In the biological examples, a “state” is interpreted either as a health phenotype region or as a region in gene-expression UMAP space (Oku, 24 Jul 2025).

3. Controlled dynamics, sparsity, and the MCSC objective

MCSC augments the uncontrolled Markov dynamics with a virtual control input: $x(m,t+1)=f(x(m,t))$5 With state feedback $x(m,t+1)=f(x(m,t))$6, the closed-loop dynamics become

$x(m,t+1)=f(x(m,t))$7

where

$x(m,t+1)=f(x(m,t))$8

The matrix $x(m,t+1)=f(x(m,t))$9 represents intervention-induced changes to transition probabilities and is required to satisfy probabilistic constraints. Column sums are preserved by imposing

$\mathbb{R}^N$0

which guarantees that each column of $\mathbb{R}^N$1 still sums to $\mathbb{R}^N$2. Nonnegativity is maintained by requiring

$\mathbb{R}^N$3

The interpretation is explicit: $\mathbb{R}^N$4 promotes the transition $\mathbb{R}^N$5, whereas $\mathbb{R}^N$6 suppresses it. The experiments emphasized in the paper focus on suppressing off-diagonal entries and moving the removed probability mass to the corresponding diagonal entry.

“Sparsity” means that most entries of $\mathbb{R}^N$7 are zero and only a small subset of transitions are modified. This is formalized by the $\mathbb{R}^N$8 norm

$\mathbb{R}^N$9

the number of nonzero entries in $K$0. The intended interpretation is intervention realism: only a few transitions or states should be targeted.

The objective function is

$K$1

Here $K$2 is a reward vector over states, $K$3 is the stationary distribution of the controlled chain $K$4, and $K$5 are regularization parameters. The first term rewards desirable long-term occupancy patterns, the second penalizes the number of modified transitions, and the third penalizes the magnitude of transition modifications in log-fold-change form.

The default formulation is infinite-horizon, with $K$6 defined by stationarity. A finite-horizon version is also given: $K$7 when control is targeted at a finite horizon $K$8. This finite-horizon form is used in the single-cell RNA-seq example.

The overarching control goal is stated as moving the stationary distribution “to a more desirable position within the standard simplex $K$9.” Operationally, undesirable states such as disease-like or pre-disease states are assigned low reward, often $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$0, while healthier states receive higher reward, often $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$1. MCSC then seeks a sparse transition modification that reduces probability mass on the undesirable states. This formulation makes the controlled object the state distribution itself rather than an individual trajectory, distinguishing it from many action-centric control or reinforcement learning approaches.

4. Algorithmic implementation and intervention interpretation

The optimization problem is nonconvex because of the $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$2 term, the logarithmic penalty, and the dependence of $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$3 on the controlled matrix $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$4. The implementation described in the paper therefore uses a greedy heuristic specialized to suppression-only interventions.

Three simplifying assumptions define the implemented procedure. Only off-diagonal transitions are eligible for modification. Only suppression is permitted, implemented by reducing a selected off-diagonal entry by a multiplicative factor such as $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$5, $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$6, or $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$7. The probability mass removed from the suppressed transition is transferred to the diagonal entry $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$8 in order to preserve column sums.

The heuristic proceeds in four stages. First, candidate transitions are restricted to the top $$\mathbb{R}^N = \bigcup_{k=1}^K S_k,\quad S_k\cap S_l = \emptyset\ \ (\forall\,k\neq l),$$9 largest off-diagonal entries of $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$0, reducing search complexity and focusing on major flows. Second, each candidate transition is individually screened by trying a small suppression and recomputing the objective $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$1; candidates that reduce $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$2 are discarded. Third, among the remaining candidates and suppression levels, an exhaustive search identifies the single best transition-suppression pair, which is then applied to update $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$3. Fourth, the procedure iterates until no further improvement in $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$4 is found. The resulting controlled matrix is denoted $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$5.

The computational burden is described as tractable in the experiments, with $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$6 up to approximately $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$7, because each evaluation requires only a stationary distribution computation or a finite-horizon matrix evolution. The paper does not claim global optimality; the emphasis is instead on interpretability and feasibility.

The nonzero entries of $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$8 are interpreted as candidate intervention targets. A suppressed entry $$g:\mathbb{R}^N \to \{1,\ldots,K\},\quad \tilde x(m,t)=g(x(m,t)).$$9 indicates that when the system occupies source state $\{x^{(1)},\ldots,x^{(K)}\}$0, interventions should act so as to prevent or slow transition to target state $\{x^{(1)},\ldots,x^{(K)}\}$1. In clinical language, these entries define where in state space intervention is suggested and which direction of progression should be opposed. The framework itself does not prescribe the treatment modality; it identifies actionable transition structure.

The regularization parameters control the sparsity-performance trade-off. Larger $\{x^{(1)},\ldots,x^{(K)}\}$2 yields fewer nonzero entries in $\{x^{(1)},\ldots,x^{(K)}\}$3, hence fewer intervention sites. Larger $\{x^{(1)},\ldots,x^{(K)}\}$4 limits the magnitude of change in each transition probability. In the one-dimensional double-well example, the paper reports that $\{x^{(1)},\ldots,x^{(K)}\}$5 suppresses only one transition, $\{x^{(1)},\ldots,x^{(K)}\}$6 suppresses two transitions, and $\{x^{(1)},\ldots,x^{(K)}\}$7 suppresses three transitions, with progressively stronger confinement to the preferred well. This is presented as an explicit illustration of the trade-off between stronger control and reduced sparsity (Oku, 24 Jul 2025).

5. Simulated dynamical systems

The paper evaluates MCSC on multiple simulated systems. In a one-dimensional double-well stochastic differential equation,

$\{x^{(1)},\ldots,x^{(K)}\}$8

with

$\{x^{(1)},\ldots,x^{(K)}\}$9

Euler–Maruyama simulation is used, the state variable is discretized into $K$0 bins, and the transition matrix is estimated from long trajectories such as $K$1. The uncontrolled stationary distribution reproduces the empirical histogram. When the reward favors the left well, sparse control suppresses a small number of transitions near the central barrier, approximately states $K$2–$K$3, and shifts the stationary distribution toward the left well. The paper also reports that moderate smoothing stabilizes transition estimates for shorter trajectories, whereas excessive smoothing destroys diagonal dominance.

A two-dimensional double-well system is defined by

$K$4

with drift $K$5 and stochastic dynamics $K$6. Two discretizations are examined: a $K$7 grid with $K$8, and Voronoi partitions with $K$9 obtained by $t$00-means or random sampling. Rewards are assigned as $t$01 in the bottom-left well, $t$02 in the top-right well, and $t$03 elsewhere. MCSC suppresses transitions near the saddle separating the wells, shifting mass from the top-right well to the bottom-left. The $t$04-means Voronoi partition is reported to yield smoother and more focused intervention patterns than random Voronoi partitioning.

A two-dimensional branching-flow or Waddington toy model uses the potential

$t$05

with drift

$t$06

Particles start at the top center and drift through a branching landscape to one of four terminal states. With $t$07 simulated trajectories and $t$08-means discretization into $t$09 states, MCSC is applied separately to each terminal state by assigning that state reward $t$10 and the others $t$11. For each target, the selected interventions suppress both an upstream branch point feeding the terminal and a downstream branch close to the terminal itself.

The method is also applied to deterministic chaotic systems. For the Lorenz system,

$t$12

and the Rössler system,

$t$13

the attractors are discretized into $t$14 regions via $t$15-means after transient removal. Rewards are defined for the right wing in Lorenz and for high-$t$16 states in Rössler. MCSC suppresses cross-wing transitions in Lorenz and suppresses transitions at the onset of $t$17-spikes, while also modifying upstream orbits to favor inner trajectories, in Rössler. The paper presents these examples as evidence that the approach can be applied not only to stochastic health data but also to complex deterministic dynamics.

Across these simulations, MCSC repeatedly identifies saddles, branch points, and spike onsets as intervention sites. The paper notes this pattern qualitatively rather than as a formal robustness theorem. This suggests that sparse control on Markov-state representations tends to prioritize dynamical bottlenecks and branching structures.

6. Real-data analyses and domain-specific findings

One real-data application uses aggregated Japanese specific health checkup data from fiscal year $t$18, organized by $t$19 secondary medical care areas, gender, and $t$20 age groups from $t$21–$t$22 to $t$23–$t$24. Twelve numerical variables are used, including BMI, WC, FPG, HbA1c, SBP, DBP, TG, HDL, LDL, AST, ALT, and $t$25-GTP. Each area-by-gender combination is treated as a “virtual individual,” with age groups serving as time points.

Principal component analysis shows that males and females cluster separately, so the analyses are conducted by gender. $t$26-means with $t$27 defines states in the $t$28-dimensional space, sorted by mean age. Empirical state distributions are computed for each age group, and the transition matrix is estimated using weighted frequencies. To mitigate strong aging drift, the final state of each virtual individual is connected back to the initial state, producing periodic boundary conditions through “resetting,” so that the stationary distribution approximates a long-term age profile. The reward assigns $t$29 to state $t$30, described as an unfavorable metabolic profile in older age, and $t$31 to all other states.

For males, the state centers show worsening FPG, HbA1c, and SBP with age, while LDL and ALT decrease and other variables are non-monotonic. States $t$32 and $t$33 dominate in older groups. MCSC suggests suppressing transitions $t$34, $t$35, $t$36, and $t$37. The predicted stationary distribution approximately halves the probability of state $t$38, increases earlier states $t$39 and $t$40, and decreases state $t$41 because of upstream suppression. Simulated controlled dynamics show more dispersed flows rather than funneling into state $t$42.

For females, state $t$43 and $t$44 again dominate older groups, but the suggested interventions differ: MCSC suppresses $t$45, $t$46, and $t$47, with no early interventions from states $t$48–$t$49. The interpretation given is that early states are rarely occupied after age $t$50, so intervening there would not be effective. Under control, the stationary probability of state $t$51 is again approximately halved, while states $t$52 and $t$53 increase. The paper notes that resetting induces artifacts, including unrealistic probability shifts in young states.

A second real-data application analyzes single-cell RNA-seq data from hepatocytes in wild-type C57BL/6 mice, dataset GSE247719, at five ages: $t$54, $t$55, $t$56, $t$57, and $t$58 months. UMAP coordinates are used as the state space, and males and females are analyzed separately. $t$59-means with $t$60 defines discrete states. Empirical distributions $t$61 are computed from cell fractions in each cluster, and a particular state, state $t$62, which appears only at $t$63 months, is treated as an undesirable late-aging state. The time series are interpolated and smoothed to account for non-equidistant sampling, and the transition matrix is estimated via optimal transport between consecutive distributions. Here the finite-horizon target

$t$64

is used.

The estimated uncontrolled dynamics reproduce the smoothed trajectories of state frequencies over time. MCSC then suggests suppressing long-range transitions feeding state $t$65, despite optimal transport’s preference for short-range flows. Simulations with the controlled transition matrix suppress the emergence of state $t$66 in both males and females. The paper characterizes this finding as counterintuitive and proposes optimal transport with relays as a future direction to decompose long hops into multiple short hops. A plausible implication is that MCSC can identify rare but influential transitions that may be obscured by local transport structure (Oku, 24 Jul 2025).

7. Relation to adjacent frameworks, limitations, and prospective directions

The framework is positioned as complementary to dynamical network biomarker and early-warning methodologies. DNB theory models biological systems as multivariate noisy dynamical systems in which healthy and disease states are equilibria of a nonlinear system and pre-disease corresponds to approaching a bifurcation such as a saddle-node. DNB-based work has emphasized detection and, in some cases, drug-target intervention using mechanistic models. MCSC differs in that it does not require explicit nonlinear equations or identification of DNB variables; it works directly from time-series state data after discretization and controls transition probabilities among macrostates.

The paper also distinguishes MCSC from reinforcement learning and Markov decision process formulations. An MDP with explicit states and actions generalizes a Markov chain, and many healthcare RL applications require action data and exploration. MCSC instead uses what the paper calls an “actionless MDP,” namely a pure Markov chain over observed state transitions without explicit actions, and control is designed offline from state-only data. It is further contrasted with Bayesian optimal path planning for pre-disease treatment, which may recommend extreme sequences of variable improvements, and with energy landscape analysis based on Ising models, which identifies stable states but does not add sparse transition control in the same way.

The paper identifies several limitations. The Markovian and time-homogeneous approximation may lose mechanistic detail because the underlying biological systems are nonlinear, nonstationary, nonautonomous, high-dimensional, hierarchical, and may contain delays. Discretization choices, including the number of states $t$67 and the clustering method, affect the estimated transition matrix and therefore the proposed interventions. The optimization procedure is heuristic, suppression-only, restricted to off-diagonal modifications, and does not provide global optimality guarantees. Although the objective function can in principle accommodate promotion as well as suppression, only suppression is implemented.

Further limitations concern the relation between the virtual controlled model and actual interventions. The controlled dynamics $t$68 are explicitly described as “virtual”; one cannot directly manipulate a population distribution, so translating suppressed transitions into concrete, feasible medical interventions remains a separate domain problem. In the health checkup data, resetting is an ad hoc response to aging-related nonstationarity and can produce artifacts. In the single-cell RNA-seq example, long-range transition suppression may be biologically questionable. Finally, the evidence so far consists of simulations and descriptive analyses of existing datasets; prospective studies, clinical trials, or causal inference studies are identified as necessary for practical validation.

The future directions listed in the paper include more principled adaptive discretization, controllers defined directly in continuous space, better optimization algorithms such as approximate convex relaxations, gradient-based methods, or mixed integer programming for structured sparsity, explicit nonstationary Markov models with time-dependent transition matrices, optimal transport with relays, and integration with DNB-based detection, mechanistic models, or reservoir computing for anomaly detection. Taken together, these points position MCSC as a phenomenological framework for selecting sparse intervention targets in discretized state space rather than as a complete mechanistic or clinical decision system.