--- title: Dual-Regime Absorbing Markov Chain url: https://www.emergentmind.com/topics/dual-regime-absorbing-markov-chain-dr-amc type: topic --- # Dual-Regime Absorbing Markov Chain A dual-regime absorbing Markov chain (DR-AMC) is a finite discrete-time Markov chain architecture characterized by a piecewise regime structure: its transient state space is partitioned into two distinct operational regimes, each associated with its own transition (and absorption) dynamics. The process evolves according to one set of sub-stochastic transition and absorption matrices in Regime 1 for a fixed, possibly policy-determined, number of timesteps or until an absorption event; should it survive this phase, it undergoes a regime switch—via a boundary transition kernel—to Regime 2, where it continues to evolve until eventual absorption. This structure enables exact stochastic modeling of systems with threshold-driven or policy-induced phase changes, enabling closed-form computation of key performance metrics through a regime-aware extension of classical phase-type (PH) distributions. The DR-AMC framework provides analytical tractability and efficient parameterization for a broad class of applications in stochastic control, information freshness, remote estimation, and computer vision [2512.04077][1808.08393][2504.10451][2008.07475]. ## 1. Formal Definition and State Space Architecture A DR-AMC is defined as a discrete-time Markov chain $\{Y_t\}_{t \geq 0}$ on a finite state space segmented into three disjoint subsets: - **Regime 1 transient states**: $\{1, 2, \dots, K_1\}$ - **Regime 2 transient states**: $\{1', 2', \dots, K_2'\}$ - **Absorbing states**: $\{a_1, a_2, \dots, a_L\}$ The process initiates in a Regime 1 transient state according to an initial distribution $\bm{\beta}_1$. Transitions within Regime 1 are governed by $\bm{A}_1$ (sub-stochastic on $K_1 \times K_1$), with absorption governed by $\bm{B}_1$ ($K_1 \times L$). If absorption does not occur prior to the deterministic switching time $\tau$, the chain transitions into Regime 2 using a boundary transition matrix $\bm{\Theta}$ ($K_1 \times K_2$). Subsequently, Regime 2 dynamics are determined by $\bm{A}_2$ and $\bm{B}_2$ of analogous dimensions. The structure of a DR-AMC is thus fully specified by the tuple $(\bm{\beta}_1, \tau, \bm{\Theta}, \bm{A}_1, \bm{A}_2, \bm{B}_1, \bm{B}_2)$ [2512.04077]. ## 2. Transition Matrices and Regime Switching The regime-specific transition matrices are defined in block-canonical form: \[ P_1 = \begin{pmatrix} A_1 & B_1 \\ 0 & I_L \end{pmatrix}, \quad P_2 = \begin{pmatrix} A_2 & B_2 \\ 0 & I_L \end{pmatrix} \] - $A_1 \in \mathbb{R}^{K_1 \times K_1}$ and $B_1 \in \mathbb{R}^{K_1 \times L}$ dictate Regime 1 evolution and absorption to $\{a_1, ..., a_L\}$. - $A_2 \in \mathbb{R}^{K_2 \times K_2}$ and $B_2 \in \mathbb{R}^{K_2 \times L}$ are defined analogously for Regime 2. - Regime switching at time $t = \tau - 1$ is accomplished by $\bm{\Theta}$, satisfying $\sum_j \theta_{ij} = 1$ for each $i$. - The initial distribution entering Regime 2 is $\bm{\beta}_2 = \bm{\beta}_1 A_1^{\tau-1} \bm{\Theta}$. The switching paradigm supports both time-based (deterministic $\tau$) and policy-based (stopping time or control) transitions, with typical analysis assuming deterministic thresholds [2512.04077][2504.10451]. ## 3. Absorption Probabilities and Fundamental Matrices Absorption in the DR-AMC is characterized piecewise. In Regime 1, the probability of absorption into each absorbing state prior to regime switch is: \[ \bm{\sigma}_1 = \bm{\beta}_1 (I - A_1^{\tau-1}) (I - A_1)^{-1} B_1 \] If the chain transitions to Regime 2, the absorption probability vector is: \[ \bm{\sigma}_2 = \bm{\beta}_2 (I - A_2)^{-1} B_2 \] The above decomposes total absorption probability based on whether absorption occurs before or after the regime change. The construction leverages truncated sums for regime-1 dwell times and classic PH-fundamental matrix formulas in Regime 2. All moments and distributions of time to absorption (absorption time $T$) follow accordingly [2512.04077]. ## 4. Dual-Regime Phase-Type Distribution of Absorption Time Let $T$ denote the random absorption time. The dual-regime discrete phase-type (DR-DPH) distribution describes the law of $T$: \[ p_T(t) = \begin{cases} \bm{\beta}_1 A_1^{t-1} (\bm{1} - A_1 \bm{1}), & 1 \leq t < \tau \\ \bm{\beta}_2 A_2^{t-\tau} (\bm{1} - A_2 \bm{1}), & t \geq \tau \end{cases} \] This piecewise construction generalizes the single-regime PH distribution, capturing the two-phase absorption process. Higher moments—including factorial moments—can be derived in closed form, with ordinary moments assembled via Stirling number combinatorics. These explicit statistics enable renewal-reward evaluations of performance in threshold-driven control systems [2512.04077][2504.10451]. ## 5. Applications in Information Freshness and Control A principal application of DR-AMCs is in modeling threshold policies for semantic-aware freshness metrics such as Age of Incorrect Information (AoII). Here, Regime 1 models periods of no transmission (passive observation), and Regime 2 models aggressive update transmission upon exceeding an AoII threshold. The DR-AMC framework precisely quantifies: - Distribution and moments of out-of-sync durations - Expected cost for arbitrary AoII penalties $g(n)$ - Transmission costs and overall system performance This analytic tractability supports semi-Markov decision process (SMDP) formulations for optimal remote estimation policies under transmission costs, outperforming single-threshold or randomized policies in empirical evaluation [2512.04077][2504.10451]. ## 6. Comparative Regime Switching and Graph-Based Perspectives DR-AMCs are naturally related to time-varying or switching Markov chains, as explored in graph-theoretic absorption analyses. In the context of multiple operating modes (e.g., two regimes), reachability and absorption can be characterized using union and intersection graphs constructed from each mode's non-absorbing transition structure. Key absorption conditions include: - Stabilizability: existence of a state-feedback switching policy ensuring absorption - Sufficient conditions under arbitrary regime switching: acyclicity, weak acyclicity, or distance contraction in the union/intersection graphs These results provide complementary perspectives on absorption in DR-AMC-like systems where the regime is determined by policy or environmental events, further broadening the modeling power [2008.07475]. ## 7. Dual-Regime AMC in Computer Vision: Saliency Detection In image saliency detection, the DR-AMC paradigm has been instantiated as a pair of absorbing random walks on superpixel graphs—one regime tracking boundary-based absorption, the other tracking absorption to foreground priors. Absorption times in each regime serve as soft probabilistic cues (foreground and background possibility), which are then fused by solving a quadratic optimization problem. Multi-scale aggregation of DR-AMC saliency maps yields state-of-the-art empirical performance, demonstrating the DR-AMC’s utility in domains beyond classical stochastic control [1808.08393]. --- **Key References:** | Reference | Application Area | Notable Contribution | |--------------------------------------------------------------------------------------|-------------------------------------------------|----------------------------------------------------------------------------------| | [2512.04077](https://arxiv.org/abs/2512.04077) | AoII minimization, remote estimation | Formal DR-AMC definition, absorption probabilities, DR-DPH formula, SMDP context | | [2504.10451](https://arxiv.org/abs/2504.10451) | AoII control, threshold policy analysis | Cycle cost/stats computation using DR-AMC/DR-PH | | [2008.07475](https://arxiv.org/abs/2008.07475) | Switching Markov chains, reachability analysis | Absorption criteria from union/intersection graph analysis | | [1808.08393](https://arxiv.org/abs/1808.08393) | Image saliency, computer vision | Bidirectional (dual-regime) AMC structure for multi-cue saliency detection | The DR-AMC framework synthesizes piecewise Markovian temporal heterogeneity into analytically tractable models, facilitating unified stochastic analysis and tractable optimization in both control and inference domains. Source: https://www.emergentmind.com/topics/dual-regime-absorbing-markov-chain-dr-amc