Sampling and Filtering with Markov Chains

Abstract: A continuous-time Markov chain rate change formula for simulation, model selection, filtering and theory is proven. It is used to develop Markov chain importance sampling, rejection sampling, branching particle filtering algorithms and filtering equations akin to the Duncan-Mortensen-Zakai equation and the Fujisaki-Kallianpur-Kunita equation but for Markov signals with general continuous-time Markov chain observations. A direct method of solving these filtering equations is given that, for example, applies to trend, volatility and/or parameter estimation in financial models given tick-by-tick market data. All the results also apply to continuous-time Hidden Markov Models (CTHMM), which have become important in applications like disease progression tracking, as special cases and the corresponding CTHMM results are stated as corollaries.