Complexity Bounds for Finite-State CTMC Activation Approximation

Establish tight state, spike, and finite-time complexity bounds for generalized finite-state continuous-time Markov chain neurons approximating useful classes of activation functions.

Background

The paper proves that a generalized finite-state continuous-time Markov chain (CTMC) neuron with affine input-dependent transition rates can uniformly approximate every continuous, nonnegative, nondecreasing activation function on a compact input interval. The constructive proof uses a birth–death chain with active states and associated refractory states, requiring 2(N+1) states, and provides a quantitative approximation bound in terms of the number of states.

The authors explicitly distinguish this expressivity theorem from a practical complexity guarantee. The main experiments use compact two- and three-state parameterizations, whereas the universal construction may require substantially more states. The unresolved problem is to determine tight requirements for the number of states, spike activity, and finite-time simulation behavior needed to obtain useful approximation accuracy for relevant activation-function classes.