An Information-Theoretic Analysis of Discrete-Time Control and Filtering Limitations by the I-MMSE Relationships

Abstract: Fundamental limitations or performance trade-offs/limits are important properties and constraints of both control and filtering systems. Among various trade-off metrics, total information rate that characterizes the sensitivity trade-offs and time-averaged performance of control and filtering systems was conventionally studied by using the differential entropy rate and Kolmogorov-Bode formula. In this paper, by extending the famous I-MMSE (mutual information -- minimum mean-square error) relationships to the discrete-time additive white Gaussian channels with and without feedback, a new paradigm is introduced to estimate and analyze total information rate as a control and filtering trade-off metric. Under this framework, we explore the trade-off properties of total information rate for a variety of the discrete-time control and filtering systems, e.g., LTI, LTV, and nonlinear, and propose an alternative approach to investigate total information rate via optimal estimation.