Shannon-McMillan-Breiman Theorem

The Shannon-McMillan-Breiman theorem is a fundamental theorem in information theory. It asserts that for a finite-valued stationary ergodic stochastic process $X$, we have $$ -\frac{\log p(X_1, X_2, \cdots, X_n)}{n} \to H(X) \mbox{ as } n \to \infty $$ with probability $1$, where $H(X)$ is the entropy rate of $X$.

The Shannon-McMillan-Breiman theorem is also referred to as the generalized asymptotic equipartition property.