Multiple descriptions

The multiple descriptions problem is a lossy source coding problem with a single encoder generating multiple descriptions (compressed versions) of the source to multiple receivers. The problem is related to the question of how compression needs to be performed in a robust manner with different receivers having access to different subsets of the descriptions generated at the source. The problem, originally introduced in, has applications in multiple domains including communication networks and distributed databases. The requirement is that while individual descriptions of the source are generated such that each description satisfies a distortion criterion individually, a receiver that has access to more messages or descriptions should be able to recover the source at a better distortion than a receiver that has access to fewer messages. In some sense, what we desire is a graceful degradation in the quality of the reconstruction with reduction in the number of descriptions that a receiver has access to.

Although the general version of the multiple descriptions problem has not been solved in so many years, significant progress has been made by various researchers on different aspects of the problem. The El Gamal-Cover coding scheme introduced by El Gamal and Cover in is a very popular scheme and has been shown to be optimal in the case of Gaussian multiple descriptions  and. Another interesting scenario of the problem involves the solution by for the case of 'no excess rate', where we require the sum rate of the problem to be equal to the rate distortion function corresponding to the most stringent distortion constraint in the system.

The key idea in the multiple description coding is the technique of generating dependent compressed versions, so that a receiver that access to more descriptions is able to recover the source better. A precise characterization of the problem involves the characterization of the amount of dependence between the messages to enable better recovery with more descriptions. A generalization of the El Gamal-Cover scheme known as the Zheng-Berger scheme includes common messages intended for different subsets of receivers and was shown to achieve a strictly better rate region.

In the case of Gaussian multiple descriptions, each description is generated based on the forward test channel model for the associated auxiliaries in the El Gamal-Cover scheme. The messages are related by correlating the amount of Gaussian noise used to generate the auxiliaries. Consider a simple two description scenario with 3 receivers for this discussion. Suppose $D_1$ and $D_2$ represent individual distortion criteria at receivers with access to respective individual descriptions and Let $D_0$ represent the distortion constraint at the central receiver with access to both descriptions. The amount of correlation depends on the relative dependence between $D_0$, $D_1$ and $D_2$. If $D_1+D_2\geq \sigma_S^2+D_0$, where $\sigma_S^2$ is the variance of the source, then this constraint implies that the individual distortions $D_1$ and $D_2$ are high. As a result, we require the noises in the forward test channels to be perfectly negatively correlated (high correlation) in order to achieve the desired distortion at the central receiver. This scenario is known as the high distortion regime on account of the high distortions at the individual distortion receivers relative to the central receiver. In the other extreme, let us consider the case where $\frac{1}{D_1}+\frac{1}{D_2}\geq \frac{1}{\sigma_S^2}+\frac{1}{D}$, which represents the case when the distortion constraints at the individual receivers are low with respect to the constraint at the central receiver. In this regime, since the individual descriptions are themselves quite good, we only need a small amount of correlation between the noises to achieve a better distortion. In fact, in this regime, it turns out, we can allow the noises in the forward test channels to be independent and achieve the optimal rate region. In between the two scenarios, we need to vary the degree of correlation to achieve the desired distortion at the central receiver.

System Model
Consider a sequence of i.i.d. observations of the source sequence $\{X_i\}_{i=1}^n$. Suppose we desire two descriptions of the source, the encoder generates codes $C_1$ and $C_2$ at rates $R_1$ and $R_2$ respectively by applying encoding functions $f_1$ and $f_2$. Let Receiver 1 and 2 have access to codes $C_1$ and $C_2$ respectively while Receiver 0 has access to all the messages in the system. The receivers apply decoding functions $\phi_1$, $\phi_2$ and $\phi_0$ resulting in estimates $\{\hat{X}_{1i}\}_{i=1}^n$, $\{\hat{X}_{2i}\}_{i=1}^n$ and $\{\hat{X}_{0i}\}_{i=1}^n$ that satisfy distortion constraints $D_1$, $D_2$ and $D_0$. Mathematically, \begin{equation*} \sum_{i=1}^n\mathbb{E}\left[d(X_i,\hat{X}_{li})\right] \leq D_l \textrm{ for } l\in\{1,2\}. ` \end{equation*}

The distortion constraint at the central receiver is given by \begin{equation*} \sum_{i=1}^n\mathbb{E}\left[d(X_i,\hat{X}_{0i})\right] \leq D_0. \end{equation*}

The objective of the problem is to find the set of all achievable tuples $(R_1,R_2,D_0,D_1,D_2)$, i.e. the set of all tuples such that there exist encoding functions $f_1$ and $f_2$ and decoding functions $\phi_1$, $\phi_2$ and $\phi_0$ satisfying the rate and the distortion requirements.

El Gamal-Cover Achievable Scheme
A brief overview of the El Gamal-Cover scheme is presented below. Given a source sequence