Introduction

The main source for this section is Feldman, Wainwright, Karger 2005.

Linear programming (LP) decoding is a relaxation of maximum likelihood (ML) decoding, proposed as an alternative to belief propagation (BP) decoding for binary linear block codes (such as LDPC codes). Although LP decoding has higher complexity (polynomial in block length) than BP decoding (linear in block length), LP decoding is often easier to analyze. In particular, LP decoding has ML certificate, in that any converged word that is not a codeword is easily identified (by containing fractional bit values).

=Problem Statement=

The problem can be framed as follows:

Define a code with length $n$ by its Tanner graph $\mathcal G(\mathcal V,F,E)$ with $\mathcal V$ set of variable nodes, $F$ set of checks, and $E$ set of edges, and define $\mathcal N(k)$ as the set of nodes in the neighborhood of node $k$. Then for all checks $j\in F$, define $E_j$ as the set of all even-sized subsets of $\mathcal(j)$. Furthermore, define $\gamma_i$ as the negative log likelihood of bit $i$, eg:

\begin{equation} \gamma_i = \log\left(\frac{\Pr(\hat{y}_i|y_i=0)}{\Pr(\hat{y}_i|y_i=1)}\right) \end{equation}

The relaxed LP decoding problem can be stated as following:

\begin{eqnarray} \hat{x}=\text{arg}\min_{f} &&\sum_{i=1}^{n}\gamma_i f_i\\ \text{subject to:}&&\sum_{i\in(N(j) \setminus S)} f_i+\sum_{i\in S}(1-f_i)\geq 1 \end{eqnarray}

= Derivation =

The relaxed LP decoding problem is derived from the integer programming formulation of the maximum likelihood

\begin{eqnarray} \hat{x}=\text{arg}\min_{f} \sum_{i=1}^{n}\gamma_i f_i\\ \text{subject to} f\in \C, \end{eqnarray}

that is, the best estimate of the sent codeword $x$ is the entry in the codebook $\C$ with the highest MAP. To first relax this into a linear programming problem, we can relax the feasible set $f\in\C$ to the convex set $f\in\text{conv}(\C)$, e.g. $f$ is within the convex hull of the codebook $\C$. Because linear programs always converge to a vertex of the feasible set, the solution to this problem is always the ML codeword. However, this problem is still NP-hard, thus motivating an even more relaxed feasible set.

As an example, consider the following code defined by the parity matrix:

\begin{equation} H=\left[ \begin{matrix} 0 & 1 & 0 & 1 & 1 & 0 & 0 & 1\\ 		1 & 1 & 1 & 0 & 0 & 1 & 0 & 0\\ 		0 & 0 & 1 & 0 & 0 & 1 & 1 & 1 \\ 		1 & 0 & 0 & 1 & 1 & 0 & 1 & 0 \\ 		\end{matrix} \right] \end{equation}

We index the bit positions $\mathcal V=\{0,1,2,3,4,5,6,7\}$ and check positions $F=\{0,1,2,3\}$. The neighborhood of check $j=1$ is $\mathcal N(j=1)=\{0,1,2,5\}$, and $E_1=\{\{\},\{0,1\},\{0,2\},\{0,5\},\{1,2\},\{1,5\},\{2,5\},\{0,1,2,5\}\}$. Then every valid word that satisfies the local check $j=1$ must follow the following pattern:

This defines the local code for $j=1$. The constraints for this local code can be enforced by defining an indicator variable $w_{j,S}$ that takes value $1$ if the local check $j$ is satisfied using pattern $S$. Then the local code is satisfied if $\sum_{S\in E_j} w_{j,S}=1$.

More specifically, in an ideal world, two things must be true:


 * 1) there exists only one $S\in E_j$ such that $w_{j,S}=1$, and
 * 2) for all code bits $f_i$, $f_i=1$ iff $i\in S$ for all $j\in\mathcal J$, $S\in E_j$ such that $w_{j,S}=1$.

The "relaxed polytope" allows multiple $S$ to partially satisfy the local check, that is, $w_{j,S}$ can take fractional values, and the previous two contraints are replaced with:


 * 1) $\sum_{S\in E_j} w_{j,S}=1$.
 * 2) $\sum_{S\in E_j,i\in S} w_{j,S}=f_i$.

Finally, we add the additional relaxation that bit values may be fractional. The relaxed LP-decoding problem, as presented by (cite feldman), is then given:

\begin{eqnarray} \hat{x}=\text{arg}\min_{f} &&\sum_{i=1}^{n}\gamma_i f_i\\ \text{subject to:}&&, 0\leq w_{j,S} \leq 1\\ && \sum_{S\in E_j} w_{j,S}=1\\ && \sum_{S\in E_j,i\in S} w_{j,S}=f_i\\ &&\forall j\in F,S\in E_j, i\in \mathcal V, \end{eqnarray}

and the constraint $0\leq f_i \leq 1$ is thus implied.

We define $Q$ as the relaxed polytope such that $(\mathbf{f},\mathbf{w})\in Q$ if $\mathbf{f}$ and $\mathbf w$ satisfy these constraints. We then define $P$ as the projected polytope such that $\mathbf f \in P$ if there exists a $\mathbf w$ such that $(\mathbf f, \mathbf w)\in Q$.

It can be shown that $P$ can be desribed as:

\begin{equation} P\triangleq \{\mathbf f \in P| \forall i\in \mathcal V, \sum_{i\in(N(j) \setminus S)} f_i+\sum_{i\in S}(1-f_i)\geq 1\} \end{equation}

where $S\in O_j$, and $O_j$ is the set of subsets of $\mathcal N(j)$ with "odd" cardinality. This projected polytope can be understood as forbidding every ""bad"" local codeword pattern by forcing them to be a distance 1 away from any valid word.

This gives the most compact formulation of the problem, presented at the top of this page.



This figure gives a 2-D pictorial analogy of the feasible set relaxation. It shows two important concepts:


 * 1) ML decoding (using the convex hull as the feasible set) will always converge to a codeword
 * 2) LP decoding (using the relaxed projected polytope) has additional vertexes, which contain nonintegral vertices (some bit values are fractional).

In this way, the ML certificate is shown: the solution set of LP decoding (vertexes of the relaxed polytope) contains ML codewords (integral) and "pseudocodewords" (with fractional bits). A solution is ML iff it is integral.

=Pseudocodewords and fractional weight=

The LP error outputs are described as pseudocodewords, in which the codeword bits are fractional. For example, consider the Hamming (7,4) code:

\begin{equation} H=\left[ \begin{matrix} 1 &1 &0 &1 &1 &0 &0\\  0 &1 &1 &1& 0& 1& 0\\    0& 0& 0& 1& 1& 1& 1\\ \end{matrix} \right] \end{equation}

Consider $\gamma=[-7/4,1,1,1,1,1,1]^T$. Then the smallest possible cost for a valid codeword is $\gamma^T f=0$. But the optimal LP decoding value is $f=[1,1/3,0,1/3,1/3,0,0]$, with a cost $\gamma^T f=-0.75$.

Given the relaxed polytope $P$, we define $\mathcal V_P$ as the set containing the vertices of $P$, e.g. the set of all valid codewords and pseudocodewords. We define the fractional weight of a pseudocodeword as the $l_1$ norm of the pseudocodeword:

\begin{equation} w_{frac}\triangleq\sum_i f_i. \end{equation}

The fractional distance of a code can then be identified as the minimum fractional weight of a word in $\mathcal V_P$:

\begin{equation} d_{frac}\triangleq\min_{\mathbf f\in \mathcal V_P} \sum_i f_i. \end{equation}

Note that since all valid codewords are also a vertex in $P$, this means that the fractional distance is a lower bound on minimal distance. Additionally, a connection between LP and ML decoding can be drawn in that the LP decoder is guaranteed successful if at most $$\lceil d_{frac}/2\rceil $$ $-1$ bits are flipped in a BSC. The max-fractional distance can be described as

\begin{equation} d_{frac}^{\max}\triangleq\min_{f\in P}\left( \frac{\sum_i f_i}{\max_i f_i} \right) \end{equation}

It can be additionally helpful to define the weight of codewords depending on the error channel. This is given by the pseudo-codeword weight, which is defined separately for BSC, BEC, and AWGN channels.

For a BSC channel, define $e$ as the smallest number such that the sum of the $e$ largest bits in a pseudocodeword $\mathbf f$ is greater than half the fractional weight. Then the pseudocodeword weight for a BSC channel is

\begin{equation} w_{BSC}(\mathbf f)\triangleq \begin{cases} 2e, &\text{ if } \sum_e p_i=(\sum_i p_i)/2\\ 2e-1, &\text{ if } \sum_e p_i>(\sum_i p_i)/2 \end{cases} \end{equation}

We can then interpret $e= $ $$\lceil w_{BSC}(\mathbf f)+1)/2\rceil $$ as the least number of bits to be flipped in the all-zero codeword such that the LP decoder converges to $\mathbf f$.

=References= Feldman