Low-density parity-check codes

Low-density parity-check (LDPC) codes constitute a family linear error-correcting codes. The were introduced by Robert Gallager in his PhD Thesis of the same name (LDPC) in 1961. A more modern view defines the codes using sparse graphical representations. Under this view the diversity of LDPC codes has expanded considerably.

Definitions
An LDPC code is, strictly defined, a code that has a representation using a sparse parity check matrix, i.e., it is the set of solutions $x$ the the equation $Hx=0$ where $H$ is sparse. An equivalent representation replaces the matrix $H$ with a graphical representation. Theoretical considerations often involve ensembles or families of LDPC codes by specified the structure of their Tanner graphs.

Regular LDPC Codes
For positive integers $d_v, d_c$ such that $d_v \leq d_c$, a $(d_v, d_c)$ regular LDPC code of block length $n$ is defined as a code whose Tanner graph has $n$ variable nodes of degree $d_v$ each and $m$ check nodes of degree $d_c$ each, where $m = n d_v/d_c$. This code has rate $$R = 1 - \frac{m}{n} = 1 - \frac{d_v}{d_c}$$ The regular LDPC code is so-called because its Tanner graph is simultaneously left-regular and right-regular.

Irregular LDPC Codes
An irregular LDPC code, in general, has a Tanner graph that is not regular. Irregular LDPC codes are defined by specifying the distribution of the node degrees in their Tanner graphs. There are several ways to specify the degree distribution in an irregular Tanner graph. A straightforward method is to specify the fraction of degree $i$ variable nodes, denoted by $L_i$, and the fraction of degree $j$ check nodes, denoted by $R_j$, for all possible indices $i$ and $j$. This way of specifying the degree distribution is called the node-perspective degree distribution. It can be seen that the sequences $\{L_i\}$ and $\{R_j\}$ both add up to unity, i.e., $$ \sum L_i = \sum R_j = 1$$

A compact way to define the node-perspective degree distribution is to specify the polynomials $L(x) = \sum L_i x^i$ and $R(x) = \sum R_j x^j$ (here $x$ is a formal variable).

More commonly, we specify the degree distribution from the edge perspective using the edge-perspective degree distribution polynomials $\lambda(x) = \sum \lambda_i x^{i-1}$ and $\rho(x) = \sum \rho_j x^{j-1}$. Here, $\lambda_i$ is the fraction of edges in the Tanner graph that are incident with a degree-$i$ variable node and $\rho_j$ is the fraction of edges in the Tanner graph that are incident with a degree-$j$ check node.

If the Tanner graph has $E$ edges, the number of variable nodes of degree $i$ is equal to $n_i = \frac{E \lambda_i}{i}$ and the number of check nodes of degree $j$ is equal to $m_j = \frac{E \rho_j}{j}$. The fraction of degree-$i$ variable nodes, $L_i$, and the fraction of degree-$j$ check nodes $R_j$ are given by \begin{align} L_i = \frac{n_i}{\sum_i n_i} = \frac{\lambda_i/i}{\sum_i \lambda_i/i} = \frac{\lambda_i}{i} \frac{1}{\int_0^1 \lambda(x) \mathrm{d}x} \\ R_j = \frac{m_j}{\sum_j m_j} = \frac{\rho_j/j}{\sum_j \rho_j/j} = \frac{\rho_j}{j} \frac{1}{\int_0^1 \rho(x) \mathrm{d}x} \end{align}

Given the polynomials $\lambda(x)$ and $\rho(x)$, a $(\lambda, \rho)$ irregular LDPC code of block length $n$ is defined as a linear code with a Tanner graph in $n$ variable nodes such that the edge-perspective variable-node degree distribution is given by $\lambda(x)$ and the edge-perspective check-node degree distribution is given by $\rho(x)$.

History
Robert Gallager introduced LDPC codes in his doctoral dissertation in 1961 where he introduce both code constructions and various decoding procedures, including what is now called belief propagation decoding. With a few notable exeptions, such as the work of Tanner in the 1980s, the concept was largely ignored until the discovery of turbo codes in 1993 and the subsequent rediscovery of LDPC by David MacKay in the late 1990s.

Properties
LDPC codes have a number of properties that are worth noting.


 * 1) LDPC codes are linear codes. This means that any bit-wise sum of any subset of codewords is itself a codeword.
 * 2) LDPC codes are block codes. As such, a block of $k$ bits are transformed into $n$ bits using the encoding process.
 * 3) LDPC codes are by definition determined by sparse graphical (or parity check) representations.
 * 4) LDPC codes have been shown to achieve capacity over the binary erasure channel (BEC) and perform very close to capacity on general binary memoryless channels.
 * 5) LDPC code ensembles exhibit concentration, which means that a randomly chosen member of the ensemble performs close to the ensemble average (in a certain restricted sense) with high probability. This result was first observed for the message-passing decoder over the binary symmetric channel (BSC).  Many techniques have been developed to find good members of the ensembles.

Encoding
LDPC codes are usually specified by either their parity check matrix or a graphical representation. Encoding is therefore typically not obvious. A universal solution if to find a generator matrix $G$ for the code. This generically gives complexity of order $n^2.$ Many other methods for deriving encoders have been developed and many ensembles have linear encoding complexity.

Decoding
Decoding graph based codes is usually done using message passing algorithms. The most powerful of these is now known as belief propagation and was introduced by Gallager. Message passing decoders are suboptimal in contrast, with e.g. maximum-likelihood decoding. Their complexity is directly proportional to the density of the graphical representation, hence the motivation for low density.

Applications
LDPC codes have been adopted in various standards including WiMax (IEEE 802.16e), IEEE 802.11n, IEEE 802.3an, and DVB-S2. They have also been employed in optical networking and data storage devices.