Hadamard code

Hadamard code is the name that is most commonly used for this code in the literature. However, in modern use these error correcting codes are referred to as Walsh–Hadamard codes. There is a reason for this: Jacques Hadamard did not invent the code himself, but he defined Hadamard matrices around 1893, long before the first error-correcting code, the Hamming code, was developed in the 1940s. The Hadamard code is based on Hadamard matrices, and while there are many different Hadamard matrices that could be used here, normally only Sylvester's construction of Hadamard matrices is used to obtain the codewords of the Hadamard code. James Joseph Sylvester developed his construction of Hadamard matrices in 1867, which actually predates Hadamard's work on Hadamard matrices. Hence the name Hadamard code is disputed and sometimes the code is called Walsh code, honoring the American mathematician Joseph Leonard Walsh. An augmented Hadamard code was used during the 1971 Mariner 9 mission to correct for picture transmission errors. The binary values used during this mission were 6 bits long, which represented 64 grayscale values. Because of limitations of the quality of the alignment of the transmitter at the time (due to Doppler Tracking Loop issues) the maximum useful data length was about 30 bits. Instead of using a repetition code, a [32, 6, 16] Hadamard code was used. Errors of up to 7 bits per 32-bit word could be corrected using this scheme. Compared to a 5-repetition code, the error correcting properties of this Hadamard code are much better, yet its rate is comparable. The efficient decoding algorithm was an important factor in the decision to use this code. The circuitry used was called the "Green Machine". It employed the fast Fourier transform which can increase the decoding speed by a factor of three. Since the 1990s use of this code by space programs has more or less ceased, and the NASA Deep Space Network does not support this error correction scheme for its dishes that are greater than 26 m. ==Constructions==

While all Hadamard codes are based on Hadamard matrices, the constructions differ in subtle ways for different scientific fields, authors, and uses. Engineers, who use the codes for data transmission, and coding theorists, who analyse extremal properties of codes, typically want the rate of the code to be as high as possible, even if this means that the construction becomes mathematically slightly less elegant. On the other hand, for many applications of Hadamard codes in theoretical computer science it is not so important to achieve the optimal rate, and hence simpler constructions of Hadamard codes are preferred since they can be analyzed more elegantly. Construction using inner products When given a binary message x\in\{0,1\}^k of length k, the Hadamard code encodes the message into a codeword \text{Had}(x) using an encoding function \text{Had} : \{0,1\}^k\to\{0,1\}^{2^k}. This function makes use of the inner product \langle x , y \rangle of two vectors x,y\in\{0,1\}^k, which is defined as follows: :\langle x , y \rangle = \sum_{i=1}^{k} x_i y_i\ \bmod\ 2\,. Then the Hadamard encoding of x is defined as the sequence of all inner products with x: :\text{Had}(x) = \Big(\langle x , y \rangle\Big)_{y\in\{0,1\}^k} As mentioned above, the augmented Hadamard code is used in practice since the Hadamard code itself is somewhat wasteful. This is because, if the first bit of y is zero, y_1=0, then the inner product contains no information whatsoever about x_1, and hence, it is impossible to fully decode x from those positions of the codeword alone. On the other hand, when the codeword is restricted to the positions where y_1=1, it is still possible to fully decode x. Hence it makes sense to restrict the Hadamard code to these positions, which gives rise to the augmented Hadamard encoding of x; that is, \text{pHad}(x) = \Big(\langle x , y \rangle\Big)_{y\in\{1\}\times\{0,1\}^{k-1}}. Construction using a generator matrix The Hadamard code is a linear code, and all linear codes can be generated by a generator matrix G. This is a matrix such that \text{Had}(x)= x\cdot G holds for all x\in\{0,1\}^k, where the message x is viewed as a row vector and the vector-matrix product is understood in the vector space over the finite field \mathbb F_2. In particular, an equivalent way to write the inner product definition for the Hadamard code arises by using the generator matrix whose columns consist of all strings y of length k, that is, :G = \begin{pmatrix} \uparrow & \uparrow & & \uparrow\\ y_1 & y_2 & \dots & y_{2^k} \\ \downarrow & \downarrow & & \downarrow \end{pmatrix}\,. where y_i \in \{0,1\}^k is the i-th binary vector in lexicographical order. For example, the generator matrix for the Hadamard code of dimension k=3 is: : G = \begin{bmatrix} 0 & 0 & 0 & 0 & 1 & 1 & 1 & 1\\ 0 & 0 & 1 & 1 & 0 & 0 & 1 & 1\\ 0 & 1 & 0 & 1 & 0 & 1 & 0 & 1 \end{bmatrix}. The matrix G is a (k\times 2^k)-matrix and gives rise to the linear operator \text{Had}:\{0,1\}^k\to\{0,1\}^{2^k}. The generator matrix of the augmented Hadamard code is obtained by restricting the matrix G to the columns whose first entry is one. For example, the generator matrix for the augmented Hadamard code of dimension k=3 is: : G' = \begin{bmatrix} 1 & 1 & 1 & 1\\ 0 & 0 & 1 & 1\\ 0 & 1 & 0 & 1 \end{bmatrix}. Then \text{pHad}:\{0,1\}^k\to\{0,1\}^{2^{k-1}} is a linear mapping with \text{pHad}(x)= x \cdot G'. For general k, the generator matrix of the augmented Hadamard code is a parity-check matrix for the extended Hamming code of length 2^{k-1} and dimension 2^{k-1}-k, which makes the augmented Hadamard code the dual code of the extended Hamming code. Hence an alternative way to define the Hadamard code is in terms of its parity-check matrix: the parity-check matrix of the Hadamard code is equal to the generator matrix of the Hamming code. Construction using general Hadamard matrices Hadamard codes are obtained from an n-by-n Hadamard matrix H. In particular, the 2n codewords of the code are the rows of H and the rows of −H. To obtain a code over the alphabet {0,1}, the mapping −1 ↦ 1, 1 ↦ 0, or, equivalently, x ↦ (1 − x)/2, is applied to the matrix elements. That the minimum distance of the code is n/2 follows from the defining property of Hadamard matrices, namely that their rows are mutually orthogonal. This implies that two distinct rows of a Hadamard matrix differ in exactly n/2 positions, and, since negation of a row does not affect orthogonality, that any row of H differs from any row of −H in n/2 positions as well, except when the rows correspond, in which case they differ in n positions. To get the augmented Hadamard code above with n=2^{k-1}, the chosen Hadamard matrix H has to be of Sylvester type, which gives rise to a message length of \log_2(2n)=k. ==Distance==

The distance of a code is the minimum Hamming distance between any two distinct codewords, i.e., the minimum number of positions at which two distinct codewords differ. Since the Walsh–Hadamard code is a linear code, the distance is equal to the minimum Hamming weight among all of its non-zero codewords. All non-zero codewords of the Walsh–Hadamard code have a Hamming weight of exactly 2^{k-1} by the following argument. Let x\in\{0,1\}^k be a non-zero message. Then the following value is exactly equal to the fraction of positions in the codeword that are equal to one: :\Pr_{y\in\{0,1\}^k} \big[ (\text{Had}(x))_y = 1 \big] = \Pr_{y\in\{0,1\}^k} \big[ \langle x,y\rangle = 1 \big]\,. The fact that the latter value is exactly 1/2 is called the random subsum principle. To see that it is true, assume without loss of generality that x_1=1. Then, when conditioned on the values of y_2,\dots,y_k, the event is equivalent to y_1 \cdot x_1 = b for some b\in\{0,1\} depending on x_2,\dots,x_k and y_2,\dots,y_k. The probability that y_1=b happens is exactly 1/2. Thus, in fact, all non-zero codewords of the Hadamard code have relative Hamming weight 1/2, and thus, its relative distance is 1/2. The relative distance of the augmented Hadamard code is 1/2 as well, but it no longer has the property that every non-zero codeword has weight exactly 1/2 since the all 1s vector 1^{2^{k-1}} is a codeword of the augmented Hadamard code. This is because the vector x=10^{k-1} encodes to \text{pHad}(10^{k-1}) = 1^{2^{k-1}}. Furthermore, whenever x is non-zero and not the vector 10^{k-1}, the random subsum principle applies again, and the relative weight of \text{Had}(x) is exactly 1/2. ==Local decodability ==

A locally decodable code is a code that allows a single bit of the original message to be recovered with high probability by only looking at a small portion of the received word. A code is q-query locally decodable if a message bit, x_i, can be recovered by checking q bits of the received word. More formally, a code, C: \{0,1\}^k \rightarrow \{0,1\}^n, is (q, \delta\geq 0, \epsilon\geq 0)-locally decodable, if there exists a probabilistic decoder, D:\{0,1\}^n \rightarrow \{0,1\}^k, such that (Note: \Delta(x,y) represents the Hamming distance between vectors x and y): \forall x \in \{0,1\}^k, \forall y \in \{0,1\}^n, \Delta(y, C(x)) \leq \delta n implies that Pr[D(y)_i = x_i] \geq \frac{1}{2} + \epsilon, \forall i \in [k] Theorem 1: The Walsh–Hadamard code is (2, \delta, \frac{1}{2}-2\delta)-locally decodable for all 0\leq \delta \leq \frac{1}{4}. Lemma 1: For all codewords, c in a Walsh–Hadamard code, C, c_i+c_j=c_{i+j}, where c_i, c_j represent the bits in c in positions i and j respectively, and c_{i+j} represents the bit at position (i+j). Proof of lemma 1 Let C(x) = c = (c_0,\dots,c_{2^n-1}) be the codeword in C corresponding to message x. Let G = \begin{pmatrix} \uparrow & \uparrow & & \uparrow\\ g_0 & g_1 & \dots & g_{2^n-1} \\ \downarrow & \downarrow & & \downarrow \end{pmatrix} be the generator matrix of C. By definition, c_i = x\cdot g_i. From this, c_i+c_j = x\cdot g_i + x\cdot g_j = x\cdot(g_i+g_j). By the construction of G, g_i + g_j = g_{i+j}. Therefore, by substitution, c_i + c_j = x\cdot g_{i+j} = c_{i+j}. Proof of theorem 1 To prove theorem 1 we will construct a decoding algorithm and prove its correctness. Algorithm Input: Received word y = (y_0, \dots, y_{2^n-1}) For each i \in \{1, \dots, n\}: • Pick j \in \{0, \dots, 2^n-1\} uniformly at random. • Pick k \in \{0, \dots, 2^n-1\} such that j+k = e_i, where e_i is the i-th standard basis vector and j+k is the bitwise xor of j and k. • x_i \gets y_j+y_k. Output: Message x = (x_1, \dots, x_n) Proof of correctness For any message, x, and received word y such that y differs from c = C(x) on at most \delta fraction of bits, x_i can be decoded with probability at least \frac{1}{2}+(\frac{1}{2}-2\delta). By lemma 1, c_j+c_k = c_{j+k} = x\cdot g_{j+k} = x\cdot e_i = x_i. Since j and k are picked uniformly, the probability that y_j \not = c_j is at most \delta. Similarly, the probability that y_k \not = c_k is at most \delta. By the union bound, the probability that either y_j or y_k do not match the corresponding bits in c is at most 2\delta. If both y_j and y_k correspond to c, then lemma 1 will apply, and therefore, the proper value of x_i will be computed. Therefore, the probability x_i is decoded properly is at least 1-2\delta. Therefore, \epsilon = \frac{1}{2} - 2\delta and for \epsilon to be positive, 0 \leq \delta \leq \frac{1}{4}. Therefore, the Walsh–Hadamard code is (2, \delta, \frac{1}{2}-2\delta) locally decodable for 0\leq \delta \leq \frac{1}{4}. ==Optimality==

For k ≤ 7 the linear Hadamard codes have been proven optimal in the sense of minimum distance. ==See also==