Although a JPEG file can be encoded in various ways, most commonly it is done with JFIF encoding. The encoding process consists of several steps: • The representation of the colors in the image is converted from
RGB to YCbCr|, consisting of one
luma component (Y'), representing brightness, and two
chroma components, (CB and CR), representing color. This step is sometimes skipped. • The resolution of the chroma data is reduced, usually by a factor of 2 or 3. This reflects the fact that the eye is less sensitive to fine color details than to fine brightness details. • The image is split into blocks of 8×8 pixels, and for each block, each of the Y, CB, and CR data undergoes the
discrete cosine transform (DCT). A DCT is similar to a
Fourier transform in the sense that it produces a kind of spatial frequency spectrum. • The amplitudes of the frequency components are
quantized. Human vision is much more sensitive to small variations in color or brightness over large areas than to the strength of high-frequency brightness variations. Therefore, the magnitudes of the high-frequency components are stored with a lower accuracy than the low-frequency components. The quality setting of the encoder (for example 50 or 95 on a scale of 0–100 in the Independent JPEG Group's library) affects to what extent the resolution of each frequency component is reduced. If an excessively low quality setting is used, the high-frequency components are discarded altogether. • The resulting data for all 8×8 blocks is further compressed with a lossless algorithm, a variant of
Huffman encoding. The decoding process reverses these steps, except the
quantization because it is irreversible. In the remainder of this section, the encoding and decoding processes are described in more detail.
Encoding Many of the options in the JPEG standard are not commonly used, and as mentioned above, most image software uses the simpler JFIF format when creating a JPEG file, which among other things specifies the encoding method. Here is a brief description of one of the more common methods of encoding when applied to an input that has 24
bits per pixel (eight each of
red, green, and blue). This particular option is a
lossy data compression method. They are represented in
matrices below.
Color space transformation First, the image should be converted from RGB (by default sRGB, The advantage of the DCT is its tendency to aggregate most of the signal in one corner of the result, as may be seen above. The quantization step to follow accentuates this effect while simultaneously reducing the overall size of the DCT coefficients, resulting in a signal that is easy to compress efficiently in the entropy stage. The DCT temporarily increases the bit-depth of the data, since the DCT coefficients of an 8-bit/component image take up to 11 or more bits (depending on fidelity of the DCT calculation) to store. This may force the codec to temporarily use 16-bit numbers to hold these coefficients, doubling the size of the image representation at this point; these values are typically reduced back to 8-bit values by the quantization step. The temporary increase in size at this stage is not a performance concern for most JPEG implementations, since typically only a very small part of the image is stored in full DCT form at any given time during the image encoding or decoding process.
Quantization The human eye is good at seeing small differences in
brightness over a relatively large area, but not so good at distinguishing the exact strength of a high frequency brightness variation. This allows one to greatly reduce the amount of information in the high frequency components. This is done by simply dividing each component in the frequency domain by a constant for that component, and then rounding to the nearest integer. This rounding operation is the only lossy operation in the whole process (other than chroma subsampling) if the DCT computation is performed with sufficiently high precision. As a result of this, it is typically the case that many of the higher frequency components are rounded to zero, and many of the rest become small positive or negative numbers, which take many fewer bits to represent. The elements in the
quantization matrix control the compression ratio, with larger values producing greater compression. A typical quantization matrix (for a quality of 50% as specified in the original JPEG Standard), is as follows: :Q= \begin{bmatrix} 16 & 11 & 10 & 16 & 24 & 40 & 51 & 61 \\ 12 & 12 & 14 & 19 & 26 & 58 & 60 & 55 \\ 14 & 13 & 16 & 24 & 40 & 57 & 69 & 56 \\ 14 & 17 & 22 & 29 & 51 & 87 & 80 & 62 \\ 18 & 22 & 37 & 56 & 68 & 109 & 103 & 77 \\ 24 & 35 & 55 & 64 & 81 & 104 & 113 & 92 \\ 49 & 64 & 78 & 87 & 103 & 121 & 120 & 101 \\ 72 & 92 & 95 & 98 & 112 & 100 & 103 & 99 \end{bmatrix}. The quantized DCT coefficients are computed with :B_{j,k} = \mathrm{round} \left( \frac{G_{j,k}}{Q_{j,k}} \right) \mbox{ for } j=0,1,2,\ldots,7; k=0,1,2,\ldots,7 where G is the unquantized DCT coefficients; Q is the quantization matrix above; and B is the quantized DCT coefficients. Using this quantization matrix with the DCT coefficient matrix from above results in: :B= \left[ \begin{array}{rrrrrrrr} -26 & -3 & -6 & 2 & 2 & -1 & 0 & 0 \\ 0 & -2 & -4 & 1 & 1 & 0 & 0 & 0 \\ -3 & 1 & 5 & -1 & -1 & 0 & 0 & 0 \\ -3 & 1 & 2 & -1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right]. For example, using −415 (the DC coefficient) and rounding to the nearest integer : \mathrm{round} \left( \frac{-415.37}{16} \right) = \mathrm{round} \left( -25.96 \right) = -26. Notice that most of the higher-frequency elements of the sub-block (i.e., those with an
x or
y spatial frequency greater than 4) are quantized into zero values.
Entropy coding Entropy coding is a special form of
lossless data compression. It involves arranging the image components in a "
zigzag" order employing
run-length encoding (RLE) algorithm that groups similar frequencies together, inserting length coding zeros, and then using
Huffman coding on what is left. The JPEG standard also allows, but does not require, decoders to support the use of
arithmetic coding, which is mathematically superior to Huffman coding. However, this feature has rarely been used, as it was historically covered by
patents requiring royalty-bearing licenses, and because it is slower to encode and decode compared to Huffman coding. Arithmetic coding typically makes files about 5–7% smaller. The previous quantized DC coefficient is used to predict the current quantized DC coefficient. The difference between the two is encoded rather than the actual value. The encoding of the 63 quantized AC coefficients does not use such prediction differencing. The zigzag sequence for the above quantized coefficients are shown below. (The format shown is just for ease of understanding/viewing.) : If the
i-th block is represented by B_i and positions within each block are represented by (p,q) where p = 0, 1, ..., 7 and q = 0, 1, ..., 7, then any coefficient in the DCT image can be represented as B_i (p,q). Thus, in the above scheme, the order of encoding pixels (for the -th block) is B_i (0,0), B_i (0,1), B_i (1,0), B_i (2,0), B_i (1,1), B_i (0,2), B_i (0,3), B_i (1,2) and so on. This encoding mode is called baseline
sequential encoding. Baseline JPEG also supports
progressive encoding. While sequential encoding encodes coefficients of a single block at a time (in a zigzag manner), progressive encoding encodes similar-positioned batch of coefficients of all blocks in one go (called a
scan), followed by the next batch of coefficients of all blocks, and so on. For example, if the image is divided into N 8×8 blocks B_0,B_1,B_2,...,B_{n-1}, then a 3-scan progressive encoding encodes DC component, B_i (0,0) for all blocks, i.e., for all i = 0, 1, 2, ..., N-1, in first scan. This is followed by the second scan which encoding a few more components (assuming four more components, they are B_i (0,1) to B_i (1,1), still in a zigzag manner) coefficients of all blocks (so the sequence is: B_0 (0,1),B_0 (1,0),B_0 (2,0),B_0 (1,1),B_1 (0,1),B_1 (1,0),...,B_N (2,0),B_N (1,1)), followed by all the remained coefficients of all blocks in the last scan. Once all similar-positioned coefficients have been encoded, the next position to be encoded is the one occurring next in the zigzag traversal as indicated in the figure above. It has been found that
baseline progressive JPEG encoding usually gives better compression as compared to
baseline sequential JPEG due to the ability to use different Huffman tables (see below) tailored for different frequencies on each "scan" or "pass" (which includes similar-positioned coefficients), though the difference is not too large. In the rest of the article, it is assumed that the coefficient pattern generated is due to sequential mode. In order to encode the above generated coefficient pattern, JPEG uses Huffman encoding. The JPEG standard provides general-purpose Huffman tables; encoders may also choose to generate Huffman tables optimized for the actual frequency distributions in images being encoded. The process of encoding the zig-zag quantized data begins with a run-length encoding explained below, where: • is the non-zero, quantized AC coefficient. •
RUNLENGTH is the number of zeroes that came before this non-zero AC coefficient. •
SIZE is the number of bits required to represent . •
AMPLITUDE is the bit-representation of . The run-length encoding works by examining each non-zero AC coefficient and determining how many zeroes came before the previous AC coefficient. With this information, two symbols are created: : Both
RUNLENGTH and
SIZE rest on the same byte, meaning that each only contains four bits of information. The higher bits deal with the number of zeroes, while the lower bits denote the number of bits necessary to encode the value of . This has the immediate implication of
Symbol 1 being only able store information regarding the first 15 zeroes preceding the non-zero AC coefficient. However, JPEG defines two special Huffman code words. One is for ending the sequence prematurely when the remaining coefficients are zero (called "End-of-Block" or "EOB"), and another when the run of zeroes goes beyond 15 before reaching a non-zero AC coefficient. In such a case where 16 zeroes are encountered before a given non-zero AC coefficient,
Symbol 1 is encoded "specially" as: (15, 0)(0). The overall process continues until "EOB" denoted by (0, 0) is reached. With this in mind, the sequence from earlier becomes: :(0, 2)(-3);(1, 2)(-3);(0, 2)(-2);(0, 3)(-6);(0, 2)(2);(0, 3)(-4);(0, 1)(1);(0, 2)(-3);(0, 1)(1);(0, 1)(1); :(0, 3)(5);(0, 1)(1);(0, 2)(2);(0, 1)(-1);(0, 1)(1);(0, 1)(-1);(0, 2)(2);(5, 1)(-1);(0, 1)(-1);(0, 0); (The first value in the matrix, −26, is the DC coefficient; it is not encoded the same way. See above.) From here, frequency calculations are made based on occurrences of the coefficients. In our example block, most of the quantized coefficients are small numbers that are not preceded immediately by a zero coefficient. These more-frequent cases will be represented by shorter code words.
Compression ratio and artifacts . The resulting compression ratio can be varied according to need by being more or less aggressive in the divisors used in the quantization phase. Ten to one compression usually results in an image that cannot be distinguished by eye from the original. A compression ratio of 100:1 is usually possible, but will look distinctly artifacted compared to the original. The appropriate level of compression depends on the use to which the image will be put. Those who use the
World Wide Web may be familiar with the irregularities known as
compression artifacts that appear in JPEG images, which may take the form of noise around contrasting edges (especially curves and corners), or "blocky" images. These are due to the quantization step of the JPEG algorithm. They are especially noticeable around sharp corners between contrasting colors (text is a good example, as it contains many such corners). The analogous artifacts in
MPEG video are referred to as
mosquito noise, as the resulting "edge busyness" and spurious dots, which change over time, resemble mosquitoes swarming around the object. These artifacts can be reduced by choosing a lower level of
compression; they may be completely avoided by saving an image using a lossless file format, though this will result in a larger file size. The images created with
ray-tracing programs have noticeable blocky shapes on the terrain. Certain low-intensity compression artifacts might be acceptable when simply viewing the images, but can be emphasized if the image is subsequently processed, usually resulting in unacceptable quality. Consider the example below, demonstrating the effect of lossy compression on an
edge detection processing step. Some programs allow the user to vary the amount by which individual blocks are compressed. Stronger compression is applied to areas of the image that show fewer artifacts. This way it is possible to manually reduce JPEG file size with less loss of quality. Since the quantization stage
always results in a loss of information, JPEG standard is always a lossy compression codec. (Information is lost both in quantizing and rounding of the floating-point numbers.) Even if the quantization matrix is a
matrix of ones, information will still be lost in the rounding step.
Decoding Decoding to display the image consists of doing all the above in reverse. Taking the DCT coefficient matrix (after adding the difference of the DC coefficient back in) : \left[ \begin{array}{rrrrrrrr} -26 & -3 & -6 & 2 & 2 & -1 & 0 & 0 \\ 0 & -2 & -4 & 1 & 1 & 0 & 0 & 0 \\ -3 & 1 & 5 & -1 & -1 & 0 & 0 & 0 \\ -3 & 1 & 2 & -1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right] and taking the
entry-for-entry product with the quantization matrix from above results in : \left[ \begin{array}{rrrrrrrr} -416 & -33 & -60 & 32 & 48 & -40 & 0 & 0 \\ 0 & -24 & -56 & 19 & 26 & 0 & 0 & 0 \\ -42 & 13 & 80 & -24 & -40 & 0 & 0 & 0 \\ -42 & 17 & 44 & -29 & 0 & 0 & 0 & 0 \\ 18 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right] which closely resembles the original DCT coefficient matrix for the top-left portion. The next step is to take the two-dimensional inverse DCT (a 2D type-III DCT), which is given by: f_{x,y} = \frac{1}{4} \sum_{u=0}^7 \sum_{v=0}^7 \alpha(u) \alpha(v) F_{u,v} \cos \left[\frac{(2x+1)u\pi}{16} \right] \cos \left[\frac{(2y+1)v\pi}{16} \right] where • \ x is the pixel row, for the integers \ 0 \leq x . • \ y is the pixel column, for the integers \ 0 \leq y . • \ \alpha(u) is the normalizing scale factor defined earlier, for the integers \ 0 \leq u . • \ F_{u,v} is the approximated DCT coefficient at coordinates \ (u,v). • \ f_{x,y} is the reconstructed pixel value at coordinates \ (x,y) Rounding the output to integer values (since the original had integer values) results in an image with values (still shifted down by 128) : \left[ \begin{array}{rrrrrrrr} -66 & -63 & -71 & -68 & -56 & -65 & -68 & -46 \\ -71 & -73 & -72 & -46 & -20 & -41 & -66 & -57 \\ -70 & -78 & -68 & -17 & 20 & -14 & -61 & -63 \\ -63 & -73 & -62 & -8 & 27 & -14 & -60 & -58 \\ -58 & -65 & -61 & -27 & -6 & -40 & -68 & -50 \\ -57 & -57 & -64 & -58 & -48 & -66 & -72 & -47 \\ -53 & -46 & -61 & -74 & -65 & -63 & -62 & -45 \\ -47 & -34 & -53 & -74 & -60 & -47 & -47 & -41 \end{array} \right] and adding 128 to each entry : \left[ \begin{array}{rrrrrrrr} 62 & 65 & 57 & 60 & 72 & 63 & 60 & 82 \\ 57 & 55 & 56 & 82 & 108 & 87 & 62 & 71 \\ 58 & 50 & 60 & 111 & 148 & 114 & 67 & 65 \\ 65 & 55 & 66 & 120 & 155 & 114 & 68 & 70 \\ 70 & 63 & 67 & 101 & 122 & 88 & 60 & 78 \\ 71 & 71 & 64 & 70 & 80 & 62 & 56 & 81 \\ 75 & 82 & 67 & 54 & 63 & 65 & 66 & 83 \\ 81 & 94 & 75 & 54 & 68 & 81 & 81 & 87 \end{array} \right]. This is the decompressed subimage. In general, the decompression process may produce values outside the original input range of [0, 255]. If this occurs, the decoder needs to clip the output values so as to keep them within that range to prevent overflow when storing the decompressed image with the original bit depth. The decompressed subimage can be compared to the original subimage (also see images to the right) by taking the difference (original − uncompressed) results in the following error values: : \left[ \begin{array}{rrrrrrrr} -10 & -10 & 4 & 6 & -2 & -2 & 4 & -9 \\ 6 & 4 & -1 & 8 & 1 & -2 & 7 & 1 \\ 4 & 9 & 8 & 2 & -4 & -10 & -1 & 8 \\ -2 & 3 & 5 & 2 & -1 & -8 & 2 & -1 \\ -3 & -2 & 1 & 3 & 4 & 0 & 8 & -8 \\ 8 & -6 & -4 & -0 & -3 & 6 & 2 & -6 \\ 10 & -11 & -3 & 5 & -8 & -4 & -1 & -0 \\ 6 & -15 & -6 & 14 & -3 & -5 & -3 & 7 \end{array} \right] with an average absolute error of about 5 values per pixels (i.e., \frac{1}{64} \sum_{x=0}^7 \sum_{y=0}^7 |e(x,y)| = 4.8750). The error is most noticeable in the bottom-left corner where the bottom-left pixel becomes darker than the pixel to its immediate right.
Required precision The required implementation precision of a JPEG codec is implicitly defined through the requirements formulated for compliance to the JPEG standard. These requirements are specified in ITU.T Recommendation T.83 | ISO/IEC 10918-2. Unlike MPEG standards and many later JPEG standards, the above document defines both required implementation precisions for the encoding and the decoding process of a JPEG codec by means of a maximal tolerable error of the forwards and inverse DCT in the DCT domain as determined by reference test streams. For example, the output of a decoder implementation must not exceed an error of one quantization unit in the DCT domain when applied to the reference testing codestreams provided as part of the above standard. While unusual, and unlike many other and more modern standards, ITU.T T.83 | ISO/IEC 10918-2 does not formulate error bounds in the image domain. ==Effects of JPEG compression==