Hybrid early-termination methods and output selection procedure for iterative turbo decoders

[00002] This application is entitled to the benefit of Provisional Patent Application, Application No. 60/227,168, filed 2000 Aug. 23.

[00003] This invention relates to iterative decoding of encoded data in communication systems, specifically to reducing the amount of computations of decoding in an effective and reliable manner, and to improving the error rate performance of the decoder.

[00004] Turbo codes have been shown to have excellent bit error rate (BER) performance even when the data rate runs close to the channel capacity. This has made the turbo codes the most popular choice for error control codes in today's sophisticated communications systems. In particular, turbo codes have been adopted by all standards for third generation (3G) wireless cell phone systems.

[00005] FIG. 1 shows a typical turbo encoder. The encoder operates on the input data on a frame basis (Throughout this application, we use binary data for the purpose of simple illustration. Binary data takes only two values, namely, 0 and 1, or, in the convention of communication systems, 1 and −1. The principles in this application, however, expand and apply to other data formats as well). A frame of data X=(x₀, x₁, . . . , x_L-1) is the input to the encoder and is encoded by a first decoder 102. The same input X is also scrambled by an interleaver 104, and then encoded by a second encoder 106. A puncturing unit 108 rearranges X, and the outputs X^P1 and X^P2 of encoders 102 and 106 into a bit stream that is then punctured to form the output frame Y=(y₀, y₁, . . . , y_M-1). The numbers L and M determine the code rate, r=L/M, of the turbo code.

[00006] The complexity of an optimum decoder for any meaningful turbo code is far beyond the realm of implementation. Practical turbo decoders are all iterative type that is non-optimum but is believed to have the performance close to that of the optimum one. FIG. 2 shows an iterative decoder. The input frame Y is broken into X, X^P1 and X^P2 by a splitting unit 202. The iteration decoding then proceeds as follows. An iteration cycle starts with a first decoder 204. The inputs to first decoder 204 are X, X^P1 and the de-interleaved extrinsic information (to be explained momentarily) X^E2 from a second decoder 206. At the first iteration, X^E2 is initialized to zero. The output of first decoder 204 is the extrinsic information X^E1 that indicates the likelihood of each bit in X to be zero or one. The extrinsic information is also called soft output, since it is a multi-bit value indicating how close the data bit is to zero or one. When first decoder 204 has finished, second decoder 206 starts decoding with inputs being the interleaved version of X, X^E1, and X^P2. A full iteration ends when second decoder 206 is done. The output X^E2 of second decoder 206 will be used by first decoder 204 in the next iteration. Interleavers 208 and 210, and a de-interleaver 212, are to ensure the proper bit alignments.

[00007] After certain number N of iterations (typically N=10˜20), the decoded frame is either correct, or the number of errors in the frame will not decrease with further iterations in a significant manner. The iteration process thus stops. X, X^E1 and X^E2 are combined and a decision is made by a combining/decision unit 214. The output of combining/decision unit 214 is the output of the iterative decoder.

[00008] Iterative decoding is not limited to decoding of the turbo codes. Other codes such as product codes and low-density-parity-check codes also call for iterative decoding as a feasible implementation.

[00009] Iterative decoding still requires tremendous amount of computations. For example, for data rate of 2 Mbits/s in 3G standard, it takes an iterative decoder about 10¹⁰ basic operations per second with 10 iterations. Such a requirement far exceeds the capability of the general-purpose computer CPUs or digital signal processors (DSP) of today and even of near future. Special circuitry such as application-specific integrated circuits (ASIC) has to be designed to meet the demand on processing capacity.

[00010] Even with ASIC implementation, the amount of processing dictates that the power dissipation of the circuit is huge. Turbo decoder alone, when operates at the high data rate, typically consumes more power than the rest of the receiver does in a 3G system. Most applications, especially wireless, are power limited; the device is generally powered by battery. Reduction in the power consumption directly translates to the extended battery life.

[00011] Another problem with iterative decoders is iteration abnormality that is inherent in iterative type of decoders. A standard iterative decoder outputs the decoded frame of iteration N, except that when early-termination is used, the output is the decoded frame of the last iteration. Recall that the iterative decoding is non-optimum. While the BER improvement with the number of iterations is expected, it is not guaranteed. In fact, this rule of thumb breaks down when channel condition improves, i.e., the signal-to-noise ratio (SNR) is higher. It has been observed that the BER can actually get worse with more iterations, and for certain number of iterations, the BER gets worse with higher SNR. Such abnormality can be extremely annoying when incorporating turbo codes into communication systems, since efforts to improve the system performance, such as increasing the number of iterations and improving SNR, may meet with adverse result.

[00012] The preceding paragraphs illustrate the some noted problems that iterative decoders possess. The rest of this section will discuss known prior arts to attack these problems.

[00013] To reduce the amount of computations hence the power dissipation, an early-termination technique has been applied to the iterative decoder. This technique is based on the fact that not all frames need to go through all N iterations. Many frames become correct after first several iterations. For those frames the iteration can be terminated early so that the computation load for the rest of the iterations is saved. Since one can never know with absolute certainty that a decoded frame is correct unless he or she knows what has been transmitted, an indirect method (a correctness indicator) has to be used to indicate the correctness of a frame. Note that additional error is introduced due to the incorrect termination by the indicators.

[00014] Depending upon the types of the indicators used, existing early-termination methods fall into the following categories: output comparison, cyclic-redundant-code (CRC) check and SNR threshold on soft output. Output-comparison-based indicators compare certain characteristics of the decoded frames from consecutive iterations. For example, Hagenauer [1] looked at the cross-entropy of the two frames, where the cross entropy is a measure of similarity of two frames. The hard-decision difference (HDD), which is the number of differing bits between two hard-decisioned frames is another indicator [2], and is called hard-decision-aided (HDA) method. The rational of this approach is that if the number of errors in a frame improves with iterations, so will the indicators. If the indicator shows that the two frames from two consecutive iterations do not change much, it would mean that the frame from current iteration is either (a) containing no errors, or (b) still having errors but unlikely to improve with further iterations. In either case it makes sense to stop the iteration process. Thus in the case of the cross-entropy method, if the cross-entropy goes below a threshold, one can conclude that further improvement is unlikely and the frame is correct. In the case of the HDA, if the number of the differing bits is zero, i.e., the two frames after hard-decision are identical, it is likely that the decoder has come to the correct frame.

[00015] The performance of the comparison-based method is mixed. It achieves significant savings in computation, hence in power consumption, generally above 70%. BER and frame error rate (FER), however, degrades due to incorrect termination. For very long frames (about several thousands data bits or more), the degradation in BER is generally kept within 50%. The degradation in FER is more evident, typically more than doubled. For short frames (from below one hundred to one thousand data bits), the degradation is worse for both BER and FER. Simulation shows that the BER and FER with early-termination are 5 and 10 times more than the BER and FER without early-termination, respectively.

[00016] The CRC-based early-termination method relies upon the CRC bits that are attached to the frame to detect if there are any errors in the frame. If the frame passes the CRC check, then the frame is declared to be correct. Use of CRC for error detection has long been practiced in communication systems. Its use in conjunction with the turbo decoder for early-termination was described in [3].

[00017] The savings in computations of the CRC-based method is slightly higher than the output-comparison-based method, since it takes two frames from consecutive iterations for the latter to make a decision, and even the first frame is correct, it has to wait for the second frame to get the comparison result. The BER and FER performances of the CRC method depend upon the number of CRC bits. In general, if the number of CRC bits is c, the additional error rate caused by CRC-based early-termination is proportional to 2^−c. Thus the larger the c, the better the performance.

[00018] Practical systems, however, put a limit on the number of CRC bits. For example, one of the 3G standards, UMTS, has c=24 for very large frames. For smaller frames, c is reduced to 16, 12 or 8. Significant degradation has been observed even for c=24. It is also not very economical to add many CRC bits to short frames. For example, UMTS has a frame size as small as 40 bits. It is a fairly high overhead for a 24-bit CRC to be in a 40-bit frame.

[00019] SNR-threshold-based strategy is described in [4] and is summarized as follows. For binary data, there is some value E such that the soft output of first decoder 204 or second decoder 206 in FIG. 2 is centered on either +E or −E, corresponding to +1 or −1. At early iterations, there is significant overlap between the two peaks in the distributions of the soft output. At later iterations where the decoded frame is more likely to be correct, the value E increases, the peaks narrow and become well separated. FIG. 3 shows this phenomenon. Since the distribution of the soft output can be approximated by the distribution of binary signaling with Gaussian noise, an estimated SNR from the soft output can be computed. The higher the soft output SNR, the more separated the peaks. The soft output SNR increases with the number of iterations. If the soft output SNR exceeds a preset threshold after certain iterations, the decoded frame is believed to be correct and the iteration process terminates.

[00020] The performance of the SNR-threshold strategy depends a lot upon the chosen threshold. If the threshold is too small, the iteration will terminate too early, resulting unacceptable BER/FER. If the threshold is too high, the decoder may take extra iterations to exceed the threshold after the decoded frame becomes correct, and the benefit of early-termination diminishes. Thus choosing the threshold becomes an undesirable trade-off between the BER/FER and power consumption. Moreover, optimum thresholds change with channel conditions, so a fixed threshold may behave poorly in practice.

[00021] In summary, existing early-termination methods cause severe degradation in BER/FER performance, increase system overhead, or force a trade-off between BER/FER and power consumption. Some may not work well, or simply not work at all, in certain conditions. To the best knowledge of this applicant, the other problem, iteration abnormality, has not seen any effective solutions.

[00022] The present invention relates to iterative decoding of an encoded frame of data. A hybrid early-termination method performs a correctness test on the decoded frame and terminates the iteration process if the decoded frame is declared correct. In addition, an output selection procedure chooses one of the decoded frames if, during iterative decoding of an encoded frame of data, all of the decoded frames fail the correctness test.

[00023] In accordance with embodiments of the present invention, the hybrid early-termination comprises output-comparison and parity-check. A decoded frame of current iteration is compared with the one of the previous iteration for “closeness test”. If the frame passes the closeness-test, it is subjected to parity-check, using the parity-check bits (the CRC check bits are a special case of the parity-check bits) in the encoded frame. If the frame passes the parity-check, it is declared to be correct and the iteration process is terminated.

[00024] A variation of the above implementation is that a decoded frame is parity-checked first. If it passes the parity-check, it is compared with a (typically, the latest) frame that also passed parity-check in previous iterations for closeness-test. The frame is declared to be correct if it passes both parity-check and closeness-test.

[00025] A salient advantage of the hybrid early-termination is its high reliability. Since the iteration process terminates only if the decoded frame passes two independent tests, there is virtually no incorrect termination, thus eliminating the BER/FER degradation that is common in existing early-termination methods. Hybrid early-termination also requires much fewer parity-check bits than the CRC-check-based early-termination does.

[00026] In accordance with embodiments of the present invention, the output selection procedure comprises choosing one of decoded frames if all of the decoded frames fail the correctness-test. An “error indicator” is computed for each decoded frame to indicate the number of errors in the decoded frame. At the end of the iteration process, the decoded frame with a best error-indicator is chosen to be the output of the decoder. A best error-indicator is defined to be an error-indicator that meets certain criterion (criteria), such as the one that indicates that the decoded frame has least errors among frames from all iterations.

[00027] The iteration abnormality phenomenon results from the fact that a standard iterative decoder always chooses the decoded frame of the last iteration as the decoder output. Study shows that the output selection procedure completely eliminates the iteration abnormality.

[00028] Accordingly, several objects and advantages of the present invention are:

• [00029] (a) to provide an early-termination method that accurately determines the correctness of a decoded frame;
• [00030] (b) to provide an early-termination method that reliably terminates the iteration process without degrading the BER/FER performance;
• [00031] (c) to provide an early-termination method that has minimum system overhead;
• [00032] (d) to provide an early-termination method that eliminates the undesirable trade-off between BER/FER and power consumption;
• [00033] (e) to provide an output selection procedure that eliminates the iteration abnormality that is inherent in the iterative decoders;
• [00034] (f) to provide an output selection procedure that improves the BER of the iterative decoder.

[00035] Further objects and advantages will become apparent from a consideration of the ensuing description and drawings.

[00041] The present application combines the output-comparison-based method and the CRC-based method to form a hybrid early-termination method that is extremely reliable and effective. In the comparison process, the present application compares the decoded frames between half iterations to further reduce the computational load. Referring to FIG. 2, one half iteration is defined to be a complete decoding operation by either first decoder 204 or second decoder 206. The present application also uses error-detection codes (EDC) to provide a more general framework that includes CRC as a special case. Best error-detection codes can often be found outside the class of CRC codes, especially when the code length (frame size) is small.

[00042] In accordance with embodiments of the present invention, an iterative decoder decodes an encoded frame that includes parity-check information, and terminates the iteration when either (a) the maximum number of iterations N has been reached, or (b) the decoded frames from the current iteration and the previous 0.5M iteration is sufficiently close by some “closeness measure” CM, and the current decoded frame passes the parity-check. The parameter M controls the separation in terms of half iterations between the two decoded frames in comparison. M satisfies 1≦M ≦2N and M=1 is the best choice.

[00043] In the case that the maximum number of iterations N has been reached, the iterative decoder computes an error-indicator EI for each decoded frame, and selects the frame with the lowest indicator as the decoder output.

[00044] The closeness-measure CM defines how close the two decoded frames are. An example of CM for two frames F1 and F2 is CM_HD, based on the hard-decisioned frames and defined as

[00045] CM_HD=hard-decision difference (HDD) between frame F1 and frame F2.

[00046] As defined previously, the HDD between frame F1 and frame F2 is the number of differing bits between two hard-decisioned frames F1 and F2.

[00047] If F1 and F2 contain soft symbols, i.e., numbers that represent the likelihood value of the bits, a hard decision that converts the likelihood value to bit must be performed first before computing CM_HD. Two frames are considered to be sufficiently close if CM_HD≦T_HD, a threshold. An appropriate choice is T_HD=0.

[00048] Another example of CM is cross-entropy based CM_CE, defined as CM_CE=the cross-entropy between frame F1 and frame F2.

[00049] Approximate expressions for CM_CE are derived in [1] and [2]. Two frames are considered sufficiently close if CM_CE≦T_CE, a cross-entropy threshold. T_CE is generally determined by simulations.

[00050] Error-indicator EI is chosen to have strong correlations with the number of errors in the frame. It turns out from the simulation study that the following two choices, EI_HD=CM_HD and EI_CE=CM_CE, yield extremely good results.

[00051] FIG. 4 shows an exemplary implementation in accordance with an embodiment of the present invention, with M=1. Referring to FIG. 4, the decoding of a new frame begins at step 401. At step 401, the iteration counter i is set to zero. At step 402, the first half iteration decoding is performed, and a decoded frame D₀ is stored. For the first half iteration, an error-indicator EI₀ is set to a very large number. This assumes that the error indicator is positively correlated with the number of errors in the frame; however, an error-indicator that has negative correlation with the number of errors in the frame can also be used, although the choice does not appear to be as natural. In this case, error-indicator EI₀ should be set to a very small number (or a negative number with a very large amplitude). Error-indicators with positive correlation are assumed in the rest of the description.

[00052] The value that error-indicator EI₀ is set to depends upon the implementation. For example, if EI_HD is used, EI₀ can be set to the size of the frame, since this is the maximum value EI_HD can have. If EI_CE is used, EI₀ can be set to the maximum value that the implemented system can represent.

[00053] Step 403 increments the iteration count i by 0.5 when M=1. Step 404 continues to perform the half iteration decoding, store decoded frame D_i and compute error-indicator EI_i. At step 405, a test compares whether i is greater than or equal to a prescribed value, i_min. i_min controls at which iteration that early-termination mechanism starts to apply to iterative decoding, and is typically set to 0, in which case step 405 is not necessary. However there are some cases where it is believed the frame is unlikely to be correct in the first several iterations, thus there is no need for testing for early-termination until after i_min iterations. Storage of decoded frame D_i and error-indicator EI_i may not be necessary in step 404 before i_min iterations. If i_min iterations have not been reached, the method proceeds to step 403.

[00054] If the test of step 405 determines that i_min iterations have been reached, the method proceeds to step 406. Step 406 computes closeness-measure CM to compare if decoded frames D_i and D_i-0.5 are sufficiently close. Step 406 differs from existing arts such as the ones described in [1] and [2] in that the separation of the two frames can be as small as half iteration in the present invention while existing arts all use frames one full iteration apart. The computational load is further reduced to the minimum by the present invention. If the test of step 406 determines that D_i and D_i-0.5 do not pass the closeness-test, the method proceeds to step 403.

[00055] If it is determined during step 406 that decoded frames D_i and D_i-0.5 are sufficiently close, the method proceeds to step 407. Step 407 checks the parity of decoded frame D_i. If the test of 407 determines that decoded frame D_i passes the parity-check, the method proceeds to step 409. Step 409 outputs decoded frame D_i and terminates the iteration process. The method proceeds to step 401 to start decoding of the next frame.

[00056] If, however, the test of step 407 determines that decoded frame D_i does not pass the parity-check, the method proceeds to step 408. Step 408 monitors if the maximum number of iterations, N, has been reached. If the test of step 408 determines that N iterations have not been reached, the method proceeds to step 403.

[00057] If, however, it is determined that N iterations have been reached, the method proceeds to step 410. Step 410 chooses an index j such that error-indicator EI_j is the smallest one among all error-indicators EI₁, i_min≦i ≦N, and selects decoded frame D_j as the output of the iterative decoder. The iterative decoding ends at the completion of step 410. The method proceeds to step 401 to start decoding of the next frame.

[00058] It appears that all decoded frames have to be stored in order to find the one with smallest EI. FIG. 5 shows an exemplary implementation that requires the storage of only one previously decoded frame. Step 501 initializes an error-indicator buffer EI_min to EI₀ and iteration counter i to zero. Step 502 performs half iteration decoding and computes error-indicator EI_i. Step 503 compares EI_min to EI_i. If the comparison of step 503 determines that EI_min is less than EI_i, the program control proceeds to step 505.

[00059] If it is determined during step 503 that EI_min is not less than EI_i, the program control proceeds to step 504. Step 504 updates error-indicator buffer EI_min and a decoded frame buffer D_min with EI_i and D_i. The program control then proceeds to step 505.

[00060] Step 505 increments the iteration counter i by 0.5 when M=1. The program control proceeds to step 502. At the end of the iteration process, decoded frame buffer D_min is the decoder output with the smallest EI.

[00061] Step 503 shows that the buffer updates take place even if EI_min=EI_i. This guarantees that if there are multiple frames with the same best error-indicator value, the output selection procedure always selects the frame in the latest iteration, since a frame generally has fewer errors in a later iteration.

[00062] There are possible variations from the foregoing implementation. To show one exemplary variation, recited below are two conditions for terminating the iteration process mentioned in the second paragraph of this section: (a) the maximum number of iterations N has been reached, or (b) the decoded frames from the current half iteration and the previous 0.5M iteration is sufficiently close by some “closeness measure” CM, and the current decoded frame passes the parity check.

[00063] Implementation of condition (b) performs the closeness-test first and then the parity-check. Condition (b) can be replaced by the following: (b′) the current decoded frame passes the parity check, and is sufficiently close to, by some “closeness measure” CM, the last previously decoded frame that also passed the parity check.

[00064] Implementation of condition (b′) performs the parity-check first and then the closeness-test. It may require an additional frame buffer to store the last previously decoded frame that also passed the parity-check. The advantage of using condition (b′) is that the BER/FER degradation can be further reduced.

[00065] There are also implementation variations in the output selection procedure. For example, computing error-indicator EI involves two decoded frames. One of the decoded frames is the one for which error-indicator EI is computed. When error-indicator EI for a decoded frame is computed, either the preceding decoded frame or next decoded frame can be used as the other decoded frame.

[00066] The output procedure can be also made more elaborate. For example, when a frame with the best error-indicator EI is determined, a separate correctness-test, such as parity-check, can be performed on that frame and the other frame that is involved in the calculation of the best error-indicator EI. If one of them passes the test, it will be selected as the output frame.

[00067] It is to be understood that the descriptions and variations shown and presented herein are merely illustrative of the principles of this application and that various modifications may be implemented by those skilled in the art without departing from the scope and the spirit of the application.

[00068] It is to be further understood that the principles of this application also apply to decoding of error control codes other than turbo codes, such as product codes, low density parity check codes, etc., the decoding of which calls for iterative types of decoder.