KR102773892B1 - Non-Volatile Memory Device for Deep Neural Network and method for improving reliability thereof - Google Patents

Abstract

The disclosed embodiment provides a nonvolatile memory device for a DNN and a method for improving the reliability thereof, which can reduce data errors and improve fault tolerance and reliability by remapping an address of a memory cell so that a sticky value of a memory cell having a sticky defect matches a weight of a DNN predetermined by learning, thereby allowing a memory cell having a sticky defect to operate like a normal memory cell as much as possible.

Description

{Non-Volatile Memory Device for Deep Neural Network and method for improving reliability thereof}

The disclosed embodiments relate to a nonvolatile memory device for DNN and a method for improving reliability thereof, and more particularly, to a nonvolatile memory device for DNN with improved fault tolerance and a method for improving reliability thereof.

Deep Neural Networks (DNNs), a type of machine learning (ML) technique, have been actively used in various fields such as computer vision, natural language processing, and big data analysis due to their high efficiency and usability compared to existing programming approaches.

Figure 1 shows a schematic structure of DNN, and shows the structure of ResNet-18, one of the representative CNNs among DNNs.

As illustrated in Fig. 1, DNNs are generally configured to include an input layer, an output layer, and a plurality of hidden layers that are located between the input layer and the output layer and perform neural network operations, each of which includes a plurality of weights determined by learning. In order for such a DNN to perform neural network operations, it must be able to store the weights of each layer and perform operations (such as convolution operations and pooling operations) according to a specified method on the input data or the output data of the previous layer and the weights of each layer. Therefore, in order to utilize a DNN, a large amount of computational resources and a large storage capacity for storing the weights are required.

Therefore, high-density, low-power consumption, and performance-efficient memory technology are required to efficiently utilize DNNs. Previously, memory devices including charge-based memory devices such as DRAM and flash were mainly used for DNNs. However, these memory devices have limitations such as limited scalability, power leakage, and increased vulnerability to defects, making them unsuitable for application to DNNs. Accordingly, the demand for new memory devices for DNNs has increased, and recently, new non-volatile memories (NVMs) such as phase change memory (PCM), resistive random-access memory (ReRAM), and spin-transfer torque magnetic memory (STT-MRAM) have attracted attention as memory devices for DNNs.

However, nonvolatile memory devices such as PCM or ReRAM may experience stuck-at faults due to low durability and immaturity of the manufacturing process. These stuck-at faults make it difficult to use nonvolatile memory devices in memory devices because they allow reading of data currently stored in the memory device but prevent writing of new data.

Korean Patent Publication No. 2021-0026767 (Public)

The disclosed embodiments aim to provide a nonvolatile memory device for DNNs capable of reducing data errors and increasing reliability, and a method for improving the reliability thereof.

The disclosed embodiments aim to provide a nonvolatile memory device for a DNN having excellent fault tolerance even to stuck-in faults and a method for improving the reliability thereof by remapping an address of a memory cell so that the stuck-in value of a memory cell having a stuck-in fault corresponds to a weight of a DNN predetermined by learning.

A nonvolatile memory device for a DNN according to an embodiment comprises: a cell array having a plurality of memory cells; and a memory controller which receives a plurality of weights, each mapped to an address designating a memory cell to be stored, and remaps an encoding address obtained by encoding the address based on an auxiliary bit composed of a plurality of bits to the weights, and stores the weights in a memory cell selected from among the plurality of memory cells according to the encoding address, wherein the auxiliary bits are set such that a memory cell in which the weights are stored is changed to a memory cell selected from among the plurality of memory cells according to the encoding address, and a stored value among the plurality of memory cells is fixed as a stuck-in value, and a bit value of the weight matches the stuck-in value of a stuck-in defective memory cell in which a stuck-in defect occurs.

The above memory controller can obtain the encoding address by performing a bitwise XOR operation on the address and the auxiliary bit.

The above auxiliary bits can be set so that the number of bit value errors that occur because the bit values of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and the stickiness value of the stickiness defective memory cell do not match as the same value is minimized.

The above auxiliary bits may be set such that a deviation between a value stored in the memory cell and the weight is minimized due to a bit value error that occurs when a bit value of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and a sticking value of the sticking defective memory cell do not match as the same value.

The above auxiliary bits can be set so that a smaller number of bit value errors occur when there are a plurality of auxiliary bits having a minimum deviation between the value stored in the memory cell and the weight.

The above-mentioned plurality of memory cells are divided into memory blocks of a size from which stored data is read out in bulk, and the auxiliary bit may have a number of bits corresponding to the number of bits of an internal block address that distinguishes and selects a plurality of memories included in the memory block among a plurality of bits of the address.

The above memory controller can detect an address of a memory cell in which a stuck-in defect has occurred, in which a stored value is fixed, among the plurality of memory cells, and determine the stuck-in value of the detected memory cell.

The above memory controller can read the weight stored in the memory cell selected according to the encoding address, decode the encoding address remapped to the weight based on the auxiliary bit, and remap it to the address.

The above plurality of memory cells can be implemented as either a phase change memory (PCM) device or a resistive random-access memory (ReRAM) device.

The above multiple weights are included in each of multiple layers of a deep neural network (DNN) and are used in neural network operations, and may have values determined in advance through learning.

A method for improving the reliability of a nonvolatile memory device for a DNN according to an embodiment comprises the steps of: obtaining a plurality of weights, each of which is mapped to an address designating a memory cell to be stored; remapping an encoding address obtained by encoding the address based on an auxiliary bit composed of a plurality of bits, to the weights; and storing the weights in a memory cell selected from among the plurality of memory cells according to the encoding address, wherein the auxiliary bits are set such that a memory cell in which the weights are stored is changed to a memory cell selected from among the plurality of memory cells according to the encoding address, so that a stored value among the plurality of memory cells is fixed as a stuck-in value, and a bit value of the weights matches the stuck-in value of a stuck-in defective memory cell in which a stuck-in defect occurs.

Therefore, the nonvolatile memory device for DNN according to the embodiment and the method for improving the reliability thereof can enable a memory cell with a stuck-on defect to operate as similarly as possible to a normal memory cell by remapping the address of the memory cell so that the stuck-on value of the memory cell with the stuck-on defect matches the weight of the DNN determined in advance by learning. Therefore, data errors can be reduced, and fault tolerance and reliability can be improved. In addition, when applied to a DNN, the method can prevent performance degradation of the DNN by reducing computational errors.

Figure 1 shows a schematic structure of DNN.
Figure 2 is a diagram for explaining a sticky defect in nonvolatile memory.
FIG. 3 illustrates a schematic structure of a non-volatile memory device for a DNN according to one embodiment.
Figure 4 is a diagram for explaining address remapping according to auxiliary bits.
Figure 5 is a diagram illustrating an example of a technique for improving the reliability of a non-volatile memory device for DNN.
Figure 6 is a diagram for explaining weight deviation according to error bit position.
Figure 7 is a diagram illustrating another example of a technique for improving the reliability of a non-volatile memory device for DNN.
FIG. 8 illustrates a method for improving the reliability of a memory device for a DNN according to one embodiment.

Hereinafter, specific embodiments of one embodiment will be described with reference to the drawings. The following detailed description is provided to help a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments, if it is judged that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the embodiment, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of their functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terms used in the detailed description are only for describing the embodiments, and should never be limited. Unless clearly used otherwise, the singular form includes the plural form. In this description, the expressions such as "comprises" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, parts or combinations thereof other than those described. Additionally, terms such as “unit,” “unit,” “module,” “memory block,” etc., described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

Figure 2 is a diagram for explaining a sticky defect in nonvolatile memory.

In Fig. 2, (b) represents a case where a stuck-on defect occurs in which a bit value is fixed to 0 or 1 in some memory cells among a plurality of memory cells of a nonvolatile memory, (a) represents data that should be stored in the memory cell of (b), and (c) represents read data obtained by reading the data of the memory cell of (b) in which the data of (a) is stored.

In Fig. 2 (b), 16 memory cells are displayed divided into groups of 4 each. In the first and second groups, the third memory cell and the first memory cell each have sticking values of 0 and 1, resulting in a sticking defect in one memory cell. In the third group, the second and third memory cells have sticking values of 0 and 1, resulting in two memory cells having sticking defects. In addition, it can be seen that in the fourth group, the bit value of the fourth memory cell has a sticking value of 0, resulting in one sticking defect.

As in (b) of Fig. 2, when four data (0111, 0101, 1001, 0110) of (a) are stored in each of four groups, the data of (a) is normally written to memory cells in which no sticking defect occurs, but the data of memory cells in which sticking defects occur is not written and is maintained as a stuck value. Therefore, when the data of the four groups are read, the stuck values do not change, so the read data are obtained as "0101", "1101", "1101", "0110", as in (c).

At this time, when comparing the write data of (a) with the read data of (c), it can be seen that while one bit value each in the first and second groups and the third group appears as an error due to a sticking defect, no error due to a sticking defect appears in the third memory cell of the third group and the fourth group. This is because, due to the nature of a sticking defect, if the bit value of the data to be stored is identical to the sticking value of the memory where the sticking defect has occurred, the bit value of the data can be read normally and thus cannot be regarded as an error. Therefore, even if a memory device includes a memory cell having a sticking defect, it can operate like a normal memory device if the bit value of the data to be stored matches the sticking value of the sticking defect memory cell.

However, in the case of a typical memory device, since arbitrary data must be able to be stored, a memory in which a sticky defect occurs, as in (b) of Fig. 2, cannot be generally used. In addition, even if the data to be stored can be confirmed in advance and the value of the confirmed data does not change, if the bit value of the data to be stored and the sticky value of the memory cell in which the sticky defect occurs are not the same, the memory device cannot be used to store the data.

FIG. 3 illustrates a schematic structure of a non-volatile memory device for a DNN according to one embodiment.

Referring to FIG. 3, a nonvolatile memory device (10) for a DNN according to one embodiment may include a cell array (11), a memory controller (12), an encoder (13), and a decoder (14).

The cell array (11) includes a plurality of memory cells (not shown). Here, each of the plurality of memory cells may be implemented as a nonvolatile memory element such as PCM or ReRAM, and some of the plurality of memory cells may have a stuck-on defect.

And most nonvolatile memory devices read or write data in memory block units including a plurality of memory cells for data processing efficiency. At this time, a plurality of memory blocks of the cell array (11) can be distinguished and selected by a specified number of upper bits among addresses each consisting of a plurality of bits, and a plurality of memory cells within each memory block can be distinguished and selected by the remaining lower bits of the address. Here, the lower bit of the address for selecting memory cells within the memory block is called an intra-block address. That is, a plurality of memory cells included in each memory block in the cell array (11) can be distinguished and selected by an intra-block address consisting of some bits of the address.

In addition, the memory device (10) may be configured so that an internal block address is individually assigned to each of a plurality of memory cells within a memory block and selected, but for data input/output efficiency, a specified number of memory cells may be configured so that they are selected as a group by one internal block address. For example, in the case of (b) of Fig. 2, four memory cells included in each of four groups distinguished from 16 memory cells may be selected by the same internal block address.

Meanwhile, when the memory device is used for DNN, a plurality of weights provided in a plurality of layers of the DNN transmitted from the DNN distribution module (20) can be stored in the cell array (11). Here, each weight is quantized to have a specified number of bits and is composed of a plurality of bit values.

In addition, although not illustrated, the cell array (11) may further include a peripheral circuit module (not illustrated) including a row decoder, a column decoder, a sense amplifier, etc. The peripheral circuit module may apply a write voltage (or current) to a memory cell of a memory block selected by an address under the control of the memory controller (12) to write a bit value to be stored. In addition, the peripheral circuit may apply a read voltage (or current) to a memory cell of a memory block selected under the control of the memory controller (12) to determine a bit value according to the state of a memory cell in the memory block and transmit the determined bit value to the memory controller (12).

When weights are applied to multiple layers constituting the DNN, the memory controller (12) selects a memory block in which to store the applied weights from the cell array (11) according to an address, and controls the peripheral circuit modules of the cell array (11) according to the internal block address to apply a light voltage (or current) according to each bit value of the weights to the memory cells included in the selected memory block so that the weights are stored.

In addition, the memory controller (12) selects a memory block according to an address, and controls a peripheral circuit module according to an internal block address to apply a read voltage (or current) to a memory cell of the selected memory block. Then, by determining the state of the memory cell from the output current (or voltage) output from the memory cell according to the read voltage (or current) that has become the peripheral circuit module of the cell array (11), and obtaining a bit value, the bit value obtained from the peripheral circuit module can be applied to check the weight.

In addition, the memory controller (12) can write a bit value to each of a plurality of memory cells included in the cell array (11) and read the bit value written to the memory cell to determine a memory cell in which a sticky defect has occurred and its sticky value. Then, when the memory cell in which a sticky defect has occurred and the sticky value are determined and the bit value of the weight to be stored in the memory cell of each memory block is confirmed, an auxiliary bit that matches the confirmed bit value with the sticky value of the memory cell in which the sticky defect has occurred to the same value can be generated and transmitted to the encoder (13) and the decoder (14), respectively.

The memory controller (12) stores a bit value of 0 in each of a plurality of memory cells included in the cell array (11), reads the stored bit value to determine a memory cell in which a sticky value of 1 has occurred, and then stores a bit value of 1 in each of a plurality of memory cells and reads the stored bit value to determine a memory cell in which a sticky value of 0 has occurred.

As described above, in a memory cell having a stuck-in defect, the currently stored bit value is not changed, so if the bit value of the weight to be stored in the memory cell having a stuck-in defect and the stuck-in value of the memory cell do not match identically, the weight has an error value. This causes an operation error of the DNN using the memory device (10), which acts as a factor that deteriorates the performance of the DNN.

Therefore, the memory controller (12) of the embodiment encodes and remaps the address mapped to the weight, thereby changing the memory cell in which each bit value of the weight is stored. At this time, the memory controller (12) can obtain auxiliary bits for converting the address so that the bit value of the weight can match the stuck value of the stuck-on fault memory cell, and transmit the obtained auxiliary bits to the encoder (13) and the decoder (14).

In the case of a general memory device, since not only arbitrary data can be stored, but also the stored data must be able to be changed at all times, even if a stuck-in fault memory cell is identified, it is not easy to match the stuck-in value of the stuck-in fault memory cell with the data to be stored. However, the memory device of the embodiment is a memory device for a DNN, and the weights determined by learning in a DNN where learning is completed are generally not changed. That is, both the stuck-in value of the stuck-in fault memory cell and the bit values of the weights can be identified. Therefore, the memory controller (12) can obtain an auxiliary bit for remapping by changing the address mapped to the weights into an encoding address so that the stuck-in value and the bit values of the weights can be optimally matched. A detailed description of a technique for obtaining the auxiliary bits will be described later.

The encoder (13) and decoder (14) may be positioned between the memory controller (12) and the cell array (11) as a configuration added to the memory device (10) in the embodiment.

The encoder (13) encodes an address mapped to a weight in the memory controller (12) according to the auxiliary bit and converts it into an encoding address, thereby causing the weight to be remapped to the encoding address. In other words, the encoder (13) causes a memory block mapped to the address to be remapped to the encoding address, thereby causing the memory cell in which the weight is stored in the cell array (11) to be changed.

For example, the encoder (13) can obtain an encoded address by performing a bitwise XOR operation on the auxiliary bit and the address. The encoder (13) can be implemented with a plurality of XOR gates (not shown) that perform a bitwise XOR operation on the auxiliary bit and the address.

At this time, if the upper bit position of the address mapped to the weight by the encoder (13) is changed, the weight may be stored in a different memory block than before being remapped in the cell array (11). This means that a plurality of weights may be distributed and stored in different memory blocks, which is inefficient from the perspective of memory access. Therefore, it is preferable that the weights be stored in memory cells within the same memory block, even if the memory cells in which they are stored are changed according to the encoding address remapped by the encoder (13). Therefore, in the embodiment, it is assumed that the auxiliary bits are obtained as the number of bits corresponding to the number of bits of the internal block address, and the encoder (13) performs encoding on the internal block address of the address using the auxiliary bits. However, the present embodiment is not limited thereto.

Meanwhile, the decoder (14) decodes the encoding address according to the auxiliary bit and reversely converts it to the address before encoding when the weight is remapped to the encoding address, the memory cell at the changed position is selected according to the encoding address, and the weight according to the state of the selected memory cell is determined and applied by the peripheral circuit module, and maps the reversed address and the determined weight and transfers it to the memory controller (12), thereby enabling the memory controller (12) to process the determined weight in the same manner as before.

Here, the decoder (14) also has a number of XOR gates similar to the encoder (13) to perform an XOR operation on the auxiliary bit and the encoded address to obtain a decoded address.

In Fig. 3, for convenience of understanding, the encoder (13) and decoder (14) are depicted as separate configurations from the memory controller (12), but the encoder (13) and decoder (14) may be configured to be included in the memory controller (12).

And the DNN distribution module (20) provides the weight of the DNN learned by the memory controller (12). The DNN distribution module (20) may be an external device separately provided outside the memory device (10). For example, the DNN distribution module (20) may be implemented as various storage media (not shown) in which the weight of the learned DNN is stored, or may be implemented as a communication module (not shown) that transmits the weight of the DNN to the memory controller (12).

In the above, it has been described that the memory controller (12) acquires the auxiliary bit, but in most cases, the memory device (10) is used together with a processor (not shown), and in this case, the auxiliary bit may be acquired from a processor (not shown), which is an external device of the memory device (10), and transmitted to the memory controller (12). In addition, the weight may be applied from the DNN distribution module (20) to the memory controller through the processor, and the address mapped to the weight may also be configured to be applied from the processor.

Figure 4 is a diagram for explaining address remapping according to auxiliary bits, showing the result of remapping an address mapped to a number of weights (W ₀ to W ₁₅ ) according to the values of various auxiliary bits into an encoding address.

In Fig. 4, the initial address represents an address mapped to a number of weights (W ₀ to W ₁₅ ), the value displayed with XOR represents the value of the auxiliary bit, and the remapped address represents an encoding address. Here, for the convenience of explanation, only the lower 4 bits of the internal block address in the address and the encoding address are displayed, and the auxiliary bits are composed of 4 bits according to the 4-bit internal block address. In addition, the number of weights (W ₀ to W ₁₅ ) are weights to be stored in the same memory block according to the mapped address, and here, the case where 16 weights are stored in one memory block is illustrated. In addition, it is assumed that the number of bit values of each weight (W ₀ to W ₁₅ ) are stored in a group of memory cells selected together by the same address. For example, a weight composed of 8 bits can be stored in 8 memory cells selected by the same address by sharing the same address.

When the encoder (13) is not provided, 16 weights (W ₀ to W ₁₅ ) are stored in memory cell groups having addresses from "0000" to "1111" in an arrangement according to the internal block address, as illustrated in FIG. 4. However, as described above, the encoder (13) encodes the internal block address and auxiliary bits of the address by an XOR operation, and remaps each weight (W ₀ to W ₁₅ ) to be stored in a memory cell according to the encoding address. At this time, the value of the auxiliary bit can be variously set from "0000" to "1111", and the storage location of each weight (W ₀ to W ₁₅ ) is changed to the remapped memory cell according to the value of the set auxiliary bit.

When the value of the auxiliary bit is "0000" as in (a), the encoding internal block address is the same as the internal block address. Therefore, the memory cell where the weight (W ₀ to W ₁₅ ) is stored does not change.

However, in the case where the value of the auxiliary bit is "0000", as in (p), the encoding internal block address appears as if the internal block address is inverted. Therefore, in the case of (p), the 16 weights (W) are stored in the arrangement order opposite to (a). In addition, in the case of (b), the value of the auxiliary bit is "0001". When the encoding address of (b) is compared with the encoding address of (a), it can be seen that the positions of the memory cells where the 16 weights (W ₀ to W ₁₅ ) are stored are interlaced by two. For example, assuming that the 16 weights (W ₀ to W ₁₅ ) are sequentially stored in the 0th to 15th memory cell groups before being remapped, the 0th weight (W ₀ ) is stored in the 1st memory cell group after being remapped to the encoding address, while the 1st weight (W ₁ ) is stored in the 0th memory cell group. In other words, the stored memory cells are interlaced.

Therefore, it can be seen that the memory cell in which each weight (W ₀ to W ₁₅ ) is stored can be changed by adjusting the value of the auxiliary bit encoding the internal block address.

Figure 5 is a diagram illustrating an example of a technique for improving the reliability of a non-volatile memory device for DNN.

In Fig. 5, a case is illustrated where four weights (1010, 0001, 0010, 1100) are mapped to each internal block address (00, 01, 10, 11), and similarly to Fig. 2, a case is illustrated where a deadlock defect occurs in five memory cells out of 16 memory cells. In addition, four memory cells form a memory cell group and are selected by the same internal block address (00, 01, 10, 11).

In the first and second groups divided into four memory cell units in Fig. 5, a sticking defect having sticking values of 0 and 1 occurred in the third memory cell and the first memory cell, respectively, and in the third group, a sticking defect having sticking values of 0 and 1 occurred in the second and third memory cells, respectively. And in the fourth group, the fourth memory cell had a sticking value of 1.

Therefore, if the internal block address mapped to the four weights (W) is not remapped, a bit value error occurs in all memory cells where a stuck-at fault occurs, resulting in errors in five bits and data errors in all four weights (W).

However, when the internal block address mapped to the four weights (W) is encoded with an auxiliary bit having a value of 01 and an XOR operation and re-fabricated as an encoded internal block address, the bit value of the weight (W) stored in the first to third groups and the sticky value of the memory cell where the sticky defect occurs match the same value, so no bit value error occurs. Although in the fourth group, despite the remapping, the bit value of the sticky value and the bit value of the weight still do not match, so one bit value error still remains, but compared to before the remapping, the bit value error is reduced from five to one, and since a data error occurs in only one of the four weights, it can be seen that the error of the memory device is significantly reduced. As a result, the reliability of the memory device can be greatly improved. In addition, considering the characteristics of the DNN, unlike other data, even if there is a data error in some weights in the DNN, the performance of the DNN may not be greatly degraded. Therefore, even if the memory device includes a memory cell with a sticky defect, the memory device can be used for the DNN.

Figure 6 is a diagram for explaining weight deviation according to error bit position.

In the above, it was explained that auxiliary bits are set simply to reduce the number of bit value errors that occur. However, in quantized data such as the weights of DNN, the bit position has a greater influence than the bit value of each bit.

In Fig. 6, a case is illustrated where a weight having a value of 179 is quantized to 8 bits and has a bit value of "10110011". In addition, if the bit values of the last 3 bit positions of such a weight are stored in a memory cell where a sticky defect occurs and a bit value error occurs, the number of bit value errors is 3, but the value is 180, so the deviation from the actual weight is 1. In contrast, if a sticky defect occurs at the 3rd bit position, the number of bit value errors is only 1, but the value is 147, so the deviation from the actual weight is 32.

In other words, the bit position where the bit value error occurs has a greater effect on the actual weight than the number of errors in the bit value. In particular, the bit value error that occurs in the upper bit position has a greater effect than the multiple bit value errors in the lower bit position. Therefore, the auxiliary bit should be set to reduce the bit value error in the upper bit position rather than to reduce the number of errors in the bit value.

Figure 7 is a diagram illustrating another example of a technique for improving the reliability of a non-volatile memory device for DNN.

In Fig. 7, as in Fig. 5, the same deadlock fault occurred in 5 memory cells out of 16 memory cells in which 4 memory cells were selected as a group by internal block addresses (00, 01, 10, 11). In addition, the case of storing 4 weighted values (W ₁ = 0111, W ₂ = 0101, W ₃ = 1011, W ₄ = 0110) quantized into 4 bits is illustrated.

Accordingly, when the internal block address (00, 01, 10, 11) mapped to four weights (W ₁ , W ₂ , W ₃ , W ₄ ) is not remapped or is remapped and stored by XOR operation with an auxiliary bit having a value of 00, the number of bit errors that occur is 5, and deviations of 2, 8, 2, and 1 occur in the four weights, respectively, resulting in a total deviation of 13 (= 2+8+2+1).

However, as in (a), when the internal block address (00, 01, 10, 11) is remapped by an XOR operation with an auxiliary bit having a value of 01, the second weight (W ₂ ) among the four weights (W ₁ , W ₂ , W ₃ , W ₄ ) is stored in the first memory group, the first weight (W ₁ ) is stored in the second memory group, the third weight (W ₃ ) is stored in the fourth memory group, and the fourth weight (W ₄ ) is stored in the third memory group. Therefore, bit value errors occur in only two bits of the first and fourth weights (W ₁ , W ₄ ) stored in the second and third memory groups, and the deviation becomes 10 (=8+2).

And as in (b), when the internal block address (00, 01, 10, 11) is remapped by an XOR operation with an auxiliary bit having a value of 10, the third and fourth weights (W ₃ , W ₄ ) among the four weights (W ₁ , W ₂ , W ₃ , W ₄ ) are stored in the first and second memory groups, and the first and second weights (W ₁ , W ₂ ) are stored in the third and fourth memory groups, respectively. Therefore, bit value errors occur in three bits in the first, third, and fourth weights (W ₁ , W ₃ , W ₄ ) stored in the first to third memory groups, and the deviation becomes 12 (=2+8+2).

Finally, as in (c), when the internal block address (00, 01, 10, 11) is remapped by an XOR operation with an auxiliary bit having a value of 11, the four weights (W ₁ , W ₂ , W ₃ , W ₄ ) are stored in the first to fourth memory groups in reverse order. Accordingly, a bit value error occurs in one bit of the fourth weight (W ₄ ) stored in the first memory group, and the deviation becomes 2.

Accordingly, the auxiliary bit can be set to have a value of 11, which minimizes the deviation.

As described above, in the embodiment, the number of bit errors and the deviation can be checked while changing the auxiliary bits to all possible values, and the auxiliary bit that minimizes the deviation can be preferentially selected, and if there are multiple auxiliary bits that minimize the deviation, the auxiliary bit that minimizes the number of bit errors can be finally selected and set.

In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below. Furthermore, in one embodiment, each component may be implemented using one or more physically separate devices, or may be implemented by one or more processors, or a combination of one or more processors and software, and may not be clearly distinguished in specific operations, unlike the illustrated example.

And the memory device illustrated in FIG. 3 can be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, and can also be implemented using a general-purpose or special-purpose computer. The device can be implemented using a hardwired device, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. In addition, the device can be implemented as a system on chip (SoC) including one or more processors and controllers.

In addition, the memory device may be installed in a computing device or server equipped with hardware elements in the form of software, hardware, or a combination thereof. The computing device or server may refer to various devices including all or part of a communication device such as a communication modem for communicating with various devices or wired/wireless communication networks, a memory for storing data for executing a program, and a microprocessor for executing a program to perform calculations and commands.

FIG. 8 illustrates a method for improving the reliability of a memory device for a DNN according to one embodiment.

Referring to FIGS. 2 to 7, a method for improving the reliability of a memory device for the DNN of FIG. 8 will be described. First, a memory cell in which a sticking defect has occurred among a plurality of memory cells of a cell array (11) and a sticking value of the corresponding memory cell are determined (31). In order to determine a memory cell in which a sticking defect has occurred, a bit value of 0 is stored in all memory cells of the cell array (11), the stored bit value is read to check a memory cell in which a bit value of 1 has been output, and a bit value of 1 is stored in all memory cells again, the stored bit value is read to check a memory cell in which a bit value of 0 has been output, thereby determining a memory cell in which a sticking defect has occurred and its sticking value. At this time, the location according to the address of the sticking defect memory cell in which the sticking defect has occurred is also determined.

And a plurality of weights included in each of a plurality of layers constituting the DNN are obtained (32). Here, the weights are quantized into multiple bits and applied, and can be mapped to an address indicating a memory cell to be stored and applied.

When the stuck-on fault memory cell and the weight are obtained, the encoded address is obtained by varying the value of the auxiliary bit in the address mapped to the weight to all possible bit values, and encoding the address based on the varied auxiliary bit (33). Here, the auxiliary bit can have a number of bits corresponding to the number of bits of the internal block address in the mapped address, and the address can be encoded by an XOR operation with the auxiliary bit.

Then, the location of the changed memory cell where the weight is stored is checked according to the encoding address obtained by each of the variable auxiliary bits, and each bit of the weight is compared with the sticking value of the checked memory cell (34).

As a result of the comparison, the number of bit value errors that occur due to the error between the bit value of the weight and the fixed value of the memory cell according to the variable auxiliary bit and the deviation between the weight stored in the memory cell and the actual weight due to the bit value error are calculated (35).

When the number of bit value errors and deviations per auxiliary bit are calculated, the auxiliary bit indicating the minimum deviation among the calculated deviations is detected (36). Then, it is determined whether there are multiple auxiliary bits detected as indicating the same minimum deviation (37). If multiple auxiliary bits indicating the minimum deviations are detected, the auxiliary bit that minimizes the number of bit value errors is detected (38). Then, the detected auxiliary bit is set as an auxiliary bit for encoding an address mapped to an applied weight (39). When the auxiliary bit is set, the address mapped to the weight is encoded using the set auxiliary bit, and the obtained encoding address is remapped to the weight (40). Then, the weight is stored in a memory cell selected according to the remapped encoding address (41).

Although not illustrated, the weights stored in the memory cells by being remapped to the encoding address can be output by being mapped to the address by decoding them again with an XOR operation using the auxiliary bits for the encoding address at read time.

In the above method for improving the reliability of a memory device, the auxiliary bit may be set by the memory controller (12). However, the memory controller (12) may transmit the weight and the location of the stuck-on fault memory cell to the processor, and the processor (not shown) may detect the auxiliary bit that minimizes the deviation and the number of bit errors and transmit it to the memory controller (12).

Although FIG. 8 describes each process as being executed sequentially, this is merely an example, and those skilled in the art may change the order described in FIG. 8 without departing from the essential characteristics of the embodiments of the present invention, or may modify and change the order described in FIG. 8 in various ways, such as executing one or more processes in parallel, or adding other processes.

Although the present invention has been described in detail above through representative examples, those skilled in the art will understand that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical idea of the appended claims.

10: Memory device 11: Cell array
12: Memory controller 13: Encoder
14: Decoder 20: DNN deployment module

Claims (20)

A cell array having a plurality of memory cells; and
A memory controller is included, which receives a plurality of weights each mapped to an address designating a memory cell to be stored, encodes the address based on auxiliary bits composed of a plurality of bits, remaps the obtained encoding address to the weights, and stores the weights in a memory cell selected according to the encoding address among the plurality of memory cells.
A memory device for a DNN in which the auxiliary bit is changed so that the memory cell in which the weight is stored is selected according to the encoding address, and the stored value among the plurality of memory cells is fixed as a stuck value, so that the stuck value of a stuck-on defective memory cell in which a stuck-on defect occurs matches the bit value of the weight. In the first paragraph, the memory controller
A memory device for a DNN that obtains the encoded address by performing a bitwise XOR operation on the above address and the auxiliary bit. In the first paragraph, the auxiliary bit is
A memory device for a DNN, wherein the number of bit value errors that occur because the bit values of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and the stickiness value of the stickiness fault memory cell do not match as the same value is set to a minimum. In the first paragraph, the auxiliary bit is
A memory device for a DNN, wherein the deviation between the values stored in the memory cells and the weights is set to be minimal due to a bit value error that occurs when the bit value of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and the stickiness value of the stickiness defective memory cell do not match as the same value. In the fourth paragraph, the auxiliary bit is
A memory device for a DNN, wherein a smaller number of bit value errors occur when there are a plurality of auxiliary bits having a minimum deviation between the value stored in the memory cell and the weight. In the first paragraph, the plurality of memory cells
The stored data is divided into memory blocks of a size that is read in batches.
The above auxiliary bits are
A memory device for a DNN having a number of bits corresponding to the number of bits of an internal block address for distinguishing and selecting a number of memories included in the memory block among a number of bits of the above address. In the first paragraph, the memory controller
A memory device for a DNN that detects an address of a memory cell in which a stuck-in defect has occurred, wherein a stored value is fixed among the plurality of memory cells, and determines the stuck-in value of the detected memory cell. In the first paragraph, the memory controller
A memory device for a DNN that decodes the encoding address remapped to the weight based on the auxiliary bit and remaps it to the address when reading the weight stored in the memory cell selected according to the encoding address. In the first paragraph, the plurality of memory cells
A memory device for DNN implemented with either a phase change memory (PCM) device or a resistive random-access memory (ReRAM) device. In the first paragraph, the plurality of weights are
A memory device for a deep neural network (DNN) that is included in each of the multiple layers and used for neural network operations and has values that are predetermined through learning. A method for improving the reliability of a memory device including a cell array having a plurality of memory cells and a memory controller,
A step of obtaining a plurality of weights, each mapped to an address specifying a memory cell to be stored;
A step of remapping the encoded address obtained by encoding the address based on auxiliary bits composed of a plurality of bits to the weight; and
A step of storing the weight in a memory cell selected according to the encoding address among the plurality of memory cells,
A method for improving the reliability of a memory device for a DNN, wherein the auxiliary bit is changed to a memory cell selected according to the encoding address in which the memory cell in which the weight is stored is fixed as a stuck value among the plurality of memory cells, and the stuck value of a stuck-on defective memory cell in which a stuck-on defect has occurred matches the bit value of the weight. In the 11th paragraph, the remapping step
A method for improving the reliability of a memory device for a DNN that obtains the encoded address by performing a bitwise XOR operation on the above address and the auxiliary bit. In the 11th paragraph, the auxiliary bit is
A method for improving the reliability of a memory device for a DNN, wherein the number of bit value errors that occur because the bit values of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and the stickiness value of the stickiness defective memory cell do not match as the same value is set to a minimum. In the 11th paragraph, the auxiliary bit is
A method for improving the reliability of a memory device for a DNN, wherein the deviation between the values stored in the memory cells and the weights is set to be minimal due to a bit value error that occurs when the bit value of each of the plurality of weights stored in the plurality of memory cells selected according to the encoding address and the stickiness value of the stickiness defective memory cell do not match as the same value. In the 14th paragraph, the auxiliary bit is
A method for improving the reliability of a memory device for a DNN, wherein a smaller number of bit value errors occur when there are a plurality of auxiliary bits having a minimum deviation between the value stored in the memory cell and the weight. In the 11th paragraph, the plurality of memory cells
The stored data is divided into memory blocks of a size that is read in batches.
The above auxiliary bits are
A method for improving the reliability of a memory device for a DNN having a number of bits corresponding to the number of bits of an internal block address for distinguishing and selecting a number of memories included in the memory block among a number of bits of the above address. In the 11th paragraph, the reliability improvement method
A method for improving the reliability of a memory device for a DNN, further comprising, prior to the step of obtaining the weights, the step of detecting an address of a memory cell in which a stuck-in defect has occurred, wherein a stored value is fixed among the plurality of memory cells, and determining the stuck-in value of the detected memory cell. In the 11th paragraph, the reliability improvement method
A method for improving the reliability of a memory device for a DNN, wherein after the step of storing the weights, the weights stored in the selected memory cells according to the encoding address are checked, and the checked weights are remapped to the encoding address by decoding the remapped weights based on the auxiliary bits. In the 11th paragraph, the plurality of memory cells
A method for improving the reliability of a memory device for DNNs implemented with either PCM devices or ReRAM devices. In the 11th paragraph, the plurality of weights are
A method for improving the reliability of a non-volatile memory device for a DNN, which is included in each of a plurality of layers of the DNN and used for neural network operations and has values determined in advance by learning.