Winograd convolution operation method, apparatus, and device, and storage medium

[0001] This application is a national stage filing of International Application No. PCT/CN2020/113155, filed Sep. 3, 2020, which claims priority to Chinese patent application No. 201911061091.9 filed on Nov. 1, 2019 and entitled “WINOGRAD CONVOLUTION OPERATION METHOD, APPARATUS, AND DEVICE, AND STORAGE MEDIUM”. The content of the applications are herein incorporated by reference in their entirety.

[0002] This disclosure relates to the technical field of artificial intelligence and in particular relates to a kind of winograd convolution operation method, apparatus, device, and storage medium.

[0003] With the development of artificial intelligence technologies, a convolution neural network model has emerged. The convolution neural network model is a kind of feed-forward neural network model that includes convolution calculations and a deep structure, and the convolution neural network model is one of representative models of deep learning. In the convolution neural network model, convolution operations on neurons and convolution kernels are required.

[0004] In the prior art, if the convolution operations are implemented by hardware, a large number of multipliers are required to be used on a chip. Since during an implementation process of hardware, overheads brought by multiplication implementations are much higher than that of addition implementations in terms of timing, power consumption and area, processing efficiency of the chip may be greatly reduced if the convolution operations are implemented by the hardware in the prior art.

[0005] Based on this, in order to solve the technical problem above, the present disclosure provides a kind of winograd convolution operation method, apparatus, device, and storage medium.

[0006] A first aspect of embodiments of the present disclosure provides a winograd convolution operation method. The method may be applied to winograd convolution transformations, and the method may include:

• • [0007] splitting data in a winograd convolution operation into a plurality of sub-tensors during a transformation process of the winograd convolution operation;
  • [0008] transforming the plurality of sub-tensors to obtain winograd transformation results of the plurality of sub-tensors and summing the winograd transformation results of the plurality of sub-tensors to obtain a winograd transformation result of the data, where a summation operation of the winograd transformation results of the plurality of sub-tensors is completed by a plurality of operation sub-apparatuses based on set strategies; and
  • [0009] completing the winograd convolution operation according to the winograd transformation result of the data.

[0010] A second aspect of embodiments of the present disclosure provides a winograd convolution operation apparatus, comprising:

• • [0011] a splitting unit configured to split data in a winograd convolution operation into a plurality of sub-tensors during a transformation process of the winograd convolution operation;
  • [0012] a transformation and summation operation unit configured to transform the plurality of sub-tensors to obtain winograd transformation results of the plurality of sub-tensors and sum the winograd transformation results of the plurality of sub-tensors to obtain a winograd transformation result of the data, where a summation operation of the winograd transformation results of the plurality of sub-tensors is completed by a plurality of operation sub-apparatuses based on set strategies; and a convolution operation unit configured to complete the winograd convolution operation according to the winograd transformation result of the data.

[0013] A third aspect of embodiments of the present disclosure provides a winograd convolution operation apparatus, including processors and a memory; where

• • [0014] the memory is configured to store a program code; and
  • [0015] the processors are configured to call the program code stored in the memory and execute the method of the first aspect.

[0016] A fourth aspect of embodiments of the present disclosure provides a computer-readable storage medium, on which an instruction is stored, where when run on a computer, the instruction enables the computer to execute the method of the first aspect.

[0017] A fifth aspect of embodiments of the present disclosure provides an artificial intelligence chip, including the apparatus of the second aspect or the third aspect.

[0018] A sixth aspect of embodiments of the present disclosure provides an electronic device, including the artificial intelligence chip of the fifth aspect.

[0019] A seventh aspect of embodiments of the present disclosure provides a board card, including a storage component, an interface apparatus, a control component and the artificial intelligence chip of the fifth aspect, where

• • [0020] the artificial intelligence chip is connected to the storage component, the control component, and the interface apparatus, respectively;
  • [0021] the storage component is configured to store data;
  • [0022] the interface apparatus is configured to implement data transfer between the artificial intelligence chip and an external device; and
  • [0023] the control component is configured to monitor a state of the artificial intelligence chip.

[0024] Further, in the board card above, the storage component may include: a plurality of groups of storage units, where each of the groups of storage units is connected to the artificial intelligence chip through a bus, and a storage unit is a double data rate (DDR) synchronous dynamic random access memory (SDRAM);

• • [0025] the chip may include: a DDR controller configured to control data transfer and data storage of each storage unit; and
  • [0026] the interface apparatus is a standard peripheral component interconnect express (PCIe) interface.

[0027] The embodiments of the present disclosure provides a winograd convolution operation method, a winograd convolution operation apparatus, a device and a storage medium. According to the method, data in a winograd convolution operation is split into a plurality of sub-tensors; the plurality of sub-tensors are transformed to obtain winograd transformation results of the plurality of sub-tensors, and the winograd transformation results of the plurality of sub-tensors are summed to obtain a winograd transformation result of the data, where a summation operation of the winograd transformation results of the plurality of sub-tensors is completed by a plurality of operation sub-apparatuses based on set strategies; and the winograd convolution operation is completed according to the winograd transformation result of the data. Since an original transformation process of the winograd convolution operation is changed from matrix multiplications to matrix summation operations, overheads brought by the multiplications may be reduced. In addition, since during the summation operation, the plurality of operation sub-apparatuses are used for processing, processing efficiency of a chip may be improved.

[0038] Technical solutions in embodiments of the present disclosure will be described clearly and completely hereinafter with reference to drawings in the embodiments of the present disclosure. Obviously, the embodiments to be described are merely some of, but not all of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.

[0039] It should be understood that terms such as “first”, “second”, “third” and “fourth” in the claims, the specification, and the drawings are used for distinguishing different objects rather than describing a specific order. It should be understood that terms “including” and “comprising” used in the specification and the claims indicate the presence of a feature, an entity, a step, an operation, an element, and/or a component, but do not exclude the existence or addition of one or more other features, entities, steps, operations, elements, components, and/or collections thereof.

[0040] It should also be understood that terms used in the specification of the present disclosure are merely for a purpose of describing a particular embodiment rather than limiting the present disclosure. As being used in the specification and the claims of the disclosure, unless the context clearly indicates other conditions, singular forms “a”, “an” and “the” are intended to include plural forms. It should also be understood that a term “and/or” used in the specification and the claims refers to any and all possible combinations of one or more of relevant listed items and includes these combinations.

[0041] As being used in this specification and the claims, a term “if” may be interpreted as “when”, or “once” or “in response to a determination” or “in response to a case where something is detected” depending on the context. By analogy, depending on the context, a clause “if it is determined that” or “if [a described condition or event] is detected” may be interpreted as “once it is determined that”, or “in response to a determination”, or “once [a described condition or event] is detected”, or “in response to a case where [a described condition or event] is detected”.

[0042] In order to make the technical solutions of the present disclosure to be clearly understood, technical terms involved in the prior art and the embodiments of the present disclosure are explained hereinafter.

[0043] Convolution operation: a convolution operation is to open an active window with the same size as a template starting from the upper left corner of an image, where the active window corresponds to a window image which is a convolution kernel, and multiply and then add the window image with corresponding pixels in the image, and use a calculation result as a first pixel value of a new image after the convolution operation. Then, the active window is moved one column to the right and the same operation is performed. And so on, from left to right and from top to bottom, the new image may be obtained.

[0044] Winograd convolution operation: a winograd convolution operation is a kind of convolution acceleration implementation based on a polynomial interpolation algorithm. During the winograd convolution operation, winograd convolution forward transformations are performed respectively on two inputs of a convolution operation, including a first target matrix and a second target matrix, and then the first target matrix after the forward transformation is multiplied with the second target matrix after the forward transformation in an element-wise manner, and finally a winograd convolution inverse transformation is performed on an element-wise multiplication result again to obtain a convolution result equivalent to an original convolution operation.

[0045] Convolution neural network model: a convolution neural network model is a kind of feed-forward neural network model that includes convolution calculations and a deep structure, and is one of representative models of deep learning. In network layers of the convolution neural network model, such as a convolution layer and a fully-connected layer, convolution operations on neurons and convolution kernels are required to be performed to obtain feature data. The convolution neural network model is widely applied to image classification, image identification, and the like

[0046] Since matrix multiplications are still performed in the winograd convolution forward transformation and the winograd convolution inverse transformation, there are large overheads during a hardware implementation. Therefore, in order to further improve processing efficiency, the embodiments of the present disclosure provide a winograd convolution operation method. The winograd convolution operation method may be used in the hardware implementation of a convolution neural network.

[0047] Specifically, the winograd convolution operation method of the embodiments of the present disclosure may be used in various scenarios where the convolution neural network is used for identification, classification and prediction. For example, the winograd convolution operation method may be used in fields of autonomous driving, robotics, smart home, office management, and the like. FIG. 1 shows an application scenario of a winograd convolution operation method, according to the present disclosure. In the application scenario, an electronic device is equipped with a vision sensor and a convolution neural network chip. The vision sensor collects human feature data of a user, such as face data, iris data, and fingerprint data, and sends the human feature data to the electronic device. The electronic device identifies the human feature data through the convolution neural network chip. If the user is identified as a legitimate user, an access door will be opened; otherwise the access door will keep closed. Since the winograd convolution operation method of the embodiments of the present disclosure is used in the convolution neural network chip, data processing efficiency of the convolution neural network chip may be improved and the identification, classification and prediction may be performed quickly.

[0048] The embodiments of the present disclosure are described hereinafter with reference to the drawings.

[0049] FIG. 2 is a flowchart of a winograd convolution operation method, according to a first embodiment of the present disclosure. As shown in FIG. 2, an execution subject of the embodiment is a winograd convolution operation apparatus, and the winograd convolution operation method of this embodiment includes the following steps.

[0050] In a step 101, during a transformation process of a winograd convolution operation, data in the winograd convolution operation is split into a plurality of sub-tensors.

[0051] A transformation type of the winograd convolution operation includes a winograd forward transformation and a winograd inverse transformation. If the transformation type of the winograd convolution operation is the winograd forward transformation, the data in the winograd convolution operation is neuron data or weight data, where the neuron data may be a neuron matrix and the weight data may be a weight matrix. If the transformation type of the winograd convolution operation is the winograd inverse transformation, the data in the winograd convolution operation is transformed element-wise multiplication data, where the transformed element-wise multiplication data is a transformed element-wise multiplication matrix, and the transformed element-wise multiplication matrix is a matrix formed after a result matrix after the neuron matrix is transformed forward is multiplied with a result matrix after the weight matrix is transformed forward in an element-wise manner.

[0052] The result matrix after the neuron matrix is transformed forward may be called a neuron-transformed matrix. The result matrix after the weight matrix is transformed forward may be called a weight-transformed matrix.

[0053] If the data is one-dimensional data, the winograd convolution operation may be expressed as F=A^T((Gg)⊙(B^Td)), and if the data is two-dimensional data, the winograd convolution operation may be expressed as F=A^T((GgG^T)⊙(B^TdB))A

[0054] If the data is the one-dimensional data, F is a convolution output matrix, and d is first data, and B^r is a transformation matrix of the winograd convolution forward transformation of the d, and g is second data, and G is the transformation matrix of the winograd convolution forward transformation of the g. Additionally, (Gg)⊙(B^Td) is data formed after the first data after the forward transformation is multiplied with the second data after the forward transformation in the element-wise manner, which may be expressed as a transformed element-wise multiplication matrix and A^T is the transformation matrix of the winograd convolution inverse transformation of the transformed element-wise multiplication matrix.

[0055] If the data is the two-dimensional data, F is the convolution output matrix, and d is the first data, which may be the neuron matrix, and B^T is a left multiplication matrix of the winograd convolution forward transformation of the d, and B is a right multiplication matrix of the winograd convolution forward transformation of the d, and g is the second data, which may be the weight matrix, and G is the left multiplication matrix of the winograd convolution forward transformation of the g, and G^T is the right multiplication matrix of the winograd convolution forward transformation of the g Additionally, (GgG^T)⊙(B^TdB) is the matrix formed after the first data after the forward transformation is multiplied with the second data after the forward transformation in the element-wise manner, which may be called the transformed element-wise multiplication matrix, and A^T is the left multiplication matrix of the winograd convolution inverse transformation of the transformed element-wise multiplication matrix, and A is the right multiplication matrix of the winograd convolution inversion of the transformed element-wise multiplication matrix.

[0056] In this embodiment, the winograd convolution operation may include the winograd forward transformation and the winograd inverse transformation. And if the transformation type of the winograd convolution operation is the winograd forward transformation in this embodiment, the winograd forward transformation may be both the winograd forward transformation of the neuron matrix and the winograd forward transformation of the weight matrix. Moreover, this embodiment does not restrict a size of the data in the winograd convolution operation.

[0057] In this embodiment, if the data in the winograd convolution operation is split into the plurality of sub-tensors, the data may be parsed as a sum of the plurality of sub-tensors.

[0058] For example, the data may be represented as

[0059] $\left[\begin{array}{cccc}{d}_{00}& d_{01}& \mathrm{\dots \dots }& d_{0Y}\\ d_{10}& d_{11}& \mathrm{\dots \dots }& d_{1Y}\\ \mathrm{\dots \dots }& \mathrm{\dots \dots }& \mathrm{\dots \dots }& \mathrm{\dots \dots }\\ d_{X0}& d_{X1}& \mathrm{\dots \dots }& d_{\mathrm{XY}}\end{array}\right],$
and then the plurality of sub-tensors obtained after the data is split may be represented as follows:

[0060] $\left[\begin{array}{cccc}{d}_{00}& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& d_{01}& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots ,$ $\left[\begin{array}{cccc}0& 0& \dots \dots & d_{0Y}\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ d_{10}& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& d_{11}& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & d_{1Y}\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots \dots ,\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ d_{X0}& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& d_{X1}& 0& 0\end{array}\right],\dots \dots ,$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& d_{\mathrm{XY}}\end{array}\right].$

[0061] In a step 102, the plurality of sub-tensors are transformed to obtain the winograd transformation results of the plurality of sub-tensors, and the winograd transformation results of the plurality of sub-tensors are summed to obtain a winograd transformation result of the data, where a summation operation of the winograd transformation results of the plurality of sub-tensors is completed by a plurality of operation sub-apparatuses based on set strategies.

[0062] Specifically, in this embodiment, meta-tensors corresponding to the plurality of sub-tensors are obtained first and then winograd transformations of the meta-tensors are performed, and the winograd transformation results of the sub-tensors are determined according to the sub-tensors and the winograd transformation results of the meta-tensors. Then the summation operation of the winograd transformation results of the plurality of sub-tensors is completed by the plurality of operation sub-apparatuses based on the set strategies, and the winograd transformation result of the data is obtained.

[0063] The set strategies may be a load-balancing strategy. According to the strategy, if the summation operation of the winograd transformation results of the plurality of sub-tensors is performed by the plurality of operation sub-apparatuses, each operation sub-apparatus undertakes balanced addition execution tasks of elements in the result matrix.

[0064] In this embodiment, since the summation operation of the winograd transformation results of the plurality of sub-tensors is performed by the operation sub-apparatuses, the operation sub-apparatuses may be adders.

[0065] In a step 103, the winograd convolution operation is completed based on the winograd transformation result of the data.

[0066] In this embodiment, if the data is the two-dimensional data, the transformation result of the neuron matrix is determined first, and then the transformation result of the weight matrix is determined. Then, after the transformation result of the neuron matrix is multiplied with the transformation result of the weight matrix in the element-wise manner to obtain the transformed element-wise multiplication matrix, the transformation result of the element-wise multiplication matrix is determined, so as to complete the winograd convolution operation.

[0067] According to the winograd convolution operation method of this embodiment, the data in the winograd convolution operation is split into the plurality of sub-tensors; the plurality of sub-tensors are transformed to obtain the winograd transformation results of the plurality of sub-tensors, and the winograd transformation results of the plurality of sub-tensors are summed to obtain the winograd transformation result of the data, where the summation operation of the winograd transformation results of the plurality of sub-tensors is completed by the plurality of operation sub-apparatuses based on the set strategies; and the winograd convolution operation is completed according to the winograd transformation result of the data. Since an original transformation process of the winograd convolution operation is changed from a matrix multiplication to a matrix summation operation, overheads brought by the multiplication may be reduced. In addition, since the plurality of operation sub-apparatuses are involved in processing simultaneously during the summation operation, processing efficiency of a chip may be improved.

[0068] FIG. 3 is a flowchart of a winograd convolution operation method, according to a second embodiment of the present disclosure. As shown in FIG. 3, the winograd convolution operation method of this embodiment is a further refinement of the steps 101 to 103 based on the embodiment shown in FIG. 2. Then the winograd convolution operation method of this embodiment includes the following steps.

[0069] In a step 201, during a transformation process of a winograd convolution operation, data in the winograd convolution operation is split into a plurality of sub-tensors.

[0070] Further, in this embodiment, splitting the data in the winograd convolution operation into the plurality of sub-tensors includes:

• • [0071] parsing the data to obtain the plurality of sub-tensors, where the data is a sum of the plurality of sub-tensors, and the number of the plurality of sub-tensors is the same as the number of non-zero elements in the data, and each sub-tensor has a single non-zero element, and the non-zero elements in the sub-tensors are the same as the non-zero elements in corresponding positions in the data.

[0072] Specifically, assuming that the data is represented as

[0073] $\left[\begin{array}{cccc}{d}_{00}& d_{01}& \dots \dots & d_{0Y}\\ d_{10}& d_{11}& \dots \dots & d_{1Y}\\ \dots \dots & \dots \dots & \dots \dots & \dots \dots \\ d_{X0}& d_{X1}& \dots \dots & d_{\mathrm{XY}}\end{array}\right],$
if each element in the data is a non-zero element, the plurality of sub-tensors obtained by splitting the data may be represented as follows:

[0074] $\left[\begin{array}{cccc}{d}_{00}& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& d_{01}& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots \dots ,\left[\begin{array}{cccc}0& 0& \dots \dots & d_{0Y}\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ d_{10}& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& d_{11}& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & d_{1Y}\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots \dots ,\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ d_{X0}& 0& 0& 0\end{array}\right],$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& d_{X1}& 0& 0\end{array}\right],\dots \dots ,\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& d_{\mathrm{XY}}\end{array}\right].$

[0075] The number of the plurality of sub-tensors is X*Y, and the number of the non-zero elements in the data is also X*Y. In other words, the number of the plurality of sub-tensors is the same as the number of the non-zero elements in the data. Besides, from the plurality of sub-tensors, it may be shown that each sub-tensor has the single non-zero element, and the non-zero elements in the sub-tensors are the same as the non-zero elements in the corresponding positions in the data.

[0076] In a step 202, the plurality of sub-tensors are transformed to obtain winograd transformation results of the plurality of sub-tensors.

[0077] Further, in this embodiment, the step 202 includes the following steps.

[0078] In a step 2021, a winograd transformation result of a meta-tensor corresponding to each sub-tensor is obtained.

[0079] The meta-tensor is a tensor that sets the non-zero element of a sub-tensor as 1.

[0080] Specifically, corresponding to the plurality of sub-tensors illustrated in the step 201, corresponding meta-tensors are denoted as follows respectively:

[0081] $\left[\begin{array}{cccc}1& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 1& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots \dots ,$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 1\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 1& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 1& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 1\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right],\dots \dots ,\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 1& 0& 0& 0\end{array}\right],\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 1& 0& 0\end{array}\right],\dots \dots ,$ $\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 1\end{array}\right].$

[0082] As an optional implementation, in a step 2021, the winograd transformation result of the meta-tensor corresponding to each sub-tensor is obtained, where specifically,

• • [0083] for each sub-tensor, a left side of the meta-tensor corresponding to the sub-tensor is multiplied by a left multiplication matrix and a right side by a right multiplication matrix, so as to obtain the winograd transformation result of the meta-tensor.

[0084] Specifically, if the left multiplication matrix is denoted as and the right multiplication matrix is denoted as after the left side of the meta-tensor corresponding to each sub-tensor is multiplied by the left multiplication matrix and the right side by the right multiplication matrix, the obtained winograd transformation results of the meta-tensors may be denoted as follows respectively:

[0085] $B_{X\times X}^T\left[\begin{array}{cccc}1& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},B_{X\times X}^T\left[\begin{array}{cccc}0& 1& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},\dots \dots ,$ $B_{X\times Y}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 1\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 1& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},$ $B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 1& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 1\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},\dots \dots ,$ $B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 1& 0& 0& 0\end{array}\right]B_{Y\times Y},B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 1& 0& 0\end{array}\right]B_{Y\times Y},\dots \dots ,$ $B_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 1\end{array}\right]B_{Y\times Y}.$

[0086] Both the left multiplication matrix and the right multiplication matrix are determined by a size of the sub-tensors and a type of winograd transformation, where the type of winograd transformation includes a winograd forward transformation and a winograd inverse transformation. In other words, if the size of the sub-tensors and the type of winograd transformation are fixed, each element in the left multiplication matrix and the right multiplication matrix will also be fixed.

[0087] In a step 2022, a non-zero element value of the sub-tensor is set as a coefficient to be multiplied by the winograd transformation result of a corresponding meta-tensor to obtain a winograd transformation result of the sub-tensor.

[0088] Further, in this embodiment, each sub-tensor has the non-zero element value, and a corresponding non-zero element value is obtained from each sub-tensor, and the non-zero element value is multiplied by the winograd transformation result of the corresponding meta-tensor to obtain the winograd transformation result of the sub-tensor.

[0089] Then the winograd transformation results of the sub-tensors that are listed above may be expressed as follows:

[0090] $$\begin{gathered} d_{00}{B}_{X\times X}^T\left[\begin{array}{cccc}1& 0& \dots & 0\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},d_{01}{B}_{X\times X}^T\left[\left[\begin{array}{cccc}0& 1& \dots & 0\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]\right]B_{Y\times Y},\dots ,d_{0Y}{B}_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots & 1\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]]B_{Y\times Y},d_{10}{B}_{X\times X}^T\left[\left[\begin{array}{cccc}0& 0& \dots & 0\\ 1& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]\right]B_{Y\times Y}, \\ {d}_{11}{B}_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots & 0\\ 0& 1& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y},d_{1Y}{B}_{X\times X}^T\left[\left[\begin{array}{cccc}0& 0& \dots & 0\\ 0& 0& \dots & 1\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 0\end{array}\right]\right]B_{Y\times Y},\dots ,d_{X0}{B}_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots & 0\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 1& 0& 0& 0\end{array}\right]B_{Y\times Y},d_{X1}{B}_{X\times X}^T[\left[\begin{array}{cccc}0& 0& \dots & 0\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 1& 0& 0\end{array}\right]B_{Y\times Y},\dots ,d_{XY}{B}_{X\times X}^T\left[\begin{array}{cccc}0& 0& \dots & 0\\ 0& 0& \dots & 0\\ \dots & \dots & \dots & 0\\ 0& 0& 0& 1\end{array}\right]B_{Y\times Y}. \end{gathered}$$

[0091] The winograd transformation result of the data may be expressed as:

S=Wino{d}=B^T dB. Then summing the winograd transformation results of the plurality of sub-tensors to obtain the winograd transformation result of the data maybe expressed as shown in a formula (1):

[0092] $\begin{array}{cc}S=\mathrm{Wino}\left\{d\right\}=B^T\mathrm{dB}=d_{00}{B}_{X\times X}^T\left[\begin{array}{cccc}1& 0& \dots \dots & 0\\ 0& 0& \dots \dots & 0\\ \dots \dots & \dots \dots & \dots \dots & 0\\ 0& 0& 0& 0\end{array}\right]B_{Y\times Y}+\dots +d_{\mathrm{XY}}{B}_{X\times X}^T\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right]B_{Y\times Y}.& \left(1\right)\end{array}$

[0093] In a step 203, the winograd transformation results of the plurality of sub-tensors are summed to obtain the winograd transformation result of the data, and during a summation operation process of the winograd transformation results of the plurality of sub-tensors, the summation operation of the winograd transformation results of the plurality of sub-tensors is distributed to a plurality of operation sub-apparatuses for an execution based on a principle of balancing loads of the plurality of operation sub-apparatuses.

[0094] Further, in this embodiment, in the step 203, distributing the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for the execution based on the principle of balancing the loads of the plurality of operation sub-apparatuses includes the following steps.

[0095] In a step 2031, the number of the non-zero elements in a transformation matrix corresponding to each sub-tensor is determined.

[0096] Further, in this embodiment, a matrix formed by the winograd transformation result of each meta-tensor may be represented as the transformation matrix of a corresponding sub-tensor. Therefore, the number of the non-zero elements in the matrix corresponding to the winograd transformation result of the meta-tensor is determined as the number of the non-zero elements in the transformation matrix of the corresponding sub-tensor.

[0097] In a step 2032, an order of the winograd transformation results of the plurality of sub-tensors in the summation operation is determined according to the number of the non-zero elements.

[0098] Further, in this embodiment, in order to enable the plurality of operation sub-apparatuses to perform the summation operation of the winograd transformation results of the plurality of sub-tensors in a manner of load balancing, when the order of the winograd transformation results of the plurality of sub-tensors in the summation operation is determined, the winograd transformation results of the sub-tensors with the larger number of the non-zero elements should be matched with the winograd transformation results of the sub-tensors with the smaller number of the non-zero elements as much as possible for the summation operation, so that all the plurality of operation sub-apparatuses are involved in the summation operation within the same clock.

[0099] In a step 2033, addition execution tasks are distributed to the plurality of operation sub-apparatuses correspondingly according to the determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and a preset mapping relationship between element positions in the result matrix and operation sub-apparatuses.

[0100] The addition execution tasks are used to calculate each element in the result matrix.

[0101] Additive numbers in the addition execution tasks of the elements in the result matrix are composed of non-zero element values in data that is formed by the winograd transformation results of the plurality of sub-tensors.

[0102] The summation operation at the same element position in the result matrix is performed by the same operation sub-apparatus.

[0103] Further, in this embodiment, in order to enable the plurality of operation sub-apparatuses to complete addition calculations of the elements in the result matrix in the manner of load balancing, a mapping relationship between the element positions in the result matrix and the operation sub-apparatuses may be preset first based on the element positions of the non-zero elements of transformation matrixes in all the sub-tensors and the number of the operation sub-apparatuses. This mapping relationship indicates which operation sub-apparatus performs the summation operation at each element position in the result matrix. In other words, the same operation sub-apparatus performs the summation operation at the same element position in the result matrix. Then, according to the mapping relationship between the element positions in the result matrix and the operation sub-apparatuses and the determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation, the addition execution tasks used to calculate the elements in the result matrix are distributed to the operation sub-apparatuses correspondingly, so as to enable the operation sub-apparatuses to complete the addition calculations of the elements in the result matrix in the manner of load balancing.

[0104] In a step 204, according to the winograd transformation result of the data, the winograd convolution operation is completed.

[0105] In this embodiment, an implementation of the step 204 is similar to that of the step 103 in embodiment one, which will not be repeated here.

[0106] According to the winograd convolution operation method of this embodiment, during the summation operation process of the winograd transformation results of the plurality of sub-tensors, the summation operation of the winograd transformation results of the plurality of sub-tensors are distributed to the plurality of operation sub-apparatuses correspondingly based on the principle of balancing the loads of the plurality of operation sub-apparatuses. This method not only enables the plurality of operation sub-apparatuses to be involved in processing simultaneously during the summation operation, which improves processing efficiency of a chip, but also achieves the load balancing among the plurality of operation sub-apparatuses during the summation operation.

[0107] FIG. 4 is a flowchart of a winograd convolution operation method, according to a third embodiment of the present disclosure. As shown in FIG. 4, the winograd convolution operation method of this embodiment is a further refinement of the step 203 based on the embodiment shown in FIG. 3, and then the winograd convolution operation method of this embodiment includes the following steps.

[0108] In a step 301, during a transformation process of a winograd convolution operation, data in the winograd convolution operation is split into a plurality of sub-tensors.

[0109] In a step 302, the plurality of sub-tensors are transformed to obtain winograd transformation results of the plurality of sub-tensors.

[0110] In this embodiment, implementations of the step 301 and the step 302 are similar to those of the step 201 and the step 202 in the embodiment shown in FIG. 3, which will not be repeated here.

[0111] In a step 303, the number of addition executions required for a summation operation is determined according to the winograd transformation results of the plurality of sub-tensors.

[0112] Further, in this embodiment, since each operation sub-apparatus may perform one operation per clock during a hardware implementation of the winograd convolution operation, after a forward or an inverse transformation of the data in the winograd convolution operation is added and split according to a formula (1), the number of addition executions is much more than that of multiplication executions. In order to match addition execution tasks and multiplication execution tasks in term of computing power in the winograd convolution operation, the operation sub-apparatuses for multiplication tasks may also perform the matched multiplication tasks if the operation sub-apparatuses for addition tasks complete the addition tasks, and therefore, in this embodiment, it is necessary to determine the number of clocks required to calculate a summation result and determine the number of the addition executions required to calculate the summation result, so that the operation sub-apparatuses may complete the addition execution tasks within the number of the clocks.

[0113] Specifically, in this embodiment, transformation matrixes corresponding to the winograd transformation results of the plurality of sub-tensors include elements with zero element values and elements with non-zero element values. The summation operation of the winograd transformation results of the sub-tensors is related to the non-zero element values in each transformation matrix. Therefore, the number of the addition executions required to calculate the summation result may be determined according to the number of the non-zero element values in the transformation matrixes corresponding to the winograd transformation results of the plurality of sub-tensors and a size of a result matrix.

[0114] As an optional implementation, the step 303 includes the following steps.

[0115] In a step 3031, a non-zero element value at a same element position in data that is formed by a winograd transformation result of each sub-tensor is determined as an additive number of the element position in the result matrix.

[0116] In a step 3032, the number of summation operations at each element position is determined based on the additive number of each element position in the result matrix.

[0117] In a step 3033, a sum of the number of the summation operations at each element position is determined as the number of the addition executions required to calculate the summation result.

[0118] The data that is formed by the winograd transformation result of each sub-tensor may be in the form of a corresponding matrix.

[0119] Further, in this embodiment, since during the summation operation of the winograd transformation results of the plurality of sub-tensors, element values at the same position in a matrix that is formed by the winograd transformation result of each sub-tensor are summed to be used as a value of the element position in the result matrix, while in the matrix that is formed by the winograd transformation result of the sub-tensor, there are several element positions whose element values are zero, and therefore, the non-zero element value at the same element position in the matrix that is formed by the winograd transformation result of each sub-tensor may be determined as the additive number at the element position in the result matrix. However, since a difference between the number of additive numbers at each element position and the number of the summation operations performed at the element position is 1, a difference between the number of the additive numbers at each element position in the result matrix and 1 may be determined as the number of the summation operations at each element position. The sum of the number of the summation operations at all element positions may be determined as the number of the addition executions required to calculate the summation result.

[0120] In order to facilitate understanding, a size of data in the winograd convolution operation may be set as 4×4 as an example for explanation. For data d with a

size of 4×4, corresponding winograd transformation results of the plurality of sub-tensors may be expressed as shown in a formula (2). The matrix that is formed by the winograd transformation results of the plurality of sub-tensors may be expressed as shown in a formula (3). A size of a corresponding result matrix S is 4×4. In other words, the result matrix has 16 elements, which may be expressed as shown in a formula (4).

[0121] Then the corresponding winograd transformation results of the plurality of sub-tensors may be expressed as:

[0122] $\begin{array}{cc}{d}_{00}:d_{00}*\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{01}:d_{01}*\left[\begin{array}{cccc}0& 1& -1& 1\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]& \left(2\right).\end{array}$ $d_{10}:d_{10}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 1& 0& 0& 0\\ -1& 0& 0& 0\\ 1& 0& 0& 0\end{array}\right],d_{11}:d_{11}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 1& -1& 1\\ 0& -1& 1& -1\\ 0& 1& -1& 1\end{array}\right]$ $d_{02}:d_{02}*\left[\begin{array}{cccc}-1& 1& 1& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{03}:d_{03}*\left[\begin{array}{cccc}0& 0& 0& -1\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]$ $d_{12}:d_{12}*\left[\begin{array}{cccc}0& 0& 0& 0\\ -1& 1& 1& 0\\ 1& -1& -1& 0\\ -1& 1& 1& 0\end{array}\right],d_{13}:d_{13}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& -1\\ 0& 0& 0& 1\\ 0& 0& 0& -1\end{array}\right]$ $d_{S-20}:d_{S-20}*\left[\begin{array}{cccc}-1& 0& 0& 0\\ 1& 0& 0& 0\\ 1& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{21}:d_{21}*\left[\begin{array}{cccc}0& -1& 1& -1\\ 0& 1& -1& 1\\ 0& 1& -1& 1\\ 0& 0& 0& 0\end{array}\right]$ $d_{30}:d_{30}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ -1& 0& 0& 0\end{array}\right],d_{31}:d_{31}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& -1& 1& -1\end{array}\right]$ $d_{22}:d_{22}*\left[\begin{array}{cccc}1& -1& -1& 0\\ -1& 1& 1& 0\\ -1& 1& 1& 0\\ 0& 0& 0& 0\end{array}\right],d_{23}:d_{23}*\left[\begin{array}{cccc}0& 0& 0& 1\\ 0& 0& 0& -1\\ 0& 0& 0& -1\\ 0& 0& 0& 0\end{array}\right]$ $d_{32}:d_{32}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 1& -1& -1& 0\end{array}\right],d_{33}:d_{33}*\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\end{array}\right].$

[0123] The corresponding matrix that is formed by the winograd transformation results of the plurality of sub-tensors may be expressed as:

[0124] $d_{00}:\left[\begin{array}{cccc}{d}_{00}& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{01}:\left[\begin{array}{cccc}0& d_{01}& -d_{01}& d_{01}\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]d_{10}:\left[\begin{array}{cccc}0& 0& 0& 0\\ d_{10}& 0& 0& 0\\ -d_{10}& 0& 0& 0\\ d_{10}& 0& 0& 0\end{array}\right],d_{11}:\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& d_{11}& -d_{11}& d_{11}\\ 0& -d_{11}& d_{11}& -d_{11}\\ 0& d_{11}& -d_{11}& 1d_{11}\end{array}\right]d_{02}:\left[\begin{array}{cccc}-d_{02}& d_{02}& d_{02}& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{03}:\left[\begin{array}{cccc}0& 0& 0& -d_{03}\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]d_{12}:\left[\begin{array}{cccc}0& 0& 0& 0\\ -d_{12}& d_{12}& d_{12}& 0\\ d_{12}& -d_{12}& -d_{12}& 0\\ -d_{12}& d_{12}& d_{12}& 0\end{array}\right],d_{13}:\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& -d_{13}\\ 0& 0& 0& d_{13}\\ 0& 0& 0& -d_{13}\end{array}\right],$

[0125] $d_{20}\left[\begin{array}{cccc}-d_{20}& 0& 0& 0\\ d_{20}& 0& 0& 0\\ d_{20}& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right],d_{21}:\left[\begin{array}{cccc}0& -d_{21}& d_{21}& -d_{21}\\ 0& d_{21}& -d_{21}& d_{21}\\ 0& d_{21}& -d_{21}& d_{21}\\ 0& 0& 0& 0\end{array}\right]d_{30}:\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ -d_{30}& 0& 0& 0\end{array}\right],d_{31}:\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& -d_{31}& d_{31}& -d_{31}\end{array}\right]d_{22}:\left[\begin{array}{cccc}{d}_{22}& -d_{22}& -d_{22}& 0\\ -d_{22}& d_{22}& d_{22}& 0\\ -d_{22}& d_{22}& d_{22}& 0\\ 0& 0& 0& 0\end{array}\right],d_{23}:\left[\begin{array}{cccc}0& 0& 0& d_{23}\\ 0& 0& 0& -d_{23}\\ 0& 0& 0& -d_{23}\\ 0& 0& 0& 0\end{array}\right]$

[0126] $\begin{array}{cc}{d}_{32}\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ d_{32}& -d_{32}& -d_{32}& 0\end{array}\right],d_{33}:\left[\begin{array}{cccc}0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& d_{33}\end{array}\right]& \left(3\right)\end{array}$

[0127] The result matrix S may be expressed as:

[0128] $\begin{array}{cc}\left[\begin{array}{cccc}{S}_{00}& S_{01}& \dots \dots & S_{03}\\ S_{10}& S_{11}& \dots \dots & S_{13}\\ \dots \dots & \dots \dots & \dots \dots & \dots \dots \\ S_{30}& S_{31}& S_{32}& S_{33}\end{array}\right],& \left(4\right).\end{array}$

• • [0129] Firstly, the non-zero element value at the same element position in the matrix that is formed by the winograd transformation results of the plurality of sub-tensors is determined as the additive number at the element position in the result matrix. For example, the additive number at S₀₀ is 4, which comes from the winograd transformation results of the plurality of sub-tensors of d₀₀, d₀₂, d₂₀, d₂₂, respectively. The additive number at S₀₁ is 4, which comes from the winograd transformation results of the plurality of sub-tensors of d₀₁, d₀₂, d₂₁, d₂₁, respectively. Similarly, the additive numbers at other element positions of the result matrix are all 4.

[0130] Next, the number of the summation operations at each element position is determined based on the additive number at each element position in the result matrix. Since the additive number at each element position in the result matrix is 4, the number of the summation operations at each element position is 3.

[0131] Finally, the sum of the number of the summation operations at each element position is determined as the number of the addition executions required to calculate the summation result. Since the number of the summation operations at each element position in the result matrix is 3 and there are 16 element positions in the result matrix, the total number of the summation operations at each element position is 48. In other words, the number of the addition executions required to calculate the summation result is 48.

[0132] In a step 304, the number of clocks is determined based on a size of the sub-tensors or the result matrix.

[0133] The result matrix is a matrix after a winograd convolution forward transformation or a winograd convolution inverse transformation is performed, such as in the formula (4), the result matrix S after the winograd convolution forward transformation is performed.

[0134] When the number of the clocks used to calculate the summation result is determined, in order to match the addition execution tasks and the multiplication execution tasks in term of computing power, the number of the clocks required for the multiplication execution tasks is determined as the number of the clocks required for the addition execution tasks.

[0135] Optionally, if the winograd convolution transformation is the winograd convolution forward transformation, the number of the clocks required for the multiplication execution tasks is the same as the size of the result matrix. If the winograd convolution transformation is the winograd convolution inverse transformation, the number of the clocks required for the multiplication execution tasks is the same as a size of a transformed element-wise multiplication matrix, which in turn is the same as a size of corresponding sub-tensors, so if the winograd convolution transformation is the winograd convolution inverse transformation, the number of the clocks required for the multiplication execution tasks is the same as the size of the sub-tensors.

[0136] In other words, if the winograd convolution transformation is the winograd convolution forward transformation, the size of the result matrix is X×Y, and the determined number of the clocks used to calculate the summation result is X×Y If the winograd convolution transformation is the winograd convolution inverse transformation, the size of the sub-tensors is X×Y, and the determined number of the clocks used to calculate the summation result is X×Y.

[0137] As an optional implementation, determining the number of the clocks according to the size of the sub-tensors or the result matrix includes:

• • [0138] determining a maximum size of the sub-tensors and the result matrix; and determining the number of the clocks according to the maximum size.

[0139] If the winograd convolution transformation is the winograd convolution forward transformation, the number of the clocks required for the multiplication execution tasks is the same as the size of the result matrix corresponding to a neuron matrix or the size of the result matrix corresponding to a weight matrix in the winograd convolution operation. If the winograd convolution transformation is the winograd convolution inverse transformation, the number of the clocks required for the multiplication execution tasks is the same as the size of the transformation element-wise matrix, which is a matrix corresponding to data of the winograd convolution inverse transformation. Moreover, the size of the matrix corresponding to the data of the winograd convolution inverse transformation is equal to the size of the sub-tensors of the data, so if the winograd convolution transformation is the winograd convolution inverse transformation, the number of the clocks required for the multiplication execution tasks is equal to the size of the sub-tensors of the data. Since the winograd convolution forward transformation and the winograd convolution inverse transformation are corresponding, the maximum size of the sub-tensors in the winograd convolution transformation and the corresponding result matrix may be determined as the number of the clocks used to calculate the summation result.

[0140] In a step 305, the number of the operation sub-apparatuses is determined according to the number of the clocks and the number of the addition executions required for the summation operation.

[0141] Further, in this embodiment, the step 305 includes the following steps.

[0142] In a step 3051, a ratio of the number of the addition executions required to calculate the summation result to the number of the clocks used to calculate the summation result is calculated.

[0143] In a step 3052, based on the ratio, the number of the operation sub-apparatuses is determined, where the number of the operation sub-apparatuses is greater than the ratio.

[0144] Further, if the number of the addition executions required to calculate the summation result is expressed as NumAdd, the number of the clocks used by the summation result is expressed as NumClk, and the number of the operation sub-apparatuses is expressed as NumUnit, the determined number of the operation sub-apparatuses may be expressed as NumUnit>NumAdd/NumClk.

[0145] If the size of the result matrix is 4×4, the number of the addition executions is 48, and the number of the clocks used by the summation result is 16, then the number of the operation sub-apparatuses may be greater than 3, such as 4.

[0146] In this embodiment, the number of the operation sub-apparatuses is set to a value that is greater than the ratio of the number of the addition executions required for the summation result to the number of the clocks used by the summation result, so that addition calculations of elements in the result matrix may be completed within the number of the clocks. If the number of the operation sub-apparatuses is set to a value which is slightly greater than the ratio, all the operation sub-apparatuses may be controlled to be in a processing state within each clock, and finally the addition calculations of the elements in the result matrix may be completed within the number of the clocks.

[0147] In a step 306, the addition execution tasks are distributed to the plurality of operation sub-apparatuses correspondingly according to the determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and a preset mapping relationship between element positions in the result matrix and the plurality of operation sub-apparatuses, so as to complete the addition calculations of the elements in the result matrix within the number of the clocks.

[0148] The addition execution tasks are used to calculate each element in the result matrix.

[0149] Further, in this embodiment, after the number of the addition executions required for the summation operation and the number of the clocks and the number of the operation sub-apparatuses are determined, by distributing the addition execution tasks to the plurality of operation sub-apparatuses correspondingly according to the determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and the preset mapping relationship between the element positions in the result matrix and the operation sub-apparatuses, not only load balancing may be realized when the summation operation is performed among the plurality of operation sub-apparatuses, but also completing the addition calculations of the elements in the result matrix within the number of the clocks may be ensured. If the number of the operation sub-apparatuses is set to a value that is slightly larger than the ratio, all the operation sub-apparatuses may be controlled to be in the processing state within each clock.

[0150] Therefore, as an optional implementation, in the step 306, distributing the addition execution task used to calculate each element in the result matrix to the plurality of operation sub-apparatuses correspondingly to complete the addition calculations of the elements in the result matrix within the preset number of the clocks includes the following steps.

[0151] In a step 3061, within first N consecutive clocks, one of additive numbers used to calculate each element position in the result matrix is input to a corresponding operation sub-apparatus to perform an initial assignment on each element in the result matrix.

[0152] In a step 3062, within the remaining clocks, the remaining additive numbers used to calculate each element position in the result matrix are input to corresponding operation sub-apparatuses for the addition calculations until each element value in the result matrix is obtained.

[0153] All the operation sub-apparatuses are in the processing state within each clock.

[0154] Further, in this embodiment, firstly, the number of first consecutive clocks required to input the one of additive numbers required to calculate each element position in the result matrix to the corresponding operation sub-apparatus may be determined based on the number of the elements of the result matrix and the number of the operation sub-apparatuses. The number of the first consecutive clocks N is a ratio of the number of the elements of the result matrix to the number of the operation sub-apparatuses.

[0155] If the size of the result matrix is X×Y, and the number of the elements of the result matrix is also X×Y, and the number of the operation sub-apparatuses is t, and then the number of the first consecutive clocks is N=(X×Y)/t.

[0156] Then, within each clock of the first N consecutive clocks, the one of additive numbers used to calculate n element positions in the result matrix may be input to the corresponding operation sub-apparatus to perform the initial assignment on the elements at the n element positions in the result matrix, where the number of n is the same as the number of the operation sub-apparatuses. In other words, each operation unit performs the initial assignment on a corresponding element in the result matrix within a same clock. After the first N consecutive clocks, the plurality of the operation sub-apparatuses perform the initial assignment on all elements in the result matrix.

[0157] It may be understood that each operation sub-apparatus may include registers, and the number of the registers may be the same as the number of the element positions corresponding to the operation sub-apparatus, so that each register may store the additive number of the corresponding element position or each addition operation result. After the first N consecutive clocks, the registers corresponding to the operation sub-apparatuses store the one of the additive numbers of the corresponding element positions.

[0158] It should be noted that the one of additive numbers that is input to the operation sub-apparatus for calculating each element position in the result matrix is derived from the non-zero element in the matrix that is formed by the winograd transformation results of the plurality of sub-tensors in the formula (3). When the one of additive numbers used to calculate each element position in the result matrix is input to the corresponding operation sub-apparatus, an input order of the matrix that is formed by the winograd transformation results of the plurality of sub-tensors is required to ensure that within each clock, each operation sub-apparatus may be in a state of initially assigning the elements in the corresponding element position in the result matrix.

[0159] Finally, within each clock of the remaining clocks, each operation sub-apparatus receives one remaining additive number used to calculate the corresponding element position in the result matrix, and the operation sub-apparatus adds the remaining additive number received this time and the additive number stored in a register and stores a result of the addition operation in a corresponding register again. Until the operation sub-apparatus receives a last remaining additive number used to calculate the corresponding element position in the result matrix and the addition operation on the last remaining additive number and the result of the addition operation in the register is performed, the element values of the element positions in the result matrix are obtained.

[0160] It should be noted that the remaining additive numbers that are input to the operation sub-apparatuses for calculating the element positions in the result matrix are also derived from the non-zero elements in the matrix that is formed by the winograd transformation results of the plurality of sub-tensors. Therefore, when the remaining additive numbers used to calculate the element positions in the result matrix are input to the corresponding operation apparatuses, the input order of the matrix that is formed by the winograd transformation results of the plurality of sub-tensors is also required to ensure that each operation sub-apparatus within each clock is in the state of performing the addition operation on the elements in the corresponding element position in the result matrix.

[0161] In this optional implementation, within each clock of the first N consecutive clocks, all the operation sub-apparatuses are in the state of initially assigning the elements at the corresponding element positions in the result matrix. Within each clock of the remaining clocks, all the operation sub-apparatuses are in the state of performing the addition operations on the elements of the corresponding element positions in the result matrix.

[0162] In this embodiment, when the addition execution tasks used to calculate each element in the result matrix are distributed to the plurality of operation sub-apparatuses correspondingly to complete the addition calculations of the elements in the result matrix within the preset number of the clocks, the one of the additive numbers used to calculate each element position in the result matrix is input to the corresponding operation apparatus within the first N consecutive clocks to initially assign each element in the result matrix. Within the remaining clocks, the remaining additive numbers used to calculate the element positions in the result matrix are input to the corresponding operation sub-apparatuses for the addition calculations until each element value in the result matrix is obtained. This embodiment provides a first method of controlling each operation sub-apparatus to be in the processing state within each clock and performing the addition operation at the same element position in the result matrix by the same operation sub-apparatus, which may ensure that the plurality of operation sub-apparatuses are in the processing state within all clocks, thereby further improving an execution speed of a chip during the winograd convolution transformation and further improving processing efficiency of the chip.

[0163] As another optional implementation, the step 306 includes the following steps.

[0164] In a step 306a, the number of the clocks is divided into a plurality of clock segments, and each element position in the result matrix is divided into a plurality of element position sets, where the clock segments correspond to the element position sets one to one.

[0165] Optionally, in this embodiment, the number of the clocks is averagely divided into the plurality of clock segments based on the number of the operation sub-apparatuses. For the result matrix with a size of 4×4, the number of the clocks is 16, and the number of the operation sub-apparatuses is 4, and then the divided clock segments are 4 segments.

[0166] Optionally, in this embodiment, a plurality of element positions in the result matrix are averagely divided into the plurality of element position sets based on the number of the elements in the result matrix and the number of the operation sub-apparatuses. For each divided element position set, the operation sub-apparatuses corresponding to the element positions are required to be different to ensure that all the operation sub-apparatuses are in the processing state within each clock.

[0167] Since the number of the elements in the result matrix is the same as the number of the clocks, the number of the element position sets is the same as the number of the clock segments. Moreover, the clock segments correspond to the element positions sets one to one. In other words, it is possible to complete the addition operations of the element positions in the corresponding element position sets by the plurality of operation sub-apparatuses within one clock segment.

[0168] In a step 306b, for each element position set, within a first clock of a corresponding clock segment, the one of additive numbers used to calculate all element positions in that element position set is input to the corresponding operation sub-apparatus to initially assign the element positions in that element position set; within the remaining clocks of the clock segment, the remaining additive numbers used to calculate all element positions in each element position set are input to the corresponding operation sub-apparatuses to perform the addition operations until the each element value in that element position set is obtained.

[0169] All the operation sub-apparatuses are in the processing state within each clock.

[0170] Further, in this embodiment, when the addition execution tasks used to calculate each element in the result matrix are distributed to the operation sub-apparatuses correspondingly according to the clock segment and the element position set corresponding to the clock segment.

[0171] It should be noted that each operation sub-apparatus includes the registers, and the number of the registers is the same as the number of the element positions corresponding to the operation sub-apparatus.

[0172] Specifically, in a first clock segment, calculation tasks of a first element position set are distributed to each operation sub-apparatus. In the first clock of the first clock segment, the one of additive numbers used to calculate all element positions in the element position set is input to the corresponding operation sub-apparatus, and after the initial assignment of each element position in the element position set is performed by the operation sub-apparatus, the additive number is stored in a first register For each clock within the remaining clocks of the clock segment, the one remaining additive number used to calculate all element positions in each element position set is input to the corresponding operation sub-apparatus, and the addition calculation is performed on the stored additive number obtained from the first register and the remaining additive number, and so on until the operation sub-apparatuses complete the addition operations of the corresponding element positions in the element position set. Then addition operation results corresponding to the element positions in the first element position set are stored in the first register of each operation sub-apparatus.

[0173] By analogy, in a M-th clock segment, the calculation tasks of a M-th element position set are distributed to each operation sub-apparatus. In the first clock of the M-th clock segment, the one of additive numbers used to calculate all element positions in the element position set is input to the corresponding operation sub-apparatus. After the initial assignment of each element position in the element position set is performed, the additive number is stored in a corresponding M-th register. For each clock within the remaining clocks of the clock segment, the one remaining additive number used to calculate all element positions in each element position set is input to the corresponding operation sub-apparatus, and the addition calculation on the stored additive number obtained from the M-th register and the remaining additive number is performed, and so on until the operation apparatuses complete the addition operations of the corresponding element positions in the element position set. Then, the results of the addition operations of the corresponding element positions in the M-th element position set are stored in the M-th register of each operation sub-apparatus.

[0174] Finally, each element value in each element position set is obtained from the corresponding register, and the result matrix is obtained according to the preset mapping relationship between the element positions in the result matrix and the operation sub-apparatuses.

[0175] According to the winograd convolution operation method of this embodiment, when the addition execution tasks are distributed to the plurality of operation sub-apparatuses correspondingly to complete the addition calculations of the elements in the result matrix within the preset number of the clocks, the number of the clocks is divided into the plurality of clock segments, and each element position in the result matrix is divided into the plurality of element position sets, where the clock segments correspond to the element position sets one to one; for each element position set, within the first clock of the corresponding clock segment, the one of additive numbers used to calculate all element positions in the element position set is input to the corresponding operation sub-apparatus to initially assign the element positions in the element position set; within the remaining clocks of the clock segment, the remaining additive numbers used to calculate all element positions in each element position set are input to the corresponding operation sub-apparatuses for the addition calculations until each element value in the element position set is obtained. This embodiment provides a second method of controlling each operation sub-apparatus to be in the processing state within each clock and performing the addition operation at the same element position in the result matrix by the same operation sub-apparatus, which may ensure that the plurality of operation sub-apparatuses are in the processing state within all clocks, thereby further improving the execution speed of the chip during the winograd convolution transformation and further improving the processing efficiency of the chip.

[0176] In the case that the data in the winograd convolution operation is the neuron matrix, the weight matrix and the transformed element-wise multiplication matrix respectively, the summation operation of the winograd transformation results of the plurality of sub-tensors by the plurality of operation sub-apparatuses is illustrated below. A size of the neuron matrix is 4×4, 4×3, 3×4, 3×3, 4×1, 1×4, 3×1, 1×3, respectively, and a size of the weight matrix is 3×3, 3×2, 2×3, 2×2, 3×1, 1×3, 2×1, 1×2, respectively, and a size of the transformed element-wise multiplication matrix is 2×2, 2×2, 2×2, 2×2, 2×1, 1×2, 2×1, 1×2 correspondingly.

[0177] For the matrix with a neuron matrix size of 4×4, a matrix multiplication between a neuron matrix d and matrixes B^T and B of the winograd convolution forward transformation is expressed as shown in a formula (5):

[0178] $\begin{array}{cc}S=\mathrm{Wino}\left\{d\right\}=B_{4\times 4}^T{d}_{4\times 4}{B}_{4\times 4}=\left[\begin{array}{cccc}1& 0& -1& 0\\ 0& 1& 1& 0\\ 0& -1& 1& 0\\ 0& 1& 0& -1\end{array}\right]\left[\begin{array}{cccc}{d}_{00}& d_{01}& d_{02}& d_{03}\\ d_{10}& d_{11}& d_{12}& d_{13}\\ d_{20}& d_{21}& d_{22}& d_{23}\\ d_{30}& d_{31}& d_{32}& d_{33}\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& -1& 1\\ -1& 1& 1& 0\\ 0& 0& 0& -1\end{array}\right].& \left(5\right)\end{array}$

[0179] Transforming the plurality of sub-tensors to obtain the winograd transformation results of the plurality of sub-tensors maybe expressed as shown in the formula (2):

• • [0180] Due to the winograd convolution forward transformation of the neuron matrix, based on the size of the result matrix, the number of the clocks used to calculate the summation result is determined as 4×4=16. When the number of the addition executions required to calculate the summation result is determined based on the winograd transformation results of the plurality of sub-tensors, since the additive numbers of the element positions in the result matrix are 4, the number of the summation operations at each element position in the result matrix is 3, and the sum of the number of the summation operations at each element position in the result matrix is 48 and the number of the addition executions required to calculate the summation result is 48.

[0181] A determining method for the number of the addition executions required to calculate the summation result is essentially the same as a determining method for the difference between the number of the non-zero elements in the matrix that is formed by the winograd transformation results of all the plurality of sub-tensors and the size of the result matrix. Therefore, for the matrix with a result matrix size of 4×4, the number of the addition executions required to calculate the summation result is determined as: 1*4+3*8+9*4−4*4=48.

[0182] 1*4 means that the number of matrixes that are formed by the winograd transformation results of the sub-tensors with only 1 non-zero element is 4; 3*8 means that the number of matrixes that are formed by the winograd transformation results of the sub-tensors with 3 non-zero elements is 8; 9*4 means that the number of matrixes that are formed by the winograd transformation results of the sub-tensors with 9 non-zero elements is 4; and 4*4 is the size of the result matrix.

[0183] Based on the number of the clocks and the number of the addition executions required, the number of the operation sub-apparatuses is determined to be greater than 3. Since for the matrix with the neuron matrix size of 4×4, 4 operation sub-apparatuses may complete the addition calculations of the elements in the result matrix within 16 clocks, the number of the operation sub-apparatuses is 4. The 4 operation sub-apparatuses are numbered 0-3, and the operation sub-apparatuses corresponding to each element position in the result matrix are represented as shown in Table 1.

[0184]

TABLE 1
For a Neuron Matrix Size of 4 × 4, Operation Units Corresponding
to Each Element in a Result Matrix with a Size of 4 × 4
	0	2	1	3
	2	2	1	1
	3	3	0	0
	1	3	0	2

[0185] For the winograd convolution forward transformation with the neuron matrix size of 4×4, according to the mapping relationship between the element positions in the result matrix and the operation sub-apparatuses in Table 1, the summation operation at the same element position in the result matrix may be performed by the same operation sub-apparatus. When the addition execution tasks used to calculate each element in the result matrix are distributed to the operation sub-apparatuses correspondingly, the matrixes that are formed by the winograd transformation results of the sub-tensors are read in an order of d₁₁→d₀₁→d₁₀→d₀₀→d₁₂→d₀₃→d₀₂→d₁₃→d₂₂→d₃₀→d₂₁→d₃₁→d₂₃→d₃₃→d₂₃→d₃₂, so as to ensure that all the operation sub-apparatuses are in the processing state within each clock.

[0186] Then during the 16 clocks, within the first 4 consecutive clocks, 4 operation units input the one of additive numbers used to calculate each element position in the result matrix to the corresponding operation sub-apparatus to initially assign each element in the result matrix, where the 4 operation sub-apparatuses are in the state of performing the initial assignment within the 4 consecutive clocks. Within the remaining 12 clocks, the remaining additive numbers used to calculate each element position in the result matrix are input to the corresponding operation apparatuses for the addition calculations until each element value in the result matrix is obtained, where the operation sub-apparatuses are in the state of performing the summation operation within the 12 clocks.

[0187] It may be understood that after the element values in the result matrix are obtained, the element values may be sequentially output to other components, such as a main processing memory, so that the result matrix may be stored by the other components.

[0188] For the matrix d with a neuron matrix size of 4×3, the matrix multiplication between the neuron matrix and the matrixes B^T B of the winograd convolution forward transformation is expressed as shown in a formula (6):

[0189] $\begin{array}{cc}S=\mathrm{Wino}\left\{d\right\}=B_{4\times 4}^T{d}_{4\times 3}{B}_{3\times 3}=\left[\begin{array}{cccc}1& 0& -1& 0\\ 0& 1& 1& 0\\ 0& -1& 1& 0\\ 0& 1& 0& -1\end{array}\right]\left[\begin{array}{cccc}{d}_{00}& d_{01}& d_{02}& \\ d_{10}& d_{11}& d_{12}& \\ d_{20}& d_{21}& d_{22}& \\ d_{30}& d_{31}& d_{32}& \end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ -1& 1& -1\\ 0& 0& 1\end{array}\right].& \left(6\right)\end{array}$

[0190] Transforming the plurality of sub-tensor to obtain the winograd transformation results of the plurality of sub-tensors may be expressed as shown in a formula (7):

[0191] $\begin{array}{cc}{d}_{00}:d_{00}*\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right],d_{01}:d_{01}*\left[\begin{array}{ccc}-1& 1& -1\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right],d_{02}:d_{02}*\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]& \left(7\right)\end{array}$ $d_{10}:d_{10}*\left[\begin{array}{ccc}0& 0& 0\\ 1& 0& 0\\ -1& 0& 0\\ 1& 0& 0\end{array}\right],d_{11}:d_{11}*\left[\begin{array}{ccc}0& 0& 0\\ -1& 1& -1\\ 1& -1& 1\\ -1& 1& -1\end{array}\right],d_{12}:d_{12}*\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\\ 0& 0& -1\\ 0& 0& 1\end{array}\right]$ $d_{20}:d_{20}*\left[\begin{array}{ccc}-1& 0& 0\\ 1& 0& 0\\ 1& 0& 0\\ 0& 0& 0\end{array}\right],d_{21}:d_{21}*\left[\begin{array}{ccc}1& -1& 1\\ -1& 1& -1\\ -1& 1& -1\\ 0& 0& 0\end{array}\right],d_{22}:d_{22}*\left[\begin{array}{ccc}0& 0& -1\\ 0& 0& 1\\ 0& 0& 1\\ 0& 0& 0\end{array}\right]$ $d_{30}:d_{30}*\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ -1& 0& 0\end{array}\right],d_{31}:d_{31}*\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 1& -1& 1\end{array}\right],d_{32}:d_{32}*\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& -1\end{array}\right].$

[0192] Due to the winograd convolution forward transformation of the neuron matrix, based on the size of the result matrix, the number of the clocks used to calculate the summation result is determined as 4×3=12. The number of the addition executions required to calculate the summation result is determined as 1*4+3*6+9*2−4*3=28.

[0193] 1*4 means that the number of the matrixes that are formed by the winograd transformation results of the sub-tensors with only 1 non-zero element is 4; 3*6 means that the number of the matrixes that are formed by the winograd transformation result of the sub-tensors with 3 non-zero elements is 6; 9*2 means that the number of the matrixes that are formed by the winograd transformation results of the sub-tensors with 9 non-zero elements is 2; and 4*3 is the size of the result matrix.

[0194] Based on the number of the clocks and the number of the addition executions required, the number of the operation sub-apparatuses is determined as 4. The 4 operation sub-apparatuses are numbered 0-3, and the operation sub-apparatuses corresponding to each element position in the result matrix are shown in Table 2.

[0195] For the winograd convolution forward transformation with the neuron matrix size of 4×3, according to the mapping relationship between the element positions in the result matrix and the operation sub-apparatuses in Table 2, the summation operation at the same element position in the result matrix may be performed by the same operation sub-apparatus, and when the addition execution tasks used to calculate each element in the result matrix are distributed to the operation sub-apparatuses correspondingly, the matrixes that are formed by the winograd transformation results of the sub-tensors are read in an order of d₁₁→d₀₁→d₁₀→d₀₀→d₁₂→d₀₂→d₂₁→d₃₁→d₂₀→d₃₀→d₂₂→d₃₃, so as to ensure that all operation sub-apparatuses are in the processing state within each clock.

[0196]

TABLE 2
For a Neuron Matrix Size of 4 × 3, Operation Sub-apparatuses
Corresponding to Each Element Position in a Result Matrix
with a Size of 4 × 3
0	0	0
1	1	1
2	2	2
3	3	3

[0197] Then during the 12 clocks, within the first 3 consecutive clocks, the 4 operation units input the one of additive numbers used to calculate each element position in the result matrix to the corresponding operation sub-apparatus to initially assign each element in the result matrix, where the operation sub-apparatuses are in the state of performing the initial assignment within the 3 consecutive clocks. Within the remaining 9 clocks, the remaining additive numbers used to calculate the element positions in the result matrix are input to the corresponding operation sub-apparatuses for the addition calculations until each element value in the result matrix is obtained, where the operation sub-apparatuses are in the state of performing the summation operation within the 9 clocks.

[0198] It may be understood that after the element values in the result matrix are obtained, the element values may be sequentially output to the other components, such as the master processing memory, so that the result matrix may be stored by the other components.

[0199] For the matrix d with a neuron matrix size of 3×4, the matrix multiplication between the neuron matrix and the matrixes B^T and B of the winograd convolution forward transformation is expressed as shown in a formula (8):

[0200] $\begin{array}{cc}S=\mathrm{Wino}\left\{d\right\}=B_{3\times 3}^T{d}_{3\times 4}{B}_{4\times 4}=\left[\begin{array}{ccc}1& -1& 0\\ 0& 1& 0\\ 0& -1& 1\end{array}\right]\left[\begin{array}{cccc}{d}_{00}& d_{01}& d_{02}& d_{03}\\ d_{10}& d_{11}& d_{12}& d_{13}\\ d_{20}& d_{21}& d_{22}& d_{23}\end{array}\right]\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& -1& 1\\ -1& 1& 1& 0\\ 0& 0& 0& -1\end{array}\right].& \left(8\right)\end{array}$

[0201] Transforming the plurality of sub-tensors to obtain the winograd transformation results of the plurality of sub-tensors may be expressed as shown in a formula (9):

[0202] $\begin{array}{cc}{d}_{00}:d_{00}*{\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]}^T,d_{10}:d_{10}*{\left[\begin{array}{ccc}-1& 1& -1\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]}^T,d_{20}:d_{20}*{\left[\begin{array}{ccc}0& 0& 1\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]}^T& \left(9\right)\end{array}$ $d_{01}:d_{01}*{\left[\begin{array}{ccc}0& 0& 0\\ 1& 0& 0\\ -1& 0& 0\\ 1& 0& 0\end{array}\right]}^T,d_{11}:d_{11}*{\left[\begin{array}{ccc}0& 0& 0\\ -1& 1& -1\\ 1& -1& 1\\ -1& 1& -1\end{array}\right]}^T,d_{21}:d_{21}*{\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 1\\ 0& 0& -1\\ 0& 0& 1\end{array}\right]}^T$ $d_{02}:d_{02}*{\left[\begin{array}{ccc}-1& 0& 0\\ 1& 0& 0\\ 1& 0& 0\\ 0& 0& 0\end{array}\right]}^T,d_{12}:d_{12}*{\left[\begin{array}{ccc}1& -1& 1\\ -1& 1& -1\\ -1& 1& -1\\ 0& 0& 0\end{array}\right]}^T,d_{22}:d_{22}*{\left[\begin{array}{ccc}0& 0& -1\\ 0& 0& 1\\ 0& 0& 1\\ 0& 0& 0\end{array}\right]}^T$ $d_{03}:d_{03}*{\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ -1& 0& 0\end{array}\right]}^T,d_{13}:d_{13}*{\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 1& -1& 1\end{array}\right]}^T,d_{23}:d_{23}*{\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ 0& 0& 0\\ 0& 0& -1\end{array}\right]}^T.$