Object-detection method multi-class Bhattacharyya Boost algorithm used therein

[0001] 1. Field of the Invention

[0002] The present invention relates to a detection method and an algorithm, particularly to an object-detection method, which can process objects having multiple classes of appearances, and a Multi-class Bhattacharyya Boost (MBH-Boost) algorithm used therein.

[0003] 2. Description of the Related Art

[0004] To integrate the digital image processing technology with automatic monitoring/control, building security, or man-machine interaction is an active field the academic circle and the industry endeavor to study and develop. Various achievements thereof will enable automatic processes to replace some human works. Among this field, object detection is a critical preprocessing because it is only after the system has known the positions of target objects that the succeeding processes, such as identifying human faces or other objects in a security system, or monitoring/tracing human beings or vehicles in a monitoring/control system, can be executed. Therefore, object-detection technology has universal utility. And, some objective and measurable factors, such as detection speed, detection accuracy and detection robustness, can be used to evaluate the performance of a detection algorithm.

[0005] Recently, the learning mechanism of a boosting algorithm has been often used in object detection, and it can also be integrated with a cascade framework to increase detection speed. The efficiency and accuracy of the object-detection technology based on the boosted cascade has acquired general attention in academic circles, and further researches are also being undertaken, for example, determining the interference factors in detecting human faces, such as illumination, occlusion and rotation angles, and promoting detection accuracy under the influence of a variety of extrinsic factors.

[0006] In resolving those discussed above, the present invention proposes an object-detection method and a multi-class Bhattacharyya Boost algorithm used therein to overcome the problems.

[0007] The primary objective of the present invention is to provide a method to detect objects with multiple classes of appearances, wherein the multi-class object can be detected via only using a single boosted cascade, and the execution efficiency and the detection accuracy thereof is promoted thereby.

[0008] Another objective of the present invention is to propose a novel Multi-class Bhattacharyya Boost (MBH-Boost) algorithm to overcome the difficulties resulting from the diversified classes of the object appearances shown in different conditions and to enhance the robustness of the system.

[0009] According to the present invention, the objection-detection method is based on a multi-class cascade framework. Initially, an object-detection system is provided with an image data; multiple rectangles on the image data are separately scanned by the multi-class object-detection method, and the integral image thereof is used to calculate the projection of the image data; the calculation results will be used to perform multi-class classifications on those rectangles.

[0010] According to the object-detection method of the present invention, the Multi-class Bhattacharyya Boost (MBH-Boost) algorithm is used in the object-detection system. Firstly, training data D^Γ of a set Γ of multiple classes, a set φ of the projection directions of weak learners, and an iteration number T are provided; a weighted value w_t^x(i) with respect to each item in D^Γ is worked out via iterative calculations; an optimal projection direction φ_t is worked out via the weighted value w_t^x(i); w_t^x(i) is updated to be w_t+1^x(i) according to the classification result of the optimal projection direction φ_t; and after T iterations, a vector-valued classifier F, which is used as a decision boundary, can be worked out. The class-wise calculation results for the set Γ={A, B, C, . . . } correspondingly give the component-wise values of the classifier F, i.e., H^A, H^B, H^C, . . . , whose maximal positive value will be used as the classification standard.

[0011] To enable the objectives, technical contents, characteristics and accomplishments of the present invention to be more easily understood, the embodiments of the present invention are described below in cooperation with the attached drawings.

[0021] The primary objective of the present invention is to provide a method to detect objects with multiple classes of appearances, and the method of the present invention is based on two core elements: (1) a classification algorithm—a Multi-class Bhattacharyya Boost (MBH-Boost) algorithm, (2) a detection framework—a multi-class cascade, which are to be described below.

[0022] The novel Multi-class Bhattacharyya Boost (MBH-Boost) algorithm primarily functions to simultaneously process the detection of multiple kinds of objects with multiple classes of appearances, and classifier sharing is the primary principle thereof. All classes of objects have a common projection direction; however, an independent decision boundary of each class will be separately learned according to the training data of each kind. Via sharing a common projection, the calculation amount is obviously reduced, and the execution speed increases. In the other side, the independent decision boundary of each kind speeds up the convergence of classification error rate. Therefore, even though only few weak learners are used, classification accuracy rate still maintains high. In theory, the present invention provides a mechanism of selecting an optimal projection direction, and it is proved that the error upper bound is directly lowered thereby; in experiment, the present invention is verified in the detection of four kinds of human faces, and the results thereof show the Multi-class Bhattacharyya Boost (MBH-Boost) algorithm of the present invention is superior to the conventional methods.

[0023] In general, the target object is a very scarce incident in comparison with the others in the image, and such a characteristic is often used to speed up the detection process. Among the approaches thereby, the cascade detection framework proposed in 2001 is a well-known one; however, it can only be used to detect a single-class object. The present invention proposes a multi-class cascade framework to solve the problem. In each node of the cascade structure, the embedded classifiers are sequentially learned from the training data varying from simplicity to complexity. In contrast with the other methods for detecting the multi-class object, the algorithm of the present invention needs only a single boosted cascade framework; therefore, the algorithm of the present invention is superior to the conventional ones in detection speed, accuracy, and robustness.

[0024] Besides, the detection method of the present invention neither applies any further limitation to the desired object nor applies any special presumption between different classes of the object; therefore, the present invention has superior generality and flexibility in application; for example, the present invention is adaptable to (1) identify various kinds of objects, such as human beings, vehicles, or human faces, (2) identify multiple classes of variant appearances of a single object, such as the frontal face, the profile face and the rotated face of a human being, or human faces under different lighting conditions, (3) identify the combination case of two abovementioned ones. Thus, the present invention can be widely applied to the automatic processing system based on digital image processing technology.

[0025] The MBH-Boost of the present invention is an algorithm based on machine learning, and the system must be constructed through a well-designed training stage before it can be used to reliably perform the testing stage for its practical application. These two stages are to be described below.

Training Stage

[0026] The method of the present invention will be exemplified by the detection of profile faces. As shown in FIG. 1, Γ={A, B, . . . , I} designates the set of the denotations of nine profile faces having different rotation angles, and |Γ|=9.

Training Data:

[0027] Training data D^Γ is a set of object images and non-object images and has the correct labels thereof. There are nine classes in the exemplifying profile face detection, and nine types of training data are expressed as:

[0028] $\begin{array}{cc}\begin{array}{c}{D}^A=\left\{\left(x_1^A,y_1^A\right),\dots ,\left(x_{D^A}^A,y_{D^A}^A\right)\right\}=D^{A+}\bigcup D^{A-}\\ D^B=\left\{\left(x_1^B,y_1^B\right),\dots ,\left(x_{D^B}^B,y_{D^B}^B\right)\right\}=D^{B+}\bigcup D^{B-}\\ ⋮\\ D^I=\left\{\left(x_1^I,y_1^I\right),\dots ,\left(x_{D^I}^I,y_{D^I}^I\right)\right\}=D^{I+}\bigcup D^{I-}\\ D^{\Gamma }=D^A\bigcup D^B\bigcup \dots D^I\end{array}& \left(1\right)\end{array}$
MBH Weak Learner:

[0029] Each MBH weak learner has a projection direction, and in the present invention, the MBH weak learners share a common projection direction for all the classes of the desired object. According to the positive and negative training data of each class, two distributions are formed in a one-dimensional subspace after projection. According to the distributions, each class learns and obtains its own decision boundary independently. Let f be a weak leaner, which comprises a projection direction φ, and x be a pattern input to f. The decision process can be expressed as:

[0030] $\begin{array}{cc}\begin{array}{c}f\left(x\right)=\left[h^A\left(x\right),\dots ,h^I\left(x\right)\right]\left(\mathrm{profile}\mathrm{face}\mathrm{detection}\right)\\ =[h^{\chi }\left(x\right)\chi \in \Gamma ],\left(\mathrm{general}\mathrm{case}\right)\end{array}& \left(2\right)\end{array}$
wherein

[0031] $\begin{array}{cc}{h}^{\chi }\left(x\right)=\ln \sqrt{p_k^{\chi +}\left(\varphi \right)/p_k^{\chi -}\left(\varphi \right)}\mathrm{if}{\varphi }^{T}x\in b_k.p_k^{\chi +}\left(\varphi \right)=\sum _{i_k^{\chi +}\left(\varphi \right)}^{}{w}^{\chi }\left(i\right)\mathrm{and}{p}_k^{\chi -}\left(\varphi \right)=\sum _{i_k^{\chi -}\left(\varphi \right)}^{}{w}^{\chi }\left(i\right)i_k^{\chi }\left(\varphi \right)=\{ix_i\in D^{\chi },{\varphi }^T{x}_i\in b_k\}& \left(3\right)\end{array}$

[0032] Refer to FIG. 2 showing the action of two projection directions and corresponding weak learners on the classes of A, C, and E in the 10-bin subspace.

Multi-Class Bhattacharyya Boost (MBH-Boost) Algorithm:

[0033] To utilize the MBH-Boost algorithm, the following parameters and data are needed: (1) the training data D^Γ of detection classes Γ={A, B, C, . . . I}, (2) the set Φ of projection directions of the weak learner, (3) the number of iteration T. In the “t”th iteration of MBH-Boost algorithm, the weighted value w_t^x(i) of each component in the training data D^Γ is updated, and an optimal projection direction φ_t will be derived from the weighted value w_t^x(i); f_t the vector-valued weak learner can also be worked out from the optimal projection direction φ_t in the “t”th iteration. The execution process can be expressed by the following algorithm.

[0034]

Algorithm:MBHBoost
Input : Face ⁢ ⁢ classes , Γ ; D Γ = ⋃ 𝓍 ∈ Γ ⁢ D 𝓍 ; Projection ⁢ ⁢ set , Φ ; Number ⁢ ⁢ of ⁢ ⁢ iterations , T . ⁢ Output : A ⁢ ⁢ vector ⁢ - ⁢ valued ⁢ ⁢ MBH ⁢ ⁢ classifier ⁢ ⁢ F . ⁢ Initialize : the ⁢ ⁢ weight ⁢ ⁢ vector ⁢ ⁢ w 1 𝓍 ⁡ ( i ) = 1 /  D 𝓍  , for ⁢ ⁢ i = 1 , 2 , … ⁢ ,  D 𝓍  ⁢ ⁢ and ⁢ ⁢ ∀ 𝒳 ∈ Γ . ⁢ for ⁢ ⁢ t ← 1 , 2 , … ⁢ , T ⁢ ⁢ do ⁢ ⌊ ⁢ 1. ⁢ ⁢ Determine ⁢ ⁢ the ⁢ ⁢ optimal ⁢ ⁢ projection ⁢ ⁢ ϕ t ⁢ ⁢ from ⁢ ⁢ Φ ⁢ by ⁢ ⁢ solving ⁢ ⁢ ( 4 ) . ⁢ 2. ⁢ ⁢ Construct ⁢ ⁢ the ⁢ ⁢ MBH ⁢ ⁢ weak ⁢ ⁢ learner ⁢ ⁢ f t ⁢ associated ⁢ ⁢ with ⁢ ⁢ ϕ t ⁢ ⁢ using ⁢ ⁢ ( 2 ) . ⁢ 3. ⁢ ⁢ w t + 1 𝓍 ⁡ ( i ) ← w t 𝓍 ⁡ ( i ) ⁢ exp ⁡ ( - y i 𝓍 ⁢ h t 𝓍 ⁡ ( x i 𝓍 ) ) / Z t 𝓍 , ⁢ for ⁢ ⁢ i = 1 , 2 , … ⁢ ,  D 𝓍  ⁢ ⁢ and ⁢ ⁢ ∀ 𝒳 ∈ Γ . ⁢ ( Z t 𝓍 ⁢ ⁢ is ⁢ ⁢ a ⁢ ⁢ normalization ⁢ ⁢ factor ⁢ ⁢ such ⁢ ⁢ that ⁢ ⁢ w t + 1 𝓍 ⁢ is ⁢ ⁢ a ⁢ ⁢ distribution . ) ⁢ Output ⁢ ⁢ an ⁢ ⁢ MBH ⁢ ⁢ classifier ⁢ ⁢ F : F ⁡ ( x ) = ∑ t = 1 T ⁢ ⁢ f t ⁡ ( x ) = ⁢ [ H 𝓍 ⁡ ( x ) = ∑ t = 1 T ⁢ ⁢ h t 𝓍 ⁡ ( x ) ❘ 𝒳 ∈ Γ ] .

[0035] The vector-valued output of the classifier F includes the calculation results H^A, H^B, H^C . . . respectively corresponding to each class in each set Γ, which can be used as the decision boundaries of the corresponding classes. In each of the iterations, the projection direction is selected according to:

[0036] $\begin{array}{cc}{\varphi }_t=\underset{\varphi \in \Phi }{\arg \min }\sum _{\chi \in \Gamma }{\Delta }_t^{\chi }\times {\mathrm{BHC}}_t^{\chi }\left(\varphi \right),\mathrm{where}{\mathrm{BHC}}_t^{\chi }\left(\varphi \right)=\sum _{k=1}^m\sqrt{p_k^{\chi +}\left(\varphi \right)p_k^{\chi -}\left(\varphi \right)}.& \left(4\right)\end{array}$
In theory, it is based on the following formulae:

[0037] $\begin{array}{cc}{\mathrm{BHC}}_t^{\chi }\left(\varphi \right)=Z_t^{\chi }/2.& \left(5\right)\\ \begin{array}{c}{\Delta }_t^{\chi }=D^{\chi }\sum _{i=1}^{D^{\chi }}{w}_1^{\chi }\left(i\right)\exp \left(-y_i^{\chi }\sum _{\tau =1}^{t-1}{h}_{\tau }^{\chi }\left(x_i^{\chi }\right)\right)\\ =D^{\chi }Z_1^{\chi }\sum _{i=1}^{D^{\chi }}{w}_2^{\chi }\left(i\right)\exp \left(-y_i^{\chi }\sum _{\tau =2}^{t-1}{h}_{\tau }^{\chi }\left(x_i^{\chi }\right)\right)=\dots \\ =D^{\chi }Z_1^{\chi }\dots Z_{t-1}^{\chi }\sum _{i=1}^{D^{\chi }}{w}_t^{\chi }\left(i\right)\\ =D^{\chi }\prod _{\tau =1}^{t-1}{Z}_{\tau }^{\chi }.\end{array}& \left(6\right)\end{array}$

[0038] The optimal solution of the upper bound of the error rate under the projection direction selected by formula (4) can be derived from formulae (5) and (6), as shown below:

[0039] $\begin{array}{cc}\sum _{\chi \in \Gamma }{\Delta }_t^{\chi }\times {\mathrm{BHC}}_t^{\chi }\left(\varphi \right)=\frac{1}{2}\sum _{\chi \in \Gamma }D^{\chi }\prod _{\tau =1}^t{Z}_{\tau }^{\chi }\propto \sum _{\chi \in \Gamma }D^{\chi }\prod _{\tau =1}^t{Z}_{\tau }^{\chi }.& \left(7\right)\end{array}$
Multi-Class Cascade:

[0040] As mentioned above, the multi-class cascade can be used to improve detection accuracy, execution speed, and robustness. In utilizing the MBH-Boost algorithm to construct the nodes of the cascade framework, firstly, the training targets (target detection rate μ and false-positive rate ν) of each node are to be preset, and target detection rate μ is usually preset to range from 99.5%˜99.9%, and false-positive rate ν is usually preset to range from 40%˜50%. Let F_k be the classifier embedded into the k th node, and the training data in the k th node would be the data passing through the anterior k−1 nodes. Additionally, a voluminous database Q, which collects various images except the object images (human faces in this embodiment), will be constructed to create a negative training data.

[0041] The constructing process of the multi-class cascade framework can be expressed by the following algorithm.

[0042]

Algorithm:Multi-Class Cascade:Training
Input : Φ ; D Γ ; A ⁢ ⁢ set ⁢ ⁢ of ⁢ ⁢ images ⁢ ⁢ containing ⁢ ⁢ no ⁢ ⁢ faces ⁢ ⁢ Q ; Target ⁢ ⁢ detection ⁢ ⁢ rate ⁢ ⁢ μ ; False ⁢ - ⁢ positive ⁢ ⁢ rate ⁢ ⁢ ν . ⁢ Output : A ⁢ ⁢ cascade ⁢ ⁢ of ⁢ ⁢ MBH ⁢ ⁢ classifiers ⁢ ⁢ { F 1 , … ⁢ , F ɛ } ; Number ⁢ ⁢ of ⁢ ⁢ stage ⁢ ⁢ used ⁢ ⁢ s 𝓍 , ∀ 𝒳 ∈ Γ . ⁢ Initialize : k ← 1 ; Γ k ← Γ ; while ⁢ ⁢ Γ k ≠ 0 / do ⁢ ⌊ With ⁢ ⁢ D Γk ⁢ ⁢ and ⁢ ⁢ Φ , apply ⁢ ⁢ MBH ⁢ - ⁢ Boost ⁢ ⁢ to ⁢ ⁢ derive F k = [ H k 𝓍 ❘ 𝒳 ∈ Γ k ] ⁢ ⁢ where ⁢ ⁢ each ⁢ ⁢ component ⁢ ⁢ H k 𝓍 should ⁢ ⁢ achieve ⁢ ⁢ the ⁢ ⁢ preset ⁢ ⁢ ( μ , ν ) ⁢ ⁢ requirements ; foreach ⁢ ⁢ 𝒳 ∈ Γ k ⁢ ⁢ do ⌊ D 𝓍 + ← { ( x , y ) ❘ ( x , y ) ∈ D 𝓍 + ⋀ H k 𝓍 ⁡ ( x ) ≥ θ k 𝓍 } ; D 𝓍 - ← False ⁢ - ⁢ Positives ⁢ ⁢ from ⁢ ⁢ D 𝓍 - ⁢ ⁢ or ⁢ ⁢ from Q ⁢ ⁢ such ⁢ ⁢ that ⁢ ⁢  D 𝓍 -  =  D 𝓍 +  ; if ⁢ ⁢ not ⁢ ⁢ enough ⁢ ⁢ False ⁢ - ⁢ Positives ⁢ ⁢ then ⌊ s 𝓍 ← k ; Γ k ← Γ k - { 𝒳 } : s ← k ; Γ k + 1 ← Γ k ; k ← k + 1 ;

Testing Stage

[0043] As shown in FIG. 3, during testing stage, the first step S1 is to provide an image data for the detection system; referring to FIG. 4, the second step S2 is that the detection system scans multiple rectangles 10 in the image data, and the rectangles 10 are of different sizes, such as 24×24, 30×30, 38×38 . . . ; in the third step S3, the integral images are used to perform the rapid calculation of the projection in the image data defined by each weak learner, and referring to FIG. 5, the projection calculation is to calculate the difference between the sums of the gray-level values in balck region(s) of each rectangle 10 and the sum of gray-level values in the remaining white region(s), and the calculation results will be input to the weak learner; in the fourth step S4, according to the classification results output by multiple previously trained weak learners, the MBH-Boost algorithm is used to perform multi-class classification on the rectangle 10, and a detection framework cascaded by vector-valued classifiers is used to perform the detection in order to determine whether the object is the desired one. Thus, the detection process is completed.

[0044] When a test pattern “x” (a rectangle) is input into a multi-class cascade framework having completed the learning, it will be processed as follows:

[0045]

Algorithm:Multi-Class Cascade:Testing
Input : A ⁢ ⁢ test ⁢ ⁢ pattern ⁢ ⁢ x ; Face ⁢ ⁢ classes ⁢ ⁢ Γ ; A ⁢ ⁢ cascade ⁢ ⁢ of detectors ⁢ ⁢ { F 1 , … ⁢ , F s } ; Number ⁢ ⁢ of ⁢ ⁢ stages , s 𝓍 , ∀ 𝒳 ∈ Γ . ⁢ Output : A ⁢ ⁢ vector ⁢ ⁢ of ⁢ ⁢ boolean ⁢ ⁢ output ⁢ s , output ⁡ ( Γ ) . ⁢ Initialize : k ← 1 ; Λ ← Γ ; while ⁢ ⁢ Λ ≠ 0 / do ⁢ ⌊ Jointly ⁢ ⁢ evaluate ⁢ ⁢ H k 𝓍 ⁡ ( x ) , ∀ 𝒳 ∈ Λ ; foreach ⁢ ⁢ 𝒳 ∈ Λ ⁢ ⁢ do ⌊ if ⁢ ⁢ H k 𝓍 ⁡ ( x ) < θ k 𝓍 ⁢ ⁢ then ⌊ ⁢ output ⁡ ( 𝒳 ) ← False ; Λ ← Λ - { 𝒳 } ; else ⁢ ⁢ if ⁢ ⁢ k = s 𝓍 ⁢ ⁢ then ⌊ ⁢ output ⁡ ( 𝒳 ) ← True ; Λ ← Λ - { 𝒳 } ; k ← k + 1 ;

[0046] The algorithm of the present invention provides each class of the object with a Boolean output, which denotes whether the test pattern (rectangle) is the object of one specified class.

[0047] When the detection of a human face 14 is used for exemplification, the detection process is as follows: firstly, referring to FIG. 6, the image data 16 is provided; referring to FIG. 7, different sizes of rectangles of the image data 16 are scanned, and the integral images are used to calculate the projections; the MBH-Boost algorithm is used to perform multi-class classification of the human face 14; the single cascade of vector-value classifiers is used to speed up the detection in order to determine the viewing angles and the lighting conditions of the human face 14, as shown in FIG. 8 and FIG. 9.

[0048] The object-detection method of the present invention can designate the desired object and the class thereof. When it is exemplified by the human face detection under different lighting conditions or with different rotation angles, the execution results are shown in FIG. 9, wherein the light incident angles and the rotation angles of the human face are all appropriately designated. From the above examples, it can be concluded: the object-detection method of the present invention can overcome the detection difficulties resulting from the diversified classes of the appearances under different conditions, and the robustness of the system can also be obviously improved. As shown in the embodiment of human face detection, in comparison with other detection algorithms, the algorithm of the present invention has the characteristics of instant detection, higher detection accuracy, capability of dealing with multi-class appearances resulting from the rotation angle of the object, viewing angle of the camera, lighting conditions, and occlusion.

[0049] Those embodiments described above are not to limit the scope of the present invention but only to clarify the present invention to enable the person skilled in the art to understand, make, and use the present invention. Any equivalent modification and variation according to the spirit of the present invention disclosed herein is to be included within the scope of the present invention.