Traffic control apparatus, method thereof, and program therefor

[0001] This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2013-21398, filed on Feb. 6, 2013; the entire contents of which are incorporated herein by reference.

[0002] Embodiments described herein relate generally to a traffic control apparatus by using face recognition, a method thereof, and a program therefor.

[0003] A traffic control apparatus configured to control whether or not to grant a permission of traffic by face recognition takes an image of the face of a user by a camera, and performs face recognition by detection of a face area from the image.

[0004] In order to enhance the face recognition, there is provided a traffic control apparatus configured to display information that gives instructions to a user which position the face of the user should be in and which posture the user should take to ensure identification of the face of the user on a display.

[0005] However, in the traffic control apparatus described above, the user cannot know how his or her face is being shot unless there is a display. Therefore, there is a problem that face identification is difficult if the shot image of the user's face is out of alignment from the center of the image, or the face in the image is too small.

[0006] Therefore, it is an object of the embodiments to provide a traffic control apparatus configured to prompt a user to put his or her face in a state suitable for face recognition, a method thereof, and a program therefor.

[0026] According to embodiments, there is provided a traffic control apparatus including: an image input unit configured to input an image including a face of a user taken by a camera; a face detecting unit configured to detect a face area of the user from the image; a generating unit configured to obtain a difference between a state of the detected face area and a state of a face area suitable for face recognition, and generate presentation information for instructing the user to move the face to a position and an orientation suitable for the face recognition when the difference is larger than a threshold value; and a noticing unit having a plurality of keys arranged in a matrix pattern, and configured to illuminate, blink, or extinguish one or the plurality of keys for specifying the position or the direction that the face is to be moved to on the basis of the presentation information.

[0027] Referring now to the drawings, a traffic control apparatus 1 of an embodiment will be described below.

[0028] Referring now to FIG. 1 to FIG. 5, the traffic control apparatus 1 according to Embodiment 1 will be described.

[0029] An appearance and a state of installation of the traffic control apparatus 1 will be described with reference to FIG. 1. FIG. 1 is an appearance drawing of the traffic control apparatus 1, and FIG. 2 is a drawing illustrating the state of illustration of the traffic control apparatus 1.

[0030] As illustrated in FIG. 1, the traffic control apparatus 1 includes a plurality of keys 3 arranged on a front surface of an apparatus body 2 in a matrix pattern, for example, in a lattice pattern of 5 rows×5 columns, and a single camera 4 installed at an upper center of the apparatus body 2. The keys 3 include ten-key (“0”, “1” to “9”), function keys (“F1” to “F4”), and other keys (“L”, “F”, “*”, “B”, “R”, “OK”, “NG”, “AL”, “GD”, “C”, “E”) arranged in a lattice pattern of 5 rows×5 columns, the respective keys 3 have translucency, and light-emitting elements such as LED lamps are provided therein.

[0031] As illustrated in FIG. 2, the traffic control apparatus 1 is provided on the side of a door 5 of a room at a height of the face of a user 10. Then, the user 10 stands in front of the apparatus body 2 of the traffic control apparatus 1, and the traffic control apparatus 1 takes an image (ie. face image) of the user 10 by the camera 4. Subsequently, the traffic control apparatus 1 unlocks the door 5 when the feature obtained from the image matches the stored in advance registered feature of the person who is authorized to pass through after comparison.

[0032] A configuration of the traffic control apparatus 1 will be described with reference to FIG. 3. FIG. 3 is a block diagram illustrating the traffic control apparatus 1.

[0033] As illustrated in FIG. 3, the traffic control apparatus 1 includes an image input unit 12, a face detecting unit 14, an extracting unit 16, a storage unit 18, a recognizing unit 20, a generating unit 22, and a noticing unit 24.

[0034] The image input unit 12 inputs a face image of the user 10 from the camera 4 as an object of face recognition.

[0035] The face detecting unit 14 detects a face area from the input face image.

[0036] The extracting unit 16 extracts the feature of the face of the user 10 from the detected face area. The extracting unit 16 extracts the feature of persons allowed to pass through the door 5 by using the image input unit 12 and the face detecting unit 14 as registered amounts of characteristics.

[0037] The storage unit 18 stores the features of the persons allowed to pass through the door 5 obtained as described above as the registered features.

[0038] The recognizing unit 20 recognizes the face of the user 10 by matching a feature of the user 10 extracted by the extracting unit 16 and the registered feature stored in the storage unit 18.

[0039] The generating unit 22 generates presentation information for giving an instruction to the user 10 to move his or her face to a state of the face area suitable for the face recognition (the position and the size of the face area).

[0040] The noticing unit 24 includes the keys 3 respectively having a light-emitting element, and a control circuit configured to cause the light-emitting elements of the respective keys 3 to emit light, and illuminates or blinks the keys 3 arranged in a lattice manner on the basis of the presentation information generated by the generating unit 22.

[0041] Subsequently, an operation of the traffic control apparatus 1 at the time of face recognition will be described with reference to a flowchart in FIG. 8.

[0042] First of all, in Step S1, when the user 10 stands in front of the apparatus body 2 of the traffic control apparatus 1 and, for example, presses the key 3 of “*”, the image input unit 12 inputs a face image in which the user 10 appears from the camera 4. Then, the procedure goes to Step S2.

[0043] Then, in Step S2, the face detecting unit 14 detects the face area from the input face image. For example, the following method is used for the detection of the face area.

[0044] First of all, in order to identify whether the face of the user 10 is in an area of concern of the face image, the face detecting unit 14 sets a plurality of sets of pixel areas in the area of concern.

[0045] Secondly, the face detecting unit 14 calculates a differential value (Harr-Like feature) of brightness of the respective pixel area sets.

[0046] Thirdly, the face detecting unit 14 identifies whether or not the face is in the area of concern by comparing the differential value and a threshold value predetermined by preparatory learning by using the face image of the user 10. The face detecting unit 14 is capable of determining whether or not the area of concern is the face area with higher degree of accuracy by combining a plurality of threshold processes of brightness differential values (Joint Harr-Like feature) for evaluating the correlation (co-occurrence property) between the plurality of features.

[0047] Fourthly, the face detecting unit 14 detects the position and the size of the face area by determining the area of concern while changing the position and the size of the area of concern in the face image. Then, the procedure goes to Step S3.

[0048] Subsequently, in Step S3, the generating unit 22 calculates differences between the position and the size of the current face area on the face image detected by the face detecting unit 14 and the position and the size of the face area on the face image suitable for the face recognition (hereinafter, referred to as “the optimal face area”).

[0049] The term the “position of the current face area” means a coordinate value (x1, y1) of the position of the center of the detected face area on the face image or a coordinate value (x1, y1) of a position of center of gravity, and the term “the size of the current face area” means a size L1 of the detected face area. The term “coordinate values” mean coordinate values in an x-y orthogonal coordinate system which specifies the pixel position of a certain point on the face image. The term the “size of the face area” means the size in the vertical direction (y-direction) or the size in the lateral direction (x-direction) of the face area.

[0050] The term the “position of the optimal face area” means, for example, a coordinate value (x0, y0) of the center position on the face image, and the term the “size of the face area” means a size L0 at the center position on the face image in the predetermined vertical direction or lateral direction optimal to the face recognition.

[0051] The term “difference in position of the face area” means a difference D(x0−x1, y0−y1)=D(xd, yd) between the coordinate value (x1, y1) of the center position or the position of center of gravity of the position of the current face area and the coordinate value (x0, y0) of the center portion of the optimal face area.

[0052] The term the “difference in size of the face areas” means a difference Ld between the size of the current face area and the size of the optimal face area, where Ld=L0−L1.

[0053] The generating unit 22 generates first presentation information indicating the position of the current face area (x1, y1) and the size L1. The “first presentation information” illuminates one or the plurality of keys 3 included in a range corresponding to the position of the face (x1, y1) and the size L1 for giving an instruction to the user 10 by using light-emitting patterns of the plurality of keys 3 arranged in the lattice pattern.

[0054] Subsequently, the generating unit 22 generates second presentation information for giving an instruction to move to the state suitable for the face recognition when the difference of the position of the face area D (xd, yd) exceeds the predetermined threshold values (for example, threshold values determined respectively in the x-direction and the y-direction). As a method of generating, the generating unit obtains the positions and the range of the keys 3 corresponding to the positions and the range indicated by a vector (=difference D) to the center position (x0, y0) on the face image from the center position (x1, y1) of the current face area of the user 10 and generates a light-emitting pattern for causing the positions or the range of the keys 3 to blink as the second presentation information.

[0055] The generating unit 22 generates the second presentation information for giving an instruction to be a size suitable for the face recognition even when the difference Ld in size of the face area exceeds a predetermined threshold value. As a method of generating, when a relationship; the difference Ld in size of the face area>threshold value Lmin is satisfied, a light-emitting pattern for causing the key 3 at the center of the plurality of keys 3 arranged in the lattice pattern of 5 rows×5 columns to blink is generated as the second presentation information.

[0056] Examples of the presentation information will be illustrated in FIG. 4 to FIG. 7. FIG. 4A, FIG. 5, FIG. 6A, and FIG. 7 illustrate a light-emitting pattern of the keys 3 of the traffic control apparatus 1. Hatched portions of the keys 3 indicate the illuminated keys 3, and portions where star mark is provided indicate the blinking keys 3. FIG. 4B and FIG. 6B illustrate face images.

[0057] Referring to FIG. 4 and FIG. 5, a case where an instruction is given to the user 10 to move his or her face rightward because the position of the face is too far on the left side will be described. Since the current face area on the face image is too far on the left side with respect to the optimal face area illustrated in FIG. 4B, the generating unit 22 generates a light-emitting pattern for illuminating the left side range (the keys 3 of “OK”, “NG”, “AL”) of the plurality of keys 3 arranged in the lattice pattern as illustrated in FIG. 4A as the first presentation information indicating the position and the size of the current face area. With this first presentation information, the user 10 knows that his or her face is too far on the left side only by looking at the light-emitting pattern of the plurality of keys 3 arranged in the lattice pattern of 5 rows×5 columns without viewing the display. Subsequently, the generating unit 22 generates the second presentation information as the light-emitting pattern for blinking the key 3 of “R” as illustrated in FIG. 5 for giving an instruction to the user 10 to move his or her face rightward. With the first presentation information and the second presentation information, the user 10 knows that he or she is instructed to move his or her face rightward only by looking at the light-emitting pattern of the plurality of keys 3 arranged in the lattice pattern of 5 rows×5 columns without viewing the display.

[0058] Referring now to FIG. 6 and FIG. 7, a case where the face of the user 10 in the face image is too small will be described. As illustrated in FIG. 6B, since the current face area is too small in the face image, the generating unit 22 generates the second presentation information as a light-emitting pattern for illuminating only the center portion (the key 3 of “5”) of the plurality of keys 3 arranged in the lattice pattern as illustrated in FIG. 6A, and then the generating unit 22 generates the second presentation information as a light-emitting pattern for blinking the element of “F” in order to instruct the user 10 to perform an action to move forward (an action to approach the apparatus body 2) as illustrated in FIG. 7.

[0059] Then, the procedure goes to Step S4.

[0060] Subsequently, in Step S4, the noticing unit 24 operates a control circuit of the light-emitting element on the basis of the first presentation information and the second presentation information generated by the generating unit 22, and illuminates or blinks the light-emitting elements of the plurality of keys 3 arranged in a lattice pattern to present the first presentation information and the second presentation information to the user 10. Accordingly, the user 10 is capable of moving to a position where his or her face is readily recognized only by looking at the light-emitting pattern of the plurality of keys 3 arranged in the lattice pattern of 5 rows×5 columns without viewing the display. Then, the procedure goes to Step S5.

[0061] Subsequently, in Steps S5 and S6, the traffic control apparatus 1 repeats the processes in Steps S2 to S4 until it is determined that the face area detected from the face image is in a state of being suitable for the face recognition, or until a predetermined limit time is exceeded.

[0062] In other words, in Step S5, the traffic control apparatus 1 determines whether or not it is within the limit time and, if yes, the procedure goes to Step S6, and the procedure terminates if the limit time is exceeded. If the state of the face suitable for the face recognition is not achieved even though the user 10 has moved his or her face within the limit time, the traffic control apparatus 1 may notify the user 10 of that effect by illuminating or blinking all the keys 3.

[0063] In Step S6, the traffic control apparatus 1 goes to Step S7 if it is determined that the face area detected within the limit time is in a state suitable for the face recognition, and goes back to Step S1 when the face area detected within the limit time is not in a state suitable for the face recognition.

[0064] Subsequently, in Step S7, the extracting unit 16 extracts the feature required for the face recognition of the user 10 from the face area of the face image. In other words, the extracting unit 16 detects a plurality of face feature points from the face image. A method of detecting the face feature points as described below is used. The extracting unit 16 uses six points in total of face feature points including two feature points each for eyes, nostrils, and mouth corners as the face feature points.

[0065] First of all, the extracting unit 16 detects the image feature point from the face image by using a separability filter. The image feature point may be, for example, corner detection.

[0066] Subsequently, the extracting unit 16 performs a pattern matching process on the detected respective image feature points. Specifically, the extracting unit 16 acquires a local image from the face image in the vicinities of the respective image feature points.

[0067] Subsequently, the extracting unit 16 calculates a similarity to dictionary information prepared in advance from the image around the eyes, the nostrils, and the mouth corners. A subspace method is used for the calculation of the similarity.

[0068] Subsequently, the extracting unit 16 leaves similarities exceeding a predetermined threshold value among the respective similarities calculated at the respective image feature points as feature point candidates.

[0069] Subsequently, the extracting unit 16 selects a combination of left and right eyes, left and right nostrils, and left and right mouth corners that satisfies predetermined positional conditions from among the respective feature point candidates as feature points. The predetermined positional relationship includes the distance or the angle of a segment connecting points at the left and right eyes or the like. The extracting unit 16 may obtain the similarity to information of the dictionary prepared in advance in the same manner as the local image by the image normalized with reference to the two points such as the left and right eyes, and set the fact that the obtained similarity exceeds the predetermined threshold value as a condition.

[0070] Subsequently, the extracting unit 16 corrects the face orientation to the front by the correspondence between two-dimensional coordinates of the selected feature points and three-dimensional standard face shape models, and normalizes the vector in which normalized image luminance values are arranged into “1” to obtain the amount of feature. However, the feature is not limited thereto, and the vector obtained by applying Sobel filter or Gabor filter on the image normalized to the front and arranging the luminance values of the image may be employed as the feature. The above-described vector applied with linear conversion such as whitening conversion may be employed as the feature.

[0071] Subsequently, in Step S8, the recognizing unit 20 calculates the similarity between the detected feature described above, and the registered feature stored in the storage unit 18 as in the same method. Simple similarity between the features is used as calculation of the similarity, for example.

[0072] The recognizing unit 20 determines whether the user 10 is a person who is allowed to pass by, for example, whether or not the similarity exceeds a predetermine threshold value, and if the similarity exceeds the threshold value, determines that the user 10 is the person who is allowed to pass to release the key lock of the door 5 or to illuminate or blinks the key 3 of “OK”. Then, the traffic control apparatus 1 terminates the traffic control.

[0073] According to Embodiment 1, a method of movement to the state suitable for the face recognition may be instructed to the user 10 by using the light-emitting pattern of the plurality of keys 3 arranged in a lattice pattern even though the display is not provided, and the traffic control apparatus 1 having a high rate of acceptance may be realized.

[0074] Referring now to FIG. 9 to FIG. 10, the traffic control apparatus 1 according to Embodiment 2 will be described below.

[0075] Referring now to FIG. 9 to FIG. 10, the traffic control apparatus 1 according to Embodiment 2 will be described. FIG. 9 is a block diagram illustrating the traffic control apparatus 1 of Embodiment 2, and FIG. 10 is a drawing illustrating the state of installation of the traffic control apparatus 1.

[0076] As illustrated in FIG. 9, the traffic control apparatus 1 of Embodiment 2 includes an entire image input unit 26 and a person detecting unit 28 in addition to the image input unit 12, the face detecting unit 14, the extracting unit 16, the storage unit 18, the recognizing unit 20, the generating unit 22, and the noticing unit 24 of the traffic control apparatus 1 of Embodiment 1.

[0077] The entire image input unit 26 receives an input of the entire image including an entire body of the user 10 therein from a monitoring camera 6 installed at a position different from the apparatus body 2 of the traffic control apparatus 1 (for example, at a ceiling of a corridor where the door 5 exists), as illustrated in FIG. 10.

[0078] The person detecting unit 28 detects an entire area of the user 10 instead of the face or a partial area such as an upper body area of the user 10 from the entire image input by the entire image input unit 26. The face detecting method performed by the face detecting unit 14 described in Embodiment 1 is used as a method of detecting the entire area or the partial area of the user 10. Specifically, the person detecting unit 28 detects by using the entire area or the partial area of the user 10 instead of the face area by learning in advance.

[0079] The generating unit 22 determines whether or not the state of the face area (the position and the size) of the user 10 detected by the face detecting unit 14 is suitable for the face recognition from the entire area or the partial area of the user 10 detected by the person detecting unit 28 and generates presentation information from the content of determination. The generating method is as follows.

[0080] First of all, the generating unit 22 acquires the positions and the sizes of the entire area or the partial area of the user 10 at a moment when the state of the face area appearing in the face image input to the image input unit 12 is suitable for the face recognition, and a moment when it is not suitable in advance as comparison information in advance.

[0081] Secondly, the generating unit 22 stores information indicating how the user 10 should move to achieve the suitable state (for example, move rightward) from the difference between the position or the size in the suitable state and the position or the size not in the suitable state in a instruction table in advance.

[0082] The first and second processes are registration process that the generating unit 22 prepares in advance before performing the face recognition of the user 10. From a third process onward, a process of performing the face recognition of the user 10 when entering or going out from the room.

[0083] Thirdly, the generating unit 22 inputs the entire area or the partial area of the body of the user 10 detected by the person detecting unit 28.

[0084] Fourthly, the generating unit 22 compares the input entire area or partial area with the comparison information, and, if the state of the face area in the face image input to the image input unit 12 is determined to be in the state suitable for the face recognition, gives an instruction to the recognizing unit 20 to perform the face recognition. In contrast, if the generating unit 22 determines that it is the state not suitable, the generating unit 22 generates the presentation information (light-emitting pattern) on the basis of the instruction table corresponding to the comparison information and outputs to the noticing unit 24. The noticing unit 24 presents the user 10 by using the light-emitting pattern of the plurality of keys 3 arranged in the lattice pattern as in Embodiment 1.

[0085] According to Embodiment 2, even when the face cannot be detected like a case where the face of the user 10 is not in the image in the image input unit 12, the instruction may be given to the user 10 to move the face to the state suitable for the face recognition, so that the traffic control apparatus 1 having a high rate of acceptance may be realized.

[0086] As the presentation information of Embodiment 1, the light-emitting elements of the keys 3 are illuminated or blinked to indicate the position or the size of the face area and instruct the user to move the the face. However, the invention is not limited thereto.

[0087] For example, the light-emitting elements of the keys 3 may be changed in color or may be extinguished to indicate the position or the size of the face area and instruct the user to move the the face.

[0088] As illustrated in FIG. 11, the presentation information may be presented by illuminating or blinking the light-emitting elements of the keys 3 arranged in the lattice pattern in an arrow shape by using all the light-emitting elements.

[0089] Also, as illustrated in FIG. 12, the presentation information may be expressed by a dynamic lamp action which moves the light-emitting pattern having an arrow shape of the light-emitting elements of a plurality of keys 3 arranged in the lattice pattern from the left to the right.

[0090] Also, as illustrated in FIG. 13, a presenting light-emitting portion 8 may be provided on the apparatus body 2. As the presenting light-emitting portion 8, elongated light-emitting elements are provided at four sides of the keys 3 arranged in the lattice pattern, respectively, and, for example, since the face area detected from the face image is positioned on the left side with respect to the optimal face area as illustrated in FIG. 13B, when the rightward movement is required, the presenting light-emitting portion 8 on the right side is illuminated as the presentation information as illustrated in FIG. 13A. In order to present the position or the size of the current face area, a noticing unit different from the presenting light-emitting portion 8 may be provided. For example, as illustrated in FIG. 3, a line 7 surrounding the ten-key from “1” to “9” may be displayed so as to match the current position or the size of the face. It is also possible to indicate the current position and the size of the face in different colors.

[0091] In Embodiment 2, the entire image input unit 26 is used for detecting the entire area or the partial area of the user 10. However, the entire area or the partial area of the user 10 may be detected only by the image input unit 12. The face area of the user 10 may be detected instead of the entire area by using the image from the entire image input unit 26.

[0092] Also, the partial face area may be detected instead of the entire area or the face area of the user 10. As a detecting method of the partial face area, detection is performed by using partial face areas such as a right side, a left side, an upper half, and a lower half of the face learned in advance by using the face detecting unit 14 of Embodiment 1.

[0093] Also, a change of the state of the face orientation may be instructed instead of, or in addition to the position or the size of the face area in Embodiment 1. In this case, when the face is oriented leftward as illustrated in FIG. 14B, a state in which the user faces leftward is indicted to the user as illustrated in FIG. 14A. Also, as illustrated in FIG. 15, presentation information which prompts a change of the face orientation from the leftward direction to the direction looking at the front is displayed. The face orientations such as upward, downward, leftward, rightward, and frontward are prepared in advance by orientation as the learning data in advance when detecting the face area, and determination is made depending on the orientation at which the face is detected. It is also possible to detect the face feature points, and then calculate the face orientation on the basis of the coordinate thereof.

[0094] In the above-described embodiments, the keys 3 are arranged in the lattice pattern. However, the invention is not limited thereto and other types of arrangement are also applicable as long as the keys or the switches are arranged in the matrix pattern.

[0095] While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions, and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.