Biometrics apparatus, biometrics method, and recording medium

[0001] This application is a National Stage Entry of PCT/JP2021/040336 filed on Nov. 2, 2021, the contents of all of which are incorporated herein by reference, in their entirety.

[0002] This disclosure relates to technical fields of a biometrics apparatus, a biometrics method, and a recording medium.

[0003] Patent literature 1 describes a technique/technology of comparing vectors respectively extracted from face areas selected by two face images to be compared, thereby to acquire a face recognition score. Furthermore, Patent literature 2 describes a technique/technology of determining a position variation of a background separation image and of preventing an unsuitable image from being used in a processing after generation of the background separation image, by prohibiting a recognition processing in a case where an index that provides a determination of being stationary is less than or equal to a certain number, or by discarding the background separation image without using it as training data in a case where a magnitude of the position variation of the background separation image is greater than a certain threshold. Furthermore, Patent literature 3 describes a technique/technology of determining a recognition error and notifying a display unit, a speaker, or the like of the fact and reason of the recognition error, in a case where a degree of coincidence or a degree of correlation with recognition information in reliability information is determined to be lower than a predetermined value, and in a case where an amplitude of a pulse wave signal of survival information in the reliability information is determined to be smaller than a predetermined amplitude. In addition, Patent literature 4 describes a technique/technology of setting a different threshold for a success in face recognition in accordance with the number of registered face images or feature quantities such that a calculation criterion of a recognition score varies depending on the number of registered face images or feature quantities. Furthermore, Patent literature 5 describes a technique/technology of correcting a score that is a collation/verification result of face verification, on the basis of a score distribution. In addition, Non-Patent Literature 1 describes a technique/technology of preventing incorrect/false recognition caused by inputting an image that is unsuitable for recognition by utilizing the fact that images in which individuality is lost, have similar feature quantities, in face recognition in which a cosine similarity is used as a matching score.

• [0004] Patent Literature 1: JP2017-517076A
• [0005] Patent Literature 2: JP2020-129298A
• [0006] Patent Literature 3: International Publication No. WO2016/067556
• [0007] Patent Literature 4: JP2013-142930A
• [0008] Patent Literature 5: JP2008-40874A

• [0009] Non-Patent Literature 1: Siqi Deng, “Harnessing Unrecognizable Faces for Face Recognition” (arXiv 2021)

[0010] It is an example object of this disclosure to provide a biometrics apparatus, a biometrics method, and a recording medium that are intended to improve the techniques/technologies described in Citation List.

[0011] A biometrics apparatus according to an example aspect of this disclosure includes: an extraction unit that extracts a target feature quantity that is a feature quantity of target data used to perform a recognition operation; a calculation unit that calculates an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and a determination unit that determines whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0012] A biometrics method according to an example aspect of this disclosure includes: extracting a target feature quantity that is a feature quantity of target data used to perform a recognition operation; calculating an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and determining whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0013] A recording medium according to an example aspect of this disclosure is a recording medium on which a computer program that allows a computer to execute a biometrics method is recorded, the biometrics method including: extracting a target feature quantity that is a feature quantity of target data used to perform a recognition operation;

• • [0014] calculating an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and
  • [0015] determining whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0021] Hereinafter, a biometrics apparatus, a biometrics method, and a recording medium according to example embodiments will be described with reference to the drawings.

[0022] First, a biometrics apparatus, a biometrics method, and a recording medium according to a first example embodiment will be described. The following describes the biometrics apparatus, the biometrics method, and the recording medium according to the first example embodiment, by using a biometrics apparatus 1 to which the biometrics apparatus, the biometrics method, and the recording medium according to the first example embodiment are applied.

[1-1: Configuration of Biometrics Apparatus 1]

[0023] FIG. 1 is a block diagram illustrating a configuration of the biometrics apparatus 1 in the first example embodiment. As illustrated in FIG. 1, the biometrics apparatus 1 includes an extraction unit 11, a calculation unit 12, and a determination unit 13.

[0024] The extraction unit 11 extracts a target feature quantity that is a feature quantity of target data BD used to perform a recognition operation. The target data are data about a recognition target. The calculation unit 12 calculates an unsuitability score indicating a degree of similarity between an unsuitability feature quantity and the target feature quantity. The unsuitability feature quantity is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate accuracy of a recognition operation due to having the same type of unsuitability for recognition. The determination unit 13 determines whether or not the target data BD are unsuitable for the recognition operation, in accordance with the unsuitability score.

[0025] In biometrics, it is known that the accuracy of the recognition operation is deteriorated and incorrect recognition is increased when the recognition is performed by using data of particular properties. Such data that deteriorate the accuracy of the recognition operation are referred to as data having the unsuitability for recognition. The data having the unsuitability for recognition may be an image in which a factor that prevents an adequate recognition operation, is included with the recognition target, in a case where the target data are an image about the recognition target, for example. A specific example in this situation includes an image captured in a condition where the recognition target is unsuitable, an image captured in a condition where a surrounding environment of the recognition target is unsuitable, an image captured in a condition where setting or the like of an imaging device is unsuitable, or the like. In addition, the data having the unsuitability for recognition may be an image in which the recognition target is included in a condition where individuality of the recognition target is lost, or an image in which the recognition target is not actually included, or the like. A specific example includes an image in which a main part of the recognition target is hidden, an image in which the surrounding of the recognition target is extremely bright or dark, an image in which a boundary between the recognition target and the surrounding environment is hard to see, an image in which an object is not considerably focused, an image of remarkably low resolution, an image captured by a camera with a dirty lens, or the like. In addition, the data having the unsuitability for recognition may be an image that is possibly erroneously determined to include a person's face and is possibly acquired as a face image, in a case where the target data are the face image in which the person's face is captured. A specific example in such a case includes an image of a painting such as a portrait, an image of a statue such as a sculpture, a face image of an animal such as a monkey, an image of a pattern that looks like a person's face, or the like.

[0026] In a case where an individual is recognized by using the data having the unsuitability for recognition, another person could be incorrectly/falsely recognized. In this way, in a case where the target data are the data having the unsuitability for recognition, the data are unsuitable for the recognition operation, and it is thus not desirable to use the target data for the recognition operation.

[0027] The data having the unsuitability for recognition have similar feature quantities. Furthermore, of the unsuitability for recognition, the respective feature quantities of the data having the same type of unsuitability for recognition are particularly similar. Therefore, in a case where the unsuitability feature quantity that is set on the basis of the feature quantity of each piece of a plurality of sample data having the same type of unsuitability for recognition, is similar to the feature quantity of the target data used to perform the recognition operation, the target data are considered to have the corresponding type of unsuitability for recognition. Since the target data are considered to possibly deteriorate the accuracy of the recognition operation due to having the unsuitability for recognition, it is desirable not to use the target data for the recognition.

[0028] Therefore, the biometrics apparatus 1 in the first example embodiment calculates the unsuitability score indicating the degree of similarity between the unsuitability feature quantity and the target feature quantity, and determines whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score. The unsuitability feature quantity is a feature quantity that is set on the basis of the feature quantity of each piece of the plurality of sample data having the same type, i.e., one type of unsuitability for recognition. Therefore, the unsuitability feature quantity is superior, in representing one type of unsuitability for recognition, to a feature quantity that is set on the basis of the feature quantity of each piece of the plurality of sample data having a plurality of types of unsuitability for recognition.

[1-2: Technical Effect of Biometrics Apparatus 1]

[0029] As described above, the biometrics apparatus 1 in the first example embodiment is configured to accurately determine whether or not the target data are unsuitable for the recognition operation, and it is thus possible to prevent the incorrect recognition.

[0030] Next, a biometrics apparatus, a biometrics method, and a recording medium according to a second example embodiment will be described. The following describes the biometrics apparatus, the biometrics method, and the recording medium according to the second example embodiment, by using a biometrics apparatus 2 to which the biometrics apparatus, the biometrics method, and the recording medium according to the second example embodiment are applied.

[2-1: Configuration of Biometrics Apparatus 2]

[0031] First, with reference to FIG. 2, a configuration of the biometrics apparatus 2 in the second example embodiment will be described. FIG. 2 is a block diagram illustrating the configuration of the biometrics apparatus 2 in the second example embodiment. In the following description, the components already described carry the same reference numerals, and a detailed description thereof will be omitted.

[0032] As illustrated in FIG. 2, the biometrics apparatus 2 includes an arithmetic apparatus 21 and a storage apparatus 22. In addition, the biometrics apparatus 2 may include a communications apparatus 23, an input apparatus 24, and an output apparatus 25. The biometrics apparatus 2, however, may not include at least one of the communication apparatus 23, the input apparatus 24, and the output apparatus 25. The arithmetic apparatus 21, the storage apparatus 22, the communication apparatus 23, the input apparatus 24, and the output apparatus 25 may be connected through a data bus 26.

[0033] The arithmetic apparatus 21 includes at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a FPGA (Field Programmable Gate Array), for example. The arithmetic apparatus 21 reads a computer program. For example, the arithmetic apparatus 21 may read a computer program stored in the storage apparatus 22. For example, the arithmetic apparatus 21 may read a computer program stored by a computer-readable and non-transitory recording medium, by using a not-illustrated recording medium reading apparatus provided in the biometrics apparatus 2 (e.g., the input apparatus 24 described later). The arithmetic apparatus 21 may acquire (i.e., download or read) a computer program from a not-illustrated apparatus disposed outside the biometrics apparatus 2 through the communication apparatus 23 (or another communication apparatus). The arithmetic apparatus 21 executes the read computer program. Consequently, a logical functional block for performing an operation to be performed by the biometrics apparatus 2 is realized or implemented in the arithmetic apparatus 21. That is, the arithmetic apparatus 21 is allowed to function as a controller for realizing or implementing the logical functional block for performing an operation (in other words, a processing) to be performed by the biometrics apparatus 2.

[0034] FIG. 2 illustrates an example of the logical functional block realized or implemented in the arithmetic apparatus 21 to perform a biometrics operation. As illustrated in FIG. 2, an extraction unit 211 that is a specific example of the “extraction unit”, a calculation unit 212 that is a specific example of the “calculation unit”, a determination unit 213 that is a specific example of the “determination unit”, a presentation unit 214 that is a specific example of the “presentation unit”, a correction unit 215 that is a specific example of the “correction unit”, and a recognition unit 216, are realized or implemented in the arithmetic apparatus 21.

[0035] Details of operation of each of the extraction unit 211, the calculation unit 212, the determination unit 213, the presentation unit 214, and the correction unit 215 will be described in detail later with reference to FIG. 3 and FIG. 4. The arithmetic apparatus 21, however, may not include at least one of the presentation unit 214, the correction unit 215, and the recognition unit 216.

[0036] The storage apparatus 22 is configured to store desired data. For example, the storage apparatus 22 may temporarily store a computer program to be executed by the arithmetic apparatus 21. The storage apparatus 22 may temporarily store data that are temporarily used by the arithmetic apparatus 21 when the arithmetic apparatus 21 executes the computer program. The storage apparatus 22 may store data that are stored by the biometrics apparatus 2 for a long time. The storage apparatus 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk apparatus, a magneto-optical disk apparatus, a SSD (Solid State Drive), and a disk array apparatus. That is, the storage apparatus 22 may include a non-transitory recording medium.

[0037] The storage apparatus 22 may store a feature quantity generation model GM, a registered feature quantity CC, and an unsuitability feature quantity WC. The storage apparatus 22, however, may not store the feature quantity generation model GM, the registered feature quantity CC, and the unsuitability feature quantity WC. The feature quantity generation model GM and the registered feature quantity CC will be described later.

[0038] The communication apparatus 23 is configured to communicate with an external apparatus of the biometrics apparatus 2 through a not-illustrated communication network.

[0039] The input apparatus 24 is an apparatus that receives an input of information to the biometrics apparatus 2 from an outside of the biometrics apparatus 2. For example, the input apparatus 24 may include an operating apparatus (e.g., at least one of a keyboard, a mouse, and a touch panel) that is operable by an operator of the biometrics apparatus 2. For example, the input apparatus 24 may include a reading apparatus that is configured to read information recorded as data on a recording medium that is externally attachable to the biometrics apparatus 2.

[0040] The output apparatus 25 is an apparatus that outputs information to the outside of the biometrics apparatus 2. For example, the output apparatus 25 may output the information as an image. That is, the output apparatus 25 may include a display apparatus (a so-called display) that is configured to display an image indicating the information that is desirably outputted. For example, the output apparatus 25 may output information as audio. That is, the output apparatus 25 may include an audio apparatus (a so-called speaker) that is configured to output audio. For example, the output apparatus 25 may output information onto a paper surface. That is, the output apparatus 25 may include a print apparatus (a so-called printer) that is configured to print desired information on the paper surface.

[0041] In the second example embodiment, each of the target data BD and the plurality of sample data may be a biological image in which a living body that is a target of the recognition operation is included. The unsuitability for recognition may include at least one of (i) unsuitability that the living body is included in the biological image in a condition where at least a part of the living body is blurred, (ii) unsuitability that the living body is included in the biological image in a condition where at least a part of the living body is shielded by a shield object, and (iii) unsuitability that the biological image is superimposed with a noise.

[0042] The unsuitability feature quantity WC includes at least a first unsuitability feature quantity 1WC, which is set on the basis of the feature quantity of each piece of a plurality of first sample data that deteriorate the accuracy of the recognition operation due to having a first type of unsuitability for recognition, and a second unsuitability feature quantity 2WC, which is set on the basis of the feature quantity of each piece of a plurality of second sample data that deteriorate the accuracy of the recognition operation due to having a second type of unsuitability for recognition, which is different from the first type. For example, the first type of unsuitability for recognition may be one of the above (i) to (iii), and the second type of unsuitability for recognition may be another one of the above (i) to (iii). In addition, there may be a third type of unsuitability for recognition, and the third type of unsuitability for recognition may be one other than the first type of unsuitability for recognition, and the second type of unsuitability for recognition that are the above (i) to (iii). In addition, there may be a fourth or more types of unsuitability for recognition. In an example illustrated below, a case where there are the first type of unsuitability for recognition and the second type of unsuitability for recognition will be described.

[0043] Before the biometrics operation, the feature quantity generation model GM is constructed, the registered feature quantity CC is registered, and the unsuitability feature quantity WC is registered.

[2-2: Construction of Feature Quantity Generation Model GM]

[0044] The feature quantity generation model GM is a model capable of generating a feature quantity of the biological image when the biological image is inputted. The feature quantity generation model GM may be constructed by machine learning to generate the same feature quantity in a case where the biological image of the same individual is inputted. Specifically, the feature quantity generation model GM may be constructed by adjusting a parameter of the feature quantity generation model GM to reduce (preferably, minimize) a loss function that is set on the basis of an error of a plurality of feature quantities generated from the biological image of the same individual. The feature quantity generation model GM may be constructed as a convolution neural network that generates the feature quantity by a convolution processing, for example. The feature quantity generation model GM may be, for example, VGG or RewNet, and may be learned/trained by using DeepFace orArcFace. The feature quantity generation model GM may be a model capable of accurately generating the feature quantity, and may be another learned/trained engine.

[0045] For example, there are assumed to be the three types of unsuitability for recognition (i), (ii), and (iii). Assumed is a case where a decision model for determining whether or not the data have each of the types of the unsuitability is constructed and used by machine learning. In this case, it is necessary to construct three types of decision models: a decision model for the unsuitability for recognition (i), a decision model for the unsuitability for recognition (ii), and a decision model for the unsuitability for recognition (iii). Apart from each decision model, there is also a need to construct a model for the biometrics, such as a model for generating the feature quantity of the biological image.

[0046] In contrast, the biometrics apparatus 2 in the second example embodiment may prepare only the feature quantity generation model GM that is machine-learned and constructed to generate the feature quantity to be used in the recognition. That is, in the biometrics apparatus 2 in the second example embodiment, the model to be constructed is only one type, which is the feature quantity generation model GM. This feature quantity generation model GM generates the registered feature quantity CC that is a feature quantity of registered data CD, and generates a target feature quantity BC that is a feature quantity of the target data. In addition, the feature quantity generation model GM also generates the feature quantity of each piece of the plurality of sample data having the unsuitability for recognition. The unsuitability feature quantity WC is set on the basis of the feature quantity generated by the feature quantity generation model GM.

[0047] Therefore, the biometrics apparatus 2 in the second example embodiment has a simple configuration. The generation of each of the registered feature quantity CC, the target feature quantity BC, and the feature quantity of each piece of the plurality of sample data having the unsuitability for recognition, will be described later with reference to FIG. 3 and FIG. 4.

[2-3: Operation of Setting Unsuitability Feature Quantity WC Performed by Biometrics Apparatus 2]

[0048] Next, with reference to FIG. 3, the operation of setting the unsuitability feature quantity WC performed by the biometrics apparatus 2 in the second example embodiment will be described. FIG. 3 is a flowchart illustrating a flow of the operation of setting the unsuitability feature quantity WC performed by the biometrics apparatus 2 in the second example embodiment.

[0049] As illustrated in FIG. 3, the extraction unit 211 acquires a plurality of sample data that deteriorate the accuracy of the recognition operation (step S31). The extraction unit 211 determines whether or not there are sample data that are not processed (step S32).

[0050] As a result of the determination in the step S32, when there are sample data that are not processed (the step S32: Yes), the extraction unit 211 extracts the feature quantities of a plurality of sample data having the same type of unsuitability for recognition unsuitability for recognition from the sample data that are not processed (step S33). The extraction unit 211 may extract the feature quantities from the plurality of sample data by using the feature quantity generation model GM.

[0051] The extraction unit 211 sets the unsuitability feature quantity WC on the basis of the feature quantity of each piece of the plurality of sample data having the same type of unsuitability for recognition (step S34), and stores it in the storage apparatus 22 (step S35). Here, the unsuitability feature quantity WC may be, for example, a mean value of the respective feature quantities of the plurality of sample data. For example, in a case where the feature quantity of each piece of the plurality of sample data can be expressed by a vector, the unsuitability feature quantity WC may be a normalized mean vector of respective feature vectors of the plurality of sample data. In a case where the unsuitability feature quantity WC is a mean value, it is possible to generate the unsuitability feature quantity that equally includes the unsuitability for recognition possessed by each piece of the sample data, and that well represents the feature quantity that is unsuitable for the recognition. The unsuitability feature quantity WC may be a representative feature quantity of the unsuitability for recognition, and may be obtained by an arbitrary method, such as calculation of a weighted mean value.

[0052] After the step S35, the processing proceeds to the step S32. As a result of the determination, when there are no sample data that are not processed (the step S32: No), the operation of setting the unsuitability feature quantity WC is ended.

[0053] For example, in a case where the sample data having two types of unsuitability for recognition are included, the operation in the step S33 to the step S35 is repeated twice.

[2-4: Biometrics Operation Performed by Biometrics Apparatus 21]

[0054] Next, with reference to FIG. 4, the biometrics operation performed by the biometrics apparatus 2 in the second example embodiment will be described. FIG. 4 is a flowchart illustrating a flow of the biometrics operation performed by the biometrics apparatus 2 in the second example embodiment.

[0055] As illustrated in FIG. 4, the extraction unit 211 acquires the biological image serving as the target data BD (step S21). The extraction unit 211 extracts the target feature quantity BC that is the feature quantity of the biological image (step S22). The extraction unit 211 may extract the target feature quantity BC from target data BD by using the feature quantity generation model GM.

[0056] The calculation unit 212 calculates, as an unsuitability score WS, a first unsuitability score 1WS indicating a degree of similarity between the first unsuitability feature quantity 1WC and the target feature quantity BC, and a second unsuitability score 2WS indicating a degree of similarity between the second unsuitability feature quantity 2WC and the target feature quantity BC (steps S23).

[0057] The determination unit 213 determines whether or not the target data BD are unsuitable for the recognition operation, in response to the first unsuitability score 1WS and the second unsuitability score 2WS (step S24).

[0058] The feature quantities of the sample data having the unsuitability for recognition (i) are similar. Furthermore, the feature quantities of the sample data having the unsuitability for recognition (ii) are similar. In addition, the feature quantities of the sample data having the unsuitability for recognition (iii) are similar. On the other hand, the feature quantities of the sample data having the unsuitability for recognition (i), the feature quantities of the sample data having the unsuitability for recognition (ii), and the feature quantities of the sample data having the unsuitability for recognition (iii) are not similar to one another. Accordingly, in a case where the target data BD have the unsuitability for recognition (i), the target feature quantity BC that is the feature quantity of the target data BD, is similar to the feature quantity of the sample data having the unsuitability for recognition (i). Furthermore, in a case where the target data BD have the unsuitability for recognition (ii), the target feature quantity BC is similar to the feature quantity of the sample data having the unsuitability for recognition (ii). In addition, in a case where the target data BD have the unsuitability for recognition (iii), the target feature quantity BC is similar to the feature quantity of the sample data having the unsuitability for recognition (iii). That is, the determination unit 213 is allowed to determine that the target data BD whose feature quantity is similar to the unsuitability feature quantity WC set from the feature quantity of the sample data having any of the types of unsuitability for recognition, are unsuitable for the recognition operation, due to the unsuitability for recognition that causes the similar feature quantity.

[0059] For example, when the feature quantity generation model GM is expressed by a function f, the following can be satisfied;

• • [0060] Feature quantity of sample data having unsuitability for recognition=f (Sample data having unsuitability for recognition), and
  • [0061] Target feature quantity BC=f (Target data BD).

[0062] The sample data having the unsuitability for recognition and the target data BD have a size of W100×H100×3 ch, for example. In addition, the feature quantity of the sample data having the unsuitability for recognition and the target feature quantity BC can be expressed by a 100-dimensional vector, for example. In this instance, the unsuitability feature quantity WC set on the basis of the feature quantities of the plurality of sample data having the unsuitability for recognition, can also be expressed by a 100-dimensional vector. A norm of each feature quantity may be 1.

[0063] In this instance, the calculation unit 212 may calculate the unsuitability score WS of the unsuitability feature quantity WC and the target feature quantity BC, by using a cosine similarity=<WC, BC> (an inner product of the unsuitability feature quantity WC and the target feature quantity BC). The cosine similarity corresponds to a value of cos(O) when an angle formed by two vectors is 0, and has such a property that it is a maximum value of 1 when the two vectors are the same, that it is 0 when the two vectors are orthogonal, and that it is a minimum value of −1 when the two vectors are oppositely directed. That is, the cosine similarity utilizes such a property that the feature quantities of data on the same individual are similar and tend to be directed in the same direction. In a case where the cosine similarity is used, the unsuitability score WS may take a value of −1 and 1. Hereinafter described is a case where the unsuitability score WS may take a value of −1 and 1.

[0064] The branching of a processing related to a value of the unsuitability score WS can be determined by a user as appropriate. For example, the determination unit 213 may determine that the corresponding target data BD are absolutely unsuitable for the recognition when one or more unsuitability scores WS of the corresponding target data BD are greater than or equal to a first value. Then, the biometrics apparatus 2 may end the processing without calculating a matching score CS with the registered feature quantity CC. For example, when the first value is 0.7, the first unsuitability score 1WS is 0.75, and the second unsuitability score 2WS is −0.8, since one or more unsuitability scores WS are greater than or equal to the first value, the corresponding target data BD are determined to be absolutely unsuitable for the recognition, and the processing may be ended without calculating the matching score CS with the registered feature quantity CC.

[0065] As a result of the determination in the step S24, when the target data BD are unsuitable for the recognition operation (the step S24: Yes), the determination unit 213 determines a reason why the target data BD are unsuitable for the recognition operation, in accordance with the first unsuitability score 1WS and the second unsuitability score 2WS (step S25). The determination unit 213 determines that the target data BD are unsuitable for the recognition operation is a first type of unsuitability, in a case where the first unsuitability score 1WS and the second unsuitability score 2WS indicate that the degree of similarity between the first unsuitability feature quantity 1WC and the target feature quantity BC is higher than the degree of similarity between the second unsuitability feature quantity 2WC and the target feature quantity BC. On the other hand, the determination unit 213 determines that the target data BD are unsuitable for the recognition operation is a second type of unsuitability, in a case where the first unsuitability score 1WS and the second unsuitability score 2WS indicate that the degree of similarity between the second unsuitability feature quantity 2WC and the target feature quantity BC is higher than the degree of similarity between the first unsuitability feature quantity 1WC and the target feature quantity BC.

[0066] The presentation unit 214 presents the reason why the target data BD are unsuitable for the recognition operation (step S26).

[0067] On the other hand, as a result of the determination in the step S24, when the target data BD are not unsuitable for the recognition operation (the step S24: No), the calculation unit 212 calculates the matching score CS indicating a degree of similarity between the target feature quantity BC and the registered feature quantity CC registered in advance (step S27).

[0068] Here, the extraction unit 211 may extract and register the registered feature quantity CC in advance, by using the feature quantity generation model GM. The feature quantity generation model GM may receive an input of the registered data CD and may generate the registered feature quantity CC that is a feature quantity of the registered data CD. The generated verification feature quantity CC may be stored and registered in the storage apparatus 22.

[0069] The recognition unit 216 may determine whether or not an identical person in accordance with the value of the matching score CS, as the biometrics. For example, the registered feature quantity CC and the target feature quantity BC may also be represented by vectors, and the determination unit 213 obtains an inner product of the registered feature quantity CC and the target feature quantity BC and may calculate the cosine similarity to be used as the matching score CS. In this instance, the matching score CS may take a value of −1 to 1. For example, when the matching score CS is greater than or equal to 0.5, a person may be determined to be an identical person, and when it is less than 0.5, the person may be determined to be another person. A value used for this determination may be arbitrarily determined by the user in accordance with the requirements.

[0070] That is, the calculation unit 212 that calculates the unsuitability score and the calculation unit 212 that calculates the matching score may have the same mechanism. The calculation unit 212 may calculate the unsuitability score WS indicating the degree of similarity between the unsuitability feature quantity WC and the target feature quantity BC, or may calculate the matching score CS indicating the degree of similarity between the target feature quantity BC and the registered feature quantity CC registered in advance. The calculation unit 212 is allowed to calculate the score by the same operation, even when inputted data are different. The calculation unit 212 may not calculate the unsuitability score WS and the matching score CS by using the cosine similarity, and may obtain a degree of similarity of an inter-vector distance such as Euclidean space distance, and may perform the calculation by any known method.

[0071] The correction unit 215 corrects the matching score CS in a case where the determination unit 213 determines that there is a possibility that the target data BD are unsuitable for the recognition (step S28). For example, it may be determined that there is a possibility that the corresponding target data BD are unsuitable for the recognition when one or more unsuitability scores WS of the target data BD are less than the first value and are greater than or equal to less than a second value, and the calculated value of the matching score CS may be corrected to be low. For example, when the first value is 0.7, the second value is −0.7, the first unsuitability score 1WS is 0.5, and the second unsuitability score 2WS is −0.8, one or more unsuitability scores WS are less than the first value and are greater than or equal to the second value, and it may be thus determined that there is a possibility that the corresponding target data BD are unsuitable for the recognition, and the calculated value of the matching score CS may be corrected to be low. On the other hand, for example, it may be determined that the corresponding target data BD are adequate for the recognition when all the unsuitability scores WS of the target data BD are less than the second value, and the calculated value of the matching score CS may not be corrected.

[0072] The correction unit 215 may correct the matching score CS to be lower as the value of the unsuitability score WS increases. That is, the correction unit 215 may correct the matching score CS such that authenticity of a person is evaluated smaller. In a case where there are a plurality of types of unsuitability for recognition, a method of correcting the matching score CS may be determined on the basis of the highest value of the unsuitability score WS. Correcting the matching score CS to be lower by the correction unit 215 may be linear transformation, or may be nonlinear transformation such as gamma correction, for example. For example, the correction unit 215 may perform the correction as follows: reducing the value of the matching score by 5% when one or more unsuitability scores are 0.65, reducing the value of the matching score by 3% when one or more unsuitability scores are 0, reducing the value of the matching score by 1% when one or more unsuitability scores are −0.65, and the like. In this case, for example, even if the matching score with certain registered data CD is 0.9, in a case where one or more first unsuitability scores 1WS are 0, the correction unit 215 corrects the corresponding matching score to 0.63. That is, the evaluation of probability of being the same target as the registered data CD, is lowered due to the correction by the correction unit 215.

[0073] Although described is an example in which the correction unit 215 changes reliability of the recognition in accordance with the unsuitability score WS, but in a case where the value of the unsuitability score WS exceeds a threshold, the correction unit 215 may uniformly correct the matching score CS to be low.

[0074] The recognition unit 216 determines whether or not an identical person, in accordance with the corrected matching score CS (step S29). The presentation unit 214 presents a recognition result of the target data BD (a determination result of whether or not the identical person) (step S30).

[2-4: Technical Effect of Biometrics Apparatus 2]

[0075] Since the biometrics apparatus 2 in the second example embodiment determines whether or not the data have a plurality of types of unsuitability for recognition for each type, it is possible to accurately determine whether or not the target data are unsuitable for the recognition operation, in regard to various types of unsuitability for recognition, thereby preventing the incorrect recognition. In addition, the biometrics apparatus 2 in the second example embodiment is configured to determine which target data having which type of unsuitability for recognition are unsuitable for the recognition operation, depending on which data having which type of unsuitability for recognition are adopted as the sample data.

[0076] In addition, since the biometrics apparatus 2 in the second example embodiment calculates the unsuitability score for each type of unsuitability for recognition, it is possible to determine the type of the unsuitability for recognition included in the target data. In a case where the feature quantity of the target data is similar to the feature quantity of data having certain unsuitability for recognition, it may be determined that the corresponding target data include the certain unsuitability for recognition. Therefore, the biometrics apparatus 2 in the second example embodiment is configured not only to determine that the corresponding target data are unsuitable for the recognition operation, but also to know why the target data are unsuitable for the recognition.

[0077] In addition, since the biometrics apparatus 2 in the second example embodiment calculates the unsuitability score for each type of different unsuitability for recognition, it is possible to determine to what extent the target data include which type of unsuitability for recognition. That is, the biometrics apparatus 2 in the second example embodiment is configured to determine the unsuitability for recognition with the highest unsuitability score, as the reason why the target data are unsuitable for the recognition operation.

[0078] In addition, since the biometrics apparatus 2 in the second example embodiment presents, to the outside, the reason why the target data are unsuitable for the recognition, an executor who performs the biometrics is able to know the reason why the target data are unsuitable for the recognition. By knowing the type of the unsuitability for recognition, the executor may be able to newly consider/examine a recognition mechanism specialized for the type. In addition, the executor may be able to determine that a situation of acquiring the target data is unsuitable, by knowing the type of the unsuitability for recognition. For example, the target data may be determined to be unsuitable for the recognition operation due to a malfunction in a device, such as dirt on a lens, or the like, of a camera that acquires the biological image. In this case, the unsuitability for recognition included in the target data may be improved by the executor adjusting the device.

[0079] In addition, since the biometrics apparatus 2 in the second example embodiment generates the target feature quantity and the unsuitability feature quantity by using the same feature quantity generation model GM, it is possible to accurately determine whether or not the data are unsuitable for the recognition operation. Furthermore, since the biometrics apparatus 2 in the second example embodiment performs the recognition by using the degree of similarity between the feature quantities generated by using the same feature quantity generation model GM, it is possible to accurately perform the biometrics.

[0080] Furthermore, since the biometrics apparatus 2 in the second example embodiment is configured to correct the matching score in a case where the target data include the unsuitability for recognition, it is possible to further increase the accuracy of the recognition. For example, when the unsuitability score of the target data is greater than or equal to or greater than a certain threshold, the determination unit 213 may determine that the target data are not used as data to be used for matching. On the other hand, when the determination unit 213 determines that the target data include the unsuitability for recognition, but are data usable for matching, the correction unit 215 may correct the matching score, thereby making it hard for the recognition unit 216 to determine that a person is an identical person.

[0081] Especially in the recognition of a living body, the incorrect recognition caused by an unsuitable image should be prevented. The biometrics apparatus 2 in the second example embodiment is configured to accurately recognize the biological image.

[0082] In the second example embodiment, described is a case where the target data BD are the biological image. The biological image may include a face image, an iris image, a fingerprint image, another image acquired from a living body, or the like. The target data BD may be data other than the biological image, and may be, for example, audio data other than the image. In the case of the audio data, the data having the unsuitability for recognition that deteriorates the accuracy of the recognition operation, can be considered to be data that do not include speech in which a noise tends to be intense.

[0083] Next, a biometrics apparatus, a biometrics method, and a recording medium according to a third example embodiment will be described. The following describes the biometrics apparatus, the biometrics method, and the recording medium according to the third example embodiment, by using a biometrics apparatus 3 to which the biometrics apparatus, the biometrics method, and the recording medium according to the third example embodiment are applied.

[0084] The biometrics apparatus 3 in the third example embodiment may have the same configuration as that of the biometrics apparatus 2 in the second example embodiment described above. The biometrics apparatus 3 in the third example embodiment is different in the type of the unsuitability for recognition, from the biometrics apparatus 2 in the second example embodiment. Other characteristics of the biometrics apparatus may be the same as those of the biometrics apparatus 2

[0085] In the third example embodiment, each of the target data BD and the plurality of sample data is an image in which a living body that is a target of the recognition operation living body is included. In the third example embodiment, the unsuitability for recognition includes at least one of (iv) unsuitability for recognition that the living body is included in the image in a condition where a predetermined area of at least a part of the living body is shielded, and (v) unsuitability for recognition that the living body is included in the image in a condition where another area that is different from the predetermined area of at least a part of the living body, is shielded. The predetermined area of at least a part of the living body may include a surrounding area of the living body. As a specific example, each of the target data BD and the plurality of sample data may be a face image in which a face of the living body that is the target of the recognition operation, is included. In this instance, the unsuitability for recognition may include at least one of (iv) unsuitability that the face is included in the face image in a condition where at least a part of the face is shielded by a mask worn by the living body, and (v) unsuitability that the face is included in the face image in a condition where at least a part of the face is shielded by a shield object that is different from the mask. In this specific example, an area shielded by the mask worn by the living body, may be an example of the predetermined area of at least a part of the living body, including a surrounding area of the living body.

[3-1: Biometrics Operation Performed by Biometrics Apparatus 3]

[0086] Hereinafter, a flow of the biometrics operation in the third example embodiment will be described with reference to FIG. 5. FIG. 5 is a flowchart illustrating the flow of the biometrics operation in the third example embodiment. In the following description, the processing steps already described carry the same step numbers, and a detailed description thereof will be omitted.

[0087] As illustrated in FIG. 5, even in the third example embodiment, as in the second example embodiment, the information processing apparatus 3 performs the operation from the step S21 to the step S25. In the third example embodiment, the determination unit 213 determines whether or not the reason why the target data BD are unsuitable for the recognition operation, which is determined in the step S25, is (iv) that the face is included in the face image in a condition where at least a part of the face is shielded by a mask worn by the living body (step S41).

[0088] As a result of the determination in step S41, when the reason why the target data BD are unsuitable for the recognition operation is (iv) (the step S41: Yes), the determination unit 213 determines that the corresponding target data BD are not unsuitable for the recognition operation. Specifically, the determination unit 213 may determine to perform, on the corresponding target data BD, face image recognition only for the face image in a condition where at least a part of the face is not shielded by the mask (step S42).

[0089] On the other hand, as a result of the determination in the step S41, when the reason why the target data BD are unsuitable for the recognition operation is not (iv) (the step S41: No), the processing proceeds to the step S26.

[0090] The face image in which the face is included in a condition where at least a part of the face is shielded by a shield object, is unsuitable to be used to perform a face recognition operation in many cases. For example, a face image of the living body wearing sunglasses by which the eyes and around the eyes are shielded, is considered to be unsuitable target data. These days, however, a wearing rate of the mask is so high that there are many cases where the face is included in the face image in a condition where at least a part of the face is shielded by the mask worn by the living body (hereinafter referred to as a “mask wearing image”). In addition, frequently wearing and taking off the mask for the biometrics is not desirable from the viewpoint of preventing the spread of infectious diseases. Note that the above-described area shielded by the sunglasses may be an example of another area that is different from the predetermined area of at least a part of the living body.

[0091] In response to this, progress is made in development of a mechanism that is configured to accurately recognize the mask wearing image. For example, it is possible to construct a recognition model capable of performing the recognition from the mask wearing image, by performing machine learning using a large amount of mask wearing images.

[3-2: Technical Effect of Biometrics Apparatus 31]

[0092] That is, even for the mask wearing image, it is possible to accurately perform the face recognize by using a dedicated mechanism. Therefore, it is possible to accurately recognize the mask wearing image, by distinguishing the mask wearing image from a shield image in which the living body is shielded by a shield object that is different from the mask such as sunglasses, for example.

[0093] The biometrics apparatus in the third example embodiment does not determine that the mask wearing image is an image that is eventually unsuitable for the recognition, but allows the mechanism suitable for the recognition of the mask wearing image, to perform the recognition. This allows the biometrics of a recognized target person, without the recognized target person taking off the mask.

[0094] For example, in a situation where the police use the face recognition, a worst-case scenario that should not happen is identifying an unrelated person as a person to look for due to the incorrect recognition, or the like. There are, however, some cases where an unrelated person is arrested due to the incorrect recognition, and it is mandatory to take countermeasures.

[0095] The biometrics apparatus described above, is configured to accurately determine whether or not the target data are unsuitable for the recognition operation, thereby preventing the incorrect recognition.

[0096] With respect to the example embodiment described above, the following Supplementary Notes are further disclosed.

[0097] A biometrics apparatus including:

• • [0098] an extraction unit that extracts a target feature quantity that is a feature quantity of target data used to perform a recognition operation;
  • [0099] a calculation unit that calculates an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and
  • [0100] a determination unit that determines whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0101] The biometrics apparatus according to supplementary note 1, wherein

• • [0102] the unsuitability feature quantity includes at least a first unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of first sample data that deteriorate the accuracy of the recognition operation due to having a first type of unsuitability for recognition, and a second unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of second sample data that deteriorate the accuracy of the recognition operation due to having a second type of unsuitability for recognition, which is different from the first type,
  • [0103] the calculation unit calculates, as the unsuitability score, a first unsuitability score indicating a degree of similarity between the first unsuitability feature quantity and the target feature quantity, and a second unsuitability score indicating a degree of similarity between the second unsuitability feature quantity and the target feature quantity, and
  • [0104] the determination unit determines whether or not the target data are unsuitable for the recognition operation in accordance with the first and second unsuitability scores.

[0105] The biometrics apparatus according to supplementary note 2, wherein the determination unit determines a reason why the target data are unsuitable for the recognition operation in accordance with the first and second unsuitability scores.

[0106] The biometrics apparatus according to supplementary note 3, wherein the determination unit

• • [0107] determines that the reason why the target data are unsuitable for the recognition operation is the first type of unsuitability, in a case where the first and second unsuitability scores indicate that the degree of similarity between the first unsuitability feature quantity and the target feature quantity is higher than the degree of similarity between the second unsuitability feature quantity and the target feature quantity, and
  • [0108] determines that the reason why the target data are unsuitable for the recognition operation is the second type of unsuitability, in a case where the first and second unsuitability scores indicate that the degree of similarity between the second unsuitability feature quantity and the target feature quantity is higher than the degree of similarity between the first unsuitability feature quantity and the target feature quantity.

[0109] The biometrics apparatus according to supplementary note 3 or 4, further comprising a presentation unit that presents the reason why the target data are unsuitable for the recognition operation.

[0110] The biometrics apparatus according to any one of supplementary notes 1 to 5, wherein

• • [0111] the extraction unit extracts the target feature quantity from the target data by using a feature quantity generation model, and
  • [0112] the unsuitability feature quantity is set on the basis of the feature quantity extracted from the plurality of sample data by using the feature quantity generation model.

[0113] The biometrics apparatus according to any one of supplementary notes 1 to 6, wherein

• • [0114] the calculation unit calculates a matching score indicating a degree of similarity between the target feature quantity and a registered feature quantity registered in advance, and
  • [0115] the biometrics apparatus further comprises a correction unit that corrects the matching score in a case where the determination unit determines that the target data are unsuitable for the recognition operation.

[0116] The biometrics apparatus according to supplementary note 7, wherein

• • [0117] the extraction unit extracts the registered feature quantity, and
  • [0118] the biometrics apparatus further comprises a storage unit that stores the registered feature quantity.

[0119] The biometrics apparatus according to any one of supplementary notes 1 to 8, wherein

• • [0120] each of the target data and the plurality of sample data is a biological image in which a living body that is a target of the recognition operation is included, and
  • [0121] the unsuitability for recognition includes at least one of unsuitability that the living body is included in the biological image in a condition where at least a part of the living body is blurred, unsuitability that the living body is included in the biological image in a condition where at least a part of the living body is shielded by a shield object, and unsuitability that a noise is superimposed on the biological image.

[0122] The biometrics apparatus according to any one of supplementary notes 1 to 9, wherein

• • [0123] each of the target data and the plurality of sample data is a face image in which a face of a living body that is a target of the recognition operation is included, and
  • [0124] the unsuitability for recognition includes at least one of unsuitability that the face is included in the face image in a condition where at least a part of the face is shielded by a mask worn by the living body, and unsuitability that the face is included in the face image in a condition where at least a part of the face is shielded by a shield object that is different from the mask.

[0125] The biometrics apparatus according to any one of supplementary notes 1 to 10, wherein

• • [0126] each of the target data and the plurality of sample data is an image in which a target of the recognition operation is included,
  • [0127] the unsuitability for recognition includes at least one of a third type of unsuitability for recognition that the target is included in the image in a condition where a predetermined area of at least a part of the target is shielded, and a fourth type of unsuitability for recognition that the target is included in the image in a condition another area that is different from the predetermined area of at least a part of the target, is shielded, and
  • [0128] the determination unit determines that the target data are not unsuitable for the recognition operation, in a case where it determines that the reason why the target data are unsuitable for the recognition operation, is the third type of unsuitability for recognition.

[0129] The biometrics apparatus according to supplementary note 11, wherein

• • [0130] the target is a living body, and
  • [0131] the predetermined area includes a surrounding area of the living body.

[0132] The biometrics apparatus according to any one of supplementary notes 1 to 12, wherein the unsuitability feature quantity is a mean value of feature quantities of the plurality of sample data.

[0133] A biometrics method including:

• • [0134] extracting a target feature quantity that is a feature quantity of target data used to perform a recognition operation;
  • [0135] calculating an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and
  • [0136] determining whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0137] A recording medium on which a computer program that allows a computer to execute a biometrics method is recorded, the biometrics method including:

• • [0138] extracting a target feature quantity that is a feature quantity of target data used to perform a recognition operation;
  • [0139] calculating an unsuitability score indicating a degree of similarity between the target feature quantity and an unsuitability feature quantity that is set on the basis of a feature quantity of each piece of a plurality of sample data that deteriorate an accuracy of the recognition operation due to having the same type of unsuitability for recognition; and
  • [0140] determining whether or not the target data are unsuitable for the recognition operation in accordance with the unsuitability score.

[0141] At least a part of the constituent components of each of the example embodiments described above may be combined with at least another part of the constituent components of each of the example embodiments described above, as appropriate. A part of the constituent components of each of the example embodiments described above may not be used. Furthermore, to the extent permitted by law, all the references (e.g., publications) cited in this disclosure are incorporated by reference as a part of the description of this disclosure.

[0142] This disclosure is allowed to be changed, if desired, without departing from the essence or spirit of this disclosure which can be read from the claims and the entire identification. A biometrics apparatus, a biometrics method, and a recording medium with such changes are also intended to be within the technical scope of this disclosure.

• • [0143] 1, 2 biometrics apparatus
  • [0144] 11, 211 Extraction unit
  • [0145] 12, 212 Calculation unit
  • [0146] 13, 213 Determination unit
  • [0147] 214 Presentation unit
  • [0148] 215 Correction unit
  • [0149] GM Feature quantity generation model
  • [0150] BD Target data
  • [0151] BC Target feature quantity
  • [0152] WC Unsuitability feature quantity
  • [0153] WS Unsuitability score
  • [0154] 1WC First unsuitability feature quantity
  • [0155] 2WC Second unsuitability feature quantity
  • [0156] 1WS First unsuitability score
  • [0157] 2WS Second unsuitability score
  • [0158] CD Registered data
  • [0159] CC Registered feature quantity
  • [0160] CS Matching score