Behavior control apparatus and method

[0001] The present invention relates to a behavior control apparatus and method for mobile unit, in particular, to a behavior control apparatus and method for recognizing a target object in acquired images and controlling behavior of the mobile unit with high accuracy based on the recognized target object.

[0002] To control a mobile unit with high accuracy based on input images, it is necessary for a control system to recognize an object in the image as a target for behavior of the mobile unit. One approach is that the control system learns training data pre-selected by an operator prior to recognition. Specifically, the control system searches the input images to extract some shapes or colors therefrom designated as features of the target. Then, the control system outputs commands to make the mobile unit move toward the extracted target.

[0003] However, it is necessary for the operator to teach features such as shape or color of the target in detail to the control system and therefore preparation for that is a burden in terms of time and labor. In addition, since the control would be interrupted when the target goes off the input image, it is difficult to apply this approach to practical use.

[0004] An alternative approach is that a template for the target is prepared and during controlling the mobile unit the template is always applied to input images to search and extract shape and location of the target in detail. In this case, however, computational cost would become huge because a computer has to keep calculating the shape and location of the target. Furthermore, the calculation of searching the target may fall into a local solution.

[0005] Therefore, to control behavior of the mobile unit efficiently and flexibly, it is preferable to make the mobile unit move autonomously rather than utilizing supervised learning method as a target is specified beforehand. To achieve that, a method for recognizing the target autonomously and learning the location of the target is needed. In Japanese Patent Application Unexamined Publication (Kokai) No. H8-126981, image position recognition method in robot system is disclosed. According to the method, the target object is searched out autonomically even when the target object is missing out of input image due to the error. However, the method requires that work plane for recognizing images is painted with various colors prior to working, which is substantially time-consuming task.

[0006] In Japanese Patent Application Unexamined Publication (Kokai) No. H7-13461, a method for leading autonomous moving robots for managing indoor air-conditioning units is disclosed. According to the method, a target object for leading is detected through image processing and the robot is leaded toward the target. However, the method needs blowing outlets of air-conditioning units as target objects, which lacks generality.

[0007] Therefore, it is objective of the present invention to provide a behavior control apparatus and method for autonomously controlling a mobile unit based on visual information in practical application without the needs of a great deal of preparation or computational cost and limiting the type of target object.

[0008] According to one aspect of the invention, a behavior control apparatus for controlling behavior of a mobile unit is provided. The apparatus comprises sensory input capturing method for capturing sensory inputs and motion estimating method for estimating motion of the mobile unit. The apparatus further comprises target segregation method for segregating the portion which includes a target object to be target for behavior of the mobile unit from sensory inputs, and target object matching method for extracting the target object from the segregated portion. The apparatus still further comprises target location acquiring method for acquiring the location of the target object and behavior decision method for deciding behavior command for controlling the mobile unit based on the location of the target object.

[0009] The behavior control apparatus roughly segregate the portion that includes a target object of behavior from sensory inputs, such as images, based on the estimation of motion. The apparatus then specifies a target object from the portion, acquires location of the target object and output behavior command which moves the mobile unit toward the location. Thus, detailed feature of the target object need not be predetermined. In addition, because the features irrelevant to present behavior are eliminated, the computational load is reduced. Therefore, highly efficient and accurate control for the mobile unit may be implemented.

[0010] As used herein, “mobile unit” refers to a unit which has a driving mechanism and moves in accordance with behavior commands.

[0011] The sensory inputs may be images of the external environment of the mobile unit.

[0012] The motion estimating method comprises behavior command output method for outputting the behavior command and behavior evaluation method for evaluating the result of the behavior of the mobile unit. The motion estimating method further comprises learning method for learning the motion of the mobile unit using the relationship between the sensory inputs and the behavior result and storing method for storing the learning result.

[0013] The behavior control apparatus pre-learns the relationship between sensory inputs and behavior commands. Then the apparatus updates the learning result when new feature is acquired on behavior control stage. The learning result is represented as probabilistic density distribution. Thus, motion of the mobile unit on behavior control stage may be estimated with high accuracy.

[0014] The motion of the mobile unit may be captured using a gyroscope instead of estimating it.

[0015] The target segregation method segregates the portion by comparing the sensory inputs and the estimated motion using such as optical flow. Thus the behavior control apparatus may roughly segregate the portion that includes a target object

[0016] The target location acquiring method defines the center of the target object as the location of the target object and the behavior decision method outputs the behavior command to move the mobile unit toward the location of the target object. Thus the mobile unit may be controlled stably.

[0017] The behavior decision method calculates the distance between the mobile unit and the location of the target object, and deciding the behavior command to decrease the calculated distance. This calculation is very simple and helps to reduce the amount of computation.

[0018] If the calculated distance is greater than a predetermined value, the target segregation method repeats segregating the portion which includes a target object.

[0019] The target object matching method extracts the target object by pattern matching between the sensory inputs and predetermined templates. Thus the target object may be extracted more accurately.

[0020] Other embodiments and features will be apparent by reference to the following description in connection with the accompanying drawings.

[0031] Preferred embodiments of the present invention will be described as follows with reference to the drawings.

[0032] A behavior control apparatus according to the invention recognizes a target object, which is a reference for controlling a mobile unit, from input images and then controls behavior of the mobile unit based on the recognized target object. The apparatus is used as installed on the mobile unit, which has driving mechanism and is movable by itself.

Configuration

[0033] FIG. 1 shows a radio-controlled (RC) helicopter 100 according to one embodiment of the invention. The RC helicopter 100 consists of body 101, main rotor 102 and tail rotor 103. On the body 101 are installed a CCD camera 104, a behavior control apparatus 105 and a servomotor 106. At the base of the tail rotor 103, there is link mechanism 107, which is coupled with the servomotor 106 through a rod 108. The RC helicopter 100 can float in the air by rotating the main rotor 102 and the tail rotor 103.

[0034] The CCD camera 104 takes images of frontal vision of the RC helicopter. Area taken by the camera is showed in FIG. 1 as visual space 109. The behavior control apparatus 105 autonomically recognizes a location of a target object 110 (hereinafter simply referred to as “target location 110”), which is to be a target for behavior control, and also recognizes self-referential point in the visual space 109 based on the image taken by the CCD camera 104. The target location 110 is represented as probabilistic density distribution, as described later, and is conceptually illustrated as ellipse in FIG. 1.

[0035] The RC helicopter 100 is tuned as that only the control of yaw orientation (as an arrow in FIG. 1, around the vertical line) is enabled. Therefore, term “stable” as used herein means that vibration of the RC helicopter's directed orientation is small.

[0036] The behavior control apparatus 105 outputs behavior commands to move the self-referential point (for example, center. This is hereinafter referred to as COM 111, acronym of “center of motion”) of the image captured by CCD camera 104 (the visual space 109) toward the target location 110 in order to control the RC helicopter 100 stably. The behavior commands are sent to the servomotor 106. In response to the behavior commands, the servomotor 106 drives the rod 108, activating the link mechanism to alter the angle of tail rotor 103 so as to rotate the RC helicopter 100 in yaw orientation.

[0037] In the embodiment described above, controllable orientation is limited in one-dimensional operation such that COM moves from side to side for the purpose of simple explanation. However, the present invention may be also applied to position control in two or three dimensions.

[0038] Although the RC helicopter 100 is described as an example of the mobile unit having the behavior control apparatus of the present invention, the apparatus may be installed on any of mobile unit having driving mechanism and being able to move by itself. In addition, the mobile unit is not limited to flying objects like a helicopter, but includes, for example, vehicles traveling on the ground. The mobile unit further includes the unit only the part of which can moves. For example, the behavior control apparatus of the present invention may be installed on industrial robots of which base is fixed to floor, to recognize an operation target of the robot.

[0039] FIG. 2 is a functional block diagram of the behavior control apparatus 105. The behavior control apparatus 105 comprises an image capturing block 202, a behavior command output block 204, a behavior evaluation block 206, a learning block 208, a storage block 210, a target segregation block 212, a matching block 214, a target location acquiring block 216 and a behavior decision block 218. The behavior control apparatus 105 may be implemented by running a program according to the present invention on a general-purpose computer, and it can also be implemented by means of hardware having functionality of the invention.

[0040] The behavior control apparatus 105 first learns relationship between features of inputs (e.g., images taken by the CCD camera 104) and behavior of the mobile unit. These operations are inclusively referred to as “learning stage”. Completing the learning stage, the apparatus may estimate motion of the mobile unit based on the captured images using learned knowledge. The apparatus further searches and extracts target location in the image autonomously using estimated motion. Finally, the apparatus controls the motion of the mobile unit with the reference to the target location. These operations are inclusively referred to as “behavior control stage”.

[0041] It should be noted that the behavior control apparatus 105 shown in FIG. 2 is configured for use on the RC helicopter 100, and the apparatus may be configured in various manner depending on the characteristic of the mobile unit installed thereon. For example, the apparatus may further include a gyroscope sensor. In this case, the apparatus uses the signals generated from the gyroscope sensor to estimate motion of the mobile unit, and uses the sensory input captured by the image capturing block 202 only for recognizing the target location.

Learning

[0042] In learning stage, while moving the mobile unit, the behavior control apparatus 105 learns relationship between features of input images taken by an image pickup device and behavior result in response to behavior command from the behavior command output block 204. The apparatus then stores learning result in the storage block 210. This learning enables the apparatus to estimate motion of the mobile unit accurately based on input images in the behavior control stage described later.

[0043] The image capturing block 202 receives images every predetermined interval from an image pickup device such as CCD camera 104 installed in front of the RC helicopter 100. Then the block 202 extracts features as sensory inputs I_i(t) (i=1,2, . . . ) from the images. This feature extraction may be implemented by any of prior-art approaches such as optical flow. The extracted features are sent to the behavioral evaluation block 206.

[0044] The behavior command output block 204 outputs behavior commands Q_i(t), which directs behavior of the mobile unit. While the learning is immature in initial stage, behavior commands are read from command sequence which is selected randomly beforehand. During the mobile unit moves randomly, the behavior control apparatus 105 may learn necessary knowledge for estimating the motion of the mobile unit. As for the RC helicopter 100 shown in FIG. 1, the behavior commands correspond to driving current of the servomotor 106, which drives link mechanism 107 to change the yaw orientation. The behavior command is sent to driving mechanism such as the servomotor 106 and the behavior evaluation block 206. The relationship between the sensory inputs I_i(t) and the behavior commands Q_i(t) is represented by the following mapping ƒ.
ƒ:I_i(t)Q_i(t)  (1)
where subscript i (i=1,2, . . . ) means i-th data. For example, the mapping ƒ may be given as a non-linear approximation translation using well-known Fourier series or the like.

[0045] In alternative embodiment, the behavior command output block 204 receives signal from an external device and outputs behavior commands in accordance with the signal.

[0046] The behavior evaluation block 206 generates reward depending on both sensory inputs I_i(t) from image capturing block 202 and the behavior result in response to behavior command Q_i(t) based on predetermined evaluation function under a reinforcement learning scheme. The example of the evaluation function is a function that yields reward “1” when the mobile unit controlled by behavior command is stable, otherwise yields reward “2”. After the rewards are yielded, the behavior evaluation block 206 generates a plurality of columns 1,2,3, . . . , m as many as the number of type of the rewards and distributes behavior commands into each column responsive to the type of their rewards. Hereinafter the behavior commands Q_i(t) distributed in column l are denoted as “Q_i^l(t)”. Sensory inputs I_i(t) and behavior command Q_i(t) are supplied to learning block 208 and used for learning the relationship between them.

[0047] The purpose of the evaluation function is to minimize the variance of the behavior commands. In other words, the reinforcement learning satisfying σ(Q¹)<σ(Q²) is executed with the evaluation function. The minimum variance of the behavior commands needs to be reduced for smooth control. Learning with the evaluation function allows the behavior control apparatus 105 to eliminate unnecessary sensory inputs and to learn important sensory inputs selectively.

[0048] In each column, both sensory inputs and the behavior commands are stored according to the type of rewards given to the behavior commands.

[0049] Each column 1,2,3, . . . ,m corresponds to a cluster model of the behavior commands. Each column is used to calculate generative models g(Ω_l) where l denotes the number of attention classes applied. Generative model is a storage model generated through learning, and may be represented by probabilistic density function in statistic learning. Non-linear estimation such as neural network may be used to model g(Ω_l), which gives the estimation of probabilistic density distribution P(Q|Ω_l). In the present embodiment, it is assumed that P(Q|Ω_l) takes the form of Gaussian mixture model, which may make approximation for any of probabilistic density function. FIG. 3 shows the relationship between the number of generative models (horizontal axis) and the minimum variance (vertical axis).

[0050] Only one column will not accelerate the convergence of the learning because, if so, it will take much time until the normal distribution curve of behavior commands stored in the column is sharpened and the variance gets small. In order to control the mobile unit stably, it needs to learn in such a way that the variance of normal distribution of motor output becomes smaller. One feature of the invention is that the normal distribution curve is sharpened rapidly since a plurality of columns are generated. A method utilizing such minimum variance theory is described in Japanese Patent Application Unexamined Publication (Kokai) No. 2001-028758.

[0051] Then the learning process described later is executed in the learning block 208. After the learning process is completed, a behavior command for minimizing the variance of the normal distribution curve of behavior commands for a new sensory input may be selected out of the column by means of a statistical learning scheme, and the rapid stability of the mobile unit may be attained.

[0052] Now the learning process at the learning block 208 will be described in detail.

[0053] The learning block 208 calculates the class of attention Ω_l corresponding one by one to each column l which contains the behavior commands using identity mapping translation. This translation is represented by the following mapping h.
h:Q_i(t)Ω_l(t)  (2)

[0054] The purpose of the class of attention Ω_l is efficient learning by focusing on the particular sensory inputs from massive sensory inputs when new sensory inputs are given. Generally, the amount of sensory inputs far exceeds the processing capacity of the computer. Thus, appropriate filtering for sensory inputs with the classes of attention Ω_l improves the efficiency of the learning. Therefore, the learning block 208 may eliminate the sensory inputs except the selected small subset of them.

[0055] When the learning goes forward, the learning block 208 may know directly the class of attention corresponding to the sensory input using the statistical probability without calculating the mapping f and/or h one by one. More specifically, each of the classes of attention Ω_l is a parameter for modeling the behavior commands Q_i^l(t) stored in each column using the probabilistic density function of the normal distribution. To obtain the probabilistic density function, a mean μ and covariance Σ need to be calculated for the behavior commands Q_i^l(t) stored in each column. This calculation is performed by unsupervised Expectation Maximization (EM) algorithm using clustered component algorithm (CCA), which will be described later. It should be noted that the classes of attention Ω_l are modeled on the assumption that true probabilistic distribution p(I_i(t)|Ω_l) will exists for each class of attention Ω_l.

[0056] Using the obtained parameters, probabilistic density function of each class of attention Ω₁ may be obtained. The obtained density functions are used as prior probability {overscore (p)}(Ω_l(t))(={overscore (p)}(Q_l(t)|Ω_l(t))) of each class of attention before sensory inputs are given. In other words, each class of attention Ω_l is assigned as an element of the probabilistic density function p(Q_i^l(t)|Ω_l,θ).

[0057] After the classes of attention Ω_l are calculated, the learning block 208 learns the relation between the sensory inputs and the classes of attention by means of supervised learning scheme using neural network. More specifically, this learning is executed by obtaining conditional probabilistic density function p_λ(I_i(t)|Ω_l) of the class of attention Ω₁ and the sensory input I_i(t) using hierarchical neural network with the class of attention Ω_l as supervising signal. It should be noted that the class of attention may be calculated by synthetic function f·h . The obtained conditional probabilistic density function p (I_i(t)|Ω_l) corresponds to the probabilistic relation between the sensory input and the class of attention.

[0058] New sensory inputs gained by CCD camera 104 are provided to the behavior control apparatus 105 after the learning is over. The learning block 208 selects the class of attention corresponding to provide sensory input using statistical learning scheme such as bayes' learning. This operation corresponds to calculating conditional probabilistic density function p(Ω_l|I_i(t)) of the class of attention Ω_l relative to the sensory inputs I_i(t). As noted above, since the probabilistic density function of the sensory inputs and the class of attention has been already estimated by the hierarchical neural network, newly given sensory inputs may be directly assigned to particular class of attention. In other words, after the supervised learning with neural network is over, calculation of the mapping ƒ and/or h become unnecessary for selecting class of attention Ω_l relative to sensory input I_i(t).

[0059] In this embodiment, bayes' learning scheme is used as the statistical learning scheme. Assume that sensory inputs I_i(t) are given and both prior probability {overscore (p)}(Ω_l(t)) and probabilistic density function p(I_i(t)|Ω_l) have been calculated beforehand. Maximum posterior probability for each class of attention is calculated by following bayes' rule.

[0060] $\begin{array}{cc}p\left({\Omega }_l\left(t\right)\right)=\frac{\stackrel{\_}{p}\left({\Omega }_l\left(t\right)\right)p\left(I_i\left(t\right)|{\Omega }_l\left(t\right)\right)}{\sum \stackrel{\_}{p}\left({\Omega }_k\left(t\right)\right)p\left(I_i\left(t\right)|{\Omega }_k\left(t\right)\right)}& \left(3\right)\end{array}$

[0061] The p(Ω_l(t)) may be called the “belief” of Ω_l and is the probability that a sensory input I_i(t) belongs to a class of attention Ω₁(t). Calculating the probability that a sensory input I_i(t) belongs to a class of attention Ω_l using bayes' rule implies that one class of attention Ω_l can be identified selectively by increasing the belief (weight) by learning of bayes' rule.

[0062] The class with highest probability (belief) is selected as class of attention Ω_l corresponding to the provided sensory input I_i(t). Thus, the behavior control apparatus 105 may obtain the class of attention Ω_l that is hidden parameter from directly observable sensory input I_i(t) using bayes' rule and to assign the sensory input I_i(t) to corresponding class of attention Ω_l.

[0063] The learning block 208 further searches behavior command according to the sensory input stored in the column corresponding to the selected class of attention, then send the searched behavior command to the target segregation block 212.

[0064] As noted above, using the blocks 204–210, the behavior control apparatus may estimate motion of the mobile unit accurately based on input images. Therefore, these blocks are inclusively referred to as “motion estimating method” in appended claims.

Behavior Control

[0065] On behavior control stage, the behavior control apparatus 105 estimates the motion based on input image and roughly segregates the location of the target object (target location). Then the apparatus performs pattern matching with templates which are stored in the memory as target object and calculate the target location more accurately. And the apparatus indicates to output the behavior command based on the distance between the target location and center of motion (COM). By repeating this process, the target location is getting refined and the mobile unit reaches in stably controlled status. In other words, the apparatus segregates the target based on motion estimation and understands what is to be target object.

[0066] Now the functionality of each block is described.

[0067] Target segregation block 212 roughly segregates and extracts a potion including target object, which are to be the behavior reference of the mobile unit, from visual space. For example, the segregation is done by comparing optical flow of the image and the estimated motion.

[0068] Target object matching block 214 uses templates to extract the target object more accurately. The target object matching block 214 compares the template and the segregated portion and determines whether the portion is the object to be targeted or not. The templates are prepared beforehand. If there are plurality of target objects, or if there are plurality of objects which match with the templates, the object having largest matching index is selected.

[0069] A target location acquiring block 216 defines the center point of the target object as the target location.

[0070] When the target location is defined, behavior decision block 218 supplies request signal to behavior command output block 204. When the request signal is received, behavior command output block 204 outputs the behavior command to move such that center of motion (COM) of the mobile unit overlaps the location of the target object.

[0071] It is indispensable for determining behavior command autonomously to segregate the target and non-target. The reason is because a target object segregated by target segregation may be used to select the optimal behavior to control the target object toward the location. In other words, the actual most suitable behavior is selected by predicting center of motion (COM) based on selective attention. Thus it allows the behavior control apparatus to search the location of the target object accurately in captured image. FIG. 4 is a diagram illustrating a target object segregation recognized by the target segregation block 212. Ellipses 401, 402, 403 are the cluster to be the location of the target object calculated based on the estimated motion and represented as normal distribution Ω₁, Ω₂, Ω₃, respectively. These are attention classes extracted from feature information of the image. Mixture distribution of three normal distribution model Ω₁, Ω₂, Ω₃ are showed as a dotted-lined ellipse in FIG. 4. Center of motion is acquired as center of mixture distribution in visual space. Each gaussian distribution in visual space is produced by projecting clustered behavior space based on center of motion on visual space with non-linear mapping like neural network.

[0072] Assuming that Ω_TL represents the target location and σ represents the area where segregating may be executed in captured image, the location of the target object is modeled by probability density function P(Ω_TL|σ). Since the location Ω_TL is basically uncertain value, it is assumed that the location has behavior control noise (that is, the variance of probabilistic density distribution). By repeating feedback process, noise (variance) of the target location is reduced and refined. In the present invention, reduction of noise (variance) depends on the accuracy of the motion estimation of the mobile unit.

[0073] FIG. 5 is a chart illustrating that range of the target location is refined (reduced) by the learning. Learning block 208 narrows down uncertain probability range (in other words, variance of probabilistic density distribution) σ of the location of the target location by, for example, bayes' learning.

CCA Reinforced EM Algorithm

[0074] Now CCA reinforced EM algorithm is described in detail.

[0075] The EM algorithm is an iterative algorithm for estimating the maximum likelihood parameter when observed data can be viewed as incomplete data. When the observed data is the normal distribution, the parameter θ is represented by θ(μ, Σ).

[0076] In one embodiment of the invention, the model of feature vector is built by means of bayes' parameter estimation. This is employed to estimate the number of clusters which represents data structure best.

[0077] Algorithm to estimate a parameter of Gaussian mixture model will be described. This algorithm is similar to conventional clustering essentially, but is different in that it can estimate parameters closely when clusters are overlapped. Therefore, sample of training data is used to determine the number of subclass and the parameters of each subclass.

[0078] Let Y be an M dimensional random vector to be modeled using a Gaussian mixture distribution. Assume that this model has K subclasses. The following parameters are required to completely specify the k-th subclass.

• • [0079] π_k: the probability that a pixel has subclass k
  • [0080] μ_k: the M dimensional spectral mean vector for subclass k
  • [0081] R_k: the M times M spectral covariance matrix for subclass k

[0082] π, μ, R denote the following parameter sets, respectively.

[0083] $\begin{array}{cc}{\left\{{\pi }_k\right\}}_{k=1}^K,{\left\{{\mu }_k\right\}}_{k=1}^K,{\left\{R_k\right\}}_{k=1}^K& \left(4\right)\end{array}$

[0084] The complete set of parameters for the class are then given by K and θ=(π, μ,R). Note that the parameters are constrained in a variety of ways. In particular, K must be an integer greater than 0, π_k≧0 with Σπ_k=1, and det(R)≧ε, where might be chosen depending on the application. The set of admissible θ for a k-th order model is denoted by ρ.

[0085] Let Y₁, Y₂, . . . , Y_n be N multispectral pixels sampled from the class of interest. Moreover, assume that the subclass of that pixel is given by the random variable X_n for each pixel Y_i. Certainly, Ω_n is normally not known, and which can also be useful for analyzing the problem.

[0086] Letting each subclass be a multivariate Gaussian distribution, the probability density function for the pixel Y_n for Ω_n=k is given by

[0087] $\begin{array}{cc}\begin{array}{c}{p}_{y_n|x_n}\left(y_n|k,\theta \right)=\frac{1}{{\left(2\pi \right)}^{M/2}}{R_k}^{-1/2}\\ \exp \left\{-0.5{\left(y_n-{\mu }_k\right)}^t{R}_k^{-1}\left(y_n-{\mu }_k\right)\right\}\end{array}& \left(5\right)\end{array}$

[0088] Since the subclass Ω_n of each sample is not known, to compute the density function of Y_nm for given parameter θ, the following definition of conditional probability is applied.

[0089] $\begin{array}{cc}{p}_{y_n}\left(y_n|\theta \right)=\sum _{k=1}^K{p}_{y_n|x_n}\left(y_n|k,\theta \right){\pi }_k& \left(6\right)\end{array}$

[0090] The logarithm of the probability of the entire sequence

[0091] $\begin{array}{cc}Y={\left\{Y_n\right\}}_{n=1}^N& \left(7\right)\end{array}$
is as follows.

[0092] $\begin{array}{cc}\log p_y\left(y|K,\theta \right)=\sum _{n=1}^N\log \left(\sum _{k=1}^K{p}_{y_n|x_n}\left(y_n|k,\theta \right){\pi }_k\right)& \left(8\right)\end{array}$

[0093] The objective is then to estimate the parameters K and θ∈ρ^(K).

[0094] Minimum description length (MDL) estimator works by attempting to find the model order which minimizes the number of bits that would be required to code both the data samples y_n and the parameter vector θ. MDL reference is expressed like the following expression.
MDL(K,θ)=−log p_y(y|K,θ)+2L log(NM)  (9)

[0095] Therefore, the objective is to minimize the MDL criteria

[0096] $\begin{array}{cc}\begin{array}{c}\mathrm{MDL}\left(K,\theta \right)=-\sum _{n=1}^N\log \left(\sum _{i=1}^K{p}_{y_n|x_n}\left(y_n|k,\theta \right){\pi }_k\right)+\\ \frac{1}{2}L\log \left(\mathrm{NM}\right)\end{array}& \left(10\right)\end{array}$

[0097] In order to derive the EM algorithm update equations, it is required to compute the following equation (Expectation step)

[0098] $\begin{array}{cc}Q\left(\theta ;{\theta }^{\left(i\right)}\right)=E\left[\log p_{y,x}\left(y,X❘\theta \right)❘Y=y,{\theta }^{\left(i\right)}\right]-\frac{1}{2}L\log \left(\mathrm{NM}\right)& \left(11\right)\end{array}$
where Y and X are the sets of random variables

[0099] $\begin{array}{cc}{\left\{Y_n\right\}}_{n=1}^N,{\left\{X_n\right\}}_{n=1}^N& \left(12\right)\end{array}$
respectively, and y and x are realizations of these random objects.

[0100] Thus the following equation holds.
MDL(K,θ)−MDL(K,θ⁽ⁱ⁾)<Q(θ⁽ⁱ⁾;θ⁽ⁱ⁾)−Q(θ;θ⁽ⁱ⁾)  (13)

[0101] This results in a useful optimization method since any value of θ that increases the value of Q(θ;θ⁽ⁱ⁾) is guaranteed to reduce the MDL criteria. The objective of the EM algorithm is hereby to iteratively optimize with respect to θ until a local minimum of the MDL function is reached.

[0102] The Q function is optimized in the following way.
Q(E,π;E⁽ⁱ⁾,π⁽ⁱ⁾)=E[log p_y,x(Y,X|E,π)|y,E⁽ⁱ⁾, π⁽ⁱ⁾]−KM log(NM)  (14)

[0103] In this case,

[0104] $\begin{array}{cc}\begin{array}{c}Q\approx \sum _{k=1}^K\left\{-\frac{1}{2}\mathrm{tr}\left(P_k{\stackrel{\_}{R}}_k\right)-\frac{\left(M-1\right){\stackrel{\_}{N}}_k}{2}\log \left(2\pi \right)+{\stackrel{\_}{N}}_k\log {\pi }_k\right\}-\\ \mathrm{KM}\log \left(\mathrm{NM}\right)\end{array}& \left(15\right)\end{array}$
where

[0105] $\begin{array}{cc}{\stackrel{-}{N}}_k=\sum _{n=1}^N{p}_{x_n|y_n}\left(k|y_n,E^{\left(i\right)},{\pi }^{\left(i\right)}\right){\stackrel{-}{R}}_k=\sum _{n=1}^N{y}_n{y}^t{p}_{x_n|y_n}\left(k|y_n,E^{\left(i\right)},{\pi }^{\left(i\right)}\right)& \left(16\right)\end{array}$

[0106] The EM update equations then are following.
(E⁽ⁱ⁺¹⁾,π⁽ⁱ⁺¹⁾)=argmin_E,πQ(E,π;E⁽ⁱ⁾,π⁽ⁱ⁾)  (17)
The solution is given as follows.
e_k⁽ⁱ⁺¹⁾=principal eigenvector {overscore (R)}_kπ_k⁽ⁱ⁺¹⁾={overscore (N)}_k|N  (18)

[0107] Initially, the number K of subclasses will be started with sufficiently large, and then be decremented sequentially. For each value of K, the EM algorithm is applied until it is converged to a local maximum of the MDL function. Eventually, the value of K may be selected simply and corresponding parameters that resulted in the largest value for the MDL criteria may be selected.

[0108] One method to effectively reduce K is to constrain the parameters of two classes to be equal, such that e_l=e_m for classes l and m. Moreover, letting E* and E*_lm be the unconstrained and constrained solutions to Eq (17), a distance function may be defined as follows.
d(l,m)=Q(E*,π*;E⁽ⁱ⁾,π⁽ⁱ⁾)−Q(E*_l,m,π*;E⁽ⁱ⁾,π⁽ⁱ⁾)=σ_max(R_l) σ_max(R_m)−σ_max(R_l+R_m)≧0  (19)
where σ_max(R) denotes the principal eigenvalue of R. At each step, the two components that minimized the class distance are computed.
(l*,m*)=argmin_l,md(l,m)  (20)

[0109] After all, the two classes are merged and the number of subclass K is decreased.

Process of Behavior Control Apparatus

[0110] It should be noted that the learning stage and behavior control stage are not also divided clearly, but both of them may be executed simultaneously as one example described bellow.

[0111] In other words, behavior evaluation block 206 determines whether feature of image provided afresh should be reflected to knowledge acquired by previous learning in behavior control stage. Furthermore, behavior evaluation block 206 receives the motion estimated from the image. When change of the external environment that was not learned in previous learning is captured by image capturing block 202, the feature is sent to behavior evaluation block 206, which outputs attentional demanding for indicating generation of an attention class. In response to this, learning block 208 generates an attention class. Thus learning result is always updated; therefore, precision of the motion estimation is improved, too.

[0112] Now the control process in practical application will be described of the behavior control apparatus of the invention installed on RC helicopter. FIG. 6 is a flowchart of the process. This chart can be divided into two step showed as two dotted line rectangular in FIG. 6. One is coarse step of left side column where rough segregation of target/non-target is executed. The other is fine step of right side column where the target location is narrowed (refined) gradually.

[0113] At step 602, probabilistic density distribution P(Ω_l) for all attention classes Ω_l of motion are assumed to be uniform. At step 604, the mobile unit moves randomly for collecting data for learning. In this example, data set collected for stabilizing the RC helicopter 100 was used to generate 500 training data points and 200 test points.

[0114] At step 606, the CCA reinforced EM algorithm is executed for calculating parameters θ (μ, Σ) which defines the probabilistic density distribution Ω_l. In the present example, 20 subclasses was used at first, but the number of subclasses converges by CCA reinforced EM algorithm and finally reduced to 3 as shown in FIG. 4.

[0115] At step 608, P(Q|Ω_l) is calculated with θ, where Q represents behavior command. At step 610, probabilistic relation between feature vector I and attention class Ω₁ is calculated with neural network. At step 612, motion of the mobile unit is estimated by bayes' rule. Steps 602 to 612 correspond to the learning stage.

[0116] At step 614, Gaussian mixture model is calculated with the use of each probabilistic density function. Part of the image which is not included in Gaussian mixture model is separated as non-target.

[0117] At step 616, the target object is recognized by template matching and probabilistic density distribution Ω_TL of the target location is calculated. At step 618 the center of this is defined as target location.

[0118] At step 620, difference D between center of motion (COM) and the target location (TL) is calculated. At step 622, the map outputs behavior command expanding the width of motion when the helicopter is far from the target location, otherwise outputs command reducing the width of the motion. FIG. 7 shows an example of output behavior command. As seen, a map is stored in memory which takes different output value depending on D and corresponding value is searched and transmitted to the servomotor.

[0119] At step 624, it is determined whether D is smaller than the allowable error ε. If D is larger than ε, the accuracy of the target location is not sufficient and the process returns to step 606 to re-calculate θ. That is, it attributes to the normalization problem how many number of gaussian mixture function is need to estimate the state of motion. By increasing the applied number of mixture gaussian function every time the process returns to step 606, the unit may estimate θ accurately and thus predict the target location accurately.

[0120] When D is smaller than ε at step 624, it shows that the helicopter is stable with sufficient accuracy for target location and so the process is terminated. By setting ε small, the unit may control both the location of helicopter and the duration during which the helicopter remains at that location. Steps 614 to 624 correspond to the behavior control stage.

Results

[0121] FIGS. 8 to 10 are graphs illustrating control status of the RC helicopter. In these graphs, horizontal axis represents the number of trial and vertical axis represents the distance between center of motion (COM) and the target location (TL) when controlling the helicopter to be stable. Two dotted straight line in the graphs represent threshold values ε to determine stability of the control. The value ε is set to 0.1826 in the graphs.

[0122] FIG. 8 is graph of control immediately after the behavior control is initiated. In this case, the distance D does not become lower than ε and the vibration is still large, so the control is determined as to be unstable. As the target location is narrowed, the vibration becomes smaller (as FIG. 9). Finally, the control status becomes stable as shown in FIG. 10.

[0123] Some preferred embodiments have been described, but this invention is not limited to such embodiments. For example, the behavior control apparatus may not be installed on the mobile unit. In this case, only the CCD camera is installed on the mobile unit and the behavior control apparatus is installed on another place. Then information is transmitted through wireless communication between the camera and the apparatus.

[0124] According to one aspect of the invention, the behavior control apparatus roughly segregate target area that includes a target object of behavior from sensory inputs, such as images, based on the estimation of motion. The apparatus then specifies a target object from the target area, acquires location of the target object and output behavior command which moves the mobile unit toward the location. Thus, detailed feature of the target object need not be predetermined. In addition, because the features irrelevant to present behavior are eliminated, the computational load is reduced. Therefore, highly efficient and accurate control for the mobile unit may be implemented.

[0125] According to another aspect of the invention, the behavior control apparatus pre-learns the relationship between sensory inputs and behavior commands. Then the apparatus updates the learning result when new feature is acquired on behavior control stage. The learning result is represented as probabilistic density distribution. Thus, motion of the mobile unit on behavior control stage may be estimated with high accuracy.