Motion editing apparatus and method for legged mobile robot and computer program

[0001] 1. Field of the Invention

[0002] This invention relates to a motion editing apparatus and a motion editing method for supporting the creation and the editing of motions, stating a predetermined movement pattern of a robot, and to a computer program. This invention especially relates to a motion editing apparatus and a motion editing method for a legged mobile robot for doing various tasks by movable legs, and to a computer program.

[0003] More particularly, the present invention relates to a motion editing apparatus and a motion editing method for supporting the editing of a movement pattern, as feasibility of the movement pattern on an actual robot is scrutinized, and to a computer program. More specifically, the invention relates to a motion editing apparatus and a motion editing method for a legged mobile robot in which the motion as edited is checked on the actual robot, and a computer program.

[0004] The application claims priority of Japanese Patent Application No. 2002-298348 filed on Oct. 11, 2002, the entirety of which is incorporated by reference herein.

[0005] 2. Description of Related Art

[0006] A mechanical apparatus for performing movements simulating the movements of the human being, with the use of electrical or magnetic movements, is termed a robot. The etymology of the term robot is said to be ROBOTA (slave machine) of the Slavic language. In Japan, the robots started to be used towards the end of the 1960s. Most of these robots used were industrial robots, such as manipulators or transporting robots, aimed to automate or perform unmanned tasks in plant movements.

[0007] In recent years, researches and developments in legged mobile robots, simulating the bodily mechanism and movements of an animal which is erected and walks on two feet, such as human beings or monkeys, are progressing, such that there are good prospects for practical utilization of this robot type. The movement system by legs, erected and walking on two legs, is labile as compared to the crawler type system or the system walking on four or six legs, and hence is difficult to control as to posture or walking. However, the movement system by legs, erected and walking on two legs, is favorable in such respects that it is able to cope with a work route presenting an irregular walking surface, such as non-leveled terrain or obstacles, or a non-continuous walking surface, such as staircase or ladder, thereby achieving more flexible movements.

[0008] On the other hand, the legged mobile robot, regenerating the mechanism of the living body or movements of the human beings, is termed a humanoid or a humanoid robot. The humanoid robot is able to support the human life, that is to support human activities, in various aspects of our everyday life, such as in our living environments.

[0009] The major portions of the task space or the living space of the human beings are tailored to the bodily mechanism and the behavior patterns of the human beings, which are erect and walk on two feet, while presenting many obstacles to movements of the state-of-the-art mechanical system, having wheeled driving device or the like as movement means. Thus, in order for the mechanical system, that is, the robot, to take the place of the human beings in a large variety of tasks and to adapt itself to the living environment of the human beings, it is desirable that the possible range of movement of the robot is substantially the same as that of the human beings. This accounts for great general expectations for practical utilization of legged mobile robots.

[0010] The up-to-date legged mobile robot has a high information processing capability, such that the robot itself may be comprehended as a sort of the computer system. Stated differently, the highly advanced complicated sequence of movements, or motions, constructed by movement patterns, realized on a robot, or by a combination of plural fundamental movement patterns, are constructed by a movement similar to computer programming.

[0011] In order for the robot body to come into widespread use, it is imperative that a large number of motion data for actuating the robot body become practically usable. Thus, it is strongly desired to construct a development environment for enabling motion editing for robots.

[0012] It may also be anticipated that the robot will be widely used in the near future not only in industry but also in households and in our everyday life. In particular, as to an entertainment-oriented product, it may be anticipated that general consumers at large, not having specialized knowledge about computers or computer programming, purchase and use robots. For the general consumers at large, it would be desirable to provide a tool which will support them in formulating and editing the robot's movement sequence by interactive processing, that is, a motion editing system efficiently and with relative ease.

[0013] The robot is constructed by a plural number of control points, such as joints, so that, by sequentially inputting the positions or velocities (joint angles or angular accelerations) at respective control points, it is possible to edit the movements of the robot body. In this respect, the movement of such formulation and editing may be likened to the generation of character animation in computer graphics. However, there is an explicit difference between the movement in a virtual space and those of an actual apparatus (robot). In the case of the legged mobile robot, desired movements cannot be executed by simply actuating the angles of joints, such that the movements on legs need to be continued without falldown of the robot. In other words, in order to realize a desired movement, it is imperative that the movements realized on the actual robot are checked and stability in posture of the robot body is maintained in the course of the motion execution.

[0014] In controlling posture stability of the legged mobile robot, a ZMP stability decision criterium of searching for a zero moment point on or inwardly of a side of a supporting polygon, formed by the for sole touchdown point and the road surface, is used. In the case of the two-legged mobile robot, this supporting polygon is of a marked height, as a result of which it is retained to be difficult to control the posture of the robot in stability.

[0015] There has already been known a motion editing system in which command values in each control point of the robot body is entered on a display to help implement the robot's motion. However, there lacks up to now a system in which posture stability in case of realizing the edited motion is checked on the actual robot or in which the desired motion is corrected for stabilizing the posture. In fact, motion edition would be a failure if the posture stability of the robot body cannot be maintained with the implemented motion, such that the motion itself cannot be put into practice.

[0016] It is therefore an object of the present invention to provide a method and an apparatus for a legged mobile robot and a computer program in which it is possible to support user in editing a movement pattern, as feasibility thereof on the actual apparatus (robot) is taken into account.

[0017] It is another object of the present invention to provide a method and an apparatus and a computer program for a legged mobile robot in which the motion once edited can be corrected as the actual movements are checked on the actual apparatus.

[0018] In one aspect, the present invention provides a motion editing apparatus or method for a legged mobile robot having a plurality of degrees of freedom in joints and a sensor for measuring an external environment, in which the apparatus or method comprises a data input unit or step of inputting motion data, a data reproducing step of reproducing the motion data on an actual apparatus, a sensor information acquisition unit or step of acquiring the sensor information from the sensor during the time when the motion data is being reproduced, a motion evaluation unit or step of evaluating the motion based on the acquired sensor information, and a motion correction unit or step of correcting the motion data based on the results of evaluation, and an motion editing apparatus for a legged mobile robot for carrying out the above motion editing method.

[0019] In the motion editing apparatus and method, according to the first aspect of the present invention, it is possible to formulate, by acquiring the sensor information of the actual robot by communication, the motion which takes the response of the actual robot into account. It is also possible to check the motion data, formulated in a reference environment, as to whether or not the actual robot operates as scheduled when the motion data is executed in an environment different from the reference environment, such as in the living environment.

[0020] The motion editing apparatus or method for the legged mobile robot of the first aspect of the present invention may further include a motion data outputting unit or step for embedding the sensor information acquired by the sensor information acquisition unit as reference data in the motion data which satisfy a criterium of evaluation in the motion evaluation unit, and for outputting the resulting motion data having the reference data embedded therein.

[0021] Even if robot movements stated in the motion data are the same, the sensor outputs differ in dependence on the outside environment or on the working environment. For example, if the walking pattern is the same, the sensor output value differs when the robot is walking on a road presenting a gradient, or on the labile road surface, such as on the gravel or on the thick-piled carpet. With the motion data having the reference sensor information embedded therein, such advantage may be derived that the motion data consistent with specified working environments (or the robot using configuration) can be stated.

[0022] The motion data outputting unit or step may output the information on the angles of joints, formed by the combination of angle command values for respective joints and measured values acquired on executing the motion, as the motion data having the reference data embedded therein. Or, the motion data outputting unit or step may output the posture information composed of the combination of the target values for the respective sensors at the time of motion edition, measured values at the time of motion execution and filtered values of the measured values of the sensor outputs as the motion data having the reference data embedded therein. Alternatively, the motion data outputting unit or step may output the ZMP trajectory information formed by the combination of the target ZMP trajectory for left and right foot soles at the time of editing and the ZMP trajectory following the correction by stabilization control at the time of execution of the motion, as the motion data having the reference data embedded therein. Still alternatively, the motion data outputting unit or step may output the foot sole touchdown information formed by the combination of a target value at the time of editing of a floor reaction sensor and a measured value thereof at the time of motion execution and/or the contact information as motion data having the reference data embedded therein.

[0023] The motion evaluation unit or step may chronologically evaluate followup characteristics on executing the motion on the actual apparatus.

[0024] The motion evaluation unit or step may chronologically acquire a torque value of an actuator and the number of revolutions on executing the motion on an actual robot body and compare the acquired data to a NT curve representing the actuator characteristics, in order to evaluate whether or not there is any movement which surpasses the limit torque of the actuator.

[0025] The motion evaluation unit or step may calculate a difference between posture sensor values and the ZMP trajectory as scheduled at the time of the motion edition, and sensor values and the ZMP trajectory as acquired on executing the motion on an actual robot body, in order to evaluate the posture.

[0026] The motion evaluation unit or step may calculate a difference between the posture at the time of motion edition and measured values obtained on executing the motion on the actual robot body, in order to evaluate the touchdown and/or contact.

[0027] The motion evaluation unit may calculate the degree of improvement in measured values as to the motion corrected by last and previous evaluation events, in order to evaluate the degree of achievement of correction.

[0028] The motion evaluation unit or step may calculate the effect of an impact due to contact with an outside object on an actuator torque, ZMP trajectory or on the acceleration, in order to evaluate the impact due to contact with the outside object.

[0029] The motion correction unit or step may correct a command angle value to the actuator based on the result of evaluation of response properties of the actuator and/or corrects control parameters of the actuator.

[0030] The motion correction unit or step may change the contents of a posture stabilization processing block based on the result of evaluation of the actuator torque.

[0031] The motion correction unit or step may change the contents of a posture stabilization processing block based on the result of evaluation of the touchdown and/or contact.

[0032] The motion correction unit or step may change the control of the posture stabilization processing block, as the contact with the outside object is taken into account, based on the result of evaluation of the impact due to contact with the outside object.

[0033] The motion reproducing unit or step may take out only an optional range of motion data, in order to reproduce the range thus taken out on the actual apparatus.

[0034] By taking out only an optional range of the motion data and reproducing the so taken out motion data on the actual apparatus (robot), in reproducing the motion data-on the actual apparatus, the motion editing movement may be improved in efficiency.

[0035] The data reproducing unit or step may set a start time point in motion data, calculate the dynamic posture at the start time point, generate a transient motion with the dynamic posture at the start time point as a terminal point, reproduce the motion on the actual apparatus using the transient motion, set a stop time point in the motion data, calculate the dynamic posture at the time point, generate a transient motion with the stop posture as a start point, and halt the movement of the actual apparatus using the transient motion.

[0036] It is noted that the motion is formed by chronological combination of two or more postures. When a dynamic motion (continuous dynamic movement), which positively and continuously employs the acceleration, it is impossible to execute the motion from an intermediate point to make the evaluation. According to the present invention, the reproduction and stop of the continuous dynamic movement as from an optional time point are enabled to reduce the time needed for motion evaluation significantly.

[0037] In a second aspect, the present invention provides computer program in a computer readable form executing the motion editing processing for a legged mobile robot having a plurality of degrees of freedom in joints and a sensor for measuring an external environment. The computer program comprises a data inputting step of inputting motion data, a data reproducing step of reproducing the motion data on an actual apparatus, a sensor information acquisition step of acquiring the sensor information from the sensor during the time when the motion data is being reproduced, a motion evaluation step of evaluating the motion based on the acquired sensor information, and a motion correction step of correcting the motion data based on the results of evaluation.

[0038] The computer program in the second aspect of the present invention has defined, in a computer readable form, a computer program which allows a preset processing to be implemented on a computer system. In other words, by having the computer program of the second aspect of the present invention installed on the computer system, concerted movements may be realized on the computer system, whereby the operation and effect comparable to those of the motion editing apparatus or method for the legged mobile robot according to the first aspect of the present invention may be achieved.

[0039] Thus, according to the present invention, there are provided a motion editing apparatus, a motion editing method and a computer program for a legged mobile robot supporting the editing of the movement pattern as the feasibility on the actual apparatus (robot) is taken into consideration.

[0040] According to the present invention, there are also provided a motion editing apparatus, a motion editing method and a computer program for a legged mobile robot capable of correcting the edited motion as the movement on the actual apparatus is checked.

[0041] Other objects, features and advantages of the present invention will become more apparent from reading the embodiments of the present invention as shown in the drawings.

[0042] FIG. 1 shows an appearance and an overall structure of a legged mobile robot as a subject entity of motion edition by a motion editing system according to the present invention, looking from an oblique front side.

[0043] FIG. 2 shows an appearance and an overall structure of a legged mobile robot as a subject entity of motion edition by a motion editing system according to the present invention, looking from an oblique rear side.

[0044] FIG. 3 schematically shows the structure of the degrees of freedom of a legged mobile robot 100 embodying the present invention.

[0045] FIG. 4 schematically shows a control system structure of the legged mobile robot 100.

[0046] FIG. 5 schematically shows the processing flow in a motion editing system according to an embodiment of the present invention.

[0047] FIG. 6 shows a data structure of the information on the angles of joints contained in motion data.

[0048] FIG. 7 shows a data structure of the information on the posture contained in motion data.

[0049] FIG. 8 shows a data structure of the information on the ZMP trajectory contained in motion data.

[0050] FIG. 9 shows a data structure of the information on the contact of the foot sole on the floor contained in motion data.

[0051] FIG. 10 is a flowchart showing a processing sequence for evaluating motion data of the robot 100 based on the sensor information.

[0052] FIG. 11 is a flowchart showing a processing sequence for motion correction.

[0053] FIG. 12 is a flowchart showing a processing sequence for processing the reproduction and stop as from an optional time point of the continuous dynamic movements stated in motion data of the robot 100.

[0054] FIG. 13 is a flowchart showing a modification of the motion editing processing.

[0055] Referring to the drawings, a certain preferred embodiment of the present invention will be explained in detail.

[0056] A. Mechanical Structure of Legged Mobile Robot

[0057] FIGS. 1 and 2 show the appearance and the overall structure of the legged mobile robot 100 as a subject entity of motion edition by a motion editing system according to the present invention. This legged mobile robot is termed a humanoid robot. As shown, the legged mobile robot 100 is made up by a trunk unit, a head unit, left and right upper limbs and left and right lower limbs, responsible for movement on legs. The movements of the entire robot is comprehensively controlled by a controlling unit enclosed e.g. in its trunk portion (not shown).

[0058] Each of the left and right lower limbs is made up by a thigh unit, a knee joint, a shank unit, an ankle and a foot flat, and is connected to approximately the lowermost end of the body trunk unit by a hip joint. Each of the left and right upper limbs is made up by an upper arm, an elbow joint, and a forearm, and is connected to upper left and right side edges of the body trunk by the shoulder joint. The head unit is connected by the neck joint to approximately an uppermost mid point of the trunk portion.

[0059] The controlling unit is a casing on which there are mounted a controller (main controlling unit), taking charge of driving control of each joint actuator constituting the legged mobile robot 100, and of processing an external input from each sensor, as later explained, and peripherals, such as a power supply circuit. The controlling unit may also include a communication interface for remote control or communication devices.

[0060] The legged mobile robot 100, constructed as described above, is able to realize walking on two feet by concerted whole-body movement control by the controlling unit. This walking on two feet is usually realized by repetition of a walking period divided into the following respective movement periods:

[0061] (1) a period during which a right leg is uplifted, with the robot being supported on only the left leg;

[0062] (2) a period during which the right leg touches the floor, with the robot being supported on both legs;

[0063] (3) a period during which the left leg is uplifted, with the robot being supported on only the right leg; and

[0064] (4) a period during which the left leg touches the floor, with the robot being supported on both legs.

[0065] The walking control in the legged mobile robot 100 may be achieved by planning a target trajectory of the lower limbs at the outset and by correcting the target trajectory during each of the above periods. That is, during the time period when the robot is supported by both legs, the correction of the lower limb trajectory is halted and, using the total correction quantity for the target trajectory, the waist height is corrected to a constant value. During the period when the robot is supported on the sole leg, a corrected trajectory is generated so that the position relationships between the ankle of the leg, the trajectory of which has been corrected, and the waist, will revert to the scheduled trajectory.

[0066] For posture stabilizing control for the robot body, to say nothing of the correction of the trajectory of the walking movements, interpolating calculations, employing a five-degree polynominal, are carried out in general in order to assure continuous transitions of the positions, velocity and the acceleration aimed to provide for a reduced offset with respect to the zero moment point (ZMP). This ZMP is used as a criterium for verifying the degree of walking stability.

[0067] This criterium for verifying the degree of walking stability by the ZMP is based on the D-Alembert's principle which states that the gravity and the force of inertia from the walking system to the road surface and the moment thereof are in equilibrium with the force of reaction from the floor as the reaction from the road surface to the walking system and the moment of the force of reaction from the floor. As a conclusion of the inference of mechanics, there exists a point of zero moment of the pitch and roll axes, that is the zero moment point (ZMP), on or inwardly of a side of a supporting polygon defined by the touchdown point of the foot sole and the road surface (ZMP stabilized area).

[0068] FIG. 3 schematically shows the structure of the degrees of freedom of the legged mobile robot 100 embodying the present invention.

[0069] Referring to FIG. 3, the present robot is made up by a trunk part of the robot to which are mounted four limbs. These four limbs are formed by left and right arms having seven degrees of freedom, that is, a shoulder joint pitch axis, a shoulder joint roll axis, an upper arm yaw axis, an elbow joint pitch axis, a forearm yaw axis, a wrist roll axis and a wrist pitch axis, and by left and right legs having six degrees of freedom, that is, a hip joint yaw axis, a hip joint roll axis, a hip joint pitch axis, a knee joint pitch axis, an ankle pitch axis and an ankle roll axis.

[0070] In actuality, these degrees of freedom in the joints are implemented by actuator motors. In the present embodiment, there is mounted a small-sized direct gear coupling type AC servo actuator in which a servo control system is designed as one chip and enclosed in a motor unit. Meanwhile, this sort of the AC servo actuator is disclosed in e.g. the Japanese Laying-Open Patent Publication 2000-299970 (Japanese Patent Application H11-33386) transferred to the present Assignee.

[0071] The robot body is loaded with an acceleration sensor A1 and with a gyro G1. On the four corners of left and right foot soles, there are mounted uniaxial load cells F1 to F8 for detecting the force of reaction of the floor in a direction perpendicular to the foot sole surface, and infra-red distance measuring sensors D1 to D8 for measuring the distance to the floor surface. At a mid portion of each of the left and right foot soles, there are mounted acceleration sensors A2 and A3 and gyros G2, G3.

[0072] It is noted that the head unit has a degree of freedom of the joint about the neck joint yaw axis, first and second neck joint pitch axes and the neck joint roll axis, with respect to the body trunk part, although this is not shown in FIG. 3 for avoiding complexities in the drawings. The body trunk part also has a degree of freedom of the joint about each of the body trunk roll axis and the body trunk pitch axis.

[0073] B. Structure of the Control System for Legged Mobile Robot

[0074] FIG. 4 schematically shows the structure of the control system of the legged mobile robot 100. In this figure, the legged mobile robot 100 is made up by respective mechanical units 30, 40, 50R/L and 60 R/L, representing four limbs of the human being, and a controlling unit 80 for exercising adaptive control for realizing concerted movements among the respective mechanical units. It is noted that, throughout the specification, R and L are each suffixes depicting right and left, respectively.

[0075] The overall movements of the legged mobile robot 100 are comprehensively controlled by the controlling unit 80. This controlling unit 80 is made up by a main controller 81 and a peripheral circuit 82. The main controller 81 is formed by main circuit components, composed e.g. of a CPU (central processing unit) and a memory, while the peripheral circuit 82 includes an interface, not shown, for exchanging data or commands with the power supply circuit or with respective component units of the robot 200.

[0076] In practicing the present invention, there is no particular limitation to the mounting site of the controlling unit 80. Although the controlling unit is mounted to the body trunk unit 40 in FIG. 4, it may also be mounted to the head unit 30. Or, the controlling unit 80 may be mounted externally of the legged mobile robot 100 and connected over a wired or wireless path to the body unit of the legged mobile robot 100 for communication.

[0077] The degree of freedom of each joint in the legged mobile robot 100 shown in FIG. 3 is realized by a relevant actuator motor M. That is, the head unit 30 is provided with a neck joint yaw axis actuator M1, neck joint pitch axis actuators M_2A, an M_2B and a neck joint roll actuator M₃, representing the neck joint yaw axis, first and second neck joint pitch axes and the neck joint roll axis, respectively.

[0078] The body trunk unit 40 is provided with a body trunk pitch axis actuator M₁₁ and a body trunk roll axis actuator M₁₂ representing the body trunk pitch axis and the body trunk roll axis, respectively.

[0079] The arm units 50R/L are subdivided into upper arm units 51R/L, elbow joint units 52R/L and forearm units 53R/L. It is noted that the arm units are each provided with a shoulder joint pitch axis actuator M₄, a shoulder joint roll axis actuator M₅,

[0080] an upper arm yaw axis actuator M₆, an elbow joint pitch axis actuator M₇, an elbow joint yaw axis actuator M₈, a wrist joint roll axis actuator M₉ and a wrist joint pitch axis actuator M₁₀, representing a shoulder joint pitch axis, a shoulder joint roll axis, an upper arm yaw axis, an elbow joint pitch axis, an elbow joint roll axis, a wrist joint roll axis and a wrist joint pitch axis, respectively.

[0081] The leg units 60R/L are subdivided into thigh units 61R/L, knee units 62R/L and shank units 63R/L. It is noted that the leg units 60R/L are each provided with a hip joint yaw axis actuator M₁₃, a hip joint pitch axis actuator M₁₄, a hip joint roll axis actuator M₁₅, a knee joint pitch axis actuator M₁₆, an ankle joint pitch axis actuator M₁₇ and an ankle joint roll axis actuator M₁₈, representing the hip joint yaw axis, hip joint pitch axis, hip joint roll axis, knee joint pitch axis, ankle joint pitch axis and the ankle joint roll axis, respectively.

[0082] More preferably; the actuators M₁, M₂, M₃, . . . , used for the respective joints, may each be constructed by a small-sized AC servo actuator, as described before, of the direct gear coupled type in which the servo control system is arranged as one chip and loaded in a motor unit.

[0083] The respective mechanical units of the head unit 30, body trunk unit 40, arm units 50 and the leg units 60 are provided with sub-controlling units 35, 45, 55, 65 for driving and controlling the actuators, respectively.

[0084] The body trunk unit 40 of the robot body is provided with an posture sensor G1 composed e.g. of an acceleration sensor A1 and a gyro sensor G1. The acceleration sensor A1 is arranged e.g. along X, Y and Z axes. By arranging the acceleration sensor A1 to the waist part of the robot body, it is possible to set the waist, representing the site with a large weight mass from the perspective of robot movement movements, as a control target point, and to directly measure the posture or the acceleration on this site, in order to exercise ZMP-based posture stabilizing control.

[0085] The leg units 60R, 60L are provided respectively with floor reaction sensors F1 to F4 and F5 to F8, acceleration sensors A2, A3, and with posture sensors G2, G3. The floor reaction sensors F1 to F8 may be constructed by pressure sensors mounted to e.g. foot soles. It can be detected, based on the presence or absence of the force of reaction exerted from the floor, whether or not the foot sole has touched the floor. The acceleration sensors A2, A3 are arranged at least along the X and Y axes, respectively. The equation of ZMP equilibrium may directly be set at the foot closest to the ZMP position by providing the acceleration sensors A2, A3 on the left and right leg units.

[0086] If the acceleration sensors are provided only to the waist part, representing the site with a large weight mass from the perspective of robot movement movements, solely the waist part is set as a control target point. In this case, the state of the foot sole has to be relatively calculated, on the basis of the results of the calculations of the control target point, such that

[0087] (1) the condition that the road surface is not moved without dependency on which force or torque acts thereon; and

[0088] (2) the condition that the frictional resistance against translational movement on the road surface is satisfactorily large such that no slip is produced need to be met, as a matter of premises, between the foot and the road surface.

[0089] In the present embodiment, a reaction sensor system for directly detecting the ZMP and the force is provided to the foot as the site of contact on the road surface, whilst a local coordinate used for control and an acceleration sensor for directly measuring the coordinate are provided to the foot sole. As a consequence, the equation of ZMP equilibrium may directly be set at the foot closest to the ZMP position, so that more strict posture stabilizing control not dependent on the above-mentioned premises may be achieved at a higher speed. The result is that stabilized walking (movement) of the robot body may be assured even on the road surface which may be moved under application of a force or a torque, such as gravel, on a thick-piled carpet or, on a tile used in a household where slip is likely to be produced because satisfactory frictional coefficients of translational movement cannot be assured.

[0090] The main controller 81 is able to dynamically correct the control target responsive to outputs of the sensors A1 to A3, G1 to G3 or F1 to F8. More specifically, the controlling unit 80 adaptively controls the sub-controlling units 35, 45, 55, 65 to realize a whole-body movement pattern of the legged mobile robot 100 in which the upper limbs, body trunk and the lower limbs are actuated in concert.

[0091] For realizing the whole-body movements of the robot body of the legged mobile robot 100, the movements of the foot units, ZMP (zero moment point) trajectory, body trunk movements, upper limb movements and the height of the waist part etc, are set, whilst commands for instructing the movements pursuant to the setting contents, are transmitted to the sub-controlling units 35, 45, 55, 65. The respective sub-controlling units 35, 45, . . . interpret the commands received from the controlling unit 80 from the main controller 81 to output driving control signals to the respective actuators M₁, M₂, M₃, . . . . It should be noted that the ZMP denotes a point on the floor surface where the moment by the force of reaction from the floor during walking becomes zero, while the ZMP trajectory means, as aforesaid, the locus of movement along which the ZMP is moved during the walking period of the robot 100.

[0092] C. Motion Editing System

[0093] FIG. 5 schematically shows the processing flow in a motion editing system in an embodiment of the present invention.

[0094] The user first edits motion data for the legged mobile robot 100 off-line on a motion editing system, having a motion editing application installed thereon, such as a personal computer (step S1).

[0095] It should be noted that the motion data may be edited by chronologically combining two or more poses (postures) of the robot 100. The poses of the robot 100 can be stated by displacements of the respective angles of joints. On the other hand, the motion data may be formed by the displacements, velocity or the acceleration of the respective angles of joints. Meanwhile, the motion data itself may be edited with advantage by employing the movement editing apparatus described in for example the Japanese Patent Application 2000-73423 transferred to the present Assignee.

[0096] The so prepared motion data is then installed on the legged mobile robot 100 to check for robot movements using the actual apparatus (step S2). The motion data itself is the chronological data representing a continuum of movements, such as displacements and velocities of the respective angles of joints, and is sometimes voluminous. In the present embodiment, an optional range of the motion data is taken out and reproduced on the actual apparatus to improve the efficiency of the motion editing operation, as will be explained subsequently in detail.

[0097] Outputs from respective sensors mounted on the actual apparatus when the optional range of the motion data is reproduced using the actual apparatus are transmitted to the motion editing system (step S3). The sensor information herein includes the acceleration sensor A1 and the gyro sensor G1, mounted to the body trunk unit 40 (mounted approximately at the center of gravity of the robot body), the acceleration sensors A2 and A3, gyro sensors G2, G3 and floor reaction sensors F1 to F8, mounted to the left and right foot soles, in addition to rotation signals from the encoder mounted to the joint actuators.

[0098] The movements of the robot are then evaluated, on the motion editing system, based on the sensor information acquired during motion reproduction (step S4). The method of evaluation of the movements of the robot 100 based on the sensor information will be explained subsequently.

[0099] If, as a result of the robot movements, the preset evaluation criterium has not been met, the motion correcting processing is executed (step S5), after which processing reverts to the step S2 to re-evaluate the movements which are based on the reproduction on the actual apparatus.

[0100] If, as a result of the evaluation of the robot movements, the preset criterium has been met, a motion data file, in which the sensor information obtained in the step S3 is embedded as the reference sensor information, is prepared (step S6) to complete the present processing routine.

[0101] By evaluating and correcting the motion data in accordance with the processing sequence shown in FIG. 5, it is possible to acquire the sensor information of the actual robot by communication to prepare the motion which has taken the responses of the actual robot into account. It is also possible to check whether or not the actual robot is being operated, as scheduled, when the motion data prepared in a reference environment is run in a different environment, such as in a living environment.

[0102] Such evaluation of the motion data on the actual robot is mainly significant in the following points:

[0103] (1) Difference in the impression of the acceleration

[0104] Since the animation reproduction by computer graphics (CG) in a virtual space significantly differs from the reproduction on the actual apparatus in a real space, as to the sensitivity to the velocity or acceleration of the various components, it is necessary to use an actual robot for adjustment thereof.

[0105] (2) Difference in the impression of the posture

[0106] Since the animation reproduction by computer graphics (CG) in a virtual space significantly differs from the reproduction on the actual apparatus in a real space as to sensitivity to the positions and postures of the various components, it is necessary to use an actual robot for its adjustment. For example, when the site referred to for stabilization is limited to only the waist part, there are occasions where the waist movements, which were not objectionable as far as the computer graphics are concerned, are felt to be excessive in a check on reproduction on an actual apparatus, such that the desired dynamic posture is not achieved. The dynamic posture may be corrected to the desired dynamic posture by carrying out the partial reproduction as the number of the sites referred to for stabilization is increased and as the priority sequence is changed. For example, the body trunk and the head unit may be added to the waist part, as sites for reference for stabilization, whereby the waist movement of the actual robot may be diminished to achieve desired movements.

[0107] (3) Difficulties met in estimating the actuator torque (current) on the actual apparatus For correct current estimation on the actual apparatus, it is necessary to construct the motor model extremely rigorously and to identify the respective elements extremely accurately. Moreover, the time needed in the simulation employing this model exceeds the practically tolerable level for a routine computer system, such as PC. Thus, by performing actual measurements by motion reproduction on the actual apparatus and by correcting details, it is possible to construct more robust motions without appreciably increasing the time needed in preparing the motion.

[0108] (4) Difficulties met in specifying correct contact and touchdown time

[0109] An offset from the scheduled movement of the contact point and the contact timing of the robot with an outside world leads to application to the robot of an unknown and impulse-like external force and an external force moment, so that, even if real-time adaptive control is applied, the effect such offset has on the stability of the robot movement may become significant. However, in constructing a rigorous shape model of an actual robot, high precision identification of various components of the shape model, inclusive of an external environment, is needed. Moreover, simulation of the contact state employing the model is not realistic because such simulation needs time appreciably surpassing the time actually tolerable with a routine computer system, such as PC. Thus, an extremely robust motion may be generated by again carrying out the motion correction and the processing for stabilizing the posture (movement) using an offset between the touchdown point and the states of touchdown and flight of the motion generated using a shape model and an environment model simplified to permit processing within a time tolerable for a routine PC on one hand and those of same motion executed in a set of actual standard environments on the other hand.

[0110] Even if the movements themselves of the robot stated by motion data remain the same, the sensor outputs differ in dependence on the external environment or on the movement environment. For example, if the walking pattern remains the same, the sensor output values differ when the robot is walking on a road presenting a gradient, on the labile road surface, such as on the gravel, or on the thick-piled carpet. The present embodiment is meritorious in that, by burying the reference sensor information in the motion data, it is possible to write motion data in keeping up with a specified movement environment or with the robot using configuration.

[0111] FIGS. 6 to 9 show typical file formats of the motion data. In the present embodiment, the motion data includes the information on the angles of joints, posture information, information on the ZMP trajectory, and the information on the contact of the foot sole with the floor surface. Referring to FIG. 6, the information on the angles of joints is the chronological data comprised of the displacements, arrayed chronologically, that is, every sampling interval of the joint actuators representing the degrees of freedom of the respective joints of the legged mobile robot 100 at the time of execution of the motions. The records at an interval of a sampling interval are formed by the combination of the angle command values and the measured angle values of the respective joints in executing the motions. In the above figures, R_joint name denotes the angle command value for the relevant actuator at the time of editing, while M_joint name denotes the measured value of the relevant actuator when the robot executes the motion.

[0112] Referring to FIG. 7, the posture information is the chronological data comprised of the information, arrayed chronologically, that is, every sampling interval of the posture sensors (gyro sensors) installed on the respective sites on the robot body of the legged mobile robot 100 at the time of the motion execution. The records at the respective sampling time points are constructed by the combination of the target values for the respective sensors, measured values at the time of motion edition and the measured values corresponding to filtered sensor outputs. In FIG. 7, R_sensor name denotes a target value of a relevant sensor at the time of editing, M_sensor name denotes a measured value in the relevant sensor at the time of executing the robot motion and F_sensor name denotes measured values corresponding to filtered sensor outputs at the time of the execution of the robot movements.

[0113] Referring to FIG. 8, the ZMP trajectory information is the chronological data comprised of ZMP positions of the legged mobile robot 100 at the time of motion execution, which are arrayed chronologically, that is, every sampling time interval. The records taken every sampling period are comprised of the combination of the target ZMP trajectory, at the time of editing, of the left and right foot soles, and the ZMP trajectory as corrected by stabilization control in motion execution. Referring to FIG. 8, the R_ZMP trajectory is the target ZMP trajectory at the time of editing, whilst M_ZMP trajectory denotes the ZMP trajectory as corrected by stabilization control at the time of the motion execution by the robot.

[0114] Referring to FIG. 9, the information on the foot sole touchdown is the chronological data comprised of measured values, arrayed chronologically, that is, every sampling time interval, of the respective floor reaction sensors mounted to the foot soles of the legged mobile robot 100, at the time of the motion execution. The records taken every sampling time interval are formed by the combination of the target values at the time of editing of the floor reaction sensors and the measured values thereof at the time of the motion execution. In FIG. 9, the R_touchdown information denotes the target value at the time of editing, whilst M_touchdown information denotes measured values at the time of motion execution.

[0115] FIG. 10 shows, in the form of a flowchart, the processing sequence in evaluating motion data of the robot 100, based on the sensor information. This processing sequence corresponds to step S4 of the flowchart of FIG. 5.

[0116] The step S11 evaluates response properties of the actuator. More specifically, this step S11 evaluates followup characteristics to the angle command value to the actuator at the time of editing, when the motion data is executed on the actual robot (step S12). The actuator is affected in response characteristics depending on the upper limit velocity (acceleration) and on the upper limit angle. Moreover, if the inter-link interference occurs, the actuator becomes unable to follow up with the target. The difference between the target value and the measured value is calculated (step S13) and, if the difference is increased, the result of the evaluation is degraded. If the evaluation is not satisfactory, the contents are saved (step S14) and, if the evaluation is satisfactory, the evaluation is terminated.

[0117] In a step S15, the actuator torque is evaluated. More specifically, the actuator torque value and the number of revolutions in case of execution on the actual robot are acquired chronologically. The so acquired data is compared to a NT curve representing actuator characteristics to verify whether or not there is any movement which has exceeded the limit torque of the actuator (step S16). If the result is unsatisfactory, the contents are saved (step S17). If the result is satisfactory, the evaluation is terminated. Although the calculations of the torque by simulation are extremely time-consuming and are not exempt from errors, the correct information can be acquired instantaneously by acquiring the torque values from the actual robot connected to the editing system.

[0118] In a step S18, the posture is evaluated. More specifically, the difference between the values of the posture sensor and the ZMP trajectory, as scheduled at the time of motion edition, on one hand, and the sensor values and the ZMP trajectory, executed on the actual robot, on the other hand, is calculated (step S19). If the difference is increased, the evaluation is degraded (step S20). If the evaluation has not yet been achieved, the contents are saved (step S21) and, if the evaluation has been achieved, the evaluation comes to a close. The so saved difference information is utilized as a parameter for posture stabilizing control at the time of re-editing.

[0119] In a step S22, the touchdown is evaluated. More specifically, the difference value between the posture at the time of motion editing and the measured values when the motion is executed on the actual robot is calculated (step S23) to evaluate the difference value (step S24). If the evaluation has not yet been achieved, the contents are saved (step S25) and, if the evaluation has been achieved, the evaluation comes to a close. An offset from the scheduled movement of the contact point and the contact timing of the robot with an outside world leads to application to the robot of unknown and impulse-like external force and the moment of the external force, so that, even if real-time adaptive control is applied, the effect such offset has on the stability of the robot movement is significant. However, an extremely robust motion may be generated by again carrying out the motion correction and the processing for stabilizing the posture (movement) using an offset between the touchdown point and the states of touchdown and flight, when the motion generated in an ideal virtual space is executed, on one hand, and those when the motion is executed on the actual robot, on the other hand.

[0120] In a step S26, the degree of achievement of the correction is evaluated. That is, the degree of the improvement of the measured values over the motion corrected by the evaluation of the last and previous evaluation events calculated (step S27) and comprehensive evaluation is again carried out in dependence on the falling priority sequence of the respective items (step S28). If the evaluation has not been achieved, the contents are saved (step S29) and, if the evaluation has been achieved, the evaluation is terminated.

[0121] In a step S30, impact evaluation by external contact is evaluated. More specifically, the effect of the impact on the actuator torque, ZMP offset and on the acceleration is calculated (step S31) to evaluate the effect (step S32). If the evaluation has not been achieved, the contents are saved (step S33) and, if the evaluation has been achieved, the evaluation is terminated. It is extremely difficult to model the outside world of the robot accurately, if in particular the human living environment is taken into account. Thus, a motion accompanied by collision against the outer object is first generated, in an ideal virtual space, using a simple collision model in which the run time by a simulator is not impracticably protracted. Then, using an offset from the force information on the occasion of contact with the outside object in case the motion is executed on the actual robot, the motion correction and the processing of stabilizing the posture (movements) are again performed to allow generation of an extremely robust motion in a short time.

[0122] FIG. 11 shows, in the form of a flowchart, the sequence of movements for correcting the motion, which is equivalent to the step S5 in the flowchart shown in FIG. 5.

[0123] The motion correction may be executed manually or automatically on a motion editing system.

[0124] In the case of manual correction, the correction is carried out as the difference between the command value and the measured value and the improved contents by the last and previous corrections are visually checked by having reference to e.g. a graph in a step S41.

[0125] In the case of the automatic correction, an angle command value to the actuator is corrected, in a step S42, by exploiting the contents of the results of evaluation of the response characteristics in the step S11. The control parameter, such as PID, of the actuator is corrected, in a step S43, by exploiting the results of evaluation of the response characteristics. Also, the site of priority for stabilization of the posture stabilizing processing block is changed in a step S44 by exploiting the results of evaluation of the actuator torque in the step S15. Moreover, the contents of the posture stabilization processing block is changed in a step S45 by exploiting the results of evaluation of posture stabilization in the step S18. Also, the result of touchdown evaluation in the step S22 is used to change the contents of the posture stabilization block (step s46). Moreover, the control of the posture stabilization processing block, which takes contact with an outside object into account, is changed in a step S47, by exploiting the result of evaluation of the impact due to contact with the outside object in the step S30.

[0126] The motion editing processing of the present embodiment is featured by the fact that motion data may be reproduced and evaluated on the actual robot body, that the motion may be corrected on the basis of the sensor information acquired during reproduction of the motion data and that the motion data having the reference information embedded therein may be acquired as the processing results.

[0127] When the motion data is reproduced on the actual robot body, only an optional range of the motion data is taken out and reproduced on the actual robot body to improve the efficiency of the motion editing movement.

[0128] It is noted that a motion is composed of a chronological combination of two or more poses. If a dynamic motion exploiting positive continuous acceleration (continuous dynamic movement) is to be executed, it is impossible to execute the motion from an optional intermediate portion to make an evaluation.

[0129] In the present embodiment, such continuous dynamic movement is enabled to be reproduced as from an optional time point and such reproduction is also enabled to be halted as from an optional time point to reduce the time needed for motion evaluation appreciably.

[0130] FIG. 12 shows, in the form of a flowchart, the sequence of movements for processing the reproduction and the halting of the reproduction as from an optional time point of the continuous dynamic movement stated by the motion data of the robot 100.

[0131] First, the start time point in the motion data is set by e.g. a user input (step S51).

[0132] Next, the dynamic posture at this start time point is calculated (step S52). The transient motion, having the dynamic posture at the start time point as a terminal point, is generated (step S53) and, using this transient motion, the motion on the actual robot is reproduced (step S54).

[0133] Then, by e.g. a user input, the stop time in the motion data is set (step S55). The dynamic posture at the stop time point is calculated (step S56) and the transient motion having the stop posture as a start point is generated (step S57) and, using this transient motion, the movement of the actual robot is halted (step S58).

[0134] FIG. 13 shows, in the form of a flowchart, a modification pertinent to motion editing processing.

[0135] First, motion data is formulated on the motion formulation editing system (step S61).

[0136] In re-editing the motion, the motion data is re-edited manually or automatically in accordance with the processing sequence described above with reference to FIG. 11 (step S62).

[0137] The sites for stabilization on the virtual space are then selected and the priority sequence of the selected sites is specified in order to stabilize the motion (step S63).

[0138] The movement impression of the stabilized motion is checked on the motion formulation editing system (step S64). If the motion is the desirable motion, processing transfers to a step S65 and, if otherwise, processing reverts to the step S62.

[0139] In the step S65, the motion data for editing is a taken into the actual robot. In a step S66, the actual robot is used to check the impression of the movement.

[0140] It should be noted that the motion may be reproduced in its entirety or only partially subject to designation of partial reproduction. If the motion is a desired one, processing transfers to a step S67. If the motion is not the desired one, processing reverts to the step S62 for re-selection of the site for stabilization, re-designation of the priority sequence and for motion re-edition.

[0141] In the step S67, the sensor information of the actual robot is taken into the motion formulation editing device.

[0142] The movements of the robot are evaluated on the motion formulation editing apparatus in accordance with the processing sequence described above with reference to FIG. 10 in a step S68. If the criterium for evaluation has been met, processing transfers to a step S69 and, if otherwise, processing transfers to a step S70.

[0143] In the step S69, a final motion data file, composed of the motion data for editing and the reference data file, embedded therein, is formulated (see FIGS. 6 to 9).

[0144] In the step S70, the contents of the evaluation are saved. These saved contents are utilized in the motion re-editing (see FIGS. 6 to 9).

[0145] The present invention has so far been elucidated with reference to certain specific embodiments thereof. However, as may be apparent to those skilled in the art, various changes, substitutions or equivalents may be envisaged without departing from the scope and the purport of the invention as defined in the appended claims. The purport of the present invention is not necessarily limited to a product termed a robot. That is, the present invention may be applied to any mechanical apparatus performing movements similar to those of the human being, based on electrical or magnetic actions, even though the apparatus belongs to other field of the industry, such as toys.

[0146] In sum, the present invention has been disclosed by way of illustration and the contents of the description of the present specification is not to be construed in a limiting sense. For understanding the purport of the present invention, reference is to be made to the description of the appended claims.