ROBOT SYSTEM AND CALIBRATION METHOD

[0001] This application is based on and claims priority to Japanese Patent Application No. 2018-142560 filed on Jul. 30, 2018, the entire content of which is incorporated herein by reference.

[0002] This invention relates to a robot system and a calibration method.

[0003] A first patent document discloses an apparatus which performs calibration by using a calibration jig so as to correct distortion of a lens and the like of an imaging device. (See PTL 1, for example.)

• [0004] {PTL 1}

[0005] Japanese Patent Publication No. 4021413

[0006] A robot system according to a first aspect of the present invention includes a robot; a tool which is attached to a distal end portion of the robot; an imaging device which is attached to the robot or a predetermined support portion; and a controller which controls the robot and the tool; wherein, the controller performs: operation processing for calibration which arranges the distal end portion of the robot at a plurality of calibration positions; tool operation processing which operates the tool at each of the plurality of the calibration positions, imaging processing which causes the imaging device to capture light irradiated to a predetermined object or a trace left on the predetermined object in the tool operation processing, and, calibration processing which performs calibration based on comparison between irradiation positions of the light or positions of the traces, which are in an image captured by the imaging device, and predetermined reference positions.

[0007] A calibration method according to a second aspect of the present invention includes a calibration arrangement step which places a distal end portion of a robot, at which a tool is provided, at a plurality of calibration positions; a tool operation step which operates the tool at each of the plurality of the calibration positions; an imaging step which causes an imaging device to capture light irradiated to a predetermined object or a trace left on the predetermined object due to the tool operation step; and, a calibration step which performs calibration based on comparison between irradiation positions of the light or positions of the traces, which are in an image captured by the imaging device, and predetermined reference positions.

[0008] A calibration method according to a third aspect of the present invention includes a calibration arrangement step which places a distal end portion of a robot, at which a laser processing tool for performing a predetermined laser processing is attached, at a plurality of calibration positions; a tool operation step which causes the laser processing tool, at each of the plurality of the calibration positions, to radiate laser beam whose intensity is lower than that used for the predetermined laser processing; an imaging step which causes an imaging device to capture an image of the laser beam irradiated to a predetermined object due to the tool operation step; and a calibration step which performs calibration based on comparison between irradiation positions of the laser beam in the image captured by the imaging device and predetermined reference positions.

[0017] A robot system according to a first embodiment of the present invention will be described below with reference to the accompanying drawings.

[0018] The robot system according to this embodiment includes a robot 10, a tool 20 which is attached at a distal end of the robot 10, a controller 30 which controls the robot 10 and the tool 20, and an imaging device 40 which is a two-dimensional camera, a three-dimensional camera, a three-dimensional distance sensor, or the like. The imaging device 40 is supported by a support member 41.

[0019] In this embodiment, the tool 20 is a laser welding tool which irradiates laser beam so as to perform laser welding, a laser processing tool which performs laser cutting, or the like. In such a case where the tool 20 is the laser welding tool, a nozzle 21 is provided at a distal end of the tool 20, and a through-hole (not shown) which has, for example, a diameter of approximately 1 mm is provided in the nozzle 21. The through-hole extends in a direction along a longitudinal axis line of the tool 20. The tool 20 is connected to a laser oscillator 22 (FIG. 2), and laser beam which is generated from the laser oscillator 22 is irradiated onto a processing object W from the through-hole of the nozzle 21. Also, the tool 20 is connected to an assist gas supply device 23 (FIG. 2). Assist gas is supplied to the tool 20 from the assist gas supply device 23, and the assist gas is made to blow on the processing object W via the through-hole of the nozzle 21 or a hole for the assist gas. In order to cut the processing object W, the laser beam is irradiated to the processing object W and the assist gas is made to blow on the processing object W. Also, the tool 20 may be a laser processing tool which performs other processing.

[0020] The robot 10 includes a plurality of arm members and a plurality of joints. And, the robot 10 includes a plurality of servo motors 11 which respectively drive the plurality of joints (refer to FIG. 2). As the servo motors 11, a various kinds of servo motors, such as a rotary motor, a linear motor, and the like may be used. Each of the servo motors 11 has an operation position detection device which detects an operation position and operation speed of each of the servo motors 11, and one example of the operation position detection device is an encoder. Detected values of the operation position detection devices are sent to the controller 30.

[0021] As shown in FIG. 2, the controller 30 includes a controller 31 having a processor and the like, a display 32, a storage unit 33 having a non-volatile storage, a ROM, a RAM, and the like, an input device 34 which is a teach pendant and the like, a transmitting and receiving device 35 for transmitting and receiving signals, and servo controllers 36 which are respectively connected to the servo motors 21. The input device 34 and the transmitting and receiving device 35 function as an input unit.

[0022] As shown in FIG. 2, the controller 30 is connected to the laser oscillator 22 and the assist gas supply device 23. On the basis of an operation program 33b which is described below, the controller 30 controls the laser oscillator 22 and the assist gas supply device 23 so as to cut the object W.

[0023] A system program 33a is stored in the storage unit 33, and the system program 33a provides a basic function of the controller 30. Also, an operation program 33b is stored in the storage unit 33. The controller 31 reads the operation program 33b, and controls the servo controllers 36, the laser oscillator 22, and the assist gas supply device 23 on the basis of the operation program 33b. By this, the robot 10 moves a tool 40 along a processing locus PL, which is shown in FIG. 1, for example, so that the processing object W is cut along the processing locus PL.

[0024] An operation program for calibration 33c, a tool operation program 33d, an imaging program 33e, and a calibration program 33f are stored in the storage unit 33.

[0025] Due to distortion of a lens of the imaging device 40 and the like, a position of an article and the like in the images captured by the imaging device 40 may be different from an actual position. In order to correct such difference, conventionally, a plate-like jig having a plurality of dots painted on its surface is fixed to a predetermined position, and images of the dots on the fixed jig are captured by the imaging device 40. And, correction data is created (calibration) on the basis of difference between positions of the plurality of dots in the images captured by the imaging device 40 and positions where the dots are supposed to be placed (predetermined reference positions). This correction data is used in order to correct the images captured by the imaging device 40.

[0026] On the other hand, in this embodiment, the controller 31 performs the below described calibration on the basis of the operation program for calibration 33c, the tool operation program 33d, the imaging program 33e, and the calibration program 33f. Processing of the controller 31 at the time of performing the calibration is shown in the flowchart of FIG. 3.

[0027] At first, when the controller 31 receives a start signal which is input by using the input device 34, the transmitting and receiving device 35, and the like (Step S1-1), on the basis of the operation program for calibration 33c, the controller 31 places the distal end portion of the robot 10 at one of a plurality of calibration positions (Step S1-2). Also, on the basis of the tool operation program 33d, the controller 31 causes the tool 20 to radiate laser beam of which intensity is lower than that used for predetermined laser processing such as cutting processing and the like when the distal end portion of the robot 10 is placed at the calibration position in Step S1-2 (Step S1-3). The laser beam which is irradiated in Step S1-3 has an intensity of few percent of the intensity of the laser beam which is used for the laser processing, and the intensity of less than 5% is desirable.

[0028] In this embodiment, the tool 40 which is attached at the distal end portion of the robot 10 irradiates the laser beam in a diagonally downward direction at each of the calibration positions. For example, when the upper surface of the object W extends in a horizontal direction, the laser beam and the upper surface of the object W form an angle of 80 degrees. Also, the laser beam and the upper surface of the object W may form an angle of 90 degrees, and may form any other angle.

[0029] Also, on the basis of the imaging program 33e, the controller 31 uses the imaging device 40 so as to capture images of the upper surface of the object W in a state where the low intensity laser beam is being radiated from the tool 20 (Step S1-4).

[0030] Also, on the basis of the imaging program 33e, the controller 31 performs known image processing such as binarization processing, machining processing, and the like on the image captured by the imaging device 40, and by this, the controller 31 detects irradiation position of the low intensity laser beam on the upper surface of the object W in each of the images (Step S1-5), and the controller 31 stores the detected irradiation position so as to be associated with the position of the distal end portion of the robot 10 and the like in the storage unit 33 (Step S1-6). The position of the distal end portion of the robot 10 may be a position which is calculated from detected values of the operation position detection devices, and it may also be a position which is intended by the operation program for calibration 33c.

[0031] The controller 31 repeatedly performs Steps S1-2 to S1-6 for a predetermined number of times (Step S1-7). Also, in Step S1-2, the distal end portion of the robot 10 is sequentially positioned at the plurality of calibration positions. After Steps S1-2 to S1-6 have repeatedly performed for the predetermined number of times (Step S1-7), on the basis of the calibration program 33f, the controller 31 creates the correction data, as the calibration, on the basis of the difference between each of the irradiation positions, which are stored in the storage unit 33, and each of the positions where the irradiation positions are supposed to be placed in the images (the predetermined reference positions) (Step S1-8), and stores the created correction data in the storage unit 33 (Step S1-9). The correction data is data which corrects distortion occurred due to aberration of the lens of the imaging device 40 and the like, such as distortion seen in the image of end portion sides of the field of view of the imaging device 40. That is to say, by correcting the images captured by the imaging device 40 by using the correction data, the distortion disappears or the distortion is suppressed in images after correction.

[0032] In one example, as shown in FIG. 4, the low intensity laser beam is irradiated onto a plurality of irradiation points P which are on the upper surface of the object W by the operation program for calibration 33c and the tool operation program 33d. That is to say, each of the positions of the irradiation points P, relative positional relationship between the irradiation positions P, and the like are known. On the basis of the positions of the irradiation points P and the relative positional relationship of the irradiation points P, which are known, it is possible to obtain the positions where the irradiation points P are supposed to be placed.

[0033] The irradiation positions of the low intensity laser beam which is irradiated as described above is, for example, detected by the imaging program 33e as shown in FIG. 5. In FIG. 5, a shape that the object W is supposed to be and positions where the irradiation points P are supposed to be placed are shown in a broken line, and the detected irradiation positions IP are shown in a solid line. And, the end portion of the object W where the aberration of the lens is likely to occur is drawn in FIG. 5.

[0034] The correction data matches each of the irradiation positions IP, which are shown in FIG. 5, to each of the irradiation points P. The controller 31 also obtains interpolation correction data between the irradiation points P in the images by a known interpolation, and the obtained interpolation correction data is stored in the storage unit 33 as the correction data. The correction data is based on the difference between the irradiation positions and the positions where the irradiation positions are supposed to be placed, and the correction data eliminates or reduces this difference.

[0035] A robot system according to a second embodiment of the present invention will be described below with reference to FIG. 6 to FIG. 8.

[0036] The second embodiment is different from the first embodiment in that the tool 20 is an arc welding tool. The configurations which are the same as or similar to those in the first embodiment are assigned the same reference symbols and their explanation is omitted.

[0037] The tool 20 includes a tungsten electrode 24 and a meltable metal wire supply device 25 (FIG. 6), and the control unit 31 of the controller 30 controls the servo controllers 36, the tungsten electrode 24, and the meltable metal wire supply device 25 on the basis of the operation program 33b, and by this, the welding processing with respect to the object W is performed.

[0038] In the second embodiment also, the calibration which is described below is performed. Processing of the control unit 31 at the time of performing the calibration of the second embodiment is shown in FIG. 7.

[0039] At first, when the control unit 31 receives the start signal which is input by using the input device 34, the transmitting and receiving device 35, and the like (Step S2-1), on the basis of the operation program for calibration 33c, the control unit 31 places the distal end portion of the robot 10 at one of the plurality of calibration positions (Step S2-2). Also, on the basis of the tool operation program 33d, the control unit 31 controls the tungsten electrode 24 and the meltable metal wire supply device 25 of the tool 20 so that the tool 20 is caused to perform the welding when the distal end portion of the robot 10 is placed at the calibration position in Step S2-2 (Step S2-3).

[0040] The control unit 31 repeatedly performs Steps S2-2 to S2-3 for a predetermined number of times (Step S2-4). Also, in Step S2-2, the distal end portion of the robot 10 is sequentially positioned at the plurality of calibration positions. After Steps S2-2 to S2-3 have repeatedly been performed for the predetermined number of times (Step S2-4), the control unit 31 uses the imaging device 40 so as to capture the images of the upper surface of the object W on the basis of the imaging program 33e (Step S2-5).

[0041] Also, on the basis of the imaging program 33e, the control unit 31 performs known image processing such as the binarization processing, the machining processing, and the like on the images captured by the imaging device 40, and by this, the control unit 31 detects positions of the welding marks (traces), respectively, on the upper surface of the object W in the obtained images (Step S2-6), and stores the detected positions of the welding marks in the storage unit 33 (Step S2-7).

[0042] Also, similar to the first embodiment, every time the welding is performed in Steps S2-2 and S2-3, it may be possible to perform processing which captures images by using the imaging device 40, and processing which stores the detected positions of the welding marks the captured images so as to be associated with the positions and the likes of the distal end portion of the robot 10 in Step S2-2.

[0043] Next, on the basis of the calibration program 33f, the control unit 31 creates correction data, which plays the role of the calibration, on the basis of differences between the positions of the welding marks, which are stored in the storage unit 33, and the positions where the welding marks are supposed to be positioned in the images (predetermined reference positions) (Step S2-8), and the control unit 31 stores the created correction data in the storage unit 33 (Step S2-9).

[0044] In one example, the welding marks are formed in a grid pattern on the upper surface of the object W due to the operation program for calibration 33c and the tool operation program 33d. That is to say, each of the positions of the plurality of welding marks, positional relationship between the welding marks with each other, and the like are known. On the basis of the positions of the welding marks and the positional relationship between the welding marks with each other, which are known, it is possible to obtain positions where the welding marks are supposed to be placed.

[0045] The positions of the welding marks MP which are irradiated as described above is, for example, detected by the imaging program 33e as shown in FIG. 8. In FIG. 8, a shape in which the object W is supposed to be and positions where the welding marks MP are supposed to be placed are shown in a broken line, and the detected positions of the welding marks MP are shown in a solid line. And, FIG. 8 shows the end portion of the object W where the aberration of the lens of the imaging device 40 is likely to occur.

[0046] Also, in the second embodiment, the tool 20 may be a dispenser tool which applies a sealing agent. In this case, the controller 30 controls the robot 10 and the tool 20 so as to perform work which applies the sealing agent to the object W. Moreover, in Step S2-3, the control unit 31 controls the tool 20 so as to cause the tool 20 to apply the sealing agent. It is preferable that the sealing agent at this time is applied in a dot shape.

[0047] And, in Step S2-6, the control unit 31 detects positions of the application portions (traces) in each of which the sealing agent is applied on the upper surface of the object W, and in Step S2-7, the control unit 31 stores the detected positions of the application portions in the storage unit 33. Also, in Step S2-8, the control unit 31 creates the correction data, which plays the role of the calibration, on the basis of differences between the positions of the application portions, which are stored in the storage unit 33, and the positions where the application portions are supposed to be placed in the images.

[0048] In the second embodiment, the tool 20 may be a laser processing tool which performs the laser processing. In this case, in Step S2-3, the control unit 31 controls the tool 20 so as to cause the tool 20 to perform the laser processing. It is preferable that the traces of the laser processing at this time is in a dot shape.

[0049] And, in Step S2-6, the control unit 31 detects the positions of the traces of the laser processing on the upper surface of the object W, and in Step S2-7, the control unit 31 stores the detected positions of the traces in the storage unit 33. Also, in Step S2-8, the control unit 31 creates the correction data as the role of the calibration on the basis of differences between the positions of the traces, which are stored in the storage unit 33, and the positions where the traces are supposed to be placed in the images.

[0050] Also, in each of the above described embodiments, it is possible to perform calibration (modification) for the robot 10 or for the operation program 33b. This calibration is performed in a state where the calibration of the aberration of the lens of the imaging device 40 and the like has already been performed in Steps S1-8, S1-9, S-2-8, and S2-9, for example, and where an influence of the distortion of the lens can be eliminated from the images captured by the imaging device 40.

[0051] For example, on the basis of the operation program 33b, the control unit 31 controls the robot 10 and the tool 20, and at this time, the control unit 31 causes the tool 20 to irradiate the low intensity laser beam so that the control unit 31 can detect the position, a locus, and the like of the laser beam by using the images captured by the imaging device 40. And, the control unit 31, an operator, and the like can perform the calibration of the robot 10 or the operation program 33b by using the position, the locus, and the like of the laser beam which are detected.

[0052] In one example, in response to rigidity of the robot 10, internal mass of the robot 10, internal mass of the tool 20, operation speed of the robot 10, orientation of the robot 10, and the like, there may be a case where the distal end portion of the robot 10 vibrates during the operation of the robot 10, which is on the basis of the operation program 33b. When the laser processing, such as cutting processing using the laser beam, is performed in a state where the distal end portion of the robot 10 is shaking, quality of a processing portion is degraded. And therefore, the calibration of the robot 10 or the calibration of the operation program 33b may be performed in order to improve the quality of the processing portion.

[0053] Also, each of the above described embodiments shows that the irradiation of the laser beam, the welding, or the application of the sealing agent is performed with respect to the object W, which is a plate-like work, however, anything can be used as the object W. For example, the object W may be a frame of a bicycle, a various kinds of jigs, an upper surface of a transportation device, a floor surface of the transportation device, and the like.

[0054] In this embodiment, the controller 30 is a robot controller which controls the operation of the robot 10, however, the controller 30 may be other controller having the configuration which is described in the embodiment. For example, the controller 30 may be another controller or a host controller, which is connected to the robot 10. In this case, the other controller or the host controller stores at least one of the operation program for the calibration 33c, the tool operation program 33d, the imaging program 33e, and the calibration program 33f, and the other controller or the host controller performs the above processing on the basis of the stored program.

[0055] And, in each of the above described embodiments, the imaging device 40 may be attached at the distal end portion and the like of the robot 10. In this case also, the imaging device 40 captures the images of the upper surface of the object W having the welding marks, the application portions of the sealing agent, and the like, and processing which is the same as or similar to the above described processing is performed.

[0056] In each of the above described embodiments, when the distal end portion of the robot 10 is placed at each of calibration positions, the laser beam is irradiated to the object W or the traces are formed on the object W due to the operation of the tool 20, and the calibration is performed on the basis of comparison between the irradiation positions of the laser beam or the positions of the traces, which are in the images captured by the imaging device 40, and predetermined reference positions. And therefore, for example, it is not necessary to prepare a calibration jig on which marks are positioned in a grid pattern. And, since it is also possible to use a work which is a work target as the object W, the calibration can accurately be performed on the imaging device 40 with respect to the operation positions. It is also possible to perform the calibration intensively on a surrounding of the operation positions.

[0057] Also, in some of the above described embodiments, the tool 20 is the laser processing tool which performs predetermined laser processing by irradiating the laser beam, and in order to perform the calibration, the control unit 31 causes the tool 20 to irradiate the laser beam of which intensity is lower than that used for the laser processing.

[0058] With this configuration, the tool 20 irradiates the laser beam of which intensity is lower than that used for the laser processing. And therefore, in such a case where the object W is a work which is a target of the laser processing, it is possible to perform actual laser processing on the work after performing the calibration by using the work. In this case, the calibration is accurately performed on the object on which the laser processing is performed.

[0059] Also, in one of the above described embodiments, the tool 20 is the arc welding tool which performs the arc welding, and the control unit 31 causes the tool 20 to perform the arc welding in order to perform the calibration.

[0060] With this configuration, in such a case where the object W is a work which is a target of the arc welding, the calibration is accurately performed on the object on which the arc welding is performed.

[0061] Also, in one of the above described embodiments, the tool 20 is the dispenser tool which applies the sealing agent, and the control unit 31 causes the tool 20 to apply the sealing agent in order to perform the calibration.

[0062] With this configuration, in such a case where the object W is a work which is an application target of the sealing agent, the calibration is accurately performed on the object to which the sealing agent is applied.

[0063] The following aspects of the present invention are derived from the above disclosure.

[0064] A robot system according to a first aspect of the present invention includes a robot; a tool which is attached to a distal end portion of the robot; an imaging device which is attached to the robot or a predetermined support portion; and a controller which controls the robot and the tool; wherein, the controller performs: operation processing for calibration which arranges the distal end portion of the robot at a plurality of calibration positions; tool operation processing which operates the tool at each of the plurality of the calibration positions, imaging processing which causes the imaging device to capture light irradiated to a predetermined object or a trace left on the predetermined object in the tool operation processing, and, calibration processing which performs calibration based on comparison between irradiation positions of the light or positions of the traces, which are in an image captured by the imaging device, and predetermined reference positions.

[0065] With the first aspect, when the distal end portion of the robot is placed at each of the calibration positions, the light is irradiated to the object or the trace is formed on the object due to the operation of the tool, and the calibration is performed on the basis of the comparison between the irradiation positions of the light or the positions of the traces, which are in the images captured by the imaging device, and the predetermined reference positions. Therefore, it is not necessary to prepare a dedicated calibration jig on which marks are positioned in a grid pattern, for example. Also, since it is also possible to use a work which is a work target as the object, it is possible to accurately perform the calibration on the imaging device with respect to the operation position. It is also possible to perform the calibration precisely on a surrounding of the operation position.

[0066] In the above aspect, preferably, the tool is a laser processing tool which performs predetermined laser processing by radiating laser beam, and the controller causes, in the tool operation processing, the tool to radiate the laser beam whose intensity is lower than that used for the predetermined laser processing.

[0067] With this aspect, the tool irradiates the laser beam whose intensity is lower than that used for the laser processing. And therefore, in such a case where the object is a work which is a target of the laser processing, it is possible to perform actual laser processing on the work after performing the calibration by using the work. In this case, the calibration is accurately performed on the object which is to be laser processed.

[0068] In order to reduce or eliminate damage of the object, it is preferable that the laser beam whose intensity is lower than that used for the predetermined laser processing is laser beam having an intensity with that the predetermined laser processing object is not processed.

[0069] In the above aspect, preferably, the tool is an arc welding tool which performs arc welding, and the controller causes the tool to perform the arc welding in the tool operation processing.

[0070] With this aspect, in such a case where the object is a work which is a target of the arc welding, the calibration is accurately performed on the target of the arc welding.

[0071] In the above aspect, preferably, the tool is a dispenser tool which applies a sealing agent, and the controller causes the tool to apply the sealing agent in the tool operation processing.

[0072] With this aspect, in such a case where the object is a work which is an application target of the sealing agent, the calibration is accurately performed on the application target of the sealing agent.

[0073] A calibration method according to a second aspect of the present invention includes a calibration arrangement step which places a distal end portion of a robot, at which a tool is provided, at a plurality of calibration positions; a tool operation step which operates the tool at each of the plurality of the calibration positions; an imaging step which causes an imaging device to capture light irradiated to a predetermined object or a trace left on the predetermined object due to the tool operation step; and, a calibration step which performs calibration based on comparison between irradiation positions of the light or positions of the traces, which are in an image captured by the imaging device, and predetermined reference positions.

[0074] A calibration method according to a third aspect of the present invention includes a calibration arrangement step which places a distal end portion of a robot, at which a laser processing tool for performing a predetermined laser processing is attached, at a plurality of calibration positions; a tool operation step which causes the laser processing tool, at each of the plurality of the calibration positions, to radiate laser beam whose intensity is lower than that used for the predetermined laser processing; an imaging step which causes an imaging device to capture an image of the laser beam irradiated to a predetermined object due to the tool operation step; and a calibration step which performs calibration based on comparison between irradiation positions of the laser beam in the image captured by the imaging device and predetermined reference positions.

[0075] According to the above aspects, it is possible to reduce effort which is required for adjusting a robot system including an imaging device, and it is possible to accurately perform calibration of the imaging device.

• [0076] 10 robot
• [0077] 20 tool
• [0078] 21 nozzle
• [0079] 22 laser oscillator
• [0080] 23 assist gas supply device
• [0081] 24 tungsten electrode
• [0082] 25 meltable metal wire supply device
• [0083] 30 controller
• [0084] 31 control unit
• [0085] 32 display
• [0086] 33 storage unit
• [0087] 33a system program
• [0088] 33b operation program
• [0089] 33c operation program for calibration
• [0090] 33d tool operation program
• [0091] 33e imaging program
• [0092] 33f calibration program
• [0093] 34 input device
• [0094] 35 transmitting and receiving device
• [0095] 36 servo controllers
• [0096] 40 imaging device
• [0097] W object
• [0098] PL processing locus
• [0099] IP detected irradiation position or detected position of welding mark
• [0100] P irradiation point
• [0101] MP positions of welding marks