Optical measurement system

[0001] This application claims the benefit of foreign priority to European Patent Office (EPO) Application EP10004264.7, filed on Apr. 22, 2010.

[0002] The present invention relates to an optical measurement system for measuring spatial coordinates and/or orientation of a probe or probing system.

[0003] From EP 0 607 303 B1, an opto-electronic system for point-by-point measurement of spatial coordinates is known. The system described provides a probe comprising a minimum of three light sources at known coordinates relative to a local probe fixed coordinate system and with a touch point at a known location relative to said local coordinate system is brought into contact with the point for which the spatial coordinates are to be determined. An opto-electronic spatial direction sensor determines the angles in two dimensions (spatial direction) towards the light sources. Based on the known positions of the minimum three light sources and the touch point in the local probe fixed coordinate system with respect to each other and based on the determined spatial directions of the individual light sources and the spatial direction sensor, the system can deduce by photogrammetric calculation methods the position and orientation of the probe and therefore the spatial coordinates of the touch point.

[0004] Compared to optical measurement systems of the prior art, the present invention provides unlimited or spherical field of view in combination with a variety of probes and probing systems. The present invention therefore provides a new level of flexibility in terms of measurement tasks that can be handled with a single system compared to the prior art. This provides significant operational and investment advantages in industries such as aerospace, automotive and other industries manufacturing complex assembled products.

[0005] It is the object of the present invention to overcome shortcomings of the prior art and to present a system which provides a highly accurate measurement of spatial coordinates and/or orientation of a probe, which can be used in enclosed setups and where space is limited and which is simple, easy and quick to use.

[0006] The present invention encompasses a system for measurement of spatial coordinates, a method for measurement of spatial coordinates, and a computer program for measurement of spatial coordinates. Advantageous embodiments are claimed in the dependent claims.

[0007] The inventive system has the advantage that the first spatial direction sensor has associated targets that permit it to be moved and turned around in every spatial direction as long as it stays within the field of view of the second spatial direction sensor. Therefore, the system is highly flexible and provides a 360 degree field of view. The first spatial direction sensor with its associated targets can be freely moved around—either manually or motorized—inside the field of view of the second spatial direction sensor without intermediate alignment to the coordinate system of the object to be measured. The probe has targets that are observed by the first spatial direction sensor, thus enabling the probe to measure anywhere regardless of the orientation and field-of-view of the second spatial direction sensor.

[0008] In addition, the system increases accuracy compared to prior art as the large field of view enables the distance from the spatial direction sensors to the targets and the probe to be minimized while avoiding the difficulties associated with extreme wide-angle optics. In addition, the positioning flexibility of the spatial direction sensors enables selection of an optimal setup dependent of the shape and size of the object to be measured.

[0009] Furthermore, due to the use of two independent spatial direction sensors, the system is able to look corners and into areas which are not accessible with a single spatial direction sensor.

[0010] In an advantageous embodiment, the probe has targets with known positions relative to each other, the processing means is further configured to compute the spatial coordinates and/or orientation of the probe based on the known orientation and/or spatial coordinates of the target, the known positions of the targets of the probe relative to each other and a determination of the spatial directions of the targets of the probe with respect to the first spatial direction sensor and at least three targets of the probe are in the field of view of the first spatial direction sensor.

[0011] By having at least three targets of the probe in the field of view of the first spatial direction sensor, the spatial coordinates and/or orientation of the probe can be determined.

[0012] In a further advantageous embodiment, the system comprises at least one further probe and/or at least one further first spatial direction sensor and/or at least one further second spatial direction sensor.

[0013] Due to this, the accuracy of determining the position and orientation of the first spatial direction sensor is increased by using more than one second spatial directions sensor in positions known relative to each other.

[0014] In a further advantageous embodiment, the first spatial direction sensor and its associated target can be rotated.

[0015] With this function, the first spatial direction sensor associated with the target is able to view in different directions, such that the spatial coordinates and/or orientation of probes located around the first spatial direction sensor can be determined.

[0016] In a further advantageous embodiment, the first spatial direction sensor and the second spatial direction sensor are mechanically connected.

[0017] Due to the fact that the second spatial direction sensor and the target with the first spatial direction sensor are mechanically connected, the distance from the second spatial direction sensor to the first spatial direction sensor is known. Therefore, only the orientation of the target of the first spatial direction sensor has to be determined. This significantly reduces the requirements with respect to calculation capacities of the processing means and improves accuracy.

[0018] Furthermore, the distance from the second spatial direction sensor to the target of the first spatial direction sensor can be kept short, while still having the possibility of accomplishing all spatial measurements of a system without mechanical or contact measuring. For example, the inventive system can be designed as compact and self-contained unit, which can be placed in a location whose surrounding has to be measured. The alignment in a global coordinate system may then be performed by reference points in the surrounding to be measured (for example reference points on a welding robot on the assembly line).

[0019] In a further advantageous embodiment, the mechanical connection is a frame structure consisting mainly of carbon fiber reinforced polymer.

[0020] Due to this, the mechanical connection is highly rigid and changes very little in response to changes in temperature. As the distance of the spatial direction sensors is firmly fixed to this rigid structure, all measurements become even more accurate.

[0021] In a further advantageous embodiment, at least one reference point is in the fields of view of the second spatial direction sensor and a further second spatial direction sensor and the processing means is further adapted to calibrate the spatial coordinate systems of the second spatial direction sensors based on the respective spatial coordinates of the at least one reference point.

[0022] This embodiment of the invention allows that the target moves, leaving the field of view of the second spatial direction sensor and entering in the field of view of a further second spatial direction sensor. As both second spatial direction sensors view the same reference point, their spatial coordinate system can be calibrated and thus melted into one spatial coordinate system, in which the target can move freely. This is especially important in assembly line applications where the target can be put on the product transported by the assembly line.

[0023] In a further advantageous embodiment, the target moves from the field of view of the second spatial direction sensor in a field of view of a further second spatial direction sensor, wherein at least one reference point is in the field of view of the first spatial direction sensor before and after the movement and the processing means is further adapted to calibrate the spatial coordinate systems of the second spatial direction sensors based on the respective spatial coordinates of the at least one reference point.

[0024] While this embodiment is similar to the preceding embodiment in that the target can be moved around freely, the reference point in this embodiment does not have to be in the fields of view of the second spatial direction sensors. This is advantageous as it is not always easy to find a reference point that is viewed by two second spatial direction sensors at the same time.

[0034] In the appended figures, similar components and/or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0035] There are two main weaknesses with the system described by EP 0 607 303 B1.

[0036] Firstly, the accuracy of the measurement is influenced by the ratio of the distance between the opto-electronic spatial direction sensor and the touch probe as well as the distance of the light sources on the touch probe with respect to each other. If the distance between the opto-electronic spatial direction sensor and the touch probe becomes too large, the angles measured by the opto-electronic spatial direction sensor in order to determine the spatial directions decrease to such a degree that accurate measurement is no longer possible.

[0037] In order to overcome this problem, the distance of the light sources on the touch probe with respect to each other may be increased, such that the angles become measurable again. However, if the touch probe is inclined with respect to the opto-electronic spatial direction sensor, even increasing the distances would not necessarily increase the measured angles sufficiently.

[0038] Also, in many applications, increasing the distance of the light sources on the touch probe with respect to each other is not practical, since the overall size of the touch probe is in practice limited due to the environment in which the system is to be used.

[0039] Thus, the measurement accuracy for the system described in EP 0 607 303 B1 deteriorates for longer distances, as the angles measured by the opto-electronic spatial direction sensor in order to determine the spatial directions of the light-sources decrease.

[0040] The addition of a second spatial direction sensor placed so that the lines from the two spatial direction sensors intersect at approximately right angles at the light sources can be used to overcome this problem. The accuracy is then improved by triangulation between the spatial direction sensors. Nevertheless, in such systems, the object to be measured must be inside the field of view of both cameras, as must the light sources of the probe.

[0041] Secondly, the opto-electronic spatial direction sensor has a finite field of view. For an object of a given size, this dictates the minimum distance between the object and the opto-electronic spatial direction sensor of the system. For large objects, this distance may become too large to achieve the required accuracy.

[0042] For other objects, a finite field of view may not be acceptable. As an example, measuring the exact shape of an enclosed space, such as a room to be fitted with a custom parquet floor, or a boiler to be fitted with new pipes, requires a measurement system that can operate through 360 degrees field of view, i.e. a system with unlimited field of view.

[0043] A typical example from industry is the so-called ‘framing station’ or ‘framer’ in an automotive production line. FIG. 1 schematically illustrates an automotive framer or framing station with roof 201, sides 202-1, 202-2 and floor tooling 203. This is the tool where two side panels 208-1, 209 and a roof panel 210, each held in the corresponding tooling, is aligned and welded to a floor assembly 211. Typically, the overall references or coordinate system definition is found either in external structures, or in the floor tooling 203.

[0044] Of critical importance to the overall geometry of the finished vehicle, the framer must be measured in the closed position that is with the tools for each of the sides roof and floor assemblies interlocked while side panels 208-1, 209 and roof panel 210 as well as the floor assembly 211 are not in place. Due to the complexity of the tooling, the closed framer appears as a closed box with all the details to be measured on the inside.

[0045] Sheet-metal locating points 206—typically pins and clamps for holding the panels and the floor assembly 211 whose spatial coordinates have to be determined, are measured with a touch point 204 of a first touch probe 205-1. The spatial coordinates of the touch point 204 shall be determined by the above mentioned prior art system. However, the distance between the spatial direction sensor 207 and the first touch probe 205-1 is too long for the spatial coordinates to be measured accurately.

[0046] In addition to that, parts of the framer or related equipment may cover a second touch probe 205-2 and a third touch probe 205-3, such that the second touch probe 205-2 and the third touch probe 205-3 are not in the field of view of the spatial direction sensor 207 and, therefore, spatial coordinates of the second touch probe 205-2 and the third touch probe 205-3 cannot be determined.

[0047] As typically no position for the spatial direction sensor described in EP 0 607 303 B1 can be found that simultaneously provides both a view of all parts of the framer to be measured and sufficiently short distances to all parts of the framer to provide the required accuracy, this prior art is not adapted for measuring framers. Since framers are a critical part of automotive production lines, this significantly reduces the usefulness and flexibility of systems such as that presented in EP 0 607 303 B1 in the automotive industry and other industries with similar enclosed objects to measure.

[0048] In this type of application the two camera approach provides no improvement, rather the need to find two suitable locations for spatial direction sensors while maintaining proper intersection angles makes the problem much more severe.

[0049] A number of approaches for expanding the effective field-of-view of a camera are known.

[0050] The most obvious is to choose wide-angle or ‘fish-eye’ optics. While wide-angle optics may increase field-of-view, it is not feasible to achieve an unlimited field-of-view. Additionally, as the field-of-view increases, so do the challenges related to accurately determining and cancelling the effects of lens imperfections—critical in a measurement application.

[0051] Using multiple cameras with partially-overlapping fields-of-view is an alternative or even complementary approach. Once the relative position and orientation of the cameras in the cluster of cameras is known, the measurements made with one camera can easily be related to measurements from any other camera, effectively creating a single camera with infinite field-of-view. Unfortunately, this approach also has challenges in practice, including high cost and complexity as well as the reliance on the relative camera positions to remain stable within extremely tight tolerances when subjected to the external influences such as temperature variations, shocks and vibration. An alternative approach would therefore be advantageous.

[0052] From FR 2870 594 A1 an approach is known by which a series of cameras form a ‘video arm’ from an area where the references are observed by the first camera, to the area of interest which is measured by a probe attached to the last camera, thus by passing any obstructions to direct line-of-sight between the references and the area of interest. Each camera in the ‘arm’ has targets placed upon it, and each camera observes the targets on the previous camera thus calculating its own position relative to the reference marks and thus the coordinate system.

[0053] The approach presented is well suited for e.g. aircraft boresighting applications, but presents a number of challenges for general measurement use. For example, in many applications, it is impossible or impractical for all reference marks to be within the field-of-view of the first camera—refer e.g. to the automotive framer presented above where this requirement would dictate long camera-to-object distances thus negating the advantage we are seeking. Also, having a camera associated with the probe is a challenge both regarding weight, fragility and cabling, as the probe is the unit the system operator will be handling during measurement. Perhaps most importantly, the approach lacks the critical element of providing targets viewable from all directions. Therefore, to achieve an infinite field-of-view will require a higher number of cameras than the following will show is necessary, adding cost and complexity both to the system per se and to its operation. At the same time, each additional ‘link’ in the chain of cameras adds measurement error and thus decreases overall system performance.

[0054] Alternative approaches involving the same basic concept of ‘cameras viewing camera with targets’ are also known from other documents, but none of these provide an electro-optical probing system with an infinite field-of-view capable of measuring a complete enclosed space.

[0055] In FIG. 2, the principle elements of a system 100 according to the invention are illustrated. The system 100 comprises a probe 101 to be measured, a first spatial direction sensor 102, a second spatial direction sensor 103, and a processing means 106.

[0056] The probe 101 is situated in the field of view FOV1 of the first spatial direction sensor 102. The first spatial direction sensor 102 is again situated in the field of view FOV2 of the second spatial direction sensor 103. While the second spatial direction sensor 103 determines the position and orientation of the first spatial direction sensor 102, the first spatial direction sensor 102 again determines the position and/or orientation of the probe 101.

[0057] In order to determine the position and/or orientation of the probe 101, the second spatial direction sensor 103 determines the spatial directions, i.e. spatial angles or solid angles, relative to targets 105 associated with the first spatial direction sensor 102 arranged in a pattern 104. Hence, the position and/or orientation of the pattern 104 and thus of the first spatial direction sensor 102 can be determined. The first spatial direction sensor 102 again determines the spatial coordinates and/or orientation of the probe 101.

[0058] The first spatial direction sensor 102 is free to move and rotate according to all six degrees of freedom in order to observe the probe 101 as long as the second spatial direction sensor 103 remains stationary and at least three targets 105 associated with the first spatial direction sensor 102 remain inside the field of view FOV2 of the sec- and spatial direction sensor 103. This provides a virtually unlimited field of view for the system 100. Therefore, the system is able to look behind corners or through 360 degrees inside an enclosed measurement volume.

[0059] For measuring, the first spatial direction sensor 102 is moved and rotated until its field of view FOV1 is pointing in the direction of a probe 101 whose position and/or orientation needs to be determined. The system 100 is able to transform the local coordinate system of the first spatial direction sensor 102 in the coordinate system of the system 100 by determining the spatial directions of at least three targets 105 associated with the first spatial direction sensor with respect to the second spatial direction sensor 103. Hence, the position and/or orientation of the probe 101, being in the field of view FOV1 of the first spatial direction sensor 102 only, is firstly determined in the local coordinate system of the first spatial direction sensor 102 and then transformed into the position and/or orientation in the coordinate system of the system 100.

[0060] The first and second spatial direction sensors 102, 103 are connected to the processing means 106. This connection may be a wired or wireless connection. Therefore, the processing means 106 as well as the first and second spatial direction sensors 102, 103 may be equipped with a transceiver. Also the probe 101 could have a connection to the processing means 106 in the same manner, e.g., to control the targets.

[0061] The spatial direction sensors 102, 103 are any type of electro-optical camera such as e.g. a CMOS camera or a CCD camera.

[0062] The probe 101 is any type of instrument suitable for measurement of the location of a geometrical entity such as a surface point or a hole—such as e.g. a touch probe with a stylus for point-by-point measurement or a stand-off scanning camera measuring multiple surface points in a so-called patch—and/or a physical characteristic associated with such an entity e.g. a combined sensor for measuring a physical characteristic combined with a location, such as a color temperature probe or ultrasonic thickness probe.

[0063] The targets 105 are any type of mark or target suitable for observation by the spatial direction sensors, such as light emitting diodes, fiducial marks, retro-reflective targets or shapes. The targets 105 may be active or passive. As active targets, LEDs (Light Emitting Diodes) are particularly well suited, but any light source with a well-defined spectro-emission may be employed. The passive targets may reflect light from an external active light source.

[0064] FIG. 3 represents an application of the invention according to the first embodiment in an automotive framing station as already illustrated in FIG. 1. In this industrial application, not all of the probes 101-1, 101-2, 101-3 are in the field of view FOV2 of the second spatial direction sensor 103. However, the first spatial direction sensor 102 is arranged in such a manner that it is always in the field of view FOV2 of the second spatial direction sensor 103. The first spatial direction sensor 102 then again detects the position and/or orientation of the probes 101-1, 101-2, 101-3. Therefore, the first spatial direction sensor 102 is mounted in such a way that it can rotate its field of view FOV1 in all directions. The acquired data is transmitted to the processing means 106 that in turn executes a computer program to perform the following steps.

[0065] In a first step, the spatial coordinates and/or orientation of the first probe 101-1 is determined in the local coordinate system of the first spatial direction sensor 102. Then, in a second step, the position and/or orientation of the first spatial direction sensor 102 is determined in the coordinate system of the system 100 by the second spatial direction sensor 103. This is done by measuring the spatial directions of at least three targets 105 associated to the first spatial direction sensor 102 arranged in a pattern 104. The local coordinate system of the first spatial direction sensor 102 is then transferred into the spatial coordinate system of the system 100. Hence, also the spatial coordinates and/or orientation of the probe 101-1 are transformed in the coordinate system of the system 100.

[0066] Due to the fact that the first direction sensor 102 is able to rotate or even to move, it is able to image the whole space surrounding it, rendering it possible to determine the position and/or orientation of the probes being located around it. Since the first spatial direction sensor 102 is situated in a location which is much closer to the probes 101-1, 101-2, 101-3 to be measured, it is less likely that a welding robot or an element prevents a spatial coordinates and/or orientation measurement. Also, due to the smaller measurement distance for the determination of the position and/or orientation of the probes 101-1, 101-2, 101-3, the measurements become more exact than if the second spatial direction sensor 103 measures the position and/or orientation of the probes 101-1, 101-2, 101-3 directly.

[0067] In FIG. 4, a second embodiment of the present invention is illustrated. Additionally to the first embodiment, the probe 101 also comprises targets 107. The spatial coordinates and/or orientation of the probe 101 are then determined based on a measurement of the spatial directions of these targets 107 relative to the first spatial direction sensor 102. In fact, the position of the targets 107 is known in a local coordinate system of the probe 101. Based on this information and the determined spatial directions, the system 400 is able to determine the position and/or orientation of the probe 101 in the local coordinate system of the first spatial direction sensor 102. This position and/or orientation can then be transformed in the position and/or orientation in the coordinate system of the system 400 as explained for the first embodiment.

[0068] In a third embodiment according to the present invention, illustrated in FIG. 5, the first spatial direction sensor 102 and the second spatial direction sensor 103 are mechanically connected. For example, the spatial direction sensors can be mounted in a frame structure 110. The first spatial direction sensor 102 is mounted in the frame structure 110 in such a way that it can rotate freely. Furthermore, the spatial direction sensors 102, 103 are arranged in such a manner that at least three targets 105 associated to the first spatial direction sensor 102 are always in the field of view FOV2 of the second spatial direction sensor 103, independent of the rotational orientation of the first spatial direction sensor 102.

[0069] The frame structure 110 may consist of carbon fiber reinforced polymer or a metal tubing or any other solid and rigid material. Preferably, the material has minimum temperature expansion properties. The entity of first and second spatial direction sensors 102, 103 and the frame structure 110 may form a self-contained, compact and stable unit, which is easy to transport and which resists high external stresses. The spatial direction sensors 102, 103 may also be protected by the frame structure 110. Alternatively, the frame structure 110 is partly covered or the frame structure may even be replaced by a closed construction and form a closed unit.

[0070] FIG. 6 illustrates the third embodiment of the present invention, having probes 101-1, 101-2, 101-3, 101-4, in an exemplary industrial setup in a moving production line. As can be seen, the entity with first and second spatial direction sensors 102, 103 and frame structure 110 can be placed on a tooling pallet or processed product as a self-contained system.

[0071] FIG. 7 illustrates a fourth embodiment of the present invention. In this embodiment, the system 700 comprises a second spatial direction sensor 103 having a field of view (FOV2) and a third spatial direction sensor 113 having a different field of view FOV3. These two fields of view FOV2 and FOV3, however, have common sectors in which reference points 111 are situated. By means of these reference points 111, which also may have targets, the local coordinate systems of each second spatial direction sensor 103 can be calibrated to the coordinate system of the system 700.

[0072] In this embodiment, the first spatial direction sensor 102 can move from a first position A within the field of view FOV2 of a second spatial direction sensor to a position B in the field of view FOV3 of the third spatial direction sensor 113. Due to the fact that the second spatial direction sensors 103 and the third spatial direction sensor 113 are calibrated, the local coordinate system of the first spatial direction sensor 102 can be calibrated to the coordinate system of the system 700 in the positions A and B.

[0073] The reference points 111 may also be replaced by a probe 101 which is static and which remains in the fields of view FOV2, FOV3 of both the second spatial direction sensor 103 and the third spatial direction sensor 113.

[0074] FIG. 8 refers to a fifth embodiment of the present invention. Again, the first spatial direction sensor 102 can be moved from a position A to a position B. Nevertheless, as in the fourth embodiment, the local coordinate system of the first direction sensor 102 stays calibrated with the coordinate system of the system 800. In contrast to the fourth embodiment, the system 800 is not calibrated by common reference points 111 or probes 101 in the fields of view FOV2 and FOV3 of the second spatial direction sensor 103 and the third spatial direction sensor 113. Instead, the first spatial direction sensor 102 determines the relative positions of the reference points 111 and/or a fixed probe 101 in its positions A and B. The reference points 111 and/or the fixed probes 101 are then in the fields of view of the first spatial direction sensor 102 before the movement from a first field of view of the first spatial direction sensor FOV1-1 to a second field of view of the first spatial direction sensor FOV1-2. From this measurement, the local coordinate systems of the second spatial direction sensor 103 and the third spatial direction sensor 113 are calibrated and the first spatial direction sensor 102 can perform measurements in the coordinate system of the system 800.

[0075]

System	100, 400, 700, 800
Probe	101, 101-1, 101-2,
	101-3 & 101-4
First spatial direction sensor	102
Second spatial direction sensor	103
Thrid spatial direction sensor	113
Pattern of targets	104
Target	105
Processing means	106
Target of the probe	107
Object	108
Touch point	109
Frame structure	110
Reference point	111
Roof tooling of framing station	201
Side tooling of framing station	202-1, 202-2
Floor tooling of framing station	203
Touch point	204
Touch probe	205
Sheet-metal locating device	206
Spatial direction sensor	207
Side panel	208-1, 209
Roof panel	210
Floor assembly	211
Field of view of a first spatial direction sensor	FOV1
Field of view of a second spatial direction sensor	FOV2
Position before movement	A
Position after movement	B