Camera guidance system

[0002] This is a continuation-in-part of application Ser. No. 08/666,299, filed Jun. 24, 1996, now abandoned, which is a 371 of PCT/IB94/00431 filed Dec. 16, 1994, and the entire disclosure of this prior application is considered to be part of the disclosure of the accompanying application and is hereby incorporated by reference therein.

[0003] The present invention concerns a camera guidance system allowing to acquire video signals of at least one target, by means of at least one camera, and to transmit such video signals to a receiver station. In particular, it concerns a help to the acquisition, by a camera, of close-up visual information of a target, said target being a competitor in a race, or a participant in a show.

[0004] One purpose of sports show business is to give the viewer the impression that he is in the middle of the action. For this purpose, multiple cameras are preferably acquiring close-up, breath-taking images of the leading competitors of the race, or of the star participants of the show. If such leading competitor or star participant is typically too far from the cameras, it is known to place such cameras on the competitor or participant himself, showing him at least partially. The imagery is constantly selected and re-evaluated, taking into account the most interesting views.

[0005] Shows and racing competitions have implied in a known manner the presence of cameras on or close to the participants or competitors. For example, in bicycle races, cameras are customarily mounted on a neighboring motorcycle. In sailing competition, cameras can be mounted on the top of the masts of competitors. The most difficult case for imaging is an air race or show where the competitors can very quickly move around in three dimensions, and are easily lost by the camera attendant.

[0006] The technical problem to be solved in such an environment is to allow for capture of the best imagery with automatically guided cameras, without having to carry a camera attendant which in many sports cannot be carried in the competitor's immediate vicinity. Various systems exist to monitor the position of moving bodies. Such systems are described e.g. in European Patent Applications Nos. 92303372.4 and 92303373.2. However, such systems are limited to the monitoring of the position of the bodies, and not of their orientation or attitude.

[0007] The system according to the invention allows to replace the camera attendant by a highly efficient tracking system; in particular, it allows to automatically keep a selected target in the viewing frame of a camera. For this purpose, the targets continuously provide information on their position and attitude coordinates to the camera platform with the communication means. The information on the position and attitude of the targets are processed by the information processing means of the platform, to be converted into platform coordinates and fed to the pointing systems of the cameras. Each target can be followed by a plurality of cameras on a platform, each camera having its own pointing system. All camera pointing systems use the information on the position of the particular target that has been allocated to them. This allows all candidates for the show to be constantly in view, ready for selection.

[0008] The present invention accomplishes this by a platform supporting the camera. The platform is movable into a plurality of positions for pointing the camera at the target. Target position means is mounted on the target for determining a position of the target and transmitting the position of the target to a processing means. The processing means receives the position of the target from the target position means, and determines a relative target position of the target position means with respect to the platform. The processing means then moves the platform to point the camera at the target.

[0009] In a first embodiment of the invention, the receiver station is on the platform and stores the video signals acquired by the cameras, either definitively, with e.g. classic 16 mm film technology, or erasable, with e.g. video tape technology.

[0010] In a second embodiment of the invention, the receiver station is not located on the platform and its attendants can instantly modify the allocation of the cameras to the target. This allows for optimization of the allocation of the targets to the cameras.

[0011] In a particular embodiment of the invention, each target can also serve as a platform, allowing the receiver station to select the most appropriate platform, and cameras on said competitors for the show. In this embodiment, information on position and attitude of all the targets and platform are delivered by said targets and platforms to the information processing systems of the other targets and platforms via the communication means. The information on attitude of the target is not necessarily useful for the guidance of the allocated cameras, but it becomes important if the target is later selected as a platform, as well as for extrapolation purposes if the efficiency of the communication means is temporarily reduced.

[0012] In a particular embodiment of the invention, if said receiver station, generally a ground station, is remote from the race, the means for transmitting the video signals can include one or more relay stations; said relay stations are equipped with antennas to convey the video signals from the platform's cameras. It should be noted that in the latter case, the relay stations also advantageously provide their coordinates to the information processing means, so that the antennas can be efficiently oriented towards each other to ensure optimal transmission quality.

[0013] In a particular embodiment of the invention, the position and attitude data are acquired by global navigation satellite systems, such as systems-known under the names of NAVSTAR/GPS and GLONASS, or more generally GNSS (Global Navigation Satellite System). Such systems are described e.g. in the European Application No. 92310832.8, or in UK Patent Application No. 8827257. U.S. Pat. No. 5,467,271 teaches how to generate agricultural maps with the help of GPS. U.S. Pat. No. 5,583,513 to Cohen describes a system for attitude determinations. U.S. Pat. No. 5,130,794 discloses a panoramic display system. U.S. Pat. No. 5,166,789 discloses an earth surface surveying system using infrared camera.

[0014] U.S. Pat. No. 5,396,431 discloses a navigation system with position measuring device and aerial photographic storage capability.

[0015] In a particular embodiment of the invention, the position of the target in platform coordinates is obtained by subtracting the platform's global position vector from the target's global position vector and projecting the resulting vector onto the platform coordinate system. Such data can be obtained in particular with the help of a differential global navigation satellite system, or D-GNSS, which will provide attitude data by triangulation.

[0016] An example of such a D-GNSS system is described in UK Patent Application No. 9004433.0.

[0017] In a particular embodiment of the invention, the pointing systems of the cameras allow them to point to any direction.

[0018] In a simplified environment where every object has a fixed assignment, and e.g. one platform can film one target, it is generally not necessary to acquire attitude data of the target, so that a GNSS receiver on the target will suffice for determination of the target's position; for the platform and any moving relay stations, on the contrary, attitude is important for the pointing of cameras and antennas. Thus the platform and each relay station will need GNSS receivers for position and at least two additional GNSS receivers for attitude.

[0019] U.S. Pat. No. 5,296,861 to Knight teaches determination of attitude data of objects.

[0020] In a particular embodiment of the invention, the global positioning system of a target is supplemented by at least an Inertial Measurement Unit (IMU) or more generally by an inertial navigation system to which such an IMU belongs. An IMU is typically constituted by three accelerometers coupled with gyroscopes. The accelerometers measures linear acceleration, and the gyroscopes measures angular speeds.

[0021] In a preferred embodiment, an IMU is used for motion measurements for each axis Ox, Oy, Oz of the coordinate system of the target. A platform or a relay station can also be equipped with an IMU. An IMU is useful if, for instance, communication between the satellites and the global positioning systems of the object is made less efficient or interrupted by interposition of other objects. A typical cycle duration, for a global positioning system as practically used in the invention, being of one second, an interruption of transmission between the GNSS satellites and a GNSS receiver can last several seconds; with the input of an IMU as described above, the information processing means of the object can extrapolate from the last available position and attitude data of the object before the interruption took place, until a final correction is made when communication with the satellites is re-established.

[0022] In a particular embodiment of the invention, where space or weight constraints may exist on the targets, on the platforms or on the relay stations, there is a need to reduce the number of GNSS receivers. A target or platform or relay station can then be equipped by only one GNSS receiver and one IMU to obtain both position and attitude, on the condition that the target or platform or relay station is moving along a true three-dimensional trajectory with a speed that is superior to a minimal value. This value depends on the characteristics of the IMU, which functions best when the speed is high and the trajectory curved. However, the speed needs not to be determined in advance; neither does the path that will be followed by the target or platform or relay station. The only constraint is that the speed is high enough for the IMU to function adequately. Such a system entails using the data from the same GNSS receiver obtained at known time intervals to determine the position of the target or platform or relay station, and using the IMU to precisely measure the relative evolution of the position of the target or platform or relay station between two position determinations by the GNSS receiver. Such measurement allows the calculation of the orientation of the mobile coordinate system of the target, platform or relay station in relation to a coordinate system that is earth fixed. This orientation can be expressed by the Euler angles of the mobile coordinate system in relation to the earth-fixed coordinate system.

[0023] It is known that IMU data suffer from errors called drift errors. These drift errors accumulate with the elapsed measurement time, and are detrimental to the accurateness of the calculations. In a particular embodiment of the invention, the length of the time intervals, i.e. the duration between two GNSS measurements, can be adapted to adequately correct said drifts errors before they grow to an unacceptable value. The drift errors of the IMU used dictate the required performance of the GNSS receiver: the higher the drift the shorter the interval must be between data received from the GNSS receiver and vice versa.

[0024] In a particular embodiment of the invention, the trajectories of the various targets, platforms and relay stations stay confined within known boundaries. This is the case, for instance, in automotive competition. Motor races are run on a circuit where the tracks are limited, in a more or less flexible manner, by barriers, obstacles, sand pits or other track boundaries. In such a case, the circuit topography is known, i.e.: variations in altitude, presence of bridges and hills, and particularly tunnels. Tunnels are particularly important because they preclude adequate function of the GNSS part of the system. The track boundaries and its three-dimensional topography can in such cases advantageously be pre-recorded in the information processing means. The position and attitude of a target, platform, or relay station can then be determined by correlating of the three-dimensional accelerations and rotations as detected by the IMU with a computer-based map. Stored track boundaries and surface topology allow in this case to compensate the drift errors of the IMU; for this method to work the vehicle must, however, move quite fast and get close to the track boundaries from time to time.

[0025] More precisely, when the target or platform or relay station trespasses or touches a track boundary, the resulting motion generated by the curbs or change in surface roughness are detected by the IMU and interpreted by the information processing means.

[0026] This allows to suppress in some cases the GNSS part of the equipment, or to make the position determination process more robust through redundancy.

[0027] In particular, it is known that in the case of a motor racing circuit, the performance of GNSS is lower not only in tunnel, but also when certain parts of the circuit are prone to reflections of satellite of signals, leading to so-called “multi-path” phenomenons. This is the case for high buildings, bridges and fences. In these parts, the data measured by the IMU are essential to the functioning of the camera guidance system.

[0028] In a particular embodiment of the invention, also applying to the previous case where the trajectories are within known boundaries, the initialization of the system is facilitated by providing the information processing means with additional, complementary data. These data consist, for example, in signals that are sent by a photo-electrical cell that is placed in a given place in the race circuit, and detects the passing by of a target, platform or relay station. This embodiment provides redundant data that can be used to enhance corrections of drift errors and to complete refresh operations, as previously described, which are performed when track boundaries are touched.

[0029] In a particular embodiment of the invention, specially applying to systems where the IMUs used have a tendency to under-perform due to low speeds that are detrimental to their efficiency, wheel rotations sensors or steering equipment sensors are installed on the target, platform or relay station. These sensors provide other kinds of supplemental data that help the information processing means to verify and correct the data otherwise provided. It is clear that in the case of high speeds on the race track, these supplemental data must be analyzed or filtered because of skidding that affect the wheels; such skidding introduces an erratic bias in the data provided by these sensors.

[0030] In a particular embodiment of the invention, GNSS satellites are replaced by pseudo-satellites generally referred to as pseudolites. These pseudolites are disposed at fixed locations in close vicinity of an elevated support or a ceiling, and transmit signals that are similar to GNSS satellites; through the information processing means they can efficiently provide platforms and relay stations with data that are necessary to film the targets, as will now be explained.

[0031] Sources of electromagnetic signals are disposed on the pseudolites. Receivers of these signals, which may indeed be normal GNSS receivers, are disposed on the targets, platforms or relay stations; these signals, once received, allow information processing means to:

[0032] calculate the distances between the receivers and each of the sources;

[0033] calculate the position and attitude data of the target, platform or relay station carrying the receivers.

[0034] This allows in turn the cameras to point exactly to the targets, and the antennas conveying the video information to point to the relay stations. The speed of the signals is close to the speed of light, and the distance between a source and a receiver is calculated from the time taken by a signal to cover such distance.

[0035] Practically, experience has shown that some redundancy in measurements and the subsequent calculations is useful. It allows to perform averaging calculations and to eliminate erroneous measurements and results by plausibility tests that are known in the art.

[0036] The invention will now be illustrated in more detail by reference to the accompanying drawing, in which:

[0037] FIG. 1 shows schematically two aircraft, a relay aircraft, three global positioning satellites, a video link ground station, and a communications link ground station;

[0038] FIG. 2 shows schematically how the global navigating satellite system (abbreviated in GNSS) functions with the help of three satellites, and illustrates how this system can be used to perform attitude measurements;

[0039] FIG. 3 shows the path of the video signals, illustrating how the camera platform aircraft films the target aircraft and the operation of the means for transmitting the video signals;

[0040] FIG. 4 shows schematically the operation of the communication means, particularly how the three aircraft and the ground station communicate;

[0041] FIG. 5 shows a target being filmed with the help of an IMU;

[0042] FIG. 6 shows an embodiment with two cameras on a motor race circuit;

[0043] FIG. 7 shows a similar embodiment with one camera and three pseudolites.

[0044] FIG. 1 shows two aircraft 4, 5 as viewed by a hypothetical viewer, supposedly aboard an auxiliary observing plane. Three global navigation stations, such satellites 1, 2, 3 are represented, although such systems generally comprise many more satellites. A relay aircraft 6 permits the aircraft 4 and 5 to transmit video signals to a ground based receiver 8. The relay aircraft 6 also relays communications between the aircraft 4, 5 and a ground based communication station 7. It is to be noted that any of the two aircraft 4, 5 can a priori be the target or the platform; the information processing means (not shown) are distributed between the aircraft 4, 5 , 6, the communication station 7 and the receiver 8.

[0045] FIG. 2 shows how any one of the aircraft represented in FIG. 1, namely the two aircraft 4, 5 and the relay aircraft 6 have (at least) three position determining means, such GNSS receivers 9, 10, 11 enabling the information processing means (not shown) to calculate the position of the aircraft and its attitude. It can be noted that a typical GNSS provides a precision of the range of 30 to 100 meters; however, measurement of the variations in distances are much more accurate, in the range of the millimeter, because the error in the distance information provided by a given satellite is constant, and introduces a highly accomodable bias. The multiplicity of available satellites introduce additional measurement safety by way of redundancy. The information processing means, known in themselves e.g. from the previously cited patent document, allow in particular to calculate the global position vector of all three receivers 9, 10, 11, and consequently the orientation of an aircraft-bound coordinate system, its Euler angles, and the variation of these data at every calculation cycle.

[0046] FIG. 3 shows how the video signals acquired by the camera 16 from the target 4 are first transmitted (in dotted lines) via the antennas 17, 18 from platform 5 to relay station 6, then via antenna 19, to ground station 8 with a television monitor 20. The operation of such means for transmitting video signals includes precise pointing, not only of camera 16 to target 4 thanks to GNSS 12 and 13, but also of antennas 17 and 18 to each other thanks to GNSS 13 and 14, and in turn of antenna 19 and ground station 8 to each other, thanks to GNSS 19 and 15. Such precise pointing is allowed by the information processing means (not shown), fed by GNSS data, that can convert any global position vector from earth coordinates to relative coordinates. For example, the global position vector of target 4 is a vector having its origin at the center of earth and its extremity at GNSS 12; the global position vector of the platform 5 is a vector having its origin in the center of earth and its extremity at GNSS 13. The information processing means allow to take into account the attitude of platform 5, as determined by the three GNSS 9, 10, 11 (FIG. 2) in order to correctly point camera 16 to target 4, according to a vector having its origin at GNSS 13 and its extremity at GNSS 12.

[0047] FIG. 4 shows schematically how the three aircraft 4, 5, 6 and the ground station 7 may communicate using radio equipment with omni-directional antennas 21, 22, 23. Such antennas do not need to be pointed to one another to communicate, unlike antennas of the means for transmitting video signals. This has been shown schematically by doubles arrows on FIG. 4. All the above entities of the exemplified system may use the same frequency for broadcasting their global position vectors, the current satellite usage, camera control information, and other relevant data. To permit the use of one single frequency a wireless adaptation of the Carrier Sense Multiple Access with Collision Detection communications standard may be used (ANSI/IEEE Std 802.3-1985 and ISI/DIS 8802/3). Also shown is a ground control unit 24, where camera selection, target selection, and zoom control takes place. The information processing means of the ground control unit operate with the information processing means of the aircraft to coordinate activities and to distribute information processing tasks.

[0048] FIG. 5 shows a target 28, equipped with a GNSS receptor and filmed by a camera 16 on a platform 5. Platform 5 is moving on a trajectory T. It is equipped by a GNSS receiver 9 and an Inertial Measurement Unit 25.

[0049] Three positions of the platform 5 on the trajectory T have been represented at T₃, T₄and T₅. These positions are measured by reference to a fixed, earth-bound system of coordinates T₂.

[0050] Simultaneously, IMU 25 detects the accelerations (biased by the earth's gravitation) affecting platform 5 with respect to an inertial system of coordinates T₆.

[0051] Information processing means use these positions measurements to calculate the precise position and attitude of the platform 5, i.e. its Euler angles with reference to T₂. The position of target 28 is determined by its own GNSS receiver.

[0052] The information processing means then calculate the pointing angles of the camera 16 and its zoom data, in order to constantly keep target 28 in the frame of camera 16, with the correct focus.

[0053] FIG. 6 represents a motor race circuit 27 filmed by at least two cameras. One of the cameras is on-board a plane 4 flying above the circuit 27. Information processing means include image analysis capabilities; these process the images from the on-board camera and provide an estimate of the position of a target 28 which is a competitor in a race; they provide to the pointing mechanism of a second camera 16 the necessary information to effectively point this second camera 16 to target 28, thus providing commercially usable images of the competitor in the race. The images shot by second camera 16 can further be analyzed by image analysis capabilities, thus helping more cameras along the track of circuit 27 to capture close-up images of target 28 as its passes by them.

[0054] FIG. 7 shows a motor race circuit. On high poles surrounding the circuit (or underneath the roof of an indoor arena) are mounted pseudolites 29, 20, 31.

[0055] The target 28 is equipped with a receiver that receives electromagnetic signals from pseudolites 29, 30, 31; information processing means calculate, from said signals, the distances between pseudolites 29, 30, 31 and target 28. These distances allow to locate precisely target 28 on the track of circuit 32 and to effectively point camera 16 on target 28 with the right focus and zoom data. This permits acquisition of top quality imagery of any competitor in the race, including clear exposure of sponsor advertising carried by the competitors.