Panoramic aerial imaging device

[0001] This application claims benefit of U.S. Provisional Application No. 60/275,378, filed Mar. 13, 2001, which is incorporated by reference herein.

[0002] The present invention relates generally to a device to take a spherical or hemispherical panoramic image from a specific point in the air, and more particularly, to a simple device with no moving parts that rotates about two axes to scan the imaging device over the ground in two axes, such that the images can be put together to create a panoramic image.

[0003] A person on the ground, such as a soldier, may need to know what is in the immediately surrounding area. Reconnaissance of an area can often be efficiently done from the air. (For example, it has been reported that tanks operating in urban environments require helicopter support before turning a street corner, to detect anti-tank weapons). However, manned aircraft are expensive, not always readily available, may not have access to the area to be looked at and a direct link to the person at the site is not always available. Satellites are only available a few times each day at predictable times, the resolution of commercial imaging satellites is not very good, and again, the ability to get the information to the person at the site is limited. Unmanned aircraft flown by the person at the site are large, expensive or difficult to operate, and require an open area for launch and recovery. Recovery is necessary because of the expense involved. Larger unmanned aircraft flown from fixed bases have many of the same problems as manned aircraft. Balloons are time consuming to inflate and fly, and depend too much on calm wind conditions.

[0004] Aircraft or satellites over-flying an area typically take vertical photographs of the earth directly below the vehicle, or panoramic photographs that extend out to either side. A large amount of ground can then be covered by flying the vehicle in a forward direction and combining the photographs into a long strip.

[0005] These methods are expensive, and not in the direct control of the person requiring the photographs, so there is an inherent time delay in getting the images to him.

[0006] Systems that are portable, and can be carried into the field, require input from the user to aim the imaging sensor at targets of interest. This is also time consuming, and involves some level of skill and practice to be able to control the imaging device accurately.

[0007] A device that loiters above the operator may also give away his position if it is detected. The chances of detection go up with time.

[0008] A lightweight, man-portable device that automatically gives the person an immediate and detailed view of the entire surrounding area, centered on his position is not available.

[0009] An object of the invention is to provide the user immediate access to detailed aerial photographs of the entire surrounding area out to a certain radius.

[0010] Another object of the invention is to provide a device that gathers this information with an imaging device from a substantially stationary point in the air above the operator.

[0011] Another object of the invention is to provide a device that can generate its pictures with no control from the ground, or user input, once it has been put into the air. To achieve this objective, the vehicle system of the invention gathers a series of images by rotating the imager in a predictable pattern along two separate axes that covers a spherical area, or a portion of a sphere. Optionally, if the device loiters in the air, it may be controlled to further examine an object of interest discovered in the initial set of images.

[0012] In some aspects, the invention provides a device that is spun up to the scanning speed along the first axis using fins that impart a torque about the roll axis as it moves through the air. The imaging device may be fixed with respect to the body of the device, with the optical axis perpendicular to the roll axis. To prevent blurred images, the shutter speed, or data collection time of the imager, is short enough to be considered insignificant compared to the angular rotation rate. The second scanning axis is provided by the device pitching over at the peak of a steep ballistic trajectory.

[0013] Another object of the invention is to provide a sensor that can be used to measure the orientation of the imaging device to assist in orienting the information from the imager. Alternately the imaging device may gather this information without scanning, using a wide angle or fisheye lens.

[0014] Another object of the invention is to provide a system where the data from the imaging device is stored or transmitted down to the operator and the operator can map all of the imaging data onto a spherical surface using a computer. The computer may then display the data on a display, such as the screen of a laptop computer, and allow the user to pan, zoom or perform other operations using the data, including, but not limited to object recognition, motion detection, range finding and geo-location.

[0015] As used herein, an “image” or “imagery data” is a collection of sensor data where each datum is associated with a relative location (e.g., a rectangular array of pixels). An “imager” or an “imaging device” is any sensor that collects imagery data.

[0016] As used herein, a “gun” includes any member of the class of small arms and light weapons, as defined by the UN Panel of Experts on Small Arms and approved by the United Nations General Assembly in 1997.

[0017] As used herein, a “data transmitter” includes wireless, wired, fiber optic, and other equivalent systems for transferring data from one location to another, although wireless systems are preferred in most embodiments of the invention.

[0018] As used herein, a “portable” vehicle is one that can be transported on a vehicle such as a light truck or a tank. A “man-portable” vehicle is one that can be carried by one to four men, and preferably one that can be carried by a single man.

[0024] FIG. 1 shows a device 1 according to the invention launched in a ballistic trajectory, from a small, portable launch pad 6. The images from the device 1 are transmitted to the operator with a portable ground station 5 where they are received by the receiver 3 using the antenna 2, and transferred to the portable computer 4. The computer arranges the images in a complete panorama, covering the entire area as seen by the flight vehicle 1 at the top of its trajectory.

[0025] FIG. 2 shows a cut away detail of one flight vehicle according to the invention. It is launched into the air using a small solid fueled rocket motor 9. Those of ordinary skill in the art will appreciate that the ideal height of launch will depend on the area to be viewed and the quality of the imager, and may range from as low as ten feet to as high as 30,000 feet, or even up to 100,000 feet or more for some vehicles. The best propulsion device will depend upon the desired height, and possibly on other factors such as radar visibility and noise. A great many propulsion devices are possible, such as motors and engines, springs, explosives, electromagnetic launchers (e.g., rail guns), electrostatic launchers, compressed gas launchers, mechanical launchers (including handheld mechanical launchers such as slings), or pressurized fluids (such as a “water rocket”). If only a modest area is to be imaged, the device can simply be thrown in the air.

[0026] The size of the vehicle will depend on the sensors on board, the desired height of the flight path, the launch method and other considerations. It is preferred that the vehicle be man-portable, and preferably that it weigh under 5 pounds, if it is to be used by soldiers in the field. It is also preferred that the vehicle be small (e.g., 6-12 inches in length) for this type of application, but larger vehicles (e.g., five feet or more) are also within the scope of the invention.

[0027] As the vehicle shown in FIG. 1 ascends, the fins 7, which are canted at an angle, cause the airframe 8 to rotate about its roll axis. In the embodiment shown, the fins 7 are fixed, but deployable and/or adaptable fins (e.g., fins that change angle to maintain a specific roll rate) may also be used. The imager 10 has a lens with an optical axis perpendicular to the airframe's 8 roll axis, and looks out a port in the side of the airframe. An image is captured by the imager 10, and is transmitted to the ground station using transmitter 11. Both the imager and transmitter are powered by the power source 12, which could be a battery, capacitor or other power source. A directional light sensor 13 is tilted at an angle that is between the roll axis and perpendicular to said axis. The purpose of the light sensor 13 is to observe the horizon passing as the vehicle rolls. Since the light sensor has an angular component in the roll plane and the pitch plane, it can obtain data on the roll rate and the pitch rate at the top of the flight path. This is explained in more detail in FIG. 4.

[0028] In the embodiment shown in FIG. 2, the imager is a visible light device such as a linear array imager, a rectangular CCD, or the like. Other sensors, such as infrared sensors, chemical sensors, biological sensors, radar, sonar, or other sensors may be appropriate for some applications. Those of ordinary skill in the art will see how to select an appropriate sensor for a given embodiment. In addition, it may be desirable in many cases to include multiple sensors (e.g., a chemical or biological “sniffer” may be mounted on the vehicle in addition to a light camera), not all of which need generate locationally specified image data.

[0029] FIG. 3 shows the aerial vehicle 14 flying along the ballistic flight-path 15. As the airframe 8 spins along the roll axis 20, the imaging device's 10 field of view 16 is swept along the ground 17 in a plane 18 perpendicular to the aerial vehicle's flight path. As the aerial vehicle 14 arcs over at the peak of its ballistic trajectory 15, the rotation along the pitch axis 21 causes the imaging device to sweep out incrementally different paths along the ground with each sweep intercepting the horizon 19 at a point perpendicular to the aerial vehicle's flight path 15 at its apogee. In an aerodynamic vehicle such as the one shown in FIG. 3, the center of mass of the vehicle should be placed forward of the center of pressure, so that the vehicle will arc over the ground and sweep through a large enough pitch range. In some preferred embodiments, the flight path and roll rate are selected so that images can be captured in all directions over a very short flight distance at apogee, so that image data is collected from a substantially stationary point.

[0030] FIG. 4 shows how the light sensor 13 is used to determine the orientation of the vehicle when each image is taken by the imager 10. As the vehicle 30 rotates about its roll axis 25, the directional light sensor's field of view 23 traces out a circle 28. When the light sensor's field of view 23 crosses the horizon at 22, it detects a different light level. The frequency of change of this difference in light level can be used to determine the vehicle's roll rate, and by interpolating between sequential horizon crossings, the imager's roll position can be determined. As the vehicle 30 arcs over in the pitch direction 26, the circles traced out by the light sensor 27, 28 will intersect the horizon at different points. The fraction of the circle above or below the horizon can be used to determine the pitch angle, and interpolating between two different circles can be used to find the pitch rate and angle at any moment in time. The data from the light sensor is transmitted to the ground using the audio channel of the TV transmitter 11, and saved on the computer 4 on the ground to be used to reconstruct the flight path.

[0031] As each image or datum is transmitted from the imager 10 to the ground station, the computer 4 places the data in the appropriate place in a spherical data set by using the position information interpolated from the light sensor 13 data. If the light sensor is not present, the same flight information can be obtained from the images themselves. The roll rate is determined by the frequency of the horizon passing through the images, and the pitch is obtained from the angle of the horizon to that of the imager. (In appropriate situations, features other than the horizon may be used in the same way to determine roll and pitch). Orientation may also be measured in a variety of other ways, such as with an inertial device such as a gyroscope; using a wide-angle lens to create a lower resolution, wide-area view with which images may be correlated; with a magnetic sensor such as a compass; using an altimeter and/or a time-of-flight monitor to infer the ballistic trajectory; by measuring the polarization of ambient light to detect the position of the sun; or by using a light intensity sensor to detect the sun (or another distinctive light source such as the moon, the stars, or artificial lighting) directly.

[0032] After the imager 10 has been swept over all of the ground visible from the top of the flight path, the vehicle may be recovered by parachute, or allowed to crash to the ground (e.g., if it is disposable). In preferred embodiments of the invention, the images collected by the vehicle are transmitted by a data transmitter to a ground station. The ground station may be, for example, a laptop computer. If the vehicle is recoverable, it can be “plugged in” to the ground station to download data, or it may transmit data to the ground station during flight. Disposable vehicles will generally transmit data to the ground system during flight, for example by wireless transmission or via a tether wire.

[0033] The transmitted data set can then be displayed on a computer screen. In preferred embodiments, the operator can zoom and pan around the image, or perform other manipulations such as overlaying the images on a map. Since a typical data set covers all or a significant portion of a sphere, the image can even be viewed in a three-dimensional virtual reality system.

[0034] It will often be desired to determine the absolute location, as well as the orientation, of images captured by the system. This may be done, for example, by placing a Global Positioning System (GPS) receiver on the vehicle. The GPS receiver may also be used to orient the images, since the position of the vehicle can be determined with high accuracy. The GPS data is preferably used to determine the exact trajectory of the vehicle, which is combined with roll rate data measured by one of the other means described above to place images. The trajectory can also be measured by knowing the initial launch point and obtaining directional information from the vehicle or launcher, for example by using a magnetic compass. Location may also be determined by manual or automatic matching of the image with maps, survey photos, or other recorded information about the area. If the location of the launch point and the orientation of launch are known, these data can be used to project the trajectory of the vehicle to determine the location of the image.

[0035] One advantage of this embodiment of the invention is that the vehicle should be very difficult to detect. It is small, quiet, and is only in the air for a matter of seconds. This invention is low cost, can be disposable, and obtains a single set of images per flight. It is small enough that the operator could carry several expendable flight vehicles for use as needs dictate.

[0036] Other solutions to obtaining biaxial scans of the imaging device are also contemplated within the scope of the invention. Imagers may be gimbaled in one or two axes on the launch vehicle, instead of using roll and pitch to scan the imager. Gimbaled systems may deploy parachutes, collecting data while the vehicle descends, for example. “Loitering” systems may also be used with gimbaled cameras. In such systems, the operator may be able to override the scanning function of the imager in order to focus on particular areas of interest.

[0037] In yet another useful embodiment of the invention, a ballistic launch vehicle can be launched over a feature of interest (such as a building), and images collected from the full arc of the ballistic trajectory can be used to create a three-dimensional model of the overflight area.

[0038] Other embodiments of the invention will be apparent to those skilled in the art from a consideration of the specification or practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.