System and method for standoff detection of human carried explosives

[0001] This application is a National Stage entry of International Application No. PCT/US2005/036593, filed Oct. 11, 2005, the entire specification claims and drawings of which are incorporated herewith by reference.

[0002] The present invention relates to devices, systems and methods for remotely detecting concealed weapons, and particularly to systems and methods for remotely detecting human carried explosives.

[0003] The U.S. military and homeland security officials are keenly interested in new technologies that remotely detect Human Carried Explosives (HCE). The archetype threat is the suicide bomber. For example, on Mar. 2, 2004 three human suicide bombers detonated in a crowd in Baghdad, killing fifty-eight people and wounding more than two hundred. Other countries also have an interest in detecting terrorists carrying explosives on their person. Since September 2000, suicide bombers have killed over three hundred seventy-five Israeli civilians in over ninety separate attacks.

[0004] U.S. military commanders believe similar attacks will be carried out against military targets, and have placed a high priority on protecting forces from suicide bomber threats at home and abroad. Civilian law enforcement and corrections authorities are also highly interested in concealed weapons detection technologies. For example, the National Institute of Justice (NIJ) has funded the development of hidden weapon detection technology that works reasonably well when the subject is within fifteen meters of the detector. However, this range is well within the lethal zone of an HCE device, making such technologies ineffective for deployment for military force protection.

[0005] The ability to detect HCE devices concealed under clothing at distances up to one hundred meters would be extremely beneficial to military commanders concerned with protecting forces from human suicide bombers. By design, these weapons are impossible to detect visibly.

[0006] Many new technologies, including non-lethal suppression devices, are being developed to stop threats as they enter the mitigation zone. In addition, a great deal of effort is underway to improve existing technologies capable of identifying concealed threats at close-in ranges up to fifteen meters. These include infrared, ultrasound, X-ray, magnetometers, and imaging radars.

[0007] However, infrared sensing is impractical due to very small temperature contrasts between a hidden weapon and outer layers of clothing. Ultrasound is inexpensive and readily available; however, it is less effective at penetrating heavy clothing than radar. X-ray and magnetic portal detectors have proven much more effective at detecting both metallic and non-metallic concealed weapons, but portal detection technologies are inherently limited by the inability to operate such devices at a distance.

[0008] Currently, no technologies exist to reliably detect HCE devices at standoff (safe evacuation) ranges. According to Explosive Ordinance Disposal guidelines, the safe evacuation distance is proportional to the cube root of the device weight multiplied by a destruction factor that varies with munitions type. A typical suicide bomber payload consisting of thirty pounds of explosive surrounded by fragmentary shrapnel yields a safe evacuation distance of approximately one hundred meters. Similarly, a Claymore mine with a shaped charge has a safe evacuation distance of approximately one hundred meters, whereas a car bomb typically has a safe evacuation distance of four hundred fifty-seven meters.

[0009] Although radar easily reaches potential threats at safe evacuation distances (standoff ranges), devices lacking metal fragmentation are transparent to conventional radar solutions. Moreover, conventional radar solutions that rely on radar cross section (RCS) measurements alone cannot reliably discriminate between humans, humans with body clutter (e.g., cell phone, belt buckle, etc.), and humans carrying fragmentation and explosives.

[0010] In order to separate a potential detection from random materials in the background environment of a potential threat and to localize a detection to a specific individual, a sensor is required having a small enough field of view so that it has enough resolution on a single individual being interrogated as a potential threat that a positive detection can be associated with the individual separate from his background and other individuals around him. Solving this problem requires a sensor with a relatively narrow field of view (FOV) of approximately a one-half to one degree visual arc at 100 meters if the sensor's FOV is going to correspond to one individual person. Such a narrow FOV requires some precise method for pointing the sensor at potential threats.

[0011] To examine a useful field of regard (FOR), for example, 60° of visual arc, the sensor must be moved sequentially to every potential individual who might be carrying an explosive device. Returns from vehicles, buildings, signs, and other objects in the background must be ignored. Many proposed detection systems postulate a skilled human operator to point the device at individual potential threats to solve this problem. However, suicide bombers have been observed to have been driven near to their target by accomplices and then to have exited the vehicle and to have run to their target and detonate. Consequently, the time to detect and interdict a suicide bomber may be only tens of seconds, and a practical system will have a method to continuously scan and examine all potential threats in a field of regard. Examining and identifying individual people at distances greater than 25 meters and rapidly pointing a narrow FOV detection sensor at each in turn covering a wide FOR every few seconds is beyond the capability of human operators in situations with more than a few potential threats.

[0012] In addition to requiring the human operator to point the sensor precisely at numerous potential threats in rapid succession, previously proposed systems also require that a skilled human operator simultaneously examine the output of the sensor to make a subjective judgment as to the likelihood that a given person is a threat. It is unlikely that any proposed system will be effective if more than a few people could be in the field of regard at any one time. In order to meet these demanding timelines of precise pointing and analysis of the sensor's output, a system that provides a method to cue a narrow field of view sensor in a rapid, precise, and automated fashion and that has a method for automatically accessing the likelihood of a threat based on processing of the narrow FOV sensors signals is required.

[0013] Bomb detection technologies generally fall into two categories: direct methods designed to detect high explosives based on chemical properties, and indirect methods that look for anomalous signatures associated with bomb devices. None of these techniques are capable of detecting HCE threats at standoff ranges.

[0014] Direct methods for bomb detection exploit the fact that high explosives contain large amounts of nitrogen and oxygen in their molecular composition. Both bulk and trace detection methods use sampling methods designed to measure the chemical properties of the material, and both require that the material be placed in close proximity to the sensor. The main application of direct explosive detection methods is in portal systems.

[0015] Bulk detection methods measure the interaction between the sample under investigation and a penetrating radiation wave, such as ionizing radiation or electromagnetic radiation. Bulk detection methods can be used on sealed packages, including concealed threats. However, methods that use ionizing radiation may not be suitable for use in screening of human subjects.

[0016] Trace detection methods are based on direct interaction with the material and require that some amount of the explosive be present on the outside of the package in the form of particle residue or vapors. Both canines and gas chromatography methods ingest vapors and hence need to be close enough to the suspect article to acquire a sample of the vapor. Particle-based detection methods, such as laser ionizing mass spectroscopy, use optical methods for interrogating samples, but detector noise limitations require that the sample be proximal to the laser.

[0017] Indirect detection methods are designed to detect anomalies in human signatures that are consistent with concealed bomb devices. As with direct detection methods, the current standard practice is limited to short-range or portal based systems. Metal detectors are used to detect the presence of metal fragmentation and are of questionable use against devices that do not employ metallic fragmentation. Thermal and passive millimeter wave (MMW) systems exploit the fact that human skin reflectivity differs from that of metal or explosive materials. Both thermal and MMW radiation pass through clothing. Passive imaging systems collect an image of devices concealed under clothing in outdoor settings using the illumination of the cold sky. Imaging devices require trained operators to point the device and interpret the images, resulting in an increase in labor and privacy concerns, and in longer times to check each potential threat.

[0018] Various devices have been developed applying the above principles. U.S. Pat. No. 4,975,968, issued to T. Uki in December 1990, discloses a dielectrometry monitoring method and apparatus for three-dimensional profiling and colorable imaging of the material contents of articles carried on a conveyor belt. U.S. Pat. Nos. 5,073,782 and 5,227,800, issued to Huguenin et al. in December 1991 and July 1993, respectively, disclose a contraband detection system suitable for detecting concealed non-metallic contraband, such as ceramic or plastic weapons or illegal drugs using quasi-coherent millimeter wave radiation. The Huguenin system is an active imaging system designed for detecting contraband on a conveyor belt with a constant background at close range. Polarized returns are used by an operator to adjust contrast with the background, and a skilled operator is required to interpret the images as the conveyor scrolls objects past the sensor.

[0019] U.S. Pat. No. 5,177,445, issued to T. Cross in January 1993, discloses a method by which particular non-metallic materials are detected by recognizing the way in which their electrical properties vary with the frequency of an applied alternating electric field. U.S. Pat. No. 5,592,170, issued to Price et al. in January 1997, discloses a frequency-agile, narrow-instantaneous bandwidth radar system which detects objects and discriminates between different types of objects from a safe stand-off distance. The field of regard, however, is fixed and is dependent upon manual sweeping of an area under surveillance by a skilled human operator. A skilled operator would have difficulty in accurately pointing and tracking a person with a 1° radar beam at ranges greater than short standoff distances. In the Price system, the discrimination of threats is also performed manually by a skilled human operator.

[0020] Unlike the previous devices that detect concealed weapons, U.S. Pat. No. 5,829,437, issued to J. Bridges in November 1998, discloses a system and method by which cancers in heterogeneous tissue is located and detected by using the backscatter signal returns from microwave radiation. Unlike the previously disclosed devices, the '437 device is not a stand-off device and requires contact with the patient.

[0021] U.S. Pat. Nos. 6,243,036 and 6,342,696, issued to G. Chadwick in June 2001 and January 2003, respectively, disclose methods and apparatuses for detecting objects by comparing the differences in amplitudes of polarized radiation reflecting off a target illuminated with low-power polarized radiation with a predetermined value representative of an expected difference if the object were not present. The Chadwick patents are specifically illustrated by examples devoted to the detection of handguns, generally from laboratory studies, and do not address the problem of focusing the radar over a wide field at ranges up to one hundred meters or greater, and do not address the problem of detecting and identifying human subjects at such ranges, nor the problem of detecting nonmetallic objects by radar.

[0022] U.S. Pat. No. 6,359,582, issued to MacAleese et al. in March 2002, discloses a weapons detector utilizing a handheld radar system and signal processor to detect the presence of a plurality of self-resonant frequencies in the backscattered signals of a target between 4-15 yards. Although suitable for guns and similar devices, HCE devices must be detected at a much greater stand-off distance.

[0023] The aforementioned devices have a fixed field of regard and require an operator to direct the field of regard of the sensor upon the subject matter. U.S. Pat. No. 6,507,366, issued to H. Lee in January 2003, on the other hand, is a device that automatically tracks a moving object using a camera having a zoom lens, an auto focus lens and a charge-coupled device for converting an image into electrical signals. The disclosure, however, is silent on how the device manipulates multiple moving targets.

[0024] Articles entitled “Radar-Based Intruder Detection for a Robotic Security System”, Cory et al., SPIE Proc. 3525:Mobile Robots XIII and Intelligent Transportation Systems, Boston, Mass., 1-5 Nov. 1998, pp. 62-72, “Mobile Robots for Outdoor Security Applications”, Pastore et al., American Nuclear Society 8^th International Topical Meeting on Robotics and Remote Systems (ANS'99), Pittsburgh, Pa., 25-29 Apr. 1999, and “Robotic Security Systems”, Everett, H. R., IEEE Instrumentation and Measurement Magazine, December 2003, pp. 30-34 describe a robotic system for the detection of intruders in storage yards, arsenals, and the like. The system includes an infrared/vision based system (FLIR) and a millimeter wave radar at 77 GHz slaved to the vision system on a two-axis pan and tilt mechanism. The vision system is stepped across a field of interest, and when motion is detected, a target track is established. The vision system is used to define geometric shape and angular location, and Doppler radar pulses provide range and speed of movement, which are fused to establish a radar cross section and to confirm target and range. The system has been reported successful in detecting a human at ranges of 100 m-300 m. A scanning radar is added to the system for 360° detection of other potential targets while the system is tracking an initial target.

[0025] None of the above inventions and patents, taken either singly or in combination, is seen to describe the instant invention as claimed. Thus, the system and method for standoff detection of human carried explosives of the present invention solving the aforementioned problems is desired.

[0026] The current invention comprises a system for standoff detection of human carried explosives (HCE) within an area under surveillance. The system comprises an active radar system with a radar transmitter, a radar receiver, at least one radar antenna, and a field of regard. The detection system also comprises a radar tracking control system in physical cooperation with the active radar system. The field of regard of the active radar system is targeted within the area under surveillance. The area of surveillance is greater than the radar field of regard. The detection system further comprises a computer and a user interface with a video display terminal and a data input means. The computer is in electrical communication with the radar system, the radar tracking control system, and the user interface. The computer has a central processing unit, a memory, and computer readable program code stored in the memory. The code includes a radar tracking instruction means for targeting the radar system, and an HCE detection instruction means for detection of a human carried explosive device.

[0027] The current invention also includes a method for the standoff detection of HCEs within an area of observation using a plurality of HCE detection systems. Each HCE system has a radar system, a radar field of regard, an imaging system, and a computer. The method includes communicatively linking together the plurality of HCE detection systems and overlapping the radar field of regard of the plurality of the HCE detection systems. A plurality of simultaneous radar signatures of the radar field of regard from different look angles is then created and a determination is made regarding the threat potential of a target based upon the combined data of the plurality of radar signatures.

[0028] The current invention further includes a method for detecting explosive devices. The method includes illuminating a candidate threat with polarized radiation. A first reflected radiation from the candidate threat is then collected. The first reflected radiation has a polarization identical to that of the illuminating radiation. A second reflected radiation is then collected from the candidate threat. The second reflected radiation has a polarization orthogonal to that of the illuminating radiation. The change in the polarization state induced by the candidate threat is then detected.

[0029] The current invention additionally comprises a system for standoff detection of HCEs within an area of surveillance. The system comprises a detecting means for repetitively scanning the area of surveillance and for detecting and tracking a human target within the area of surveillance. The system also comprises a radar transmitter for transmitting a narrow beam of polarized electromagnetic radiation and a focusing means for automatically focusing and triggering the beam of polarized radiation emitted by the transmitter onto at least a portion of the human target detected by the detecting means. A radar receiver receives a portion of the beam that is transmitted by the radar transmitter and reflected from the human target and polarized orthogonal to the transmitted beam. The system further includes a data storage device with target reference data stored thereon. The target reference data includes radar signatures corresponding to humans with and without explosives carried on their person. The system includes a computing means for continuously comparing a reflected beam received by the radar receiver to the reference data to determine when the human target is carrying explosives and for continuously assessing the target detection and tracking data to determine the threat level. An alarm means is also included for alerting an operator of the system when the threat level determined by the computing means exceeds a threshold.

[0030] The current invention further comprises a method for the standoff detection of HCEs within an area of observation using a plurality of HCE detection systems. Each HCE system has a radar system, a radar field of regard, an imaging system, and a computer. The method includes communicatively linking together the plurality of HCE detection systems and coordinating the radar field of regard of the plurality of HCE detection systems to encompass a range of interest. A plurality of simultaneous radar signatures of the radar field of regard is created from different look angles and a determination is made regarding the threat potential of a target based upon the combined data of the plurality of radar signatures.

[0031] These and other objects of the present invention will become readily apparent upon further review of the following specification and drawings.

[0042] Similar reference characters denote corresponding features consistently throughout the attached drawings.

[0043] The present invention is a Human Carried Explosive Detection system and method for standoff detection of human carried explosives (HCE). The system and method automatically detects HCE at ranges between 10 m-200 m, preferably about one hundred meters, and within seconds alerts an operator to HCE threats. The system and method employ a polarized radar system emitting a narrow beam, preferably between about 0.5° to 1°, to detect either fragmentation or nonfragmentation explosives by comparison of the energy differential between co-polarized and cross-polarized reflections from a target to reference values of similar targets not carrying explosives. In a first embodiment, the system and method employ radar only, operated in either a scan mode or a track mode, mounted on a pan and tilt two-axis gimbal, and software for assessment of the target threat. In a second embodiment, the system and method use a vision-based sensor collocated with the radar to detect and assess the target threat.

[0044] The system is preferably low-cost and/or portable, and may include a multi-sensor processor, an operator console, and a wide-band wireless communications link to a handheld display. The multi-sensor processor receives video feed from at least one camera and automatically detects and tracks all humans in the field of view. Using track data to continuously cue the narrow beam radar to a subject of interest, the radar repeatedly interrogates cued objects, producing a multi-polarity radar range profile for each interrogation event.

[0045] Range profiles and associated features are automatically fused over time until sufficient evidence is accrued to support a threat/non-threat declaration hypothesis. Once a decision is made, the system alerts operators through a wireless handheld display.

[0046] The vision-based system may be any imaging system, and may include at least one Ladar, Lidar, infrared, multispectral, hyperspectral, or imaging radar output signal instead of or in addition to a video output signal. The radar system may be dual polarized with either HH (horizontal transmit-horizontal receive) and HV (horizontal transmit-vertical receive), or VV (vertical transmit-horizontal receive) and VH (vertical transmit-horizontal receive), alternating polarization (HH and HV alternating with VV and VH), fully polarimetric (HH, VV, HV and VH), or may use either right or left circular polarization.

[0047] The differential between the co-polarized reflection and the cross-polarized reflection may be a simple difference in energy levels, but is preferably calculated as a ratio referred to as an extract RF discriminant.

[0048] Accordingly, the invention to provides a system and method for standoff detection of human explosives that protects stationary and mobile positions from concealed human-carried explosives from a range outside a blast zone of the explosives. The invention also provides improved elements and arrangements thereof for the purposes described which is inexpensive, dependable and fully effective in accomplishing its intended purposes. The system has several embodiments and is designed to automatically detect HCE out to a range of between about ten meters to two hundred meters, preferably about one hundred meters, and within seconds alerts an operator to HCE threats.

[0049] The human chest gives a characteristic scattered response signature when excited by the electromagnetic field produced by radar. This chest wall response signature is modified when non radar-transparent objects, such as explosive devices or concealed weapons, are placed on or near the chest. Signature modification can take the form of additional energy returns from highly reflective components (metal objects) or attenuation if the device obscures the radar view of the chest.

[0050] Most clothing types have a relatively low dielectric constant and are highly transparent over a wide range of radar frequencies. The body, on the other hand, is highly conductive, since it contains a large fraction of water. Research regarding the measurement of the radar reflectivity of various tissue types has been performed which characterizes specific absorption rates of RF energy in the context of Electro Magnetic (EM) radiation hazard analysis. Measurements and models place the radar cross-section (RCS) of a human at roughly one square meter, or equivalent to the radar return from a perfectly conducting metal sphere with a diameter of one meter.

[0051] When a hidden threat device is concealed on or near the chest wall, the radar view of the chest wall can become obscured and there is the possibility of collecting a radar return from the device, either directly or indirectly, as in the case of an interaction between the chest wall and the device. Obscuration of the chest induces changes in the expected chest wall signature. Direct radar returns from a threat device placed between the chest wall and the radar produce near-range returns that appear closer to the radar and can be separated from the chest wall return. The HCE Detection System detects the induced changes in the return RF signature from the chest wall due to non radar-transparent objects, both reflective and non-reflective.

[0052] The present inventors have conducted tests with 3-D geometric models of a six-foot (1.829 meters) human using three dielectric layers that model the electrical properties of skin, subcutaneous or fatty tissue, and muscle tissue. The tests were conducted on a model not carrying explosives, a model equipped with a suicide bomber vest carrying fragmentation explosives, and a suicide bomber vest carrying explosives only without fragments. When illuminated with a 77 GHz radar beam, both the fragmentation explosives model and the explosives only model showed a 30 dB energy gain over the bare torso in a cross-polarized channel (VH) as compared to a bare torso. Similarly, the ratio of co=polarized energy to cross-polarized energy for a bare torso was significantly higher than for the fragmentation explosives and explosives only model over a wide range (about 60°) of look angles. Consequently, humans carrying explosives, whether fragmentation explosives or explosives only, can be detected with at least a dual polarized radar system. The problem is to focus a narrow beam (between 0.5° to 1°) of polarized radiation on a target within a wide field of regard to obtain sufficient look angles to confirm a threat at standoff distances automatically and within as short a time period as possible to effectuate evacuation or mitigation measures.

[0053] FIG. 1 and the system block diagram of FIG. 8A illustrate the radar-only embodiment 100 of the System. The radar-only embodiment 100 of the System is lightweight, portable, and includes a radar system mounted on a sensor platform 124, a computer system 102 connected to the radar system, and an operator console 120.

[0054] The radar system includes a radar transceiver 106 electrically connected to at least one radar antenna 108. The radar transceiver 106 is comprised of a transmitter portion and a receiver portion, and upon command of a radar controller 816 in computer 102, the radar transceiver 106 and antenna 108 transmit a narrow beam radar towards a field of regard 118, thereby illuminating the subject 110 and threat device 112 with electromagnetic radiofrequency (RF) energy. The beam is polarized, either horizontally, vertically, or with right or left circular polarization.

[0055] The sensor platform 124 is mounted to a portable stand 114 by a gimbal 104 such that a pan/tilt controller 104 in computer 102 may automatically adjust the position of the platform 124.

[0056] The computer 102 has an architecture similar to that of commercially available computers, and includes, at a minimum, a keyboard 812, a pointing device 814, random access memory (RAM) memory 804, and central processor unit (CPU) 802. The program instruction code implementing the HCE detection and tracking algorithms is recorded onto standard computer readable medium and loaded into memory using a compact disk (CD) reader 806 or floppy disk drive 808. Alternatively, the program instruction code may be read directly from another computer via a standard communication port 818 available on most computers. The communication port 818 may be a network connection enabling real-time collaboration between multiple HCE detection systems in the tracking and detection of human carried explosives.

[0057] In addition to the standard computer interfaces, the computer 102 has a radar controller 816, which under control of the program instruction code triggers the radar and digitizes analog data received from the radar receiver portion of the radar transceiver. Furthermore, the computer 102 contains an interface 820 to the pan/tilt two-axis gimbal 104 that controls the horizontal and vertical positioning of the sensor platform 124.

[0058] Reliable radar-based detection of hidden HCE 112 requires collecting several RF signature measurements of the subject 110 as it is moving within the radar field of regard 118. Two reasons for this are collection geometry and scintillation. There is no guarantee that the explosive device 112 will be visible from the radar look direction given an arbitrary view of a subject 110 (i.e., if their back is toward the sensor 108). However, as the system tracks the subject 110 in the radar's field of regard, the number of opportunities to gain favorable views of the HCE device 112 increases. Additionally, radar signatures of man-made objects and clutter can be highly variable due to the specular nature of returns from flat surfaces and the effect of constructive and destructive interference between coherent scatterers that lie in adjacent range bins.

[0059] In operation, the radar assembly is steered via commands to the gimbal controller 104 sent from the processor 102 so that the main-lobe of the radar beam 118 intersects the subject 110 and threat device 112 at a nominal standoff range R of one hundred meters. Energy scattered off of the subject 110 and threat device 112 is collected at the antenna 108, sent to the transceiver 106, and converted to digital format by the processor 102. Signal processing algorithms hosted on the processor 102 are then used to automatically classify the return signal as “threat” or “non-threat.”

[0060] Still referring to FIG. 1, the operator console function and user interface may consist of a standard display terminal 120 connected to computer 102. Alternatively, the operator's console 120 may be a portable or notebook computer having a display, keyboard and pointing device electrically connected to the computer 102 via a hardwired or wireless link. Finally, notifications such as a warning and indication and associated video signal may be sent to a remote field operator for viewing on a wireless handheld computer 122 over a wireless broadband communication link.

[0061] FIG. 2 is a flowchart representing the processing performed by the radar-only embodiment 100 of the System. Two modes of operation are available in the radar-only embodiment 100: scan-mode 206 and track mode 204. An operator initiates the system by selecting the desired operation mode 202. In scan mode 206, the operator defines a scan pattern consisting of the azimuth and elevation extent of the radar field of regard, a sweep rate and a radar pulse repetition frequency (PRF). The scan pattern generator software within the processor 102 uses these parameters to generate a sequence of radar control signals that drive the radar beam position control module 208 and triggers the radar transceiver 106.

[0062] Track mode 204 has two operator-selected modes: manual and automatic. Manual mode allows the operator to steer the radar beam using a joystick, trackball, keyboard or similar two-axis control input. In manual track mode, the operator has a further option as to the mode of radar triggering: either manual triggering or automatic triggering with a user selected PRF. In automatic track mode error signals are fed back from the radar and used in an adaptive scan pattern to maintain the beam on moving subjects. As with the manual track mode, the user selects a desired radar PRF.

[0063] FIG. 5 illustrates a high-level block diagram of the sensor platform 124 with the radar transceiver 106 and radar antenna 108 mounted thereon. When the radar transceiver 106 is triggered, the antenna 108 launches RF energy at the heading set by the beam position coordinates supplied by the scan-mode 206 or track-mode 204 controllers. After the radar has been positioned, the computer 102 sends a trigger pulse 504 to the radar transmitter portion 502 of the radar transceiver 106. Upon receiving the trigger pulse 504, the radar transmitter 502 generates a signal 506 that is upconverted to millimeter-wave frequencies and is transmitted through the radar antenna 108. Backscattered radar returns 620 and 622 are collected at the radar antenna 108 and sent to the radar receiver portion 508 of the radar transceiver 106 where they are downconverted to baseband and digitized. Digital radar signature data is then transmitted to the processor via digital data streams 510 and 512 to the processor 302 along with heading information extracted from the pan/tilt controller 104.

[0064] FIG. 6 illustrates a representative high-level schematic of the radar assembly of transceiver 106 and antenna 108. The radar transceiver 106 incorporates a low-cost, commercially available FMCW (frequency-modulated continuous wave) radar, preferably operating at either a 77 GHz or 94 GHz center frequency, although a center frequency anywhere in the range from about 10 GHz to 110 GHz may be used. A trigger pulse 504 is sent from the radar controller 820. The trigger 504 initiates a voltage-controlled oscillator (VCO) 602 that generates a frequency tone that is swept over a predefined range, f_min to f_max. In the preferred embodiment, the bandwidth, f_max−f_min, is at least 500 MHz and preferably closer to 1 GHz.

[0065] In the preferred embodiment, the output of a MMW (millimeter wave) local oscillator (LO) 604 is collocated with the VCO 602 and both oscillators are protected 606 from temperature variations. The output of the LO 604 is split into two parts using a power divider 608. One half of the LO signal is sent through an isolator 610 and then mixed with the VCO output to upconvert the VCO signal to the MMW frequency band. The resulting signal is amplified by a high-gain MMW power amplifier 614 into a signal 506, which is then transmitted through a Gaussian optic antenna 108. For a system operating at a 94 GHz center frequency, the preferred embodiment uses a 12″ (0.3048 meters) diameter round antenna 616 with either a vertical or horizontally polarized feed element 630.

[0066] The RF signature collection 210 of the backscattered response from objects within the footprint of radar beam 118 may be collected by using a common transmit and receive aperture, as in a mono-static radar system, or using separate transmit and receive apertures, as in a bi-static or multi-static configuration. Furthermore, a multi-element, phased-array antenna and space-time adaptive processing may be deployed. In the preferred embodiment, shown in FIG. 6, the reflected radar signal is collected using a separate receive antenna 618. The receive antenna 618 is a single 12″ (0.3048 meters) diameter round Gaussian optic antenna and simultaneously collects both H and V polarizations. Dual polarization collection is accomplished by inserting a polarizing grid 632 in the antenna 618 and splitting the V and H components of the return signal into two different feeds 620 and 622. The two polarizations are down-converted to baseband by being mixed with the same LO used to upconvert the transmitted VCO output. In this case, the other half 612 of the LO outputs after the first power divider 608 is sent through a second power divider 624, each of the two outputs is then sent through an isolator 626 or 628 before being mixed with the two received antenna signals 620 and 622. The resulting signals, in this case, VV 510 and VH 512, are sent to the radar controller 820 where they are converted into digital format via an analog-to-digital converter.

[0067] The received radar signals are then processed through two decision stages 212 and 216. The first processing stage is detection 212, in which the return signature is temporally adaptively filtered using a constant false-alarm rate (CFAR) technique to extract range bins where the scattered response energy exceeds the energy of spurious returns caused by clutter and receiver noise. Signals passing the CFAR test are then processed in a subsequent sequential hypothesis testing stage 216 to determine if they are consistent with returns from a threat device 112. Signals that do not pass the CFAR threshold test are discarded 214.

[0068] Three outcomes are possible at the output of the sequential hypothesis test 216: “Yes” indicating a definite “threat”, “No” indicating a definite “non-threat”, and “?” indicating more information is needed. In the event that insufficient evidence is available to support either the “threat” or “non-threat” hypotheses, the processor 102 can direct the operator, if in manual track mode, to hold the radar beam 118 on target until a sufficiently large number of radar samples has been collected to allow a reliable declaration of the threat status. In automatic track mode or scan mode the processor 102 automatically positions the beam 118 until sufficient data is received.

[0069] Events that trigger a “threat” determination may then be used to initiate one or more mitigation responses 220 to be discussed later. Information available at the output of the radar-only embodiment 100 of the invention includes the estimated range and heading of the threat relative to the radar sensor, as well as the time at which the threat was detected. Additional characteristics, such as velocity vector, radar cross-section and other distinguishing signature features might also be provided to the operator as an aid to intercepting the threat 112. As an example, one may consider continuing the radar track during the interception operation to provide real-time updates of position and heading information on detected threats 112.

[0070] The preferred embodiment incorporates a radar system having at least one transmit polarization and at least one receive polarization, i.e., at least a dual polarized system. However, in alternative embodiments the radar system may be alternately polarized, fully polarized, or use left or right circular polarization. In the event of multiple polarizations the span of polarizations will contain an orthogonal pair of either or both of transmit or receive polarizations.

[0071] FIG. 3 and FIG. 8B illustrate an alternative embodiment of the HCE Detection System, which incorporates at least one video camera to improve threat determination and tracking capabilities of the System. In addition to the system components of the radar-only embodiment 100, the radar plus video embodiment 300 includes a fixed, wide field of view (WFOV) video camera 306 that is aimed at the area under surveillance. For greater resolution, a steerable narrow field of view (NFOV) video camera 304 may be mounted on platform 124 and co-boresighted with the radar beam 118. As best illustrated in FIG. 8B, the video outputs of the WFOV camera 306 and the NFOV camera 304, if present, are connected to one of several frame-grabbing interface products 830 and 832 known and available to those skilled in the art. Depending upon system requirements, the cameras 304, 306 may be any combination of black and white, color, infrared, Ladar, Lidar, imaging radar, hyperspectral, or multi-spectral cameras.

[0072] It is well within the abilities of one skilled in the art of methods for detecting and tracking humans on foot in RBG (red-blue-green) color video data to extend the methods in order to detect and track people with data from other types of imaging sensors having only one intensity value at each pixel instead of the three intensity values provided by a color video camera. For example, a static background viewed by a stationary camera can represent the background as patterns of light and dark in black/white intensity data from a B/W video camera, a LADAR/LIDAR sensor, or an imaging MMW radar sensor. Moving people in the scene can then be detected by subtracting an average view of the static background intensities from each new frame of sensor data to find foreground pixels that represent a significant change from the background and therefore potential images of people moving in the foreground across the background. Foreground pixels can be grouped and detected as people based on the size and shape of the group of pixels. A detected person can then be tracked across subsequent frames by performing subtraction of the background intensity values.

[0073] Similarly, hyperspectral and multi-spectral video cameras can be used to detect humans by subtracting a representation of the background in color space where the data is represented by more than the three colors of a standard RBG video camera. Also, in a similar manner, range images in an angle-angle-range image from a LADAR, LIDAR, or MMW imaging radar can be used to detect and track people by subtracting an average representation of the background in range-image space. Whether these alternative sensors have better detection and tracking performance than standard color video depends on the nature of the scene and the illumination. Improved detection and tracking performance can be obtained by using multiple types of imaging sensors in combination, but at an added expense is incurred by adding additional sensors to the system and additional processors to interpret the data from the additional sensors.

[0074] FIG. 7 is a high level flowchart illustrating the process by which the video system (either one or two cameras) 702, maintains a track on the object until the radar system 704 has accumulated enough evidence to conclusively determine 706 the object's “threat” versus “non-threat” status. In the event that there is not enough evidence to make one of those two declarations, there is a feedback mechanism 708 by which the radar system continues to request track information from the video system. When “non-threat” is declared, the track is dropped from the list of objects being tracked. When “threat” is declared, on the other hand, the detection event and track information is passed back to the video system for final tracking and notification of an operator.

[0075] As previously disclosed, in the radar plus video embodiment 300, video signals from the WFOV video camera 306 are operated upon by a video-tracking algorithm stored in program memory that is designed to detect and extract moving subjects from the WFOV camera 306 field of regard 308. The heading of each detected moving subject is estimated, and corresponding pan/tilt position commands are sent to the gimbal 104. Also at this time, the processor 302 sends a trigger to the radar transceiver 106 to initiate the transmission of one or more RF signals through the antenna 108 toward the subject 110 and any threat device 112 present on the subject 110. Energy scattered off of the device 112 and subject 110 is collected at the antenna 108 and forwarded to the transceiver 106, which then transmits the data to the processor 302 where the data is processed using signal detection and classification algorithms.

[0076] The signal detection and classification algorithms classify the reflected radar signal as being either “threat” or “non-threat”. In the preferred embodiment, the NFOV video 304 refines the track-heading information extracted from within the field of view 308 of the WFOV video data to provide an estimate of the pose and posture of each detected subject 112.

[0077] Two modes of operation are available in the radar plus video embodiment 300. In the first mode, referred to as “video cued radar,” video motion detection and tracking algorithms implemented in software stored in memory in processor 302 are used to detect, segment and track moving subjects 112 in the video field of view. Track headings of subjects 112 detected in the video stream are used to control the radar beam position 118 and to trigger the radar in order to interrogate the threat status of each tracked object in the video field of view.

[0078] In the second operation mode, the radar operates in continuous scan mode with a scan pattern that covers the WFOV video camera 306 field of view 308. Radar pulse returns are tagged with the time and position of the pan/tilt head to allow registration of the radar data to the video sequence, both in time and look angle.

[0079] Whereas FIG. 7 illustrates the high level process by which the radar plus video embodiment 300 tracks and performs threat determination, FIG. 4 illustrates in greater detail the interaction between the video and radar sensors, as well as the HCE threat detection process. The operation of the radar plus video embodiment 300 can be broken down into four blocks A-D. Starting with Block A, at step 404, the WFOV camera 306 collects video data and transmits the data to the processor 302 which applies motion detection and segmentation software implemented algorithms 406 to separate moving objects from stationary background. A two-sided composite hypothesis test 410 is then applied to classify each detected moving object as being “human” or “other.” A database of constraints 408 on human characteristics is utilized to classify the object type of each moving object detected. This database 408 contains data elements including, but not limited to size, shape, thermal profile (if applicable for Infrared or multi-spectral cameras) and color (if applicable, for color cameras), and motion as an aid in classifying object type. Moving objects that are not consistent with human motion are discarded from further consideration 414.

[0080] Detected moving objects that have been classified as human-like are then sent to the motion tracker module 412 along with the video stream collected by the WFOV camera 306 at step 404. Information passed to the Tracker Module 412 by the Motion Detector Module 406 includes the video frame number, a mask or region of interest delimiters that describe the location of the moving object in the video frame, and the ancillary statistics used in the initial hypothesis test 410. The Motion Tracker Module 412 generates and maintains a track for each object cued by the motion detector 406. Tracks are maintained until the subject 110 leaves the field of view 310 of the WFOV camera 306.

[0081] A Threat Motion Analysis Module 416 examines each track based upon a database 414 of stored characteristics, and determines the threat potential of the moving track. Database 414 contains measurements or parameters that have been characterized as threat motions, either by an operator or from analysis of past threats. These characteristics may take the form of region constraints such as, “anybody who enters this demarked area is considered a threat”, or may consist of time and region constraints such as, “anybody who enters the demarked area at the wrong time”. Other threats could be someone moving toward the protected area at a higher rate of speed than other subjects in the area. An alternate strategy would have the Threat Motion Analysis Module 416 compare the motion statistics of new tracks to database 414 estimates of “normal” motion statistics within the surveillance area to determine if the motion of the tracked object represents an anomaly.

[0082] At any one time, a track may be classified as a “threat”, a “non-threat”, or “indeterminate”. The Threat Motion Analysis Module 416 operates to detect dynamic or emerging threats in the presence of uncertainty by applying a sequential hypothesis test. A sequential hypothesis test is a statistic based test known to those skilled in the art of statistics. Unlike a normal hypothesis test which outputs a binary “yes/no” answer, a sequential hypothesis test allows a third answer, “don't know, collect more data.” The idea is that at each point in time, you collect additional information until you have enough information to make a decision about a given hypothesis. Preset or operator selected parameters within the sequential hypothesis test enable one to incorporate the cost of collecting more data into the optimization.

[0083] With “indeterminate” threats, the hypothesis test 416 is sequentially reapplied with each new observation until a threat determination can be made or the subject has left the field of view. Note that the definition of a threatening motion can vary depending on scenario. Threatening motions could be defined as a subject with a motion vector toward the protected area and/or motion velocities that are markedly different from the average velocity of other subjects in the field of view 308.

[0084] Still referring to FIG. 4, Block B includes steps implementing the motion detection and tracking algorithm incorporated in processor 302 and is capable of simultaneously detecting and tracking multiple objects within the field of view 308 of the WFOV camera 306. Motion tracks that are designated as potential threats are forwarded from the Threat Motion Analysis Module 416 to the Track History Database 426. Each track is assigned a priority by a queuing algorithm that continuously compares the threat status of each track relative to other tracks in the database 426. Several different approaches can be used to rank track priority. One approach is to rank tracks according to the range and velocity of each subject in the video field of view 308. Subjects that are closer to the protected area and are closing at a faster rate are assigned the highest priority in the queue. Tracks associated with subjects that are exiting the field of view 308 of the WFOV camera 306 are assigned a low priority and eventually dropped from the queue when the subject leaves the field of view.

[0085] The Track Priority Queue and Scan Pattern Generator Module 422 extracts heading and velocity estimates associated with the top track in the queue and commands the position controller 442 in Module C to steer the radar antenna 108 to point at the subject 110. As best shown in FIG. 5, a trigger pulse 504 is sent from the Track Priority Queue and Scan Pattern Generator Module 422 to the radar transceiver 106 to initiate a sequence of radar pulses illuminating the subject 110 and collect scattered radar signatures to determine the presence or absence of hidden threat objects 112 on the subject 110. The nominal distance R within which the System 100 can detect a threat object 112 is one hundred meters.

[0086] Depending on the accuracy of the location estimates provided by the Motion Tracker Module 412, there are several approaches for managing the handoff between video and radar. In the current embodiment, the Position Control Module 442 is configured to scan over a narrow sector of angles centered about the heading obtained from the Motion Tracker 412. A sequence of radar returns is collected at step 444 over the sector 118. Each radar return is collected, digitized and sent through a composite binary hypothesis test 448 to determine if the return is consistent with clutter or a human subject.

[0087] In the preferred embodiment a two-parameter, constant false-alarm (CFAR) algorithm is implemented in the software of processor 302 to test for human return versus clutter. Upon a determination of “clutter”, the radar is commanded to the next position in the designated sector. As previously disclosed, the CFAR test operates on a collected RF signature of a target, the RF signature comprising a pair of complex-valued radar impulse response measurements, the co-polarized response, VV and the cross-polarized response, VH. (Note that VV=VV(k,r) and VH=VH(r) where k is the pulse number and r is the range along the radar line of sight from the radar to the far end of the region under surveillance.) For the CFAR test, the squared magnitude of each signal is computed and then summed to obtain an estimate of the total scattered energy as a function of range, i.e., |VV|²+|VH|².

[0088] Note that in a fully-polarimetric system the corresponding quantity would be referred to as the polarimetric span, |VV|²+|VH|²+|HV|²+|HH|². For the two-parameter CFAR algorithm we slide two local windows along the range dimension of |VV|²+|VH|², a “target” window and a “background” window. The average signal energy is then computed over each window at each range shift and forms the ratio between the target and background averages. Ratios that fall below a predetermined threshold are classified as clutter and are dropped from further processing. Ratios that exceed the threshold are classified as “not clutter” and the corresponding radar measurements (VV and VH) are forwarded to the feature extraction module 454. The optimal window sizes and clutter threshold are a function of the radar system design and operating environment. Typically, one selects the target window to be approximately the same length as the range extent of a human subject.

[0089] As an example, for a one-foot resolution radar with an over-sampling factor of 2:1, one skilled in the art would expect the range extent of a person viewed from the side at ground level to be 2-3 feet, or 4-6 samples. The background window size is typically chosen to be greater than or equal to the target window size, with the goal that it be smaller than any background inhomogeneities to avoid problems with estimating spatially-varying background statistics.

[0090] Previously we had disclosed how the presence of hidden threat devices on the human body induces a change in the polarization state or radiofrequency energy scattered off of the body. In the preferred embodiment, the Extract RF Discriminant Module 454 computes the polarization energy ratio:
rho(k,r)=<|VV(k,r)|²>/<|VH(k,r)|²>,
over peak regions detected by the CFAR algorithm. In the preferred embodiment we use a local window that is the same size as the CFAR target window. The quantity rho(k,r) is then forwarded to the Associate Track module 424 where it is associated with one or more of the tracks in the Track History database 426.

[0091] The beamwidth of the radar and required accuracy required to hit a human-sized object at one hundred meters is equal to the width of the radar beam, which is about 0.5 degrees. A typical video tracking system, however, can have a pointing error that is on the order of 2-3 degrees. Thus, in the initial handoff from video to radar, the radar system will need to “reacquire” the target by performing a fine search about the heading provided by the video system. Once the target has been reacquired, the new signature is compared with previous signatures from that track and an accumulated statistic is calculated.

[0092] Radar target reacquisition is accomplished by scanning the radar beam 118 in azimuth along the error diameter of the cued video heading. At 50 m-100 m ranges a 6′ (1.829 meters) tall person is between two and four radar beamwidths high. Therefore, by notionally scanning the radar at two or three different elevation angles, the System can separate upper body returns from legs and feet. At each elevation and azimuth location a radar pulse is collected and examined for the presence of a large-amplitude return along the range dimension by utilizing a cell-averaging, two-parameter, constant false-alarm rate (CFAR) detection algorithm. The statistic is a local signal-to-noise (SNR) estimate and consists of the ratio between a local signal mean within a window centered at, divided by the average background level estimated from regions outside the window. Large local SNR measurements indicate the possible presence of a target at time delay in the radar beam 118.

[0093] After the target 110 has been reacquired by the radar, a classification algorithm is applied to separate threat signatures from normal chest response signatures. As tracks and associated radar returns are accumulating in the track history database, the evidence accrual module 428 continuously polls each track and constructs a running detection statistic.

[0094] In the preferred embodiment, the evidence accrual module 428 computes a running sum of polarization ratios for each track. For the j^th track in the database, this statistic is:
phi(j)=Sum{pulse k in track j}rho(k,r)

[0095] A sequential hypothesis test 430 is then applied to each track statistic:
tau(j)=+1, phi(j)>T2, else
tau(j)=0, T1<phi(j)<=T2
tau(j)=−1, phi(j)<=T1

[0096] Three outcomes are possible: “threat” (yes, or tau(j)=+1), “non-threat” (no, or tau(j)=−1), “insufficient evidence” (?, or tau(j)=0). Tracks declared “non-threat” are marked and maintained in the track history database 426 to allow for eliminating conflicts in the case of overlapping tracks or occlusions. Tracks for which there is insufficient evidence to declare “threat” or “non-threat” are maintained and the evidence accrual process 428 is continued until a definitive “threat” or “non-threat” declaration can be made 430. Tracks declared “threat” are sent to the operator for intervention action as shown in Block D. As with the CFAR test, thresholds T1 and T2 are pre-determined. In the preferred embodiment, these thresholds are set by the operator and/or preset prior to deployment in order to achieve a desired detection versus false-alarm rate and to incorporate the cost of waiting to collect additional sample data. Note that tau(j) also has an implicit time variable, “t” which represents the time-length of the track. A further embodiment would incorporate time dependence into the thresholds, i.e., T1=T1(t) and T2=T2(t), to account for scene dynamics or time-varying statistics.

[0097] Block C represents an enhancement to the video-radar cued embodiment which contemplates the use of an optional narrow field of view (NFOV) camera 304 that is co-boresighted with the radar beam 118 under command of the Position Control Module 442. In step 446 the NFOV camera 304 collects high-resolution, zoomed-in video 310 of the object in the current track under interrogation. Video from the NFOV camera 304 is processed to extract estimates of the subject body pose 450. Pose and position estimates obtained from the NFOV camera 304 are compared with the nominal heading information provided by the WFOV camera 306. If the radar/video sensors are deemed to be out of alignment with the subject of their “gaze”, a correction vector is generated and forwarded to the Position Control Module 442 and the position of the system is updated until the radar antenna 108 and NFOV 304 camera are aligned on the subject 110. Once alignment is achieved, radar discriminants are extracted from the radar signature 454 and combined with estimated pose information in the track history database 426. Evidence accrual proceeds as before and three outcomes, “threat”, “non-threat” and “insufficient evidence” are possible.

[0098] Regardless of whether a single WFOV camera 306 is used, or a combination of WFOV 306 and NFOV 304 cameras are deployed, in the event of a “yes” decision from the HCE threat determination logic 430, the corresponding threat declaration and track information is passed at step 432 to the operator console 120 or to an optional wireless display device 122. The information displayed on the operator console 120 includes video data and associated radar statistical evidence. Based upon this data, an interdiction response can be initiated 434. Forms of response can vary from automated triggering of a mitigation system, such as automated notification of first responders to more proactive activities, e.g., the aiming, arming and firing of a high-powered water cannon, 94 GHz. active denial radar, or similar non-lethal weapon.

[0099] Additionally, video information can be forwarded to first responders in the field to provide visual information to aid in identification and apprehension of the threat subject 110.

[0100] FIG. 9 illustrates an application in which a plurality of HCE Detection Systems 904A-904C collaborate to protect a specific site 910. As shown, HCE Detection Systems 904A-904C are networked together to form a collaborative detection and tracking system in order to achieve improved detection and false alarm rejection within an area under surveillance 902. In order to coordinate surveillance between multiple HCE detection systems, each System 904A, 904B and 904C must be connected to a wideband network 930. Threat mitigation devices 926 in communication with and under control of the HCE Detection Systems operate to neutralize the threat 936 with a water cannon or active radar beam 922. The network 930 may be a hardwired or wireless network.

[0101] Collaborative tracking and detection can be implemented in several ways. In one implementation, inter-computer communication coordinates the detection and tracking of potential targets 932, 934, and 936 by overlapping the radar field of regard 914A-914B of each system 904A-904B. The resulting measurements of the RF signatures are combined to achieve simultaneous views of the field of regard from different look directions to increase the validity of the resultant threat determination. In multiple system collaborative mode, feature-based tracking has been implemented to associate space-time coordinates and subject appearances as the threat passes through crowds having occluded regions.

[0102] A second implementation of a collaborative network of HCE detection systems can be used to expand the area of surveillance 902 by linking HCE radar systems 904A-904C having non-overlapping fields of regards (not shown). Intersystem communication would coordinate the detection and tracking of potential human carried explosives over the union of the system wide fields of regard. Feature-based tracking would associate space-time coordinates and appearances of subjects that move from one system field of regard to another system's field of regard. Decision processing and threat mitigation, performed centrally or decentralized at the individual system level would detect and classify the nature of the threat, and as heretofore disclosed, neutralize threat 936 as necessary.

[0103] The radar system is comprised of relatively low cost, commercially available components. Furthermore, the specific radar system deployed is not limited to the specific embodiments heretofore disclosed. In the preferred embodiment, the radar system is similar to radar-controlled automobile cruise control systems having a center frequency of about 77 GHz. and has a nominal range of approximately one hundred meters. Non-limiting, the radar system may operate at different or multiple frequencies, including the Ka-band, the W-band, UHF, and UWB frequencies. Furthermore, the radar system need not be comprised of a Frequency Modulated Continuous Wave (FMCW) real beam radar. An alternate radar system may consist of a linear chirp waveform.

[0104] Furthermore, an alternative to the previous embodiments 100 and 300, in which the radar transmitter is collocated with the radar receiver, would entail a multi-static system in which one or more receivers is mounted apart from the one or more transmitters.

[0105] It is to be understood that the present invention is not limited to the embodiments described above, but encompasses any and all embodiments within the scope of the following claims.