Traffic sensor

[0002] A claim of priority is made to U.S. Provisional Patent Application Serial No. 60/043,690, entitled TRAFFIC SENSOR, filed Apr. 14, 1997.

[0003] The development of at least some of the presently disclosed subject matter may have been funded in part by the United States Government. The United States government may have rights in the invention.

[0004] The present invention is related to traffic monitoring systems, and more particularly to a traffic monitoring system for detecting, measuring and anticipating vehicle motion.

[0005] Systems for monitoring vehicular traffic are known. For example, it is known to detect vehicles by employing inductive loop sensors. At least one loop of wire or a similar conductive element may be disposed beneath the surface of a roadway at a predetermined location. Electromagnetic induction occurs when a vehicle occupies the roadway above the loop. The induction can be detected via a simple electronic circuit that is coupled with the loop. The inductive loop and associated detection circuitry can be coupled with an electronic counter circuit to count the number of vehicles that pass over the loop. However, inductive loops are subjected to harsh environmental conditions and consequently have a relatively short expected lifespan.

[0006] It is also known to employ optical sensors to monitor vehicular traffic. For example, traffic monitoring systems that employ “machine vision” technology such as video cameras are known. Machine vision traffic monitoring systems are generally mounted above the surface of the roadway and have the potential for much longer lifespan than inductive loop systems. Further, machine vision traffic monitoring systems have the potential to provide more information about traffic conditions than inductive loop traffic monitoring systems. However, known machine vision traffic monitoring systems have not achieved these potentials.

[0007] In accordance with the present invention, a traffic monitoring station employs at least one video camera and a computation unit to detect and track vehicles passing through the field of view of the video camera. The camera provides a video image of a section of roadway in the form of successive individual video frames. Motion is detected through edge analysis and changes in luminance relative to an edge reference frame and a luminance reference frame. The frames are organized into a plurality of sets of pixels. Each set of pixels (“tile”) is in either an “active” state or an “inactive” state. A tile becomes active when the luminance or edge values of the pixels of the tile differ from the luminance and edge values of the corresponding tiles in the corresponding reference frames in accordance with predetermined criteria. The tile becomes inactive when the luminance and edge values of the pixels of the tile do not differ from the corresponding reference frame tiles in accordance with the predetermined criteria.

[0008] The reference frames, which represent the view of the camera without moving vehicles, may be dynamically updated in response to conditions in the field of view of the camera. The reference frames are updated by combining each new frame with the respective reference frames. The combining calculation is weighted in favor of the reference frames to provide a gradual rate of change in the reference frames. A previous frame may also be employed in a “frame-to-frame” comparison with the new frame to detect motion. The frame-to-frame comparison may provide improved results relative to use of the reference frame in conditions of low light and darkness.

[0009] Each object is represented by at least one group of proximate active tiles (“quanta”). Individual quantum, each of which contains a predetermined maximum number of tiles, are tracked through successive video frames. The distance travelled by each quantum is readily calculable from the change in position of the quantum relative to stationary features in the field of view of the camera. The time taken to travel the distance is readily calculable since the period of time between successive frames is known. Physical parameters such as velocity, acceleration and direction of travel of the quantum are calculated based on change in quantum position over time. Physical parameters that describe vehicle motion are calculated by employing the physical parameters calculated for the quanta. For example, the velocities calculated for the quanta that comprise the vehicle may be combined and averaged to ascertain the velocity of the vehicle.

[0010] The motion and shape of quanta are employed to delineate vehicles from other objects. A plurality of segmenter algorithms is employed to perform grouping, dividing and pattern matching functions on the quanta. For example, some segmenter algorithms employ pattern matching to facilitate identification of types of vehicles, such as passenger automobiles and trucks. A physical mapping of vehicle models may be employed to facilitate the proper segmentation of vehicles. A list of possible new objects is generated from the output of the segmenter algorithms. The list of possible new objects is compared with a master list of objects, and objects from the list of possible new objects that cannot be found in the master list are designated as new objects. The object master list is then updated by adding the new objects to the object master list.

[0011] At least one feature extractor is employed to generate a descriptive vector for each object. The descriptive vector is provided to a neural network classification engine which classifies and scores each object. The resultant score indicates the probability of the object being a vehicle of a particular type. Objects that produce a score that exceeds a predetermined threshold are determined to be vehicles.

[0012] The traffic monitoring station may be employed to facilitate traffic control in real time. Predetermined parameters that describe vehicle motion may be employed to anticipate future vehicle motion. Proactive action may then be taken to control traffic in response to the anticipated motion of the vehicle. For example, if on the basis of station determined values for vehicle distance from the intersection, speed, accelleration, and vehicle class (truck, car, etc.), the traffic monitoring station determines that the vehicle will “run a red light,” traversing an intersection during a period of time when the traffic signal will be otherwise be indicating “green” for vehicles entering the intersection from another direction, the traffic monitoring station can delay the green light for the other vehicles or cause some other actions to be taken to reduce the likelihood of a collision. Such actions may also include displaying the green light for the other vehicles in an altered mode (e.g., flashing) or in some combination with another signal light (e.g., yellow or red), or initiating an audible alarm at the intersection until the danger has passed. Further, the traffic monitoring station may track the offending vehicle through the intersection and record a full motion video movie of the event for vehicle identification and evidentiary purposes.

[0013] The foregoing features of this invention, as well as the invention itself, may be more fully understood from the following Detailed Description of the Invention, and Drawing, of which:

[0014] FIG. 1A is a perspective diagram of a traffic monitoring station that illustrates configuration;

[0015] FIG. 1B is a side view diagram of a traffic monitoring station that illustrates tilt angle;

[0016] FIG. 1C is a top view diagram of a traffic monitoring station that illustrates pan angle;

[0017] FIG. 2 is a flow diagram that illustrates the vehicle detection and tracking method of the traffic monitoring station;

[0018] FIG. 3 is a diagram of a new frame that illustrates use of tiles and quanta to identify and track objects;

[0019] FIG. 4 is a diagram of a reference frame;

[0020] FIG. 5 is a diagram that illustrates edge detect tile comparison to determine tile activation;

[0021] FIG. 6 is a diagram that illustrates adjustment of segmenter algorithm weighting;

[0022] FIG. 7 is a diagram that illustrates feature vector generation by a feature extractor;

[0023] FIG. 8 is a diagram of the traffic monitoring station of FIG. 1 that illustrates the processing module and network connections;

[0024] FIG. 9 is a block diagram of the video capture card of FIG. 8;

[0025] FIG. 10A is a diagram that illustrates use of the new frame for image stabilization;

[0026] FIG. 10B is a diagram that illustrates use of the reference frame for image stabilization;

[0027] FIG. 11 is diagram of the field of view of a camera that illustrates use of entry and exit zones;

[0028] FIG. 12 is a block diagram of traffic monitoring stations networked through a graphic user interface;

[0029] FIG. 13 is a flow diagram that illustrates station to station vehicle tracking; and

[0030] FIG. 14 is a diagram of an intersection that illustrates traffic control based on data gathered by the monitoring station.

[0031] U.S. Provisional Patent Application Serial No. 60/043,690, entitled TRAFFIC SENSOR, filed Apr. 14, 1997, is incorporated herein by reference.

[0032] Referring to FIGS. 1A, 1B and 1C, a traffic monitoring station 8 includes at least one camera 10 and a computation unit 12. The camera 10 is employed to acquire a video image of a section of a roadway 14. The computation unit 12 is employed to analyze the acquired video images to detect and track vehicles.

[0033] A three dimensional geometric representation of the site is calculated from parameters entered by the user in order to configure the traffic monitoring station 8 for operation. The position of a selected reference feature 16 relative to the camera 10 is measured and entered into memory by employing a graphic user interface. In particular, a distance Y along the ground between the camera 10 and the reference feature 16 on a line that is parallel with the lane markings 17 and a distance X along a line that is perpendicular with the lane markings are measured and entered into memory. The camera height H, lane widths of all lanes W1, W2, W3 and position of each lane in the field of view of the camera are also entered into memory. The tilt angle 15 and pan angle 13 of the camera are trigonometrically calculated from the user-entered information, such as shown in Appendix A. The tilt angle 15 is the angle between a line 2 directly out of the lens of the camera 10 and a line 6 that is parallel to the road. The pan angle 13 is the angle between line 2 and a line 3 that is parallel to the lane lines and passes directly under the camera 10. A value used for scaling (“scaler”) is calculated for facilitating distance calculations. The scaler is a fixed factor for the entire image that is used for conversion between real distances and pixel displacements. Hence, the distance and direction from the camera to any point in the field of view of the camera, and the distance and direction between any two points in the field of view of the camera can be determined.

[0034] Corrections for roadway grade and bank may also be calculated during configuration. “Grade” refers to the change in height of the roadway relative to the height of the camera within the field of view of the camera. “Bank” refers to the difference in height of portions of the roadway along a line perpendicular with the lane markings. The user determines the grade and bank of the roadway and enters the determined values into memory by employing a graphic user interface. The grade and bank corrections are achieved by translating the reference plane to match the specified grade and bank.

[0035] Referring to FIGS. 2 and 3, operation of the traffic monitoring station will now be described. A video frame 18 is acquired from the camera as depicted in step 20. If an interlaced camera is employed, the acquired frame is deinterlaced. If a progressive scan camera is employed then deinterlacing is not necessary. Image stabilization techniques may also be employed to compensate for movement of the camera due to wind, vibration and other environmental factors, as will be described below. The acquired frame 18 is organized into tiles 22 as depicted in step 24. Each tile 22 is a region of predetermined dimensions. In the illustrated embodiment, each frame contains 80 tiles per row and 60 tiles per column and the dimensions of each tile are 8 pixels by 8 pixels. Tile dimensions may be adjusted, may be non-square, and may overlap other tiles.

[0036] Referring to FIGS. 2, 3 and 4, a list 26 of tiles in which motion is detected (“active tiles”) 38 is generated by employing either or both of reference frames 28, 29 and a previously acquired frame 30 in separate comparisons with the acquired frame 18. The reference frame 28 represents the luminance of the image from the camera in the absence of moving vehicles. The reference frame 29 represents edges detected in the image from the camera in the absence of moving vehicles. In the illustrated embodiment, both the reference frames 28, 29 and the previous frame 30 are employed. If a color camera is employed, the chrominance (color) portion of each tile 22 in the acquired frame is separated from the luminance (black and white) portion prior to comparison.

[0037] As illustrated in FIG. 5, an edge detect comparison may be employed to detect motion and activate tiles. For each tile 22 of the new frame 18 (FIG. 3), the tile is organized into four “quartiles” 32 of equal size. The pixel-luminance values in each quartile 32 are summed to provide a representative luminance value for each quartile. In the illustrated embodiment, each pixel has a luminance represented by a value from 0 to 255, where greater values indicate greater luminance. The quartile having the maximum representative luminance value is then identified and employed as a baseline for analyzing the other quartiles. In particular, the maximum luminance quartile 34 is designated to be in a first state, illustrated as logic 1. The other quartiles in the tile are designated to be in the first state if their representative luminance value exceeds a threshold defined by a predetermined percentage of the luminance value of the maximum luminance quartile 34 (lum ≧βlum_max). β (“the gain”) can be fixed at a specific level or may be allowed to vary based upon the characteristics of the image. Quartiles with a representative value that fails to exceed the threshold are designated to be in a second state, illustrated by logic 0. Each quartile is then compared with the corresponding quartile from the corresponding tile 36 from the reference frame 29 (FIG. 4) and, separately, the previously acquired frame. The tile 22 is designated as “eactive” if the comparison indicates a difference in the state of more than one quartile. If the comparison indicates a difference in the state of one or fewer quartiles and at least one quartile of the tile is in the second state, the tile is designated as “inactive.”

[0038] In the case where each quartile 32 in the corresponding tiles of the current frame and the reference frame are designated to be in the first state a luminance activation technique is employed. A luminance intensity value is determined by summing the luminance of all pixels in the tile and dividing the sum by the total number of pixels in the tile, i.e., computing the average luminance. The average luminance of the tile is compared with the average luminance of the tile 36 in the corresponding location of the reference frame 28 and the previous frame to detect any difference therebetween. In particular, the average luminance of the reference tile is subtracted from the average luminance of the new tile to produce a difference value and, if the magnitude of the difference value exceeds a predetermined threshold, motion is indicated and the tile is designated as “active.” The model using tiles, quartiles and pixels is isomorphic to a neural model of several layers.

[0039] Referring again to FIGS. 3 and 4, the reference frames 28, 29 may be either static or dynamic. A static reference frame may be generated by storing a video frame from the roadway or portion(s) of the roadway when there are no moving objects in the field of view of the camera. In the illustrated embodiment the reference frames 28, 29 are dynamically updated in order to filter differences between frames that are attributable to gradually changing conditions such as shadows. The reference frames are updated by combining each new frame 18 with the reference frames. The combining calculation may be weighted in favor of the reference frames to filter quickly occurring events, such as the passage of vehicles, while incorporating slowly occurring events such as shadows and changes in the ambient light level.

[0040] Referring to FIGS. 2 and 3, and Appendix B, active tiles 38 in the list 26 of active tiles are organized into sets of proximately grouped active tiles (“quanta”) 40 as depicted by step 42. The quanta 40 are employed to track moving objects such as vehicles on successive frames. The distance travelled by each quantum is calculated based upon the change in position of the quantum from frame to frame. Matching and identifying of quantum is facilitated by a “grab phase” and an “expansion phase” as depicted by step 44. Each quantum has a shape. In the “grab phase,” active tiles are sought in a predicted position that is calculated for the quantum in the new frame, within the shape defined by the quantum. The predicted position is determined by the previously observed velocity and direction of travel of the quantum. If any active tiles are located within the quantum shape region in the predicted position of the quantum in the new frame, the quantum is considered found. If no active tiles are located in the quantum shape region in the predicted position in the new frame, the quantum is considered lost. In the “expansion phase,” active tiles that are adjacent to a found quantum and that have not been claimed by other quanta are incorporated into the found quantum, thereby allowing each quantum to change shape. Unclaimed active tiles are grouped together to form new quanta unless the number of active tiles is insufficient to form a quantum. If any of the quanta that have changed shape now exceed a predetermined maximum size then these “parent” known quantum are reorganized into a plurality of “children” quantum. Each child quantum inherits the characteristics of its parent quantum, such as velocity, acceleration and direction.

[0041] The identified quanta are organized into objects as depicted in step 46. The traffic sensor employs a plurality of segmenter algorithms to organize the identified quanta into objects. Each segmenter algorithm-performs a grouping, dividing or pattern matching function. For example, a “blob segmenter” groups quanta that are connected. Some segmenter algorithms facilitate identification of types of vehicles, such as passenger automobiles and trucks. Some segmenter algorithms facilitate identification of vehicle features such as headlights. Some segmenter algorithms reorganize groups of quanta to facilitate identification of features.

[0042] Referring to FIG. 6, the segmenter algorithms are employed in accordance with a dynamic weighting technique to facilitate operation under changing conditions. Five segmenter algorithms designated by numbers 1-5 are employed in the illustrative example. One segmenter algorithm is employed in each time slot. In particular, the segmenter algorithm in the time slot designated by an advancing pointer 48 is employed. When a segmenter algorithm successfully detects and tracks an object that is determined to be a vehicle by the neural network and is consistent across a plurality of frames then that segmenter algorithm is granted an additional time slot. Consequently, the segmenter algorithms that are more successful under the prevailing conditions are weighted more heavily than the unsuccessful segmenter algorithms. However, each segmenter algorithm is assigned at least one permanent time slot 50 in order to assure that each of the segmenter algorithms remains active without regard to performance. Hence, operation dynamically adjusts to changing conditions to maintain optimum performance. It should be apparent that the number of segmenters, and number and position of the time slot allocations may be altered from the illustrative example.

[0043] A list of possible new objects represented by their component quanta is generated by the segmenters as depicted by step 54. The list of possible new objects is compared with a master list of objects, and any objects from the list of possible new objects that cannot be found in the master list is designated as a new object as depicted by step 56. The object master list is updated by adding the new objects to the object master list as depicted in step 57. The objects in the updated object master list are then classified and scored as depicted in step 58.

[0044] Referring to FIGS. 2 and 7, the objects in the master list are examined by employing at least one feature extractor 49 as depicted by step 47. Each feature extractor produces a vector 51 of predetermined length that describes an aspect of the object, such as shape. The illustrated feature extractor overlays the object with a 5×5 grid and generates a vector that describes the shape of the object. Because the number of cells 53 in the grid does not change, the representative vector 51 is relatively stable when the size of the object (in number of pixels) changes, such as when the object approaches or moves away from the camera. The vector 51 is concatenated with vectors 55 provided from other feature extractors, if any, to produce a larger vector. 57 that represents the object. Other grid patterns and combinations of overlays may be used to achieve improved results based upon camera position relative to the vehicles and other environmental factors.

[0045] Masking using a vehicle template may be employed to remove background information prior to feature extraction. The object is then compared with templates 136 that depict the shape of known types of vehicles such as cars, vans, trucks etc. When the best fit match is determined, the center of the object, where the center of the template is located in the match position, is marked and only portions of the object that are within the template are employed for generating-the vectors.

[0046] The descriptive vectors generated by the feature extractors are provided to a neural network classification engine that assigns a score to each object. The score indicates the probability of the object being a vehicle, including the type of vehicle, e.g., passenger automobile, van, truck. Objects that produce a score that exceeds a predetermined threshold are determined to be vehicles of the type indicated. If there are regions of overlap between objects in the updated object master list, ownership of the quanta in those regions is resolved in a competition phase as depicted in step 60. Of the objects in competition for each quantum in the overlap region, the object that was assigned the highest score by the neural network obtains ownership of the quanta.

[0047] Physical characteristics relating to object motion, such as velocity, acceleration, direction of travel and distance between objects, are calculated in step 62. The calculations are based on changes in position of a plurality of quanta from frame to frame. In particular, vehicle velocity may be calculated as the average velocity of the quanta of the vehicle, by the change in location of a specific portion of the vehicle such as the center front, or by other techniques. Similarly, vehicle acceleration may be calculated as the change in vehicle Velocity over time and vehicle direction may be calculated by extrapolating from direction of travel of quanta over a plurality of frames. The velocity, acceleration and direction of travel of the quanta are calculated based on known length and width dimensions of each pixel and the known period of time between successive frames.

[0048] Referring to FIG. 8, the computation unit 12 includes at least one video capture card 66. The video capture card 66 performs initial processing on video signals received from the camera 10. The computation unit 12 operates on the output of the video capture card 66. The functions described with regard to the embodiment illustrated in FIGS. 8 and 9 are implemented with a custom video capture card 66. These functions may alternatively be implemented with a commercially available frame grabber and software. In the illustrative embodiment the computation unit 12 is a commercially available IBM compatible computer that employs the Windows 95 operating system. The IBM compatible computer includes a Peripheral Controller Interconnect (“FPCI”) bus interface.

[0049] Referring to FIG. 9, the video capture card 66 is operative to process new video frames, establish and maintain the reference frame, and compare the new frames with the reference frame in order to ascertain luminance and/or edge differences therebetween that are indicative of motion. A digitizer circuit 70 is employed to convert the analog video signals from the camera. The camera may provide analog video signals in either National Television Standards Committee (“NTSC”) format or Phase Alteration Line (“PAL”) format. The chrominance portion, if any, of the video signal is separated from the luminance portion of the video signal by the digitizer circuit 70. The resulting digital signals are provided to an image state machine 72 where the video signal is deinterlaced, if necessary. In particular, the video signal is deinterlaced unless a progressive scan camera is employed. The output of the image state machine 72 is a succession of deinterlaced video frames, each frame being 640 pixels by 480 pixels in size. The image state machine is coupled to a Random Access Memory (“RAM”) 74 that includes a ring of three buffers where frame data is collected prior to transmission of the frames over a digital bus 76 via pixel fetch circuitry 78.

[0050] Referring to FIGS. 9 and 10, image stabilization is employed by the video control processor 99 to compensate for camera movement due to environmental factors such as wind. Up to two anchor features 69 that are identified by the user during configuration of the traffic monitoring station are employed. The location of each anchor 69 on the new frame 18 is determined, and the new frame is adjusted accordingly. Each anchor 69 is located by matching a template 162 to the image in the new frame. The template 162 is a copy of the rectangular region of the reference frame 28 that includes a representation of the anchor feature 69. A pixel by pixel comparison is made between the template 162 and a selected region of the new frame to determine whether a match has been found based on average luminance difference. The selected region of the new frame is adjusted until the best match is located (best match=_minΣ|New_x,y−Ref_x,y|). The first comparison may be made at the coordinates at which the anchor 69 is located in the reference frame, or at the coordinates at which the anchor was located in the previous frame. If a _minΣ calculation that is less than or equal to the _minΣ calculation in the previous frame is found, the location is determined to be a match, i.e., the anchor is found. If a mine calculation that is less than or equal to the _minΣ calculation in the previous frame is not found, the location of the selected region is adjusted until the best match is located. The location of the selected region is adjustable within an area of up to 8 pixels in any direction from the matching coordinates of the previous frame. The selected region is shifted in turn both vertically and horizontally by a distance of four pixels to yield four _min93 calculation results. If the lowest of the four results is lower than the result at the start point, the selected region is moved to the place that yielded the lowest result. If none of the results is lower than the result at the start point, the selected region is shifted in turn both vertically and horizontally by half the original distance, i.e., by two pixels, to yield four new _minΣ calculation results. If the lowest of the four results is lower than the result at the start point, the selected region is moved to the place that yielded the lowest result. The distance may be halved again to one pixel to yield four new _minΣ calculation results. When the best result is found, the anchor is considered found if the result achieves a predetermined threshold of accuracy. If the best result fails to achieve the predetermined threshold of accuracy, an edge comparison is undertaken. The edge comparison is made between the template and the region that defines the best _minΣ calculation results. If at least one vertical edge, at least one horizontal edge, and at least 75% of all constituent edges are matched, the anchor is considered found. Otherwise, the anchor is considered not found.

[0051] The new frame 18 is adjusted to produce a stabilized frame based upon how many anchors 69 were found, and where the anchors were found. In the event that both anchors are found and the anchors were found at the same coordinates as in the reference frame, the camera did not move and no correction is necessary. If both anchors moved by the same distance in the same direction, a two-dimensional X-Y offset vector is calculated. If both anchors moved in different directions, the camera may have zoomed and/or rotated. A zoom is indicated when the anchors have moved either towards or away from the center 164 of the image. For example, the anchors appear larger and further from the center of the image when the camera zooms in, and smaller and closer to the center of the image when the camera zooms out. In the case of a camera that is zoomed out, the image is “inflated” by periodically duplicating pixels so that the anchors appear in the expected dimensions. In the case of a camera that is zoomed in, the image is “deflated” by periodically discarding pixels so that the anchors appear in the size that is expected.

[0052] In the event that only one anchor was found, adjustment is based upon the location of that one anchor. If the anchor did not move, no correction is applied. If the previous frame was not zoomed, it is assumed that the new frame is not zoomed. If the previous frame was zoomed, it is assumed that the new frame is also zoomed by the same amount. In the event that neither anchor is found, the corrections that were calculated for the previous frame are employed.

[0053] From the number of anchors found and their positions, the video control processor 99 calculates a set of correction factors as described above and sends them to the pixel fetch circuitry 78. These correction factors include instructions for shifting the frame horizontally, vertically, or both to correct for camera pan and tilt motion, and/or directions for inflating or deflating the frame to compensate for camera zoom motion. If no correction is needed, the video control processor calculates a set of correction factors which instructs the pixel fetch circuitry to do a simple copy operation. The correction factors allow the pixel fetch circuitry to select pixels from RAM 74 for transmission on the bus 76 in stabilized order. The pixels are collected into a stabilized frame 84 for use by the computation unit 12 (FIG. 8).

[0054] Referring to FIG. 9, a differencing unit 82 employs the contents of the reference frame buffer 80 and the incoming pixels on the bus 76 to compare the reference frame with the stabilized frame, pixel by pixel, in order to determine the differences. The difference values are stored in the difference frame buffer 86. The computation unit 12 may access the difference frames over the PCI bus 94.

[0055] A tiling unit 88 is operative to organize the incoming pixels on bus 76 into tiles 22 (FIG. 3). The tiles are stored in a tile buffer 90 for use by the computation unit 12 (FIG. 8), which may access the tiles via the PCI bus 94.

[0056] Referring to FIG. 11, user-defined zones may be employed to facilitate operation where the view of the camera is partially obstructed and where sections of roadway converge.

[0057] An entry zone is employed to designate an area of the video image in which new objects may be formed. Objects are not allowed to form outside of the entry zone. In the illustrated example, an overpass 100 partially obstructs the roadway being monitored. By placing an entry zone 102 in front of the overpass 100, undesirable detection and tracking of vehicles travelling on the overpass is avoided. A second entry zone 104 is defined for a second section of roadway within the view of the camera. Vehicles entering the roadway through either entry zone 102 or entry zone 104 are tracked. An exit zone 106 is employed to designate an area where individual vehicles are “counted.” Because of the perspective of the field of view of the camera, more distant vehicles appear smaller and closer together. To reduce the likelihood of multiple vehicles being counted as a single vehicle, the number of vehicles included in the vehicle count is determined in the exit zone 106, which is proximate to the camera.

[0058] Referring now to FIG. 12, a plurality of traffic monitoring stations 8 may be employed to monitor and share data from multiple sections of roadway. Information gathered from different sections of roadway may be shared via a computer network. The gathered information may be displayed on a graphic user interface 108 located at a separate operations center 110. Video images 112 from the camera are provided to the graphic user interface 108 through flow manager software 114. The flow manager maintains near actual time display of the video image through balance of video smoothness and delay by controlling buffering of video data and adapting to available bandwidth. Data resulting from statistical analysis of the video image is provided to the graphic user interface 108 from an analysis engine 116 that includes the tiling unit, segmenter algorithms and neural network described above. The controller card may be employed to transmit the data through an interface 118 to the operations center 110, as well as optional transmission to other traffic monitoring stations. The interface 118 may be shared memory in the case of a standalone monitoring station/graphic user interface combination or sockets in the case of an independent monitoring station and graphic user interface. The operations center 110 contains an integration tool set for post-processing the traffic data. The tool set enables presentation of data in both graphical and spreadsheet formats. The data may also be exported in different formats for further analysis. The video may also be displayed with an overlay representing vehicle position and type.

[0059] Alarm parameters may be defined for the data generated by the analysis engine 116. For example, an alarm may be set to trigger if the average velocity of the vehicles passing through the field of view of the camera drops below a predetermined limit. Alarm calculations may be done by an alarm engine 122 in the traffic monitoring station or at the graphic user interface. Alarm conditions are defined via the graphic user interface.

[0060] Networked traffic monitoring stations may be employed to identify and track individual vehicles to determine transit time between stations. The shape of the vehicle represented by active tiles is employed to distinguish individual vehicles. At a first traffic control station, a rectangular region (“snippet”) that contains the active tiles that represent a vehicle is obtained as depicted by step 132. Correction may be made to restore detail obscured by inter-field distortion as depicted by step 134. The snippet is then compared with templates 136 that depict the shape of known types of vehicles such as cars, vans, trucks etc, as depicted in step 138. When the best fit match is determined, the center of the snippet, where the center of the template is located in the match position, is marked as depicted by step 140. Further, the size of the snippet may be reduced to the size of the matching template. First and second signatures that respectively represent image intensity and image edges are calculated from the snippet as depicted-by step 142. The signatures, matching template type, vehicle speed and a vehicle lane indicator are then transmitted to a second traffic monitoring station as depicted by step 144. The second traffic monitoring station enters the information into a list that is employed for comparison purposes as depicted in step 146. As depicted by step 148, information that represents vehicles passing the second traffic monitoring station is calculated by gathering snippets of vehicles and calculating signatures, a lane indicator, speed and vehicle type in the same manner as described with respect to the first traffic monitoring station. The information is then compared with entries selected in step 149 from the list by employing comparitor 150. In particular, entries that are so recent that incredibly high speed would be required for the vehicle to be passing the second traffic monitoring station are not employed. Further, older entries that would indicate an incredibly slow travel rate are discarded. The signatures may be accorded greater weight in the comparison than the lane indicator and vehicle type. Each comparison yields a score, and the highest score 152 is compared with a predetermined threshold score as depicted by step 154. If the score does not exceed the threshold, the “match” is disregarded as depicted by step 156. If the score exceeds the threshold, the match is saved as depicted by step 158. At the end of a predetermined interval of time, a ratio is calculated by dividing the difference between the best score and the second best score by the best score as depicted by step 160. If the ratio is greater than or equal to a predetermined value, a vehicle match is indicated. The transit time and average speed of the vehicle between traffic monitoring stations is then reported to the graphic user interface.

[0061] Inter-field distortion is a by-product of standard video camera scanning technology. An NTSC format video camera will alternately scan even or odd scan lines every 60th of a second. A fast moving vehicle will move enough during the scan to “blur,” seeming to partially appear in two different places at once. Typically, the car will move about 1.5 ft during the scan (approx. 60 mph). Greater distortion is observed when the car travels at higher velocity. Greater distortion is also observed when the vehicle is nearer to the camera. The distortion compensating algorithm is based on knowledge of the “camera parameters” and the speed of the vehicle. The camera parameters enable mapping between motion lo in the real world and motion in the image plane of the camera. The algorithm predicts, based on the camera parameters and the known speed of the vehicle, how much the vehicle has moved in the real world (in the direction of travel). The movement of the vehicle on the image plane is then calculated. In particular, the number of scan lines and distance to the left or right on the image is calculated. Correction is implemented by moving the odd scan lines ‘back’ to where the odd scan lines would have been if the car had stayed still (where the car was when the even scan lines were acquired). For example, to move 4 scan lines back, scan line n would be copied back to scan line n−4, where n is any odd scan line. The right/left movement is simply where the scan line is positioned when copied back. An offset may be added or subtracted to move the pixels back into the corrected position. For example, scan line n may have an offset of 8 pixels when moved back to scan line n−4, so pixel 0 in scan line n is copied to pixel 7 in scan line n−4, etc. If the speed of a particular vehicle cannot be determined, the average speed for that lane may be employed to attempt the correction. Distortion correction is not necessary when a progressive scan camera is employed.

[0062] Referring to FIG. 14, the traffic monitoring station may be employed to facilitate traffic control. In the illustrated embodiment, the traffic monitoring station is deployed such that an intersection is within the field of view of the camera. Vehicle detection can be employed to control traffic light cycles independently for left and right turns, and non-turning traffic. Such control, which would require multiple inductive loops, can be exerted for a plurality of lanes with a single camera. Predetermined parameters that describe vehicle motion are employed to anticipate future vehicle motion, and proactive action may be taken to control traffic in response to the anticipated motion of the vehicle. For example, if the traffic monitoring station determines that a vehicle 124 will “run” a red light signal 125 by traversing an intersection 126 during a period of time when a traffic signal 128 will be indicating “green” for a vehicle 130 entering the intersection from another direction, the traffic monitoring station can provide a warning or control such as an audible warning, flashing light and/or delayed green light for the other vehicle 130 in order to reduce the likelihood of a collision. Further, the traffic monitoring station may track the offending vehicle 124 through the intersection 126 and use the tracking information to control a separate camera to zoom in on the vehicle and/or the vehicle license plate to record a single frame, multiple frames or a full motion video movie of the event for vehicle identification and evidentiary purposes. The cameras are coordinated via shared reference features in a field of view overlap area. Once the second camera acquires the target, the second camera zooms in to record the license plate of the offending vehicle. The traffic monitoring station could also be used to detect other types of violations such as illegal lane changes, speed violations, and tailgating. Additionally, the traffic monitoring station can be employed to determine the optimal times to cycle a traffic light based upon detected gaps in traffic and lenghts of queues of cars at the intersection.

[0063] The determination of whether the vehicle will run the red light may be based upon the speed of the vehicle and distance of the vehicle from the intersection. In particular, if the vehicle speed exceeds a predetermined speed within a predetermined distance from the intersection it may be inferred that the vehicle cannot or is unlikely to stop before entering the intersection.

[0064] Having described the embodiments consistent with the present invention, other embodiments and variations consistent with the present invention will be apparent to those skilled in the art. Therefore, the invention should not be viewed as limited to the disclosed embodiments but rather should be viewed as limited only by the spirit and scope of the appended claims.