Radio frequency environment object monitoring system and methods of use

[0001] 1. Field of the Invention

[0002] This invention relates to a radio frequency environment monitoring system for object presence feedback. More particularly, the invention relates to a radio frequency environment monitoring system for RFID tagged and/or untagged object monitoring via analysis of changes to the monitored RF environment occurring as the objects present in the monitored area are varied in position and or number.

[0003] 2. Description of Related Art

[0004] Previous RFID tag inventory and or object passage gateway monitoring systems have relied upon reading one or a plurality of tags, each tag representing a unit or known quantity of units of associated objects.

[0005] It is not practical to apply an RFID tag to each object to be monitored, and/or it is desirable to monitor the presence of quantities of the objects to be monitored with greater precision than just a binary indication of an RFID tag associated with a quantity of objects each of which may or may not be tagged, as might be the case with items contained in a parts bin.

[0006] In prior RFID portal systems, RFID readers monitor RFID tags attached to objects and/or object loads during passage through, for example, a number of dock doors in a distribution center. Because of the nature of known RFID systems it is not always possible to determine whether particular tags are in a particular door. It is possible for example for a reader in door 2 to read a tag in door 1 or door 3. Without special door sensors, it is not possible to determine whether a particular door is open or closed and whether an object is moving out of the distribution center or into the distribution center.

[0007] Further, many items have packaging that is not complementary to being tagged, such as metal cans and/or plastic bottles, such as soft drinks. Also, it is not economical to tag individual items, such as nails, bolts, batteries, coins or the like. To optimize inventory operations, for example for Just In Time inventory systems, it is required to know the rate of depletion and/or when inventory of a specific item is about to be depleted, before actual depletion occurs.

[0008] Therefore, it is an object of the invention to provide an object monitoring solution that overcomes deficiencies in the prior art.

[0029] The inventor has recognized that the entirety of the monitored radio frequency environment, hereafter referred to as the RF environment fingerprint, may be used to derive changes of object presence and/or location(s), even where the individual objects within the target area are not each provided with their own individual RFID tag.

[0030] It is possible to monitor the presence and/or direction of movement of an RFID tag within a three dimensional target area via RFID monitoring systems. For example, International Patent Application serial number PCT/US08/58824, titled “Radio Frequency Signal Acquisition and Source Location System” filed Mar. 30, 2008 by Bloy et al, and International Patent Application serial number PCT/IB2008/053643, titled “Steerable Phase Array Antenna RFID Tag Locater and Tracking System”, filed Sep. 9, 2008 by Bloy, both applications commonly owned with the present application and hereby incorporated by reference in their entirety, describe systems of cooperating steerable phased array antennas performing beam scans of a target area, via an electronic beam steering circuit such as an array of phase shifters coupled to a corresponding array of antenna elements of a panel antenna, from which the presence and location of individual RFID tags is derived by logical processing of historical signal data obtained from prior scans of the target area.

[0031] To derive the location of the RFID tags, the RF signal intelligent tracking and control system (ITCS) of PCT/US08/58824 processes a plurality of signals received in response to a scanning narrow beam interrogation signal directed through the target area to derive which are true signals received directly from an individual RFID tag and which are pseudo emitter reflections of the same RFID tag response signal (carrying the same RFID tag identifier) reflecting off of secondary surfaces before finally reaching an antenna. The received signal information may be processed to reject the pseudo emitters, thus identifying the signal response to focus three-dimensional location routines upon, thereby identifying the RFID tag position in three dimensions. The identified RFID tag locations may be combined to generate a three dimensional picture of each of the RFID tag locations in the target area and, over time, the movement if any of the RFID tags.

[0032] Further, the ITCS may apply highly focused and/or varied interrogation signal power, for example to penetrate adjacent objects and/or packaging surrounding a specific RFID tag.

[0033] An ITCS includes an electronically steerable phase array antenna connected to a phase shift network and a radio frequency transmitter-receiver unit. For purposes of the present invention, the transmitter-receiver unit may or may not be specifically configured for identifying, tracking and locating RFID tags. The transmitter-receiver unit may contain circuit and/or logic capabilities to perform, in cooperation with an electronic steering circuit and antenna, functions such as:

• • [0034] transmission of a continuous wave signal
  • [0035] modification and/or modulation of the continuous wave signal
  • [0036] reception of a return signal
  • [0037] demodulation of the return signal
  • [0038] return signal signal strength measurement
  • [0039] time delay measurement between a transmitter signal and the return signal
  • [0040] phase measurement of the return signal compared with the phase of the transmitter signal; and
  • [0041] reporting the strength, relative phase and time delay of the returned signal.

[0042] The transmitter-receiver unit is under the control of and provides received signal information to a processing unit that may be local to or remote from the antenna and/or transmitter-receiver. The processing unit causes the transmitter-receiver to transmit a continuous wave signal and further causes the continuous wave signal to be modified/modulated as desired. The processing unit also commands the electronic steering circuit, either directly or via a communications link to a dedicated electronic steering sub-processor/controller, to direct the beam emitted from the steerable phased array antenna into different directions, for example directing the beam direction through a raster scan or the like of the target area. At each point or position of the beam an output report of the transmitter-receiver unit is passed to the processor containing, for example:

• • [0043] a scan direction and time code;
  • [0044] refractive frequency index of any RFID tag(s) detected;
  • [0045] relative phase and time delay;
  • [0046] Return Signal Strength Indicator (RSSI)—the coverage or lack of coverage of the RFID tag by objects will change the RSSI (a delta of signal absorption); and
  • [0047] multipath pseudo emitter shift (error)—the multiple pseudo reflections and their monitored location will change as the position and or coverage of the signal source by other objects changes.

[0048] The output report may be stored in a memory, register or table that may be temporary or permanent, hereafter referred to as the data matrix.

[0049] As disclosed in PCT/US08/58824, the ITCS is able to detect so called pseudo emitters and use multi-path ambiguity resolution algorithms to reduce or eliminate positional ambiguity these signals introduce when attempting to identify the locations of RFID tagged items in a target area. While the prior ITCS RFID tracking and location systems included extensive logic dedicated to the identification and rejection of these pseudo emitter signals, the present invention recognizes that these signals, especially when their appearance, location and/or disappearance is tracked over time as part of the overall RF environment fingerprint present in a target area, may be interpreted as an object presence and/or location indicator for untagged objects within the target area.

[0050] As shown in FIG. 1, an exemplary system according to the invention is a variation of an ITCS including a plurality of antenna(s) 7 coupled to a transceiver 2. The antenna(s) 7 providing a signal beam steerable via a separate or integrated beam controller directed by the transceiver 2 and/or processor 1. The processor unit 1 also includes a rules engine 3 operating in concert with the normal ITCS processor functions upon the present RF environment fingerprint and a collection of baseline configuration data, for example stored in a data matrix 5 of the processor unit 1. Alternatively, the ITCS may include multiple transceiver(s) 2, each dedicated to a separate or multiple antenna(s) 7. System commands are input and results are output via an operator interface 4

[0051] The rules engine 5 compares the real time RF environment fingerprint with a set of baseline calibration RF environment fingerprint data sets to identify pseudo emitter and/or response signal degradation indications of changes to the number and/or position of untagged objects present in the target area. Resolution of the location, pseudo emitter and/or response signal degradation indications is greatly increased when multiple antenna(s) 7 of the ITCS are arrayed about the target area 9, for example as shown in FIG. 2. At least two antenna(s) 7 enabling the application of triangulation logic between a beam vector of each antenna 7 to generate an accurate z-axis position parameter of signal sources useful alone or as a further accuracy check upon z-axis estimates derived from signal timing routines.

[0052] The rules engine 3 operates upon two primary indications of an untagged object(s) 11 presence in the target area 9, demonstrated in FIGS. 3-5. First, in a line of sight parameter 13, the response signal from a reference RFID tag 15, will become measurably degraded and/or cut off entirely as untagged object(s) 11 and/or structures 17 occur in and/or are placed in the signal path between the reference RFID tag 15 and an antenna 7, for example covering and/or obscuring the signal path to a reference RFID tag 15, thereby identifying the presence of an untagged object 11 along a signal path between the reference RFID tag 15 and an antenna 7. The reference RFID tag(s) 15 may be passive RFID tags permanently attached around the target area 9, for example on the floors, shelving and/or ceiling of a storeroom, warehouse or other space to be monitored. Second, in a pseudo emitter parameter 19, untagged objects and/or structures 17 including RF reflective surfaces (metal and/or metal coated surfaces) occurring and/or placed proximate a reference RFID tag 15 signal path will generate pseudo emitter responses with respect to that RFID tag 15 that may be located in three dimensions within the target area 9 as if they were RFID tags, themselves. Further, where separate antenna(s) 7 are arrayed about the target area 9, the different lines of sight between each antenna 7, various reference RFID tag(s) 15 and an untagged object 11 may generate both line of sight signal parameter 13 and pseudo emitter parameter 19 responses with respect to the same untagged object(s) 11, greatly improving the rules engine accuracy (for diagram clarity, representative signal paths/parameters are diagrammed with respect to only one of the antenna 7).

[0053] In an exemplary embodiment of the invention, the signals received from the ITCS in real time and/or as accessed from the ITCS data matrix 5 include the following signal parameters applied as inputs to signal processor(s) 11 integrated with and/or coupled to the processor unit 1:

• • [0054] The phase φ of a received signal at an instant in time t_i;
  • [0055] the amplitude of the received signal at the same instant in time t_i;
  • [0056] a fast Fourier transform (FFT) of the received signal envelope at the same instant in time t_i;; and
  • [0057] a time reference.

[0058] The resolution and accuracy of the time reference is preferably very high, for example derived from an atomic clock, GPS signal or the like. The signal processor(s) 11 output a time coded signature (sig) which is a unique signal signature for time t, generated by the signals received by the ITCS from each of the antenna(s) 7 along a known antenna signal beam orientation, stored as a mathematical representation, for example resulting from FFT processing of frequency and phase, as well as frequency and amplitude signal data. Where multiple antenna(s) 7 are applied, the target spatial coordinate where the antenna signal beams are directed to intersect may be applied to each signal record as a specific three-dimensional reference location of the signal signature obtained.

[0059] The signal signature feeds a function processor 23 which may be a separate component coupled with or an internal logical function of the processor unit 1. The function processor 23 derives and outputs a running average sub-signature f(sig), from a plurality of samples of the time coded instantaneous signature (sig) received by the function processor 23 over an observation period.

[0060] The observation period may be varied by the processing unit 1 to take account of changes in the environment or other user defined parameters. A plurality of sub signatures is further processed by the processing unit 1 to derive two further signatures, a primary signature and a reference signature for each spatial coordinate.

[0061] FIG. 6, demonstrates signal/data flow through a basic signal processor 21 and function processor 23 module. As shown in FIG. 7, multiple signal processor(s) 21 and function processor(s) 23 modules may be applied, for example each fed by signals from a different antenna 7. Signals generated by signal processor(s) 21 and function processor(s) 23 are mathematically summed by respective summing processor(s) 25 to produce a composite running average sub-signature f(sig) and instantaneous signature (sig). These two signatures represent a unique instantaneous signature (sig) and a running average sub-signature f(sig) for a known location within the target area 9. Thus, the data matrix 5 may be loaded with a signal signature representing the signal response for each of a plurality of unique spatial positions within the target area 9 of the ITCS. The system is explained herein with reference to a two antenna system. One skilled in the art will appreciate that the system result will improve as additional antenna's and associated signal processor and function processor modules are added to the ITCS.

[0062] As shown in FIG. 8, the processor unit 1 may be selected to have significant processing power for both mathematical and signal processing.

[0063] In its most simple form, the processor unit 1 receives two inputs from a single signature generator, average sub-signature f(sig) and (sig) for the X and Y (azimuth and elevation) coordinates of the point in space from which the signature was derived. This information is stored in the data matrix 5. The processor unit 1 causes the beam steered antenna(s) 7 to scan the environment in steps whose size depends on the beam illumination, preferably such that there is a slight overlap between steps. This can be best understood by picturing a flashlight being swept across a wall. The flashlight is moved a step at a time. At each step the light is momentarily switched on and a picture taken. The flashlight is moved and the light is again momentarily switched on such that the illuminated patch slightly overlaps the previous patch. This process is repeated in a raster scan until the entire wall has been examined. A series of measurements is made for each beam position and the average sub-signature f(sig) and (sig) for each spatial point is recorded in the data matrix 5 with the X,Y or where known, X,Y,Z position of the target point in space.

[0064] A recording cycle may be defined as one complete sweep of the target area 9. The processor unit 1 causes a plurality of recording cycles to be undertaken to build a running average for an average sub-signature f(sig) of each spatial location that may be further processed to derive an updated reference signature for each spatial location. The reference signature (Ref sig) is an average of many primary signatures and is used as a reference for determining whether change has occurred in the environment. Thus, the data matrix 5 may be updated with each sweep of the target area 9.

[0065] In addition, the reference signature (Ref sig) may also include a signature of the background Radio Frequency noise and or signals with the interrogator switched off, i.e. the background RF noise due to sources other than the primary emitter. This background signature is used by the processor to specifically exclude environmental changes due to Radio Frequency sources not generated by the system of this invention. These noise sources may include but are not exclusively due to transmissions or emitters such as TV or mobile radio base stations, Cellular base stations, low power and Part 15 devices operating in the same radio neighborhood, etc. as well as unintentional radiators such as computing and other non radio transmitting devices.

[0066] To improve signal quality, the background noise signature may be subtracted from any of the other signatures, such as the instantaneous signature, the average signature and the running reference signature, a procedure referred to as noise canceling.

[0067] For the purposes of further explanation reference will be made to a single spatial point. One skilled in the art will appreciate that similar processing may be applied to each spatial point and/or a series of partially overlapping points in the target area to derive a change analysis and thus inventory of the entire target area by extension of the single point processing disclosed hereafter.

[0068] The processor unit 1 uses three pieces of information to determine a change in the environment and measure the magnitude and nature of the change. The change analysis is then routed to a “rules” engine 3 which then takes action based on the nature of the change and pre-determined or preset user requirements.

[0069] The three pieces of information used are:

• • [0070] The instantaneous signature (sig) representing a single sample of a single point in the environment,
  • [0071] the average sub-signature f(sig) of the same point; and
  • [0072] the reference signature for the same point in space.

[0073] The processor unit 1, compares the instantaneous signature (sig), for example as represented graphically with respect to frequency and signal level in FIG. 9 and contemporaneously with respect to phase angle with respect to frequency in FIG. 10, with the values of the average sub-signature f(sig), for example as graphically shown in corresponding FIGS. 11 and 12, in a comparator processor 27 in order to determine whether the instantaneous signature (sig) is within preset limits. In other words the average sub-signature f(sig) is used to check the validity of the instantaneous signature (sig) to make sure that it was not a spurious read or a one-time aberration. As described herein above each of these signatures, also referred to as RF fingerprints, may also be processed to subtract the background environmental noise that may be present, for example as graphically represented by FIG. 13. The result of this check is a primary signature (sig prime) for that particular point in space.

[0074] The primary signature (sig prime) is then compared with the reference signature from the data matrix 5 in a further comparator processor 27. If there is a significant change, then the resulting nature of the change is routed to the rules engine 3, which triggers the appropriate action according to the established rules framework. If there is no significant change, then it is assumed that the environment has remained stable over the preceding period of time. The method using two antenna systems is the same. However because there are two steerable beam base stations, it is possible to more accurately determine a point in space using the azimuth and elevation positions from the two base stations to derive X, Y and Z positional information via triangulation. Further, dual (or more) signal beams intersecting at a desired location increase the interrogation signal intensity, for the desired point of interest, without radiating RF power beyond established limits, outside of the target area 9.

[0075] Baseline calibration RF environment fingerprint data is collected during system configuration to represent a range of possible target area 9 configurations, for example of the target area 9 entirely empty of untagged objects 11, and then in multiple instances as the target area 9 is successively filled with target objects.

[0076] FIGS. 14-19 graphically demonstrate a simplified set of representative signals from which RF environment fingerprint obtained from the same target area 9 as it is successively scanned either during initial calibration sweeps and/or during area monitoring, for example when initially empty of untagged object(s) 11 (FIGS. 14, 15), half of the untagged object(s) 11 removed (FIGS. 16, 17) and finally fully filled with the untagged object(s) 11 (FIGS. 18, 19). In the entirely empty state, pseudo emitter signal parameters 19 related to structure(s) 17 of the target area 9, identified by their longer interrogation to backscatter response signal reception time delay (resulting from a longer signal path via, for example, reflective surfaces of supports and or shelves) may be mapped for later exclusion from rules engine pseudo emitter parameter 19 signal location/processing. Once an “empty” RF environment fingerprint of a target area 9 has been mapped, further changes to the RF environment fingerprint may be recorded, associated with known events, such as the appearance and location of dynamic pseudo emitter signal responses that appear/disappear during calibrated re-stocking and/or depletion of specific tagged and/or untagged object(s) 11 within the target area. Multiple RF environment fingerprint(s) obtained during each calibration step and also during ongoing real time monitoring may be averaged to improve accuracy and/or discard erroneous data points.

[0077] Dynamic pseudo emitter signals may be further identified with the assistance of signal specific scans of the target area 9. For example, while an interrogation signal is focused upon a particular location known to include a reference RFID tag 15 or other response signal source, the entire target area 9 may then be scanned to identify the locations of any associated pseudo emitter parameter 19 signals that may have appeared or disappeared since the last RF environment fingerprint was scanned.

[0078] In addition to purely radio frequency inputs from within the target area 9, the qualified appearance and/or disappearance of specific tag identifications within the monitored RF environment may be used to dynamically and automatically determine actions to be taken by the RF environment monitor, such as altering the coverage pattern to sweep particular areas in a predetermined manner. For example, in one embodiment of the system, a known reference RFID tag 15 is attached to the lower edge of each of a plurality of dock roller-doors such that as the door is opened the reference RFID tag 15 rises from the floor to the top of the opening. The radio environment monitor observes and monitors the position of the door reference RFID tag 15 of each door as well as the location of individual tags on items or objects in the vicinity of each of the dock doors. The rules engine is configured to recognize these changes to the target area access and further apply an association between an individual dock door and the items that should be loaded through the dock door. If the dock door is opened when it should not be open or if the incorrect items are loaded through the dock door then an action may be triggered.

[0079] In an environment in which objects are stocked on shelves or movable promotional display modules, as might typically be the case in a grocery retail store for example, the baseline data set of the target area 9 RF fingerprint may include a scan of a target area 9 including a higher resolution focus on individual storage shelving on which a plurality of reference RFID tag(s) 15 are arrayed. The baseline data set identifies positions of each of the reference RFID tag(s) 15, as well as any static pseudo emitter parameter 19 signals generated by any RF reflective surfaces present in the target area when each of the shelves is empty of the untagged object(s) 11 to be monitored. As untagged object(s) 11 to be monitored are then successively added to the shelves, the signal response characteristics of each of the reference RFID tag(s) 15 attached to the shelves and/or wall(s) behind the shelves changes and/or new pseudo emitter parameter 19 signals are introduced into the RF environment fingerprint by RF reflective surfaces of the untagged object(s) 11. Also, the baseline data set may include changing response signal absorption characteristics, if any, which may occur as the untagged object(s) 11 to be monitored begin to cover and or obscure the reference RFID tag(s) 15 line of sight parameter 13 (the signal response with the shortest time delay between interrogation and backscatter signal response from an RFID tag, such as a reference RFID tag 15) and/or pseudo emitter parameter 19 signal paths with respect to one or more of the antenna(s) 7.

[0080] By monitoring changes in the RF environment fingerprint, event rules may be applied by the rules engine to generate user outputs and/or situation reports. For example, changes to stock presence and/or quantity in a specific bin may be estimated; the bin presence and/or object quantity therein may be tracked, for example generating reports of objects for which the inventory is running low. Thus inventory monitoring or the like may be performed via RFID, without requiring an individual tag on each object.

[0081] Large groups of tagged items may be similarly monitored by observing the number and location of signal responses received from a large concentration of RFID tagged objects, even if a significant number of the RFID tags are obscuring/covering one another and preventing them from being detected. Further any RFID tags affixed to objects within the target area 9, may be treated for purposes of the RF environment fingerprint as reference RFID tag(s) 15, with any associated signal degradation and or pseudo emitter data applied only as long as these RFID tags are monitored to be in the position associated with the particular RF environment fingerprint.

[0082] This baseline data set may be updated from time to time to compensate for changes in the environment. The system may be configured to automatically and dynamically update the baseline data set as small changes occur.

[0083] By monitoring the environment and comparing large changes to the reference RF environment fingerprint of the stored baseline data set, the system may compute that something in the target area 9 has changed and make a decision based on this change using the rules engine 3. Where multiple factors within the rules engine 3 indicate an objects presence, such as signal degradation and/or dynamic pseudo emitter parameter 19 signal detection from multiple paths, reference RFID tag(s) 15 and/or antenna(s) 7, an increased probability factor may be assigned to the resulting rules engine 3 conclusion. Rules engine 3 conclusions may then be assessed depending upon their assigned probability factor totals and only those meeting a threshold probability factor higher than a preset level output for pre-assigned action(s).

[0084] During the calibration procedure, specific shelf locations may be allocated to specific objects to enable monitoring and reporting upon the presence of multiple separate object classes, corresponding for example to different consumer products in a grocery store environment and or multi-item distribution warehouse. If the system is further coupled to the electronic stock-keeping computer system, the change in the RF environment fingerprint may be associated with items removed from the shelves and the rules engine 3 may then, for example, trigger a purchase order to be raised.

[0085] One skilled in the art will appreciate an advantage of the invention is that objects may be tracked and located without a need to individually tag them. By tagging the environment, for example the shelves and or object bins instead of each individual object, and recording how the RF environment fingerprint, including pseudo emitter parameter 19 signals associated with desired objects, then changes as the target area 9 is filled and/or emptied of objects, for example by comparison with a baseline data set, it is then possible to monitor the movement or change of position of untagged objects in the target area 9 with significant precision.

[0086]

Table of Parts
1	processor unit
2	transceiver
3	rules engine
4	operator interface
5	data matrix
7	antenna
9	target area
11	untagged object
13	line of sight parameter
15	reference RFID tag
17	structure
19	pseudo emitter parameter
21	signal processor
23	function processor
25	summing processor
27	comparator processor

[0087] Where in the foregoing description reference has been made to ratios, integers, components or modules having known equivalents then such equivalents are herein incorporated as if individually set forth.

[0088] While the present invention has been illustrated by the description of the embodiments thereof, and while the embodiments have been described in considerable detail, it is not the intention of the applicant to restrict or in any way limit the scope of the appended claims to such detail. Additional advantages and modifications will readily appear to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details, representative apparatus, methods, and illustrative examples shown and described. Accordingly, departures may be made from such details without departure from the spirit or scope of applicant's general inventive concept. Further, it is to be appreciated that improvements and/or modifications may be made thereto without departing from the scope or spirit of the present invention as defined by the following claims.