Method and system for location accuracy analysis

[0001] 1. Field of the Invention

[0002] The present invention relates to telecommunications and, more particularly, to methods and systems for analyzing the accuracy of a wireless location technology used in a wireless telecommunications network.

[0003] 2. Description of Related Art

[0004] The Federal Communications Commission (FCC) has adopted rules to implement enhanced 911 (E911) services for wireless calls. In Phase I of the E911 requirements, wireless carriers are required to provide the Public Safety Answering Point (PSAP) with the telephone number of the originator of a 911 wireless call and the cell site or base station serving the call originator. In Phase II of the E911 requirements, wireless carriers are also required to provide the PSAP with an Automatic Location Identification (ALI) of the 911 wireless call originator. For handset-based wireless location technologies, the FCC standards require the ALI to have a location accuracy of 50 meters for 67 percent of the calls and 150 meters for 95 percent of the calls.

[0005] The FCC has provided non-mandatory guidelines and suggestions for determining whether wireless location systems comply with the FCC's accuracy standards in its “Guidelines for Testing and Verifying the Accuracy of Wireless E911 Locations Systems,” OET Bulletin No. 71 (Apr. 12, 2000). The FCC's Guidelines suggest an empirical approach for verifying compliance with its accuracy standards. Specifically, the FCC's Guidelines suggest performing tests at various sample locations to determine the distance between the actual location and the location reported by the ALI system. The FCC's Guidelines also provide a table indicating, for various sample sizes, how many location measurements need to have location errors less than 50 meters and how many need to have location errors less than 150 meters in order to demonstrate compliance at the 90% confidence level.

[0006] The statistical approach given in the FCC's Guidelines has several drawbacks. First, it is rather cumbersome. Second, the approach does not provide a measure of how accurate the wireless location system is. The latter point is significant in that wireless location services are of interest not only because of the FCC's mandates but also because other entities may be interested in obtaining location information, e.g., to provide location-based services to wireless customers. Thus, there is a need to efficiently analyze the accuracy of wireless location technologies with regard to the requirements of potential consumers of location information, as well as for compliance with FCC's E911 requirements.

[0007] In a first principal aspect, an exemplary embodiment of the present invention provides a method of analyzing the accuracy of a wireless location technology used in a wireless telecommunications network. In accordance with the method, a test site that has a known location is selected. A wireless location technology is used to determine a measured location for at least one mobile station located at the test site. A location error between the measured location and the known location is calculated. The measurement and calculation steps are repeated for the at least one mobile station to obtain n measured locations and n calculated location errors. An average location error is calculated from the n calculated location errors, wherein n is sufficient to calculate the average location error to within a predetermined uncertainty at a predetermined confidence level. The average location error for that test site is stored in at least one network element of the wireless telecommunications network.

[0008] In a second principal aspect, an exemplary embodiment of the present invention provides a system for analyzing the accuracy of a wireless location technology used in a wireless telecommunications network. The system comprises a position determining entity (PDE) and a data analysis system communicatively coupled to the PDE. The PDE provides a plurality of location measurements made by the wireless location technology for at least one mobile station located at a test site having a known location. The data analysis system calculates an average location error from the plurality of location measurements and the known location of the test site.

[0011] The present invention, in exemplary embodiments, uses an average location error, such as a mean radial location error, calculated for a given test site to characterize the accuracy of a wireless location technology. The average location error may be determined to within a desired uncertainty at a desired confidence level in accordance with the exemplary methods described below. The average location errors may be reported to emergency services networks and/or other consumers of location information. Moreover, the implementation of the wireless location technology may be adjusted in order to minimize the average location errors.

1. Exemplary Architecture

[0012] The wireless location technology is used in a wireless telecommunications network that provides wireless communications to mobile station located in wireless coverage areas served by the network. The wireless location technology may be mobile-based, network-based, or it may use a combination of mobile station and network resources to determine location. For example, the wireless location technology may use the Global Positioning System (GPS) and may be network-assisted. In the network-assisted approach, the mobile station includes a GPS receiver but also communicates with the network to determine its position. A position determining entity (PDE) in the wireless telecommunications network may then calculate the mobile station's location in terms of geographic coordinates, such as latitude and longitude. As described in more detail below, a data analysis system, which may include a computer, may receive the measured locations from the PDE and calculate the location errors.

[0013] Referring to the drawings, FIG. 1 is a simplified block diagram of an exemplary wireless telecommunications network 10 in which exemplary embodiments of the present invention may be employed. Wireless telecommunications network 10 includes a base transceiver station (BTS) 12 that provides a wireless coverage area within which BTS 12 may communicate with one or more mobile stations, such as mobile station 14, over an air interface. Mobile station 14 may be a wireless telephone, a wirelessly-equipped personal digital assistant (PDA), or other wireless communication device. The communications between BTS 12 and mobile station 14 may occur in a digital format, such as CDMA, TDMA, GSM, and/or 802.1 x, and/or an analog format, such as AMPS.

[0014] BTS 12 may be controlled by a base station controller (BSC) 16, which, in turn, may be controlled by a mobile switching center (MSC) 18. MSC 18 may be connected to a network, such as the public switched telephone network (PSTN) 20. and may use an out-of-band signaling system, such as Signaling System 7 (SS7) to route calls through PSTN 20. MSC 18 is also able to signal to a home location register (HLR) 22 and to a service control point (SCP) 24. This signaling may occur via one or more signal transfer points (STPs), such as STP 26. The signaling between MSC 18 and HLR 22 may conform to IS-41 specifications. A recent revision incorporated herein by reference. The signaling between MSC 18 and SCP 24 may conform to the specification “Wireless Intelligent Network,” TIA/EIA/IS-771, published in July 1999, which is incorporated herein by reference. Other signaling protocols could be used, however. In this way, MSC 18, BSC 16, and BTS 12 may connect incoming calls from PSTN 20, which calls may originate from calling parties using landline telephones, mobile stations, or other communication devices, to mobile station 14. Similarly, MSC 18, BSC 16, and BTS 12 may connect calls originating from mobile station 14 to their destinations, via PSTN 20.

[0015] Wireless telecommunications network 10 uses a wireless location technology to obtain the location of mobile stations, such as mobile station 14. The specific manner in which network 10 obtains the locations of mobile stations, and the specific network elements that network 10 uses for location determination, will, in general, depend on the wireless location technology that is used. In an exemplary embodiment, network 10 may use a position determining entity (PDE) 30 and a mobile position center (MPC) 32 in connection with location determinations. For example, PDE 30 may determine the location (such as in terms of latitude and longitude) of a mobile station, such as mobile station 14, based at least in part on information provided by mobile station 14 during a location determination session. MPC 32 may store the locations determined by PDE 30. MPC 32 may also control aspects of how network 10 obtains, uses, and provides location information. PDE 30 and MPC 32 may communicate with other elements of network 10 via STP 26, a packet-switched network, and/or some other type of communication link.

[0016] In an exemplary embodiment, wireless assisted GPS is used to determine the location of mobile station 14. In the wireless assisted GPS approach, mobile station 14 includes a Global Positioning System (GPS) receiver but does not determine its location on its own. Instead, PDE 30, determines the location of mobile station 14 in terms of latitude and longitude, based at least in part on information provided by mobile station 14. For example, mobile station 14 may first send an indication of its approximate location, such as the cell, sector, or other wireless coverage area it is operating in, to PDE 30. Using this approximate location, PDE 30 determines what GPS satellites are in view and their Doppler offsets and then sends this information to mobile station 14 as “assistance data.” Mobile station 14 then uses this assistance data to acquire GPS signals from the GPS satellites and, thereby, obtains information, such as pseudoranges, from which its location may be calculated. Mobile station 14 sends the pseudoranges and/or other location-related information to PDE 30. PDE 30 then calculates the location of mobile station 14, in terms of latitude and longitude, based on the information from mobile station 14 and, optionally, various corrections.

[0017] Network 10 may, however, use other wireless location technologies for determining the location of mobile stations, such as mobile station 14. For example, mobile station 14 may use its GPS receiver to determine its location without network assistance. In that case, mobile station 14 may simply send to PDE 30 or other network element its calculated location, e.g., its latitude and longitude, during the location determination session.

[0018] Alternatively, network 10 may use wireless location technologies that do not rely on GPS. For example network 10 may use Advanced Forward Link Triangulation (AFLT) to locate mobile stations. In the AFLT approach, a mobile station's location is determined (such as by PDE 30) from the round-trip delays associated with signals between the mobile station and multiple BTSs. In some cases, network 10 may be able to use more than one type of wireless location technology to determine mobile station location. For example, in some or all of the areas served by network 10, network 10 may have the ability to use wireless assisted GPS, AFLT, or both to determine mobile station locations.

[0019] The communications between mobile station 14 and wireless telecommunications network 10 may conform to the specification “Position Determination Service Standard for Dual-Mode Spread Spectrum Systems,” TIA/EIA/IS-801, dated Oct. 15, 1999, which is incorporated herein by reference. Other protocols may be used, however.

[0020] Network 10 may also include one or more databases that store parameters specifying aspects of how to carry out location determinations. Such parameters may specify, for example, the power levels, forward link and reverse link calibrations, sector center latitude and longitudes, maximum antenna ranges, antenna latitude and longitudes, and orientation accuracies to use in location determinations. In this regard, FIG. 1 shows RF database 34 as representative of such databases in network 10. RF database 34 may be communicatively coupled to BSC 16, as shown in FIG. 1. Alternatively, RF database 34 may be a part of BSC 16, or RF database 34 may be a part of, or accessible by, other network elements, such as PDE 30 and/or MPC 32.

[0021] Wireless telecommunications network 10 may also make the locations stored in MPC 32 available to one or more location information consumers 36. Such location information consumers may include providers of emergency services. For example, MPC 32 may report the location of a mobile station engaged in an E911 call to a PSAP or other element of an emergency services network. Location information consumers 36 could also include other entities, which may be commercial or non-commercial, that make use of location information. Such location information consumers may, for example, monitor the locations of mobile stations and/or provide location-based services to mobile stations or to others.

[0022] Network 10 may also include a data analysis system 38 for location accuracy analysis. Data analysis system 38 may obtain the mobile stations locations determined by PDE 30, compare the locations to “ground truth,” and calculate the location errors, as described in more detail below. Data analysis system 38 may include a computer that is programmed to perform these calculations automatically. Thus, data analysis system 38 may include one or more processors, data storage, and programming in the form of machine language instructions stored in the data storage. Data analysis system 38 may be a separate network element of network 10. Alternatively, data analysis system 38 may be integrated with a network element, such as PDE 30.

2. Exemplary Operation

[0023] To analyze the accuracy of the wireless location technology, a plurality of test sites may be chosen in areas served by the wireless telecommunications network. The test sites may be located in the same or in different cells or sectors. Preferably, the test sites correspond to likely spots where mobile stations may originate calls that trigger E911 or other location-based services. The locations of the test sites are determined to a high degree of accuracy, i.e., their “ground truth” locations are determined, such as by using surveys. In this regard, surveys are often able to determine locations to within 1 cm. The “ground truth” locations, which may be expressed in terms of latitude and longitude, may be stored in data analysis system 38.

[0024] One or more mobile stations are then located at the test sites, and the wireless location technology is used to measure the locations of the mobile station(s). Data analysis system 38 receives the measured locations, e.g., from PDE 30, and calculates the location error for each measurement. As described in more detail below, the location errors may be radial location errors, i.e., expressed as radial distances from the “ground truth” locations of the test sites. Data analysis system 38 may also obtain other information from PDE 30 or other elements of network 10, such as the mobile identification numbers (MINs) or other identifiers of the mobile station(s) or types of mobile station(s) used in the testing.

[0025] Data analysis system 38 may calculate the radial location error as follows. First, the differences between the measured and “ground truth” latitudes and longitudes are calculated as set forth below:
Δ_long=A_test−A_truth
Δ_lat=B_test−B_truth
In this expression, A_test and A_truth are the measured and “ground truth” longitude values, respectively, and B_test and B_truth are the measured and “ground truth” latitude values, respectively. The location errors in “x” and “y” directions, expressed as distances, may then be calculated as follows:
Δx=K_longΔ_long
Δy=K_latΔ_lat
In this expression, K_long and K_lat are correction factors to convert longitude and latitude differences, respectively, to distances. If the latitudes are expressed in degrees and the location errors are to be expressed in meters, then K_lat=1852×60 may be used, based on the conversion factor that one minute of arc is nominally equal to 1852 meters. Alternatively, a correction factor that varies with latitude may be used in order to take into account the earth's deviation from a perfect sphere. The corresponding correction factor for longitude may take into account the fact that the distance between longitudinal lines varies with latitude, even for a perfect sphere. Thus, K_long=1852×60×cos(A_test) may be used. Data analysis system 38 may calculate the radial location error, r, for a given measurement as follows: $r=\sqrt{{\left(\Delta x\right)}^2+{\left(\Delta y\right)}^2}$

[0026] Data analysis system 38 obtains a total of n location measurements for a given test site. The n location measurements may all be of the same mobile station. Alternatively, the n location measurements may be of several different mobile stations at the test site. Data analysis system 38 may calculate n radial location errors, r_l, from the n location measurements, in the manner described above. Data analysis system 38 may also calculate an average radial location error from the n radial location errors. The average radial location error may be a mean radial location error, calculated as follows: ${\mu }_{\mathrm{calc}}=\left(1/n\right)\Sigma r_l$
However, other types of averages may be used. For example, data analysis system 38 may calculate a root-mean-square radial location error, as follows: ${\rho }_{\mathrm{calc}}=\sqrt{\left(1/n\right)\Sigma r_l^2}$

[0027] It is to be understood that the calculated average location errors may themselves be random variables that may differ from the true average location errors. Thus, the calculated mean radial location error, μ_calc, may estimate the true value, μ, to within an uncertainty δ (dependent on the statistical confidence level), as summarized by the following expression:
μ=μ_calc±δ
Of course, δ increases as the desired statistical confidence level increases. In addition, for a given confidence level, δ decreases as more location measurements are taken, i.e., as n increases.

[0028] Thus, the number, n, of location measurements used to calculate μ_calc, may be chosen so as to be able to calculate μ to within a predetermined uncertainty at a predetermined confidence level, such as the 95% confidence level. The number, n, of location measurements required to achieve this level of accuracy may be estimated based on the following model.

[0029] In this model, the location errors in both the x and y directions are assumed to be random variables that have a normal, i.e., Gaussian distribution, with an expected value of zero and a standard deviation of σ. Based on this model, it can be shown that the radial location error, r, is a random variable with a Rayleigh distribution given by the following expression:
f(r)=(r/σ²)exp(−r²/2σ²)

[0030] The mean, μ, for this distribution, i.e., the true mean radial location error, is given by the following expression: $\mu =\sigma \sqrt{\left(\pi /2\right)}$

[0031] Given this relation between μ, the true mean radial location error, and σ, the standard deviation in the underlying Gaussian distribution of the location errors in the x and y directions, confidence intervals for μ may be estimated from the confidence intervals for σ. In particular, at the 95% confidence level, σ may be estimated to lie within the following interval: $s\sqrt{\left(n/{\chi }_{975,v}^2\right)}\le \sigma \le s\sqrt{\left(n/{\chi }_{025,v}^2\right)}$
where X_975,ν² and X_025,ν² are the 97.5 and 2.5 percentile values, respectively, for the chi-square distribution with ν=n−1 degrees of freedom. The quantity, s, is the standard deviation applicable to the location errors in either the x or y directions. Thus, s may be taken to be $\sqrt{\left(1/n\right){\Sigma \left(\Delta x_i\right)}^2},$
with Δx, representing the n measured location errors in the x direction, or s may be taken to be $\sqrt{\left(1/n\right){\Sigma \left(\Delta y_l\right)}^2},$
with Δy_i representing the n measured location errors in the y direction. Alternatively, since the measured radial location errors are related to the measured location errors in the x and y directions by r_l²=(Δx_l)²+(Δy_l)², s may be taken to be $\sqrt{\left(1/2n\right)\Sigma r_l^2},$
with r_l representing the n measured radial location errors. Alternatively, other ways of estimating s could be used.

[0032] Thus, given a desired uncertainty, δ, and using the 95% confidence level, the required number of location measurements, n, may be found using the following expression: $s\sqrt{\left(n/{\chi }_{\mathrm{.025},v}^2\right)}-s\sqrt{\left(n/{\chi }_{\mathrm{.975},v}^2\right)}\le 2\delta \sqrt{\left(\pi /2\right)}$
However, for sufficiently large n: ${\chi }_{975}^2\approx \left(1/2\right){\left(\sqrt{2n}+1.96\right)}^2\mathrm{and}{\chi }_{025}^2\approx \left(1/2\right){\left(\sqrt{2n}-1.96\right)}^2$
Using these approximations, the above expression to find n may be simplified to the following: $n\ge {\left(\pi /2\right)\left[\left(1.96/\sqrt{2}\right)\left(s/\delta \right)\right]}^2\approx 3{\left(s/\delta \right)}^2$

[0033] Thus, n may be chosen to achieve a particular level of uncertainty expressed as a distance, i.e., δ. Alternatively, n may be chosen to achieve a particular level of uncertainty expressed as a ratio, e.g., s/δ. Other ratios could be used, however, and the level of uncertainty could be characterized in other ways. Moreover, other ways of estimating n could be used. Finally, other statistical models, such as models that do not rely on assuming Gaussian or Rayleigh distributions, could be used.

[0034] In this way, data analysis system 38 may calculate the average location errors (e.g., the mean radial location errors) at a plurality of test sites to within a predetermined uncertainty (whether characterized as a distance, as a ratio, or in some other manner) at a predetermined confidence level, such as the 95% confidence level. Data analysis system 38 may also store the calculated average location errors for each test site in its own data storage and/or in one or more network elements, such as PDE 30, MPC 32, and/or RF database 34. Moreover, data analysis system 38 may store each average location error in a data record that includes other information, such as the location of the test site, the type of mobile station(s) used for the location measurements, and/or other information. In this way, wireless telecommunications network 10 may maintain information that characterizes how accurate location determinations typically are for different areas served by network 10.

[0035] FIG. 2 is a flow chart illustrating, in simplified form, an exemplary location accuracy analysis process. The process may begin by choosing a test site in an area served by wireless telecommunications network 10, as indicated by step 100. For example, the test site could be located in the wireless coverage area of BTS 12. As indicated by step 102, the “ground truth” latitude and longitude is then obtained for the test site, such as by using surveys. Data analysis system 38 may store the “ground truth” latitude and longitude. Network 10 uses a wireless location technology to obtain a plurality of latitude and longitude measurements for a mobile station, such as mobile station 14, located at the test site, as indicated by step 104. As indicated by step 106, the radial location errors are then calculated, such as by data analysis system 38, based on the latitude and longitude measurements from step 104 and from the “ground truth” latitude and longitude. The radial location errors may be expressed as distances, as described above. The mean radial location error is then calculated, such as by data analysis system 38, as indicated by step 108.

[0036] Data analysis system 38 may then determine whether the uncertainty, at the 95% or other desired confidence level, in the calculated mean radial location error is at or below the desired level of uncertainty, as indicated by step 110. If the desired level of uncertainty at the desired confidence level has been achieved, then data analysis system 38 may store a data record that indicates the mean radial location error. The data record may include other information as well, such as the location of the test site used to determine the mean radial location error, an identification of the mobile station(s), such as by mobile identification numbers (MINs), or the type of mobile station(s) used for the location measurements, an identification of the wireless location technology location technology or parameters of the wireless location technology used for the location measurements, the dates and times of the location measurements, and/or other circumstances of the location measurements. If, on the other hand, the desired uncertainty at the desired confidence level has not been achieved, then network 10 may use the wireless location technology to obtain additional latitude and longitude measurements for the mobile station located at the test site.

[0037] The average location errors (e.g., mean radial location errors) stored in network 10, as described above, may be used in various ways. For example, network 10 may report average location errors to providers of emergency services. When a mobile station, such as mobile station 14, makes an E911 call, network 14 uses a wireless location technology to determine the location of mobile station 14. Network 10 may also retrieve the average location error for that wireless location technology that was previously determined for a nearby test site, e.g., a test site in the same cell or sector as mobile station 14. Then, when network 10 reports the location of mobile station 14 to the emergency services network, network 10 may also report the average location error. In this way, the PSAP or other emergency service provider, may be made aware of how accurate the reported location is likely to be. Network 10 may similarly report average location errors when it reports mobile station locations to other location information consumers.

[0038] The average location errors may also be used to increase the accuracy of the wireless location technology used in network 10. In particular, the wireless location technology may have some flexibility in how it is used by network 10. For example, RF database 34 may store one or more adjustable parameters specifying how to make location determinations for a given wireless coverage area, e.g., a cell or sector. Thus, the wireless location technology in network 10 may be “tuned,” by adjusting one or more of these adjustable parameters, in order to minimize the average location errors calculated for one or more test sites in that wireless coverage area.

[0039] In addition, whenever a mobile station's location is to be determined, network 10 may choose, in real time, the wireless location parameters that result in the smallest radial location error for that type of mobile station in that location at that time of day, based on the mean radial location errors stored by network 10. Similarly, if network 10 is able to use different types of wireless location technologies, such as wireless assisted GPS and AFLT, then network 10 may choose the wireless location technology with the lowest average location error to locate a mobile station in a given wireless coverage area.

[0040] Wireless telecommunications network 10 may also use the average location errors in other ways. For example, network 10 may do position averaging of a mobile station, based on the mobile station's rate of mobility and the average location error.

3. Conclusion

[0041] Exemplary embodiments of the present invention have been described above. Those skilled in the art will understand, however, that changes and modifications may be made to these embodiments without departing from the true scope and spirit of the invention, which is defined by the claims.