METHOD AND SYSTEM FOR ALIGNMENT OF SENSORS IN A SIMILAR ENVIRONMENT

[0001] The present invention relates in general to a method for alignment of sensors comprised in communication devices in similar environment, due to signal latency/delay and relative movement of the devices. The invention also relates to a system that comprises at least two sensors comprised in communication devices within a similar environment.

[0002] A communication device configured to be used in a wireless communication network may often include at least one sensor and a processor communicatively coupled to the sensor.

[0003] As inexpensive and small sensors, such as accelerometers, gyros and magnetometers, are widely available, they are incorporated into many modern communication devices, such as mobile phones and mobile phone accessories. The sensors may be used to determine the orientation of the device, in relation to earth, for portrait/landscape display switching, map alignment, pedometry, navigation etc.

[0004] While good orientation accuracy is important for most applications, achieving high accuracy with low cost sensors is also challenging. This is where sensor calibration often helps. Usually this process is lengthy and generally involves complicated steps for the user and is time consuming.

[0005] Some mobile devices are being calibrated in factory in specific fixtures, others by the users placing the device in the specific orientations, while other devices undergo continuous or occasional “on the fly” calibration during normal orientation.

[0006] Since mobile phones and the accessories typically uses inexpensive sensors and as a result the calibration is likely to drift in time. It would therefore be desirable to provide techniques that can compensate for this drift in time of the sensors continuously and with little if any user input.

[0007] With the above description in mind, then, an aspect of the present invention is to provide a way to improve alignment of sensors within similar environments, which seeks to mitigate, alleviate or eliminate one or more of the above-identified deficiencies in the art and disadvantages singly or in any combination.

[0008] As will be described in more details by the aspects of the invention below, one way to improve the alignment of the sensors is to continuously determine the time difference between sensor data from sensors and the relative orientations between the sensors in similar environment.

[0009] An aspect of the present invention relates to a method for alignment of sensor units comprised in at least two independent communication devices operating in the same environment, said method comprising the steps of receiving a first signal representing sensor data collected by a first sensor unit comprised in a first communication device, receiving a second signal representing sensor data collected by a second sensor unit comprised in a second communication device, calculating a correlation function between said first signal and said second signal, defining the time difference between first and second signals by finding a maximum in said correlation function while time shifting the first and the second signals and calibrate the first and second sensor units to compensate for the defined time difference.

[0010] The method may further comprises the steps of finding a first movement, based on the correlation function, and defining a first reference axis in the reference co-ordinate system in each device as the direction of the first found movement, finding a second movement, based on the correlation, which are different from the first found movement, for defining a second reference axis in the reference co-ordinate system in each device as the direction of the second found movement and determining the relative rotation between the at least two devices by combining the defined first and second reference axis within the reference co-ordinate system of each devices.

[0011] The method may further comprises the steps of repeatedly finding movements correlated between the at least two sensor units, repeatedly combining the found movement with the previously found movement between the at least two sensor units for increasing accuracy in the defined time delay and in the determined relative orientation. The second reference axis may be determined as a projection of the second movement perpendicular to the first reference axis.

[0012] The movement may be a linear acceleration or a rotational acceleration.

[0013] The direction of the first reference axis may be along the linear acceleration, opposite the linear acceleration, along a clockwise rotation axis of the rotational acceleration or along a counterclockwise rotation axis of the rotational acceleration.

[0014] The second reference axis may be the reference rotation in a plane perpendicular towards the first reference axis.

[0015] The step of defining a time delay may further comprising determining a first reception time of the first signal, determining a second reception time of the second signal and defining the time delay as the difference between the first and the second reception time.

[0016] The sensor units may be any of accelerometer, gyro, magnetometer or environmental sensor.

[0017] Another aspect of the present invention relates to a system alignment of sensor units comprised in at least two independent communication devices operating in the same environment, said system further comprises a unit for receiving and processing receiving a first signal representing sensor data collected by a first sensor unit comprised in a first communication device and receiving a second signal representing sensor data collected by a second sensor unit comprised in a second communication device, means for calculating a correlation function between said first signal and said second signal, means for defining the time delay between first and second signals by finding a maximum in said correlation function while time shifting the first and the second signals and means for calibrating the first and second sensor units to compensate for said time difference.

[0018] The system may further comprise means for finding a first movement, based on the correlation function, and defining a first reference axis in the reference co-ordinate system in each device as the direction of the first found movement means for finding a second movement, based on the correlation, which are different from the first found movement, for defining a second reference axis in the reference co-ordinate system in each device as the direction of the second found movement and means for determining the relative rotation between the at least two devices by combining the defined first and second reference axis within the reference co-ordinate system of each devices.

[0019] The unit for receiving and processing sensor data may be comprised in one of the at least two independent communication units.

[0020] The sensor units may be any of accelerometer, gyro, magnetometer or environmental sensor.

[0021] One of the at least two independent communication units may be a mobile phone.

[0022] The features of the above-mentioned aspects may be combined in any way possible to form variants of the embodiments of the present invention.

[0028] Embodiments of the present invention relate, in general, to the field of electronic communication devices. A preferred embodiment relates to a portable communication device, such as a mobile phone, including one or more accessories, e.g. a headset, microphone, camera, arm wrist sensor etc. However, for the sake of clarity and simplicity, most embodiments outlined in this specification are related to a mobile phone.

[0029] Embodiments of the present invention will be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like reference signs refer to like elements throughout.

[0030] The present disclosure is directed to a method and system that uses signals recorded by different sensors to improve the possibility to align signals received within a mobile communication system, where some of the sensors are located on different devices, by using correlation techniques. For example, one device may be a mobile phone and the other device may be a wireless/wired devices that communicates with the mobile phone, such as a headset.

[0031] Sensor data from accessories may be sent to the mobile phone for processing in the phone over serial links, such as Bluetooth and/or USB. Sensor data from an accessory are usually combined with sensor data from other accessories or the mobile phone to get additional information. By the nature of the serial communication between the mobile phone and the accessory, there will be an undetermined time delay, latency, between the different data sets, this delay often needs to be determined and accounted for.

[0032] According to one aspect of the invention it may be useful to know the relative orientation between the sensors, i.e. a sensor included in a headset and a sensor included in the mobile phone to be able to determine in what direction the wearer is looking.

[0033] In an exemplary embodiment of the invention, as shown in FIG. 1, a system, 10, includes at least two mobile communication devices 11, 12, for example a mobile phone and a headset, and a signal processing unit, a CPU, 13.

[0034] The devices 11, 12 are equipped with at least one motion sensor unit, not shown, and preferably the same kind of sensors. The sensor units may include accelerometers or other motion sensing devices, such as gyros and magnetometers, or any environmental sensor e.g. 50/60-Hz signal from surrounding electricity, light sensors, etc. for monitoring the activity or movement of the devices 11, 12. A first sensor unit, may be located in a mobile phone, and a second sensor unit, may be located in a headset.

[0035] The signal processing unit 13, is configured to receive and process sensor data signals 14, 15 collected from each of the sensor units within the system, 10. The signal processing unit 13 may be located in one of the communication devices.

[0036] In the system, as shown in FIG. 2, sensor data signals 14, 15 from the sensor unit may be transferred over i.e. Bluetooth to the signal processing unit 13 in which the sensor data received from the sensor unit in the first device, i.e. headset, is combined with sensor data from the sensor unit in the second device, i.e. mobile phone. It is anticipated that other protocols may be used over the wireless link.

[0037] Sensor data may be an electrical signal representing i.e. the sound output signal from a microphone, a light output signal from a camera or a movement signal from a movement sensor, i.e. an accelerometer.

[0038] Sensor data from the sensor units will experience time delays due to e.g. the nature of the serial connection and will vary over time due to circumstances such as Bluetooth traffic intensity. A delay compensation block is required before the at least two sensor data signals, 14, 15 can be processed.

[0039] If the first sensor unit and the second sensor unit are located in similar environments, i.e. if the sensor output of the first sensor unit and the second sensor unit are similar, but not necessarily identical, the similarity of the sensor data may be used to calculate the time delay, ΔT, between reception time, Δt₁, of the sensor data from the first sensor unit and reception time, Δt₂, of the sensor data from the second sensor unit. One example of similar environment is when the user wearing a mobile phone and a headset is travelling by car, bus or other transportation vehicles. The time delay, ΔT, will be calculated according to formula 1.

ΔT=Δt₂−Δt₁  Formula 1

[0040] ΔT can be determined in a processing unit, CPU, by continuously evaluating and finding the maximum for the normalized time shifted correlation functions between the signals, sensor data, from the first sensor unit and the second sensor unit, according to formula 2.

$\begin{array}{cc}\left(f_1*f_2\right)\left[\Delta T\right]=\frac{1}{n-1}\sum _{m=0}^n\frac{\left(f_1^{*}\left[m\right]-\stackrel{\_}{f_1}\right)\left(f_2\left[\Delta T+m\right]-\stackrel{\_}{f_2}\right)}{{\sigma }_{f_1}{\sigma }_{f_2}}& \mathrm{Formula}2\end{array}$

[0041] In which f₁ is the function of the sensor data of the first sensor unit and f₂ is the function of sensor data of the second sensor unit, n is the number of points in the data series, f₁ and f₂ are the mean values of the sensor data from the first and second sensor unit, respectively, σ_f1 and σ_f2 are the standard deviations of the sensor data from the first and second sensor unit, respectively.

[0042] If the sensor data is multidimensional, the dimensionality could for example be decreased to one by using the magnitude of the sensor data. This may be used when calculating the delay, ΔT, in which case it is not necessary to compare more than one dimension.

[0043] By continuously calculating the delay, ΔT, sensor data can at any time be combined in an ideal way and time delays be accounted for.

[0044] In a first example, sensor data 14 from an movement sensor in a headset 11 and sensor data 15 from a movement sensor in a mobile phone 12, as shown in FIG. 2, are used. The sensor data could look something like the two artificial series 21, 22 of accelerometer sensor data, as shown in FIG. 3, where sensor data graph 21 may be a sensor data signal from the mobile phone 12 and sensor data graph 22 may be a sensor data signal from the headset 11.

[0045] The units on the y-axis show the arbitrary units and the unit on the x-axis shows the number of samples. Both accelerometers have the same sample rate in this example.

[0046] The time shifted correlation function of sensor data 23 received from the first and second sensor units as a function of ΔT is shown in FIG. 4. A distinct maximum 24 is found, which gives ΔT.

[0047] Once the latency or delay alignment has been done, the above described method can also be utilized to align the magnitude of a low accuracy sensor to an accurate sensor by calculating a scaling factor between the readings of the two sensor units. An example would be a cheap accelerometer in a headset that could be calibrated towards a more accurate accelerometer in a mobile phone

[0048] When the system has been aligned due to the latency/delay, ΔT, the relative movement, rotation or linear, between the devices within the system may be determined. This may be useful when the at least two devices moves related to each other.

[0049] In one embodiment a two dimensional relative orientation between at least two sensor units, comprised in the communication devices, may be determined. Two consecutive, non-parallel movements 19, 20, same movement each time in both system, are needed. The first movement could be a common rotation, or a common translation, and the second movement could be a common rotation or a common translation.

[0050] Prerequisite for the determination of the relative orientation is that the sensor data has been aligned, as disclosed above, and that the sensors are in a similar environments, i.e. the sensor readings are similar but not necessarily identical.

[0051] In one embodiment may a similar environment be the surrounding static gravitation, i.e. the earth gravitation. It may be possible to determine the angles of the movement, θ and φ, and the direction of the static acceleration 19, 20 of a device relative the static gravitation. The relative orientation between the at least two devices is however not possible to be determined based only on the static gravitation.

[0052] In one embodiment may a similar environment be any environmental sensor e.g. 50- or 60-Hz signal from surrounding electricity, light sensors, etc. The 50- or 60-Hz signal is a 3 dimensional electromagnetic vector, radiated by surrounding electric wires and surrounding equipment. The vectors from electric radiation may have different direction in the two measurement points for the devices, thus this vectors may only be used for alignment in time. For alignment time we the measured electromagnetic 50- or 60-Hz signal in the two measurement spots for the devices has to belong to radiation from the same phase. There are normally 3-phase electric systems and there is often multiple overlaid fields, delayed 120 degrees apart. Determining if the measured electromagnetic 50- or 60-Hz belong to radiation from the same phase may be done by correlation of disturbances, i.e. if there is a Dirac by some current change, or a specific harmonic pattern well correlated in the two measurements, then it may be concluded that they origin from the same phase, and then the measurements can be used for alignment in time between the at least two devices.

[0053] To be able to determine the relative orientation between at least two devices, i.e. the relative orientation between the reference co-ordinate systems of the first sensor unit and the second sensor unit, a movement of the devices or a perturbation of the sensor readings i.e. acceleration, is needed. The perturbation of the sensor readings for each sensor units should ideally be identical, but using statistical methods it is only necessary that the perturbations are similar over time, as long as the perturbation timescale is much faster than the timescale of the change in orientation of the sensor units relative each other.

[0054] A real life example with a near ideal conditions would be a headset and a mobile phone with accelerometers on a table in a boat during a storm. The relative orientation of the headset and the mobile phone is constant and the acceleration perturbation is large due to the rocking of the boat.

[0055] A more realistic, but still feasible, situation would be a headset worn by a person with the mobile phone in the pocket of a user, who is walking.

[0056] A stepwise process regarding how to define relative orientation between two devices is disclosed below and sensors for determination of linear movement and/or rotational movement in each device are used in this process.

[0057] In the first step, the first found linear or rotational movement that is enough correlated between the two devices 11, 12 is used to determine a first reference axis 16a, 17a within a reference co-ordinate system 16, 17, one in each device 11, 12. If the movement is linear, the reference axis is directed in the direction of the linear movement. If the movement is a rotation, the reference axis is the rotation axis. Enough correlated is defined by a predetermined value, which is determined dependent of which application the alignment is used for. The value is also dependent on the availability of sensor data and what demands are sat on the precision of the definition of the relative orientation.

[0058] The first reference axis 16a, 17a is determined as an axis in the direction of the first found enough correlated movements within each devices 11, 12. The first reference axis are parallel to each other. One dimension is now defined in the reference co-ordinate system 16, 17 within each device 11, 12. The first determined reference axis may be stored in an internal memory within each device.

[0059] In the second step, a second linear or rotational movement is found. The second found movement should be enough correlated between the two devices, but not parallel to the first found movement. This found second movement is used to determine the second reference axis 16b, 17b within each reference co-ordinate systems 16, 17. The second reference axis is determined as an axis in the direction of the second found movement in a plane that may be perpendicular towards the direction of the first reference axis.

[0060] The relative orientation between the two devices is determined by combining the reference co-ordinate systems of each device, 18.

[0061] The first and second step may be repeated continuously by finding linear or rotational movements that are enough correlated in each system. The found movements may then be used to recalculate the latest determined at least first and second reference axis.

[0062] Consecutively, the found linear or rotational movement that is enough correlated between the at least two devices are used to fine tune the previously determined relative orientation between the at least two devices 11, 12, and to compensate for deviations in the calculation of reference axis 16a, 16b, 17a, 17b. This increases the precision of the system. It may also be possible to pre-set when and how often this recalculation step may be performed.

[0063] As an alternative embodiment, the following method may be used to defining the relative orientation between two devices in a system described above.

[0064] In the first step, two angles, θ and φ, are defined which are used for transforming the reference co-ordinate system 16 of the first sensor unit, into the reference co-ordinate system 17 of the second sensor unit, as shown in FIG. 2.

[0065] In the second step, sensor data 14, 15 from both sensor units are collected and sampled during perturbation of the acceleration.

[0066] In the third step, the collected sensor data is aligned by using the time delay.

[0067] In the forth step, the environment is investigated. If the correlation degree of the collected sensor data set is high, the devices experience similar environment and the sensor data can be used. If there is no correlated sensor data it is an indication that the devices do not experience similar environments.

[0068] In the fifth step a 2D correlation function is set up a to sample the rotational space spanned by the two angles, θ and φ.

[0069] In the sixth step the correlation is maximized for finding the two angles, θ and φ.

[0070] The above described methods may be used for more than two devices.

[0071] In one embodiment, e.g. a gamming environment, the system may include a network of wearable sensors attached, for example, to a user's arms and/or legs. The method and system according to the invention may allow the user's to interact with the game.

[0072] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” “comprising,” “includes” and/or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0073] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0074] The foregoing has described the principles, preferred embodiments and modes of operation of the present invention. However, the invention should be regarded as illustrative rather than restrictive, and not as being limited to the particular embodiments discussed above. The different features of the various embodiments of the invention can be combined in other combinations than those explicitly described. It should therefore be appreciated that variations may be made in those embodiments by those skilled in the art without departing from the scope of the present invention as defined by the following claims.