Predicting a position of an object over time

[0001] Motion detection can play an important role in the medical field because the ability to ascertain and track the position (e.g., a three dimensional (3D) position) of a target object, such as a person or a body part of the person, may be crucial to the success of many medical applications or procedures including, for example, computed tomography (CT) or magnetic resonance imaging (MRI) scanning where a failure to detect a movement of the target object may significantly reduce the scan quality of the CT or MRI. However, a medical environment may involve “noisy” conditions for tracking the position of the target object. For example, the medical environment may include many other moving objects which may occlude some portion of the target object or even confuse some position tracking methods causing sensing errors and/or algorithmic uncertainty over time. As another example, due to detection challenges, approximations, etc., it may be difficult to distinguish the actual motion of a patient versus 2D or 3D prediction noise. These prediction or detection noises may be constant in the medical environment and may become heavier in some challenging scenarios (e.g., 3D position predictions may vary significantly from one timestep to another), whereas patient motion may be less frequent but important not to miss. Accordingly, systems, methods, instrumentalities are desired for improving the quality and efficiency of conventional position tracking methods.

[0002] Disclosed herein are apparatus and methods associated with predicting the position of an object (e.g., person or body part) over time. According to embodiments of the present disclosure, an apparatus may be configured to obtain multiple predictions about a position of an object during a time period, wherein each of the multiple predictions indicates the position of the object at a respective time during the time period. The apparatus may also obtain respective validity indications corresponding to the multiple predictions, wherein each respective validity indication indicates an accuracy of the corresponding prediction. The apparatus may then select a first subset of predictions and a second subset of predictions from the multiple predictions and determine whether a change has occurred between a first statistical distribution associated with the first subset of predictions and a second statistical distribution associated with the second subset of predictions. Based on a determination that the change has occurred between the first statistical distribution and the second statistical distribution, the apparatus may select a prediction from the first subset of predictions or the second subset of predictions as an indication of the position of the object.

[0003] In some embodiments, the apparatus may determine, based on the respective validity indications corresponding to the second subset of predictions, that the second subset of predictions is valid and select one of the second subset of predictions as the indication of the position of the object based on a further determination that the second subset of predictions is more accurate than the first subset of predictions or that a difference between the first subset of predictions and the second subset of predictions indicates that the object has moved. In some embodiments, the apparatus may determine that the object has moved based on a determination that the difference between the first subset of predictions and the second subset of predictions is above a movement threshold value. For example, the apparatus may determine the difference between the first subset of predictions and the second subset of predictions based on the average of the first subset of predictions and the average of the second subset of predictions.

[0004] In some embodiments, the apparatus may determine that the second subset of predictions is valid based on a determination that the respective validity indications corresponding to the second subset of predictions indicate that an accuracy of the second subset of predictions is above a validity threshold value. For example, the apparatus may determine that the second subset of predictions is more accurate than the first subset of predictions by comparing the accuracy of the second subset of predictions to an accuracy of the first subset of predictions as indicated by the respective validity indications associated with the first subset of predictions.

[0005] In some embodiments, the apparatus may determining, based on the respective validity indications corresponding to the second subset of predictions, that the second subset of predictions is valid and select one of the first subset of predictions as the indication of the position of the object further based on a determination that the first subset of predictions is more accurate than the second subset of predictions and that a difference between the first subset of predictions and the second subset of predictions indicates that the object has not moved. In some embodiments, the apparatus may determine, based on the respective validity indications corresponding to the second subset of predictions, that the second subset of predictions is invalid and selecting one of the first subset of predictions as the indication of the position of the object.

[0006] In some embodiments, based on a determination that no change has occurred between the first statistical distribution associated with the first subset of predictions and the second statistical distribution associated with the second subset of predictions, and that a latest prediction of the multiple predictions is valid, the apparatus may be configured to further determine, based on the latest prediction and one or more other predictions of the multiple predictions, whether the object has moved (e.g., a motion of the object has occurred). Based on a determination that the object has moved, the apparatus may select the latest prediction as the indication of the position of the object.

[0007] In some embodiments, based on a determination that no change has occurred between the first statistical distribution associated with the first subset of predictions and the second statistical distribution associated with the second subset of predictions, and that the latest prediction of the multiple predictions is valid, the apparatus may be configured to further determine, based on the respective validity indications corresponding to the latest prediction and a previous prediction about the position of the object, whether the latest prediction is more accurate than the previous prediction. Based on a determination that the latest prediction is more accurate than the previous prediction, the apparatus may select the latest prediction as the indication of the position of the object.

[0008] In some embodiments, each of the respective validity indications may include at least one of a first score that indicates a confidence in the corresponding prediction or a second score that indicates a visibility of the object. In some embodiments, the first statistical distribution associated with the first subset of predictions may include a temporal distribution of the first subset of predictions or a temporal distribution of the respective validity indications corresponding to the first subset of predictions, while the second statistical distribution associated with the second subset of predictions may include a temporal distribution of the second subset of predictions or a temporal distribution of the respective validity indications corresponding to the second subset of predictions. In some embodiments, the object may include a person or a joint of the person, and the position of the object may include a triangulated three-dimensional position of the person or the joint.

[0016] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. A detailed description of illustrative embodiments will be provided with reference to the figures. Although these embodiments may be described with certain technical details, it should be noted that the details are not intended to limit the scope of the disclosure. Further, while some embodiments may be described in a medical setting, those skilled in the art will understand that the techniques disclosed in those embodiments may also be applicable to other settings or use cases.

[0017] FIG. 1 shows a simplified block diagram of an example apparatus 100 that may be used to perform the operations associated with predicting the position of an object over time as described herein. Apparatus 100 may be a standalone computer system, or a networked computing resource implemented in a computing cloud. Apparatus 100 may include processing device(s) 102 and storage device(s) 104, where the storage device 104 may be communicatively coupled to processing device 102. Processing device(s) 102 may include one or more processors such as a central processing unit (CPU), a graphic processing unit (GPU), or an accelerator circuit. The storage device(s) 104 may include a memory device, a hard disc, and/or a cloud storage device connected to processing device 102 through a network interface card (not shown in FIG. 1). Processing device(s) 102 may be programmed to use historical data regarding the position of the object (e.g., obtained from storage device(s) 104 and/or some other storage device) to predict the position of the object over a time period, as described herein, via instructions 106.

[0018] The processing device(s) 102 may execute instructions 106 and perform one or more of the following operations for predicting the position of the object over a time period. At operation 108, multiple predictions about a position of an object during a time period may be obtained, wherein each of the multiple predictions may indicate the position of the object at a respective time during the time period. Throughout this disclosure, the positional candidate (e.g., prediction) for the object at time t (e.g., during the time period) may be denoted as X_t (e.g., X_t∈² or X_t∈³ for 2D or 3D positions). For example, a medical imaging system may estimate a 3D pose of a person, as defined by the 3D position of the person's joints (e.g., the object described herein), in a scanning or surgery room. Multiple visual sensors may be placed in the room, capturing sequences of images (RGB, depth, and/or IR) of the person, which may then be processed by a computer vision algorithm (e.g., implemented by apparatus 100 or another apparatus). This algorithm may rely on the multi-sensor data to determine (e.g., via triangulation) the position of the person's joints at each time step during the time period. This data regarding the position of the joints may then be obtained by processing device(s) 102 (e.g., obtained from storage device(s) 104 or from another apparatus). However, due to a variety of factors (such as occlusion of the patient, movement of the patient, sensing errors, algorithmic uncertainty, etc.), these predications regarding the 3D position of the joints may be inaccurate and/or inconsistent over time.

[0019] At operation 110, respective validity indications corresponding to the multiple predictions may be obtained, wherein each respective validity indication may indicate an accuracy of the corresponding prediction. For example, the processing device(s) 102, or other separate automated or user-controlled processor(s), configured to provide, at each time step t, a prediction with regard to the 2D or 3D positions of the target object, may also provide a validity indication V_t(e.g., scores) respectively assigned to each corresponding prediction X_t in order to provide some information on the confidence/quality of the said corresponding prediction X_t. In some embodiments, these validity indications V_t may be self-estimated by the processing device(s) 102 (and/or other separate processor(s)), or they may be assigned by a human user of the apparatus 100 (and/or the other separate processor(s)). These validity indications V_t may provide some non-exhaustive information on the quality of the positional candidates X_t. In some embodiments, these validity indications V_t may take the form of confidence scores (e.g., scores indicating the confidence about each prediction X_t), visibility scores (e.g., scores indicating how visible the target object(s) were at time t, where a lower visibility may indicate a lower validity of a positional candidate X_t), reprojection scores (e.g., scores indicating the consistency of the 3D-to-2D reprojection for 3D predictions X_t that were triangulated from 2D predictions X_t), etc. Throughout this disclosure, the list of n validity indications (e.g., confidence scores, visibility scores, etc.) assigned to the positional candidate X_t may be denoted as V_t∈ⁿ and the paired prediction/validity indication for one target object at time t may be denoted as Z_t=(X_t, V_t).

[0020] At operation 112, the processing device(s) 102 may group the paired prediction values (e.g., past Z_t values from the same time period) into at least 2 subsets (e.g., each corresponding to two separate “concepts” regarding the position of the target object). A first subset (e.g., Z_old) may correspond to the Z_t values from a first time period and a second subset (e.g., Z_new) corresponding to the Z_t values from a second time period. The processing device(s) 102 may then determine whether a change has occurred between a first statistical distribution associated with the first subset of paired predictions (e.g., Z_old) and a second statistical distribution associated with the second subset of paired predictions (e.g., Z_new). The first statistical distribution may include a temporal distribution of the first subset of predictions or a temporal distribution of the respective validity indications corresponding to the first subset of predictions, while the second statistical distribution may include a temporal distribution of the second subset of predictions or a temporal distribution of the respective validity indications corresponding to the second subset of predictions. For example, the processing device(s) 102 may apply a “concept-drift” detection algorithm to analyze the temporal distribution of the first or second subset of paired predictions Z_t for each target object and determine if a change between the distributions is detected (e.g., a change in the distributions of X or V over the corresponding time period is detected between the first subset of predictions and the second subset of predictions). In some embodiments, the processing device(s) 102 may detect the change in the two distributions based on the difference between the two distributions being larger than some predefined change threshold value.

[0021] At operation 114, the apparatus 100 may, based on a determination that the change in the first statistical distribution and the second statistical distribution has occurred, select a prediction from the first subset of predictions or the second subset of predictions as an indication (e.g., a best indication) of the position of the object. As described herein, the two subsets may correspond to two separate “concepts” regarding the position of the target object. The processing device(s) 102 may compare the two subsets to determine which concept (e.g., a predicted position of the object from the first subset or from the second subset) should be selected to represent the position of the object. In some embodiments, when comparing two subsets of values, the processing device(s) 102 may aggregate the values of each subset in order to facilitate their comparison. For example, the average/mean or median value of the paired prediction values Z_t of each subset may be computed.

[0022] Accordingly, the processing device(s) 102 may receive the latest paired prediction Z_t, memorize it (e.g., store it in storage device(s) 104), and provide a “best” paired prediction Z_s=(X_s, V_s) that accounts for the previous paired prediction values {Z_(t-1), Z_(t-2), . . . , Z_(t-w)}(e.g., based on a memory size w for storage of position prediction values for the object(s)). The processing device(s) 102 may return Z_s=Z_t-τ) (e.g., with τ∈[0, w]) wherein the value of τ may be decided based on the comparison of the different concepts regarding the position of the object as explained above.

[0023] FIGS. 2A-2C show graphs 200A-200C illustrating operations associated with predicting the position of an object over time as described herein.

[0024] FIG. 2A shows graph 200A illustrating an operation (e.g., memory operation) associated with adding (e.g., by the processing device(s) 102 of FIG. 1) a new paired prediction result Z_t to a memory configured to store paired prediction values associated with the position of the target object (e.g., the paired prediction value Z_t may be added to the end of the sequence of paired prediction values after Z_t-1). As noted above, the memory used for storing the paired prediction values may include storage device(s) 104 of FIG. 1 and/or other remote storage devices.

[0025] FIG. 2B shows graph 200B illustrating an operation (e.g., memory operation) associated with obtaining subsets of the paired prediction values Z from the memory configured to store past paired prediction values associated with the position of the target object. The first subset of paired prediction values may correspond to the “old” concept associated with the position of the object

[0026] $\left(e.g.,Z_{\mathrm{old}}={\left\{z_{t-{\mathrm{rand}}_i\left(n_s,n_w\right)}\right\}}_{i=1}^{n_s}\right)$
and may include a predefined number n_s of paired prediction values chosen randomly from the paired prediction values. The second subset of paired prediction values may correspond to the “new” concept associated with the position of the object

[0027] $\left(e.g.,Z_{\mathrm{new}}={\left\{z_{t-i+1}\right\}}_{i=1}^{n_s}\right)$
and may include the same predefined number n_s of paired prediction values (e.g., so the two subsets have an equal number of values to be compared) chosen from the latest paired prediction values.

[0028] FIG. 2C shows graph 200C illustrating an operation (e.g., a memory operation) associated with obtaining the current best paired prediction value Z_s and the latest paired prediction value Z_t from the memory that stores past paired prediction values associated with the position of the target object. As described more fully below with respect to the methods 500A and 500B of FIGS. 5A and 5B respectively, if no change is detected in the temporal distribution of the paired prediction values, the processing device(s) 102 may obtain the current values of the best paired prediction Z_s and of the latest paired prediction Z_t.

[0029] FIG. 3 shows a flow diagram of an example method 300 (e.g., implemented by processing device(s) 102 of FIG. 1) for predicting the position of an object over time as described herein.

[0030] Method 300 may include obtaining, at 302, multiple predictions about a position of an object during a time period, wherein each of the multiple predictions may indicate the position of the object at a respective time during the time period (e.g., the position may be a triangulated 3D position of a person's joint). As noted above, a medical imaging system may need to estimate a 3D pose of a person, as defined by the 3D position of the person's joints (e.g., objects), in a scanning or surgery room. The data regarding the position of the joints may then be provided to storage device(s) 104 of FIG. 1 so that it may be obtained by processing device(s) 102. Also as noted above, the positional candidate (e.g., prediction) for one object at time t (e.g., during the time period) may be denoted as X_t(e.g., X∈² or X_t∈³ for 2D or 3D positions).

[0031] Method 300 may further include, at 304, obtaining respective validity indications corresponding to the multiple predictions, wherein each respective validity indication may indicate an accuracy of the corresponding prediction. As noted above, the processing device(s) 102, or other separate automated or user-controlled processor(s), configured to provide, at each time step t, predictions with regard to the 2D or 3D positions of the target object(s), may also provide validity indications V_t (e.g., scores) respectively assigned to each corresponding prediction X_t in order to provide some information on the confidence/quality of the said corresponding prediction X_t. In some embodiments, these validity indications V_t may take the form of confidence scores (e.g., scores indicating the confidence of each prediction X_t), visibility scores (e.g., scores indicating how visible the target object(s) were at time t, where a lower visibility may indicate a lower validity of a positional candidate X_t), reprojection scores (e.g., scores indicating the consistency of the 3D-to-2D reprojection for 3D predictions X_t that were triangulated from 2D predictions X_t), etc.

[0032] At 306, method 300 may include selecting a first subset of predictions and a second subset of predictions from the multiple predictions obtained at 302, and determining whether a change has occurred between a first statistical distribution associated with the first subset of predictions and a second statistical distribution associated with the second subset of predictions (e.g., the statistical distributions may include respective temporal distributions of the two subsets of predictions or the respective validity indications corresponding to the subsets of predictions). For example, the first subset of predictions (e.g., Z_old including X_old and V_old) may be associated with a first time period, and the second subset of predictions (e.g., Z_new, including X_new, and V_new) associated with a second time period. As noted above, the first subset of paired prediction values may correspond to the “old” concept associated with the position of the object

[0033] $\left(e.g.,Z_{\mathrm{old}}={\left\{z_{t-{\mathrm{rand}}_i\left(n_s,n_w\right)}\right\}}_{i=1}^{n_s}\right)$
and may include a predefined number n_s of paired prediction values chosen randomly from the paired prediction values. Also as noted above, the second subset of paired prediction values may correspond to the “new” concept associated with the position of the object

[0034] $\left(e.g.,Z_{\mathrm{new}}={\left\{z_{t-i+1}\right\}}_{i=1}^{n_s}\right)$
and may include the same predefined number n_s of paired prediction values chosen randomly from the paired prediction values (e.g., so the two subsets have an equal number of values to be compared). In examples, the processing device(s) 102 may perform a “concept-drift” based determination to analyze the respective temporal distributions of the two subsets of paired predictions and determine if a change in the distributions is detected (e.g., a difference in the distributions being above a threshold value). As noted above, when comparing the two subsets of values, the processing device(s) 102 may aggregate the values of each subset in order to facilitate their comparison. For example, the change in the temporal distributions may be detected based on the following:

[0035] $\mathrm{dist}\left(Z_{\mathrm{old}},Z_{new}\right)>\sqrt{\frac{-\ln \alpha }{n_s}},$
wherein dist( ) may represent a statistical distance between the subsets of paired prediction values Z_old and Z_new. In examples, such a statistical distance may be determined based on a predefined probability a (e.g., for a Kolmogorov-Smirnov test (K-S test or KS test) of the equality of probability distributions) and the predefined number of values n_s for each of the subsets.

[0036] At 308, based on a determination that the change has occurred between the first and second statistical distributions described above, method 300 may proceed to 310, where a prediction from the first subset of predictions or the second subset of predictions may be selected as the indication (e.g., a best indication) of the position of the object. As noted above, the processing device(s) 102 may compare the two subsets (e.g., Z_old and Z_new) in order to determine which concept (e.g., a predicted position of the object from the first subset or from the second subset) should be selected. The selection may be made, for example, by computing average position and validity values for each subset:

[0037] $\mathrm{avg}{Z}_{\mathrm{old}}=\frac{1}{n_s}{\sum }_{i=1}^{n_s}{Z}_{old}^i\mathrm{and}{Z}_{new}=\frac{1}{n_s}{\sum }_{i=1}^{n_s}{Z}_{\mathrm{new}}^i$
and then comparing the averages.

[0038] As explained with respect to FIGS. 4A-4B below, if the second subset of prediction values (e.g., Z_new) are deemed valid (e.g., based on the validity indication values and a validity threshold value) and if the difference between the first subset Z_old and the second subset Z_new is large enough to suspect that the target object has moved (e.g., based on a movement threshold value), or if the second subset of prediction values are deemed more accurate than the first subset of prediction values (e.g., based on a comparison of respective validity indication values), the method 300 may return a result (e.g., the best result) from the second subset Z_new. Otherwise, if the second subset of prediction values is not deemed valid (e.g., based on the validity indication values and the validity threshold value), or if no motion is suspected (e.g., based on the movement threshold value) and the first subset of prediction values is deemed more accurate (e.g., based on a comparison of respective validity indication values), the method 300 may return the same result as the previous timestep. In some embodiments, the movement threshold value (e.g., distance) for detecting object motion may be a predefined constant for each or all target objects or may be defined relative to other properties of the target object and corresponding predictions. For example, the threshold value may be a multiple of the standard deviation of past prediction results.

[0039] Furthermore, as explained with respect to FIGS. 5A-5B below, if no change in the temporal distribution of the two subsets of paired predictions is detected, method 300 may still compare the latest paired prediction Z_t with the previously returned best paired prediction Z_s. If Z_t is deemed valid, and if a motion of the object since last time step is suspected or if Z_t is deemed more accurate, then method 300 may switch to Z_t (Z_s←Z_t) and return its value as the indication (e.g., best indication) of the position of the object. Otherwise, method 300 may simply return the same result Z_s as returned in the previous timestep.

[0040] FIGS. 4A-4B show flow diagrams illustrating example methods 400A and 400B for selecting a prediction of the position of an object as an indication (e.g., the best indication) of the position of the object as described herein.

[0041] FIG. 4A illustrates the method 400A, which may continue, at 402A, from operation 310 of method 300 shown in FIG. 3 (e.g., associated with selecting a prediction from the first subset of predictions or the second subset of predictions as an indication of the position of the object). As shown in FIG. 4A, method 400A may include determining, at operation 404A, that the second subset of predictions is valid based on the respective validity indications corresponding to the second subset of predictions. For example, the validity may be evaluated based on criteria such as confidence scores (e.g., scores indicating the confidence of the processing device(s) 102, or other separate processor(s), with regard to each prediction X_t), visibility scores (e.g., scores indicating how visible the target object(s) were at time t, with a lower visibility indicating a lower validity of a positional candidate X_t), reprojection scores (e.g., scores indicating the consistency of the 3D-to-2D reprojection for 3D predictions X_t that were triangulated from 2D predictions X_t), etc. If one of the subsets (e.g., the second subset Z_new) does not meet the validity threshold value with respect to any of the criteria, the subset may be deemed invalid and a prediction value (e.g., best prediction value) from the other subset (e.g., the first subset Z_old) may be returned.

[0042] At 406A, method 400A may further include selecting one of the second subset of predictions as the indication of the position of the object based on a further determination that the second subset of predictions is more accurate than the first subset of predictions (e.g., based on comparing the respective validity indication values V_t corresponding to each of the position prediction values X_t for each subset), or that a difference between the first subset of predictions and the second subset of predictions indicates that the object has moved (e.g., based on the difference between the position prediction values being greater than the movement threshold value). For example, the position prediction value may be selected from the second subset (e.g., Z_new) based on the following: Z_s←Z_t w/τ=argmax V_t-i+1 wherein i∈[1, n].

[0043] FIG. 4B illustrates the method 400B, which may continue, at 402B, from operation 310 of method 300 shown in FIG. 3 (e.g., associated with selecting a prediction from the first subset of predictions or the second subset of predictions as an indication of the position of the object).

[0044] As shown in FIG. 4B, method 400B may include determining, at 404B, that the second subset of predictions is valid based on the respective validity indications corresponding to the second subset of predictions. As noted above, the validity may be evaluated based on several criteria (e.g., confidence scores, visibility scores, reprojection scores, etc.), and if a subset does not meet the validity threshold value with respect to any of the criteria, the subset may be deemed invalid. At 406B, method 400B may further include selecting a prediction from the first subset of predictions as the indication (e.g., the best indication) of the position of the object based on a further determination that the first subset of predictions is more accurate than the second subset of predictions (e.g., based on comparing the respective validity indication values V_t corresponding to each of the position prediction values X_t for each subset), and that a difference between the first subset of predictions and the second subset of predictions indicates that the object has not moved (e.g., based on the difference between the position prediction values being less than the movement threshold value).

[0045] FIGS. 5A-5B show flow diagrams illustrating example methods 500A and 500B for selecting a prediction of the position of an object as an indication (e.g., the best indication) of the position of the object when no change in the distribution of the predictions or the validation indications is detected as described herein.

[0046] FIG. 5A shows the method 500A, which may include determining, at 502A, that no change between the respective statistical distributions associated with a first subset of predictions and a second subset of predictions has occurred, and that the latest prediction of the multiple predictions is valid. Method 500A may further include determining, at 504A, whether the object has moved (e.g., whether a motion of the object has occurred) based on the latest prediction and one or more other predictions of the multiple predictions. As noted above, the movement of the object may be determined by comparing the respective position prediction values of the latest prediction and one or more other predictions to determine whether the difference between the values is greater than a movement threshold value. At 506A, method 500A may further include selecting the latest prediction as the best indication of the position of the object based on a determination that the object has moved (e.g., based on the difference between the respective position prediction values of the latest prediction and the one or more other predictions being greater than the movement threshold value).

[0047] FIG. 5B shows the method 500B, which may include determining, at 502B, that no change between the respective statistical distributions associated with a first subset of predictions and a second subset of predictions has occurred, and that the latest prediction of the multiple predictions is valid. The method 500B may further include determining, at 504B, whether the latest prediction Z_t is more accurate than a previous prediction (e.g., the previously determined best prediction) Z_s. Such a determination may be made, for example, based on the respective validity indications V_t and V_s corresponding to the latest prediction Z_t and the previous prediction Z_s about the position of the object. As noted above, the respective validation (or accuracy) of the predictions may be compared by comparing the respective validity indication values V_t and V_s (e.g., which score(s) are higher).

[0048] At 506B, the method 500B may include selecting the latest prediction as an indication (e.g., the best indication) of the position of the object based on a determination that the latest prediction is more accurate than the previous prediction (e.g., based on the validity indication value V_t being greater than the validity indication value V_s).

[0049] For simplicity of explanation, the operations of the methods (e.g., performed by apparatus 100 of FIG. 1) are depicted and described herein with a specific order. It should be appreciated, however, that these operations may occur in various orders, concurrently, and/or with other operations not presented or described herein. Furthermore, it should be noted that not all operations that apparatus 100 is capable of performing are depicted in FIGS. 3, 4A-4B and 5A-5B or described herein. It is also noted that not all illustrated operations may be required to be performed.

[0050] FIG. 6 is a block diagram illustrating an apparatus in the example form of a computer system 600, within which a set or sequence of instructions may be executed to cause the system to perform any one of the methods discussed herein.

[0051] The apparatus may operate as a standalone device or may be connected (e.g., networked) to other machines or devices. In a networked deployment, the apparatus may operate in the capacity of either a server or a client machine in server-client network environments, or it may act as a peer machine in peer-to-peer (or distributed) network environments. The apparatus may be a wearable device, a personal computer (PC), a tablet PC, a hybrid tablet, a personal digital assistant (PDA), a mobile telephone, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single apparatus is illustrated, the term “apparatus” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. Similarly, the term “processor-based system” shall be taken to include any set of one or more machines that are controlled by or operated by a processor (e.g., a computer) to individually or jointly execute instructions to perform any one or more of the methodologies discussed herein (e.g., method 300 of FIG. 3, methods 400A and 400B of FIGS. 4A-4B and methods 500A and 500B of FIGS. 5A-5B).

[0052] Example computer system 600 includes at least one processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both, processor cores, compute nodes, etc.), a main memory 604 and a static memory 606, which communicate with each other via a link 608 (e.g., bus). The computer system 600 may further include a video display unit 610, an alphanumeric input device 612 (e.g., a keyboard), and a user interface (UI) navigation device 614 (e.g., a mouse). In one embodiment, the video display unit 610, input device 612 and UI navigation device 614 are incorporated into a touch screen display. The computer system 600 may additionally include a storage device 616 (e.g., a drive unit), a signal generation device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 622, such as a global positioning system (GPS) sensor, accelerometer, gyro meter, magnetometer, or other such sensor.

[0053] The storage device 616 includes a machine-readable medium 624 on which is stored one or more sets of data structures and instructions 626 (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The instructions 626 may also reside, completely or at least partially, within the main memory 604, static memory 606, and/or within the processor 602 during execution thereof by the computer system 600, with main memory 604, static memory 606, and the processor 602 comprising machine-readable media.

[0054] While the machine-readable medium 624 is illustrated in an example embodiment to be a single medium, the term “machine-readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more instructions 626. The term “machine-readable medium” shall also be taken to include any tangible medium that is capable of storing, encoding or carrying instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure or that is capable of storing, encoding or carrying data structures utilized by or associated with such instructions. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine-readable media include volatile or non-volatile memory, including but not limited to, by way of example, semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0055] The instructions 626 may further be transmitted or received over a communications network 628 using a transmission medium via the network interface device 620 utilizing any one of a number of well-known transfer protocols (e.g., HTTP). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, mobile telephone networks, plain old telephone (POTS) networks, and wireless data networks (e.g., Wi-Fi, 3G, and 16G LTE/LTE-A or WiMAX networks). The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying instructions for execution by the machine, and includes digital or analog signals or other intangible medium to facilitate communication of such software.

[0056] Example computer system 600 may also include an input/output controller 630 to receive input and output requests from at least one central processor 602, and then send device-specific control signals to the device they control. The input/output controller 630 may free at least one central processor 602 from having to deal with the details of controlling each separate kind of device.

[0057] The term “computer-readable storage medium” used herein may include any tangible medium that is capable of storing or encoding a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term “computer-readable storage medium” used herein may include, but not be limited to, solid-state memories, optical media, and magnetic media.

[0058] The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated in the functionality of other hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, the methods, components, and features may be implemented by firmware modules or functional circuitry within hardware devices. Further, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in computer programs.

[0059] While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. Accordingly, the above description of example embodiments does not constrain this disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this disclosure. In addition, unless specifically stated otherwise, discussions utilizing terms such as “analyzing,” “determining,” “enabling,” “identifying,” “modifying” or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (e.g., electronic) quantities within the computer system's registers and memories into other data represented as physical quantities within the computer system memories or other such information storage, transmission or display devices.

[0060] It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other implementations will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the disclosure should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.