System and method for controlling a mobile industrial robot using a probabilistic occupancy grid

[0001] The present disclosure relates to the field of robot navigation and, in particular, to methods and devices for controlling a mobile industrial robot on the basis of a probabilistic occupancy grid representing a substrate on which the robot can move.

[0002] Significant research has been and is being devoted to the problem of developing perception mechanisms for supporting control systems in mobile industrial robots. A satisfactory perception mechanism should enable the robot to navigate constrained and populated environments efficiently, while at the same time avoiding destructive collisions. For this purpose, many regard sensor fusion as a promising approach, wherein information captured using different sensor technologies or different measuring principles is combined. Such combining is likely to improve the robot's discovery and correct assessment of difficult obstacles, including objects at different heights, objects with small dimensions, transparent or reflective objects, dark or black objects.

[0003] For example, US20170197311A1 describes a method for building a probability map of absence or presence of obstacles in a navigable environment of an autonomous robot. A probability map is initialized with predefined values of a probability of the absence or presence of an obstacle, and the values are then updated based on acquired data from at least one sensor type, such as a lidar, a three-dimensional camera, an ultrasonic sensor or contact sensors mounted on the robot.

[0004] The problem of updating such a probability map based on a sequence of sensor measurements from a moving platform is discussed in a preprint by Chr. Robbiano et al., “Bayesian Learning of Occupancy Grids”, retrieved from arxiv.org/abs/1911.07915.

[0005] Particularly addressing an indoor mobile robot application where the navigable environment is modeled by a two-dimensional occupancy grid, the present disclosure aims to refine and improve the prior art techniques.

[0006] One objective is to make available methods and devices for providing an occupancy grid to support the controlling of a mobile industrial robot. Another objective is to enable maintenance and successive refinement of such occupancy grid on the basis of continuing measurements by a sensor at an elevated point on the robot. A particular objective is to propose efficient data processing techniques suited for use cases where RGB-D (red-green-blue plus depth, i.e., color image and depth map) measurements or millimeter-wave radar measurements, or both, are available.

[0007] These and other objectives are achieved by the method and robot controller according to the invention defined by the independent claims. The dependent claims are directed to embodiments of the invention.

[0008] In a first aspect, a method for controlling an industrial robot movable on a substrate comprises: initializing an occupancy grid of cells which each represents a portion of the substrate and is associated with an occupancy probability that some physical object is present in the cell; obtaining occupancy-related measurements using at least one measuring principle that includes sensing an incident electromagnetic wave at an elevated point on the robot; assigning an occupancy probability on the basis of an obtained occupancy-related measurement at an angle of incidence S; and controlling the industrial robot on the basis of the occupancy grid. According to an embodiment of the invention, the assigning step includes evaluating whether the measurement indicates that some physical object is present in the angle of incidence, selecting a first predetermined model if the evaluation is positive and selecting a second predetermined model if the evaluation is negative, and determining a new occupancy probability for cells in the angle of incidence on the basis of the measurement and in accordance with the selected model.

[0009] The use of dual models for determining the new occupancy probability has proved advantageous. More precisely, the measurement data can be processed in two independently configurable ways depending on whether the measurement predominantly indicates that an object is present or not.

[0010] In a second aspect, there is provided a robot controller for controlling at least one mobile industrial robot. The robot controller comprises an input interface, processing circuitry and output interface and it is configured to perform the above method.

[0011] The invention further relates to a computer program containing instructions for causing a computer, or the robot controller in particular, to carry out the above method. The computer program may be stored or distributed on a data carrier. As used herein, a “data carrier” may be a transitory data carrier, such as modulated electromagnetic or optical waves, or a non-transitory data carrier. Non-transitory data carriers include volatile and non-volatile memories, such as permanent and non-permanent storages of magnetic, optical or solid-state type. Still within the scope of “data carrier”, such memories may be fixedly mounted or portable.

[0012] As used herein, an “angle” may be a planar angle (e.g., direction, unit: 1 radian) or a solid angle (e.g., sensor's field of view, frustum; unit: 1 steradian). The specific term “angle of incidence” includes not only the size/measure of the angle that the sensor covers but also the current orientation of the sensor. The orientation may be expressed with respect to a frame of reference that is stationary with respect to the sensor.

[0013] As used herein, a “measurement” includes the measured quantity as well as indications relating to the measuring process as such, e.g., noisiness, quality, accuracy.

[0014] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a/an/the element, apparatus, component, means, step, etc.” are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

[0021] The aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, on which certain embodiments of the invention are shown. These aspects may, however, be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and to fully convey the scope of all aspects of invention to those skilled in the art. Like numbers refer to like elements throughout the description.

[0022] FIG. 2 depicts a setting where the present invention may be applied. Here, an industrial robot 210 is movable over a substrate 220 by means of wheels 214, bands, claws, movable suction cups or other means of propulsion and/or attachment. The substrate 220 may be a planar or curved surface with a horizontal, slanted or vertical orientation; it may optionally be provided with rails or other movement guides adapted to cooperate with the robot 210. Example substrates 220 include an indoor floor, outdoor ground, a surface preconditioned to facilitate driving etc. Present are furthermore physical objects which may act as obstacles of different seriousness to the robot's 210 movement over the substrate 220. To exemplify, FIG. 2 shows a box-shaped object 240 comparable to the robot's 210 height, a puddle of liquid 241 and a moderately sized heap of powdery material 242. From these, the puddle of liquid 241 will normally not impede the robot's 210 movement and may therefore be treated as a non-obstacle.

[0023] The bookkeeping of obstacles relies on an occupancy grid 230, a data structure which associates each of the multiple cells 231 of the grid with an occupancy probability p. An occupancy probability of a cell at grid indices (x, y) may be denoted p(x, y). The resolution of the grid may be expressed as a cell size parameter w_res, which may be of the order of 0.01 m to 1 m, such as 0.05 m, 0.10 m or 0.20 m. For navigation of relatively slower-moving and sensitive robots 210, a smaller cell size is preferable. In the present disclosure, unless otherwise indicated, grid indices (x, y) refer to the discretized grid at resolution w_res. The grid indices may for example be represented as integer pairs, and real-valued cartesian coordinates (X, Y, Z) (which may have been read from a sensor) may be converted into grid indices (x, y) by orthogonal projection on the Z=0 plane (corresponding to the substrate 220) combined with a rounding operation. The grid indices may optionally be extended with a height index z into an integer triplet (x, y, z). In this disclosure, uppercase letters are preferred for coordinates and lowercase letters for grid and height indices.

[0024] The present disclosure will rely on ordinary terminology and the commonly embraced theoretical basics of occupancy grids. In particular, an occupancy probability of a cell may be understood as the posterior probability that a physical object is present in the cell. The highest likelihood that a physical object is present corresponds to the value p=1. Cells with occupancy probability values in the range (0.5,1.0] may be treated as occupied cells, and cells with values in [0.0,0.5) may be taken to be free. The occupancy grid 230 is a probabilistic representation that goes beyond the binary distinction occupied/free and thereby enables sensor fusion, as discussed above, and the nuanced combining of multiple measurements by a single sensor. The occupancy probability is uniform or model-agnostic in the sense that it may have been produced or partially produced by the first predetermined model (for the case of a measurement predominantly suggesting that the cell is occupied) or the second predetermined model (for the case of a measurement predominantly suggesting that the cell is free). In some embodiments, an alternative occupancy grid is used where each cell is associated with a probability (a ‘non-occupancy probability’) p′ that the cell is free, i.e., no physical object is present. This means that the highest likelihood that a physical object is present corresponds to the value p′=0.

[0025] The robot 210 is shown equipped with a movable arm 213, end effector 212 and sensor 211. The sensor may be elevated, i.e., located some distance above the level of the substrate 220 to make efficient use of its field of view. The sensor 211 may be configured to employ at least one measuring principle that includes sensing an incident electromagnetic wave reflected or emitted by the objects 240, 241, 242 on the substrate 230 or the substrate 230 itself. To exemplify, an incident ray from the box 240 is shown in FIG. 2. The measuring principle may be one or more of optical, electromagnetic reflection, electromagnetic scattering, electromagnetic diffraction, lidar, RGB-D sensing, mm-wave radar, ultra-wideband radar (UWB). Alternatively, the sensor 211 may be acoustic. In FIG. 2, the symbol S denotes an angle of incidence in the sense discussed above. The sensor 211 may have an angle of incidence S which is variable as regards its extent and/or its orientation.

[0026] FIG. 3 is a schematic representation of a robot controller 300 suitable for controlling the robot 210 of FIG. 2. The robot controller 300 may be integral with the robot 210 or may be implemented as a separate, stationary or mobile unit. It comprises an input interface 310 configured to receive occupancy-related measurements obtained by the sensor 211 on the robot, processing circuitry 320 configured to assign an occupancy probability on the basis of the received occupancy-related measurements, an output interface 330 for supplying commands to the robot 210, as well as an operator interface 340. The interfaces 320, 340 and processing circuitry 320 may be local resources or networked (or cloud) resources; in particular, the processing circuitry 320 may include one processor or multiple processors. The operator interface 340 may be a text-based, graphical, auditory and/or haptic interface, by which the robot controller 300 may receive instructions directing on high level the commands to be fed to the robot 210, and by which it may output human-intelligible information indicative of a state or condition of the robot 210. FIG. 3 furthermore indicates an example set of communicative connections (data connections) among the internal functional components 310, 320, 330, 340 of the robot controller 300. Alternatively, the connections may be implemented as a central bus providing multipoint connectivity.

[0027] To establish and maintain the occupancy grid, the robot controller 300 may perform the method 100 depicted in FIG. 1.

[0028] In a first step 110, the occupancy grid is initiated. It may take into account the geometry of the substrate 220 and a configured value of the cell size w_res. The initiation may further include assigning a neutral/unknown occupancy probability (e.g., p=0.5) to all cells 231 of the occupancy grid 230. Alternatively, a specific placeholder value may be assigned, so as to signify that the occupancy probability has not yet been updated in view of an occupancy-related measurement.

[0029] In a second step 120, occupancy-related measurements are obtained using the robot-carried sensor 211 shown in FIG. 2. Each measurement includes data representing the measured quantity but may also contain indications describing the measuring process as such, e.g., noisiness, quality, accuracy, irregularities.

[0030] A third step 130 of the method 100 comprises multiple substeps where the measurement is processed with the ultimate aim of determining a new occupancy probability to some cells 231 in the occupancy grid 230. This may include adjusting or gradually refining the neutral or placeholder values assigned in the initiation step 110 using the occupancy-related measurement as guidance.

[0031] In a first substep 132 of the third step 130, it is evaluated whether the measurement indicates that some physical object is present in the angle of incidence S. Put differently, it is evaluated whether some physical object is present in any of the cells that overlap (by at least a predetermined percentage, such as 50%) with the orthogonal projection of the angle of incidence S. The evaluation is meaningful as long as the elevation (e.g., defined as the angle between the substrate 220 and the line of sight of the sensor 211) of the angle of incidence is moderate; if a measurement is captured at such angle of incidence S that the lower boundary of the sensor's 211 field of view is above the height of expected obstacles in the remotest measurable areas of the substrate 220, this measurement may be disregarded. It is remarked that while the output of successful completion of this substep 132 is binary (object present/absent), it will form the basis of the assignment of an occupancy probability anywhere in the real interval [0,1] or a digital representation thereof. The resolution of the evaluation in substep 132 may be the full sensor frustum or a sub-interval of angles of incidence. For example, if a sub-interval S₁ of the sensor frustum S is obscured by a physical object, then the evaluation may be positive for S₁ and negative for the complement S\S₁.

[0032] Assuming that it can be determined whether some physical object is present in the angle of incidence S, the execution of the third step 130 moves on to the second substep 134, which selects a first predetermined model M1 if the determination was positive (objects are likely present) and a second predetermined model M2 if the determination was negative (objects are likely absent). For a given cell in the angle of incidence S, a single one of the predetermined models M1, M2 is selected; in an execution of the third step 130, each cell is updated on the basis of either model, not both.

[0033] In the third substep 136, then, a new value p_new of the occupancy probability for cells in the angle of incidence S is determined on the basis of the measurement and in accordance with the selected first or second predetermined model. It is noted that each of the predetermined models M1, M2 may be adaptive in the sense that different processing is foreseen for different measuring principles, as exemplified below.

[0034] If a pre-existing value p_old(x, y) of the probability for a cell at grid indices (x, y) in the angle of incidence S exists, an optional fourth substep 138 of merging the new occupancy probability p_new(x, y) and the pre-existing one may be executed. For this purpose, a recursive rule, as Bayes' rule, may be applied. The merging may proceed as follows:

[0035] $p_{\mathrm{merged}}\left(x,y\right)=\frac{p_{\mathrm{old}}\left(x,y\right)p_{\mathrm{new}}\left(x,y\right)}{p_{\mathrm{old}}\left(x,y\right)p_{\mathrm{new}}\left(x,y\right)+\left[1-p_{\mathrm{old}}\left(x,y\right)\right]\left[1-p_{\mathrm{new}}\left(x,y\right)\right]}.$
If no pre-existing value exists, the new value determined in substep 136 is adopted.

[0036] After completion of the third step 130 of the method 100, the robot controller 300 may execute a fourth step 140, in which the industrial robot 310 is controlled on the basis of the occupancy grid. Such controlling may include local motion planning, tactical route planning, collision avoidance, speed regulation and similar tasks.

[0037] With reference to FIG. 4, the first and second predetermined models M1 and M2 will now be exemplified in the special case where the sensor 211 is or includes an RGB-D sensor. An RGB-D sensor may produce a two-dimensional color bitmap, where at least some pixels are annotated with depth values, representing an image plane orthogonal to the orientation of the angle of incidence S. Alternatively, the RGB-D sensor may produce a point cloud, where each point is represented in three-dimensional coordinates (e.g., cartesian, spherical, radial) and additionally carries a color or chromaticity value. RGB-D sensors are commercially available and easy to integrate. Since however the color information is not relied upon in the processing to be described below, a sensor producing a two-dimensional monochrome or grey-tone bitmap with associated depth information shall be considered equivalent to an RGB-D sensor.

[0038] In the coordinate system shown in FIG. 4, the RGB-D sensor 211 is located at coordinates (X_a, Y_a, Z_a)=(0,0,1) and is aimed approximately at (4,0,0), so that a region of the substrate 220 surrounding that point is imaged together with any objects present thereon. The conical region C(S) represents the sensor's 211 frustum (field of view) at an angle of incidence S. For purposes of the robot controller 300 described herein, it may be suitable to disregard depth values or point cloud points that are further away than a predetermined cutoff distance r₀ from the sensor 211, where the measurements may be less reliable. The cutoff distance may be of the order of meters or tens of meters. For a sensor 211 with a frustum C(S) as wide as illustrated in FIG. 4, a cutoff distance of r₀=8 m will exclude depth values only when the sensor's 211 frustum is rotated (pitched) upwards from the position shown, so that it approaches a horizontal orientation where the upper boundary of the frustum reaches beyond the line (X,Z)=(8,0).

[0039] For the RGB-D sensor, the first predetermined model M1 (positive evaluation in substep 132) may be defined as follows in one embodiment. It is assumed that the RGB-D sensor is located at (X_a, Y_a, Z_a) and a physical object is detected at (X_b, Y_b, Z_b), corresponding to a cell with grid indices (x_b, y_b). Detecting a physical object with an RGB-D sensor may correspond to obtaining a depth (D) value less than the cutoff distance r₀. The new probability is given by

[0040] $M{1}_{\mathrm{RGB}}:p_{\mathrm{new}}\left(x,y\right)=\{\begin{array}{cc}0.8& \mathrm{if}\left(x,y\right)=\left(x_b,y_b\right)\\ p_1\left(x,y\right)& \mathrm{if}\left(x,y\right)\in L_{\mathrm{ab}}\end{array}$
where L_ab is the set of grid indices of all cells intersecting the line segment from (X_a, Y_a, 0) to (X_b, Y_b, 0) on the substrate 220, and the function p₁ may be defined as

[0041] $p_1\left(x,y\right)=\frac{1}{2}\left(1-\frac{❘\left\{\left(X,Y,Z\right):X=x,Y=y,Z\ge 0\right\}\bigcap C\left(S\right)❘}{Z_a}\right).$
Here, |⋅| denotes measure, and the point set {(X, Y, Z):X=x, Y=y, Z≥0}∩C(S) is clearly a vertical line segment contained inside the frustum C(S) above the grid cell (x, y). Accordingly, the second term represents the ratio between the local vertical thickness of the frustum C(S) and the sensor height Z_a. Direct inspection shows that p₁ will tend to the neutral value 0.5 in incompletely covered grid cells and decrease to zero (i.e., cell is likely free) for grid cells well covered by the frustum C(S). The dependence on the thickness is linear.

[0042] In a discretized representation, where the three-dimensional navigation space is divided into cubic cells (voxels) of length w_res, the function p₁ may be implemented as follows:

[0043] $p_1\left(x,y\right)=\frac{1}{2}\left(1-\frac{w_{\mathrm{res}}}{Z_a}\sum _{t=z_{\min }\left(x,y\right)}^{z_{\max }\left(x,y\right)}1\right)$
where z_min(x, y) is the height index of a lowest cubic cell inside the frustum C(S) at grid indices (x, y) and Z_max(x, y) is the height index of a highest cubic cell inside the frustum C(S). Accordingly, the summation counts the number of cubic cells contained inside the frustum at grid indices (x, y) in the vertical dimension. This number is compared to the number Z_a/w_res of stacked cubic cells needed to reach the sensor height Z_a, in the same manner as described in the preceding paragraph.

[0044] In a variation of this embodiment, the constant value P(x_b, y_b)=0.8 may be modified by a correction term or correction factor which accounts for a measuring uncertainty. The measuring uncertainty may vary across different regions of the substrate 220 or depending on conditions in which the RGB-D measurement was made.

[0045] In a further variation, the numerator in the expression

[0046] $\frac{❘\left\{\left(X,Y,Z\right):X=x,Y=y,Z\ge 0\right\}\bigcap C\left(S\right)❘}{Z_a}$
may be extended by a weighting operating that gives relatively more weight to positions closer to the substrate 220. This reflects an assumption that the expected obstacles have a height variation, so that the information gained by searching the navigation space at lower height (smaller Z) is relatively more useful for obstacle avoidance than by searching higher up. The dependence on the thickness is nonlinear.

[0047] Still referring to an RGB-D sensor, the second predetermined model M2 (negative evaluation in substep 132) may be defined as follows in one embodiment. It is assumed that the RGB-D sensor is located at (X_a, Y_a, Z_a) and oriented in the angle of incidence S and that no physical object is detected. Then,
M2_RGB:p_new(x,y)=p₁(x,y)
for all cells in a solid angle of incidence S and with p₁ defined (and implemented) in the manner described above. Accordingly, the new occupancy probability in cells in a solid angle of incidence S shall be in a positive relation to a local vertical thickness of the solid angle, i.e., a thickness of the frustum C(S) for that angle S.

[0048] As explained above, the new probability value p_new may be combined with an existing one p_old into a merged value p_merged which is assigned as the occupancy probability of a cell in the occupancy grid.

[0049] With reference now to FIG. 5, radar measurements, in particular three-dimensional millimeter-wave and ultra-wideband radar measurements, will be discussed next. Radars of these respective types may operate anywhere in the millimeter-wavelength range, such as around 60 GHz or around 80 GHz, and in the ultra-wideband range, such as around 24 GHz. FIG. 5 shows a frustum C(S) of a radar sensor 211, which is mounted to a chassis of the robot 210 with an approximately horizontal orientation. While the figure includes cartesian coordinate axes (X, Y, Z), some of the calculations to be presented refer to a local system in polar coordinates (r, θ, φ), where r is the distance from the sensor 211 and the line (θ, φ)=(0,0) corresponds to the center line of the radar's field of view at each time. Transformations between the local sensor system and the cartesian system are possible if the angle of incidence S is known; equivalence under such transformation is denoted (X, Y, Z)˜(r, θ, φ)_s. Similarly, a transformation between the local sensor system and grid indices is possible as long as the angle of incidence S is known, and it will be written, as the case may be, (x, y)˜(r, θ, φ)_s or (x, y, z)˜(r, θ, φ)_s.

[0050] When the robot-mounted sensor 211 is or includes a radar, the first predetermined model M1 may include the following operations. If a radar reflection is detected above a cell with grid indices (x_b, y_b), then the new occupancy probability is given by

[0051] $M{1}_{\mathrm{radar}}:p_{\mathrm{new}}\left(x_b,y_b\right)=P_{\mathrm{fa}}^{\frac{1}{1+\mathrm{SNR}}}$
where 0<P_fa<<1 and SNR denotes the signal-to-noise ratio (SNR) or signal-to-interference-plus-noise ratio (SINR) of that measurement. One may for example set P_fa=10⁻³. The SNR and SINR are inversely proportional to the noisiness of the signal. The first predetermined model M1 for the radar case produces a new probability only for the grid cells where the reflection is detected somewhere in the field of view.

[0052] Turning to the case where the radar detects no reflection, an assignment of the occupancy probability similar to the one described for the RGB-D sensor is used. For the radar case, the inventor has realized that the end accuracy is improved if a pre-correction factor P(x, y, z) is introduced:

[0053] $M{2}_{\mathrm{radar}}:p_{\mathrm{new}}\left(x,y\right)=p_2\left(x,y\right)=\frac{1}{z_{\max }\left(x,y\right)-z_{\min }\left(x,y\right)}\sum _{t=z_{\min }\left(x,y\right)}^{z_{\max }\left(x,y\right)}P\left(x,y,t\right)$ $\mathrm{where}$ $P\left[\left(x,y,z\right)~{\left(r,\theta ,\varphi \right)}_S\right]=\frac{1}{2}-\left(\frac{1}{2}-P_{\mathrm{fb}}\right)\exp \left(-\frac{1}{2}\left(\frac{r^2}{r_0^2}+\frac{{\theta }^2}{{\theta }_0^2}+\frac{{\varphi }^2}{{\varphi }_0^2}\right)\right)$
with constants r₀, θ₀, φ₀, P_fb. For cells on the center line of the radar's field of view, the pre-correction factor P(x, y, z) is equal to P_fb, while for cells in off-center positions, where the measurement is less reliable, the factor P(x, y, z) will tend to the neutral value 0.5. In other words, the pre-correction factor P(x, y, z) tends to maintain the new occupancy probability in off-center cells closer to the neutral probability. The expression P₂(x, y) may be understood as an average over the cubic grid cells at indices (x, y) which lie inside the frustum C(S). Cubic grid cells that are off-center will contribute neutral or near-neutral probability values to the average; if this holds for the majority of the grid cells, the occupancy status of the corresponding grid cell will remain uncertain.

[0054] The constants r₀, θ₀, φ₀ define the useful field of view (energy lobe) of the radar, in a similar manner as the cutoff distance r₀ for the RGB-D sensor, though with a gradual decrease rather than a sharp cutoff. The constants may be assigned in view of known characteristics and performance of the sensor hardware to be used, as may be read from a manufacturer's datasheet. Example values for a 60 GHz sensor, which is commercially available at the priority date and has an approximate range of 10 m, are r₀=7 m, B₀=40°, φ₀=40°.

[0055] The constant P_fb>0 may be set to a value less than 0.5 and may reflect the perceived certainty of the radar measurement. In a use case where the millimeter-wave or UWB radar is operated alongside with another measuring principle, the liberty of assigning an exact value to P_fb may be utilized for relative tuning of the two measuring principles, to render these consistent as data sources. In the concrete case of the RGB-D sensor discussed above, evaluating the discretized implementation of M1_RGB for w_res=0.05 m and Z_a=1.3 m gives p_new(x, y)=0.4808. Accordingly, one may set P_fb=0.4808.

[0056] The predetermined first and second models M1 and M2 may be expressed as models which are adaptive in dependence of the measuring principle. In the use case where an RGB-D sensor is fused with a radar sensor, the first predetermined model becomes:

[0057] $M1:p_{\mathrm{new}}\left(x,y\right)=\{\begin{array}{ccc}0.8& \mathrm{for}\mathrm{RGB}-D& \mathrm{if}\left(x,y\right)=\left(x_b,y_b\right)\\ P_{\mathrm{fa}}^{\frac{1}{1+\mathrm{SNR}}}& \mathrm{for}\mathrm{radar}& \mathrm{if}\left(x,y\right)=\left(x_b,y_b\right)\\ p_1\left(x,y\right)& \mathrm{for}\mathrm{RGB}-D& \mathrm{if}\left(x,y\right)\in L_{\mathrm{ab}}\end{array}$
with the notation introduced above. Similarly, the second predetermined model can be written:

[0058] $M2:p_{\mathrm{new}}\left(x,y\right)=\{\begin{array}{ccc}{p}_1\left(x,y\right)& \mathrm{for}\mathrm{RGB}-D& \mathrm{if}\left(x,y\right)\in C\left(S\right)\\ p_2\left(x,y\right)& \mathrm{for}\mathrm{radar}& \mathrm{if}\left(x,y\right)\in C\left(S\right)\end{array}$
Here, the condition (x, y)∈C(S) states that the expressions are valid only for cells lying in the solid angle of incidence S, i.e., in the frustum C(S) for that angle S.

[0059] The aspects of the present disclosure have mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.