Mobile robot map construction method and apparatus, and computer device and storage medium

[0001] The present invention relates to the technical field of robot map construction, in particular to a mobile robot map construction method and apparatus, computer device and storage medium.

[0002] At present, intelligent mobile robots are increasingly widely used in various industries and/or in all walks of life. With the advent of the intelligent era, it is urgently necessary for robots to construct environment maps involving multi-floor complex scenes efficiently and accurately without human intervention.

[0003] At present, a multi-floor environment map is usually constructed by making robots traverse the whole floor by trial and error to realize constructing the floor environment map; and then the robot is manually transferred to the next floor to complete constructing the environment map of the next floor. It takes a long time and is inefficient to traverse the floors by trial and error. However, an intervention with an operator is necessary for manually transfer the robot to the next floor, therefore, the robot could not be automatically transferred between floors. In addition, for the position where there is glass in the floor, the prior art treats the glass in the floor as an ordinary obstacle when constructing the floor environment map, and ignores a possible region behind the glass that may be necessary to construct the environment map.

[0004] Therefore, the prior art does not have a complete solution for constructing an environment map with multi-floor scenes and glass interferences in floors, and thus improvements are needed.

[0005] In view of the above, it is necessary to provide a mobile robot map construction method and apparatus, computer device and storage medium to solve the above problems.

[0006] The embodiment of the present disclosure is realized as follows: a mobile robot map construction method, comprising steps of:

• • [0007] acquiring a number of floors and current floor information;
  • [0008] acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map and labeling a position of glass according to the point cloud data and the image data, wherein the environment map comprises an obstacle region, a passable region, and a region to be detected, wherein the position of glass is located in the passable region;
  • [0009] determining whether there is a region to be detected in the environment map, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule; and
  • [0010] if there is no region to be detected, moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0011] In one embodiment, the present disclosure further provides a mobile robot map construction apparatus, comprising:

• • [0012] an acquisition module configured for acquiring a number of floors and current floor information;
  • [0013] a map construction module configured for acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map and labeling a position of glass according to the point cloud data and the image data, wherein the environment map comprises an obstacle region, a passable region and a region to be detected, wherein the position of glass is located in the passable region;
  • [0014] a determination module configured for determining whether there is a region to be detected in the environment map, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule; and
  • [0015] a transfer module configured for, if there is no region to be detected, moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0016] In one embodiment, the present disclosure further provides a computer device, comprising a memory and a processor, wherein the memory stored a computer program therein, and the computer program, when executed by the processor, causes the processor to execute the steps of the above mobile robot map construction method.

[0017] In one embodiment, the present disclosure further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, causes the processor to execute the steps of the above mobile robot map construction method.

[0018] The mobile robot map construction method provided by the embodiment of the present disclosure constructs an environment map by utilizing the point cloud data and the image data. And glass features can be identified through comprehensive processing of the point cloud data and the image data. Thus, the problem of inaccurate identification of glass features in the prior art is overcome. In addition, the mobile robot map construction method provided by the present disclosure enables the robot to take the elevator to realize automatic cross-floor detection, and this process can realize automatic cross-floor map construction without manual intervention, with high intelligence.

[0027] In order to make the purpose, technical solution, and advantages of the present disclosure clearer, the present disclosure will be further described in detail with the attached drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0028] It is understood that the terms “first” and “second” used in this application can be used to describe various elements herein, but these elements are not limited by these terms unless otherwise specified. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of this application, a first xx script may be called a second xx script, and similarly, a second xx script may be called a first xx script.

[0029] FIG. 1 is an application environment diagram of a mobile robot map construction method provided in one embodiment. As shown in FIG. 1, in this application environment, a mobile robot 100 and a control device 200 are included.

[0030] The mobile robot 100 includes mobile components that can be used for moving, devices for data acquisition, such as a laser radar, a camera, etc.; in addition, it can also include an infrared sensor, a communication module, a voice recognition module, etc. This is an optional specific implementation, and the embodiment of the present disclosure does not specifically limit this. It should be understood that the hardware structure of the mobile robot 100 can be varied, and the method of the present disclosure mainly involves the algorithm of the mobile robot 100 in automatic map construction, and the specific hardware settings are not further limited.

[0031] In addition, a control device 200 may be included. The control device 200 can communicate with the mobile robot 100, which can be wired or wireless. Through the control device, the operator can operate the mobile robot 100. It should be noted that the operation of the operator may not involve the process of automatic map construction, but only be used for basic operations such as starting and stopping, and the mobile robot 100 can realize constructing cross-floor maps by implementing the map construction method provided by the embodiment of the present disclosure. The composition and specific form of the control device 200 are not specifically limited by the embodiment of the present disclosure.

[0032] As shown in FIG. 2, in one embodiment, a mobile robot map construction method is proposed, and this embodiment is mainly illustrated by applying this method to the mobile robot 100 in FIG. 1. Specifically, it can include steps of:

[0033] In Step S202, acquiring a number of floors and current floor information.

[0034] In the embodiment of the present disclosure, the current floor information includes, but not limited to, the total number of floors, the use of floors, the number and distribution of merchants and households in floors, etc., and such information is mainly used to assist constructing environment maps. In the present disclosure, the number of floors and the information of the current floor can be obtained by the operator input or the mobile robot through wireless communication, and in addition, it can be obtained from the corresponding signs in the floor through image and character recognition.

[0035] In Step S204, acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map according to the point cloud data and the image data and labeling a position of glass; the environment map comprises an obstacle region, a passable region, and a region to be detected, wherein the position of glass is located in the passable region.

[0036] In the embodiment of the present disclosure, the construction of the environment map needs to combine point cloud data and image data, and the recognition of glass features also needs to combine the processing results of point cloud data and image data. In the embodiment of the present disclosure, the environment map includes an obstacle region, a passable region, and a region to be detected. It should be noted that the three types of regions divided here do not necessarily exist in the constructed environment map at the same time, and the environment map may include one or more types of regions, which is only the division of the types of regions in the environment map by the present disclosure.

[0037] In Step S206, whether there is a region to be detected in the environment map is determined, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule.

[0038] In the embodiment of the present disclosure, before the detection of the current position is completed, there may be a region to be detected, so it is necessary to detect the region to be detected according to a pre-set rule.

[0039] In Step S208, if there is no region to be detected, the robot moves to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and takes the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0040] In the embodiment of the present disclosure, the elevator position on the map environment can be marked before or during constructing the environment map.

[0041] The mobile robot map construction method provided by the embodiment of the present disclosure constructs an environment map by utilizing the point cloud data and the image data. And glass features can be identified through comprehensive processing of the point cloud data and the image data, Thus the problem of inaccurate identification of glass features in the prior art is overcome. In addition, the mobile robot map construction method provided by the present disclosure enables the robot to take the elevator to realize automatic cross-floor detection, and this process can realize automatic cross-floor map construction without manual intervention, with high intelligence.

[0042] In one embodiment, as shown in FIG. 3, in step S204 of labeling the position of glass according to the point cloud data and the image data, specifically including steps S302-S310.

[0043] In Step S302, inputting the point cloud data into a pre-trained first neural network model, and outputting a probability p(x, y, z) that a coordinate point (x, y, z) belongs to glass.

[0044] In the embodiment of the present disclosure, the training method of the neural network module can refer to the prior art, and the embodiment of the present disclosure does not specifically limit this. It should be noted that the present disclosure uses the diffuse reflection of glass to make the reflection spectrum appear a specific shape, and it is determined whether an object is glass according to a similarity between the specific shape and the point cloud data collected on site.

[0045] In Step S304, inputting the image data into a pre-trained second neural network model, and outputting a probability f(x, y, z) that the coordinate point (x, y, z) belongs to glass.

[0046] In the embodiment of the present disclosure, the image data is also processed by the neural network model, and the specific training process of the neural network model is not specifically limited in the embodiment of the present disclosure.

[0047] In step S306, if p(x, y, z) and f(x, y, z) are both greater than or both less than a corresponding pre-set threshold and a comparison result is obtained, determining whether the position is glass according to the comparison result.

[0048] In the embodiment of the present disclosure, it should be noted that the pre-set thresholds corresponding to the two probabilities may be the same or different, and there is no direct relationship between them. According to the comparison result, it is determined whether the position is glass or not. Specifically, if the two probabilities are both greater than or both less than the corresponding pre-set threshold, it means that the determination results obtained by the two methods are the same and can be mutually confirmed, so it can be determined whether the position is glass or not.

[0049] In Step S308, otherwise, whether a comprehensive probability s(x, y, z)=a₁·p(x, y, z)+a₂·f(x, y, z) is greater than the pre-set threshold is compared and a comparison result is obtained, and determining whether the position is glass according to the comparison result;

[0050] In the embodiment of the present disclosure, different from the previous step, if the determination results by two probabilities are different and the two determination methods cannot be mutually confirmed, a comprehensive probability is determined by the above formula, and whether the target position belongs to the glass is determined by the relationship between the comprehensive probability and its pre-set threshold. This method makes up for a deficiency of an independent evaluation.

[0051] In Step S310, if the position is determined to be glass, the position of glass is labeled in the environment map;

• • [0052] where s(x, y, z) is the comprehensive probability of the coordinate point (x, y, z), a₁, a₂ are pre-set coefficients, p(x, y, z) is the probability that the position is glass calculated from the point cloud data, and f(x, y, z) is the probability that the position is glass calculated from the image data.

[0053] In the embodiment of the present disclosure, the position of glass is marked in the environment map, and the marking method can be annotative, for example, it can be explained by words, and the pre-set images and models of glass features can also be called and displayed in the environment map.

[0054] The mobile robot map construction method provided by the embodiment of the present disclosure uses point cloud data and image data to identify the glass, which can improve the accuracy of glass identification.

[0055] In one embodiment, as shown in FIG. 4, in step S206, the region to be detected is detected according to the pre-set rule, which may specifically include steps S402-S408.

[0056] In Step S402, selecting a position point on a boundary of the region to be detected with a lowest movement cost from the current position of a robot, as a path candidate position point; the movement cost comprises a steering cost, a straight-ahead cost, and a danger warning cost.

[0057] In the embodiment of the present disclosure, the movement cost is determined according to the following formula:
F(x)=α_d·d(x,O)+αω·ω(x,O)+α_g·g(x)

• • [0058] where F(x) is the movement cost of a x^th boundary position point of the region to be detected, α_d is a distance weight parameter pre-set by a system, d(x, O) is a linear distance between the x^th boundary position point of the region to be detected and the current position of the robot, a_ω is a steering weight parameter pre-set by the system, ω(x, O) is a steering angle of the x^th boundary position point of the region to be detected from the current position of the robot, a g is a danger warning weight parameter pre-set by the system, and g(x) is a nearest distance from the x^th boundary position point of the region to be detected to a boundary of a detected obstacle.

[0059] In Step S404, determining a reachable region, wherein that reachable region covers the path candidate position point, a center position of the reachable region is closest to a current position of the robot, and the reachable region can accommodate an outer contour of the whole robot with a safe distance.

[0060] In the embodiment of the present disclosure, the size of the safety distance can be set by itself.

[0061] In Step S406, determining whether the robot can reach the reachable region from the current position without colliding with obstacles and glass, and if so, the robot moves to the reachable region in an obstacle avoidance mode;

[0062] In the embodiment of the present disclosure, the determination process can be combined with the pre-constructed environment map, such as the hypothetical planning of the path in the constructed environment map.

[0063] In Step S408, otherwise, labeling the path candidate position point as a boundary point in the region to be detect in the glass, and selecting a new path candidate position point.

[0064] In one embodiment, as shown in FIG. 5, a new path candidate position point is selected in step S408, which may specifically include steps S502-S504.

[0065] In Step S502, determining whether a new path candidate position point and a reachable region can be selected so that the robot can reach the reachable region from the current position without colliding with glass and obstacles, and if so, the new path candidate position point and the reachable region are selected and the robot moves to the reachable region in an obstacle avoidance mode.

[0066] In Step S504, otherwise, selecting a position point which is on the glass boundary, is in a side close to the robot, and has a lowest movement cost from the current position of the robot, as the new path candidate position point, and the robot moves to the path candidate position point in an obstacle avoidance mode.

[0067] In the embodiment of the present disclosure, the determination of movement cost refers to the aforementioned embodiment, and the mode of avoiding obstacles can be realized by referring to the prior art, which is not specifically limited in the embodiment of the present disclosure.

[0068] In one embodiment, as shown in FIG. 6, before step S208, steps S602-S604 may be specifically included.

[0069] In Step S602, starting an elevator position sensing mode, identifying the elevator position according to the point cloud data or the image data, or sensing the elevator position by using an elevator sensor;

[0070] In the embodiment of the present disclosure, the elevator sensor can be used to sense the elevator position, specifically, the elevator is equipped with a near-field signal transmitter, and the robot is equipped with a near-field signal receiver, so that the mobile robot can sense the elevator by sending and receiving the near-field signal.

[0071] In Step S604, determining whether the elevator position can be marked in the detected passable region, and if so, marking the elevator position in the passable region.

[0072] In one embodiment, before the step of moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor in Step 208, the following steps S702-S708 are included.

[0073] In Step 702, when there are a plurality of undetected floors, labeling the undetected floors with {1, 2, . . . , R}, where R is a number of currently undetected floors, and {e₁, e₂, . . . , e_R} is a floor number difference between each undetected floor and the floor where the robot is currently located.

[0074] In Step 704, selecting an undetected floor corresponding to the smallest floor number difference, as a floor to be reached.

[0075] In Step 706, setting the current position as (x_o,y_o), and setting the numbers of the elevators in a current floor are set as {1, 2, . . . , N}, where N is a total number of elevators in the current floor, {d₁, d₂, . . . , d_N} is a path distance from the current position to the elevators/elevator in the current floor, and {t₁, t₂, . . . , t_N} is a time when a carriage of each elevator in the current floor reaches the current floor.

[0076] In Step 708, calculating a priority decision value of the robot taking elevators according to the following formula to determine a preferred elevator:

[0077] $Q_n=\frac{1}{\frac{d_n}{v}+t_n}$

• • [0078] where Q_n is a priority decision value for the robot to take a n^th elevator on the current floor, where n∈{1, 2, . . . , N}, d_n is a path distance from the current position of the robot to the n^th elevator on the current floor, v is a speed set for the robot, and t_n is a time when the carriage of the n^th elevator on the current floor reaches the current floor.

[0079] In the embodiment of the present disclosure, a priority decision of taking elevators is determined by the above formula, which can shorten the time to reach the undetected floor and reduce the movement cost of the mobile robot.

[0080] As shown in FIG. 7, in one embodiment, a mobile robot map construction apparatus is provided, which can be integrated in the above-mentioned mobile robot 100, and specifically includes:

• • [0081] an acquisition module 701 configured for acquiring a number of floors and current floor information;
  • [0082] a map construction module 702 configured for acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map and labeling a position of glass according to the point cloud data and the image data, wherein the environment map comprises an obstacle region, a passable region and a region to be detected, wherein the position of glass is located in the passable region;
  • [0083] a determination module 703 configured for determining whether there is a region to be detected in the environment map, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule; and
  • [0084] a transfer module 704 configured for, if there is no region to be detected, moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0085] In the embodiment of the present disclosure, please refer to any one or a combination of the above-mentioned embodiments for the explanation of the steps executed by the above-mentioned modules, and the embodiment of the present disclosure will not repeat them here.

[0086] The mobile robot map construction apparatus provided by the embodiment of the present disclosure constructs an environment map by utilizing the point cloud data and the image data, and can identify glass features through comprehensive processing of the point cloud data and the image data, thus overcoming the problem of inaccurate identification of glass features in the prior art; in addition, the mobile robot map construction method provided by the present disclosure enables the robot to take the elevator to realize automatic cross-floor detection, and this process can realize automatic cross-floor map construction without manual intervention, with high intelligence.

[0087] FIG. 8 shows an internal structure diagram of a computer device in one embodiment. The computer device can be a mobile robot 100 in FIG. 1. As shown in FIG. 8, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. The memory includes a nonvolatile storage medium and an internal storage. The nonvolatile storage medium of the computer device stores an operating system and a computer program. When the computer program is executed by the processor, the processor can implement the mobile robot map construction method provided by the embodiment of the present disclosure. The internal storage can also store a computer program, which, when executed by the processor, can make the processor execute the mobile robot map construction method provided by the embodiment of the present disclosure. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, a button, a trackball or a touchpad arranged on the shell of the computer device, and an external keyboard, touchpad or mouse.

[0088] It can be understood by those skilled in the art that the structure shown in FIG. 8 is only a block diagram of a part of the structure related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or less components than those shown in the figure, or combine some components, or have different component arrangements.

[0089] In one embodiment, the mobile robot map construction device provided by the present disclosure can be realized in the form of a computer program, and the computer program can run on a computer device as shown in FIG. 8. The memory of the computer device can store various program modules that make up the map construction device of the mobile robot, such as the acquisition module, the map construction module, the determination module and the transfer module shown in FIG. 7. The computer program composed of each program module causes the processor to execute the steps in the mobile robot map construction method of each embodiment of the application described in this specification.

[0090] For example, the computer device shown in FIG. 8 can execute step S202 through the acquisition module in the mobile robot map construction device shown in FIG. 7; the computer device may execute step S204 through the map construction module; the computer device can execute step S206 through the determination module; the computer device may execute step S208 through the transfer module.

[0091] In one embodiment, a computer device is proposed, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:

• • [0092] acquiring a number of floors and current floor information;
  • [0093] acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map and labeling a position of glass according to the point cloud data and the image data, wherein the environment map comprises an obstacle region, a passable region and a region to be detected, wherein the position of glass is located in the passable region;
  • [0094] determining whether there is a region to be detected in the environment map, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule; and
  • [0095] if there is no region to be detected, moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, causes the processor to perform steps of:

• • [0097] acquiring a number of floors and current floor information;
  • [0098] acquiring point cloud data of an environment around a current position via a laser radar, acquiring image data of the environment around the current position via a camera, and constructing an environment map and labeling a position of glass according to the point cloud data and the image data, wherein the environment map comprises an obstacle region, a passable region and a region to be detected, wherein the position of glass is located in the passable region;
  • [0099] determining whether there is a region to be detected in the environment map, and if there is a region to be detected, detecting the region to be detected according to a pre-set rule; and
  • [0100] if there is no region to be detected, moving to a nearest elevator in an obstacle avoidance mode according to an elevator position labeled on the environment map, and taking the elevator to an undetected floor until finishing constructing the environment map for all undetected floors.

[0101] It should be understood that although each step in the flowchart of each embodiment of the present disclosure is displayed in sequence as indicated by the arrow, these steps are not necessarily executed in sequence as indicated by the arrow. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include a plurality of sub-steps or stages, which may not necessarily be completed at the same time, but may be executed at different times, and the execution order of these sub-steps or stages may not necessarily be sequentially executed, but may be alternately or alternatively executed with other steps or at least a part of sub-steps or stages of other steps.

[0102] Those skilled in the art can understand that all or part of the processes in the method for realizing the above-mentioned embodiments can be completed by instructing related hardware through a computer program, which can be stored in a nonvolatile computer-readable storage medium, and when executed, the program can include the processes of the above-mentioned embodiments. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. The nonvolatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. The volatile memory may include random access memory (RAM) or external cache memory. By way of illustration without limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), and Direct Rambus RAM (DRDRAM), and Rambus dynamic RAM (RDRAM).

[0103] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction between the combinations of these technical features, they should be considered as the scope recorded in this specification.

[0104] The above-mentioned embodiments only express several embodiments of the present disclosure, and their descriptions are more specific and detailed, but they cannot be understood as limiting the patent scope of the present disclosure. It should be pointed out that for those skilled in the art, without departing from the concept of the present disclosure, a number of variations and improvements can be made, which are within the scope of protection of the present disclosure. Therefore, the scope of protection of the patent of this present invention should be based on the appended claims.