TELEPRESENCE CONTROL SCHEMES FOR HAZARDOUS ENVIRONMENTS

[0001] The present application claims priority to and the benefit of U.S. Provisional Patent Applications No. 63/130,004, filed on Dec. 23, 2020, the entirety of which is incorporated herein by reference.

[0002] The transmission of SARS-CoV-2, the virus that causes COVID-19, may be worsened by the ability of the virus to survive on surfaces for days after an infected individual was present in the area. The specific proportion of those cases that were acquired through surfaces or through aerosol dispersion in the air may not be known. Research studies investigating the ability of SARS-CoV-2 to survive on various materials have demonstrated that the virus can remain active for up to 2 to 3 days on common surfaces, such as metals, plastic, and cardboard.

[0003] As a result, contamination with the virus, as well as its resulting disease COVID-19, can continue to occur for days after an infected person has been present in a given location. Surface disinfection procedures may be therefore necessary to eliminate contamination and reduce the spread of the virus. According to the World Health Organization, sanitation protocols must be performed meticulously at least once per day. Manual cleaning procedures can pose a risk to those performing them. Additionally, performed alone, they are less effective than when combined with automated sanitation protocols.

[0004] The present disclosure provides a semi-autonomous, remotely controlled robotic disinfection system that may combine the dynamic nature of manual cleaning with the meticulousness of an automated system. In an example, a system may include a mobile robot, a first imaging device configured to inspect an area surrounding the mobile robot, a UV disinfection mechanism attached to the mobile robot, and a telepresence control mechanism configured to control the mobile robot. The UV disinfection mechanism may be configured to disinfect the area surrounding the mobile robot. The telepresence control mechanism may be configured to allow a user of the mobile robot to control the mobile robot remotely. The mobile robot may have a mapping functionality and a navigation functionality.

[0005] In some examples, the first imaging device of the mobile robot may include a scanner configured to provide the mapping functionality by mapping the area surrounding the mobile robot.

[0006] In some examples, the first imaging device of the mobile robot may include a camera configured to provide a 360° view of the area surrounding the mobile robot.

[0007] In some examples, based on information from the first imaging device, the mobile robot may be configured to determine a potential target cleaning location.

[0008] In some examples, responsive to determining a presence of a person in the area surrounding the mobile robot, the system may be configured to turn off a UV light from the UV disinfection mechanism.

[0009] In some examples, the UV disinfection mechanism may include a UV lamp configured to emit a short-wave UV light having a wavelength in a range of 100 nm to 280 nm.

[0010] In some examples, the UV disinfection mechanism may be configured to eliminate over 99% of SARS-CoV-2 virus in a disinfected area.

[0011] In some examples, the navigation functionality of the mobile robot may be configured to drive the mobile robot autonomously.

[0012] In some examples, responsive to determining that a charging state of the mobile robot is lower than a predetermined value, the mobile robot may be configured to autonomously move to a power dock for charging.

[0013] In some examples, the telepresence control mechanism may include a user controller configured to transmit a control instruction to the mobile robot remotely.

[0014] In some examples, the user controller is configured to detect a user gesture via a second imaging device in the user controller and translate the user gesture into the control instruction.

[0015] In some examples, the user controller may be configured to detect a voice command and translate the voice command into the control instruction.

[0016] In some examples, the user controller may be configured to provide a virtual control icon to the user, receive a user command through the virtual control icon, and translate the user command into the control instruction.

[0017] In some examples, the virtual control icon may include a virtual map including a plurality of predefined target locations, and the control instruction comprises an instruction to move the mobile robot to one of the plurality of predefined target locations.

[0018] In some examples, the user controller may be configured to provide a teleportation portal to the user, receive a user command through the teleportation portal, and translate the user command into the control instruction.

[0019] In some examples, the teleportation portal may include a predefined target location. The control instruction may include an instruction to move the mobile robot to the target location.

[0020] In other examples, a system may include a mobile robot, a first imaging device configured to inspect an area surrounding the mobile robot, a UV disinfection mechanism attached to the mobile robot, and a telepresence control mechanism configured to control the mobile robot. The UV disinfection mechanism may be configured to disinfect the area surrounding the mobile robot. The telepresence control mechanism may include a wearable user controller configured to transmit a control instruction to the mobile robot remotely. The mobile robot may have a mapping functionality and a navigation functionality.

[0021] The advantages discussed herein may be found in one, or some, and perhaps not all of the embodiments disclosed herein. Additional features and advantages are described herein, and will be apparent from the following Detailed Description and the figures.

[0029] As of Oct. 2020, there are around 129,000 Covid-19 cases recorded in Qatar and 39.3 million cases worldwide. The proportion of those cases that were acquired through surfaces or through aerosol dispersion in the air may not be known. However, there is increasing scientific evidence that ultraviolet (UV) light type C can deactivate SARS-CoV-2 virus.

[0030] Aspects of the present disclosure may provide a semi-autonomous, remotely controlled robotic disinfection system that combines the dynamic nature of manual cleaning with the meticulousness of an automated system. The system may use ultraviolet light (UV-C) technology. UV-C technology may have a 99.8% kill rate against bacterial and viral strains, and may not have lingering side effects as compared to commonly used chemical disinfectants.

[0031] Aspects of the present disclosure may be in the domain of telepresence control. Telepresence may refer to devices or machines that will allow users to work remotely in another room or another location. With telepresence, the user may feel that they are present in another location even if they are not. Further, telepresence control may prevent UV-C light from harming the human eyes and skin by distancing the user from the system when the UV-C light is activated.

[0032] UV light disinfection robots may be one of the emerging and successful of robotics. Aspects of the present disclosure may be a semi-automated disinfection approach utilizing robotics and UV light to eliminate surface contamination in public spaces. A telepresence system may be integrated into a working autonomous UV-C robot that may allow a user to navigate potentially hazardous environments. The robotic system may include a mobile robot, the robot's mapping and navigation capability, a UV disinfection system, and a telepresence control system.

[0033] With the addition of a telepresence system, a safe method can be achieved for the effective disinfection of public spaces that are otherwise difficult to thoroughly sanitize. By doing so, aspects of the present disclosure may provide a solution to deal with the infectious nature of COVID-19, as well as other transmittable pathogen-borne diseases.

[0034] Referring to FIGS. 1 to 3, in some examples, a system 100 according to the present disclosure may include a mobile robot 110 and a UV disinfection system/mechanism 120 attached to the mobile robot 110. The UV disinfection mechanism 120 may be configured to disinfect the area surrounding the mobile robot 110.

[0035] In some examples, the mobile robot 110 may be an automated guided vehicle. In some examples, the mobile robot 110 may have a size of 620×440×330 mm³ and a payload of 100 kg, and a speed of 1.2 m/s. In other examples, the mobile robot 110 may have any other suitable size, payload, and speed.

[0036] In some examples, the mobile robot 110 may have a mapping functionality and a navigation functionality. In some examples, the system 100 may further include an imaging device configured to inspect an area surrounding the mobile robot 110. In some examples, the imaging device of the mobile robot 110 may include a camera 140. The camera 140 may provide a 360° view of the area surrounding the mobile robot 110. In some examples, the camera 140 may be disposed on the top surface of the UV disinfection mechanism 120. In other examples, the camera 140 may be disposed in any other suitable location in the system. In some examples, the imaging device may provide the mapping functionality by mapping the area surrounding the mobile robot 110. In some examples, the imaging device of the mobile robot 110 may further include a scanner (e.g., a laser scanner) configured to provide the mapping functionality by mapping the area surrounding the mobile robot 110.

[0037] In some examples, based on information from the imaging device, the mobile robot 110 may determine a potential target cleaning location. In some examples, the system 100 (e.g., the mobile robot 110) may determine a presence of a person in the area surrounding the mobile robot 110. Responsive to determining the presence of a person in the area surrounding the mobile robot 110 and/or near the mobile robot 110 (e.g., within a predetermined distance), the system 100 may turn off a UV light from the UV disinfection mechanism 120.

[0038] In some examples, the UV disinfection mechanism 120 may include UV lamps configured to emit a UV light. The UV light from the UV lamps may be a short-wave UV light having a wavelength in a range of 100 nm to 280 nm. The UV disinfection mechanism 120 may eliminate over 99% of SARS-CoV-2 virus in an area disinfected by the UV disinfection mechanism 120. For example, with 5 mJ/cm² dose of UV-C light for 6 seconds, it has been found that there was 99% reduction of the SARS-CoV-2 virus.

[0039] In some examples, the UV disinfection mechanism 120 may include a low-pressure mercury vapor ultraviolet (UV) type C light source. In some examples, four UV lamps (e.g., Philips, TUV T8) may be installed in the mobile robot 110. In other examples, any other suitable number of UV lamps (e.g., 1, 2, 3, 5, 6, 7, 8, 9, 10, and so on) may be installed in the mobile robot 110. In some examples, the UV lamp may emit short-wave UV radiation with a peak at 253.7 nm in the spectrum of UV type C wavelength. In some examples, the short-wave UV radiation of the the UV lamp may have any other suitable peak value in the spectrum of UV type C wavelength. In some examples, these lamps may be used in industrial water and air disinfection units.

[0040] In some examples, the navigation functionality of the mobile robot 110 may allow the mobile robot 110 to be driven autonomously. For example, using the navigation functionality, the mobile robot 110 may be automatically driven according to a preprogramed algorithm.

[0041] In some examples, the system 100 may further include a power dock 130. Responsive to determining that a charging state of the mobile robot 110 is lower than a predetermined charging value (e.g., 10%, 20%, or 30%), the mobile robot 110 may autonomously move to the power dock 130 for charging.

[0042] In some examples, the system 100 may further include a telepresence control system/mechanism configured to control the mobile robot 110. The telepresence control mechanism may allow a user of the mobile robot to control the mobile robot 110 remotely. In some examples, the telepresence control mechanism may include one or more user controllers 210, 220, and 230. In some examples, as illustrated in FIGS. 2 and 3, the user controllers 210, 220 may be a hand controller, and the user controller 230 may be a virtual reality headset. The one or more user controllers may transmit a control instruction to the mobile robot 110 to control the mobile robot 110 remotely.

[0043] In some examples, the telepresence control mechanism may include multiple telepresence modalities. As discussed in detail below, the multiple telepresence modalities/modes may use a virtual reality head-mounted display (e.g., in the controller 230). Each of the multiple telepresence modalities/modes may use different control schemes, which may include the use of (i) the controllers 210, 220; (ii) hand gestures; (iii) voice commands; (iv) virtual icons; and (v) teleportation portals. Each of these telepresence modalities/modes will be discussed in detail below.

[0044] The first mode may use the user controllers 210, 220 to control the mobile robot 110. As illustrated in FIG. 2, the user controllers 210, 220 may be a hand controller (e.g., one for the left hand, and the other one for the right hand). Each of the user controllers 210, 220 may include various control switches/buttons. For example, the user controller 210 (e.g., for the left hand) may include a thumbstick 220, button Y 230, button X 240, button menu 250, and trigger 260. Similarly, the user controller 220 (e.g., for the right hand) may include a thumbstick 221, button B 231, button A 241, button menu 251, and trigger 261. The user controllers 210, 220 may receive user commands through these various switches/buttons, generate a control instruction based on the user commands, and transmit the control instruction to the mobile robot 110 to control the mobile robot 110 remotely. Each of the user controllers 210, 220 may also include a grip 270, 271 on the inner side of the user controllers 210, 220 to help the user hold the user controllers 210, 220.

[0045] The user commands may be input into the controllers 210, 220 by physically pressing the buttons in the controllers 210, 220 or a combination of multiple buttons to activate commands to operate the mobile robot 110. Examples of such commands may include: Movement (forward/backward); Turn (Left/Right); Increase/Decrease velocity; Turn On/Off the UV lights.

[0046] The second mode may use the user's gestures (e.g., hand gestures). As illustrated in FIG. 4, in some examples, the user controller 230 may detect a user gesture. For example, the user controller 230 may detect the user gesture via one or more imaging devices 231, 232, 233, 234 in the user controller 230 and translate the user gesture into the control instruction to be transmitted to the mobile robot 110. In some examples, the one or more imaging devices 231, 232, 233, 234 may be a camera. The one or more imaging devices 231, 232, 233, 234 of the user controller 230 may detect and scan the surrounding of the user and send a visual feedback to avoid accidental hits and bumps while the user is fully immersed in a virtual world. The one or more imaging devices 231, 232, 233, 234 of the user controller 230 may also track the user's gestures (e.g., user's hands and finger articulations), accurately map the gestures (e.g., hands' bones) to virtual ones, and display a digital replica of them in the virtual world.

[0047] Using hand gestures to control the mobile robot 110 and its functions may be achieved by using pre-recorded specific gestures of the hands corresponding to tasks needed to be performed by the robot 110. The action may be performed once the user's gesture (e.g., hand gesture) matches the pre-recorded gesture. Some examples of the gestures may include: Right hand, thumb up: Move forward; Right hand, thumb down: Move backward; Left hand, middle and index fingers extended (V sign): turn on UV lights. This command scheme may not require the usage of the controllers 210, 220, but can complement the controllers 210, 220, for example, in case there is a loss of battery of the controllers 210, 220 during mission critical tasks.

[0048] The third mode may use the user's voice commands. In some examples, one or more user controllers 210, 220, 230 may detect a voice command 250 and translate the voice command into the control instruction. For example, as illustrated in FIG. 5, the user controller 230 may detect a voice command 250 and translate the voice command into the control instruction. In some examples, a microphone 236 may be integrated inside the user controller 230. The user's voice command can be used to trigger predefined actions for each vocal command. It is beneficial to have this feature as an alternate to the hand controller 210, 220, for example, in case the hand controller 210, 220 loses battery power. Some examples of the voice/verbal commands may include: Directional commands: “Forward”, “Back”, “Turn Left”, “Turn Right”, “Stop”; UV light activation: “Light On”, “Light Off.”

[0049] The fourth mode may include virtual icons. As illustrated in FIG. 6, in some examples, the user controller 230 may provide one or more virtual control icons to the user, for example, through the head-mounted display in the user controller 230. In some examples, the one or more virtual control icons may include four sections. The first virtual section 310 may include an auto mode turn-on button and a target selection button. Next to the target selection button, a virtual map 315 may be provided. The virtual map 315 may be a map showing the area surrounding the mobile robot 110. The virtual map 315 may be generated using the information from the imaging device (scanner and/or camera 140) of the mobile robot 110. With these virtual icons (the target selection button and/or the map 315), the user can select several predefined target locations to move the mobile robot 110 to the selected location.

[0050] The second virtual section 320 may include an on/off button, which may turn on/off the mobile robot 110. The third virtual section 330 may include a UV lights button, an automatic navigation button, a manual navigation button, an emergency stop button, and a help button. The UV lights button, when selected, may turn on/off the UV light of the UV disinfection mechanism 120. When the automatic navigation button is selected, the mobile robot 110 may drive in an automatic navigation mode (e.g., according to a random or preprogramed algorithm) without the specific instructions from the user. When the manual navigation button is selected, the mobile robot 110 may drive in a manual navigation mode by receiving specific instructions from the user through the controllers 210, 220, 230. When the emergency stop button is selected, the mobile robot 110 and/or the UV disinfection mechanism 120 may turn off right away. When the help button is selected, the controller 230 may make a call to a local emergency agency (e.g., by dialing 911) or someone who can connect the user to the local emergency agency for an emergency situation.

[0051] The fourth virtual section 340 may include direction buttons: forward, backward, left, and right. The mobile robot 110 may move according to the selected direction buttons. The fourth virtual section 340 may further include a plus button and a minus button. These plus/minus buttons may be used to control other parameters of the robot and/or the UV disinfection mechanism 120 (e.g., speed of the robot 110, wavelength of the UV light from the UV disinfection mechanism 120, etc.). In some examples, the virtual icons may appear near the controller 210 or 220 (e.g., on the display of the headset controller 230) as shown in FIG. 6. In other examples, the virtual icons may appear in any other suitable location on the display of the headset controller 230.

[0052] In some examples, the system 100 may receive a user command through the one or more virtual control icons, and translate the user command into the control instruction to be transmitted to the mobile robot 110. For example, the control instruction may include an instruction to move the mobile robot to the target location (e.g., the location selected on the map 315). Additional virtual buttons similar to the buttons on the controllers 210, 220, which are described with respect to FIG. 2, can be added to have a larger control with a virtual dashboard.

[0053] The fifth mode may use a teleportation portal. As illustrated in FIG. 7, in some examples, the user controller 230 may provide teleportation portals to the user. The teleportation portals may be projected in augmented reality. In some examples, the user controller 230 may receive a user command through the teleportation portal, and translate the user command into the control instruction. In some example, the teleportation portal may include a predefined target location. For example, using the teleportation portals, the user may be able to point and select any point (e.g., point 410) in the navigable area within the camera's point of view to move the mobile robot 110 to the selected point. The telepresence control mechanism may project the target destinations on the user's headset (e.g., controller 230) as selectable “teleportation portals” to simplify navigation control. The control instruction may include an instruction to move the mobile robot to the target location. For example, the selection of a target destination may trigger the robot's autonomous navigation system, which may allow the robot 110 to maneuver the environment to reach its destination.

[0054] In some examples, the first to third modes may be useful for undefined and detailed locations for navigation. The fourth and fifth modes include predefined locations and, thus, these modes may be useful for deploying the robot 110 in large public spaces, such as in shopping malls.

[0055] It will be appreciated that each of the systems, structures, methods and procedures described herein may be implemented using one or more computer program or component. These programs and components may be provided as a series of computer instructions on any conventional computer-readable medium, including random access memory (“RAM”), read only memory (“ROM”), flash memory, magnetic or optical disks, optical memory, or other storage media, and combinations and derivatives thereof. The instructions may be configured to be executed by a processor, which when executing the series of computer instructions performs or facilitates the performance of all or part of the disclosed methods and procedures.

[0056] In the descriptions above and in the claims, phrases such as “at least one of” or “one or more of” may occur followed by a conjunctive list of elements or features. The term “and/or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and B;” “one or more of A and B;” and “A and/or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and/or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.

[0057] The subject matter described herein can be embodied in systems, apparatus, methods, and/or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and/or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several further features disclosed above.

[0058] The drawing figures are not necessarily to scale. Certain features and components herein may be shown exaggerated in scale or in somewhat schematic form and some details of conventional elements may not be shown in interest of clarity and conciseness.

[0059] The terminology used herein is intended to describe particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless otherwise indicated. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the terms “about,” “approximately,” “substantially,” “generally,” and the like mean plus or minus 10% of the stated value or range.

[0060] It should be understood that various changes and modifications to the example embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present subject matter and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims. Moreover, consistent with current U.S. law, it should be appreciated that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, paragraph 6 is not intended to be invoked unless the terms “means” or “step” are explicitly recited in the claims. Accordingly, the claims are not meant to be limited to the corresponding structure, material, or actions described in the specification or equivalents thereof.