Position detection using directed light events

[0001] The present invention relates to a system, a position detecting device, a position detecting method, and a program.

[0002] There is known an event-driven type vision senor in which a pixel having detected a change in intensity of incident light generates a signal time-asynchronously. The event-driven type vision sensor is advantageous in that it is capable of operating at a high speed and with low power consumption, as compared with a frame type vision sensor, specifically, an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) which scans all pixels at intervals of a predetermined cycle. Techniques for such an event-driven type vision sensor are described in, for example, PTL 1 and PTL 2.

• • [0003] [PTL 1] JP 2014-535098T [PTL 2] JP 2018-85725A

[0004] Regarding the event-driven type sensor, however, although such advantages as described above are known, it is hard to say that peripheral techniques that take into consideration its characteristics different from those of a conventional vision sensor, for example, the frame type vision sensor, have been sufficiently proposed.

[0005] Thus, the present invention is intended to provide a system, a position detecting device, a position detecting method, and a program that are capable of detecting a position of an object in a space at a high speed and with accuracy by using an event-driven type vision sensor.

[0006] According to an aspect of the present invention, provided is a system including an event-driven type vision sensor that outputs an event signal indicating a two-dimensional position which is located within a field of view of the vision sensor and at which a light intensity change has occurred in a space, and a time of day at which the light intensity change has occurred, a light source that emits light having directivity toward the space, and a position detecting device including an event signal receiving section that receives the event signal, and a position detecting section that detects a three-dimensional position of the light intensity change on a basis of an angle at which the light source has emitted the light at the time of day, the two-dimensional position, and a positional relation between the vision sensor and the light source.

[0007] According to another aspect of the present invention, provided is a position detecting device including an event signal receiving section that, from an event-driven type vision sensor oriented toward a space, receives an event signal indicating a two-dimensional position which is located within a field of view of the vision sensor and at which a light intensity change has occurred in the space, and a time of day at which the light intensity change has occurred, and a position detecting section that detects a three-dimensional position of the light intensity change on a basis of an angle at which a light source that emits light having directivity toward the space has emitted the light at the time of day, the two-dimensional position, and a positional relation between the vision sensor and the light source.

[0008] According to still another aspect of the present invention, provided is a position detecting method including a step of receiving, from an event-driven type vision sensor oriented toward a space, an event signal indicating a two-dimensional position which is located within a field of view of the vision sensor and at which a light intensity change has occurred in the space, and a time of day at which the light intensity change has occurred, and a step of detecting a three-dimensional position of the light intensity change on a basis of an angle at which a light source that emits light having directivity toward the space has emitted the light at the time of day, the two-dimensional position, and a positional relation between the vision sensor and the light source.

[0009] According to yet another aspect of the present invention, provided is a program that causes a computer to function as a position detecting device including an event signal receiving section that, from an event-driven type vision sensor oriented toward a space, receives an event signal indicating a two-dimensional position which is located within a field of view of the vision sensor and at which a light intensity change has occurred in the space, and a time of day at which the light intensity change has occurred, and a position detecting section that detects a three-dimensional position of the light intensity change on a basis of an angle at which a light source that emits light having directivity toward the space has emitted the light at the time of day, the two-dimensional position, and a positional relation between the vision sensor and the light source.

[0010] According to the above-described configurations, a position of an object in a space can be detected at a high speed and with accuracy by using an event-driven type vision sensor.

[0017] Hereinafter, a preferred embodiment of the present invention will be described in detail referring to the accompanying drawings. Note that, in the present description and the drawings, for constituent elements having substantially the same functional configuration, duplicated descriptions will be omitted by denoting them by the same reference sign.

[0018] FIG. 1 is a diagram illustrating a schematic configuration of a system according to an embodiment of the present invention. In the illustrated example, a system 10 includes an event-driven type vision sensor 100, a light source 200, and a position detecting device 300. The vision sensor 100 includes a sensor array in which arrayed are asynchronous-type solid-state image sensors that each generate an event signal upon detection of an intensity change, more specifically, a luminance change, in light that is incident through an unillustrated optical system.

[0019] Here, the asynchronous-type solid-state image sensors included in the vision sensor 100 each include an address event representation (AER) circuit. The address event means that, at a certain pixel address, an amount of light of a pixel has varied and an amount of the variation thereof has exceeded a threshold value. Specifically, the address event includes an on-event indicating that an amount of light of a pixel has varied and has overrun a predetermined upper limit value, and an off-event indicating that an amount of light of a pixel has varied and has underrun a predetermined lower limit value. The event signal output from the vision sensor 100 includes a pixel address, a flag indicating the kind of the address event (the on-event or the off-event), and a time stamp. The pixel address included in the event signal indicates a two-dimensional position pos which is located within a field of view of the vision sensor 100 and at which a light intensity change has occurred in a space toward which the vision sensor 100 is oriented. Further, the time stamp indicates a time of day at which the light intensity change has occurred.

[0020] The light source 200 emits light having directivity toward the same space as the space toward which the vision sensor 100 is oriented. The light having directivity is, for example, infrared laser. For example, the light source 200 scans the space by emitting the light having directivity in the form of a line, and changing an angle of the emission in a time-series pattern in a direction intersecting with the line. In this case, the light source 200 changes the angle of the emission for each time of day that is synchronized with or can be matched with the time stamp of the event signal output by the vision sensor 100. The angle of the emission at each time of day is specified in advance to the light source 200 from the position detecting device 300 or is transmitted to the position detecting device 300 from the light source 200.

[0021] FIG. 2 is a block diagram illustrating a functional configuration of the position detecting device illustrated in FIG. 1. In the illustrated example of FIG. 2, the position detecting device 300 is implemented by a computer including, for example, a communication interface, a processor, and a memory, and includes functional sections of an event signal receiving section 310, a light source controlling section 320, and a position detecting section 330 that are implemented by causing the processor to operate according to a program stored in the memory or having been received via the communication interface.

[0022] Here, for example, the position detecting device 300 may be embedded in the same device as a device including the vision sensor 100 and the light source 200. Further, for example, the position detecting device 300 may be a terminal device that is disposed in the same space as a space in which the vision sensor 100 and the light source 200 are disposed and that communicates with each of the vision sensor 100 and the light source 200, or may be a server device that is coupled with the vision sensor 100 and the light source 200 via a network. Further, some of the functions of the position detecting device 300 may be implemented in the terminal device, and the other ones of the functions thereof may be implemented in the server device.

[0023] The event signal receiving section 310 receives the event signal from the vision sensor 100. As described above, the event signal is information indicating the two-dimensional position which is located within the field of view of the vision sensor 100 and at which the light intensity change has occurred in the space, and the time of day at which the light intensity change has occurred.

[0024] The light source controlling section 320 controls the light source 200 according to timing information 340 indicating an angle of the emission of the light at each time of day. Alternatively, in a case where the light source 200 transmits the angle of the emission of the light at each of time of day to the position detecting device 300, the light source controlling section 320 may not be provided.

[0025] The position detecting section 330 detects a three-dimensional position of the light intensity change having occurred in the space, on the basis of the event signal having been received by the event signal receiving section 310, the timing information 340 indicating the angle of the emission of the light by the light source 200 at each time of day, and position information 350 regarding the vision sensor 100 and the light source 200. In the present embodiment, the position detecting section 330 outputs the three-dimensional position of the detected light intensity change, as object position information 360.

[0026] The principle of the detection of an object existing in the space by the position detecting section 330 in the present embodiment will be described referring to FIG. 1 again. When light having been emitted by the light source 200 is reflected on an object obj existing in the space, a light intensity change due to reflection light occurs, and the vision sensor 100 detects an event. From a two-dimensional position pos within the field of view of the vision sensor 100 at the event, an angle θ₁ of the object obj relative to the vision sensor 100 as a reference can be identified. Further, as described above, an angle θ₂ of the emission of the light by the light source 200 at each time of day is known. Adding a positional relation between the vision sensor 100 and the light source 200 (which is illustrated as a distance L) to the angles θ₁ and θ₂ makes it possible to identify the three-dimensional position of the object obj according to the principle of triangulation.

[0027] As described above, in the present embodiment, the light source 200 emits light having directivity in the form of a line. Therefore, in a case where an event due to reflection light on the object obj has been detected at a certain time of day, it is identified that the object obj exists on a plane formed by the light in the form of a line, which is being emitted at the above time of day. In this case, the angle θ₁ identified from the two-dimensional position pos within the field of view of the vision sensor 100 at the event is a three-dimensional angle, and thus, a point at which, in a three-dimensional space, the plane formed by the above light intersects with a straight line identified by the angle θ₁ can be detected as the three-dimensional position of the object obj. Alternatively, in a case where it is assumed that the object obj is stationary, for example, changing an emission angle, which is an angle at which the light source 200 emits the light having directivity, in a time-series pattern including the change of the emission angle in each of two directions intersecting with each other allows two planes including the object obj to be identified from times of day at which the event has occurred in cases of the emissions of the light in the individual directions, and an angle θ₂ obtained as an angle of an intersection line of the above two planes can be identified as a three-dimensional angle. In this case, the three-dimensional position of the object obj is detected as the intersection point of two straight lines identified by the angles θ₁ and θ₂.

[0028] Further, as an additional configuration, as illustrated in FIG. 3, the light source 200 may emit light having directivity in the form of a line having a width. In this case, the event due to the reflection light on the object obj occurs twice with a time difference (the on-event and the off-event). In a case where two angles θ_{2_1} and θ_{2_2} corresponding to anterior and posterior edges of the light emitted at each time of day are known, the accuracy of the detection of the three-dimensional position can be enhanced by reducing error influence in a way that averages a three-dimensional position that is detected on the basis of an angle θ_{1_1} identified at the on-event and the angle θ_{2_1} of the anterior edge of the light, and another three-dimensional position that is detected on the basis of an angle θ_{1_2} identified at the off-event and the angle θ_{2_2} of the posterior edge of the light.

[0029] As another additional configuration, as illustrated in FIG. 4, the light source 200 may emit light having directivity in the form of a band having a further width. In this case, the light source 200 scans the space by changing the angle of the emission in a time-series pattern in a direction of the width of the band. In this case as well, the event due to the reflection light on the object obj occurs twice with a time difference (the on-event and the off-event). In this regard, however, the time difference is larger than that of the example of FIG. 3, and thus, in a case where, for example, the object obj is moving, the position of the object obj at the on-event is different from the position of the object obj at the off-event. Therefore, a first three-dimensional position that is detected on the basis of an angle θ₁ identified at a first event and an angle θ₂ of the anterior edge of the light and a second three-dimensional position that is detected on the basis of an angle θ₃ identified at a second event and an angle θ₄ of the posterior edge of the light are handled as positions indicating three dimensional positions of the object obj at mutually different times of day. That is, the first three-dimensional position and the second three-dimensional position are handled as time-series changes of the three-dimensional position. For example, as illustrated in FIG. 5, causing the light source 200 to emit light rays having directivity in a plurality of forms of a band so as to arrange the anterior and posterior edges of the light at equal intervals and make the occurrence intervals between the on-events and the off-events equal makes it possible to detect the three-dimensional positions of the object obj by scanning of the space at any cycle.

[0030] FIG. 6 is a flowchart illustrating an example of processing in the embodiment of the present invention. In the illustrated example, the light is emitted from the light source 200 (step S101), and in a case where an event signal having been generated by the vision sensor 100 is received by the position detecting device 300 (step S102), the position detecting section 330 detects a position at which a light intensity change has occurred, as the position of an object in the space (step S103).

[0031] In the above-described embodiment of the present invention, the position of an object in the space can be detected at a high speed and with accuracy. The event-driven type vision sensor operates at a high speed by generating the event signal time-asynchronously, and thus, a time or a cycle needed to detect the position of the object can be shortened within a range of a time resolution of control of a light emission angle by a light source. On the other hand, the position of the object can also be detected by, for example, detecting the reflection of emitted light from an image having been captured by a frame type vision sensor, but the frame type vision sensor operates at a lower speed than the event-driven type vision sensor by a time during which the frame type vision sensor captures the image by time-synchronous scanning, and as a result, the time or the cycle needed to detect the position of the object becomes longer. Further, in a case where the event signal is used, the position at which the light intensity change has occurred is directly identified, and thus, the position can accurately be detected, as compared with a case where, for example, image processing for extracting a reflected image from the image is performed.

[0032] The embodiment of the present invention can be utilized in, for example, game controllers, smartphones, and various kinds of mobilities (automobiles, electric automobiles, hybrid electric automobiles, motorcycles, bicycles, personal mobilities, airplanes, drones, ships, robots, etc.) to acquire information regarding surrounding environments, estimate the self-position from surrounding objects, and detect a coming-near flying object to take an avoidance action.

[0033] Heretofore, a preferred embodiment of the present invention has been described in detail referring to the accompanying drawings, but the present invention is not limited to such an example. Any person having normal knowledge in the technical field to which the present invention belongs is obviously able to conceive of various kinds of alternative examples and modification examples within the scope of the technical thought set forth in claims of the present specification, and it is naturally understood that such examples also belong to the technical scope of the present invention.

• • [0034] 10: System
  • [0035] 100: Vision sensor
  • [0036] 200: Light source
  • [0037] 300: Position detecting device
  • [0038] 310: Event signal receiving section
  • [0039] 320: Light source controlling section
  • [0040] 330: Position detecting section