Controlling the integral light energy of a laser pulse

[0001] This application claims the benefit of U.S. Provisional Application No. 61/799,626, filed Mar. 15, 2013, which is hereby incorporated by reference herein in its entirety, including but not limited to those portions that specifically appear hereinafter, the incorporation by reference being made with the following exception: In the event that any portion of any of the above-referenced provisional application is inconsistent with this application, this application supersedes said above-referenced provisional application.

[0002] Not Applicable.

[0003] Advances in technology have provided advances in imaging capabilities for medical use. One area that has enjoyed some of the most beneficial advances is that of endoscopic surgical procedures because of the advances in the components that make up an endoscope.

[0004] The disclosure relates generally to electromagnetic sensing and sensors, and is more particularly related to controlling a consistent amount of electromagnetic energy that may be delivered by an emitter configured to illuminate a scene. The features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by the practice of the disclosure without undue experimentation. The features and advantages of the disclosure may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims.

[0013] The disclosure extends to methods, systems, and computer based products for digital imaging that may be primarily suited to medical, industrial, marine, and automotive applications. In the following description of the disclosure, reference may be made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific implementations in which the disclosure may be practiced. It may be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the disclosure.

[0014] The disclosure describes at least one embodiment that can be used to control the duration of a single or multiple light emitting system, such as a laser or other light system, to limit output light energy of a light pulse that is held to a specified tolerance in a light controlled environment where the light pulse is the sole light source available. It will be appreciated that for the sake of simplicity, the disclosure will use a laser as an exemplary embodiment of the light emitter. However, it will be appreciated that the light emitter may be any light source, including a laser, LED or other light source without departing from the scope of the disclosure.

[0015] An embodiment may comprise a system for digital imaging within a light deficient environment comprising: a camera control unit that may comprise a microcontroller, FPGA, ASIC, hardware, software, ISP, and the like support circuitry for digital imaging.

[0016] The system may further comprise a controllable light source connected to the CCU, wherein the CCU may control enable/disabling, power level, light level, on/off, and other controls of the output of the controllable light source. The light source may comprise digital light sources such as lasers and light emitting diodes, and may also comprise analog light sources.

[0017] The system may further comprise an imaging sensor capable of detecting and digitizing reflected light from a scene, and transmitting data to the CCU.

[0018] During camera operation, a PID control algorithm may be run to ensure that the captured scene maintains a desired video exposure level to maximize the dynamic range of the sensor or to achieve a desired scene response that the end-user desires. This control algorithm be generally known as Automatic Shutter Control (ASC). In some embodiments each light pulse, such as a laser pulse, may be adjusted proportionally to the error measurement of desired exposure level compared to a measured exposure level. This may be measured using the mean pixel value of all pixels or some portion of pixels from the Pixel Array (PA). The ASC may request adjustment to be made to the light pulse in duration and/or intensity to ensure that the desired setpoint is achieved in some specified time to not affect image quality.

[0019] In an embodiment the light source will be required to consistently deliver a specified light level to meet the request of the ASC algorithm. This range can be anywhere from 2-2ⁿ bits, depending on the applications precision requirements. However, a problem may arise for the light source, due to warm-up requirements, changes in temperature at each light emitter or each laser, temperature in the box, manufacturing defects, or other typical issues associated with light emitters, such as lasers, and electronic devices that may prevent the light source from meeting the precision demands of the ASC algorithm leading to flickering light, image artifacting, or other undesirable effects causing poor image quality and/or a system unusable to the end-user. What is needed is a solution to ensure functional specification and performance demands are met to satisfy user experience.

[0020] In an embodiment a light sensing element, which may be a photo diode or another light sensing element, may be configured to read the energy transmission on individual fiber channels coming from each light emitter, such as a laser, to integrate the light energy in real-time, which is typically less than 1 ms. Each laser may have a single fiber ranging from 0.05 mm to 0.5 mm connected to an optical output from the laser. Laser can vary in wavelength and power and may be chosen to meet application need. The light sensing element, such as a photo diode, may be attached or directed to the passing fiber would provide an independent feedback that may then be compared to a register or variable retrieved from computer memory that comprises predetermined desired light levels requested by the AEC. This system may ensure that once a light energy level has been achieved the laser would be shut-off or disabled to preserve the desired image quality.

[0021] An embodiment may comprise a single photodiode sensing the electromagnetic energy of a multitude of fibers or a light sensing element, such as a photo diode or other light sensing element, looking at a single fiber individually. It is also understood that a user or system may need to calibrate this feedback to ensure precision requirements are met in the application. An example showing the control of two lasers is depicted in FIG. 1.

[0022] In an embodiment a light sensing element, such as a photo diode, may be placed internally on each laser module, such that there may be a system having a plurality of sensing elements, such as photo diodes. Since laser modules 155 are typically comprised of a plurality of lasers arranged in a linear array (as illustrated in FIG. 7) or other geometric pattern usually containing 10 or more individual laser diodes, to ensure precision light output of the laser module one could direct a small portion of those laser's outputs, say as little as one or two, to a sensing element 160 or any device that can transform light into a voltage or current level that would measure the amount of light output of one, two or N laser diode(s) that can be used with a differential amplifier comparing light output measurement from sensing element against a desired reference voltage or current level. This circuit may then provide direct feedback to the bias current (or voltage) of the laser module ensuring that desired output light level is met to a desired precision level. An example is shown in FIG. 2.

[0023] As illustrated in FIG. 2, an embodiment may comprise a bundle of laser emitters 205 that are combined through the use of fiber optics 207. The system 200, may further comprise a light sensing element or an electromagnetic sensor 210 that senses the output of one of the lasers in the laser bundle 205. Additionally, feedback may also be applied with the use of a light frequency doubler or operation amplification circuit 215 has been incorporated into certain configurations of the laser module. For a non-limiting example, green or blue laser modules may be used in applications that could benefit from this device.

[0024] In an embodiment a precision level of the light output may be 0.01%-10% depending on the application requirements.

[0025] FIG. 3 illustrates a schematic view of a paired sensor 305 and an electromagnetic emitter 310 in operation for use in producing an image in a light deficient environment. Such a configuration allows for increased functionality in light controlled or ambient light deficient environments. It should be noted that as used herein the term “light” is both a particle and a wavelength, and is intended to denote electromagnetic radiation that is detectable by a pixel array, and may be include wavelengths from the visible and non-visible spectrums of electromagnetic radiation. The term “partition” is used herein to mean a pre-determined range of wavelengths of the electromagnetic spectrum that is less than the entire spectrum, or in other words, wavelengths that make up some portion of the electromagnetic spectrum. An emitter may be a light source that is controllable as to the portion of the electromagnetic spectrum that is emitted, the intensity of the emissions, or the duration of the emission, or all three. An emitter may emit light in any dithered, diffused, or columnated emission and may be controlled digitally or through analog methods or systems.

[0026] A pixel array 305 of an image sensor may be paired with an emitter 310 electronically, such that they are synced during operation for both receiving the emissions and for the adjustments made within the system. As can be seen in FIG. 3, an emitter 310 may be tuned to emit electromagnetic radiation in the form of a laser, which may be pulsed in order to illuminate an object. The emitter 310 may pulse at an interval that corresponds to the operation and functionality of a pixel array 305. The emitter 310 may pulse light in a plurality of electromagnetic partitions, such that the pixel array receives electromagnetic energy and produces a data set that corresponds (in time) with each specific electromagnetic partition. For example, FIG. 3 illustrates a system having a monochromatic pixel array (black and white) 305, which is simply sensitive to electromagnetic radiation of any wavelength. The light emitter illustrated in the figure may be a laser emitter 310 that is capable of emitting a green electromagnetic partition, a blue electromagnetic partition, and a red electromagnetic partition in any desired sequence. It will be appreciated that other light emitters may be used in FIG. 3 without departing from the scope of the disclosure, such as digital or analog based emitters. During operation, the data created by the monochromatic sensor for any individual pulse is assigned a specific color partition, wherein the assignment is based on the timing of the pulsed color partition from the emitter. Even though the pixels are not color dedicated they can be assigned a color for any given data set based on timing. In one embodiment, three data sets representing RED, GREEN and BLUE pulses may then be combined to form a single image frame 320. It will be appreciated that the disclosure is not limited to any particular color combination or any particular electromagnetic partition, and that any color combination or any electromagnetic partition may be used in place of RED, GREEN and BLUE, such as Cyan, Magenta and Yellow, Ultraviolet, infra-red, or any other color combination, including all visible and non-visible wavelengths, without departing from the scope of the disclosure. In the figure, the object to be imaged contains a red portion, green portion and a blue portion. As illustrated in the figure, the reflected light from the electromagnetic pulses only contains the data for the portion of the object having the specific color that corresponds to the pulsed color partition. Those separate color (or color interval) data sets can then be used to reconstruct 325 the image by combining the data sets.

[0027] FIG. 4 graphically illustrates the operation of an embodiment of an electromagnetic emitter. An emitter may be timed to correspond with the cycles of a sensor, such that electromagnetic radiation is emitted within the sensor operation cycle and/or during a portion of the sensor operation cycle. In an embodiment the emitter may pulse during the read out portion of the sensor operation cycle. In an embodiment the emitter may pulse during the blanking portion of the sensor operation cycle. In an embodiment the emitter may pulse for a duration that is during portions of two or more sensor operational cycles. In an embodiment the emitter may begin a pulse during the blanking portion, or during the optical black portion of the readout portion, and end the pulse during the readout portion, or during the optical black portion of the readout portion. It will be understood that any combination of the above is intended to fall within the scope of this disclosure as long as the pulse of the emitter and the cycle of the sensor correspond.

[0028] FIG. 5 graphically represents varying the duration and magnitude of the emitted electromagnetic pulse to control exposure. An emitter having a fixed output magnitude may be pulsed during any of the cycles noted above in relation to FIG. 3 for an interval to provide the needed electromagnetic energy to the pixel array. An emitter having a fixed output magnitude may be pulsed at a longer interval of time, thereby providing more electromagnetic energy to the pixels or the emitter may be pulsed at a shorter interval of time, thereby providing less electromagnetic energy. Whether a longer or shorter interval time is needed depends upon the operational conditions.

[0029] In contrast to adjusting the interval of time that the emitter pulses a fixed output magnitude, the magnitude of the emission itself may be increased in order to provide more electromagnetic energy to the pixels. Similarly, decreasing the magnitude of the pulse provides less electromagnetic energy to the pixels. It should be noted that an embodiment of the system may have the ability to adjust both magnitude and duration concurrently, if desired. Additionally, the sensor may be adjusted to increase its sensitivity and duration as desired for optimal image quality.

[0030] Implementations of the disclosure may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed in greater detail below. Implementations within the scope of the disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions may be computer storage media (devices). Computer-readable media that carry computer-executable instructions may be transmission media. Thus, by way of example, and not limitation, implementations of the disclosure can comprise at least two distinctly different kinds of computer-readable media: computer storage media (devices) and transmission media.

[0031] Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), Flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.

[0032] A “network” may be defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. In an implementation, a sensor and camera control unit may be networked in order to communicate with each other, and other components, connected over the network to which they may be connected. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmissions media can include a network and/or data links which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

[0033] Further, upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures that can be transferred automatically from transmission media to computer storage media (devices) (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a “NIC”), and then eventually transferred to computer system RAM and/or to less volatile computer storage media (devices) at a computer system. RAM can also include solid state drives (SSDs or PCIx based real time memory tiered Storage, such as FusionIO). Thus, it should be understood that computer storage media (devices) can be included in computer system components that also (or even primarily) utilize transmission media.

[0034] Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts may be disclosed as example forms of implementing the claims.

[0035] Those skilled in the art will appreciate that the disclosure may be practiced in network computing environments with many types of computer system configurations, including, personal computers, desktop computers, laptop computers, message processors, control units, camera control units, hand-held devices, hand pieces, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. It should be noted that any of the above mentioned computing devices may be provided by or located within a brick and mortar location. The disclosure may also be practiced in distributed system environments where local and remote computer systems, which may be linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0036] Further, where appropriate, functions described herein can be performed in one or more of: hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) or field programmable gate arrays can be programmed to carry out one or more of the systems and procedures described herein. Certain terms may be used throughout the following description and Claims to refer to particular system components. As one skilled in the art will appreciate, components may be referred to by different names. This document does not intend to distinguish between components that differ in name, but not function.

[0037] FIG. 6 is a block diagram illustrating an example computing device 600. Computing device 600 may be used to perform various procedures, such as those discussed herein. Computing device 600 can function as a server, a client, or any other computing entity. Computing device can perform various monitoring functions as discussed herein, and can execute one or more application programs, such as the application programs described herein. Computing device 600 can be any of a wide variety of computing devices, such as a desktop computer, a notebook computer, a server computer, a handheld computer, camera control unit, tablet computer and the like.

[0038] Computing device 600 includes one or more processor(s) 602, one or more memory device(s) 604, one or more interface(s) 606, one or more mass storage device(s) 608, one or more Input/Output (I/O) device(s) 610, and a display device 630 all of which may be coupled to a bus 612. Processor(s) 602 include one or more processors or controllers that execute instructions stored in memory device(s) 604 and/or mass storage device(s) 608. Processor(s) 602 may also include various types of computer-readable media, such as cache memory.

[0039] Memory device(s) 604 include various computer-readable media, such as volatile memory (e.g., random access memory (RAM) 614) and/or nonvolatile memory (e.g., read-only memory (ROM) 616). Memory device(s) 604 may also include rewritable ROM, such as Flash memory.

[0040] Mass storage device(s) 608 include various computer readable media, such as magnetic tapes, magnetic disks, optical disks, solid-state memory (e.g., Flash memory), and so forth. As shown in FIG. 6, a particular mass storage device is a hard disk drive 624. Various drives may also be included in mass storage device(s) 608 to enable reading from and/or writing to the various computer readable media. Mass storage device(s) 608 include removable media 626 and/or non-removable media.

[0041] I/O device(s) 610 include various devices that allow data and/or other information to be input to or retrieved from computing device 600. Example I/O device(s) 610 include digital imaging devices, electromagnetic sensors and emitters, cursor control devices, keyboards, keypads, microphones, monitors or other display devices, speakers, printers, network interface cards, modems, lenses, CCDs or other image capture devices, and the like.

[0042] Display device 630 includes any type of device capable of displaying information to one or more users of computing device 600. Examples of display device 630 include a monitor, display terminal, video projection device, and the like.

[0043] Interface(s) 606 include various interfaces that allow computing device 600 to interact with other systems, devices, or computing environments. Example interface(s) 606 may include any number of different network interfaces 620, such as interfaces to local area networks (LANs), wide area networks (WANs), wireless networks, and the Internet. Other interface(s) include user interface 618 and peripheral device interface 622. The interface(s) 606 may also include one or more user interface elements 618. The interface(s) 606 may also include one or more peripheral interfaces such as interfaces for printers, pointing devices (mice, track pad, etc.), keyboards, and the like.

[0044] Bus 612 allows processor(s) 602, memory device(s) 604, interface(s) 606, mass storage device(s) 608, and I/O device(s) 610 to communicate with one another, as well as other devices or components coupled to bus 612. Bus 612 represents one or more of several types of bus structures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus, and so forth.

[0045] For purposes of illustration, programs and other executable program components may be shown herein as discrete blocks, although it is understood that such programs and components may reside at various times in different storage components of computing device 600, and may be executed by processor(s) 602. Alternatively, the systems and procedures described herein can be implemented in hardware, or a combination of hardware, software, and/or firmware. For example, one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein.

[0046] FIG. 7 illustrates an implementation of a laser illumination system having a plurality of laser bundles emitting a plurality of wavelengths of electromagnetic energy. As can be seen in the figure, the illumination system 700 may comprise read laser bundle 720, a green laser bundle 730, and a blue laser bundle 740 that are all optically coupled together though fiber optics 755. As can be seen in the figure, each of the laser bundles may have a corresponding light sensing element or electromagnetic sensor 725, 735, 745 respectively, for sensing the output of the specific laser bundle or wavelength.

[0047] It will be appreciated that various features disclosed herein provide significant advantages and advancements in the art. The following embodiments may be exemplary of some of those features.

[0048] In the foregoing Detailed Description of the Disclosure, various features of the disclosure may be grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed disclosure requires more features than may be expressly recited in each claim. Rather, inventive aspects lie in less than all features of a single foregoing disclosed embodiment.

[0049] It is to be understood that the above-described arrangements may be only illustrative of the application of the principles of the disclosure. Numerous modifications and alternative arrangements may be devised by those skilled in the art without departing from the spirit and scope of the disclosure and the appended claims may be intended to cover such modifications and arrangements.

[0050] Thus, while the disclosure has been shown in the drawings and described above with particularity and detail, it will be apparent to those of ordinary skill in the art that numerous modifications, including, but not limited to, variations in size, materials, shape, form, function and manner of operation, assembly and use may be made without departing from the principles and concepts set forth herein.

[0051] Further, where appropriate, functions described herein can be performed in one or more of: hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein. Certain terms may be used throughout the following description and Claims to refer to particular system components. As one skilled in the art will appreciate, components may be referred to by different names. This document does not intend to distinguish between components that differ in name, but not function.

[0052] The foregoing description has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations may be possible in light of the above teaching. Further, it should be noted that any or all of the aforementioned alternate implementations may be used in any combination desired to form additional hybrid implementations of the disclosure.

[0053] Further, although specific implementations of the disclosure have been described and illustrated, the disclosure is not to be limited to the specific forms or arrangements of parts so described and illustrated. The scope of the disclosure is to be defined by the claims appended hereto, any future claims submitted here and in different applications, and their equivalents.