SYSTEM AND METHOD FOR GRAPHICAL PROCESSING OF MEDICAL DATA

[0001] This application claims the benefit of, and priority to, U.S. Provisional Application Ser. No. 61/745,120, filed Dec. 21, 2012, the contents of which are incorporated by reference.

[0002] The invention generally relates to imaging systems in catheter labs and methods of processing data.

[0003] A catheterization laboratory, or cath lab, is an examination room in a hospital that provides the equipment to perform medical procedures that require the insertion of a catheter into a patient's arteries. Typical procedures in a cath lab include intravascular imaging, which can be used to detect vulnerable plaque in a patient's arteries before the onset of a stroke or heart attack. Cath labs also provide the equipment to treat hardened and narrowed arteries by coronary angiography—a procedure in which a doctor uses a catheter to deliver a stent or balloon to open up a narrowed artery and prevent a stroke or heart attack.

[0004] In all of these procedures, medical imaging equipment such as x-ray angiography, intravascular ultrasound (IVUS), or optical coherence tomography (OCT) systems can be used to help a doctor look at the affected arteries. Unfortunately, these imaging systems by their nature can impose some limits on the availability of cath labs.

[0005] Typical cath lab imaging systems generate large three-dimensional data sets that must be processed by high-powered computers to provide useful images. The amount of processing power required to work with the high resolution 3D images that enable plaque detection and angiographic intervention goes well beyond what is offered by typical desktop computers. Since each imaging system requires an expensive, high-powered computer, building a cath lab is a very expensive undertaking. Due to the expense required for a cath lab, some large hospitals may only build one or none, even where several are called for, and some smaller clinics may go without a cath lab entirety.

[0006] The invention provides a computer server with a graphics processor that processes data sets from multiple cath labs simultaneously. The server includes a hypervisor that creates a dedicated virtual machine for each cath lab so that independent imaging or other medical system can access the resources they require, such as unique operating systems, applications, or APIs. The graphics processor provides capabilities that are needed by the imaging or other medical systems such as, for example, the ability to perform very large numbers of transformations in parallel (e.g., linear or non-linear transformations). Since the graphical processing hardware is well-suited to medical data processing, the virtual machines can efficiently handle the work demanded by imaging or other medical systems. Since the hypervisor can allocate resources as-needed to the virtual machine and re-capture the capacity of idle resources, the server can do the work that would otherwise require a large number of dedicated machines. Using high-speed networking technologies, the computer server and each of the cath labs can be in different parts of a building or different buildings. The efficiencies offered by using a hypervisor to share the resources of a graphical processor for medical imaging processing allows a greater number of cath labs to be built or operated for a given amount of resources. Thus, hospitals may have a greater number of cath labs and individual clinics can have a cath lab that could not have one otherwise. Since a greater number of cath labs can be made available, more patients can be diagnosed and treated for conditions such as arterial plaque prior to adverse events like heart attacks or strokes.

[0007] In certain aspects, the invention provides a medical imaging or other measurment or analysis system that uses a server with a graphics processor coupled to a memory. The server uses a hypervisor to define a plurality of virtual machines sharing the system and graphics processor. The system is operable to receive a data from a cath lab comprising a three-dimensional data set describing a patient's anatomy and perform, using the graphics processor, a plurality of transforms in parallel on the data set within one of the plurality of virtual machines. The data can be a three-dimensional data set, blood flow data, or other data. The system can then provide an analysis or a visualization image of the data set, for example, on a monitor or saved to disk.

[0008] The graphics processor includes one or more graphic processing units (GPUs) operably coupled together. The graphics processor is configured to perform massively parallel data processing. Additionally, the graphics processor may perform such operations as oversampling and interpolation. One or more of the GPUs may include one or more frame buffer, a hardware accelerator, or both. The graphics processor can include one or more microchip (e.g., on each GPU) operable to execute a kernel written using OpenCL, CUDA, or a similar programming language. Additionally, one or more of each GPU could include an integrated ARM CPU. In certain embodiments, the graphics processor includes a GPU from NVIDIA, AMD/ATI, S Graphics, Intel, or Matrox, a Many Integrated Cores (MIC) processor from Intel, or other similar massively parallel computational device.

[0009] Preferably, the server is communicably coupled to an imaging instrument in each of a plurality of cath labs, a plurality of imaging instruments of different modalities in any one cath lab, or a combination thereof. The server and the plurality of cath labs can be separated from one another, e.g., on a different floors of a building or in different buildings.

[0010] In related aspects, the invention provides a method of medical imaging or data analysis that includes using a server comprising a graphics processor coupled to a memory and a hypervisor to define a plurality of virtual machines. The server can be used for receiving from a cath lab a data set such as, for example, a series of images comprising a three-dimensional data set of a patient's anatomy and performing a plurality of transforms in parallel on the data set within one of the plurality of virtual machines. A visualization image of the data set may be provided by, for example, displaying it on a monitor or storing it on a disk. The server is communicably coupled to a plurality of imaging instruments, e.g., in each of a plurality of cath labs. The method can include storing transformed data in a frame buffer on the graphics processor, using a hardware accelerator on the graphics processor, performing a variety of algorithms (e.g., oversampling or interpolation) on the data set, or a combination thereof.

[0011] In other aspects, the invention provides a method of imaging tissue by capturing data from a patient using a medical imaging instrument, transferring the data to a server computer comprising a graphical processing unit, and using the graphical processing unit to perform various algorithms on the data in a virtual machine while the server computer simultaneously uses the graphical processing unit to perform various algorithms on other data in a second virtual machine. A visualization image of the data can then be viewed or stored based on the processing operations on the data. In some embodiments, the processing operations include performing a plurality of linear and non-linear transformations in parallel. In certain embodiments, the data includes an image of a patient's tissue and the visualization image provides an image of the patient's tissue.

[0012] Aspects of the invention provide a system for medical imaging that includes a server computer comprising a graphical processing unit; a hypervisor module operable to initiate the creation of a plurality of virtual machines in the server computer and to coordinate, in each of the virtual machines, a set of processing operations by the graphical processing unit on a set of data; and a tangible, non-transitory memory coupled to the graphical processing unit and operable to store processed image data.

[0020] The invention provides a computer server with a graphical processer that can process data from multiple medical imaging systems simultaneously. Data sets from cath labs or other imaging suites (x-ray, angiography, PET scans, MRI, etc.) can be provided by any suitable imaging system and a processing system of the invention allocates resources in the form of a virtual machine, processing power, operating system, applications, etc., as-needed. Embodiments of the invention are described as applied to a catheterization lab, or cath lab, and may find particular application with cath labs due to the particular processing requirements of typical cath lab systems such as intravascular ultrasound (IVUS), optical coherence tomography (OCT), functional measurement (FM), optical-acoustic imaging, and angiographic systems. However, the processing system of the invention further may be applied to any medical imaging modality.

[0021] FIG. 1 shows a diagram of a cath lab 101 according to certain embodiments of the invention. An operator uses control station 110 and optional navigational device 125 to operate catheter 112 via patient interface module (PIM) 105. Here, control station 110 and PIM 105 will be described as for an IVUS system. However, the processing system of the invention is applicable to OCT, optical-acoustic imaging, FM, and other modalities, as well as IVUS. At a distal tip of catheter 112 is an ultrasound transducer 114 (in the case of IVUS). Imaging system 120 works with PIM 105 to coordinate imaging operations. Imaging operations proceed by rotating an imaging mechanism via catheter 112 while transmitting a series of electrical impulses to transducer 114 which results in sonic impulses being sent into the patient's tissue. Backscatter from the ultrasonic impulses is received by transducer 114 and interpreted to provide an image on monitor 103. The IVUS system is operable for use during diagnostic ultrasound imaging of the peripheral and coronary vasculature of the patient. The IVUS instruments can be configured to automatically visualize boundary features, perform spectral analysis of vascular features, provide qualitative or quantitate blood flow data, or a combination thereof. Systems for IVUS suitable for use with the invention are discussed in U.S. Pat. No. 6,673,015; U.S. Pub. 2012/0265077; and U.S. RE40,608 E, the contents of which are incorporated by reference in their entirety for all purposes. Aspects of the invention are discussed in U.S. Provisional Patent Application No. 61/473,591, as well as progeny of that application, the contents of each of which are hereby incorporated by reference in their entirety.

[0022] Operations in cath lab 101 employ a sterile, single use intravascular ultrasound imaging catheter 112. Catheter 112 is inserted into the coronary arteries and vessels of the peripheral vasculature under angiographic guidance. Catheters such as may be used for IVUS are described in U.S. Pat. No. 7,846,101; U.S. Pat. No. 5,771,895; U.S. Pat. No. 5,651,366; U.S. Pat. No. 5,176,141; U.S. Pub. 2012/0271170; U.S. Pub. 2012/0232400; U.S. Pub. 2012/0095340; U.S. Pub. 2009/0043191; U.S. Pub. 2004/0015065, the contents of which are incorporated by reference herein in their entirety for all purposes. Cath lab 101 may include industry standard input/output interfaces for hardware such as navigation device 125, which can be a bedside mounted joystick. System 101 can include interfaces for one or more of an EKG system, exam room monitor, bedside rail mounted monitor, ceiling mounted exam room monitor, and server room computer hardware.

[0023] Catheter 112 and PIM 105 may be connected to the imaging instrument 120 and/or base station 110, which may contain a type CF (intended for direct cardiac application) defibrillator proof isolation boundary. All other input/output interfaces within the patient environment may utilize both primary and secondary protective earth connections to limit enclosure leakage currents. The primary protective earth connection for controller 125 and control station 110 can be provided through the bedside rail mount. A secondary connection may be via a safety ground wire directly to the bedside protective earth system. Monitor 103 and an EKG interface can utilize the existing protective earth connections of the monitor and EKG system and a secondary protective earth connection from the bedside protective earth bus to the main chassis potential equalization post. Monitor 103 may be, for example, a standard SXGA (1280×1024) exam room monitor. System 101 includes control system 120 to coordinate operations.

[0024] Imaging instrument 120 may include one or more processor coupled to a memory. Any suitable processor can be included such as, for example, a general-purpose microprocessor, an application-specific integrated circuit, a massively parallel processing array, a field-programmable gate array, others, or a combination thereof. In some embodiments, imaging instrument 120 can include a high performance dual Xeon based system using an operating system such as Windows XP professional. Imaging instrument 120 may be provided as a single device (e.g., a desktop, laptop, or rack-mounted unit, or may include different machines coupled together (e.g., a Beowulf cluster, a network of servers, a server operating with a local client terminal, other arrangements, or a combination thereof).

[0025] Imaging instrument 120 may operate with different modality data sets in parallel, such as processing real time intravascular ultrasound imaging while simultaneously running a tissue classification algorithm referred to as virtual histology (VH). Instrument 120 may offload all or a portion of any processing to a graphic processor as described herein. The application software can include a DICOM3 compliant interface, a work list client interface, interfaces for connection to angiographic systems, or a combination thereof. Imaging instrument 120 may be located in a separate control room, the exam room, or in an equipment room and may be coupled to one or more of a custom control station, a second control station, a joystick controller, a PS2 keyboard with touchpad, a mouse, or any other computer control device.

[0026] Imaging instrument 120 will generally include memory coupled to the processor. Memory includes any one or more computer readable storage media. Memory preferably refers to tangible, non-transitory computer-readable media. Thus, any computer of the invention generally includes at least one processor (e.g., one or more silicon chip with one or more cores) coupled to at least one non-transitory memory.

[0027] Imaging instrument 120 may generally include one or more USB or similar interfaces for connecting peripheral equipment. Available USB devices for connection include the custom control stations, optional joystick 125, and a color printer. In some embodiments, imaging instrument 120 includes one or more of a USB 2.0 high speed interface, a 10/100/1000 baseT Ethernet network interface, AC power jack, PS2 jack, Potential Equalization Post, 1 GigE Ethernet interface, microphone and line jacks, VGA video, DVI video interface, PIM interface, ECG interface, other connections, or a combination thereof. In certain embodiments, imaging instrument 120 operates as a proximal collector, and optional preprocessor, of data collected via PIM 105 and catheter 112. In some embodiments, PIM 105 transmits data to a shared system without the benefit of an imaging instrument 120, e.g., either directly or through control system 110.

[0028] FIG. 2 shows a shared system 201 for processing images from a plurality of imaging systems 120a, 120b, 120c, . . . , 120n. Each of the imaging systems 120 may send data via hub 213 to a shared graphics processor 205. Processor 205 can use resources from, or be part of, networked resources 229. In operation, processor 205 includes a hypervisor 219 that allocates processing power from one or more graphical processing unit (GPU) 213 to create a plurality of virtual machines 223a, 223b, 223c, . . . , 223n. In some embodiments, each virtual machine 223 services one imaging system 120. Additionally or alternatively, any imaging system 120 could request and receive more than one virtual machine 223, and any virtual machine 223 could perform services for more than one imaging system 220. Additionally, processor 205 will generally include a connection to storage 207.

[0029] Shared system 201 virtualizes computers, operating systems, or both for the processing of images from a plurality of imaging systems 120 by means of hypervisor 219. Any suitable virtual machine monitor may perform the role of hypervisor 219. Platform virtualization is performed by system 201 (a control program), which creates a simulated computer environment, a virtual machine 223, for its guest software. The guest software is not limited to user applications; it may allow the execution of complete operating systems. The guest software executes as if it were running directly on the physical hardware. The described architecture provides a number of benefits. The system operates at significantly lower energy consumption than a similar number of cath labs that do not share a graphic processor 205. Processor 205 can be more easily maintained, inspected, updated, protected, and moved than a plurality of distributed computers.

[0030] In certain embodiments, one or more of the virtual machines 223 simulates enough hardware to allow an unmodified “guest” OS (one designed for the same instruction set) to be run in isolation. This may be allowed by including such tools as, for example, Parallels Workstation, Parallels Desktop for Mac, VirtualBox, Virtual Iron, Oracle VM, Virtual PC, Virtual Server, Hyper-V, VMware Workstation, VMware Server (formerly GSX Server), KVM, QEMU, Adeos, Mac-on-Linux, Win4BSD, Win4Lin Pro, and Egenera vBlade technology, Linux KVM, VMware Workstation, VMware Fusion, Microsoft Hyper-V, Microsoft Virtual PC, Xen, Parallels Desktop for Mac, Oracle VM Server for SPARC, VirtualBox and Parallels Workstation.

[0031] Due to the nature of image processing operations that are employed in medical imaging, processor 205 includes one or more of GPU 215. GPU 215, also occasionally called visual processing unit (VPU), provides a specialized electronic circuit to manipulate and alter memory to accelerate the building of images (e.g., within a frame buffer). GPU 215 is efficient at manipulating medical image data, and the highly parallel structure can make it more effective than general-purpose CPUs for algorithms where processing of large blocks of data is done in parallel. GPU 215 can include resources for 2D acceleration, 3D functionality, graphics-related application programming interfaces (APIs) such as OpenGL or DirectX, or general purpose GPU (GPGPU) development environments such as OpenCL or CUDA by NVIDIA. GPU 215 can include programmable shading to (e.g., each pixel can be processed by a short program that can include additional image textures as inputs; each geometric vertex can be processed by a short program; etc.). Such functionality can be offered by OpenGL API, DirectX, and the GeForce chips by NVIDIA. GPU 215 may further include support for generic stream processing. In certain embodiments, processor 205 includes a plurality of parallelized GPUs (e.g., each itself configured to perform parallel operations). Parallelized GPU computing can be implemented using any suitable platform such as, for example, products from NVIDIA, or OpenCL. OpenCL is an open standard defined by the Khronos Group. OpenCL solutions are supported by Intel, AMD, NVIDIA, and ARM. Processor 205 will include at least one GPU 215. Any suitable GPU can be used, including, for example, those made by Intel, NVIDIA, AMD/ATI, S3 Graphics (owned by VIA Technologies), and Matrox. GPU can provide any suitable algorithm or processing function known in the art such as, for example, neural networks, decision trees, graph algorithms, tree-space searching, Markov chain Monte Carlo sampling of data sets, etc. GPU 215 can include a programmable shader or other resources to manipulate vertices and textures, perform oversampling and interpolation techniques to reduce aliasing, and very high-precision color spaces. In certain embodiments, GPU 215 is a GTX680 (GK104 core), GT640M (GK107 core), GTX 660 Ti (GK104 core), GTX 660 (GK106 core), GTX 650 (GK107 core), or GTX690 by NVIDIA or a Radeon by AMD. In some embodiments, GPU 215 includes an integrated ARM CPU of its own. GPU 215 may operate via OpeNVIDIA, OpenCL, or CUDA, an SDK and API that allows using the C programming language to code algorithms. GPU 215 can process many independent vertices and fragments in parallel. In this sense, GPU 215 is a stream processor and can operate in parallel by running one kernel on many records in a stream at once.

[0032] A stream includes a set of records that require similar computation. Streams provide data parallelism. Kernels are the functions that are applied to each element in the stream. In the GPUs, vertices and fragments are the elements in streams and vertex and fragment shaders are the kernels to be run on them.

[0033] FIG. 3 shows an exemplary relation of resources in processor 205. Hypervisor 219 allows for the virtualization of the GPU 215. Hypervisor 219 may be any suitable manager such as, for example, the NVIDIA VGX Hypervisor, which allows a virtual machine to interact directly with a GPU. Hypervisor 219 manages GPU resources to allow multiple medical imaging systems to share common hardware while improving user density. Each virtual machine 223 can provide a guest operating system or processing environment. The guest OS can provide applications, drivers, APIs, and remote protocol tools. Virtualization and data processing are discussed in U.S. Pat. No. 8,239,938; U.S. Pat. No. 7,672,790; U.S. Pat. No. 7,743,189; U.S. Pub. 2011/0274329; U.S. Pub. 2008/0143707; and U.S. Pub. 2004/0111552, the contents of each of which are incorporated by reference. Processor 205 may be onsite or off-site.

[0034] FIG. 4 illustrates an architecture 203 for image processing that is conducive to the use of an off-site processor 205. Processor 205 includes the elements shown in FIG. 4 (e.g., hypervisor 219, VM 223, etc.) and is connected to hub 213 over network resource 229, which can include the Internet, a WAN or LAN, cellular telephone data networks, other methodology, or a combination thereof. Network 229 can also provide a connection for storage 207. Via hub 213, processor 205 is connected to a plurality of local imaging instruments 120. It should be appreciated that a local connection can service a medical suite such as a cath lab, and can include connections to a plurality of different imaging instruments within a medical suite.

[0035] FIG. 5 shows a local network structure in which each medical suite has its own combination of imaging modalities, all connected to processor 205. Here, hub 213a is connected to imaging suite 101a that includes angiographic, MRI, OCT, FM, and IVUS imaging instruments. Hub 213b is connected to cath lab 101b that provides angiographic, OCT, and IVUS services. Hub 213n connects to cath lab 101n. It will be appreciated that any suitable number of cath lab 101, each having any given combination of imaging modality instruments, may be connected to processor 205. By sharing a graphical processor 205 with a plurality of different imaging instruments 120, methods of the invention provide beneficial costs and qualities of image processing services.

[0036] FIG. 6 diagrams a method according to the invention. A data set is received 609 at processor 205 from a medical imaging instrument 120. Hypervisor 219 allocates 615 a virtual machine 223 for the requesting instrument 120. GPU 215 processes 621 the data set within virtual machine 223. Where GPU 215 includes one or more optional frame buffer, the nascent processed data is stored 627 in the frame buffer. Processor 205 then operates to provide 631 the image data, which can include, for example, a 2D image for viewing on a monitor or an analytical result such as from a virtual histology analysis. When done processing 621 the data, hypervisor 219 recaptures 637 the capacity of GPU 215 that was allocated 615 to virtual machine 223. This methodology according to the systems described herein may provide considerable savings in terms of efficient use of processing resources (e.g., in some embodiments, a shared GPU will provide service associated with a 10×reduction in demand for processing hardware). In certain embodiments, processor 205 coordinates pre-allocation. For pre-allocation, a cath lab indicates that it will perform an IVUS operation, and the server allocates and holds resources for that cath lab. When data flows, it flows through frame-by-frame in real-time for any and all sessions (e.g., IVUS sessions). When the lab session is done, the server can release the lock on those system resources that were held. Further, the server notifies labs or instruments of deficiencies in resources (e.g., if GPU is operating at full capacity and a request comes in, server can send a message to requestor saying so or giving an estimated delay). Additionally, the systems and methods herein allow for distributed medical imaging laboratories to each avail themselves of system processing power despite geographical separation.

[0037] FIG. 7 depicts a distributed system 701 for shared graphical processing of medical image data. Here, graphical processor 205 is operating in San Diego, Calif. Medical imaging lab 101a is operating in, for example, Ontario, Oreg. Lab 101b operates out of Creston, Iowa. Lab 101n is shown here in Calhoun, Ga. In each lab 101 an imaging instrument 120 operates while attached via a PIM 105 to a catheter 112 inserted into a patient's body. Data collected by transducer 114 is transferred via instrument 120 from Oregon, Iowa, and Georgia to processor 205 in California. A processing system 215 including one or more GPU in processor 205 invokes a virtual machine 223 for each lab. The data is processed in the respective virtual machine 223 in processor 205. Results can be displayed, for example, on monitors 103 back in the respective labs 101. While discussed with respect to FIG. 7 as a distributed embodiment, it will be appreciated that systems and methods of the invention particularly provide embodiments in which processor 205 operates at a facility that includes the labs 101 (e.g., a plurality of cath labs 101 on a hospital campus, each networked to processor 205 in a server “closet”—generally, an air conditioned room with server racks). Further discussion of client server architecture for imaging may be found in U.S. Pub. 2012/0083696; U.S. Pub. 2011/0257545; U.S. Pub. 2011/0245669; U.S. Pub. 2011/0034801; U.S. Pub. 2008/0306766; and U.S. Pub. 2007/0043597, the contents of which are incorporated by reference in their entirety.

[0038] As used herein, the word “or” means “and or or”, sometimes seen or referred to as “and/or”, unless indicated otherwise.

[0039] References and citations to other documents, such as patents, patent applications, patent publications, journals, books, papers, web contents, have been made throughout this disclosure. All such documents are hereby incorporated herein by reference in their entirety for all purposes.

[0040] Various modifications of the invention and many further embodiments thereof, in addition to those shown and described herein, will become apparent to those skilled in the art from the full contents of this document, including references to the scientific and patent literature cited herein. The subject matter herein contains important information, exemplification and guidance that can be adapted to the practice of this invention in its various embodiments and equivalents thereof.