Method and apparatus for controlling ultrasound system display

[0001] This invention relates generally to ultrasound systems and, more particularly, to methods and devices for controlling imaging in ultrasound systems.

[0002] At least some known ultrasound systems, for example, an ultrasound machine or scanner, are capable of performing three dimensional (3D) volume date acquisition and acquisition of the 3D data sets progressing over time (4D). Raw data and processed data of the 3D and 4D data sets may grow exponentially as multiple dimensional data are spanned. Large data sets can make managing and manipulating the data hardware intensive, such as requiring large amounts of random access memory (RAM), large disk storage, powerful microprocessors, and image handling chipsets. The large data sets also may make navigating though the data, locating an object of interest, and displaying a desired view of the object of interest cumbersome and difficult. Because the physical space on the ultrasound machine may be limited, it is often not possible to provide a real-time or offline 3D or 4D representation of the data available to the user at all times.

[0003] A common approach to managing the limited space for data and display capabilities is to use only a portion of the data, such as every Nth frame or image of a plurality of images that make up a data set. However, using only every Nth frame or image may result in the omission of data for which a user is searching. Additionally, allowing a user to view only a portion of the data may not represent the data in a manner familiar to the user.

[0004] In one embodiment, a method for operating a medical imaging system is provided. The method includes receiving an image data set of a region of interest in a first dimensional representation, reducing the dimensionality of the image data set to a second dimensional representation, selecting a feature of interest in the second dimensional representation, and generating an image corresponding to the selected feature in the first dimensional representation.

[0005] In another embodiment, an ultrasound system is provided. The ultrasound system includes an ultrasound probe having at least one transducer for transmitting pulses to an object, a processor programmed to receive image data in a first dimensionality and convert the image data to a second dimensionality, a viewpoint selector coupled to a user input of said ultrasound system to control a view of converted image data, and a display system configured to display an ultrasound image portion and a reduced dimensionality image portion.

[0006] In another embodiment, a method for processing ultrasound data is provided. The method includes receiving multidimensional ultrasound data representing an image, generating a reduced dimension data set from the multidimensional ultrasound data, selecting a feature of interest in the reduced dimension data set, and displaying the image corresponding to the feature of interest.

[0011] Exemplary embodiments of ultrasound systems and methods for controlling such systems are described in detail below. A detailed description of exemplary ultrasound systems will first be provided followed by a detailed description of an embodiment that facilitates managing, navigating, and displaying image data in ultrasound systems.

[0012] FIG. 1 is a block diagram of an ultrasound system in accordance with one exemplary embodiment of the present invention. Ultrasound system 10 includes a transmitter 12 that drives transducer elements 14 within a probe 16 to emit pulsed ultrasonic signals into a body. A variety of geometries may be used. The ultrasonic signals are back-scattered from structures in the body, like blood cells or muscular tissue, to produce echoes that return to transducer elements 14. The echoes are received by a receiver 18. The received echoes are provided to a beamformer 20, which performs beamforming and outputs an RF signal. The RF signal is then transmitted to an RF processor 22. Alternatively, RF processor 22 may include a complex demodulator (not shown) that demodulates the RF signal to form IQ data pairs representative of the echo signals. The RF or IQ signal data may then be routed directly to an RF/IQ buffer 24 for temporary storage. A user input device 26 as described herein may be used to control operation of ultrasound system 10. This may include using voice commands to control a viewpoint selector 28, which allows a user to select a virtual vantage point from which to view the image data, or view perspective, a region of interest, and/or an index indicator for controlling ultrasound system 10.

[0013] Ultrasound system 10 also includes a processor 30 to process the acquired ultrasound information (i.e., RF signal data or IQ data pairs) and prepare frames of ultrasound information for display on display system 32. Processor 30 is adapted to perform one or more processing operations according to a plurality of selectable ultrasound modalities on the acquired ultrasound information. Acquired ultrasound information may be processed in real-time during a scanning session as the echo signals are received. Additionally or alternatively, the ultrasound information may be stored temporarily in RF/IQ buffer 24 during a scanning session and processed in less than real-time in a live or off-line operation.

[0014] Ultrasound system 10 may continuously acquire ultrasound information at a frame rate that exceeds fifty frames per second, which is the approximate perception rate of the human eye. The acquired ultrasound information may be displayed on display system 32 at a slower frame-rate. An image buffer 34 may be included for storing processed frames of acquired ultrasound information that are not scheduled to be displayed immediately. In an exemplary embodiment, image buffer 34 is of sufficient capacity to store at least several seconds of frames of ultrasound information. The frames of ultrasound information are stored in a manner to facilitate retrieval thereof according to its order or time of acquisition. Image buffer 34 may comprise any known data storage medium.

[0015] It should be noted that various embodiments of a user interface or input, such as, for example, user input device 26, may be implemented for controlling ultrasound system 10. Such various embodiments may include control functionality, such as a set of user controls for controlling ultrasound system 10. The set of user controls may be provided, for example, as part of a touch screen or panel, or as manual inputs, including, for example, user operable switches, and/or buttons. The set of user controls may be manually operable or voice operated.

[0016] Ultrasound system 10 includes a probe 16, such as, for example, a transducer or a plurality of transducing elements, connected to a transmitter 12 and a receiver 18. Probe 16 transmits ultrasonic pulses and receives echoes from structures inside a scanned volume 36. Scanned volume 36 may be obtained by various techniques, including, for example, real-time imaging, volume scanning, scanning with transducers having positioning sensors, freehand scanning using a Voxel correlation technique or scanning with matrix array transducers.

[0017] Probe 16 may be moved, such as, along a linear or arcuate path, while scanning volume 36. At each linear or arcuate position, probe 16 obtains one or more scan planes 38. Scan planes 38 are collected for a thickness, such as from a group or set of adjacent scan planes 38. Scan planes 38 may be stored in buffer 24, and then transmitted to processor 30. In some embodiments, probe 16 may obtain lines instead of scan planes 38, and buffer 24 may store lines obtained by probe 16 rather than scan planes 38. Buffer 24 then stores lines obtained by probe 16 rather than scan planes 38. Processor 30 may receive a slice thickness setting from a slice thickness setting control within user input 26 or may set the slice thickness setting control automatically based on predetermined characteristics of the transmitted data, which identifies the thickness of a slice to be created from scan planes 38. Processor 30 creates a data slice from multiple adjacent scan planes 38. The number of adjacent scan planes 38 that may be obtained to form each data slice is dependent upon the thickness selected by the slice thickness setting control. The data slice is stored in image buffer 34 and processor 30 may access image buffer 34 to perform volume rendering upon the data slice.

[0018] Processor 30 may also generate difference maps of the stored data or may create difference maps of incoming data in real-time. For example, a difference map of respective pixels of each adjacent pair of planar slices may be generated using an intensity projection map, a sum of absolute differences, a maximum intensity projection map, a minimum intensity projection map, an average intensity projection map, an integrated projection map, a first local maximum projection map, and an arbitrarily weighted projection map. The output of the difference maps may be indexed and combined to reduce the dimensionality of the incoming data. For example, difference maps of adjacent pairs of image slices may be combined to render a two-dimensional indication of differences between all of the plurality of image slices. The indication may indicate relative values of the difference between adjacent planar slices on a display that may be traversed to quickly locate a feature of interest, such as, a portion of the display where there are relatively larger differences between adjacent planar slices than other portions. Similarly, volume data collected over time may be reduced in dimensionality by generating difference maps of temporally adjacent volumes such that a region of interest may be selected or a portion of the display that represents a greater relative change in difference over time may be selected.

[0019] The output of processor 30 may be selectably transmitted to a display system 32. Display system 32 may comprise a user wearable device such as a texture glove, a headset display, and/or a heads-up display. As used herein, a texture glove is a wearable covering for a limb comprising a plurality of sensors for transmitting the limb's movement and orientation in space to an input device or directly to a processor. The texture glove may include a plurality of sensory outputs that interact with the limb to transmit sensory indications of, for example, but not limited to, texture, temperature, force, pressure, and orientation. Display system 32 may include an active-matrix color LCD display, such as is available from Sharp Systems of America, Mahwah, N.J., wherein a second matrix, or parallax barrier is transparent when operating in a 2-D mode. During a user selectable 3-D mode a switching LCD sends alternate pixels to the left and right eyes of the user to create a 3-D effect.

[0020] Display system 32 may also include or may transmit images to a holographic projection device 40, which may display images in a heads-up mode such that projected images are used in conjunction with a users visual field to highlight or accentuate features in the user's visual field, or may be used to comprise the entire visual display. Accordingly, holographic projection device 40 may be a head-mountable wearable device that includes sensors to determine a position and orientation of holographic projection device 40 relative to an object that is being scanned and/or has been scanned.

[0021] It should be noted that the position of each echo signal sample (Voxel) is defined in terms of geometrical accuracy (i.e., the distance from one Voxel to the next) and ultrasonic response (and derived values from the ultrasonic response). Suitable ultrasonic responses include gray scale values, color flow values, and angio or power Doppler information.

[0022] It should further be noted that ultrasound system 10 may include additional or different components. For example, a user interface or input may be provided and used to control the operation of ultrasound system 10, including, to control the input of patient data, scan parameters, and/or a change of scan mode.

[0023] FIG. 2 is a perspective view of a real-time volume acquired by the system of FIG. 1 in accordance with an exemplary embodiment of the present invention. Scanned volume 36 includes a sector shaped cross-section with radial borders 122 and 124 diverging from one another at an angle 126. Probe 16 (shown in FIG. 1) electronically focuses and directs ultrasound firings longitudinally to scan along adjacent scan lines in each scan plane 38 (shown in FIG. 1) and electronically or mechanically focuses and directs ultrasound firings laterally to scan adjacent scan planes 38. Scan planes 38 obtained by probe 16 may be stored in buffer 24 and may be scan converted from spherical to Cartesian coordinates by processor 30. A volume comprising multiple image planes 134 defined by multiple scan planes is generated as a rendering box 130. Rendering box 130 is formed from multiple adjacent image planes 134.

[0024] Rendering box 130 may be defined in size by an operator using user interface or input 26 to have a slice thickness 132, width 136 and height 138. Processor 30 may be controlled by the slice thickness setting control to adjust the thickness parameter of the slice to form rendering box 130 with the desired thickness. Rendering box 130 designates the portion of scanned volume 36 that is volume rendered. Processor 30 may access image buffer 34 renders along slice thickness 132 of rendering box 130.

[0025] Referring now to FIGS. 1 and 2, during operation, a slice having a pre-defined thickness is acquired and is processed in processor 30. The echo data representing rendering box 130 may be stored in image buffer. A predefined thickness of between about two millimeters and about twenty millimeters is typical, however, a thickness of less than about two millimeters or greater than about twenty millimeters also may be suitable depending on the application and the size of the area to be scanned. The slice thickness setting control may include a rotatable knob with discrete or continuous thickness settings.

[0026] Processor 30 projects rendering box 130 onto an image portion 148 of an image plane 134. Following processing, the pixel data in image portion 148 may be transmitted through a video processor within display system 32. Rendering box 130 may be located at any position and oriented at any direction within scanned volume 36. In some situations, depending on the size of the region being scanned, it may be advantageous for the rendering box 130 to be only a small portion of scanned volume 36.

[0027] FIG. 3 is an exemplary embodiment of a user interface 300 displaying an output of ultrasound system 10 (shown in FIG. 1). User interface 300 may include an ultrasound viewing area 302 that may be used to display in real time 2D images, 2D images in less than real time, cine loops of 2D images, and 3D display of ultrasound images. Captured cine loops may be replayed in real time, at reduced speed or frame by frame in viewing area 302. Ultrasound viewing area 302 may be used to display difference maps of selected data to facilitate navigation of large amounts of image data to features of interest rapidly. User interface 300 may include a scout map area 304 that includes a scale 306 and an index indicator 308. A reduced dimensionality display 310 is used to indicate a relative difference between adjacent data slices and/or between temporally adjacent volumes. Scout map area 304 includes a selector 312 that may be used to move a cursor 314 relative to display 310. Cursor 314 indicates the portion of display 310 that is being displayed on ultrasound viewing area 302. In the exemplary embodiment, selector 312 is a slider. In other embodiments selector 312 may be, for example a toggle switch, or a numeric input field.

[0028] During a scan, an amount of data is received by system 10. Processor 30 computes a difference map of each pair of data slices contained with in the received data. The user may alter the view perspective of the image data displayed on ultrasound viewing area 302 to orient the view of each displayed image slice or difference map depending on a desired angle of view towards the scanned data. Ultrasound viewing area 302 may selectably display real time images and/or images that are stored on system 10. When system 10 computes difference maps for the adjacent image slices, a total difference between each pair of image slices may be computed and normalized for display on display 310. An exemplary trace 316 illustrates exemplary values for differences between adjacent image slices contained in the memory of system 10. A user may use slider 312 to move cursor 314 to a local maxima, such as feature of interest 318. A relatively large amount of difference between adjacent image slices may indicate a density interface between structures within scanned volume 36. Such a density interface may be between, for example, blood and a vessel wall, organ tissue and tumor tissue, and bone and muscle. After selecting a feature of interest 318, the user may select to replay the scan at full speed, reduced speed, frame-by-frame, or a combination of speeds. User interface 300 may similarly be used to find features of interest 318 in 3D data taken over time such that index indicator 308 represents an elapsed time of a scan. Accordingly, the user may select feature of interest and start playback of the image volume images in a selectable speed.

[0029] A view perspective may be selected from any direction relative to scanned volume 36. Accordingly, a difference map and reduced dimensionality display may be specified for a volume from any direction as selected by a user. The scanned volume may be navigated using a haptic input control, such as, but not limited to a mouse, a joystick, a jog wheel, and/or a texture glove coupled to user input 26. Selection of the view perspective direction may be performed using for example, a hardware button, a software key, and/or a virtual reality button or key.

[0030] Ultrasound viewing area 302 may be further used to increase a download rate of image data from system memory. Using a selected view perspective, system 10 may direct data from only a portion of memory to be downloaded. The downloaded data may correspond to only that amount of data containing data for the difference map selected, such that data that is not used for the difference map is not downloaded.

[0031] Processor 30 is also programmed to manipulate image data using software controls such as for example, spin, flip, and/or rotate on a 3D scout map, such that a quick multi-planar reformatting in arbitrary oblique planes simulates a “fly through” of the image displayed in non-linear trajectories. For example, this software also may be interfaced with a commercial visualization package, such as, Explorer™ Version 3.0, available from Numerical Applications Group, Downers Grove, Ill. The visualization package may be used to render and manipulate the image data. For example, these packages include viewers that allows a “fly through” of the difference maps and the 3D data set. The “fly-through” capability may be used to scout objects of interest, which may then be selected or otherwise identified for viewing using the full 3D data. Image data compression may be used that include lossy and/or lossless algorithms, such as, for example, IMCompress™, available from InnovMetrics Software, Quebec, Canada.

[0032] In the exemplary embodiments, system 10 may acquire 3D image data spanned over a period of time, or 4D data sets. Processor 30 may compute the differences between 3D data sets to measure a quantity of change between the 3D data sets. This change in quantity may be graphically displayed as a linear map such as display 310, to provide image representation information over a selected time span. For example, a greater value of change may signify a relatively more diagnostically interesting period in time, while a lower value of change may indicate that the diagnostic information within the period of time is relatively low. Processor 30 may automatically compute and display a difference map of received data over a period of time. The difference map may correspond to display 310 that illustrates an amount of image change between images over the period of time.

[0033] In the exemplary embodiment, a multi-dimensional structure, such as, a 3D structure, may be located within volume 36. A difference map of the multi-dimensional structure may be illustrated on ultrasound image area 302. Multi-dimensional structure may be identified manually via a user input while viewing the structure in a 3D rendering. The identified data may then be tagged such that processor 30 recognizes the tag associated with the data and provides an identification of the multi-dimensional structure on ultrasound image area 302. In an alternative embodiment, the multi-dimensional structure may be identified by matching features of the multi-dimensional structure with a plurality of structure models that may be stored in the memory of system 10. A comparison of model features to features of the multi-dimensional structure may be performed by processor 30 using, for example 3D image data, or may be performed using difference map data. Selection of a specific model to use for comparison and the comparison to the multi-dimensional structure may be performed automatically by processor 30. A user may reject the automatic selection using user input 26 wherein processor 30 may re-perform the selection and comparison process. Processor 30 may be programmed to “learn” from incorrect selections such that over time the selection of which model to use for comparing increases in accuracy.

[0034] FIG. 4 is a flow chart of an exemplary method 400 of displaying images for ultrasound system 10 (shown in FIG. 1). Method 400 includes receiving 402 an image data set of a region of interest in a first dimensional representation, for example a 3D representation, and reducing 404 the dimensionality of the image data set to a second dimensional representation. For example, data received in a 3D representation may be difficult and time consuming to navigate, but reducing the dimensionality of the image data set also facilitates reducing the data representation of the image data, such that a feature of interest may be found in the reduced dimensionality image set and selected 406 for display by generating 408 an image of the selected feature in the first dimensional representation.

[0035] System 10 includes software and/or hardware controls for selecting views of images from a plurality of view perspectives using input devices associated with a view being displayed. A rendered volume may be rotated and/or flipped about a selectable axis to provide a view perspective from any direction. The rendered volume may then be sliced relative to the view perspective selected and reduced in dimensionality according to a selected difference map algorithm. The reduced dimensionality image display and the first dimensionality image display and/or a difference map display may be displayed on user interface 300. The reduced dimensionality display permits a user to quickly identify features of interest and locate the feature of interest. The feature of interest may be selected using a cursor and index indicator, and system 10 may then display the first dimensional representation of the feature of interest on ultrasound image area 302.

[0036] Image controls may be selectably included in user interface depending on whether the control function is available during a particular imaging procedure. For example, an image contrast control slider may only be displayed during a period when adjusting an image contrast is a permitted adjustment. Adjustment may be selectively permitted based on a setup for a particular user and/or a default set-up. Additionally, image controls may be selectively displayed and/or enabled based on a previous use history for each user or for all users. Image control input devices may comprise, but are not limited to, sliders, click selectors, and drop-down menus.

[0037] System 10 is also configured to receive input from a plurality of hardware pointing devices, such as, but not limited to, haptic controls for facilitating navigation through image representations. In the exemplary embodiment, processor 30 cooperates with the video processor within display system 32, and holographic projection device 40 to generate a holographic projection image of the image representation. Display system 32 and holographic projection device 40 may be configured to display multidimensional data on wearable display devices, for example, a head mounted display device.

[0038] Exemplary embodiments of apparatus and methods that facilitate displaying imaging data in ultrasound imaging systems are described above in detail. A technical effect of reducing the dimensionality of ultrasound image data from a first dimensional representation to a reduced dimensional representation, searching the reduced dimension data for a feature of interest and displaying the feature of interest in the first dimensional representation as described herein include at least one of facilitating visualizing regions of interest in a scanned object.

[0039] It will be recognized that although the system in the disclosed embodiments comprises programmed hardware, for example, software executed by a computer or processor-based control system, it may take other forms, including hardwired hardware configurations, hardware manufactured in integrated circuit form, firmware, among others. It should be understood that the intensity projection map processor disclosed may be embodied in a hardware device or may be embodied in a software program executing on a dedicated or shared processor within the ultrasound system or may be coupled to the ultrasound system.

[0040] The above-described methods and apparatus provide a cost-effective and reliable means for facilitating the viewing multi-dimensional image data by reducing the dimensionality of the image data to reduce the computing power needed to scout the data to select a view. More specifically, the methods and apparatus facilitate improving visualization of multi-dimensional data. As a result, the methods and apparatus described herein facilitate operating multi-dimensional ultrasound systems in a cost-effective and reliable manner.

[0041] Exemplary embodiments of ultrasound imaging systems are described above in detail. However, the systems are not limited to the specific embodiments described herein, but rather, components of each system may be utilized independently and separately from other components described herein. Each system component can also be used in combination with other system components.

[0042] While the invention has been described in terms of various specific embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the claims.