Personalized motion imaging system

[0001] The present invention relates to imaging systems that incorporate personal images in computer generated image sequences. More particularly the invention relates to such systems that include a camera for capturing an image of a viewer and incorporate the captured image into a computer generated motion image sequence.

[0002] Digital motion imaging systems are traditionally used for both passive and interactive viewing. In the former case, pre-defined image sequences are presented to a viewer by, for example, a digital television or digital cinema experience. In the latter case, choices made by the viewer affect the images presented, for example in computer gaming. Passive viewing experiences are often created by mixing pre-stored image sequences (on videotape or film, for example) to create the experiences. Live (real-time) motion image sequences are often passively viewed as part of the electronic media, for example with sporting events, news shows, and the like. In contrast, interactive viewing experiences are commonly found as part of computer gaming activities in which a computer generates a scene that is generally rendered from a database of objects describing the scene. Viewer interaction through some interface device (such as a joystick or mouse) provides additional information for suitably rendering the scene.

[0003] The integration of images from multiple sources is often found as part of image sequence viewing experiences, both passive and interactive. For example, multiple scenes are presented within a larger scene or multiple image sequence sources are viewed simultaneously on a single image-sequence viewing device. However, these image sequences are not personalized, that is they do not include real-time motion image sequences of the viewers. Commercial experience has shown, for example with the Kodak Fantasy Theater and Thrill Ride imaging systems, that personalized images have a greater appeal to many image viewers than images without personalization. Moreover, for some applications, the integration of an image of the viewer himself or herself with a computer generated image sequence adds to the enjoyment of the viewing experience, for example with group gaming activities. In these applications, the presentation of participants within the image sequences viewed adds verisimilitude and additional enjoyment to the experience. See for example, U.S. Pat. No. 4,710,873, issued Dec. 1, 1987 to Breslow et al. which discloses incorporating a video still image of a viewer into an image sequence in an interactive display device. It is also known to provide interaction between multiple viewers within some imaging environments to enhance the enjoyment of viewers and improve the effectiveness of the imaging experience. See for example U.S. Pat. No. 4,521,014 issued Jun. 4, 1985 to Sitrick. The problem with these approaches is that the image of the viewer is static, thereby reducing the impact of the effect.

[0004] There is a need therefore for an improved real-time motion imaging system that provides personalization, improves communication, and does not require preparation.

[0005] The need is met according to the present invention by providing a personalized motion imaging system that includes: a computer; a motion image display device connected to the computer for displaying a motion image sequence to a viewer; a camera connected to the computer for capturing an image of the viewer; and an image processing program running on the computer for generating a real-time motion image sequence of an environment and integrating a motion image sequence of the viewer into the computer generated motion image sequence of the environment to produce a composite motion image sequence for display on a motion image display device.

[0006] The present invention has the advantage that it provides a real-time motion imaging system that provides personalization, improves communication, and does not require preparation.

[0009] Referring to FIG. 1, a personalized motion imaging system 10 includes a computer 12, a motion image sequence display device 14 connected to the computer, and a camera 16, such as a motion image camera, also connected to the computer. A variety of programs for compositing and animating image sequences can run on the computer.

[0010] The composition and animation of image sequences can operate in a variety of ways. In a first embodiment, a viewer 17 stations herself in front of the viewing device 14 and observes a motion image sequence generated by the computer 12. The camera 16 records the viewer in real-time and transmits an image sequence of the viewer to the computer 12. The computer 12 then composites the image sequence of the viewer with a computer generated image sequence to create a personalized, real-time image sequence and displays the personalized composite image sequence to the viewer. Known composition techniques using a blue or green screen to differentiate the subject from the background may be employed. See for example U.S. Pat. No. 5,867,282 issued Feb. 2, 1999 by Fredlund et al.

[0011] In an alternative embodiment, a still image of the viewer is captured by the camera 16 and transmitted to the computer 12. The computer then creates an animated image of the viewer using the still image and an animation routine. This animated image is then composited with the computer generated image sequence to form the personalized composite image sequence.

[0012] In either embodiment described above, the computer generated image sequence is generated from a stored set of image objects. The computer generated image sequence content is structured so as to complement the captured or animated personal image sequence. The computer generated image sequence is provided with locations in the individual images of the computer generated sequence where the individual images from the personalized image sequence or animation can be located so as to provide a suitable artistic and enjoyable sequence for the viewer. For example, in a game with opposing players, each of whom is controlling a virtual vehicle, the position of the vehicle operator can be fixed within the vehicle and a personalized image sequence composited into the vehicle operator position.

[0013] The present invention has applications for both single viewer environments and environments in which there are multiple viewers. For example, in the single viewer case a viewer sees an image of himself integrated into the image sequence he views. Referring to FIG. 2, in the multiple viewer case, a viewer 17 is seen by a second viewer 17′ viewing a related scene displayed on a second personalized motion imaging system 10′. Each viewer sees the other person and/or herself in the image sequence. Both of these cases are applicable to computer game playing with either single or multiple players. The multiple viewer case can be extended to more than two viewers to the extent supported by the computer and the image sequences.

[0014] The verisimilitude of the imaging environment can be greatly enhanced with the use of stereo imagery and a stereoscopic viewing device. A computer generated stereo image sequence is composited with a personalized stereo image sequence to produce a stereo composite image sequence. The personalized stereo image can be acquired with the use of a stereo camera; multiple image acquisition of the viewer from different angles using a single camera; or by the generation of a stereo animation from either single or multiple still image(s) as known in the art.

[0015] The composite stereo image sequence may be viewed in a variety of ways. Stereo image goggles can be used, for example, but the viewer will appear in the image with the goggles. For conventional viewing without goggles, a monocentric autostereoscopic display device such as that disclosed in U.S. Ser. No. 09/738,747 filed Dec. 15, 2000 by Kessler et al., and now issued as U.S. Pat. No. 6,416,181 B1 may be used. Alternatively, other stereo display devices known in the art may be employed.

[0016] The personalized display system may be extended to include other means of interaction in a single- or multi-viewer environment. For example, a multi-sensory experience can be provided by including audio speakers 15 to provide sound to the viewer. These sounds are typically associated with the image sequence being viewed, as in a motion picture or a computer game with sound effects. In a multi-viewer environment, the sound effects can be augmented by verbal interactions between players. These verbal interactions can be implemented with the use of microphones 18 and audio mixing devices (either separate or integrated into the computer) for incorporating the verbal interactions into the image sequence sound track or sound effects.

[0017] In some applications, live verbal interactions between multiple viewers may not be desired. For example, a personalized representation of a viewer may utter pre-stored speech sequences stored within the computer. These speech sequences may be stored as digitized audio clips or synthesized in real-time from stored text. In any case, to the extent that the personalized image sequence, whether animated or not, can be seen to speak, the speech sequences are synchronized with the personalized image sequence. This capability is known in the prior art, for example with the RealSpeak product from Lemout & Houspie, and the animation products from LifeFX Networks, Inc.

[0018] It is also possible to add additional verisimilitude to an animated, personalized image sequence by artificially animating emotions corresponding to the content of the image sequence. A variety of facial expressions are known to connote particular expressions; by animating personalized facial expressions at appropriate moments, viewers may further enjoy the viewing experience and interaction with other viewers. Such emotional animation is known in the art, for example by LifeFX Networks, Inc.

[0019] A viewer's enjoyment of the imaging experience available through the present invention can be further enhanced with the addition of mementos of the experience. These can be provided by connecting an image output device 20 to the computer 12. This image output device 20 might write any of a variety of images on a variety of media, such as traditional hard-copy images, three-dimensional lenticular images, image sequences on videotape or digital versatile disks (DVD) 22, and the like. The image content may be drawn from the composited image sequence and may include multiple viewers and text reminiscent of the viewing experience. For example, a hardcopy print might be provided that includes a representative image from the composited sequence, suitable branding identification, personal images, and text describing the name of the experience and any scoring achievements. As another example, a DVD might be provided with the digital image sequence written thereon together with images of all of the participants. As another example, a lenticular image print might be provided that includes several selected scenes from the sequence incorporating personal images and experience identification. See for example U.S. Pat. No. 5,764,231 issued Jun. 9, 1998 to Ray et al.

[0020] The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.

• [0021] 10 personalized motion imaging system
• [0022] 10′ second personalized motion imaging system
• [0023] 12 computer
• [0024] 14 digital motion image display
• [0025] 15 audio speakers
• [0026] 16 camera
• [0027] 17 viewer
• [0028] 17′ second viewer
• [0029] 18 microphone
• [0030] 20 output device
• [0031] 22 DVD