Apparatus for receiving broadcast data, method for displaying broadcast program, and computer program

[0001] 1. Field of the Invention

[0002] The present invention relates to a display control technique of data broadcast content in which characters, image, voice, video are multiplexed.

[0003] 2. Description of the Related Art

[0004] As digital satellite broadcasting service has started, data-broadcasting service is received by not only personal computers but also television receivers. With the television receiver having a receiving function of receiving data broadcast data and a replay software program (a data broadcast browser), even a user who has no personal computer views a variety of pieces of information and displays texts and images, associated with a television program, on the screen of the television receiver. Standards controlling such a service is Standard ARIB-STD-B24 formulated by the Association of Radio Industries and Business (ARIB) in Japan.

[0005] TV stations encodes ordinary TV program data, audio data, and data broadcasting data. After being multiplexed, these pieces of data are modulated and then transmitted from an antenna. A signal transmitted from the antenna is then delivered to a user terminal (a television receiver) through a broadcasting satellite (BS).

[0006] FIG. 17 is a block diagram showing the construction of a typical receiver for receiving typical digital satellite broadcast signals. The Standard of the ARIB permits the resolution of a video plane 107, still image plane 108, and character and drawing image plane 109 to be different from receiver to receiver. For example, the still image plane 108 may have two types of resolutions, namely, a resolution of 1920 pixels (horizontally) by 1080 lines (vertically), and a resolution of 720 pixels (horizontally) by 480 lines (vertically). With the still image plane 108 of a resolution of 720×480, an aspect ratio of width frame to height frame may be 16:9 or 4:3 on a display.

[0007] With the still image plane 108 of a resolution of 1920×1080, each pixel on the still image plane 108 is displayed as a square. But with a resolution of 720×480, each pixel of the still image plane 108 has a slightly horizontally elongated rectangular shape if the aspect ratio of the display is 16:9, and has a slightly vertically elongated rectangular shape if the aspect ratio of the display is 4:3. The display is available in a variety of sizes, for example, 14 inches, 19 inches, and 33 inches.

[0008] Digital data is repeatedly transmitted from TV stations in the data carousel method standardized in ISO/IEC13818-6. Digital data, filtered by a transport decoder, contains text information, script information, image information, and video and audio data. Text information is described using XML (extensible Markup Language) regulated by the Worldwide Web Consortium (W3C).

[0009] Attempts to provide services such as electronic shopping using data broadcasting have been made. For example, when a television-shopping program is in progress, detailed information about goods is delivered using data broadcasting. A user operates the television receiver to monitor the detailed information about a commercial product currently being introduced during the program.

[0010] In the electronic shopping using the television receiver, the size of the displayed image of each commodity becomes different depending on the size of a TV screen in use and the resolution of the TV. The real size of each commodity and the size of the commodity which users feel according to the displayed image fail to match each other.

[0011] Japanese Laid-Open Patent No. 9-006944 discloses an electronic catalog system to reproduce the size of the commodity. According to the disclosure, the electronic catalog system scales the image of a product in accordance with the size of a display screen to present the image of the product in the real size thereof.

[0012] As described above, however, the receiving environment such as the resolution of the still image plane 108 of the TV receiver, the size of the display, and the aspect ratio of the display screen during presentation may be different from user to user.

[0013] The technique, disclosed in the above-cited Japanese Laid-Open Patent No. 9-006944, simply performs a scaling process taking into consideration the screen size only. The aspect ratio of an original product may change depending on the specifications of a TV receiver of a user. This technique is unable to present a product in its real size, and thus it is difficult to cause the impression of the product on the TV display screen to the user to match the actual impression of the product.

[0014] In accordance with the cited disclosure, the user needs to input data of the screen size, and the image data of the product in the real size thereof is generated from particular image data. Depending on a combination of the size of a product, display screen size, and resolution, the image quality of the image of the product is substantially degraded during real-size presentation. On-screen realism of the product is thus damaged.

[0015] For example, if a product as large as 1 meter high is displayed on a 14-inch TV monitor in the real size thereof as shown in FIG. 18A, a portion of the product is conventionally displayed in enlargement. The image quality is thus degraded.

[0016] In the example shown in FIGS. 18A and 18B, the image of a “bicycle” as long as 1.5 m is presented in the real size thereof on a 14-inch size TV screen. A portion of the image shown in FIG. 18A is thus presented as shown in FIG. 18B. If the resolution of the image shown in FIG. 18A is not high enough, image quality substantially decreases as shown in FIG. 18B. Realism of the bicycle is lost.

[0017] It is an object of the present invention to provide an apparatus, method and computer program for displaying an image of an object in a shape reflecting the size of the original object from image data transmitted through data broadcasting, regardless of the specifications of a receiver.

[0018] In an aspect, the present invention relates to an apparatus for receiving broadcast data, and includes a receiver for receiving image data of the image of an object and size information relating to the object, transmitted through data broadcasting, an input unit for inputting screen information relating to a display screen of display, a scaling unit for determining a scale factor of the object image based on the screen information input by the input unit and the size information received by the receiver so that the object image is displayed on the display at a predetermined size, and a display control unit for controlling the display to display the object image expanded or contracted at the scale factor determined by the scaling unit.

[0019] Further objects, features, and advantages of the present invention will be apparent from the following description of the preferred embodiments with reference to the attached drawings.

[0055] The embodiments of the present invention will now be discussed with reference to the drawings.

[0056] FIG. 1 is a block diagram showing the construction of a digital TV receiver apparatus 100 incorporating the present invention.

[0057] Referring to FIG. 1, a signal received by an antenna is input to a tuner 101. The tuner 101 performs processes, such as demodulation and error correction, on the input signal, and reproduces digital data referred to as transport stream (TS). The reproduced transport stream (TS) data is output to a descrambler 102.

[0058] Upon receiving, from the tuner 101, the TS data that is scrambled to restrict viewing, the descrambler 102 descrambles the TS data based on key information contained in the TS data for descrambling and key information output from an IC card controller 117, and outputs descrambled TS data to a transport decoder 103.

[0059] The IC card controller 117 contains an IC card which stores user contract information and key information for decrypting the key information for descrambling contained in the TS data. When the key information for decrypting the key information for descrambling is present, the IC card controller 117 outputs the key information to the descrambler 102.

[0060] Upon receiving, from the tuner 101, the TS data not scrambled, the descrambler 102 outputs, the TS data to the transport decoder 103. The transport decoder 103 extracts a packet corresponding to a program selected by the user.

[0061] The video data in the extracted packet is decoded by a video decoder 106, and is then sent to a moving image plane 107 in a graphic buffer 121. The audio data in the extracted packet is decoded by an audio decoder 104, and is then output to a loudspeaker 133 through a D/A converter 105. Data broadcast data in the extracted packet is fed to a memory 112 through a bus 122 connected to a CPU 118. The data broadcast data, after being decoded by the CPU 118, is stored in a hard disk 119.

[0062] When the data broadcast data is displayed, the CPU 118 reads the data broadcast data stored in the hard disk 119, converts the data broadcast data into image data of characters and drawings using a graphic generator 110, and then sends the image data to one of character and drawing image plane 109 and still image plane 108 in the graphic buffer 121.

[0063] The character and drawing image plane 109 chiefly stores bit images such as buttons, texts, drawings, and pictures, serving as UI (user interface) in data broadcasting. The still image plane 108 stores bit images of photographs.

[0064] An image synthesizer 111 synthesizes and switches graphic data stored in each of the moving image plane 107, character and drawing image plane 109, and still image plane 108 in the graphic buffer 121, and outputs the resulting data to a display 134.

[0065] The display 134 displays the incoming image data. The display 134 has a memory storing information, such as a screen size of the display 134. Information stored in the memory is sent to the digital TV receiver apparatus 100 through a serial bus connected between the digital TV receiver apparatus 100 and display 134, and is thus sent to and stored in the memory 112 controlled by the CPU 118 through a serial interface 120 and bus 122.

[0066] Also connected to the bus 122 are an IEEE1394 interface 114 and a modem 115. The IEEE1394 interface 114 is used for the digital TV receiver apparatus 100 to perform protocol communication with an external VCR (Video Cassette Recorder) 130 and a printer 131. The modem 115 is used to be connected to the Internet through a telephone line.

Data Broadcasting

[0067] In the data broadcasting, digital data is repeatedly transmitted from TV stations in the DSM-CC data carousel method formulated in standard ISO/IEC13818-6. Digital data, filtered by the transport decoder 103, contains text information, script information, image information, and video and audio data. Text information is described using XML (extensible Markup Language) regulated by the W3C.

[0068] This embodiment employs specification based on XHTML specification which is standardized by reformulating HTML4.0 with XML1.0. In XML, a character sequence in a document is attributed to (defined by) a description between tag < > and tag </>. The tag may have a nested structure. The format is CSS (Cascading Style Sheet), and the script process uses Java (Trade Name) Script.

[0069] In this embodiment, the data broadcasting is linked with a shopping program. An object image captured from a product, being currently introduced in the shopping program, is multiplexed on the data broadcast data as a still image, and is then transmitted from a TV station. A plurality of types of object image data of a single product, such as a general image of the product, and an image that is obtained by enlarging a portion of the product to express texture of the product, are multiplexed to the data broadcast data.

[0070] FIG. 2 shows a software configuration operating on the CPU 118. A data broadcasting browser 201 is an application running on an operating system 200. The data broadcasting browser 201 controls the graphic generator 110 using a graphics library 202 and graphics driver 203, thereby rendering characters, drawings, and image on the graphic buffer 121.

[0071] The data broadcasting browser 201 also controls the image synthesizer 111 using a window control middleware 204 and image synthesis driver 205, thereby synthesizing moving images, still images, characters, and drawings. The data broadcasting browser 201 controls the transport decoder 103 using a transport decoder driver 206, thereby performing filter setting on a moving image and voice to be reproduced, and receiving data broadcast data. The data broadcasting browser 201 further controls the serial interface 120 using a serial driver 207, thereby reading the screen size and aspect ratio of the display 134. The data broadcasting browser 201 communicates with a remote controller 132 through a remote controller driver 208, thereby receiving operational inputs from a user.

[0072] The operation of the digital TV receiver apparatus 100 thus constructed is discussed below.

Operation of Data Broadcasting Browser

[0073] FIG. 3 is a flow diagram showing the process of the data broadcasting browser 201. When the user requests the digital TV receiver apparatus 100 to present broadcast data using the remote controller 132 during viewing a TV program, the CPU 118 initiates a data broadcast receiving and replaying program (data broadcasting browser 201) in step S301.

[0074] The running data broadcasting browser 201 acquires size information of the screen and the aspect ratio from the display 134 (step S302). The digital TV receiver apparatus 100, connected to the display 134 through the serial bus, reads the screen size information and aspect ratio information stored in the memory of the display 134 through the serial bus. For example, information, such as “screen size=19.2 inches and aspect ratio=16:9”, is thus acquired. The data broadcasting browser 201 stores the acquired information in the memory 112.

[0075] The data broadcasting browser 201 controls the transport decoder 103, thereby receiving data broadcast contents transmitted in the data carousel method. The data broadcasting browser 201 then acquires on-screen XML data and its associated data, such as an image, embedded in the XML data, and store these pieces of data in the memory 112 (step S303).

[0076] FIG. 4 shows an example of a startup screen of data broadcast XML data received by the digital TV receiver apparatus 100 and correspondingly associated with a shopping program. This example includes a component for <object> (object tag labeled (A) in FIG. 4) displaying a program on air, a component for a link button (labeled (B) in FIG. 4) displaying detailed information of a product currently introduced in the program, and components for link buttons (labeled (C), (D), and (E) in FIG. 4) for switching to a general view, texture view (enlarged view), real-size view, and order screen.

[0077] Each component is subject to style definition defining a display position and display format. For example, an object component labeled (A) in FIG. 4 is positioned within an area extending by a width of 1695 pixels from a position 40 pixels rightward from the left edge of the screen and a height of 856 pixels from a position 60 pixels down from the top of the screen. Each link button has a description of XML data which is displayed when the user presses the link button.

[0078] When an image associated with the XML data is acquired, the data broadcasting browser 201 executes a script function described in OnLoad attribute of <body> tag in the XML data, as an initial script (step S304 in FIG. 3). A script is described between <script> and </script> in FIG. 4, and initialize function is an initial script in this example.

[0079] FIG. 5 is a flow diagram of the initial script process described in the startup screen XML data shown in FIG. 4.

[0080] In the initial script, a function is invoked to inquire of the current data broadcasting browser 201 about the working environment, in other words, to inquire whether the digital TV receiver apparatus 100 has a real-size presentation feature (step S501 in FIG. 5).

[0081] The data broadcasting browser 201 has GetBrowserSupport ( ) as a builtin API (ApplicationProgramInterface) which is invoked from the script. If a character sequence “realsize” as an argument is given, the API returns a value “1” (TRUE) or “0” (FALSE) as a variable representing display permission attribute.

[0082] Since the digital TV receiver apparatus 100 and the data broadcasting browser 201, now under discussion, have a real-size presentation feature, the return value of the API is TRUE (step S503 in FIG. 5). If this script is executed on the data broadcasting browser 201 having no real-size presentation feature, the return value of the API becomes FALSE (step S504 in FIG. 5).

[0083] In the above script, the return value of the API is substituted for a “visibility” attribute which decides whether or not to permit displaying of “a link button to real-size presentation”. In other words, if the return value of the API is TRUE, “visible” is substituted for the visibility attribute to enable displaying the link button to the real-size presentation. If the return value of the API is FALSE, “hidden” is substituted for the visibility attribute to disable displaying the link button to the real-size presentation.

[0084] Subsequent to the execution of the initial script, the data broadcasting browser 201 renders the button image and text on the character and drawing image plane 109 using the graphic generator 110 (step S305 in FIG. 3). At the same time, the data broadcasting browser 201 controls the image synthesizer 111, thereby synthesizing the button image and text with an image in the moving image plane 107 (step S306 in FIG. 3).

[0085] FIG. 6 shows a startup screen (main screen) which is generated by the data broadcasting browser 201 from the XML data. A rectangle labeled a is a moving image window where an image of the moving image plane 107, such as a video of TV shopping, which is normally displayed on the entire screen, is presented in a contracted form. On the remaining portion other than the moving image window, the data broadcasting browser 201, namely, the CPU 118 displays drawings and characters rendered on the character and drawing image plane 109 using the graphic generator 110. A link button b for the real-size presentation shown in FIG. 6 appears because the variable of the display permission attribute is TRUE.

General Presentation

[0086] When the user selects the general presentation button c on the main screen shown in FIG. 6 using the remote controller 132, general presentation display screen XML data is newly acquired, and the digital TV receiver apparatus 100 presents a screen based on the content of the XML data. The general presentation displays a still image of an entire product within the display screen. The data broadcasting browser 201 renders received still image file data onto the still image plane 108. The data broadcasting browser 201 also controls the image synthesizer 111, thereby presenting a screen formed of a moving image window for program presentation, a still image window for still image presentation, and a link button for returning to the main screen shown in FIG. 6.

[0087] FIGS. 7A and 7B show display examples of the screen. FIG. 7A shows a display example presented on a display 134 having a diagonal size of 29 inches, and FIG. 7B shows a display example presented on a display 134 having a diagonal size of 19.2 inches.

Texture Presentation

[0088] The user may select a texture presentation button d on the main screen shown in FIG. 6 using the remote controller 132, texture presentation display screen XML data is newly acquired, and the digital TV receiver apparatus 100 presents a screen based on the content of the XML data. FIGS. 8A and 8B show display examples of the screen. FIG. 8A shows a display example presented on a display 134 having a diagonal size of 29 inches, and FIG. 8B shows a display example presented on a display 134 having a diagonal size of 19.2 inches.

[0089] Unlike the general presentation screen, the texture presentation screen presents a still image which is captured within a close range to a product so that a texture of the product is seen. The data broadcasting browser 201 renders received still image file data to the still image plane 108. The data broadcasting browser 201 also controls the image synthesizer 111, thereby presenting a screen formed of a moving image window for program presentation, a still image window for still image presentation, and a link button for returning to the main screen shown in FIG. 6.

Real-size Presentation

[0090] When the user selects a real-size presentation button b on the main screen shown in FIG. 6 using the remote controller 132, XML data for the real-size presentation is newly acquired, and the digital TV receiver apparatus 100 presents a screen based on the content of the XML data. FIGS. 9 and 10 show the XML data.

[0091] FIG. 11 is a flow diagram of an initial script process described in the XML data shown in FIGS. 9 and 10. The initial script is mainly carried out in first through fourth processes as described below.

[0092] In the first process, the screen information such as the display size and aspect ratio of the display 134 is acquired (step S1201).

[0093] In the second process, the real-size presentation parameter calculation process is carried out using the XML data and the general image of the product (steps S1202–S1203).

[0094] In the third process, an appropriate image determination process is carried out to determine whether an expansion rate and image size are appropriate (steps S1204–S1206). If it is determined that the image is not appropriate, the second step is carried out on another image (step S1207). If no other image is available, the algorithm proceeds to the fourth process.

[0095] In the fourth process, the size of the moving image window is determined taking into consideration the final window size for real-size image presentation (step S1208).

[0096] The first through fourth processes will be discussed in more detail.

[0097] The acquisition of the screen size and aspect ratio in the first process is performed by invoking GetScreenInfo ( ) as the API contained in the data broadcasting browser of the present invention from the script (see label (F) in FIG. 9).

[0098] If any of four type of character sequences “height”, “width”, “size”, and “ratio” as an argument is designated, the API returns a return value responsive to the argument. The arguments height and width respectively represent the number of pixels in a horizontal direction and the number of pixels in a vertical direction in a maximum possible size of the still image plane 108 of the digital TV receiver apparatus 100. The argument size represents a diagonal size of the display 134 of the digital TV receiver apparatus 100. The argument ratio represents the aspect ratio of frame width to frame height of the display 134. Size “0” represents an aspect ratio of 4:3, and size “1” represents an aspect ratio of 16:9.

[0099] If a data broadcasting browser is displayed on the entire display screen of the digital TV receiver apparatus 100 of high-definition grade connected to the display 134 having a diagonal size of 19.2 inches and an aspect ratio of 16:9, the return values are height=1080, width=1920, size=19.2, and ratio=1. In the digital TV receiver apparatus 100 connected to the display 134 having a diagonal size of 14.6 inches and an aspect ratio of 4:3, the return values are height=480, width=720, size=14.6, and ratio=0.

[0100] When the script invokes the API, the data broadcasting browser 201 returns the values for size and ratio based on the information which has been stored in the memory 112 at a startup, and returns the values for height and width based on the resolution of the still image plane 108 of the digital TV receiver apparatus 100.

[0101] The real-size presentation is based on scaling information of a source image. A content producer of data broadcasting describes the already measured scaling information in the XML data. The script process of the data broadcasting browser 201 is thus simply to substitute values for variables src_height and src_width, and for variable src_dip which indicates the number of pixels per inch (see label (A) in FIG. 9).

[0102] The second process of calculating the real-size presentation parameter is carried out as illustrated in a flow diagram in FIG. 12. The second process assumes three combinations of the resolution of the still image plane 108 of the digital TV receiver apparatus 100 and the aspect ratio of the display 134 (see label (G) in FIG. 9). The present invention is not limited to these combinations, and calculations corresponding to various combinations may be performed.

• • [0103] (1) Resolution of 1920×1080 and aspect ratio of 16:9
  • [0104] (2) Resolution of 720×480 and aspect ratio of 4:3
  • [0105] (3) Resolution of 720×480 and aspect ratio of 16:9

[0106] Referring to FIG. 12, height is the number of pixels in a height direction in the still image plane 108, ratio is an aspect ratio of the display 134 (16:9=1, 4:3=0), src_dpi is the number of pixels per inch in the source image, K is the number of pixels per inch in the display 134, Zx is a horizontal zoom ratio to be applied to the source image, and Zy is a vertical zoom ratio to be applied to the source image.

[0107] The value “18.37559” in the right side of the equation defining K represents a ratio of a diagonal line of the display 134 based on the aspect ratio (16:9) of the display 134, namely, 18.37559²=16²+9². Similarly, the value “5” in the right side of another equation defining K represents a ratio of diagonal line of the display 134 based on the aspect ratio (4:3), namely, 5²=4²+3².

[0108] In the combination (1), an image rendered on the still image plane 108 remains unchanged from that of the data broadcast data. The horizontal and vertical zoom ratios applied to the source image equal each other, namely, Zx=Zy (step S1304).

[0109] In the combination (2), an image rendered on the still image plane 108 has a shorter width than the source image in the data broadcast data, and the horizontal and vertical zoom ratios applied to the source image become different, namely, Zx>Zy (step S1305).

[0110] In the combination (3), an image rendered on the still image plane 108 has a longer width than the source image in the data broadcast data, and the horizontal and vertical zoom ratios applied to the source image become different, namely, Zx<Zy (step S1303).

[0111] The third process of determining an appropriate image is performed in accordance with the flow diagram shown in FIG. 13. As shown, dst_height is the number of pixels subsequent to an expansion process of the source image in the height direction, height is the number of pixels of the browser window in the height direction, and Zy is a vertical zoom ratio to be applied to the source image.

[0112] In the third process, the image is determined as being not appropriate (step S1403) if the zoom rate in the height direction calculated in the second process is larger than a predetermined value (step S1401) and if the image size is larger than a predetermined value subsequent to the zooming of the source image in accordance with the zoom rate (step S1402); otherwise, the image is determined as being appropriate (step S1404).

[0113] In this embodiment, the image is determined as being not appropriate if the vertical zoom ratio is 400% and if the number of pixels in a height direction subsequent to the zooming process of the source image at a zoom ratio of 400% is 70% or more of the number of pixels of the browser window in a height direction (see label (H) in FIG. 10).

Production of Texture Image from Real-Size Image

[0114] If the image is determined as being not appropriate in the third process, the source image is switched from the general image to an enlarged image (texture image), and the algorithm starts over again from the second process (step S1207 in FIG. 11).

[0115] The adjustment of the size of the moving image window in the fourth process is carried out in accordance with a flow diagram shown in FIG. 14 (see label (I) in FIG. 10).

[0116] As shown, dst_height is the number of pixels in a height direction subsequent to a zooming process of the source image, dst_width is the number of pixels in a width direction subsequent to the zooming process of the source image, height is the number of pixels of the browser window in a height direction, width is the number of pixels of the browser window in a width direction, real_height is the number of pixels of a real-size image display window in height direction, real_width is the number of pixels of the real-size image display window in a width direction, mov_height is the number of pixels of the moving window in a height direction, and mov_width is the number of pixels of moving window in a width direction.

[0117] In the adjustment of the moving image window size, an image size (dst_height and dst_width) is determined by zooming in (zooming out) the source image according to the zoom rate (Zx, Zy) determined in the third process. The real-size display presentation window size (real_height, real_width) and the moving image window size (mov_height, mov_width) are determined so that the image size falls within a constant percentage (70%, for example, as shown in FIG. 14) of the (browser window) screen (from step S1501 to step S1503).

Display Examples

[0118] FIGS. 15A and 15B show display examples resulting from the process of the script described in the real-size display screen XML data. FIG. 15A shows a screen of the display 134 having a diagonal size of 29 inches and an aspect ratio of 16:9, and FIG. 15B shows a screen of the display 134 having a diagonal size of 19.2 inches and an aspect ratio of 16:9.

[0119] As seen from FIGS. 15A and 15B, the image of the product is maintained even if the display 134 changes the screen size thereof. Since the size of the moving image display window is adjusted to a maximum size in response to the window size of the real-size presentation, the TV is viewed under optimum environments.

[0120] FIGS. 16A and 16B show a 19-inch display on which an image of a commercial product having a length of 1 meter or longer, such as a bicycle, is presented. The data broadcasting browser 201 of this embodiment generates an image of interest of the product of this size, namely, a real-size image as an enlarged (texture) source image, as shown in FIG. 16A rather than a general image shown in FIG. 18A. Image quality of the real-size image is thus better than the one resulting from the source image shown in FIG. 18A.

[0121] In accordance with this embodiment, realism felt by the user is improved because the image processing apparatus that receives data broadcast data presents the image of a product or commodity at the real size thereof during viewing a TV shopping program. User confidence in the data broadcast TV shopping is thus promoted.

[0122] This embodiment allows the image of an object to be displayed at the real-size regardless of specifications of the display presenting the data broadcast program, such as the screen size and resolution.

[0123] In accordance with this embodiment, a real-size presentation content is replayed only when a receiver having a real size presentation feature receives a single data broadcasting content.

[0124] Since window sizes in a concurrent display of a TV video and a product in the program associated data broadcasting are optimized, product presentation is carried out with the screen efficiently utilized.

[0125] The image quality of the real-size image is improved, and real-size presentation is performed without the need for particular user setting.

[0126] This embodiment has been discussed in connection with the “data broadcast shopping” as a content of the data broadcasting. The data broadcasting content is not limited to this. The data broadcasting content may be the image of any object as long as the viewer wants to know the size of the object. For example, in a data broadcasting associated with an educational program, the content may be the image of animals or insects.

[0127] The image data for presenting a real-size image is not only still image data but also moving image data in which commercial products, mounted on a table, turn integrally with the table, for example. The moving image data of the commercial products may be transmitted using PES (PacketizedElementaryStream) as a data stream method.

[0128] The process steps for zooming the image data may be stored beforehand in the receiver rather than being contained in the transport stream transmitted in the data broadcasting. In this case, a data broadcasting apparatus simply transmits a reference display size of the image data, and is freed from the responsibility to transmit parameters for expansion and contraction in accordance with the receiver. The data broadcasting apparatus thus allows receivers having a variety of specifications to display a real-size image. The present invention finds widespread use.

[0129] In the electronic shopping using the TV receiver, users are unable to actually pick up and touch a product. Users are unable to try a dress on before purchasing it, for example, and cannot correctly picture to themselves in the dress.

[0130] To overcome such a drawback, virtual fitting may be contemplated by synthesizing image data representing the user's own image and image data of a commercial product on a display.

[0131] There are a variety of TV receiver types with display screens thereof varied in size and resolution available for virtual fitting. The aspect ratio of a displayed image of commercial product is different from TV receiver to TV receiver. The size of the image of the product conveyed to the user may fail to match the actual size of the product.

[0132] A plurality of broadcasters present a diversity of commodities in their programs. The products are substantially different in size, and the product images cannot be synthesized with the user image.

[0133] Electronic fitting or virtual fitting using broadcasting, communication, or separate storage media has been proposed. For example, according to Japanese Laid-Open Patent No. 5-336445 has proposed an image processing method which synthesizes and displays a commercial product image based on a posture and size of a personal appearance image when the personal appearance image of a user and the commercial product image are synthesized.

[0134] When the image processing method is applied to virtual fitting in an electronic shopping based on the data broadcasting and the TV receiver, different types of the product image must be broadcast concurrently in parallel as long as there is a possibility of being used by any user. This occupies a substantial portion of available data capacity permitted for data broadcasting, thereby disabling the transmission of image data of numerous sorts of commercial products.

[0135] Japanese Laid-Open Patent No. 7-210608 has proposed a sales assisting system which contracts or expands, synthesizes and displays product image data by obtaining the size of a user image from the user image or from an input unit.

[0136] When such sales assisting system is applied to a virtual fitting in an electronic shopping that uses the data broadcasting and TV receiver, it is required to display a diversity of types of product image data provided by a plurality of broadcasters. If the actual product image is too large with respect to the size of the user image, the TV screen cannot accommodate the product image.

[0137] Since the resolution and aspect ratio of the TV screen presenting the product are not taken into consideration, the user may have the impression of the product from the TV screen, different from the actual product, if the aspect ratio of the shown product changes.

[0138] It is an object of the present invention to overcome this drawback. It is also another object of the present invention to synthesize a user image and an image derived from the data broadcast data in a correct scale regardless of specifications of receivers.

[0139] A second embodiment of the present invention will now be discussed in detail referring to the drawings. In accordance with the second embodiment, the digital TV receiver apparatus 100 uses a data broadcasting browser having a virtual fitting function, thereby synthesizing and displaying a user image and an image transmitted using data broadcasting in a correct scale.

[0140] The digital TV receiver apparatus 100 of the second embodiment remains unchanged from that shown in the block diagram in FIG. 1. The data broadcasting browser of the first embodiment has the real-size presentation function, while the data broadcasting browser of the second embodiment has a virtual fitting function. The discussion of the operation of the major portions of the digital TV receiver apparatus 100 is omitted here. The operation discussed below relates to the initiation of the data broadcasting browser based on the XML data transmitted using the data broadcasting and the virtual fitting.

[0141] The CPU 118 in the digital TV receiver apparatus 100 operates in the same manner as already discussed with reference to FIG. 3 except the startup screen XML data in the execution of the initial script in step S304. As a result, the rendering process of the startup screen in steps S305 and S306 results in a startup screen (main screen) as shown in FIG. 22.

[0142] FIGS. 19 and 20 show an example of XML data of the startup screen of data broadcasting correspondingly associated with a shopping program. In this example, the XML data contains a component <object> (object tag, see label (A) in FIG. 20) for displaying a program on air, a link button (see label (B) in FIG. 20) for displaying detailed information of a product being introduced in a current program, and components (labeled (C), (D), and (E) in FIG. 20) for switching to a general view, texture view (enlarged view), real-size view, and order screen.

[0143] Each component is subject to style definition defining a display position and display format. For example, each component represents that a corresponding object is placed in an area at a position adjusted by the initial script in each display screen. Specifically, an object component labeled (A) in FIG. 20 is placed within an area extending by a width of 1536 pixels from a position 60 pixels down from the top of the screen, and by a height of 864 pixels from a position 40 pixels rightward from the left edge of the screen (see label (F) in FIG. 19). Each link button has a description of XML data which is displayed when the user presses the link button.

[0144] When an image associated with the XML data is acquired, the data broadcasting browser 201 executes a script function described in OnLoad attribute of <body> tag in the XML data, as an initial script (step S304 in FIG. 3).

[0145] A script is described between <script> and </script> in FIG. 19, and an initialize function becomes an initial script in this example.

[0146] FIG. 21 is a flow diagram of the initial script process described in the startup screen XML data shown in FIGS. 19 and 20.

[0147] In the initial script, a function is invoked to inquire of the current data broadcasting browser 201 about the working environment there, in other words, to inquire whether the digital TV receiver apparatus 100 has a virtual fitting feature (step S501 in FIG. 21).

[0148] The data broadcasting browser 201 has GetBrowserSupport ( ) as a builtin API (ApplicationProgramInterface) which is invoked from the script (corresponding to label (G) in FIG. 19). If a character sequence “fitting” as an argument is given, the API returns a value “1” (TRUE) if the digital TV receiver apparatus 100 has the virtual fitting feature or “0” (FALSE) if the digital TV receiver apparatus 100 has no virtual fitting feature.

[0149] Since the digital TV receiver apparatus 100 and the data broadcasting browser 201 of the second embodiment have a virtual fitting feature, the return value of the API is TRUE (step S503 in FIG. 21). If this script is executed on the data broadcasting browser 201 having no virtual fitting feature, the return value of the API becomes FALSE (step S504 in FIG. 21). In the above script, the return value of the API is substituted for a “visibility” attribute which decides whether or not to permit displaying of “a link button to virtual fitting page”.

[0150] The script inquires of the digital TV receiver apparatus 100 about display information of the display (step S505). The data broadcasting browser 201 has GetScreenInfo ( ) as the API which is invoked from the script (see label (H) in FIG. 19). If any of four types of character sequences “height”, “width”, “scale”, and “ratio” as an argument is designated, the API returns a return value responsive to the argument. The arguments height and width respectively represent the number of pixels in a horizontal direction and the number of pixels in a vertical direction in a maximum possible size of the still image plane 108 of the digital TV receiver apparatus 100.

[0151] The argument scale represents the number of pixels per unit length in a height direction of the display screen of the display 134. The argument ratio represents the aspect ratio of frame width to frame height of the display 134. Size “0” represents an aspect ratio of 4:3, and size “1” represents an aspect ratio of 16:9.

[0152] In the script, the size and the position of the button area are adjusted depending on the number of pixels in the height direction, and the aspect ratio of the display screen of the display 134 (step S506). The broadcast program of the second embodiment provides a moving image and a still image on a display screen having 1920×1080 pixels and an aspect ratio of 16:9. For example, as already discussed, the moving image is placed in an area extending by a width of 1536 pixels from a position 60 pixels down from the top of the screen, and by a height of 864 pixels from a position 40 pixels rightward from the left edge of the screen. If the acquired number of pixels of the display 134 in the height direction is 480, the moving image area may be changed to 336 pixels in width by 189 pixels in height.

[0153] Subsequent to the execution of the initial script, the data broadcasting browser 201 renders the button image and text on the character and drawing image plane 109 using the graphic generator 110 (step S305 in FIG. 3). At the same time, the data broadcasting browser 201 controls the image synthesizer 111, thereby synthesizing the button image and text with an image in the moving image plane 107 (step S306 in FIG. 3).

[0154] FIG. 22 shows a display screen which is generated by the data broadcasting browser from the XML data of a startup screen. As shown, a rectangular portion 701 is a moving image window, and typically presents an image on the moving image plane 107, normally presented on the entire display screen. The rectangular portion 701 thus presents an ordinary TV video in a contracted form. On the remaining portion other than the moving image window, the data broadcasting browser 201, namely, the CPU 118 displays drawings and characters rendered on the character and drawing image plane 109 using the graphic generator 110.

[0155] Also shown are a link button 702 which requests a link to a screen which provides detailed information about a commercial product introduced in the moving image window, for example, material, size and price. A link button 704 requests a shift to a screen that presents the general image of the product. A link button 705 is used to perform a virtual fitting to be discussed later, and is displayed because a variable representing the display permission attribute is TRUE. A link button 706 is used to instruct the presentation of the image of an object at the real size thereof.

General Presentation

[0156] When the user selects the “real-size presentation” link button 704 using the remote controller 132 with the main screen shown in FIG. 22 presented, the real-size display screen XML data is newly acquired. The display based on the content of the XML data is presented. The real-size display screen presents a still image which is the general image captured from an entire product.

[0157] The data broadcasting browser 201 renders the received still image file data on the still image plane 108. The data broadcasting browser 201 also controls the image synthesizer 111, thereby generating a screen 720 formed of a moving image window 722 for program presentation and a still image window 721 for still image presentation as shown in FIGS. 23A and 23B, and a return button 723 for returning to the main screen shown in FIG. 22. FIG. 23A shows a “sweater” as a clothing item on the display 134, and FIG. 23B shows an “earring” as an accessory on the display 134.

Virtual Fitting Display Page

[0158] The user may select a “fitting” link button 705 using the remote controller 132 on the display shown in FIG. 22. Virtual fitting display screen XML data is newly acquired, and a screen based on the content of the XML data is presented. FIGS. 24 through 30 show the XML data.

[0159] FIG. 31 is a flow diagram showing an initial script process described in the XML data of the virtual fitting screen shown in FIGS. 24 through 30. The script process is performed as described below.

[0160] An acquisition process for acquiring the resolution, aspect ratio, and scale of the display 134 is performed (see step S1002 in FIG. 31 and label (A) in FIG. 24). With the resolution, aspect ratio, and scale of the display 134 acquired (see label (B) in FIG. 24), a product image is adjusted to match the display screen (step S1003). With the resolution, aspect ratio, and scale information of the user image for virtual fitting registered in the digital TV receiver apparatus 100 (see label (C) in FIG. 25), a zooming process is performed on the user image so that the product image and the user image are synthesized at a correct scale. The product image and the user image are synthesized, thereby forming a virtual fitting image (step S1004).

[0161] It is then determined whether the synthesized virtual fitting image falls within the display area of the original product image (step S1005). If it is determined that the virtual fitting image falls within the display area, the sizes and synthesis positions of the product image and the user image are readjusted (see step S1006 in FIG. 31 and label D in each of FIGS. 26 and 27).

[0162] If it is determined that the virtual fitting image falls within the display area of the original product image, the sizes and synthesis positions of the product image and the user image are readjusted. The size and display position of the product image are set (see step S1007 in FIG. 31 and label (E) in FIG. 27), and the size and display position of the user image are set (see step S1008 in FIG. 31 and label (F) in FIG. 27).

[0163] Each of the above steps is further discussed in detail.

[0164] As already discussed, the acquisition process of the screen size and aspect ratio of the display 134 in step S1002 is carried out by invoking GetScreenInfo ( ) which is the API contained in the data broadcasting browser of the present embodiment from the script.

[0165] For example, a high-definition digital TV receiver apparatus 100 is connected to the display 134 having a screen size of 19.2 inches and an aspect ratio of 16:9. If the data broadcasting browser 201 is displayed on the entire display screen of the display 134, the values of height=1080 pixels, width=1920 pixels, scale=114 pixels/inch, and ratio=1 are acquired. In the digital TV receiver apparatus 100 connected to a display 134 having a screen size of 14.6 inches and an aspect ratio of 4:3, the values of height=480 pixels, width=720 pixels, scale=54.8 pixels/inch, and ratio=0 are acquired.

[0166] When the API is invoked from the script, the data broadcasting browser 201 provides return values for “scale” and “ratio” based on information which is read from the display 134 at startup through the serial bus and then stored in the memory 112, and provides return values for “height” and “width” based on the resolution of the still image plane 108 in the digital TV receiver apparatus 100.

[0167] In broadcast program of the present embodiment, moving images and still images are broadcast on the assumption that the screen has 1920×1080 pixels and an aspect ratio of 16:9. If the still image plane 108 of the digital TV receiver apparatus 100 has a different resolution, the size and display area of the still image must be adjusted. Scale factors kx and ky in width and height directions applied to information of the product image and user image are calculated in step S1002.

[0168] The three combinations of resolution and aspect ratio of the still image plane 108 of the digital TV receiver apparatus 100 of the present embodiment are contemplated. The present invention is not limited to these combinations, and calculations corresponding to various combinations may be performed.

• • [0169] (1) Resolution of 1920×1080 and aspect ratio of 16:9
  • [0170] (2) Resolution of 720×480 and aspect ratio of 4:3
  • [0171] (3) Resolution of 720×480 and aspect ratio of 16:9

[0172] In the combination (1), broadcast data is directly rendered, and thus kx=ky=1.00.

[0173] In the combination (2), an image rendered on the still image plane 108 has a shorter width. The width of the product image is rendered to be longer accordingly on the still image plane 108, namely, kx>ky.

[0174] In the combination (3), an image rendered on the still image plane 108 has a longer width. The width of the product image is rendered to be shorter accordingly on the still image plane 108, namely, kx<ky.

[0175] A content producer of data broadcasting describes, in the XML data, information of image data set beforehand in step S1203 for acquiring the resolution, aspect ratio, and scale of the product image. The data broadcasting browser 201 in the script process thereof substitutes values for variables smpl_width, smpl_height, smpl_scale, smpl_guidex, and smpl_guidey.

[0176] Here, smpl_guidex, and smpl_guidey are position coordinates at which the user image overlaps the product image, smpl_scale represents a scale where the user image overlaps the product image, and is expressed in the number of pixels per unit length in a height direction. These values must be adjusted for the display 134 of the digital TV receiver apparatus 100 using kx and ky calculated in step S1002 if the display 134 has 720×480 pixels, different from the assumed resolution in the broadcasting.

[0177] The acquisition process of the user image, resolution, aspect ratio, and scale thereof in step S1004 is carried out by invoking, from the script, GetUserInfo ( ) which is the API contained in the data broadcasting browser 201.

[0178] If arguments “face”, “chest”, and “figure” are input, the API returns references of an image data expressing the face, the upper half of the body of a user, and the entire body of the user. These pieces of the image data of the user is the image data the user has registered beforehand in the digital TV receiver apparatus 100 for virtual fitting. To perform electronic shopping, the image data is superimposed on product image data provided by data broadcasting.

[0179] The script of the data broadcasting designates a character sequence by selecting a type of user image in use for virtual fitting in accordance with a displayed product. The resolution, aspect ratio, and scale of the user image are associated with the user image, and are obtained by referencing the user image. Like the scale of the product image, the scale of the user image is represented in the number of pixels per unit length in a height direction.

[0180] Here, “factor” is a scale factor of contraction or expansion of the user image. The scale factor is used to set the product image and the user image in the same contraction or expansion rate in a synthesis process, and is used to contract or expand the user image to obtain a dimensionally correct synthesized image. The scale factor is obtained by dividing the “scale” of the product image by the “scale” of the user image. Using the scale “factor”, usr_width, usr_height, usr_guidex, and usr_guidey are adjusted. Since the aspect ratio of the user image is already adjusted to meet the display conditions of the display 134 in use, no further adjustment is required.

[0181] FIG. 32 shows a flow diagram of a readjustment process of the size and synthesis position of the product image and the user image as necessary so that the virtual fitting image falls within the display area of the original product image (steps S1005 and S1006).

[0182] An overextended portion of the user image beyond the product image in ±X directions is determined (step S1101). The product image contains guide coordinates (guidex and guidey) serving as a guide when the user image is synthesized (displayed in superimposition). The guide coordinates are relative coordinates having the origin thereof at the top left corner of each user image. If the user image is displayed in superimposition on the product image so that their coordinates agree with each other, a synthesized image for virtual fitting is aligned in a correct positional relationship.

[0183] The displaying of the synthesized image is performed in the procedure described in the body of the script. In step S1101, “offsetx” represents an amount of overextension of the user image in the −X direction from the product image when the two images are superimposed in accordance with the position provided by the guide coordinates, “remainx” represents an amount of overextension of the user image in the X direction from the product image, and “factorx” representing a contraction or expansion rate in the X direction is calculated so that the display area of the synthesized image falls within the display area of the original product image.

[0184] In step S1102, an overextended portion of the user image beyond the product image in ±Y directions is determined. Similarly, the scale factor “factory” in the Y direction is calculated from “offsety” and “remainy”.

[0185] The process for determining a scale factor “resize” for contracting the product image and user image in (steps S1103–S1105) is selecting a value “1” requiring neither contraction nor expansion and the smallest value of “factorx” and “factory”. In other words, if no overextension occurs, no contraction is required. A contraction rate in the direction of larger overextension is adopted. If no contraction is required with resize <1 (step S1106), the product image data and user image data are contracted, and the guide coordinates, and information of amounts of position adjustment “offsetx” and “offsety” of the product image required in response to the overextension of the user image are updated (step S1107).

Display Examples

[0186] Display examples of the data broadcasting browser 201 and the script process described in the virtual fitting screen XML data are discussed.

[0187] Referring to FIGS. 34A–34C, clothing “sweater” as a product is tried on. In this example, the user image of the “face” shown in FIG. 33B is registered in the digital TV receiver apparatus 100, and the product image shown in FIG. 33A is broadcast.

[0188] FIG. 34A shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 1920×1080 pixels, an aspect ratio of 16:9, and a diagonal size of 19.2 inches. FIG. 34B shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 720×480 pixels, an aspect ratio of 16:9, and a diagonal size of 14.6 inches. FIG. 34C shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 720×480 pixels, an aspect ratio of 4:3, and a diagonal size of 14.6 inches.

[0189] FIG. 34A shows a still image window 731 for presenting a synthesized image, a moving image window 732 for program presentation, a shift button 733 for shifting a synthesized image within the still image window 731, and a return button 734 for returning to the main screen shown in FIG. 7 (The same is true of FIGS. 34B and 34C).

[0190] In any of the displays 134, the user image is contracted or expanded to a correct size with respect to the product image, and synthesized and displayed. In the displays 134 having specifications different from specifications assumed in the data broadcasting, respectively shown in FIGS. 34B and 34C, the user image and product image with the correct sizes and correct aspect ratios maintained are synthesized and displayed, although the layout of the on-screen images are slightly different from that shown in FIG. 34A. A correct image of the product in the virtual fitting is thus conveyed using the data broadcasting.

[0191] Referring to FIGS. 35A–35C, “earrings” as an accessory are tried on. FIG. 35A shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 1920×1080 pixels, an aspect ratio of 16:9, and a diagonal size of 19.2 inches. FIG. 35B shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 720×480 pixels, an aspect ratio of 16:9, and a diagonal size of 14.6 inches. FIG. 34C shows a fitting image which is presented on the display 134 of the digital TV receiver apparatus 100. The display 134 has 720×480 pixels, an aspect ratio of 4:3, and a diagonal size of 14.6 inches.

[0192] As shown in FIGS. 35A–35C, the user image registered in the digital TV receiver apparatus 100 is appropriately contracted or expanded, and displayed with respect to the product image data although the product (the earrings) is substantially different in size from the product (the sweater) shown in FIG. 33A. A correct product image is thus conveyed to the viewer.

[0193] In this embodiment, the scale of the broadcast image data is adjusted to match the scale of the display screen of the receiver. The scale factor is obtained by dividing the scale of the broadcast image data by the scale of the user image data which is set to be coincident with the scale of the display screen and stored. The user image data is contracted or expanded according to the scale factor, and synthesized with the broadcast image data. The synthesized image data is then displayed on the display screen.

[0194] In the data broadcasting service such as an electronic (data broadcasting) shopping in accordance with this embodiment, the user image registered in the receiver and the product image are contracted or expanded at a correct scale factor even in environments where receivers with displays having different specifications coexist. A virtual fitting is thus performed on the display screen of the TV receiver.

[0195] The electronic shopping, in which the viewer is unable to touch the product directly, conveys a correct product image to the viewer, and allows the viewer to try the product on virtually with an easy operation. The viewer thus feels “realism”, and user confidence in the data broadcast TV shopping is promoted.

[0196] In the present embodiment, the data broadcasting shopping is performed in the data-broadcasting program. The virtual fitting is based on the face of the user in the above discussion. But the chest, or the entire body of the user may be used in the virtual fitting. The image of the fingers, the hand, and the arm of the user may be used to try an accessory or a wristwatch on. A building may be tried using the entire body or furniture as the user image. Furthermore, an indoor view may be used as the user image to select virtually furniture.

[0197] The present invention is not limited to the virtual fitting in the electronic shopping. The image data held for individual users with correct dimensions maintained and the broadcast data are synthesized based on the data broadcasting that is available to wide public.

[0198] The synthesis process is not limited to the still image. A content may be formed of broadcast moving images and the user image. To handle the still image, the data carousel transmission method is preferable. The moving image is preferably transmitted using a PES (PacketizedElementaryStream) method.

[0199] The process for contracting or expanding the image data of a product may be stored in the receiver rather than being contained in the transport stream transmitted in the data broadcasting. In this case, a data broadcasting apparatus simply transmits the scale of the image data as data for contraction and expansion, and is freed from the responsibility to transmit process information for expansion and contraction in accordance. The arrangement reduces the amount of transmission data and allows multimedia data to be additionally transmitted.

[0200] The present invention is not limited to the above-referenced embodiments, and finds use in any of the functions to be discussed in claims or functions performed by the above-referenced embodiments.

[0201] For example, the software configuration and the hardware configuration in the above-referenced embodiments are interchangeable with each other. The elements of the above-referenced embodiments may be combined as necessary. The construction specified in each of the claims or a part or whole of each of the above-referenced embodiments may form a single apparatus, or may be combined with another apparatus, or may be an element of an apparatus.

[0202] In accordance with the present invention, the image data obtained from data broadcasting is displayed in a form reflecting the real size of an object regardless of the specifications of the receiver. The user image and the image from the data broadcasting are synthesized and displayed at a correct scale factor. The user thus correctly judges whether or not to purchase the product in the data broadcasting shopping.

[0203] While the present invention has been described with reference to what are presently considered to be the preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.