METHOD FOR PROVIDING OPTIMIZED HOLOGRAMS BY EXTRACTING SYSTEM CHARACTERISTICS OF HOLOGRAPHIC DISPLAY

[0001] This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2024-0188061, filed on Dec. 17, 2024, in the Korean Intellectual Property Office, the disclosure of which is herein incorporated by reference in its entirety.

[0002] The disclosure relates to generating and providing holograms, and more particularly, to a method for generating and providing holograms optimized for system characteristics of a holographic display.

[0003] Artificial intelligence (AI) technology is utilized in the hologram area. Specifically, a technology that generates holograms by using a deep learning-based hologram generation network and displays the holograms on a holographic display has appeared.

[0004] However, in generating holograms, the hologram generation network may not consider system characteristics of the holographic display. To this end, even the holograms that are generated by the same hologram generation network may be displayed differently according to a holographic display.

[0005] This means that the holograms generated by the hologram generation network may not be optimized for the holographic display. Accordingly, there is a need for a solution for generating holograms optimized for a holographic display.

[0006] The disclosure has been developed in order to solve the above-described problems, and an object of the disclosure is to provide, as a solution for generating and providing holograms optimized for system characteristics of a holographic display, a method for extracting system characteristics of a holographic display embedded in a hologram generation network of a holographic display system, and generating and providing optimized holograms based on the extracted system characteristics.

[0007] To achieve the above-described object, a hologram providing method according to an embodiment of the disclosure may include: receiving, by a hologram providing server, a hologram generated by a holographic display system from a reference image; extracting, by the hologram providing server, system characteristics of a holographic display by using the received hologram and a hologram that is numerically reconstructed from the reference image; and providing, by the hologram providing server, a hologram that is generated by reflecting the extracted system characteristics to the holographic display system.

[0008] The received hologram may have the system characteristics of the holographic display reflected thereon. The holographic display system may generate the hologram from the reference image, by using a hologram generation network which is a neural network that learns the system characteristics of the holographic display of the holographic display system.

[0009] According to an embodiment of the disclosure, the hologram providing method may further include generating, by the hologram providing server, the reference image based on a system parameter of the holographic display, and transferring the reference image to the holographic display system.

[0010] According to an embodiment of the disclosure, the hologram providing method may further include receiving, by the hologram providing server, the system parameter of the holographic display from the holographic display system, and the system parameter may include hologram presentation wavelengths and the number of hologram representation wavelengths, the number of hologram pixels and a size of a hologram pixel, and a depth range.

[0011] The hologram that is numerically reconstructed from the reference image may be generated by the holographic display system.

[0012] The system characteristics may include optical aberrations, gamma characteristics, and color temperature of the holographic display.

[0013] The reference image may include a point light image, a point light array image, a grayscale-modulated monochrome image, and an image in which the aforementioned images are arranged at regular intervals by distance with respect to a depth domain that the holographic display intends to represent.

[0014] Providing the hologram may include generating the hologram by using a hologram generation network that is trained to receive an image content and extracted system characteristics and to generate holograms optimized for the system characteristics.

[0015] According to another embodiment of the disclosure, a hologram providing system may include: a processor configured to extract system characteristics of a holographic display by using a hologram that is generated by a holographic display system from a reference image and a hologram that is numerically reconstructed from the reference image; and a communication unit configured to provide a hologram that is generated by reflecting the extracted system characteristics to the holographic display system.

[0016] According to still another embodiment of the disclosure, a method for extracting system characteristic of a holographic display may include: receiving, by a hologram providing server, a system parameter of the holographic display from a holographic display system; generating, by the hologram providing server, a reference image based on the system parameter of the holographic display, and transferring the reference image to the holographic display system; receiving, by the hologram providing server, a hologram that is generated by the holographic display system from the reference image; and extracting, by the hologram providing server, the system characteristics of the holographic display by using the received hologram and a hologram that is numerically reconstructed from the reference image.

[0017] As described above, according to embodiments of the disclosure, system characteristics of the holographic display which are embedded in the hologram generation network of the holographic display system may be extracted, and, based on the system characteristics, holograms that are optimized for the system characteristics of the holographic display may be generated and provided, so that realism may be maximized.

[0018] Other aspects, advantages, and salient features of the invention will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the invention.

[0019] Before undertaking the DETAILED DESCRIPTION OF THE INVENTION below, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or,” is inclusive, meaning and/or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like. Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.

[0031] Hereinafter, the disclosure will be described in more detail with reference to the accompanying drawings.

[0032] Embodiments of the disclosure propose a method for generating and providing optimized holograms by extracting system characteristics of a holographic display. The disclosure relates to a technology that extracts system characteristics of a holographic display embedded in a hologram generation network of a holographic display system, and generates and provides holograms optimized for the system characteristics of the holographic display based on the extracted system characteristics.

[0033] FIG. 1 is a view illustrating a flow of a method for providing holograms optimized for a holographic display according to an embodiment of the disclosure.

[0034] As shown in FIG. 1, a holographic display system 100 may request streaming of optimized holograms, first, while transferring system parameters of its own holographic display 130 to a hologram streaming server 200 (S310).

[0035] The system parameters of the holographic display 130, which are transferred at step S310, are listed in the table of FIG. 2. As shown in FIG. 2, the system parameters of the holographic display 130 may include hologram representation wavelengths and the number thereof, the number of hologram pixels and pixel sizes, and may further include a depth range.

[0036] Reference is made back to FIG. 1. The hologram streaming server 200 which receives the request at step S310 may generate a reference image set based on the system parameters of the holographic display 130 received with the request, and may transfer the generated reference image set to the holographic display system 100 (S320). The reference image set may be comprised of a plurality of reference images generated to identify system characteristics of the holographic display 130. The reference images will be described in detail below.

[0037] The holographic display system 100 may generate holograms from the reference image set by using its own hologram generation network #1 110 (S330), and may generate holograms from the reference image set by using an ideal numerical hologram generator 120 (S340).

[0038] The hologram generation network #1 110 and the ideal numerical hologram generator 120 which are configured in the holographic display system 100 will be described in detail below with reference to FIG. 3.

[0039] The ideal numerical hologram generator 120 may generate holograms by using a traditional numerical reconstruction method rather than using AI. When holograms generated by the ideal numerical hologram generator 120 are reconstructed as hologram images through the holographic display 130, the holograms may be distorted and noises may be added due to system characteristics of the holographic display 130 (optical aberrations, gamma characteristics, color temperature, etc.).

[0040] In order to generate holograms that compensate for the system characteristics of the holographic display 130, the hologram generation network #1 110 may be trained to generate optimized holograms by using the hologram images that are generated by the ideal numerical hologram generator 120 and are reconstructed by the holographic display 130.

[0041] However, due to technical, environmental factors, it may be difficult to optimize the holograms generated by the hologram generation network #1 110 for the system characteristics of the holographic display 130 at a high level. In order to optimize the holograms for the system characteristics of the holographic display 130 at a higher level, the system characteristics of the holographic display 130 should be identified and reflected in generating holograms.

[0042] The system characteristics of the holographic display 130 may be embedded in the hologram generation network #1 110 through training, but it is practically impossible to identify where and in what form the system characteristics of the holographic display 130 are embedded in the hologram generation network #1 due to characteristics of a deep learning network.

[0043] Accordingly, in embodiments of the disclosure, reference images may be used. Specifically, the hologram generation network #1 110 having the system characteristics of the holographic display 130 embedded therein and the ideal numerical hologram generator 120 without the system characteristics are made to generate holograms, respectively, by using the reference images, and the system characteristics of the holographic display 130 may be extracted by comparing the holograms.

[0044] By comparing the holograms, it may be determined what change the hologram generation network #1 110 makes to generate holograms that compensate for the system characteristics of the holographic display 130. This is because the holograms generated by the hologram generation network #1 110 have the system characteristics of the holographic display 130 embedded therein, but the holograms generated by the ideal numerical hologram generator 120 do not have the system characteristics of the holographic display 130 embedded therein.

[0045] However, in order to accurately extract the system characteristics of the holographic display 130, it is necessary to use appropriate reference images. The reference images may include a point light (point) image, a point light array (point matrix) image, a grayscale-modulated monochrome image, etc., and may further include an image in which the aforementioned images are arranged at regular intervals by distance with respect to a depth domain that the holographic display 130 intends to represent.

[0046] A file structure of a reference image set containing reference images is illustrated in FIG. 4. As shown in FIG. 4, the reference image set file may be comprised of a file information region and a data region in which reference images are stored.

[0047] The file information region may contain information on the file and information related to the reference images stored in the data region. Specifically, as shown in FIG. 5, the file information region may contain information on a file size, a data region size, the number of reference images, a size of each reference image, the number of bits of a depth map image, and a depth value of each grayscale of the depth map image.

[0048] In the data region, each reference image may be divided into a color image and a depth map image, and may be stored in sequence as shown in FIG. 6.

[0049] Reference is made back to FIG. 1. When the holograms are generated through steps S330 and S340, the holographic display system 100 may constitute a transfer hologram set by unifying the holograms generated at step S330 and the holograms generated at step S340, and may transfer the transfer hologram set to the hologram streaming server 200 (S350).

[0050] The file structure of the transfer hologram set is illustrated in FIG. 7. As shown in FIG. 7, the transfer hologram set file may be comprised of a file information region and a data region in which transfer hologram datasets 1, 2, are stored. The transfer hologram dataset 1 may be the holograms that are generated by the hologram generation network #1 110 at step S330, and the transfer hologram dataset 2 may be the holograms that are generated by the ideal numerical hologram generator 120 at step S340.

[0051] The file information region may contain information on the file and information related to the transfer hologram datasets stored in the data region. Specifically, as shown in FIG. 8, the file information region may contain information on a file size, the number of datasets, a size of each dataset, the number of transfer hologram images in each dataset, a size of the transfer hologram image, and information related to labels of the datasets.

[0052] As shown in FIG. 9, in the data region, each transfer hologram in each transfer hologram dataset may be divided into a real part and an imaginary part, and may be stored in sequence. In FIG. 9, the reference images based on which the transfer holograms are generated may match each other to indicate that the reference images are not stored in the data region but match and are stored in the same order as the order of the reference images based on which the holograms are generated.

[0053] Reference is made back to FIG. 1. A processor 210 of the hologram streaming server 200 which receives the transfer hologram set at step S350 may compare the two types of transfer holograms constituting the transfer hologram set, and may extract the system characteristics of the holographic display 130 (S360). The extracted system characteristics may include optical aberrations, gamma characteristics, color temperature, etc. of the holographic display 130

[0054] FIG. 10 illustrates an overall process of extracting the system characteristics of the holographic display 130. FIG. 10 illustrates the processes from step S330 to step S360.

[0055] As shown in FIG. 10, the processor 210 may reconstruct a hologram image through numerical reconstruction on the transfer holograms generated by the hologram generation network #1 110. The reconstructed hologram image does not reflect the system characteristics of the holographic display 130 and hence is different from the reference image. Specifically, the transfer hologram generated by the hologram generation network #1 110 reflects the system characteristics, but the reconstruction is ideal numerical reconstruction, and therefore, the reconstructed hologram image is a hologram image that inversely compensates for the system characteristics, rather than the reference image.

[0056] The processor 210 may reconstruct a hologram image through numerical reconstruction on the transfer holograms generated by the ideal numerical hologram generator 120. The reconstructed hologram image does not reflect the system characteristics of the holographic display 130 and may be the same as the reference image.

[0057] When a point light source or an array of point light sources is used as a reference image, the processor 210 may compare forms of a point-spread-function between the two hologram images, and may calculate optical aberrations of the holographic display 130. In particular, fitting in the form of Zernike-polynomial may cause a linear combination of major optical aberrations such as astigmatism, coma, spherical aberrations, and accordingly, it is possible for the processor 210 to numerically analyze respective optical aberration components based on the linear combination.

[0058] When monochrome images are used as a reference image set and reference images that use monochrome images for various grayscale values are used, the processor 210 may extract gamma characteristics and color temperature by pixel of the holographic display 130 by comparing the two hologram images.

[0059] Reference is made back to FIG. 1. When the system characteristics are extracted at step S360, a hologram generation network #2 of the hologram streaming server 200 may generate optimized holograms reflecting the system characteristics of the holographic display 130, and may stream the optimized holograms to the holographic display system 100 (S370).

[0060] The holograms generated at step S370 may be holograms to be streamed to the holographic display system 100, and may be generated from an image content to be serviced, rather than the reference image set.

[0061] The hologram generation network #2 220 may be a deep learning network that is trained to receive an image content and system characteristics of the holographic display 130 and to generate holograms optimized for the system characteristics. That is, the hologram generation network #2 220 may differ from the hologram generation network #1 110 of the holographic display system 100 in terms of the hologram generation method.

[0062] Up to now, the method for providing the optimized holograms by extracting the system characteristics of the holographic display has been described in detail with reference to preferred embodiments.

[0063] In the above-described embodiments, the system characteristics of the holographic display which are embedded in the hologram generation network of the holographic display system may be extracted, and, based on the system characteristics, holograms optimized for the system characteristics of the holographic display may be generated and provided, so that realism may be maximized.

[0064] In the above-described embodiments, the ideal numerical hologram generator 120 of the holographic display system 100 generates transfer holograms from the reference image set. However, this is merely an example and is changeable. For example, an ideal numerical hologram generator may be provided in the hologram streaming server 200 to allow the hologram streaming server 200 to generate transfer holograms from the reference image set. In this case, the hologram streaming server 200 does not have to receive the transfer holograms generated by the ideal numerical hologram generator 120.

[0065] Although not shown, the hologram streaming server 200 may be provided with a communication means to receive the request, the system parameters, and the transfer hologram sets from the holographic display system 100, and to stream the reference image set and the optimized hologram to the hologram streaming server 200.

[0066] The technical concept of the disclosure may be applied to a computer-readable recording medium which records a computer program for performing the functions of the apparatus and the method according to the present embodiments. In addition, the technical idea according to various embodiments of the disclosure may be implemented in the form of a computer readable code recorded on the computer-readable recording medium. The computer-readable recording medium may be any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. A computer readable code or program that is stored in the computer readable recording medium may be transmitted via a network connected between computers.

[0067] In addition, while preferred embodiments of the present disclosure have been illustrated and described, the present disclosure is not limited to the above-described specific embodiments. Various changes can be made by a person skilled in the at without departing from the scope of the present disclosure claimed in claims, and also, changed embodiments should not be understood as being separate from the technical idea or prospect of the present disclosure.