Method of and Installation for the Production of Orthostereoscopic Images

For the purpose of illustrating the invention, there is shown in the drawings a form which is presently preferred; it being understood, however, that this invention is not limited to the precise arrangements and instrumentalities shown.

FIG. 1 is a diagram explaining an optical photographic system showing the effect of pseudo stereoscopy;

FIG. 2 is an optical diagram of the invention;

FIGS. 3a-3c are a set of images showing the general method according to the invention;

FIGS. 4a-4b a more detailed set of images showing the method according to the invention;

FIG. 5 is a diagram of a video installation for carrying out the invention;

FIG. 6 is a diagram of another video installation for carrying out the invention;

FIG. 7 is a diagram of another video installation for carrying out the invention; and

FIG. 8a-c shows the invention as embodied in a CCD camera.

FIG. 1 is a plan view of an optical system for taking stereoscopic images. This optical system is composed of a lens 1 of large dimensions behind which there is a lenticular surface 2, that is to say a surface consisting of an assembly of lenses bounded by cylindrical surface which are all perpendicular to the plane of the drawing and of which only one (2i) is shown. As used herein, the term "cylindrical" in reference to the elementary lenses is understood to means any lens whose imaging surface is a portion of a cylinder. Similarly, as used herein, a "spherical" lens is one whose imaging surface is a portion of a sphere.

To understand stereoscopic photography and the problem of orthostereoscopy, FIG. 2 has been sub-divided into a left-hand part corresponding to the photograph and a right-hand part corresponding to observation of the image.

Referring first to FIG. 1, for taking the photograph, an object point A is situated in front of the lens 1; at the level of the lens, the right eye OD is shown diagrammatically and the left eye OG is shown looking at the point A and thus defining a certain point of view. The system of stereoscopic photography with a large lens 1 and a lenticular system 2, "sees" the object point A as if both eyes OD and OG were placed at the level of the lens 1. Indeed, a small part of the lens 1 situated at the position of the eyes OD and OG respectively receives the light beams emitted by the object point A and destined respectively for the right eye and the left eye (seen in plan in FIG. 1).

If we trace the light rays which are thus destined for the right eye and the left eye and pass through the optical system formed by the lens 1 and the lenticular system 2, we arrive at the image point AOD which is the image point intended for the right eye and the point AOG which is the image point intended for the left eye.

The ray of light corresponding to the right eye is represented by the solid line while the ray of light intended for the left is represented by a broken line.

The image points AOG and AOD can be formed, for instance, on a photosensitive surface 3 placed behind the lens system 2 or may appear on a ground glass screen.

In the simplest case of a single observer who stays in position (which corresponds to a stereoscopic point of view), the right eye and the left eye occupy a precise position which remains fixed. On a photosensitive surface, this would correspond to placing behind the objective lens 1 two images in fixed position. In this case, the image points AOG and AOD would result from each object point A. When restored, only a single observation point for the stereoscopic image would be produced.

However, when the photograph is taken, there is no need to limit the use of the objective lens to one stereoscopic point of view, realized, for example, by two diagphragms situated at the location of the "left eye" and of the "right eye", but instead there is provided a larger diaphragm providing a continuity of image points by the "right eye" and by the "left eye" behind each lens in the lens system. This diagram corresponds to a horizontal slit. All the image points formed by the objective lens and a single lens in the system is referred to as the "elementary image IE".

In the case of a cylindrical lens systems with a vertical axis, each part of the lens forms an image point from the object point A. Thus a number of image points are obtained which include the points AOG and AOD. This remark only applies in the horizontal direction and not in the vertical direction since the system comprises vertical cylindrical lenses.

In the case of a system of spherical lenses distributed in a horizontal direction and the vertical direction, the diaphragm is useless.

It must be remarked that the overall image formed by the objective 1 is reversed, an arrangement which is not totally modified by the lens system which only transforms the image at elementary level.

Under these conditions, in order to observe the image, it is necessary to be positioned where the left eye and the right eye occupy the position shown in FIG. 1. To have one's eyes in the proper positions, a viewer would have to stand upside-down behind the screen.

Upon observation of this overall image with the interposition of a lens system, the left eye will see the image poing AOD intended for the right eye and the right eye will see the image point AOG intended for the left eye. The result is a pseudo stereoscopic effect.

FIG. 2 shows the method and the installation according to the invention by which it is possible to restore an image to provide an orthostereoscopic view, that is to say a three-dimensional view in which the right eye and the left eye effectively see the parts of the image which are intended for them.

According to the invention, the image is broken down into elementary images (IE), of which the image points AOG, AOD are respectively received by zones A1, A2 of an optoelectronic sensor of the CCD type which analyzes the image, or a tube corrected for geometrical aberrations; the analogue signals which are thus picked up and which correspond to the image points are nubmered so that they can be processed by an electronic circuit 5 which reverses the elementary images while retaining their order in the overall image.

The reversal described may likewise be carried out in an analogue way.

Whatever type of processing is involved, this reversal is carried out electronically. It is shown diagrammatically in FIG. 2 as the unit 5. The signals thus obtained are transmitted to a display means 6 which again displays the elementary images corresponding to each lens in the system to allow them to be used by the right eye OD and the left eye OG through a lens system 7 which is optically identical to the lens system 2 which was used for taking the photograph. Thus, it is found that the right eye OD sees the image point from the zone A2 and the left eye OG sees the image point from the zone A1 as if the right eye and the left eye occupied the position shown in the photographic part. Therefore, one is viewing an orthostereoscopic image.

The method of the present invention produces three-dimensional or relief images of the orthostereoscopic type according to the principle whereby a photograph is taken or viewed with the help of a lenticular optical surface comprising different elementary lenses.

The photograph is taken with a single objective lens according to at least two different points of view corresponding to the two eyes of an observer; the series of elementaty lenses forms an image of the object for each direction of view. This image is composed of elementary images formmed respectively by the optical assembly comprising the lenticular optical surface.

In order to cause the image that is formed on the photosensitive surface 3 to become coherent so that it may be viewed on a monitor having a compatible lenticular surface, the electronic signal created by the image on the photosensitive surface must be processed electronically before it is passed to the monitor. The result of this electronic processing is to reverse individual elementary images while retaining the order of elementary images within the composite image. In a preferred embodiment of the present invention, the elementary lenses in the lenticular grating are vertically arranged relative to the photosensitive surface or the monitor, while the scanning direction of thepicture elements (pixels) of the television equipment is horizontal. Thus, the electronic processing may be effected by rearranging the order of pixels in each of the horizontal scan lines.

As an example of how the alteration of the order of pixels within each scan causes a suitable alteration in the image, consider a horizontal line through a video screen which is scanned from left to right in the course of one pass of the video scan. In a conventional television camera, the pixels along this horizontal line will be input and output in the same order. In a hypothetical example having twelve pixels across the screen, the input and output will be:

The photosensitive video screen in one embodiment of the present invention is divided among a number of cylindrical lenses. Because the scan line is perpendicular to the lens' axes, each horizontal scan line will be subdivided into as many divisions as there are cylindrical lenses. Each division of the scan will correspond to one elementary image on the screen. For the hypothetical example, consider the case of twelve pixels in a horizontal line, divided by three vertical cylindrical lenses:

In the above example, the double vertical line .parallel. represents the borders between three cylindrical lenses. In the physical world, the three cylindrical lenses will each reverse the incoming image as it forms an elementary image on the photosensitive screen. It is the function of the present invention to revese the order of pixels within each elementary image to form the corrected image. So, taking the pixels within each cylindrical lens and reversing them (but keeping the order of cylindrical lenses as they are scanned), the system of the present invention will output the following:

The present invention divides the pixels across a horizontal line into batches and reverses each batch individually. In the case of twelve pixels divided among four cylindrical lenses, each batch will have three pixels associated with it. When processed by the present invention, the output would be:

Here, the line of twelve pixels is divided into four batches of three, and each of the four batches is reversed individually. By performing this dividing-and-reversing process to each scan line in a television raster, the necessary reversal of each elementary image along its longitudinal axis is accomplished.

FIGS. 3 and 4 show an example of the method of the present invention as applied to an image of a three-dimensional object. FIGS. 3a and 3b are two views of the same automobile body, observed from slightly different angles. Generally FIG. 3a is the image of the automobile body seen with the left eye, and FIG. 3b shows the automobile body as viewed simultaneously through the right eye. The difference in view points in FIGS. 3a and 3b is exaggerated somewhat for clarity. When each eye is viewing one of these images, the brain is capable of reconstructing the spatial volume of the object. FIG. 3c shows four superimposed views of the object from varying viewpoints. These four viewpoints are only four of an infinite number of viewpoints from which the object can be observed. Each elementary lens accepts (when the object is being photographed) and emits (when the image is being viewed) a continuous series of images through the angle of field of each elementary lens, i.e., the angle which determines an angular zone. Although the image associated with each elementary lens is continuous, only two points (each corresponding to one eye) can be observed at any one time by a two-eyed observer. Any group of two points of view may be considered as being a left and a right point of view.

FIGS. 4a-b show a representation of the object in FIG. 3 as processed under the method of the present invention. FIG. 4a is an image formed by a lenticular grating such that each elementary lens is four pixels wide. (The image is simlilar to that of FIG. 3, except that the angle of the field between the two viewpoints is more realistically narrow than that of FIGS. 3a and 3b. It should also be emphasized that the processing of the image by the lenticular grating is fundamentally different from the image of FIG. 3c, which is merely a superimposition of four single viewpoints.)

Vertical lines 50 show the borders between the elementary lenses in the lenticular grating as superimposed on the image. It can be seen that within each elementary image there is room for four pixels across each elementary image. Each set of pixels in one position within each elementary image (for example on the far right within each elementary image) corresponds to an image of the object through one viewpoint. With a four-pixel-per-elementary-image resolution, four viewpoints out of an infinite family are registered on the image surface. Any two of these viewpoints can be observed to create the three-dimensional image when one set of pixels is viewed by one eye and another is viewed by the other eye.

In FIG. 4a it can be seen that, in the portion of the image relating to a line passing from the lower left to the upper right (as in the far edge of the windshield), the groups of four pixels within each elementary image are each sloped from the lower left to the upper right. However, if this image in FIG. 4a is viewed through a lenticular grating, each elementary lens in the grating will reverse the order of pixels from left to right so that the line viewed through each of the elementary lenses will be from upper left to lower right, even though the line over a series of elementary images will be from lower left to upper right. The result will be a garbled image because each elementary image will be backwards.

The present invention electronically reverses the order of pixels within each elementary image before it is viewed through a lenticular grating, so that the reversing processes of the electronic processing and the lenticular grating will cancel out. FIG. 4b shows the image of the automobile body where the sequence of pixels within each elementary image has been reversed. The revesal of pixels is clear from a comparison of FIGS. 4a and 4b. Viewed without a lenticular grating, the image of 4b would itself appear to be garbled; the lower left to upper right line of the far windshield is shown as an upper left to lower right line within each elementary image. However, when the image of FIG. 4b is viewed through a lenticular grating, the elementary lenses of the grating reverse the order of the pixels again, placing them back in their correct order. Thus, if one were to place a lenticular grating of the correct dimensions over FIG. 4c and so arrange his eyes that one family of pixels is aligned with one eye and another family of pixels is aligned with another eye, a three-dimensional effect will be created. Viewing FIG. 4a through a lenticular grating, however, will yield an incoherent image.

In practical applications of lenticular grating systems, the lenticular grating is commonly combined with objective lenses and other optical devices. As a result of observing the reconstituted image through another lenticular grating, the final image may be reversed as a whole with respect to the original image; that is the reconstituted image may be coherent but reversed as a whole from left to right. However, this reversal of the entire picture can be cured easily through electronic means known in the art.

In practical applications of the present invention, the width of the cylindrical lenses on the lenticular grating must be chosen to take into account resolution and providing a sufficient difference between the images going to each of the viewer's eyes. The cylindrical lenses must be of a width sufficient to expose enough pixels that can be sensibly reversed. For example, in the hypothetical twelve-pixel screen above, to subdivide the screen into twelve batches would make no sense, because batches of one pixel each cannot be reversed. At the same time, if the elementary images are too wide, the borders between the images behind each cylindrical lens will be conspicuous and picture quality will be unsatisfactory.

FIG. 5 is a diagram showing a video installation for taking orthostereoscopic images.

This installation consists of a photographic camera 31 which processes the image 30. The camera 31 is followed by a digitalizing circuit 32 which converts the analogue signal supplied by the camera 31 to a digital signal. This digital signal is stored in the memory 33 to be fed to a computer 34 having various peripherals such as a disc scanner 35, a monitoring screen 36, and a keyboard 37. The signal processed by the computer 34 is transmitted to a memory 38 and through a digital/analog converter 39, the signal is converted to an analog signal transmitted to a user circuit 40 such as an emitter, a recording means such as a tape recorder, etc.

The signal thus processed may likewise be transmitted to a television receiver 41 in order to be viewed on a cathode ray screen 42 in accordance with three-dimensional television techniques, that is to say through an interposed lenticular system.

The system described hereinabove is preferably embodied in electronic circuits which process the data in real time and therefore more rapidly. Such circuits are far simpler and therefore far less expensive. The electronic system consists preferably of a camera 31, a digitalizer 32, a turn-over card, a digital/analog converter, possibly a system for the transmission or storage of images and a reception and display means, still comprising a lenticular system.

Although the description given hereinabove employs a digital processing arrangement, it is likewise possible to perform similar processing on the analogue signal without passing through the digitalization stage.

The description of the invention given hereinabove in principle employs a single lens 1 for taking the photograph (FIG. 1). This lens may likewise be replaced by an equivalent means in order to modify (increase) the stereoscopic base. For this, it is necessary to replace the single photographic assembly by a plurality of correlated identical assemblies, that is to assemblies of which the diaphragms in the form of a horizontal slit have each time only a width corresponding to a fraction of the total aperture so that the superimposition of images of one and the same scene, formed behind the lenticular systems, yields a single image corresponding to that which would have been obtained with the single system with the exception of the different stereoscopic base.

Furthermore, the number of systems used simultaneously must not exceed the number of selected viewing points.

The above example is directed to an embodiment of the invention having a grating of cylindrical lenses, arranged vertically with respect to the eyes of the observer. However, a lenticular grating of spherical lenses could be used to create the three-dimensional image without the necessity of having the observer's eyes specifically oriented in one direction relative to the viewing screen. In the embodiment having spherical lenses, the rearrangement of pixels must be accomplished not only within the scan line, but among a number of neighboring scan lines so that the circular elementary images on the imaging surface will be reversed through their center point.

FIG. 6 is a block diagram showing another embodiment of the electronic image-processing system of the present invention. The apparatus shown in FIG. 6 performs the alteration of the order of pixels in each scan line, as described above. The input, a sequence of signals relating to the image along each horizontal scan line as it is scanned in a raster, is fed into separator 70. Separator 70 accepts the sequence of signals and feeds the signal to an analog/digital converter 72 and a signal generator 74 in such a way that signal generator 74 will be able to keep track of the location of each pixel on its appropriate scan line, so that the pixels will "line up" correctly as scanned lines are pieced together to form an image on a screen. Simultaneously, analog/digital converter 72 accepts the actual signal (in the simple case of a black-and-white television, a signal representative of the light intensity of a particular pixel in an image). Analog/digital converter 72 and signal generator 74 may be coordinated, through means known in the art, by a phase-locked loop 76.

A sequence of signals passes through analog/digital converter 72 and signal generator 74. Signal generator 74, keeping track of the correct location on a scan line of a pixel being processed at a given instant, relays a control signal to address counters 78 and 80 and synchronization generator 82. Address counter 78 in turn activates an address permutator 84, which has stored within it the appropriate sequence of pixels in each scan line that will correspond to reversing the elementary images on the screen, and, if necessary, reversal of the entire corrected image. The exact sequence will of course vary with the number of cylindrical lenses used on the screen, and may also take into account factors such as distortion of the image within each cylindrical lens. The appropriate sequence of pixels may be incorporated into the address permutator 84 by any number of known means, such as by microprocessor or read-only memory.

The memory in this system is divided into two parts: dynamic memory 86 and permanent memory 88. The actual permutation of the pixels is made by the hardware connections between memories 86 and 88. Each location in dynamic memory 86 is associated with a permutated address. These data are transmitted for reading to the permanent memory 88. The reading is made dependent on the signals transmitted by signal generator 74 and the rad-only address counters 80. The memory of the system is divided into a dynamic part 86 and a permanent part 88 to allow the advantage of allowing fine control over the image quality. For example, in the case of a system using four pixels across each elementary lens, an operator can use the permanent memory 88 to dissociate the four families of pixels, to view each perspective individually and perform optical adjustments on the system. This feature has been found to be very significant.

The permutated signals are then read out of permanent memory 88 into digital-analog convertor 90. The converter 90 outputs the pixels in their new sequence to a monitor or a recording device, for simultaneous or later viewing through a viewing screen having an arrangement of lenses associated therewith compatible with that of the original photosensitive screen.

The elements of the system are preferably arranged so that the re-arrangement of pixels in each scan line takes no longer than the time required for one scan of the scan line, so that what appears on the reconstituted screen is only one scan behind what is input into the system.

FIG. 7 shows a simplified system for carrying out the method of the invention, which is similar to the system of FIG. 6, except that the memory portion, shown as block 87 in FIG. 7, is not divided into dynamic and permanent portions. In this embodiment the digital signals from analog/digital converter 72 are permuted as they are read into memory 87 simultaneously with the permuted addresses from address permutator 84 and address counter 78. The permuted pixels in memory 87 are then read out through a FIFO register 94. This embodiment has the advantage of simplicity and low cost, but it lacks the importantability to allow an operator to dissociate specific view points from the composite image.

In addition to the digital process described by the systems of FIGS. 6 and 7, the inversion of each elementary image may be accomplished by hard-wired permutation of the photocells comprising the picture elements in a CCD camera. In the photosensitive surface of a typical CCD camera, as shown in FIG. 8a, an array of photocells 101-105, etc., are arranged in rows and columns and exposed to an image. The array of photocells is disposed adjacent to a lenticular grating just like the photosensitive surface in the previous embodiments. Each photocell 101-105 etc. represents a pixel in one position of the image on the photosensitive screen. In the array of FIG. 8a, photocells 101, 102, and 103 cooperate with registers such as 106, for each column of pixels, and each of the registers 106 cooperate with a main register 108.

FIG. 8b shows a similar CCD camera as in FIG. 8a, except that the photocells are permutated. The relationship between the photocells, the vertical registers 106, and the main register 108 are the same as above, except that the connection between the individual photocells and the vertical register 106 are slightly altered. In the example of FIG. 8b, the connecting wires of photocells 101 and 103 are crossed, so that the image processing system "sees" the portion of the image associated with photocell 101 as if it were going to photocell 103, and vice-versa. It is clear that such a crossing of wires will have the effect of inverting a horizontal line of the image through the axis marked by dotted line 110. Thus, this example shows a hard-wired CCD photosensitive surface which is used with a lenticular grating of horizontally-arranged elementary lenses, each elementary lens being three pixels in width.

FIG. 8c shows another type of inversion of an elementary image, except that the wiring of the vertical register 106 is such that the induced inversion of the elementary image is for a vertical elementary image of three pixels in width, inverted around line 112. This inversion is accomplished by hard-wiring vertical registers 106 and 107 in such a way that they are "seen" by the main register 108 in each other's positions. The CCD surface of FIG. 8c corresponds to a lenticular grating having elementary lenses corresponding to every three vertical columns of pixels.

It should be noted that for processing reasons (turning over of the elementary images), the number of viewing points is predetermined so that there is associated with each viewing point an image point for each lens in the system. The image point is necessarily as large as the pixel of the part of the system which has the lowest resolution (camera or digitalizer or screen). There is no interest in having the image point equal to two "eldim" pixels (to use French terminology); in this case, it would be appropriate to double the number of lenses in the system to cover a larger image or the same image with a better resolution so that it is doubled.

The electronic means of the present invention provides great versatility in manipulating elementary images, which is not possible with a purely optical system. The electronic means of the present invention does not require an expensive extra lenticular grating disposed between the first lenticular grating and the photosensitive surface. The electronic means are realizable through known electronic components, and will not cause the image to darken, as would the second lenticular grating of an optical system. Further, the manipulation of elementary images through electronic means may also include computer enhancement to improve the image. It is conceivable to create a computer program to simulate the depth effects from a lenticular grating directly on the video surface, so as to create, for example, three-dimensional cartoons. Electronic equipment is more readily miniaturizable for use in practical camera equipment, whereas additional optical components contribute greatly to excess weight and fragility of equipment.

When the present invention is used in conjunction with black-and-white television equipment, the signals entering the system are simply representative of a light intensity corresponding to an appropriate shade of gray for a particular pixel in a televised image. Each pixel has only one variable associated with it, the variable of light intensity. With color equipment, however, three sets of variables must be taken into consideration: one for each of the primary colors that make up the pixels in a color television signal. A system according to the present invention must assure that the red, blue and green color dots are not only transposed appropriately on the video screen, but also that the appropriately-colored pixel signal is directed to the appropriate gun of the color television monitor. Also, color refraction within each of the elementary lenses may also have to be taken into consideration by the system.

There are currently in popular use two types of color television screen, one in which the colored picture elements forming the picture are dot-shaped, and another in which the colored areas are in the form of rectangular stripes. For equipment of the present invention, having vertically-disposed elementary lenses, the striped screen is preferable. In either case, an important consideration is the fact that the alternating pattern of colored areas, each area producing one of three colors but together forming the color images, will have various angles of incidence relative to the associated elementary lens. In addition to the features of the image being altered by the lenticular grating, the primary colors of the image will be altered in more or less incoherently, as various color dots will be defracted in various amounts, depending on the position of individual dots relative to each lens.

One proposed solution to this garbling of color is to use a screen having vertical stripes, and laying the screen on its side, so that the colored stripes are perpendicular to the elementary lenses. As the colors are perpendicular to the lenses, the same color will appear throughout the angle of incidence of each lens, and the colors will remain coherent.

When using color equipment, it is preferable to have the ability to manipulate not only individual elementary images, but to control the appearance of the picture as a whole through electronic means, so as to correct the color distortion (Moire effect) and general picture clarity.

In conclusion, the invention makes it possible to create orthostereoscopic images either on photographic carriers or on video carriers (magnetic tapes, digital discs or digital/analogue discs). It should be noted that the invention is likewise applicable to the production of synthesized orthostereoscopic images.

The present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof and, accordingly, reference should be made to the appended claims, rather than to the foregoing specification, as indicating the scope of the invention.