Synchronization frame generator for multichannel imaging system

[0002] This invention relates to the field of signal synchronization methods and panoramic television devices.

[0003] In audio-visual recording systems, it is often necessary to synchronize several streams of audio or visual images. Multiple cameras are often used for recording a scene simultaneously from several points of view, while at the same time one or more audio devices record the sound.

[0004] All of these streams of information must be synchronized to each other in the playback process. For multiple-camera systems, one or more of three synchronization methods are commonly used:

[0005] 1. A visual cue simultaneously recorded on several media. One example of this is a clapboard, which gives simultaneous marking of a visual cue (the camera seeing the clapper coming together) and an audio cue (the sound of the clap being recorded on an audio track). This would work for multiple-camera shoots only if all of the cameras could clearly and simultaneously see the clapboard coming together. This same principle is embodied in a more convenient form for multiple-camera shoots by a device which flashes a bright light (for the camera) while emitting a sharp sound (for the audio). If any camera cannot clearly record the visual cue, then it cannot be synchronized with the others.

[0006] 2. A separate synchronization signal is recorded simultaneously on all media. One example of this is SMPTE time code, which can be recorded on multiple film, video or audio devices, and which provides a common time reference for the numbering of individual frames. By making use of this information, all of the frames or audio recorded at a given time can be aligned to one another. This requires separate devices for reading and interpreting the synchronization signal information.

[0007] 3. Physically marking visual cues on the media itself after recording is completed. For example, multiple strips of motion picture film are often used in the process of editing and when producing multi-layer effects. Audio recordings are also present in the form of mag track recordings with the same perforations as film. In a multi-gang synchronizer on an editing bench, these strips of film can be placed next to each other on linked spoked wheels, and markings made, either with grease pencil, permanent marker or by punching holes in the middle of the frame, to mark key synchronization frames common to all. By lining up these marked frames, synchronization can be achieved. Physical marking requires extra steps in the postproduction process, and is impractical for electronic media such as video where changes in the video image typically require a recopying process and the risk of loss of visual image quality.

[0008] Other approaches in the patent literature have relied upon variations in these approaches. For example, U.S. Pat. No. 4363055 “Control System for Audio Visual Projector” describes a separate control signal representing frame numbers. U.S. Pat. No. 5481307 “Method and Apparatus for Compression and Decompression Sequences of Digital Video Images Using Synchronization Frames” involves arbitrary marking of selected frames after recording for special handling in a compression process. A similar process is found in Motion Picture Experts Group Level 2 (MPEG-2) compression, where Intra (I) frames are identified and used for comparison to measure the changes in a moving video image.

[0009] The synchronization of multiple cameras usually involves making sure that all share the same frame rate. Otherwise, they will drift apart over time. Another step is to make sure that all frames begin and end together. Otherwise, if one frame ends during the exposure time for another, there is the possibility for a sync marker at a given moment in time extending over one frame on one track, and two frames on another track. For enforcing a common frame rate for film-based systems, a synchronization pulse controlling the camera motors is often used. For setting a frame rate for multiple video cameras, a single “genlock” signal is fed into all of the cameras, such as a “video black” signal.

[0010] The present invention comprises an improvement in marking synchronization points as part of a recording and playback apparatus for multi-channel imaging systems. This improvement generates an easily visible synchronization frame simultaneously for all channels, and can be applied to either video or motion-picture film-based systems. It does not rely upon any external objects or markers being simultaneously present in the field of view of any camera in the multi-camera system. It also does not rely upon a separate synchonization signal being recorded simultaneously along with the video images, or upon any marking or post-processing of the recorded image or the image media.

[0011] FIG. 1 is a block diagram of the signal flow of a multi-channel video camera system including a simultaneous synchronization frame generator.

[0012] FIG. 2 is a schematic view of multiple strips of frames, showing the synchronization frames simultaneously generated at the same moment in time for each.

[0013] FIG. 3 is a schematic view of multiple strips of frames, showing the synchronization frames simultaneously generated at the same moment in time for each with the addition of a stream-identification frame following the synchronization frame.

[0014] FIG. 4 is a block diagram of a reader circuit for automatically detecting and reacting to sync frames as part of a postproduction logging and editing process.

[0015] 2 Video Timing signal generator

[0016] 4 Video Frame generator

[0017] 6 Video timing and frame generation signal

[0018] 8 Example of video signal stream

[0019] 10-16 Video Cameras A-D

[0020] 18 Internal video signal with image information included

[0021] 20 Sync Frame Generator

[0022] 22 Frame frame interface

[0023] 24 Sync Frame Trigger

[0024] 26 Substitute signal generator

[0025] 28 Action of operator

[0026] 30 Sync Frame as recorded in Track A

[0027] 32 Sync Frame as recorded in Track B

[0028] 34 Sync Frame as recorded in Track C

[0029] 36 Sync Frame as recorded in Track D

[0030] 38 Video Output

[0031] 40 Sequence of frames for Track A

[0032] 42 Sequence of frames for Track B

[0033] 44 Sequence of frames for Track C

[0034] 46 Sequence of frames for Track D

[0035] 48 Sample Sync Frame

[0036] 50 Extra Information frame—Track A

[0037] 52 Extra Information frame—Track D

[0038] 54 Example of Video Frame in Track

[0039] 56 Frames being reproduced from a playback device

[0040] 58 Direction of travel

[0041] 60 Signal reader

[0042] 62 Signal from reader

[0043] 64 Frame Start detect

[0044] 66 Detector for sync frame signal level

[0045] 68 Frame End detect

[0046] 70 Flag for detection of sync frame

[0047] A synchronization frame circuit will be described particularly suited for the generation of an easily readable mark on all channels of a multi-channel video system. The same principle could also be applied to multi-channel film cameras or other multi-image recording systems. The use of a sync frame generator is particularly suitable for highly-integrated compound video camera systems used for panoramic imaging, such as the Dodeca™ 1000 Round About™ Immersive Video Camera System.

[0048] The principle of the present invention relies upon the substitution of a blank frame for the usual ever-changing image signal produced by an image generator such as a motion picture film or video camera. This blank frame is generated simultaneously in all of the channels of the system, regardless of what the camera image of an individual channel might be at that moment.

[0049] FIG.1 shows a block diagram of the signal flow of a multi-channel video camera system including a simultaneous synchronization frame generator. A video timing generator 2 includes a timing circuit for generation of frame divisions within the video stream, typically in the form of vertical sync pulses, in addition to any necessary color burst pulses, horizontal sync signals and other components of a typical video signal, according in National Television Standards Committee (NTSC) video, or another standard such as PAL or SECAM. The video frame generator component 4 is here shown simultaneously distributed along line 6 into four channels A-D of regular frame divisions, such as at 8. For purposes of clarity here, the genlock frame generator timeline, the video image generator signal, and the position of the sync frame are shown spread out horizontally, even though they are essentially contemporaneous. An unlimited number of channels can be included in the described system.

[0050] This “genlock” signal is thereby fed into multiple video image generators, such as the video cameras A-D shown here as 10, 12, 14, and 16, respectively. Here the frame and other timing information from the genlock signal serves as the basis for outputting the image information coming from a sensor such as a Charge-Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS) imager. In place of the usual visible-light video images, other sensors or electronics are sometimes used to output different image information as part of a video signal, such as infrared, ultraviolet, or amplified night vision images. The video signal information goes out on a line such as at 18.

[0051] The synchronization frame generator 20 is controlled by a relay 22, which is activated by an interactive trigger device 24 such as a positive-action pushbutton, which an operator can trigger 26 when a sync point is to be recorded. When the button is pushed, a substitute signal generator 28 sends a fixed-level signal to be output as part of the video stream instead of the image information from the camera on line 18. The substituted sync frame being generated is identical for all channels. The sync frame as output on tracks A-D is shown at 30, 32, 34, and 36, respectively. At 38 is an example of regular nonsync frame video output.

[0052] FIG. 2 is a schematic view of multiple strips of frames, showing the synchronization frames simultaneously generated at the same moment in time for each. A sequence of frames for Track A is at 40, a sequence of frames for Track B is at 42, a sequence of frames for Track C is at 44, and a sequence of frames for Track D is at 46, with all frames shown numbered consecutively according to a given moment in time. A sample of a substituted sync frame is at 48. These frames can be either film frames or video frames.

[0053] There are several options for the appearance of this sync frame. In a film-based system, the sync frame can be black, as caused by an override circuit preventing the shutter from opening for that frame on all cameras synchronized to the system by the usual means of a frame rate control signal for the camera motor. For mechanical shutters linked to the camera motor, an additional sync-frame shutter must be added to perform this function. Marking the otherwise unexposed sync frame can be done, either within the gate at the same time, or at a later point in time within the camera, with a small light such as from a Light-Emitting Diode (LED), either shining alone or through a mask to create a pattern. Care must of course be taken to prevent this light from exposing any part of any other film in the camera. An easily visible and distinctive sync frame pattern on the developed film will simplify the task of recognizing a sync frame as opposed to a normally unexposed frame.

[0054] In a video-based system, the sync frame should be as much of a visible contrast to the usual form of the video image signal. In that way, it will be visible to someone reviewing the recording even when it is going by at the full speed of 60 fields/30 frames per second (in NTSC) or in a fast-forward or fast-reverse review mode. If only one field of an interlaced video sequence is marked, then it may not be visible to an operator during normal playback.

[0055] A video sync frame should be solid in appearance (as a contrast to the multifareous detail in a typical picture) and be either black, a color, grey, or a form of white. For an NTSC black frame, the signal level (in the frame area usually occupied by the video image information) can be either the usual “video black” level of 7.5 IRE units, or preferably a “super black” of 0 IRE. A “super black” sync frame, because it has a different signal from that normally produced by a video camera, is more apt to be recognized by a simple signal detection device for detecting sync frames during review of any recordings made.

[0056] If the sync frame is to be a color, it should be a bright color, such as a solid frame of red, green or blue. If the sync frame is grey, with an IRE level for example of 50, it may be visibly different, but could be missed in a fast review of the image because it has less contrast with normal video images. However, a grey frame, since it is more within the IRE parameters of “normal” video image reproduction, may be easier in certain cases to reproduce accurately. White frames can be of several IRE levels. The usual peak white of video recording is 100 IRE. If the generated sync frame is at that level, it could be confused with a normal video image such as a camera pointed at a bright light. It is better to generate the white at an obviously artificial “super white” level, such as 120 to 130 IRE. However, levels above IRE 120 can sometimes cause interference in adjacent audio channels.

[0057] In addition to a generated sync frame for all channels, a second frame can be added to carry identification information about the particular track. Such a “slate frame” can have simple information such as a letter identifier of the track, or complex information about the time reference, name of the production, and other data. FIG. 3 is a schematic view of multiple strips of video frames, showing the synchronization frames simultaneously generated at the same moment in time for each with the addition of a stream-identification slate frame following the synchronization frame. An extra Information frame for Track A is shown at 50, and for Track D at 52. At 54 is an example of a numbered video frame in the track. The slate frame is here shown following the sync frame, although it could also be placed ahead of it. However, if it is placed after the sync frame, it is more suitable for a simple detection circuit that could be set up to recognize the sync frame first as a preparation to the recording of the track information in an automated fashion.

[0058] FIG. 4 is a block diagram of a reader circuit for automatically detecting and reacting to sync frames as part of a postproduction logging and editing process. The frames being reproduced from a playback device are shown at 56, with the direction of travel shown at 58. At 60 is a signal reader, such as for extracting the information usually representing the various details making up a video image. The signal 62 from the reader goes first to a detector 64 for the vertical pulse and other components marking the start of a video frame or field. After this is detected, the signal is sent to a detector for the sync frame signal level at 66. One characteristic of the sync frame signal level is that it does not vary, because the frame is blank. If there is variation in the signal, as normally happens when recording a picture, then the detector resets and waits for the next frame. If the signal matches the assigned level for a sync frame (such as IRE 0, within allowable limits for noise) throughout the frame, then when the end of the frame is detected at 68, a flag is set at 70 for the detection of the sync frame. At that point, a command could be sent to stop and park the recording machine on the sync frame, or perform some other automated playback function such as the recording of slate frame information as described above, or the recording of SMPTE time code data that is present on another track or elsewhere in the video signal, such as that represented by Vertical Interval Time Code (VITC).

[0059] A detector for sync frames in a film-based system could rely on the detection of a black frame in an otherwise continuous motion sequence, such as where the frame image before and after the sync frame is mostly identical (as is true for almost every sequence except for that in rapid motion). If extra marking of the sync film frame has been done with an LED array, then the detector can be designed to recognize only that pattern, and ignore any black frames that could normally be produced by the camera when starting and stopping scenes.

[0060] Although the description above contains many specificities, these should not be construed as limiting the scope of the invention but as merely providing illustrations of some of the presently preferred embodiments of this invention. For example, a sync frame can have an appearance of a simple graphic pattern such as blocks or stripes, rather than a solid frame. A sync frame can also be generated on a single field of an interlaced video signal. The color of the sync frame can be programmed to change in a sequence, to aid in the identification of a particular marked frame by a reader circuit.

[0061] Thus the scope of the invention should be determined by the appended claims and their legal equivalents, rather than by the examples given.