Camera control system

[0002] 1. Field of the Invention

[0003] The present invention relates to a camera control system suitable for use with surveillance cameras, video conference systems or the like which perform communication of information, for example, via a network.

[0004] 2. Description of Related Art

[0005] In recent years, with the spread of personal computers (hereinafter called PCs) and a further improvement in the performance of communication between computers which is represented by the spread of Ethernet (trademark), video conference systems, surveillance systems or the like are increasingly becoming realized comparatively inexpensively by connecting control terminals such as general-purpose PCs to a network such as the Internet.

[0006] For example, if a user transmits a control signal for a camera from a control terminal which is being used by the user, the control signal is received by the camera via a network and the camera is controlled in accordance with the control signal. A video signal obtained from the camera is received by the control terminal via the network and a picked-up image is displayed on the display screen of a CRT or the like.

[0007] During this time, control of the camera is executed in such a way that an operating panel provided with operating buttons is displayed on the display screen of the CRT or the like through a GUI (Graphical User Interface) and the user moves a cursor on the display screen with a mouse and clicks on a predetermined one of the operating buttons.

[0008] However, if an image obtained from one or more cameras is displayed oh the display screen or a displayed image is magnified, a number of problems will occur. For example, the operating panel for camera control overlaps a picked-up image of a camera to be controlled by the user and the picked-up image is hidden in whole or part by the operating panel, or the camera becomes difficult to operate.

[0009] It is, therefore, an object of the present invention to carry out display control of an operating panel and realize an orderly display of an image picked up by a camera which is being controlled.

[0010] To achieve the above-described object, in accordance with one embodiment of the present invention, there is provided a camera control system capable of displaying on display means a picked-up image obtained from one or more cameras, which system comprises selecting means for selecting at least one image obtained from the one or more cameras, for displaying purpose, display processing means for displaying an image selected by the selecting means, on the display means in a predetermined size, searching means for searching for an empty area on the display means, other than a display area occupied by the image displayed by the display processing means, and control means for causing a new image obtained from the one or more cameras to be displayed in the empty area found by the searching means, if the new image is additionally selected by the selecting means for displaying purpose.

[0011] In accordance with another aspect of the present invention, there is provided a camera control system capable of displaying on display means a picked-up image obtained from one or more cameras, which system comprises selecting means for selecting at least one image obtained from the one or more cameras, for displaying purpose, display processing means for displaying an image selected by the selecting means, on the display means in a predetermined size, an interface displayed on the display means in a predetermined size, for selectively operating the one or more cameras, searching means for searching for an empty area on the display means, other than a display area occupied by the image displayed by the display processing means, and control means for causing the interface to be displayed in the empty area found by the searching means.

[0012] In accordance with another aspect of the present invention, there is provided a control method for a camera control system capable of displaying a picked-up image obtained from one or more cameras, on a display screen of a display device, which method comprises a selecting step of selecting at least one image obtained from the one or more cameras, for displaying purpose, a display processing step of displaying an image selected by the selecting step, on the display screen in a predetermined size, a searching step of searching for an empty area on the display screen, other than a display area occupied by the image displayed by the display processing step, and a control step of causing a new image obtained from the one or more cameras to be displayed in the empty area found by the searching step, if the new image is additionally selected by the selecting step for displaying purpose.

[0013] In accordance with another aspect of the present invention, there is provided a control method for a camera control system capable of displaying a picked-up image obtained from one or more cameras, on a display screen of a display device, which method comprises a selecting step of selecting at least one image obtained from the one or more cameras, for displaying purpose, a display processing step of displaying an image selected by the selecting step, on the display screen in a predetermined size, an interface displaying step of displaying an interface for selectively operating the one or more cameras, on the display screen in a predetermined size, a searching step of searching for an empty area on the display screen, other than a display area occupied by the image displayed by the display processing step, and a control step of causing the interface to be displayed in the empty area found by the searching step.

[0014] In accordance with another aspect of the present invention, there is provided a storage medium which stores a program for executing processing for a camera control system capable of displaying a picked-up image obtained from one or more cameras, on a display screen of a display device, the storage medium allowing the camera control system to execute the following processing: selecting at least one image obtained from the one or more cameras, for displaying purpose, displaying the image selected, on the display screen in a predetermined size, searching for an empty area other than a display area occupied by the image displayed on the display screen, and causing a new image obtained from the one or more cameras to be displayed in the empty area found, if the new image is additionally selected for displaying purpose.

[0015] In accordance with another aspect of the present invention, there is provided a storage medium which stores a program for executing processing for a camera control system capable of displaying a picked-up image obtained from one or more cameras, on a display screen of a display device, the storage medium allowing the camera control system to execute the following processing: selecting at least one image obtained from the one or more cameras, for displaying purpose, displaying the image selected, on the display screen in a predetermined size, displaying an interface for selectively operating the one or more cameras, in a predetermined size, searching for an empty area other than a display area occupied by the image displayed on the display screen, and displaying the interface in the empty area found.

[0016] The above and other objects, features and advantages of the present invention will become apparent from the following detailed description of a preferred embodiment of the present invention, taken in conjunction with the accompanying drawings.

[0017] FIG. 1 is a conceptual diagram of a camera control system according to an embodiment of the present invention;

[0018] FIG. 2 is a block diagram of the construction of a control terminal according to the embodiment of the present invention;

[0019] FIG. 3 is a block diagram of the entire camera control system according to the embodiment of the present invention;

[0020] FIG. 4 is a view showing a display example on a display screen according to the embodiment of the present invention;

[0021] FIG. 5 is a flowchart of the processing of display control executed by a CPU according to the embodiment of the present invention;

[0022] FIG. 6 is a view showing another display example on the display screen according to the embodiment of the present invention;

[0023] FIG. 7 is a view showing one example of an area list according to the embodiment of the present invention;

[0024] FIG. 8 is a view showing another display example on the display screen according to the embodiment of the present invention;

[0025] FIG. 9 is a flowchart of the processing of display control executed by the CPU according to the embodiment of the present invention;

[0026] FIG. 10 is a view showing another display example on the display screen according to the embodiment of the present invention;

[0027] FIG. 11 is a view showing another display example on the display screen according to the embodiment of the present invention; and

[0028] FIG. 12 is a view showing another display example on the display screen according to the embodiment of the present invention.

[0029] A preferred embodiment of the present invention will be described below in detail with reference to the accompanying drawings.

[0030] FIG. 1 is a conceptual diagram showing a camera control system according to the preferred embodiment of the present invention.

[0031] The camera control system according to the preferred embodiment is connected to a network 901 such as a LAN, a WAN or the Internet. As will be described below in detail, the camera control system is connected to the Internet.

[0032] Client-side apparatus 300, 300a, 300band 300care situated at remote locations and output camera control commands therefrom. Cameras 101 can be controlled from outside for panning, tilting or zooming, and are respectively connected to camera servers 100, 100a, 100band 100c. The cameras 101 connected to the camera servers 100, 100a, 100band 100care operated in accordance with a control command sent from a client-side apparatus having a cameral control right, and signals of input images inputted from the cameras 101 are transmitted to the client-side apparatus.

[0033] In the present embodiment, as shown in FIG. 1, a plurality of camera servers and a plurality of client-side apparatus may be individually connected to the network 901, or a plurality of units in each of which a camera server and a client-side apparatus are integrated may be connected to the network 901. Such units each including a camera server and a client-side apparatus are capable of effecting mutual communication which enables a camera situated on one side to be operated on another side via the network 901.

[0034] In accordance with the camera control system according to the present embodiment, an infinite number of camera servers and client-side apparatus of the above-described types may be connected to each other via the network. However, the present embodiment can also be accomplished in such a way that one camera server and one client-side apparatus are connected to each other via the network.

[0035] In the camera server 100, a communication interface 902 transmits and receives signals to and from each of the camera servers and the client-side apparatus via the network 901. A command interpreting circuit 903 interprets a received signal and transmits a predetermined operation signal to the camera 101 connected to the camera server 100. An image input circuit 904 receives an input image from the camera 101 and performs signal processing on the image. A camera control circuit 905 is provided for controlling a panning, tilting or zooming operation of the camera 101.

[0036] In the client-side apparatus 300, a display control part 907 displays an image received from a camera server which is being accessed, or the state of control of the camera 101 connected to the camera server. An operation input part 908 requests a camera control right from the camera server which is being accessed, or instructs the camera 101 to alter the state of panning, tilting or the like, or inputs set values of camera parameters.

[0037] Incidentally, in such camera control system, one or more cameras may be connected to the camera server 100.

[0038] In the following description, for the sake of simplicity, it is assumed that control terminals in each of which a camera server and a client-side apparatus are integrated are connected to a network.

[0039] FIG. 2 is a detailed explanatory block diagram of the construction of a control terminal which corresponds to the aforesaid camera server and client-side apparatus. In the construction of a control terminal 20 which is shown in FIG. 2, a CPU 1 secures a work area by downloading into a RAM 14 either of an OS (operating system) 4 and a camera control program 5 which are stored in a secondary storage 3, and collectively controls the entire control terminal 20 or transmits a control command to a camera 12 via a network 13.

[0040] The secondary storage 3 stores the OS 4, the camera control program 5 and the like, and is a magneto-optical disk (MO), a compact disc (CD-ROM), a hard disk, a floppy disk or the like.

[0041] The OS 4 is basic software for processing the operation of the control terminal 20, and the camera control program 5 is software for generating a camera control signal for panning, tilting, zooming or the like of the camera 12, and for performing A/D or D/A conversion of an image signal or applying predetermined processing such as compression or expansion to an image signal. The camera control program 5 performs management of an image display area and a display area for a camera control GUI 21 (to be described later) in a display part 7, or performs display processing such as “Move”, “Deform” and “Scale”.

[0042] An input part 6 is provided with a keyboard and a pointing device such as a mouse, and plays the role of allowing a user to input a control command for the camera 12 through the camera control GUI 21 shown in FIG. 4. The display part 7 includes a display device which is a bit-map display, a CRT or an LCD, and displays a picked-up image or the camera control GUI 21 for operating the camera 12 via the network 13.

[0043] The panning, tilting or zooming operation of the camera 12 can be continuously controlled, and an image processing device 9 performs A/D conversion of an image signal sent from the camera 12 or another terminal and image processing such as compression or expansion of an image signal, and displays the obtained image on the display part 7 or at another control terminal 20 via the network 13.

[0044] A camera I/F 10 transmits a camera control signal to the camera 12. A network I/F 11 performs communication with another control terminal 20 connected to the network 13, via the network 13. At this time, the network I/F 11 performs processing corresponding to the kind of network (such as Ethernet or ATM).

[0045] As described previously, if the CPU 1 is to execute various kinds of control, the RAM 14 provides a work area to the CPU 1 when the CPU 1 downloads the OS 4 stored in the secondary storage 3 into the RAM 14. A ROM 15 is a storage medium which stores various programs similarly to the secondary storage 3.

[0046] The above-described constituent elements of the control terminal 20 are connected to one another via an internal bath 8 so as to transmit and receive signals to and from one another.

[0047] FIG. 3 is a block diagram of the entire camera control system in the present embodiment. In FIG. 3, each of control terminals 20a, 20b, 20cand 20dhas a construction similar to the control terminal 20 shown in FIG. 2, and each of the control terminals 20a, 20b, 20cand 20dcan transmit a signal such as an image signal and a camera control signal to any other control terminal via the network 13. Incidentally, although the present embodiment is provided with four control terminals 20 for the sake of simplicity of description, the number of control terminals is not limited to four.

[0048] FIG. 4 is a view showing a display example on the screen of the display part 7 of the present embodiment. In FIG. 4, reference numeral 18 denotes a display screen of the display part 7, and images picked up by cameras connected to the control terminals 20aand 20bare respectively displayed in display areas 19aand 19b. In the example shown in FIG. 4, since images picked up by cameras connected to the control terminals 20cand 20dare not selected for display on the display screen 18, neither of them is displayed on the display screen 18.

[0049] The camera control GUI (operating panel) 21 is used for selecting a control terminal to which a camera to be controlled is connected, and controlling a field of view for image pickup and an image pickup magnification of the camera connected to the control terminal selected. In the camera control GUI 21, buttons 22a, 22b, 22cand 22dare used for specifying whether to display on the display screen 18 images picked up by the respective cameras connected to the control terminals 20a, 20b, 20cand 20d, or for specifying which of the cameras connected to the respective control terminals is to be controlled.

[0050] An image to be displayed on the display screen 18 is selected in such a way that a user moves a cursor 40 with a mouse (not shown) provided in the input part 6 and clicks on one of the buttons 22a, 22b, 22cand 22dwhich corresponds to the one of the control terminals 20a, 20b, 20cand 20dto which is connected a camera which is picking up the image desired to be displayed. In the example shown in FIG. 4, the button 22ais clicked on and changed in color, and the image picked up by the camera 12 connected to the control terminal 20ais displayed in the display area 19a.

[0051] If the user desires to acquire a control right for any one of the cameras 12 connected to the respective control terminals 20a, 20b, 20cand 20d, the user double clicks on the one of the buttons 22a, 22b, 22cand 22dthat corresponds to an image displayed by the above-described operation. Then, the: CPU 1 issues a control right acquisition request and requests a control right from a control terminal to which the desired camera is connected. FIG. 4 shows that the camera 12 connected to the control terminal 20bis in its controllable state, and the button 22bis displayed with its color changed to a further extent and a thick frame which indicates that the camera 12 connected to the control terminal 20bis in the controllable state is displayed in the display area 19bin which the image picked up by that camera 12 is displayed.

[0052] Buttons 28, 29, 30 and 31 are respectively used for instructing the controllable camera 12 to move its attitude upward, downward, leftward and rightward. Buttons 26 and 27 are respectively used for varying the image pickup magnification of the controllable camera 12 toward its telephoto end and its wide-angle end.

[0053] In the present embodiment, the picked-up images from the cameras connected to the network are displayed on the display screen 18 shown in FIG. 4, without overlapping each other, and the camera control GUI 21 is reduced in scale and moved to and located at a position at which the camera control GUI 21 interferes with as few of the picked-up images as possible.

[0054] The processing of display control to be exerted on the display screen 18 by the CPU 1 will be described below with reference to the flowchart shown in FIG. 5. In the following description, it is assumed that the user operates an arbitrary one of the control terminals 20ato 20d.

[0055] As shown in FIG. 6, by way of example, it is assumed that the camera control GUI 21 and the display area 19boccupied by the image picked up by the camera 12 controlled by the camera control GUI 21 are displayed on the display screen 18. The camera control GUI 21 and the display area 19bcan be moved within the display screen 18 if the user performs a dragging operation to move the cursor 40 with the mouse (not shown).

[0056] During the state shown in FIG. 6, if the user desires to newly display on the display screen 18 an image picked up by one or more cameras connected to the control terminals 20a, 20b, 20cand 20d, for example, the camera 12 connected to the control terminal 20a, the user first clicks on the button 22aon the camera control GUI 21 (S101). When the CPU 1 detects the clicking of the button 22a, the CPU 1 requests the control terminal 20ato transmit image data picked up by the camera 12 connected to the same, and acquires the image data (S102).

[0057] In addition, the CPU 1 searches in an area list to determine whether an empty area in which to display the image data acquired from the control terminal 20ais present on the display screen 18.

[0058] The area list is stored in the RAM 14 in the form shown in FIG. 7 by way of example, and includes the following items: the coordinate values of the upper left point and the lower right point of a maximum area of the display screen, the coordinate values of the upper left point and the lower right point of the title of a picked-up image being currently displayed (the name of the control terminal to which the camera is connected), and the coordinate values of the upper left point and the lower right point of the camera control GUI 21, as well as data indicating whether the control terminal to which the camera picking up the image is connected is a target for camera control. If one of the display areas 19aand 19band display areas 19cand 19dor the camera control GUI 21 is moved by the operation of the user, the CPU 1 recognizes a position which has been reached by the one moved by the user, and rewrites the area list stored in the RAM 14.

[0059] In area search, a maximum rectangular area on the display screen 18 is searched for, other than an image (indicated at 19bin FIG. 6) from a control terminal which is currently displayed on the display screen 18 (S103).

[0060] Then, the CPU 1 determines whether the maximum rectangular area is sufficiently large to display a new image (S104). If there is no sufficient empty area, the CPU 1 moves the display position of the currently displayed image and relocate all areas displayed on the display screen 18 (S105), and updates the values of the area list stored in the RAM 14 with the values obtained from such relocation (S106). The process returns to Step S103, in which area search is again performed.

[0061] If it is determined in Step S104 that there is an empty area sufficient to display a new picked-up image, the CPU 1 displays the image at an appropriate position in the empty area (S107). Furthermore, the CPU 1 enters the position at which the image is displayed into the area list (S108).

[0062] In this manner, in the processing of displaying the image picked up by the camera 12 connected to the control terminal 20aduring the state shown in FIG. 6, the display area 19boccupied by the image picked up by the camera 12 connected to the control terminal 20bis moved for relocation within the display screen 18, and the display area 19ais set in the resultant empty area and the image picked up by the camera 12 connected to the control terminal 20ais displayed in the display area 19aso that all the picked-up images are displayed without overlapping each other, as shown in FIG. 4.

[0063] The above description has referred to a case where the camera control GUI 21 does not overlap a picked-up image. However, as the number of picked-up images increases further as shown in FIG. 8, it becomes necessary to execute the processing of relocating the camera control GUI 21 so as not to overlap any of the picked-up images. FIG. 9 shows an operational flowchart showing such processing.

[0064] By way of example, in the following description, reference will be made to the processing of further displaying an image picked up by the camera 12 connected to the control terminal 20c, on the display screen 18 when it is in the state shown in FIG. 4.

[0065] In the flowchart shown in FIG. 9, the processing of Steps S201 to S208 is similar to that of Step S101 to S108 of FIG. 5, and the description thereof is omitted.

[0066] As shown in. FIG. 8, a newly displayed picked-up image (displayed in the display area 19cin FIG. 8) will occasionally overlap the camera control GUI 21. The CPU 1 again makes a search in the area list and checks whether the camera control GUI 21 overlaps the display area occupied by the new picked-up image (S209). If the camera control GUI 21 does not overlap the display area occupied by the new picked-up image, the process is brought to an end.

[0067] If the camera control GUI 21 overlaps the display area occupied by the new picked-up image, the CPU 1 searches for an empty area into which to move the camera control GUI 21, in the area list (S210). In the case of FIG. 8, a rectangular frame 52 shown by dashed lines corresponds to such empty area. (The rectangular frame 52 actually is not displayed on the display part 7.) If there is such empty area, the CPU 1 moves the camera control GUI 21 into the empty area (in this case, the rectangular frame 52), as shown in FIG. 10 (S212). In this manner, the picked-up image can be displayed without interfering with the camera control GUI 21.

[0068] There is another case where no empty area is present in the display screen 18. For example, as shown in FIG. 11, if a picked-up image is displayed in the display area 19dfor the control terminal 20d, in addition to the picked-up images displayed in the display areas 19a, 19band 19c, nearly the whole of the display screen 18 will be occupied by the image display areas.

[0069] In this example, the CPU 1 moves the camera control GUI 21 onto a display area in which the image picked up by the camera 12 connected to a control terminal for which the user does not have a control right is displayed. For example, since the camera 12 connected to the control terminal 20bis in a selected state, the CPU 1 moves the camera control GUI 21 onto a display area other than the display area 20b, for example, the display area 19ain which the image picked up by the camera 12 connected to the control terminal 20ais displayed, as shown in FIG. 11 (S213).

[0070] Then, after the completion of the movement of the camera control GUI 21, the CPU 1 updates the area list on the basis of new data (S214). In this manner, if the display areas which display different picked-up images occupy nearly the whole of the display screen 18, the CPU 1 controls the position of the camera control GUI 21 to prevent it from overlapping the image picked up by the camera 12 which can be controlled by the user, whereby the user can view the image picked up by operating that camera 12.

[0071] However, if the display screen 18 is in the state shown in FIG. 11, the user will not be able to view all the picked-up images. For this reason, in the processing operation of Step S214, the CPU 1 may also display the camera control GUI 21 on a reduced scale so that the user can view the picked-up image overlapped by the camera control GUI 21, with the camera control GUI 21 interfering with none of the picked-up images as shown in FIG. 12.

[0072] If there is an empty area but there is no area sufficient to normally display the camera control GUI 21, the camera control GUI 21 may be reduced in scale so as to be accommodated into an area into which to move the camera control GUI 21. The reduction in scale of the camera control GUI 21 can be realized, as by reducing the space between buttons or eliminating a blank space.

[0073] As is apparent from the foregoing description, in accordance with the present embodiment, picked-up images transmitted from the network are automatically displayed on the display screen without overlapping each other, and furthermore, the camera control GUI 21 is automatically moved into an empty area or displayed on a reduced scale so that the camera control GUI 21 is prevented from interfering with the picked-up images. Accordingly, the user can perform rapid camera control and can also view the picked-up images displayed on the display screen.

[0074] Incidentally, it is also possible to achieve the present invention by supplying a system or an apparatus with a storage medium on which the program code of software for realizing the function of the above-described embodiment is recorded, and causing a computer (a CPU or an MPU) of the system or the apparatus to read out and execute the program code stored in the storage medium.

[0075] In this case, the program code itself read out from the storage medium realizes the function of the above-described embodiment, and the storage medium which stores the program code constitutes part of the present invention.

[0076] The storage medium for supplying such program code may make use of, for example, floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM or the like.

[0077] Although the function of the above-described embodiment can be realized by a computer which reads and executes a program code, the function of the above-described embodiment may also be realized by an OS (operating system) running on the computer, in such a way that the OS performs actual processing in whole or part on the basis of an instruction of the program code.

[0078] In addition, the function of the above-described embodiment may be realized in such a way that a program code read out from the storage medium is written into either a memory provided in an expansion board inserted in the computer or a memory provided in an expansion unit connected to the computer and a CPU provided on the expansion board or the expansion unit performs actual process in whole or part on an instruction of the program code.

[0079] If the present invention is applied to the storage medium, program codes :corresponding to the flowchart described previously are stored in the storage medium. In brief, modules essential to the camera control system according to the present invention are stored in the storage medium.

[0080] In the present invention, it suffices to store the program codes of at least three modules in the storage medium. The modules include a searching module for searching for an empty area in the display screen, a moving module for moving a picked-up image and a camera control GUI on the display screen according to an empty area found on the basis of a search result of the searching module, and a reducing mode for reducing in scale the camera control GUI.

[0081] As is apparent from the foregoing description, in accordance with the present embodiment, in an apparatus capable of selectively controlling one or more remote cameras which are connected to each other, for example, via a network, an image from such one or more cameras and a camera control interface are displayed on a display screen without overlapping each other, and if there is no sufficient empty area in the display screen, the camera control interface is displayed on a reduced scale without interfering with images from the cameras. Accordingly, even if the number of images displayed on the display screen is large, a user can view all of the displayed images and perform rapid camera control.