System and method performing image processing of image data of low resolution and arranging image data of high resolution

[0002] This application is a division of application Ser. No. 09/009,266, filed on Jan. 20, 1998, now U.S. Pat. No. 6,317,115, which is a continuation of application Ser. No. 08/354,118, filed on Dec. 6, 1994, now abandoned.

[0003] 1. Field of the Invention

[0004] The present invention relates to a multimedia server and, more particularly, to an apparatus and method for editing images, and an apparatus and method for controlling image display, which provide various service functions, such as printing/scanning, facsimile transmission/reception, database access, or computer conferences, to computers on a network.

[0005] 2. Description of the Related Art

[0006] As various terminals now include communication functions with the advances in networking in recent years, interconnectivity becomes very important. For example, for facsimiles, there are facsimile communication protocols defined by CCITT as to a method of accessing a public network. However, protocols for a case in which terminals are formed into a network as a facsimile server have not been standardized.

[0007] Although an SQL language of ANSI has been established as a standard for a relational database (RDB), protocols for access to an RDB and a front end interface API (Application Program Interface) are different from vendor to vendor. Although there is an industry standard for each OS for printing, architecture for network printers in a multi-vendor environment has not yet been standardized.

[0008] Regarding OCR, since there has hitherto been no concept of networking, a method of accessing a network OCR is to be determined. There is also no concept of an image processing accelerator on the network. As described above, an effort for standardization has been made as for each individual application field, such as a database for a database, or a network printer for a network printer. However, standardization across the fields has not been made. For example, to perform a series of operations such that an image received by a facsimile server is converted into codes by an OCR server and the codes are stored in a database, since there is no linkage among the respective servers, the client side initiates three client-side programs for facsimiles, OCRs and databases and the operations are performed in sequence. The above operations may possibly be automated if the three programs perform an inter-application communication by using the functions offered from the up-to-date OS. Although the up-to-date OS offers a scheme for inter-application communication, the way of the inter-application communication is left to a vendor and nothing has been decided.

[0009] However, in the conventional server system, for example, to perform a series of operations such that an image received by a facsimile server is converted into codes by an OCR server and the codes are stored in a database, since there is no linkage among the respective servers, the client side initiates three front-end programs, a facsimile front-end, an OCR front-end and a database front-end, and the operations are performed in sequence. However, most of the operation include a number of “cut and paste” and repetition of accesses to a “temporary file”, and this entails a great burden.

[0010] Also, the method of accessing each different server is different as an inevitable historical consequence of the invention of the server, and access across the service fields is nearly impossible. The present invention is designed to integrate independent servers which are distributed on the network in a virtual manner and to logically construct a huge virtual server so that compatibility among the servers is not required and a number of server functions can be used easily from one application.

[0011] Hitherto, it is not possible for a small personal computer to edit a high-resolution image on a large screen because of a shortage of memory. For example, a full-color image of an A4 size at 400 dpi (dot/inch) requires a memory of as much as 47 megabytes. Making it possible to handle such a large image by a personal computer is unrealistic in terms of cost and speed, and poses a problem from a point of view of reliability. The CPU power of a personal computer is not so large due to the limitation of a single chip architecture and is slow in speed. In the method of taking in a hard disk into a part of RAM, a despairing delay occurs each time swapping occurs. Also, in the case of the OS of today's personal computer, if such a large memory is handled, the personal computer often crashes.

[0012] It is an object of the present invention to provide an image display control apparatus and method for outputting a drawing command to a server; interpreting the drawing command output from the output means and acquiring image data written in an image memory allocated in the server; and displaying the image data acquired by the acquiring means.

[0013] It is another object of the present invention to provide an image editing apparatus and method for inputting a drawing command from a host computer; interpreting the drawing command input from the input means and drawing the image data on the allocated image drawing memory; and outputting the image data to the host computer so that the image data drawn by the drawing means is displayed on the screen memory of the host computer.

[0014] To achieve the above-described object, according to one aspect of the present invention, there is provided an image display control apparatus and method connected to a server for performing image processing, the image display control apparatus comprising: output means for outputting a drawing command to the server; acquiring means for interpreting the drawing command output from the output means and for acquiring image data written in an image memory allocated in the server; and display control means for displaying the image data acquired by the acquiring means.

[0015] According to another aspect of the present invention, there is provided an image editing apparatus connected to a host computer, comprising: input means for inputting a drawing command from the host computer; drawing means for interpreting the drawing command input from the input means and for drawing the image data in the allocated image drawing memory; and output means for outputting the image data to the host computer so that the image data drawn by the drawing means is displayed on the screen memory of the host computer.

[0016] According to a further aspect of the present invention, there is provided a network system whereby a server for performing image processing is connected to a host computer in the form of a network, the network system comprising:

[0017] the host computer which comprises: output means for outputting a drawing command to the server; acquiring means for interpreting the drawing command output from the output means and for acquiring image data written in an image memory allocated in the server; and display control means for displaying the image data acquired by the acquiring means, and

[0018] the server which comprises: input means for inputting a drawing command from the host computer; drawing means for interpreting the drawing command input from the input means and for drawing the image data in the allocated image drawing memory; and output means for transferring the image data to the host computer so that the image data drawn by the drawing means is displayed on the screen memory of the host computer.

[0019] According to a further aspect of the present invention, there is provided an image display method in an image display control apparatus connected to a server for performing image processing, the image display method comprising the steps of: outputting a drawing command to the server; interpreting the drawing command output from the output means and acquiring image data written in an image memory allocated in the server; and displaying the image data acquired by the acquiring means.

[0020] According to a further aspect of the present invention, there is provided an image editing method in an image editing apparatus connected to a host computer, the image editing method comprising the steps of: inputting a drawing command from the host computer; interpreting the drawing command input from the input means and drawing the image data in the allocated image drawing memory; and outputting the image data to the host computer so that the image data drawn by the drawing means is displayed on the screen memory of the host computer.

[0021] The above and further objects, aspects and novel features of the invention will more fully be appreciated from the following detailed description when read in connection with the accompanying drawings. It is to be expressly understood, however, that the drawings are for the purpose of illustration only and are not intended to limit the invention.

[0022] FIG. 1 is a block diagram illustrating the system configuration of a multimedia server in accordance with one embodiment of the present invention;

[0023] FIG. 2 is a diagram illustrating a method of accessing existing servers via the center server shown in FIG. 1;

[0024] FIG. 3 is a diagram illustrating the structure of a message in a multimedia server in accordance with the present invention;

[0025] FIG. 4 is a block diagram illustrating the procedure of transmitting a first process script in the multimedia server in accordance with the present invention;

[0026] FIG. 5 is a chart illustrating real-time processing in accordance with the present invention;

[0027] FIG. 6 is a chart illustrating another real-time processing in accordance with the present invention;

[0028] FIG. 7 is a chart illustrating batch processing in accordance with the present invention;

[0029] FIG. 8 is a chart illustrating a communication method employing process scripts in accordance with the present invention;

[0030] FIG. 9 is a block diagram illustrating the procedure of transmitting a second process script in the multimedia server in accordance with the present invention;

[0031] FIGS. 10(a) and 10(b) show examples of scripts which are transferred from a client to a center server in accordance with the present invention;

[0032] FIG. 11 shows an example of communication protocols among the client, the application, and the center server in accordance with the present invention;

[0033] FIG. 12 is a block diagram illustrating the system configuration formed of the center server and the multimedia server to which an exchange is connected;

[0034] FIG. 13 is a block diagram illustrating the configuration in which the multimedia server is made to work as an image drawing accelerator in accordance with the present invention;

[0035] FIG. 14 is a system block diagram of the image drawing accelerator in accordance with the present invention;

[0036] FIG. 15 is a specific block diagram of a network drawing accelerator in accordance with the present invention;

[0037] FIG. 16 is a flowchart illustrating local virtual memory processing in the multimedia server in accordance with the present invention; and

[0038] FIG. 17 is a flowchart illustrating the operation of the image drawing accelerator in accordance with the present invention.

[0039] FIG. 1 is a block diagram illustrating the system configuration of a multimedia server in accordance with one embodiment of the present invention. This configuration, in particular, corresponds to a case in which an existing server 10 is accessed via a center server 11.

[0040] In the multimedia server described in this embodiment, a server suited to handle multimedia information is formed of an “integrated server” and a “digital exchange”. One virtual integrated server is constructed on the network, and various types of services are provided.

[0041] In this embodiment, the “integrated server” refers to a server such that a group of servers distributed on the network are logically integrated, which integrated server is formed of a plurality of function servers of the conventional type for providing various services, and one center server for controlling the servers. The center server may be a dedicated server or a server shared by the client.

[0042] When the center server is seen from the client (the host computer), a group of a plurality of function servers distributed on the network is seen as one huge integrated server. Of course, each different server may be accessed individually as in the past. However, passing “process scripts” to the center server, so that the subsequent operations are left to the center server so as to perform the operations, is more efficient because the client is able to quickly proceed to the next operation.

[0043] Referring to FIG. 1, reference numeral 1 denotes an application serving as a “front end” which accesses the server. The “front end” will now be described briefly.

[0044] If the server and the client are present in the same computer, the application is able to directly call the functions of the server and use the functions. However, if the server is present on another computer over the network, it is not an easy matter because the application must access the server via the communication program.

[0045] In such a case, the application programmer cannot easily write a program for accessing the server. Therefore, it is common that the server vendor prepares an accessing scheme whereby the server is accessed by the same calling as when the server is present locally even if the server is on the network.

[0046] The scheme is like a part of the server, has communication functions, and is installed in the client. This small program is called a front end in contrast with the server, and it is possible for the programmer to write applications which access the server on the network without considering communications at all.

[0047] The user inputs and outputs data through an application 1. Reference numerals 2 and 4 each denote a client/server type communication program; reference numeral 3 denotes a local area network (LAN); reference numeral 5 denotes a center server which provides a standard access environment which is integrated for the application 1. The center server 5 functions to provide all the services from the application 1. Since the basic function of the center server 5 is to logically integrate various existing servers, the center server 5 usually does not have primary service functions of the server. Thus, the center server 5 accesses the existing server 10 through a server manager 6 and responds to a request from the application 1. Reference numeral 7 denotes a client communication program; reference numeral 8 denotes a local area network (LAN); and reference numeral 9 denotes a server communication program, data to be handled and interfacing not being the same as the communication programs 2 and 4.

[0048] A language (command) for accessing the existing server 10 has a language specific to each server due to the relation with the configuration of the respective servers. Therefore, the client/server communication programs 7 and 9 for accessing purposes are often all their own, and there may be a case in which the client/server communication programs 2 and 4 for connecting to the center server 5 for integrating a plurality of servers are different from the client/server communication programs 7 and 9.

[0049] The LAN 8 need not to be the same type as the LAN 3 because of the same reason. For example, the LAN 3 may be an AppleTalk (trademark), and the LAN 8 may be an Ethernet (trademark). The server manager 6 for controlling the existing server 10 functions as a front end of the existing server 10, and the interface with the client communication program 7 is also an existing API.

[0050] On the other hand, the center server 5 provides an access method which is common to all the servers, and an API which is provided via the client/server communication programs 2 and 4 is also common. Therefore, protocol conversion and data conversion are performed for absorbing the differences in architecture between the center server 5 and the server manager 6.

[0051] For the above purpose, the center server 5 and the server manager 6 are present in the same computer and operate by close association. The means by which the functions of the center server 5 and the server manager 6 are integrated is a function of the center servers. Therefore, the center server 5 is called a server-server when the function needs to be clearly discriminated.

[0052] FIG. 2 is a diagram illustrating a method of accessing an existing server via the center server 5 shown in FIG. 1.

[0053] In FIG. 2, when clients 20 to 25 access a center server 26 by a common access method, the center server accesses necessary function servers 27 to 30 through an individual access method in accordance with a request from the client. Therefore, the respective clients 20 to 25 need not to be concerned with the method of accessing the individual function servers A to D, much less the existence of the function servers A to D.

[0054] In the multimedia server constructed as described above, the center server which transfers messages over the network receives a message such that a predetermined process script and data are paired. The received message is issued from each client, and the center server interprets the received message and performs communication protocol conversion and data conversion for each function server. The respective process script and the data are transferred over the network while the consecutive operations for each function server are handled instead. Thus, it becomes possible for each program of the client to obtain desired result data by merely transferring process scripts corresponding to the integrated function operation and data to the center server even when the center server is made to perform a plurality of functions.

[0055] The “process script” will now be explained. The process script is such that commands are written in the form of text in a basic unit at which data communication is made in accordance with communication protocols. It is possible to transmit commands and data at the same time. Since the process script is usually used for a communication between machines, there is no difference if the command is a text or binary codes. However, since a text can be read by a human being, an unusual application is possible such that a machine transmits the text to the human being via an electronic mail and the result thereof is received via the electronic mail.

[0056] The greatest feature of the process script is that the communication protocols themselves can be transmitted in the form of scripts. Thus, it is possible to establish a communication while dynamically changing the communication procedure. Let's recall that the communication procedure of a G4 facsimile. Since the service is constructed by exceedingly strict protocols, a feat such that when a facsimile document is received, the same document is distributed to some locations which cannot perform within the standard protocols.

[0057] Since the process script is such that the service procedure is written rather than the communication procedure, any communication method or communication media may be used, and an integrated operation can be specified easily for a terminal which understands basic services. As an example, let's consider a case in which a DTP document is printed from Japan by a printer in an office in the United States. Since the network is usually not connected, such operation cannot be performed. In a new method, a print command issued from the DTP application is packed in a process script and delivered to the destination by electronic mail. When the received scripts are transmitted to the script manager automatically if the party is a mail server, otherwise manually, the contents are analyzed, and a printed output appears on the printer of the office of the other party.

[0058] Since a center server is connected to the network and receives a message such that a predetermined process script and data are paired, which message is issued from each client, interprets the received message, performs communication protocol conversion for each function server, and transfers the process script for each function server over the network, it becomes possible for each program of the client to obtain desired result data by merely transferring process scripts corresponding to the integrated function process and data to the center server even when the center server is made to perform a plurality of functions.

[0059] Further, since each function server receives the process scripts from the center server and receives data from the client over the network in accordance with ID information in the process scripts, it becomes possible to transfer scripts and data independently of each other among the clients and the respective function servers.

[0060] Also, since the center server connected to the network receives a message such that a predetermined process script and data are paired, which message is issued from each client, interprets the received message, performs communication protocol conversion and data conversion for each function server, and transfers the process script for each function server over the network, and since the exchange unit exchanges the telephone line between each client and the center server, it becomes possible to transfer integrated information in parallel in a multi-session for real-time information, such as voice information, among the clients while each function process is performed.

[0061] As a result, the operation procedure is standardized to one type of a simple integrated access such that the services of all the servers, and thus a seamless, useful server environment can be provided. For example, it becomes possible to perform a series of integrated operations such that an image received by a facsimile server is converted into codes by an OCR server and stored in a database by a file server continuously by one script. Therefore, it is possible to considerably reduce the operation required by the client side as in the past such that the client side initiates three front-end programs of a facsimile front-end, an OCR front-end and a database front-end and the operations are performed in sequence, a number of “cut and paste” and repetition of accesses to a “temporary file” are reduced.

[0062] Since independent servers distributed on the network can be integrated virtually, it becomes possible to use a number of service functions or integrated service functions easily from one application without the client considering the compatibility of each server. In addition, it is possible to provide a method which integrates a method of accessing a server apparatus, which method has hitherto been developed separately in individual companies and industry fields, and which accesses the server having different existing interfaces on the network by one integrated access method. A description will be given in detail below for each embodiment.

[0063] FIG. 3 is a diagram illustrating the structure of a message in a multimedia server in accordance with the present invention.

[0064] As shown in FIG. 3, the message in this embodiment is formed of a process script fork (process script) and a data fork (data). The process script and data have a tag element (tag) having contents common to them.

[0065] In this tag, “time” when the message is created, “life time” when the message will be erased, a series of “ID numbers”, “type” of data, “sign” of the application from which the data is created, and the like are written. Of these items, “ID numbers” are indispensable, but other items are optional.

[0066] FIG. 4 is a block diagram illustrating the procedure of transmitting a first process script in the multimedia server in accordance with the present invention.

[0067] In general, in most cases, a client 41, a center server 43, and a function server 44 are all on the same LAN. When the function server 44 is a server which can understand process scripts in conformity with the present invention, the center server 43 indicates that fact to the client 41, the client 41 separates a message 42 to a process script 45 and a data fork (data) 46 and transmits them to the center server 43 and the function server 44, respectively. The center server 43 edits the received process script 45 as required, after which a new process script 47 is transmitted to the function server 44. Since both the client 41 and the center server 43 are clients for the function server 44, the function server 44 combines the process script 47 and the data 46 which have been received separately, performs necessary operations on the data, and sends back a result message 48 to the client 41 over the LAN.

[0068] In this embodiment, the process script is a program list such that the procedure for performing a series of operations is written by the center server 43 by using a group of function servers 44. The center server 43 performs a series of operations in accordance with the program list and sends back only the result message 48 to the client 41. Thus, the load of the client 41 is considerably smaller than a conventional case in which a group of function servers are accessed individually.

[0069] The process script is sometimes automatically created by a device driver of the client 41. The process script is merely an assembly of a series of function calls of the communication protocols for the client 41, is a script language created from within the series of communication protocols for the center server 43, and is communication protocols themselves for the group of function servers 44.

[0070] The feature of the process script is that the arrangement of the conventional computer communication can be broadly classified into the following: real-time processing, and batch processing.

[0071] The batch processing is a method often used for processing by a large computer (mainframe) in which when operations are centralized by a number of users, the jobs are spooled sequentially in a queue, the operations are processed sequentially and the results are sent back to the user. There is a drawback in that the time the result is output is not known, and when it is strongly desired to determine the progress, the queue must be looked into.

[0072] In the real-time processing, a scheme by which the progress can be determined at any time is prepared by exchanging the process scripts, and a necessary operation in the process is performed.

[0073] Although the real-time processing appears convenient at first sight, conditions are imposed that the network must also be a real-time system. When, for example, the network is of a “store and forward” type like Internet, and when several hubs are relayed, a delay of several minutes occurs for each relay. In such a network, real-time processing cannot be used.

[0074] The process script is a communication method which is most suitable for such a “store and forward” type network and has a characteristic which is intermediate between real-time processing and batch processing. The script means “writes a command.” The command is written in the form of text, and the protocols are made simpler to a certain extent.

[0075] An example of real-time processing is shown in FIG. 5. When a user issues a command of “ready to send data” and a response of “ready” is received from the server, the user sends “data” and sends a delivery confirmation of “acknowledgement” in response to the response of “complete”. The reason why such protocols can be used is due to the end to end communication between the user and the server without any hubs in between.

[0076] A communication through the hubs 1, 2 and 3, as shown in FIG. 6, will now be described. Delays are accumulated in sequence from the time of “ready to send data” to the time of “ready”, and it is no easy matter to reach a stage for “data” transmission. In the meantime, some communication connection is disconnected due to timeout or the like. Thus, this communication cannot be made at all practical.

[0077] FIG. 7 illustrates an example of batch processing in which the “store and forward” type network is taken into consideration. The response of “ready” in response to the “ready to send data” is received from a near hub. If this is done, “data” can be sent relatively earlier. However, even in this method, a recovery routine for a case in which some kind of communication error occurs in the middle is significantly complex. A case in which “ready to send data” is lost in the middle can possibly be managed, but if the response “ready” is lost, the recovery thereof is probably impossible.

[0078] FIG. 8 illustrates an example of a communication method employing process scripts in accordance with the present invention. The command of “process data and respond” and “data” in a set are sent in turn. Since the command of “processed result” and the processed “data” in a set are sent, there is no communication method in this network which is faster than this communication method. Even when the process script is lost due to an abnormal communication, the recovery is easy because the command and the data are in a set.

[0079] FIG. 9 is a block diagram illustrating the procedure of transmitting a second process script in the multimedia server in accordance with the present invention, showing, in particular, an access method when the function server is an existing server. Those components in FIG. 9 which are the same as those in FIG. 4 are given the same reference numerals.

[0080] In FIG. 9, the process script and the data of the message 42 are sent as a request without being isolated to the center server 43. When the requested function is not present in the center server 43, the function server 44 which performs the function instead is searched, the request of the client 41 is transmitted to the function server 44 by an existing access method, and the result is received. The result is sent back as the message 48 to the client 41 <communication protocols>.

[0081] One role of the center server in the multimedia server is to standardize different access methods of various servers. When the application accesses the center server having a standardized interface, access to individual servers is performed by the center server instead from that point. Items to be standardized, when broadly classified, are standardization of data and standardization of an access method.

[0082] The number of standardizations of data is not one, but more than one. Data is standardized by an industry standard format which is independent of a specific application, and exchange with non-standard data is allowed.

[0083] Main formats currently in use are: QuickDraw, GDI, RTF, TIFF, PICT, Bitmap, PostScript, EPS, G3/G4, PCL, HP-GL, ANSISQL, ASCIIText, UNICODEText, and Binary Data (trademarks and product names).

[0084] On the other hand, the standardization of the access method is nearly unified. It is possible to easily use various services in an integrated manner by the unified access method.

[0085] However, the industry standard access method cannot help being supported. On the other hand, as explained with reference to FIG. 3, the contents to be transmitted is a message, that is, “process script” and “data” are in a set. “Data” is information itself to be processed, and “process script” is a command sequence of how that information is to be handled. The set of “process script” and “data” are transmitted among the servers, and necessary operations are performed one after another. By standardizing the method of exchanging “process script”, it is possible to access various servers by the same access method.

[0086] The center server 43 works as a gateway for the server having different protocols. For example, a function call for accessing the facsimile server, the contents shown in FIG. 10(a), are issued from the client 41. As a result, the scripts shown in FIG. 10(b) are passed to the center server 43.

[0087] FIGS. 10(a) and 10(b) show examples of scripts which are used in FIG. 9. FIG. 10(b) shows a script portion which is actually sent to the network. In this case, since a G3 facsimile is used as an example, MMR data flows, but the illustration thereof is omitted.

[0088] The address (server) of the server and the service type (fax₁₃service) are specified by MM₁₃open. The fact that the type of fax₁₃service is G3 is specified by MM₁₃send. The destination Fax number (destination) is also set.

[0089] The size (length) and contents (content₁₃buffer) of the transmission data buffer are sent by MM₁₃data. This service is terminated by MM₁₃close. Since the process script is flexible, it is used in a manner close to the real-time system at this time.

[0090] FIG. 11 shows in detail the sequence which is written in a simplified manner.

[0091] The communication protocols among the application, the client and the center server conform to the procedure shown in FIG. 11.

[0092] There are the following two methods for the access sequence. The first method is a store and forward type which is store-and-forward switching. When the pair of the process script and the data is completely transmitted, the communication is temporarily terminated regardless of the success/failure of the process result, and the final processed result is examined by restarting the communication again. When the operation is completely entrusted to the server, the client side is able to proceed to the next operation, and thus the use efficiency of resources is increased.

[0093] The second method is a real-time method in which the connection of the communication is ensured until the final processed result is determined. Although this method has the drawback in that the operation of the client is restricted, the operational reliability is high.

[0094] Next, a description will be given below of the function processes of a digital exchange in this embodiment.

[0095] The digital exchange of this embodiment is an exchange for controlling the public switched network and the local switched network, and performs call control in accordance with a connection instruction from the center server 43. By making a second connection having no delay via the local switched network among a plurality of clients connected with delay via the LAN, it is possible to transmit information (voice) requiring a real time response.

[0096] FIG. 12 is a block diagram illustrating the system configuration formed of the center server and the multimedia server to which an exchange is connected in accordance with the present invention, which is an application example of FIG. 2.

[0097] Referring to FIG. 12, reference numerals 61 to 65 denote various function servers; reference numerals 66 to 70 denote clients; reference numeral 71 denotes a center server for integrating the respective function servers 61 to 65; reference numeral 73 denotes a local area network (LAN); and reference numeral 72 denotes an exchange (PBX) for switching the telephone line of the extension/external line. Since data is formed into packets and transmitted through the LAN 73, this data transfer is not suitable for real-time transmission, such as voice or moving images. However, by utilizing the feature that all the clients are always connected, this data transfer is suitable for data transfer which does not require connection control and broadcast communication.

[0098] On the other hand, the PBX 72 is suitable for real-time transmission of voice and moving images. By taking full advantage of the feature of the LAN and the PBX, it is possible to construct a personal computer conference system (PC conference). A case in which an access to the client 67 is made from the client 66 so as to conduct a PC conference will now be considered. The client 66 issues to the center server 71 a start command script for starting a PC conference with the client 67. The center server 71 issues a script for connecting with the multimedia client demon (MCD) of the client 67 and instructing the start of the PC conference. In this example, the demon refers to a program which always runs in the background. Since the ID number of the PC conference is written in the script, by sending scripts to the destination of that ID number thereafter from the clients, data communication is possible among the clients.

[0099] At the same time, the center server 71 accesses the PBX 72 via a control line 74 in order to connect the telephone sets near (or built in) the clients 66 and 67 so that voice can be transmitted. In this way, the operators of the clients 66 and 67 make a conversation by voice while watching the common screen of the computer, and thus a PC conference is conducted. The PC conference need be a one-to-one connection, and a plurality of clients may be connected at the same time.

[0100] As a result, a LAN in which a packet transmission is in the main and which is not suited for real-time data transmission, and an exchange which is suited for exchange between voice and moving images and which is not suited for broadcast communication and intelligent control are interconnected to each other, and thus it is possible to construct a personal computer conference system without adding special optional parts to a commercially available personal computer.

[0101] Service functions using a high-speed data highway will now be explained below.

[0102] Although in this embodiment a system construction is considered by assuming components which are relatively easy to obtain today, data packets and voice lines may be multiplexed when a network having a transmission rate of 100 Mbps or more, such as FDDI (Fiber Distributed Data Interface: United States ANSI standard) can be used.

[0103] Also, although in this embodiment ISDN is assumed as a public network connected to the exchange PBX, B-ISDN may be used. When the United States SMDS-class public network can be used, the LAN-side connection may be connected to the outside.

[0104] When speed may be sacrificed to a certain extent, an inter-LAN connection may be connected through the ISDN network such as UNIX's SLIP (SerialLineIP). Even with current technology, use of an ISDN router makes an inter-LAN connection possible over the ISDN network.

[0105] Examples of the services to which the present invention can be applied are shown below.

[0106] (Field)—(Service)

[0107] Printing—Color printing

[0108] Scanning—Color scanner

[0109] OCR—OCR, file keeper, voucher processing

[0110] Translation—Japanes/English

[0111] File system—NFS

[0112] Database—text, still image, moving image, voice

[0113] Conference system—shared window+real-time voice

[0114] Mail—text, still image, moving image, voice

[0115] Contribution system—still image, moving image, voice

[0116] Image processing—CMM, filtering, image processing accelerator

[0117] Fax—Fax transmission/reception/distribution

[0118] MHX—text, Fax, CATS

[0119] Reservation system—conference room reservation.

[0120] As regards the service function process, this embodiment will be explained below in comparison with the prior art by using the OCR service as an example.

[0121] The operation will now be considered in which 10 documents are scanned by an OCR and converted into character codes, and saved as a text file in a disk of a data server.

[0122] In the prior art, a document is scanned by a scanner and transmitted to a client. The image data is retransferred to the OCR server, applied to an OCR in order to convert the image data into text data, the text data is transmitted to the client, and the text data is transferred to the disk of the data server. The above is repeatedly performed ten times.

[0123] On the other hand, in this embodiment, the client transmits a process script to the center server (the document is scanned by the scanner→transmits to the OCR server) (applied to the OCR in order to convert to text data) (text data is transferred to the disk of the data server) center server→notifies the client of the result.

[0124] In the prior art, four times of data transfer intervened by the client is required. Since as much as 10 documents are not scanned at a stretch by a conventional personal computer because of the limitation of the capacity of the hard disk, 40 data transfers are required for worst case. In contrast, only two data transfers intervened by the client are required, and all others are communications among the servers. The whole data transfer is reduced. Data communication/processing which is not related to the client are indicated by the parentheses. Since reduction in the number of times of data transfers indicates a reduction in the number of times the computer is operated, the operation can be considerably automated.

[0125] As regards the service function process, this embodiment will be explained below in comparison with the prior art by using the scanning service as an example.

[0126] An operation will be considered in which a document of an A3 size is scanned in a full color at 400 dpi by a network scanner, color processing is performed on the image data, and the image data is printed by a print server.

[0127] In the prior art, scanning of a document→image data is transmitted to the client→color processed image data→the image data is transmitted to the print server whereby the image data is printed.

[0128] On the other hand, in this embodiment,

[0129] client→process script is transmitted to the center server (scanning of the document) (image data→color processing) (color processed image data→transmitted to the print server) (print) center server→notifies the client of the result.

[0130] Since a normal personal computer is limited by the memory capacity, actually, the prior art is merely a desk calculated value. In reality, there is almost no such OS capable of managing such a huge main memory of 96 MBytes.

[0131] As regards the service function process, this embodiment will be explained below in comparison with the prior art by using the mail service as an example.

[0132] A case in which a moving-image file is transmitted to the server and the presentation of the moving image is made.

[0133] In the prior art,

[0134] a huge moving-image file→transmitted to the server

[0135] In this embodiment,

[0136] client→transmits moving-image scripts to the center server (center server to function server communication) center server→notifies the client of the result.

[0137] Moving-image files are generally very large in size. In some cases, a movie of tens of minutes exceed one Gigabytes.

[0138] To handle data of such a size, it is necessary to expand the memory and/or a hard disk of the client, of the conventional client and, in some cases, to replace the CPU unit with a high-speed one.

[0139] However, in this embodiment, since the server stores moving-image data, it is not necessary to expand the memory and/or a hard disk of the client, and the CPU may be a slow-speed one. If process scripts are issued to the multimedia server (center server) over the network, the reproduction of the moving image is performed by the server instead.

[0140] As regards the conference system function process, this embodiment will be explained below in comparison with the prior art by using the mail service as an example.

[0141] A plurality of clients are connected to each other at the same time, one file is opened, an electronic board which can be written from anywhere is prepared, the exchange is accessed at the same time so that the respective extension telephone sets are connected to each other, and thus a PC conference is conducted.

[0142] In the prior art,

[0143] a system, such as text-based conference system in which no voice is used

[0144] In this embodiment,

[0145] client→transmits process scripts for conference to the center server (the center server to function server communication) (PC conference among the clients) client→transmits a termination process script to the center server→the center server→notifies the client of the result.

[0146] Referring to FIG. 13, a description will be given below of a case in which the multimedia server in accordance with the present invention is made to function as an image drawing accelerator.

[0147] FIG. 13 is a block diagram illustrating the concept of a case in which the multimedia server is made to function as an image drawing accelerator in accordance with the present invention. This embodiment, in particular, is concerned with a case in which the multimedia server is used as an image drawing accelerator for a computer (Trademark: Macintosh) of Apple Computer Corporation.

[0148] For example, to edit a full-color bit-mapped image of B5 at 400 dpi, the main memory requires tens of megabytes, and thus a conventional personal computer is not capable of editing such an image. Even if the virtual memory function offered from the up-to-date OS is used, a full-color image of A4 at 400 dpi or thereabouts is a limit in terms of processing speed. On the other hand, since the multimedia server of this embodiment is used as a virtual memory on the network, it is possible to edit a full-color high-resolution image of A3 size or so.

[0149] The virtual memory will now be explained. The virtual memory is a technique to use a huge memory. The real memory usually refers to a semiconductor memory (or main memory), and only a part of the huge memory size required is present. In contrast, the virtual memory refers to a hard disk or the memory (or an auxiliary storage) of the server on the network. The access speed thereof is slow, but has a huge memory space. Such a method is called a virtual memory that main storage and auxiliary storage are well combined, and the entire memory space is seen as if it were a uniform and seamless space when seen from an application.

[0150] When the application accesses a portion not present in the main storage, the memory controller detects an error, the current information of the main storage is saved in an auxiliary storage, and necessary information is loaded from the auxiliary storage to the main storage in a reverse manner, and swapped in the memory space that the addresses of the main storage are requested. Thus, when seen from the application, it looks as if a huge memory is actually present.

[0151] Specifically, an application 101 always sends a drawing command to a drawing control program (product name: QuickDraw) 102, a part of OS, when something is to be displayed on a monitor in order to request to rasterize a screen memory 109. There is no case in which the application directly accesses the image memory.

[0152] When, on the other hand, a bit-mapped image is to be displayed on the monitor, a buffer memory for drawing is usually allocated under the responsibility of the application, and the image is drawn there in the background, after which a request to transfer memory is issued to the QuickDraw 102. Accordingly, the buffer memory for drawing is under the control of the application, and setting the real memory on the hard disk by using the buffer memory as a virtual memory has hitherto been performed.

[0153] In this embodiment, the real memory is provided in the multimedia server on the network, and when the screen is updated, a partial copy is made in the local memory.

[0154] Reference numeral 103 denotes a drawing command handler which is an application program interface (API) for passing the functions of the multimedia server to the application on the client side. The application 101 passes to the API the same parameter as the call to the QuickDraw so that the image is written in the memory allocated in the server on the network.

[0155] Since the application is not necessarily designed by considering the multimedia server, the function call to the application 101 or the QuickDraw 102 may be hooked to forcedly pass the control to the drawing command handler 103. All the function calls of QuickDraw are performed by exception handling, and the handling can be easily branched. In such a case, the drawing command for the image memory is passed through the QuickDraw. Needless to say, the application designed by considering the multimedia server is more efficient in drawing than the application designed without considering the multimedia server.

[0156] The bitmap drawing command for the memory is sent to a PDL rasterizer 106 via a client 104 and a server 105 where the bitmap drawing command is rasterized into bitmap data and written in a remote real memory 108 in the multimedia server. The remote real memory 108 has a capacity for storing all the bitmap data for one page, while the capacity of the local virtual memory 107 is limited.

[0157] The local virtual memory 107 virtually overlaps the remote real memory 108. The portion where the memory is allocated actually is only a limited area which is currently being edited. If the area which is being edited is changed, the image data in a new area is copied from the remote real memory 108. As a result, an image editing operation which was usually impossible in the past, such as edition of a full-color image of A3 at 400 dpi because of the limitation of the memory capacity, can be processed from an inexpensive terminal by using the resources on the network.

[0158] FIG. 14 is a block diagram of the image drawing accelerator in accordance with the present invention. The side on the right of the dotted line is a server side, and the side on the left of the dotted line is a client side. Those components in FIG. 14 which are the same as those in FIG. 13 are given the same reference numerals.

[0159] An example in which the client is a Macintosh (product name) will be explained. In normal use, when the application 101 draws on the screen, the application 101 calls a screen memory control system 109 called grafport in order to allocate a drawing port. The screen buffer memory allocated by grafport is of a bitmap system, and the resolution of the screen buffer memory is 72 dpi (dot/inch).

[0160] The application 101 is capable of calling a series of screen server functions called QuickDraw (product name), and usually control is passed to the OS 102 in the form of a ToolBox call (a system call of OS). The OS rasterizes the received screen drawing function call into a bit map and draws in the grafport.

[0161] In the present invention, the ToolBox call of QuickDraw is hooked (stolen) so that control is temporarily passed to a network drawing accelerator 110 (net₁₃grafport), causing a ToolBox call to be issued again. The network drawing accelerator 110 shows the portion including drawing command handler 103 and the client communication program 104.

[0162] In this example, the client communication program 104 is a communication program which merely provides a communication means, and the client side refers to the whole of the left side from the dotted line in the figure. The same holds for how the server side and the server are called, and in some cases, they refer to only the communication program, and in some cases, the server refers to the whole of the right side from the dotted line.

[0163] The network drawing accelerator 110 converts the ToolBox call of QuickDraw from the application 101 into process scripts and transmits them also to a network drawing accelerator server (the server side) over the LAN. A network drawing accelerator server 111 receiving the process scripts draws the same image as for the client in the high-resolution drawing system 108 (server₁₃grafport) at a high resolution of, for example, 400 dpi (dot/inch). At the same time when drawing in the local screen memory (the buffer memory managed by the screen memory control system 109) is completed, the drawing of a high-resolution image has been completed on the server side. If necessary, it is possible to print and output the image as it is, and to print it on a film.

[0164] FIG. 15 is a block diagram of the network drawing accelerator 110. Although it is possible to realize the entire client side by means of software, it is also possible to realize the entire client side by means of hardware using an accelerator board in order to achieve a faster processing speed. This accelerator is inserted into the bus (υbus) 209 of the computer of the client in the form of a board. A bus I/O 206 is a bus input/output for connecting an internal bus 205 inside the board and an external bus 209 and for arbitrating them. A CPU 201, a ROM 202, a RAM 203, and a LAN I/F 204 are connected to each other through the internal bus, thus a microcomputer is formed.

[0165] An interrupt handler 207 is used to guide an interrupt signal line 210 of the computer body of the client to the CPU 201 on the board. A BUS INT signal 211 is output to the mother board via an I/O 208, and is a control line for requesting the CPU of the computer body of the client to use the bus. A BUS ACK signal 212 is a control line for informing whether the bus has been released.

[0166] When the application issues a drawing command, some QuickDraw routine is called, and an interrupt (INT) occurs. Therefore, that signal can be detected via the interrupt signal line 210. The CPU 201 causes the I/O 208 to be enabled to grant the BUS INT signal 211 so as to issue a bus release request to the CPU of the mother board. When the bus is actually released and a tristate is reached, the BUS ACK signal 212 is returned, and the CPU 201 detects that the external bus 209 is placed in the usable state. Thereupon, the I/O 206 is accessed to connect the internal bus 205 to the external bus 209, and the pointer of the interrupt jump table written in a predetermined external memory (not shown) is fetched. Since the type of the interrupt can be determined by reading the contents of the memory indicated by the pointer, when the interrupt is an interrupt related to the QuickDraw routine, the QuickDraw command is converted into process scripts in the RAM 203 and the process scripts are transmitted to the server from the LAN I/F 204.

[0167] On the server side, the mirror image of the contents drawn in the remote real memory of the entire screen is reflected in the local virtual memory 107. In practice, only the portion of a local real memory 112 having the same size as that of the screen memory 109 is copied. An address converter 113 is a circuit for performing address conversion so that the real memory 112, when seen from the external bus 209, is seen as if it was a part of the virtual memory 107. If the coordinates of the origin (upper left corner) of the virtual memory 107 are X, Y and if the coordinates of the rectangular area of the real memory 112 are x, y, the address converter 113 performs the computation of X−x and Y−y with respect to the address accessed from the external bus 209.

[0168] When the CPU (not shown) of the mother board computes using the portion displayed on the CRT, that is, only inside the real memory 112, the positional relation of the real memory on the virtual memory does not vary. However, if an attempt to access outside the real memory by scroll or the like is made, an interrupt signal 114 is generated to supply the interrupt signal to the CPU 201. The CPU 201 examines the cause of the interrupt and requests the server to send the bit-mapped image at the new coordinate position over the LAN 204, and also resets the address converter so that the relative position of the real memory is changed. In practice, the interrupt signal 114 is generated by the address converter 113.

[0169] Next, a description will be given below of the summary of the local virtual memory operation in the multimedia server in accordance with the present invention with reference to FIG. 16.

[0170] FIG. 16 is a chart illustrating the summary of local virtual memory processing in the multimedia server in accordance with the present invention. The operation of the accelerator for QuickDraw under the control of, for example, MacOS, will be explained below.

[0171] In the local memory of this embodiment, instead of the application requesting the OS to allocate as a real memory primarily, the real memory allocated in the server connected through the network is processed as if it was a real memory in a virtual manner. Therefore, the local virtual memory exists virtually in the local machine of the client, and the real memory exists in the remote server.

[0172] Specifically, when the application starts drawing, the application issues a memory allocation command, for example, NewPtr (an abbreviation of New Pointer, a command for allocating the memory area) to the MacOS. The drawing command handler steals the command and converts it into process scripts (P scripts) and sends the process scripts to a PDL rasterizer/server (hereinafter referred to as a remote drawing engine) via the communication path of the client/server.

[0173] The remote drawing engine issues a malloc command (an abbreviation of memory allocate which works the same as NewPtr) to the remote MacOS (not shown) in order to allocate the real memory, and also sends back the process scripts to the client. Since the process scripts are sent back to the application from the drawing command handler as the pointer of the memory, it appears as if there was a real memory locally from the application.

[0174] Next, the application issues a “drawing command” to the MacOS in order to draw something in the local virtual memory. The “drawing command” is also stolen in the drawing command handler, converted into process scripts, sent to the “remote drawing engine” whereby the data is actually written on the remote real memory at a high resolution of 400 dpi. Since it is also necessary to draw on the monitor of the client, the remote drawing engine emulates and transmits the drawing command to the MacOS of the client. In such a case, the data is written at a low resolution of 72 dpi.

[0175] The result of the drawing is converted into process scripts, and are returned to the client similarly to that described above. This process is repeatedly performed for a required number of times (corresponds to the flow of the thick line in the figure).

[0176] When the drawing is completed and printing is performed, unless according to this embodiment, the data is rasterized at a high resolution of 400 dpi on the local client, and the large amounts of data are transmitted to the printer. According to this embodiment, since the rasterization at the high resolution is completed in the “remote engine” when the drawing is completed, it is possible to immediately start printing without sending the large amounts of data. Therefore, only the “print command” is transmitted.

[0177] Where the completed bitmap high-resolution image data is saved in a secondary storage apparatus (not shown), if, though the “save command” is issued, “data” sent from the remote drawing engine in accordance with the command is directly saved in a secondary storage apparatus or the like, it does not become necessary to provide a real memory locally.

[0178] The operation of the image drawing accelerator will be explained below in detail with reference to both the system principle diagram of FIG. 13 and the flowchart of FIG. 17.

[0179] FIG. 17 is a flowchart illustrating the operation of the image drawing accelerator 110. The left half portion from the center shows the client side, and the right half portion shows the server side. An explanation will be given by assuming that a commonly used connection operation (for example, a connection such as AppleTalk or TCP/IP) has been completed in the figure. The program shown in the flowchart in the left half portion of FIG. 16 is stored in the ROM 202 or the RAM 203 of the network drawing accelerator 110 of the client side, which program is executed by the CPU 201. The program shown in the flowchart in the right half portion of FIG. 16 is stored in the ROM (not shown) or the RAM (not shown) of the server side, which program is executed by the CPU (not shown).

[0180] Before the application of the client performs a drawing operation, an operation for allocating a memory for storing the result is performed. This operation is an operation which makes a request for the OS, and is generally called a system call. In the case of Macintosh, it is called a ToolBox call. The CPU 201 hooks the function call via the drawing command handler 103 in order to request the server side to allocate a real memory. This request is shown in the exchange in steps S12 and S23.

[0181] At the same time, the CPU 201 allocates a minimum buffer memory for the screen memory in the local machine as shown in step S13 via the drawing command handler 103. Since the screen of the local machine is 72 dpi (dot/inch), the size of the buffer memory is very small.

[0182] Next, only the pointer of the local virtual memory 107 having the same size as that of the remote real memory 108 is secured, and overwritten in the real memory by using the pointer which was returned in step 23. Further, the pointer of the screen memory secured in step S13 is assigned to an appropriate portion of the space of the local virtual memory 107. As a result, a part of the picture drawn in the huge remote real memory 108 is displayed on the screen of the client.

[0183] Thereafter, the CPU 201 waits for an event via the drawing command handler 103 (step S15). When some drawing event occurs (step S16), the drawing command is transmitted to the server side, and also the same drawing is performed in the screen memory of the local machine in order to make a screen display (step S17). In the server side receiving the drawing command, the CPU (not shown) draws at a high resolution in the remote real memory 108 via the PDL rasterizer 106 (step S24).

[0184] As stated earlier, since a part of the picture drawn in the huge remote real memory 108 is displayed on the screen of the local machine, the event is a scroll event wait for scrolling the screen vertically and horizontally (step S18). In this case, a parameter representing a rectangular area of the local screen memory 109, which is assigned in such a way that the area is present in a part of the memory space represented by the local virtual memory 107 is changed by an amount corresponding to the scroll (step S19).

[0185] When the local screen memory 109 is moved within the local virtual memory 107, the picture written in the local screen memory becomes different from the picture at the same position where the picture is drawn in the remote real memory 108. Thus, the CPU 201 requests the server to send a bit-mapped image in a new rectangular area via the drawing command handler 103 (step S25). The CPU (not shown) of the server sends the bit-mapped image in the specified area to the client side (step S26), the CPU 201 of the client side copies the image in the local virtual memory 107. As a result, the image information is overwritten in the local screen memory 109, and the display screen becomes new. Thereafter, waiting for an event is repeated.

[0186] When the application 101 edits a bit-mapped image, it edits an image in the local real memory (not shown) by using the drawing function 102 of the OS, and the result is displayed on a monitor (not shown) via the screen memory 109. However, the local memory used in the present invention is the local virtual memory 107 which has only a part of the memory as an entity. As a result, it becomes unnecessary to prepare a huge memory. In this case, as indicated by the x mark in FIG. 13, the place where all the drawing commands are primarily passed to the rasterizer 102 of the OS is stolen and guided to the drawing command handler 103, and the local virtual memory 107 and the screen memory 109 are indirectly controlled as required by the rasterizer 102.

[0187] The stolen drawing command is transmitted to the PDL rasterizer 106 of the server side via the client communication program 104 and the server communication program 105. Then, a bit-mapped image is drawn in the high-resolution remote real memory 108. The image rasterized in the remote real memory 108 can be printed as it is.

[0188] The local virtual memory 107 is a memory having the same size as that of the screen memory 109 and corresponds to a part of the remote real memory 108. When network image processing is performed, a commonly used client/server connection is made between the client 104 and the server 105, after which an image memory requested by the application is allocated. The memory allocation request is fetched by the drawing command handler 103, in practice, the local virtual memory 107 is allocated in the local machine, and the entity of the real memory is allocated as the remote real memory 108 in the server as a result of the exchange between step S12 and S23. The size of the local virtual memory 107 matches the virtual size of the remote real memory 108, and differs for each document size handled.

[0189] The buffer for the screen memory is also allocated in the local machine in step S13 at the same time. Thereafter, the application 101 is placed in a state in which a drawing input event is waited for, that is, the drawing command handler 103 is also placed in an input wait state. This state is shown in step S15. If the operator operates a mouse or the like and inputs some drawing command, and if the command is a drawing event, the drawing command is sent to both the rasterizer 102 of the local machine and the PDL rasterizer 106 of the server, and the data is drawn in the screen memory 109, the local virtual memory 107 and the remote real memory 108 by means of the CPU 201 and the CPU (not shown) of the server. These steps are indicated at 16, 17 and 24.

[0190] If the above drawing event is a screen scroll, the reference coordinates of the screen memory 109 in the local virtual memory 107 vary. Since there is no image there, the server is notified of the change of the rectangular area as shown in steps S19 and S25, and bitmap information is transferred to the client from the predetermined position of the remote real memory 108 as shown in steps S20 and 526. Thereafter, the server and the client alike are placed in a state in which they wait for the next event.

[0191] According to this embodiment, as described above, when a drawing function call issued from the application to the operating system is hooked, scripts corresponding to the drawing function call are issued to the server where the scripts are interpreted, the image is drawn on the image drawing memory allocated in the server, the drawn drawing image data is taken from the image memory to the operating system over the network and drawn in the screen memory. Thus, it is possible to perform a high speed operation of drawing an image of a large amount which exceeds the amount of data which can be processed by the client.

[0192] Also, according to this embodiment, when a drawing function call issued from the application to the operating system is hooked, scripts corresponding to the drawing function call are issued to the server where the scripts are interpreted, the image is drawn on the image drawing memory allocated in the server, the drawn drawing image data is taken from the image memory to the operating system via the local virtual memory and drawn in the screen memory. Thus, it is possible to perform an operation of drawing an image of a large amount which exceeds the amount of data which can be processed by the client, in particular, a drawing operation corresponding to the edition screen, at a high speed.

[0193] Also, according to this embodiment, scripts instructing the start of image drawing are issued from each client to the server, an image drawing program is initiated on the basis of the issued scripts, and also an image drawing memory is allocated, a parameter corresponding to the allocated is returned to the client, and a desired image is drawn in the image memory in accordance with each drawing instruction received for the client over the network. Thus, it is possible to perform an image editing operation which exceeds the image processing performance of the client.

[0194] Also, according to this embodiment, the server analyzes scripts instructing the start of image drawing from each client, and an image drawing memory for assuring image drawing of a resolution higher than that corresponding to an output device of the client is allocated. Thus, it is possible to perform an image editing operation at a high resolution which exceeds the image processing performance of the client at a high speed.

[0195] Many different embodiments of the present invention may be constructed without departing from the spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific embodiments described in this specification. To the contrary, the present invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the claims. The following claims are to be accorded the broadest interpretation, so as to encompass all such modifications and equivalent structures and functions.