Method and system for migrating connections between receive-any and receive-direct threads

[0002] The invention relates generally to computer networks, and more particularly to an improved method and mechanism for handling network connections with threads.

[0003] A receive-any thread is a thread of execution in a server process that handles both requests for new connections and incoming remote procedure call (RPC) requests on (one or more) existing connections, if any. The receive-any thread responds to a new connection request by posting an accept, and to an RPC data request by posting a receive. Receive-any threads accomplish this dual-functionality using a Windows® Sockets API (WinSock) call such as selects( ) to block, listening for new connection requests and activity on existing connections. Other environments use similar APIs. Since RPC servers typically listen on more than one protocol (e.g., TCP, SPX or ADSP) at any given time, one receive-any thread is provided to deal with each protocol.

[0004] However, because receive-any threads multiplex between requests for new connections and receiving RPC data, perform parameter validation and do extra data copying (buffering), they are relatively slow. More particularly, when receiving data via a receive-any thread, the cost of the select and the extra copying cost of transferring data from the transport buffers to the RPC runtime add substantial runtime overhead. As a result, another type of thread known as a receive-direct thread may be used for an existing data connection.

[0005] A receive-direct thread is a dedicated thread which receives RPC requests on a specific connection and dispatches those requests. A receive-direct thread pre-posts a receive on the connection to avoid the data copying and the overhead associated with select( ), i.e., using the WinSock recv( ) API, a receive-direct thread receives data into application buffers as the data arrives. Receive-direct threads thus have less runtime overhead and provide better performance when handling data than do receive-any threads. Indeed, in certain instances, the use of a receive-direct thread provides at least a twenty percent performance improvement over a receive-any thread.

[0006] However, each receive-direct thread has costs associated therewith including an increased memory size and footprint. With the above scheme of making existing data connections receive-direct connections, many receive-direct threads, up to some predetermined quota, may be allocated in memory. If a connection is idle or mostly idle, the receive-direct thread associated therewith is also idle and system resources are wasted. Moreover, a receive-direct thread may not be available for a newer, highly active connection because other for a newer, highly active connections because other connections, including idle connections, have used up the quota of receive-direct threads. The highly active connection instead has to remain receive-any, and thus suffers from the reduced performance associated with receive-any data handling.

[0007] Accordingly, it is a general object of the present invention to provide a method and mechanism for more efficiently using receive-any and receive-direct threads for handling connections.

[0008] Another object is to provide a method and mechanism of the above kind that adjusts thread usage for idle or active connections.

[0009] A related object is to provide a method and mechanism as characterized above that migrates connections between receive-any and receive-direct based upon connection activity.

[0010] In accomplishing those objects, it is another object of the invention to provide a common receive-any thread that listens on multiple protocols.

[0011] Briefly, the present invention provides a method and mechanism for handling a connection in a computer system between a client socket and a data socket of a server. The server includes a receive-direct thread associated with the data socket for handling communication on the connection, and a receive-any thread with a socket mask associated therewith. When the connection has no activity for a period of time, information referencing the data socket associated with the receive-direct thread is moved to the socket mask associated with the receive-any thread, whereby the (idle) connection becomes a receive-any connection. A receive-any connection that becomes active is migrated to a receive-direct thread, when one is available.

[0012] Other objects and advantages will become apparent from the following detailed description when taken in conjunction with the drawings, in which:

[0013] FIG. 1 is a block diagram representing a computer network into which the present invention may be incorporated;

[0014] FIG. 2 is a block diagram representing components in a server machine logically connected to one another and to client processes;

[0015] FIG. 3 is a block diagram representing an exemplary socket mask;

[0016] FIG. 4 is a block diagram representing an exemplary data structure for storing socket information;

[0017] FIG. 5 is a flow diagram representing the general steps taken by a receive-any thread in accordance with one aspect of the present invention;

[0018] FIG. 6 is a flow diagram representing the general steps taken to make a new connection;

[0019] FIG. 7 is a flow diagram representing the general steps taken by a receive-direct thread in accordance with one aspect of the present invention; and

[0020] FIG. 8 is a flow diagram representing the general steps taken in registering a new endpoint.

[0021] Turning to the drawings and referring first to FIG. 1, there is shown a computer network generally designated 20 into which the present invention may be incorporated. The computer network 20 comprises a number of computer-based machines 22₁-22_nsuch as personal computers interconnected in known manner. The machines 22₁-22_ncommunicate with one another via remote procedure calls by passing data packets. Each of the machines 22₁-22_nmay be configured to recognize many different protocols at the network and transport layers. In FIG. 1, the machine 22₃is shown as a client machine and the machine 22₆is shown as a server machine, although it can be appreciated that in a particular network many of the machines act as both servers and clients. Of course, certain of the machines may be limited so as to act only as clients or only as servers.

[0022] Each of the machines such as the machines 22₃and 22₆of FIG. 1 includes a processor 24, a memory 26, and network communication circuitry 28 that enables the machines to communicate with one another. The memory 26 includes random access memory and/or non-volatile storage, which, as understood, can together comprise a large amount of virtual memory via swapping techniques. The memory 26 may contain instantiated objects and persistent data for other objects as understood in object-oriented technology, as well as WinSock functions (e.g., in the dynamic link library wsock32.dll) as described below.

[0023] FIG. 2 shows the virtual connections between a number of clients 22₁-22₅(more particularly client processes) and the server 22₆. For simplicity, FIG. 2 shows each virtual connection to the server 22₆as coming from a separate client machine 22₁-22₅, although it is understood that a single client machine may have multiple connections to a server. As is known, each of the clients 22₁-22₅communicate with the server 22₆through a virtual connection between a client socket 30₁-30₅and an associated server socket 32₁-32₃and 33₁-33₂. The server 226 maintains information about each of its sockets 32₁-32₃and 33₁-33₂within a set of data structures 34, typically a connection object for each socket. For example, such socket information includes a socket ID, which in TCP (Transmission Control Protocol) comprises a protocol identifier, a server IP (Internet Protocol) address and a server endpoint (port number). Each of the data structures 34 representing a TCP socket includes a field to store this socket ID, along with other fields and flags.

[0024] A connection may be in one of four states, i.e., a new connection, an existing receive-any connection, an existing receive-direct connection, or a closed connection. New connections and existing receive-any connections are handled by a receive-any thread 36, through listening sockets 33 and data sockets 32, respectively. For example, in FIG. 2 the socket 30₃of client 22₃may be connected to the server data socket 32₃, while the clients 22₄and 22₅are each requesting a new connection via server listening sockets 33₁and 33₂, respectively. In accordance with one aspect of the invention, the receive-any thread 36 listens for new connections and handles data on different protocols, as described in more detail below. For this reason, the receive-any thread 36 of the present invention is alternatively referred to as a common listening thread.

[0025] Each existing receive-direct connection is handled by a dedicated receive-direct thread. FIG. 2 shows two receive-direct threads, 38₁and 38₂, which connect via server data sockets 32₁and 32₂to client sockets 30₁and 30₂, respectively. Finally, closed connections are detected and handled by whichever type of thread (receive-any or receive-direct) was handling the connection before it was closed.

[0026] A client thus establishes a new connection via the server receive-any thread 36 through a listening socket (e.g., socket 33₁or 33₂) of an appropriate protocol. In keeping with the invention, when a new connection is established, if a receive-direct thread is available, a receive-direct thread 38_nwill be allocated and the connection will be made a receive-direct connection. A counter (object) 39 or the like is maintained in the server memory 26₆such as one of the data structures 34 to determine if a receive-direct thread is available based on a quota. If a receive-direct thread is not available, the connection will be handled by the receive-any thread 36, i.e., the connection is made a receive-any connection.

[0027] To keep track of the various listening sockets 33₁-33_nand data sockets 32₁-32_nunder the control of the common listening thread 36, the common listening thread 36 maintains pointers to the sockets'data structures in an array known as a socket mask 40. The socket mask 40 is represented in more detail in FIG. 3, with a set of listening sockets 33 and data sockets 32, wherein each letter A, B or C represents a different protocol. The socket mask also includes a synchronization socket 42 (FIG. 3), described in more detail below.

[0028] In accordance with one aspect of the present invention, if a receive-direct connection is idle for a sufficient period of time, the connection is migrated to a receive-any connection. The receive-direct thread (e.g., 38₁) performs this task and essentially frees itself for possible use by a more active connection. To this end, a WinSock option is leveraged (SO_RCVTIMEO provided by Windows NT) that allows a blocked receive call to timeout if not used. The WinSock APIs 44 including select( ), accept( ), and recv( ) along with this option are described in more detail in the Windows Sockets specification version 2.0 and in “Windows™ Sockets Network Programming,” B. Quinn and D. Shute, Addison-Wesley (1996).

[0029] Conversely, existing receive-any data connections that are sufficiently active are migrated to receive-direct connections. The receive-any thread 36 performs this task based on activity on one or more of its data sockets 32 (FIG. 3). As described in more detail below, this allows the active connections to be receive-direct and the idle connections to be receive-any, which provides more-optimal overall performance while efficiently utilizing system resources.

[0030] Turning to an explanation of the operation of the invention with particular reference to the flow diagrams of FIGS. 5-8, the receive-any (common listening) thread 36 is started when the first endpoint on a WinSock protocol is registered by the server 22₆. As shown in FIG. 5, the common listening thread 36 loops forever, and first makes a copy of the socket mask 40, known as the master mask, onto a current mask 46 (FIG. 2) at step 500. At step 502, the common listening thread 36 calls one of the Winsock APIS 44, i.e., the selects API, passing the current mask 46 thereto. The common listening thread 36 is then blocked in the select using the current mask 46, waiting for activity on one of the sockets in the mask. Such activity includes requests for new connections on the mask's listening sockets and RPC requests on the data sockets.

[0031] Moreover, the (master) socket set 40 can change while the common listening thread 36 is blocked in select using the current mask 46. More particularly, a new listen socket may be added, (by RegisterEndpoint, described below), or a receive-direct connection may be migrated to a receive-any connection due to lack of activity, (by the ReceiveDirect thread, also described below). If the socket set 40 changes, the common listening thread 36 needs to wake up, update its current mask 46, and go back to the select with the updated current mask 46. To wake up the common listening thread 36, a synchronization socket 42 is provided in the socket mask 40. Writing (e.g., a byte) to the synchronization socket 42 unblocks the common listening thread 36. As evident from FIG. 5, once unblocked, the common listening thread 36 will eventually loop back to step 500 where the current-mask 46 will be updated. Step 502 will then call selects with the updated current mask 46.

[0032] In any event, one or more sockets can have activity when the select call is unblocked. Thus, the common listening thread 36 tests for listening socket and data socket activity whenever unblocked. As shown in FIG. 5, step 504 tests if a listen socket 33 has had activity, i.e., a new connection has been requested on an endpoint. If so, step 504 branches to step 506 where the NewConnection function 50 (FIG. 2, an d described in more detail below with reference to FIG. 6) is called for the requested connection. Note that for purposes of simplicity FIG. 5 shows only one test for listen socket activity and only one potential call to NewConnection 50, although it is understood that steps 504-506 may be performed multiple times if multiple requests for new connections have been received in the same select. In any event, the request or requests for new connections, if any, are handled by steps 504-506, and the common listening thread 36 continues to step 508.

[0033] Step 508 of FIG. 5 tests if activity was detected on the data sockets 32. If not, the process branches back to step 500 where the current mask 46 is updated and select is again called as described above. However, if there is data socket activity, appropriate ones of the steps 510-518 are executed depending on the type of activity. Again, it is understood that multiple data sockets having activity during the same call to select( ) can be handled by repeating the steps 508-518 as appropriate for each data socket before returning to step 500.

[0034] Thus, upon detecting data socket activity, step 508 branches to step 510 where it is determined whether the activity resulted from a data socket's connection being closed. If so, step 510 branches to step 512 where that socket is removed from the master mask 40 and the receive-any connection cleaned up in a known manner. For example, an open/closed flag in the data structure corresponding to that socket (e.g., the connection object 48 of FIG. 4) is set to closed.

[0035] If instead the activity was a data request, step 510 branches to step 514 where the counter 39 or the like tracking the number of receive-direct threads is checked against the quota of receive-direct threads to determine if a receive-direct thread is available for this connection. At present, it is assumed that any data activity is sufficient to make the connection receive-direct if a receive-direct thread is available. However, in an alternative embodiment, a threshold test of the level of activity may also be performed before migrating a receive-any connection to receive-direct. For example, the first time a receive-any connection has data, the connection's socket information is associated with a timestamp, but the connection remains receive-any. If the next time that the connection has data activity minus the timestamp is less than a certain time duration, that connection will be made receive-direct. Note further that such a threshold test may be modified based upon the number of receive-direct threads available, i.e., if many receive-direct threads are available the threshold is lower (e.g., allowed time duration between activity is longer) than if few receive-direct threads are available.

[0036] In any event, if no receive-direct thread is available (or if the level of activity is below a threshold), step 514 branches to step 518 where the data is handled and passed to the RPC runtime 49. Conversely, if a receive-direct thread is available (and a threshold level of activity, if any, is met), the connection is migrated to receive-direct by branching from step 514 to step 516. The step of migrating includes signaling a ready thread (or if a ready thread is not available, creating the receive-direct thread, e.g., instantiating an object therefor), modifying the data structure (e.g., the connection object 48, FIG. 4) for that socket, including setting the appropriate receive-direct/any field therein to receive-direct status, and removing the data socket from the socket mask 40. For example, the socket pointer 323 (FIG. 3) may be the socket information that is removed from the master mask 40. The counter 39 is incremented, and, of course, the data is also handled. The receive-direct thread 38 is discussed in more detail below with reference to FIG. 7.

[0037] FIG. 6 represents the NewConnection function 50 which is called by step 506 of FIG. 5 in response to listen socket activity as described above. Step 600 of FIG. 6 calls the WinSock accept( ) API to obtain a new socket for the newly requested connection. Assuming that the accept is posted and the socket information is returned, (i.e., the client is allowed to connect and so on), step 602 tests the receive-direct thread counter 39 against the quota of receive-direct threads to determine if a receive-direct thread is available for this connection. At present, it is assumed that a new connection will have data activity, and thus if a receive-direct thread is available, the connection is made receive-direct by branching from step 602 to step 604. Step 604 creates the receive-direct thread, increments the receive-direct thread counter 39 and marks the connection as receive-direct in the corresponding socket data structure. If no thread is available, step 602 instead branches to step 606 where the connection is added to the master mask 40 as another one of the data sockets 32 (FIG. 3), and the data structure for that socket is also modified as necessary to make the connection receive-any. Again, it should be noted that if certain threshold levels of activity are desired before making a connection receive-direct, a new connection can directly be made receive-any, from where, as discussed above, the connection will be migrated to receive-direct if the connection thereafter meets the threshold requirements.

[0038] FIG. 7 generally represents the steps taken by a receive-direct thread (such as the thread 381) to handle a receive-direct connection. Step 700 calls the WinSock recv( ) API, using the time-out socket option and passing a value (in milliseconds) as a time parameter. The passed time parameter is generally selected as a tradeoff between a desire to free idle receive-direct threads while avoiding overly-frequent migrations, and at present has been selected to time-out if no data activity is received on the socket for approximately thirteen minutes. Note that the time may be varied for a given socket based on the number of receive-direct threads available when the connection is made receive-direct, i.e., a longer time if many threads are available and a shorter time if few are available. Of course, the receive call will also unblock when the socket has some data activity, including activity to close the connection.

[0039] Once unblocked from receive, step 702 tests whether the call unblocked as a result of a lack-of-activity time-out. If not, there was activity, and step 702 branches to step 704 to test if the data activity resulted from closing the connection. If the activity was not to close the connection, but instead was a data request, the data (RPC request data) is handled, i.e., the recv( ) receives the data into application buffers as the data arrives, as represented by step 706.

[0040] However, if the activity was to close the connection, step 704 branches to step 708 where the connection is cleaned up in a known manner, e.g., the appropriate open/closed flag in the data structure 48 (FIG. 4) is marked as closed. At step 710, the receive-direct thread 38, is then marked as available by decrementing the counter 39 of receive-direct threads, and the receive-direct thread returns to the runtime where it is reclaimed. The receive-direct thread 38, then effectively ends.

[0041] In accordance with one aspect of the invention, in the situation where a receive-direct thread times out for lack of activity, step 702 branches to migrate the connection to receive-any. To accomplish the migration, step 712 adds the socket to the data sockets 32 in the master mask 40 (FIG. 3). For example, the socket (pointer) 328 is added to the mask 40. Step 714 updates the appropriate one of the data structures 34 by setting the receive-direct/any field therein to receive-any. Then, at step 716, the receive-direct thread writes a byte to the synchronization socket 42 (FIG. 3), so that the receive-any thread 36 will unblock and update its current mask 46 with the new master mask 40 that reflects the receive-any status of the migrated data socket 328. The receive-direct thread 38, is then marked as available by decrementing the counter 39 of receive-direct threads (step 710). The receive-direct thread then returns to the runtime where it is reclaimed, and the receive-direct thread 38, effectively ends.

[0042] Lastly, whenever a new endpoint is registered on any of the WinSock protocols, the common listening thread needs to be made aware of and administer the added listening socket. To this end, a RegisterEndpoint function 54 is provided as generally described in the steps of FIG. 8. Step 800 represents the step of creating the new listen socket. As is known, the server creates a listen socket to listen for new connections on a predetermined protocol, and also sets up an endpoint at step 802 so clients can make the connection requests. Each such endpoint corresponds to one (or more) listening sockets which may belong to different protocols. In keeping with the invention, at step 804, the listen socket, e.g., socket 333 of FIG. 3 is added to the master mask 40. Lastly, the common listening thread 36 is made aware of the change at step 806 as a byte is written to the synchronization socket 42 to wake up the thread 36. As described above, writing the byte unblocks the common listening thread 36 from the select whereby the current mask 46 is updated with the changed master mask 40 before returning to the select.

[0043] The use of the socket mask 40 enables the receive-any (common listening) thread 36 to listen on virtually any protocol, including those not yet established. As long as the client as server are in agreement as to a particular protocol, they can communicate via a listening socket and then a data socket. The receive-any thread 36 detects the activity on the correct socket in the mask 40, and, as described above, handles the client request appropriately.

[0044] As can be seen from the foregoing detailed description, there is provided a method and mechanism for more efficiently using receive-any and receive-direct threads for handling connections. The method and mechanism adjusts thread usage for idle or active connections by migrating connections between receive-any and receive-direct based upon connection activity. The common receive-any thread that listens on multiple protocols.

[0045] While the invention is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention.