System and method for maintaining N number of simultaneous cryptographic sessions using a distributed computing environment

[0001] This application is a continuation-in-part of U.S. Ser. No. 09/259,885 filed Feb. 27, 1999, now U.S. Pat. No. 6,484,257.

[0002] The field of the present invention relates generally to the encryption and decryption of data conducted over a distributed computer network. In particular, the field of the invention relates to a software architecture for conducting a plurality of cryptographic sessions managed over a distributed computing environment, including a mechanism for authenticating and managing policies for network elements in order to ensure that the integrity and security of data that traverses the computer network is maintained and properly audited.

[0003] An N session distributed architecture is described which solves the problems encountered with providing a secure network. The present software solution boosts performance to previously unattainably high levels and provides a practical security solution capable of servicing N simultaneous cryptographic session using a distributed computing environment without additional encryption decryption hardware at wire-speed levels. An aspect of the invention provides a solution, which overcomes the network bandwidth latency barriers to secure encryption. Another aspect of the invention provides a scalability solution to the problem of processor saturation due to encryption decryption loads.

[0004] There is a growing need to provide for secure commerce on computer networks, which does not require costly non-scalable computational resources. Corporations now have critical needs for ensuring the security of data that traverses their networks. Information Systems (IS) managers have attempted to cope with those needs by installing and managing expensive hardware to provide protection of data. In the case where data must be transferred between sites, IS managers can dictate their security needs to the telephone companies who manage the transfer of data between multiple sites. However, there are several problems limiting the transfer of data networking. Such concerns are as follows:

[0005] Network Availability (also known as uptime);

[0006] Network bandwidth (the amount of data that the overall network can handle over a particular time slice);

[0007] Quality of Service: ensuring that pre-determined service levels, such as bandwidth congestion allowances and network latency, are consistently met for all hosts connected to the network;

[0008] Security: ensuring that sensitive data are protected as it traverses the network and those unauthorized parties do not compromise that data or the network itself.

[0009] Monitoring/Auditing (the capability to verify that the above needs are being met and the ability to instantly detect and react to any deviation from preset expectations)

[0010] When considering a new technology that will impact a network, an IS manager must address the foregoing issues. After these requirements are met, factors of cost and scalability must be considered. IS managers are constantly looking for ways to meet the above requirements while reducing the cost of supporting their network. Managing the cost of expanding a network to address increased bandwidth requirements of users is a major problem for IS managers today.

Point-to-Point Encryption

[0011] Point-to-point link level encryption has a disadvantage in that it is not scaleable. For example, there is a dramatic and non-linear cost difference in installing and maintaining a 128 k Frame Relay link versus a 1.544M Frame Relay link. The cost problem is not limited to bandwidth, but rather is also greatly affected by the addition of new groups of hosts as additional connection points. Related equipment also must be installed and maintained. Point-to-point encryption also has cost disadvantages. Point-to-point link level encryption is usually all or none meaning that all data both public and private are encrypted over this link. This additional overhead is acceptable in some cases but undesirable in others.

[0012] Since link level encryption requires static routes to be created it does not integrate easily into the Internet paradigm, which requires packets to be dynamically routed from point to point. A network layer (or higher) encryption solution is required in order to fit easily into the framework of routable IP packets. Currently there exists a transport level security mechanism for application programs using SSLv3 (secure sockets layer). SSL was developed in 1995 when a universally recognized security mechanism at the IP layer did not exist. This has been the most commonly used protocol for providing secure applications.

[0013] The three protocol capabilities of SSL include authentication, encryption and key exchange. In IPSec these are provided as separate protocols (AH, ESP and IKE).

[0014] In SSL most of the communications protocol data is passed in plaintext, only the application header and actual data sent to the application is cryptographically protected. The encryption and integrity protection for the data and not the communications as in IPSec, which protects both, are handled by the record protocol. The negotiation of new crypto algorithms and keys is handled by the handshake protocol. Finally, any errors that have occurred are handled by the alert protocol. SSL maintains its security state based on the session associated with a particular set of host addresses and ports.

[0015] SSL sessions are established in four steps. In Step 1 the sender sends a hello message to the receiver containing random data. In Step 2 the receiver sends the sender his/her public key embedded in a signed certificate. In step 3 the sender encrypts a shared secret key and a change cipher spec switch (to determine the proper cipher to use) with the receiver's public key and sends it to the receiver. In step 4 the receiver sends a reply using the shared secret key (after decrypting the info in step 3 with his private key) and a “finished” message. Both sides now can begin communications. Using the record protocol, all data that passes between the two parties are encrypted and hashed and the recipient checks this hash upon decryption to make sure that the data have not been modified in transit.

[0016] The newest version of SSL (3.0) supports RSA key exchange, Diffie-Hellman anonymous or signed (the most common implementation is SKIP) and Fortezza using SKIPJACK. TLS (Transport Level Security) and PCT (Private Communication Technology) by Microsoft are both variations on SSL that are vying for standards approval by the IETF. A major disadvantage of all versions of SSL is that SSL is ineffective against many of the newer communications level (below transport level) attacks, which are technically called SYN Flooding, Buffer Overruns and Traffic Analysis.

IPSec

[0017] IPSec is a conventional protocol for securing IP traffic as it traverses the Internet, an Extranet or any IP based local, metropolitan or wide area network. IPSec can be incorporated with IPv4 to provide security for host to host, host to subnet and subnet to subnet communications, which are not available with SSL.

[0018] The objective for securing large corporate networks is to allow the proper insiders or outsiders to access corporate data transparently while keeping unintended parties from accessing the same data or denying service to those who should be accessing the data. In the past, Firewalls have been used as a means for filtering incoming and outgoing traffic. Firewalls have been combined with access servers to authenticate parties before they are allowed access to any resource inside or outside the firewall.

[0019] Firewalls have evolved to include new protocols that allow them to safely transfer data between themselves and another party over the Internet. This function is known as creating a virtual private network (a private network over the public Internet).

[0020] The IPSec protocol uses two underlying protocols to send data securely. IPSec adds two additional packet headers to a packet to handle each of the two protocols. The headers both contain a numerical value known as the SPI (security parameters index) to identify the crypto keys and procedures to use with it. The first header, AH (authentication header), provides integrity checking and keying information to keep attackers from computing alternate checksums that check correctly. The second header, ESP, encrypts the contents of the remainder of the packet.

[0021] IPSec supports a number of algorithms for authentication and encryption. Examples are KeyedMD5 and SHA-1 (for AH), DES, Triple DES, and RC4 (for ESP). In addition to this, IPSec automatically handles the creation of security associations between hosts through key management.

[0022] Manual keys can be used which allow hosts to be configured manually with the proper shared secret keys. More common is the use of Simple Key Interchange Protocol (SKIP) which negotiates and exchanges session keys between IPSec hosts. ISKAMP (Internet Security Association and Key Management Protocol) is a general purpose protocol intended to manage security associations and manage key exchanges using Oakley or IKE. Tunneling is also used. In tunnel mode the final destination IP header is encrypted and a gateway IP header is added to allow router to route the packet to the gateway server. In transport mode the IP header is not encrypted.

[0023] IPSec is meant to protect traffic between hosts. However, with the wide range of applications currently in use (email, browsers, file transfer, remote terminal access, multimedia, database and so on) it becomes cumbersome to implement.

[0024] Although IPSec provides automatic protection between hosts, it is not at the discretion of the user as in SSL. Most organizations prefer not to make individual users decide whether traffic should be protected since users do not have sufficient training or applying cryptographic protection.

[0025] IPSec provides an advantage over SSL because it can protect against the newer protocol attacks such as SYN flooding and buffer overruns. In the SYN flooding attack mentioned above IPSec would block illegitimate SYN messages because they require a valid AH with a valid cryptographic checksum. Attackers cannot generate numerous requests from random hosts because they cannot generate a valid AH for every such host. In the buffer overrun attack, the destination host will discard any IP packets which are not properly formatted for IPSec. That is, packets must come from a valid host and be properly formatted for TCP before TCP processes them, thereby protecting the host from this type of attack.

[0026] Although SSL and IPSec can be combined to gain added protection and flexibility, these systems fail to address the problems of increased cost of implementation and scalability. Another major problem not addressed by SSL and IPSec is managing the cost of expanding a network to address increased bandwidth requirements of users.

[0027] The introduction of constantly changing standards and encryption/decryption schemes has greatly increased the burden on computer resources to provide secure communication. The computational demands are currently being addressed with the addition of special encryption/decryption ASICs (Application Specific Integrated Circuits) or hardware. However, as hosts are doubled, a doubling of hardware must be added to meet demands from both source and sink ends.

[0028] A conventional attempt to address the above deficiencies includes the use of hardware to handle encryption and decryption of data traffic. However, this is expensive and slow in that it increases the computational burden on the CPU when encrypting and decrypting data. The hardware approach also has a disadvantage in that it is not scaleable.

[0029] Other problems not addressed by existing security systems are prioritizing security associations between hosts, authenticating host machines and preventing traffic analysis attacks that occur when all users of the public networks converge on a bottleneck. That is, even in tunneled situations, traffic must be directed to a particular gateway which can give rise to a breach in security.

[0030] Alternative software architectures have been tried and discarded. The conventional software point to point client server model cannot scale adequately. Increased demand for secure sessions can lead to system failure as processing resources become unavailable on either side. Employing a central server model has been tried and found inadequate. The central gateway server in a distributed system environment becomes saturated with increased demand for decryption services. Tests were done to compare the performance of a conventional centralized server architecture model, as in FIG. 1, against an invention architecture topology in FIG. 2. These configuration performance tests and results are discussed below.

[0031] Conventional distributed architecture is unable to manage the increase in secure session demand due to instability arising from uneven processor computational loads, propagation delays and computer or network latency, all of which cause loss in synchronicity with collective processors. For these reasons current solutions are inadequate to overcome the barriers mentioned above.

[0032] Therefore, what is needed is a new method for encryption/decryption which is infinitely scaleable in the number of simultaneous sessions capable of being processed by a server.

[0033] What is also needed is an encryption/decryption system which is infinitely scaleable in terms of bandwidth between clients and servers.

[0034] What is also needed is an easily implemented software solution which provides end-to-end encryption/decryption in a distributed network while increasing processing power which eliminates latency as bandwidth increases.

[0035] In accordance with the foregoing and other objectives, an aspect of the invention provides a distributed software solution for encryption/decryption which is infinitely scaleable in the number of simultaneous sessions capable of being processed by a server and in terms of bandwidth between clients and servers. Another aspect of the invention provides end-to-end encryption in a distributed network and combines the processing power of all computers connected to the system to enable bandwidth to be infinitely scaleable and to reduce latency substantially to zero.

[0036] Another aspect of the invention provides a software architecture for encryption/decryption by partitioning the client traffic into units which can be processed across a distributed network of hosts without introducing network instabilities. A further aspect of the invention increases packets per second throughput and overcomes latency. Another aspect of the invention implements a mathematical method ensuring a stable partitioning and processing of encrypted traffic to meet the increase in secure session demand.

[0037] In accordance with another aspect of the invention, the software architecture has three primary components: Manager, Client and Agent. The manager software resides on a gateway server and manages all aspects of controlling the system. Client, server, and agents are created on the manager. The manager controls client access levels. The manager controls security policies on clients, agents and other nodes on the network. Device signatures and certificate information are imported and stored by the manager or optionally generated by the manager. The manager does performance monitoring. The manger performs auditing. Network address translation is handled by the manager for tunneled traffic from the client.

[0038] The client software resides on the desktop of internal hosts, the desktop/laptop of remote users and the desktops or remote offices. The Client software provides a simple GUI interface for clients to configure dial-up information and use either a dial-up connection or a wireline/wireless network connection to the local security server.

[0039] The agent software handles the negotiation of security keys, security associations and establishes the IPSec link between itself and the server. Agent software can run as a stand alone process or exist as part of the client software. The agent software is responsible for encrypting and decrypting communication traffic as it arrives from the clients via the server. The agent software is also responsible for discovering other elements on the network and enforcing security policies on the network elements. All of the agents operate as distributed system to share the load of the encryption and decryption over all of the agent CPUs.

[0056] In accordance with an aspect of invention, the manager or main server, agent and client are all designed to operate transparently within any distributed network which uses an internet protocol (IP). Examples of such distributed networks may employ Ethernet, Token Ring, Synchronous Optical Network (SONET), ATM, Gigabit Ethernet networks, or the like. They will not disrupt network traffic flowing on the host machines or on the wire. The invention uses client server and agent technology to establish end to end or “final mile” security links to the final destination inside the business network.

Manager Initialization

[0057] Referring to FIG. 4, the manager will load the policy file (or read it from the database) upon initialization. The IKE engine will start on the server and receive setting information from the server daemon based on what it has loaded from the policy file. Encryption/decryption settings will be set, integrity checking settings will be set, re-keying settings will be set and access time information will be set on the manager. At this point, the server (and IKE engine) will establish a connection with the agents listed in the policy file and (after establishing a secure session) download security policy rules (security associations, network address translation tables, etc).

Agent Initialization

[0058] Referring to the event diagram in FIG. 4, upon initialization the agent performs an authenticated DH key exchange in order to establish a session key with the Manager. To prevent “man in the middle” attacks, the agent's and server's certificates are attached to the messages exchanged (along with signatures and message digests to verify the devices and certificates and to make sure the message is not modified in transit) for session key negotiation. After a session key is established the manager sends the SA information to the agent and the IKE engines on the manager and agent exchanging keying information.

Client Initialization/Authentication

[0059] Upon initialization the client contacts the gateway server and authenticates using RADIUS, TACACS+, a pre-shared password, device signature or X.509 certificate. Once the client is authenticated, it negotiates the session key with the gateway server. After the session key is established, the client downloads the security policy information from the server (security associations, network address translation tables, etc). With the security policy information established, the client's IPSec engine begins communication with the gateway server and ultimately with the final destination.

CryptoScale

[0060] Cryptoscale is the invention architecture comprised of manager or main server, agents and clients to boost performance to exceptional levels without the use of additional hardware. This technology allows software-based VPN solutions to perform at wire-speed levels. The architecture is based on an asynchronous distributed model but provides critical key synchronization within some components of the architecture. The system consists of separate components, agents that exist as atomic objects with zero wait states, that process data in an arbitrary order and at arbitrary relative speeds. Specific timing considerations are ignored with the exception of re-keying time constraints and IP time-to-live. The entire system is modeled as a finite state machine. Transitions in state are caused by an encryption/decryption computation (a DES CBC block for example) on an agent. There is a synchronizer component in the gateway server, which handles the problems that exist in managing an asynchronous system. FIG. 4 shows the protocol and process sequence, which transpires between the manager, agent(s) and client(s) entities. In the timing model, the exact order can be altered without departing from the scope of the invention. For instance, the FIG. 4 shows that the agent initializes and authenticates before the client initializes and authenticates. This order is not limited to the precise sequence shown in FIG. 4 and some steps can be interchanged without affecting the encryption/decryption aspect of the invention.

Timing Model

[0061] Below is a discussion regarding the events that transpire between the main server, agent(s) and client(s). Only an abbreviated number of the events mentioned are shown in FIG. 4 and only to impart the very basic ideas in establishing secure session environment, initiating secure sessions, transferring secure sessions and terminating secure sessions.

[0062] First the Main Server starts up, wherein a registry is created and initialized and the server begins execution 402. The Agent Server(s) register themselves 405 with the Main Server and define session key(s) with which to establish secure communications. The Main Server and Agent Servers become enabled to receive secure connections from Clients 410 and 415. The Client(s) connects to the Main Server and authenticates using one of several server's known authentication methods 420. The Main Server determines if it can accept a new session based on its current available processor bandwidth. If the Main Server can accept a new session based on available processor resources, then it agrees on a secret session key with the Client(s) and begins the session(s). If the Main Server has insufficient resources to service the session 425, then it will instruct an Agent Server(s) to become unblocked [wake up] and participate in a multiparty key exchange between a Client, Main Server and Agent Server. If the Agent Server has insufficient resources it will notify the server that it cannot accept a new client session or maintain an existing one. If none of the Agent Servers can accept a new client connection then the server can handle the additional load or deny the connection based on configuration settings.

[0063] If the Agent Server loses resources it will request that the Main Server pass the client connection to a new Agent which the Main Server will attempt to do. If the Main Server cannot pas the connection it will either attempt to handle the load itself or notify the client and close the connection. If the Agent Server abruptly shuts down the Main Server will automatically pass the client connection to a new Agent Server or attempt to handle the load itself 435 and 440. The Main Server will notify both Client and Agent Server of the correct cipher to use for the session. The Main Server will notify both Client and Agent Server of any special information such as special ciphers for the different types of communication formats. The Main Server will notify the Client and Agent Server of each other's addresses. The Client and Agent will independently generate a session key to exchange data. The Client will then begin encrypting its session communication to the Agent Server (via the Main Server gateway) using the key and information obtained from the Main Server. The Agent Server will decrypt the session communication and redirect this decrypted communication to the intended final destination. If the Client specifies an end peer to connect to directly, then the Main Server will attempt to redirect the Client to the peer Agent Server after authenticating both parties 450 and 460. The Main Server maintains a list of connections [sessions] and associated session information (session keys, security policies, etc) in a registry wherein a session redirect or special circumstances may require this information to initiate further action. Upon successfully transferring a session to an Agent Server, the Main Server will terminate the thread of an encrypted session communication with the Client but maintain information in the step previous to this one. At this point the Main Server will serve only to synchronize the Client and Agent communication via network address translation 455 and 465. If any Agent Server currently servicing a Clients becomes saturated [overloaded or processor resource insufficient], it notifies the Main Server to pass the session on to another Agent Server. If the Main Server receives notification from an Agent Server that it is saturated then the Main Server finds an alternate available Agent Server and passes the secret key and security association information corresponding to that session to the said alternate Agent Server and securely notifies the impacted Client to redirect itself to the said alternate Agent Server. The Client connects to the alternate Agent Server and a secure session is continued. Upon receiving notification of a successfully redirected session from the said alternate Agent Server, the Main Server will notify the said saturated Agent Server that the said session was successfully redirected. The saturated Agent Server wishing to terminate the session then terminates the session after it receives confirmation from the Main Server that the Client was successfully redirected. If a client or Server signals a session is concluded then a termination message is sent to the Main Server which then registers the session in the registry as terminated 475 and 480.

An overall system algorithm proceeds as follows:

• [0064] 1. The client authenticates to the main server using device signature or some other form of authentication
• [0065] 2. The server gets the client information including the bandwidth requirements to determine how many agents to assign to the client
• [0066] 3. The server sends the client a NAT (Network Address Translation) table, which is used to forward IP packets from host to another host. The NAT table contains the actual IP addresses of assigned agents
• [0067] 4. The server sends the agents a NAT table update to add the client to their existing client list
• [0068] 5. The client and agent perform an authenticated Diffie Hellman exchange (using device signatures or some other form of authentication) in order to negotiate the session key and proper security association
• [0069] 6. The client begins the encryption process
  • [0070] Packet 1 is tunneled to agent 1
  • [0071] Packet 2 is tunneled to agent 2
  • [0072] Packet 3 . . .
  • [0073] :
  • [0074] Packet N is tunneled to agent 1 by some mutually established order e.g., round robin.
    Individual packet network layer model decomposition is shown in Figure FIG. 5A for two clients outgoing secure packets. The packets are shown broken down by network layer model components and show an aspect of packet processing at the various locations. The tunneling packets traveling from the client 5A10 each contain DATA 5A20, TCP 5A22, AGENT IP HEADER 5A24, ESP 5A26, AH 5A28, and GATEWAY HEADER+AGENT HEADER 5A30 components. These two packets are sent across the Internet 5A80 to the Gateway 5A40. The Gateway then strips the GATEWAY HEADER+AGENT HEADER 5A30 and pre appends an AGENT ID IP HEADER 5A60 for a packet composed of output packet 5A50. The encrypted stream (forward direction) is more generally, partitioned among N agents. A basic distributed network topology is shown in FIG. 6 Cryptoscale Configuration. FIG. 6 depicts a simple distributed topology of the invention architecture. 610, 620, and 630 show potential sources for clients needing secure communication service. These all contain an agent IP table or list of available agent's IP addresses assigned to that particular client. Each client table may have a different list of agents based on client needs and server resource scheduling. These can then employ IPsec tunnel mode through Gateway server at 640, which contains the NAT, Network Address Translation, table and master IP tables and then on to assigned agents at 650, where decryption is accomplished in parallel fashion.
• [0075] 7. Agent N decrypts the packet that was sent to it and either sends it up the stack (reads) clear text data or forwards clear text data to final destination. The decryption process itself is closely tied to the cipher being used. This invention employs the established standard ciphers e.g., DES and RC4, and decryption of each packet would depend on the type of encryption standard employed. The packet processing is shown in FIG. 5B. Again, the agent processed packets are broken down into boxes to show the individual network model layer in each packet. Note that the AGENT IP HEADER, ESP, AH and AGENT ID IP HEADER layers have been stripped off by the agent. The remaining DATA 5B10, TCP 5B20, are pre-appended a DESTINATION IP HEADER 5B30, and then forwarded to the final destination host 5B40, in FIG. 5B for reconstitution of individual packets.

[0076] N Agents working in parallel on an encrypted stream are shown in FIG. 7 in a more general form. This figure shows the mode where the application data is broken into M discrete packets and sent through encryption XOR chain. These packets are encrypted separately by the low level process 710. The figure shows 8 such packets for an example. The packets traverse routers and spread as they follow the shortest path to the agents 720. More diffusion occurs as packets continue on their journey 730 Packets from each application part reach their assigned agent and are decrypted individually 740 shows the packets destined for the same device begin to draw closer and packet concentration occurs 760 as packets get closer to the destination. Decrypted packets reach the destination and are reassembled 770

[0077] These general stages above can be further broken down into more detail. FIG. 8 through FIG. 12 contain flow charts, which depict the process aspect of the invention in more depth. The flow begins in the upper left hand corner 810 of FIG. 8 and continues to 1250 in FIG. 12. The process is discussed directly below.

Distributed Encryption Environment Main Process

[0078] If the main server 810 starts successfully, it proceeds to authenticate an agent and begin a registration 814. This part of the process is continued in FIG. 9. Upon finishing 814 and returning, the main sever checks CPU load to decide 816 whether or not to offload work to an agent. Affirmative leads to 828, where the server tries to contact an agent. If no contact is established, then the Administrator is alerted 850 and the server tries to contact other agents 855. That is accomplished at 830 by invoking the agent methods sub-process, which is continued in FIG. 11. If the agent method fails to invoke, then the connection is closed 836, the server is notified and the server tries to contact another agent 828. If the agent method is successful, a client redirection 832 is attempted and success will lead to a client connection being passed from the server to the agent and a freeing of that server thread 834. A negative will initiate closing of connection and server notification 836. Once a connection is passed and server thread freed 834, then a secure communication begins 838. In the event of an error, a security violation is logged 840 and the connection is closed. If the server decides not to offload the demand for service to an agent 816, then the server waits for a client connection 820. If the client connects, then 822 a client thread is created on the server. The server checks to see if it has sufficient CPU resources 816 and begins 824 a client authentication process, which continues in FIG. 10. If the client authentication process is successful, then 82 secure communication begins.

Authentication and Registration Sub-process

[0079] The sub-process begins by attempting a 905 multiparty key agreement, MKA. A failure will prompt a security violation log and connection termination 915. A success will force the process to proceed to 910 initiation of a secure message communication. This leads to 925 client/agent supplying authentication credentials such as a device signature, password or PKI. Device signatures are hashes calculated from existing hardware identification information such as mac address, volume id, cd/dvd rom id, ide/scsi controller id, cpu id, ram size, etc. This information is sent to the server for authentication. Any failure in the previous two steps will be logged and a security violation flagged, followed by connection termination 915. A successful client/agent authentication 925 leads to a registration event on the main server 930. The main server will register the device signature of the agent to use for subsequent authentication.

Client Sub-process

[0080] The client sub-process begins by attempting to connect to the main server 1015. If this is successful then 814 client authentication sub-process is initiated as shown in FIG. 9. A failure to authenticate will prompt a failure log and 1025 server notification. A successful client 814 authentication will lead to 1030 information passing from client to server. If information passing was unsuccessful, then a failure is logged 1035 and the attempt retried. If information passing 1030 was successful, then the client requests an update from server 1040. An update failure is logged and a retry attempt is made 1045. If successful, the client receives a redirection request from the server 1050. If redirection is accepted 1055, then the client attempts to redirect to agent. If this is successful, then 1070 secure communication begins with the agent. In the event that redirection 1050 request failed, then secure communication begins with the server.

Invoking Agent Methods Connection Sub-process

[0081] Successfully invoking the agent authentication process start a pass connection check 1110, refer to FIG. 12, followed by the agent performing a CPU load check 1115 to decide if it can accept a new client connection. If the agent cannot accept more work, then it notifies the main server 1120 and if that fails, then it logs a failure and retires 1125. If a load check indicates sufficient resources are available, then the agent waits for a connection 1130. An agent connection 1140 with the server spawns a server thread 1145 on the agent. A failure to create the server thread will prompt a security violation log entry and sub-process termination 1165. A successful server thread creation initiates a multiparty key agreement, MKA, with the agent. If this is successful, and then a client authentication sub-process 814 begins, see FIG. 9. Success will cause the server to 1155 redirect the client to agent and further to 1160 spawn a client thread on the agent. Any failures in the previous four steps will prompt a security violation logging, sever notification and sub-process termination.

[0082] The main server may make requests to the agents to invoke additional methods such as discovering agents or other devices that the agent can communicate with and report the discovery information back to the main server. The main server may (upon receiving the feedback) direct the agent invoke additional methods such as security policy deployment, enforcement, device quarantine, or data gathering.

Agent Network Discovery

[0083] After the main server has authenticated the agent 930 the main server may instruct the agent to discover other agents 830, 410 and nodes on the network to help the main server determine which nodes should participate in the encryption/decryption process based on proximity. Many network management systems rely on ping for discovery but there is tremendous overhead in using this process to discover nodes. For example, a typical ping confirmation of a host (4 pings) can take 4 seconds. A ping confirmation of 100,000 nodes can take 400,000 seconds, 111 hours or 4.6 days. Some solutions can ping multiple hosts at the same time to reduce the overhead, but even an improvement of a factor of 10 would still take 11 hours or half a day. A better approach would be to leverage the agent's ability to utilize the address resolution protocol (ARP) which is much more efficient than ping to discover nodes. Also, it allows the agents to discover nodes that don't respond to pings like firewalls and other security devices. The agent leverages ARP as a means of discovery by:

• • [0084] Reading ARP tables from the local host
  • [0085] Retrieving ARP tables from gateways/routers using SNMP or CLI

[0086] This means of discovery is at least 100 times more efficient than ping alone and can significantly reduce the time of discovering other agents or other systems. Once the nodes have been discovered then the agent will attempt to connect to the node to determine if another agent is present on the node or if the right security policy is enforced on the node (see below).

Agent Security Policy Enforcement

[0087] The main server may also invoke agent methods 830 to deploy and enforce security policies on the host that the agent is running on and on hosts that the agent has discovered. The agent can communicate with the local host and hosts that it has discovered using XML, SNMP, CLI, TL1 or a pre defined protocol for communicating with the host. Some of the necessary adapters needed to communicate with the local hosts are embedded in agents. The agents can also load new adapters dynamically that are sent to them from the main server. This allows the agents to configure and check security policies on multi-vendor routers, firewalls (personal and corporate), intrusion detection systems, anti-virus, certificate authorities, RADIUS, LDAP servers and any other security endpoint.

Process for Passing Connections

[0088] Invocation of this sub-process will prompt the agent to check its CPU load and decide if it should pass existing client 1210 off. An affirmative will lead the agent 1215 to notify the main server. Failures are logged followed by a retry 1220. Success allows the server and agent to exchange connection passing information 1225 followed by the 1235 server sending the client redirection information. A further success prompts the client to attempt to redirect 1245. If the client redirection is successful, then the agent thread is freed. Any failures in the previous three steps will initiate security violation logs and sub-process termination 1230.

Parallel Processing and Reconstitution

[0089] One of the novel features of this architecture is that the client partitions encrypted bandwidth which is assigned to individual agents or multiplexed to process and decrypt. This is a new and non-obvious application of distributed algorithm technology as applied to encryption and decryption and greatly enhances scalability. This process implementation is based on automata mathematics, as is well understood. The formalized mathematics can be found in Nancy A. Lynch's, “Distributed Algorithms”, Morgan Kaufmann Publishers, 1996. The implementation of the theory to this application is described directly below.

[0090] The model is an asynchronous concurrent system. An Input/Output automaton is defined as a simple type of state machine in which the transitions are associated with named actions. Actions are classified as input output or internal. I/O automata are combined using a repetitive composition operation to form an N level automaton, which represents the concurrent system. The input for this system operating in the forward direction is an encrypted IP packet. The internal action is the decryption of the packet. The output is the unencrypted packet. The internal function of decryption occurs on each agent is visible only to that particular agent. There is another tuple of [input, process, agent response,] for controlling the agent. An example is [add new client IP to table (server message to agent), agent adds IP entry, agent responds with “client added” message]. The input for this system is the server control message, the internal action is the processing (the agent makes a decision based on the control message) of that message. The output is the response. An in-depth explanation of the implementation of the processing algorithm in a distributed network using automata follows directly below.

[0091] Process automaton Pi, see FIG. 3, represents an agent in the system. Configuration and synchronization of process Pi occurs via send and receive messages send (m)ij and receive(m)j,i.

[0092] Listed below are the definitions of the five components of the automaton. Each automata (agent) contains these five components, which fully describe the automata. Since the automata are identical in their external and internal characteristics and behavior, it is possible to compose them into a automaton that, once described in terms of the following five components, completely describes each automata within the system.
The first component is the signature. The signature S of the system consists of the set of three disjoint actions, input, process, output; in(S) represents the input set, proc(S) represents the process set and out(S) represents the output set. External actions consist of the union of in(S)∪out(S) and local actions consist of the union of out(S)∪proc(S). The union of the in(S)∪proc(S)∪out(S) represents all actions of the signature S. The external signature (also known as the external interface) of S, extsig (S) is defined to be the signature (in(S),out(S),0).
sig(A), a signature
The second component defines the set of possible states of sig(A). Each action may or may not result in a change of state for the automaton. Changes may be simple such as loading a byte onto a stack for processing or more complex such as a DES-CBC computation where the passing of a cleartext byte (octet to be more precise) to the cipher results in 16 subsequent XOR operations and at least 16 state changes of that cleartext byte.
states (A), a finite set of states
The third component defines the set of initial state/s of the automaton. The start states are the nonempty elements of states (A) set that exist when the system is initialized
Start (A), a nonempty subset of states (A) known as the start or initial states
The fourth component defines the set of changes in state as the system becomes active.
trans (A), a state transition relation, trans(A) states(A) x acts(sig(A)) x states(A)
The final component defines the set of equivalent actions that occur within an automata. These are commonly called threads.
task (A), a task partition
Transitions
(s, π, s′) is an element of trans(A). π represents an action the causes a change in state. Since an agent requires an input packet to change state, we say that state s is considered quiescent. This means that an agent can and will accept additional input packets even if it has not processed all of the previous packets. It will simply use a vector table function, which will grow and shrink as needed or a fixed sized array to store packets until they are processed (limited to the amount of available memory on the processor to create the vector or array). Unexpected packet input (corrupt packets, malicious packets or unsecured packets) result in silent destruction of the packets and a security violation flag set for the server to see when it polls the agent.
There are three tasks (or threads) that occur on the agent. The first is receiving control commands from the server. The second is sending and receiving messages to/from the server. The final task is participating in the encryption and decryption of packets. Each task runs in a separate thread.
The process I/O automaton is shown below. Here we focus on the packet processing tuple in the forward direction. For simplicity, this assumes that the client has already been authenticated and the agent is ready to begin processing packets.
V is a fixed value set vεV (v is a ciphertext packet).
Signature (Actions):
Input:

[0093] init (v)_i, vεV; Initialization occurs at the moment the packet arrives at the agent. The system initializes with a null value (before a packet arrives) but this case is thrown out since it gives no information about the systems characteristics or behavior.

[0094] receive (v)_j,i, vεV, 1≦j≦n, j≠i; The agent puts the received packet from the server (j) onto the internal stack to await processing. The restrictions that are placed on this action are that at least one sender must exist and that sender cannot be the agent itself

Internal:

[0095] decide (v)₁, vεV; The agent must decide what to do with the packet. It can decrypt it or discard it.

Output:

[0096] send(v)_1,k, vεV; vεV, 1≦k≦n, k≠j The agent sends the packet to one or more (in the case of multicast groups) final destinations (one of which may be itself). The restrictions are that there must be at least one receiver and it cannot be the sender.

[0097] discard(v)₁, vεV; The agent discards the packet

States and Start States:

[0098] packetval, a vector that is indexed by {1, . . . , n) which represents the current states of the packets in V. All of the packets are initialized in V as properly encrypted or corrupt packets. Next, one by one, each packet exists in V as decrypted or null based on the processing decision of the agent. Here is an example showing the various states of two bytes. The first byte is a valid encrypted byte. The second represents a corrupt byte.

[0099]

	V(inital)	V(input)	V(internal)	V(output)
	Null	11110011	11101111	11101111
	Null	11110000	00000011	null

Transitions:

[0100] The transitions for the above states are as follows:

For the valid byte:

[0101] init (v)₁, vεV→receive(v)_j,i, vεV→val (j):=v→decide(v)₁, vεV (val (j) cannot be null)→v′=f(val(j))→send(v′)_1,k

For the invalid byte:

[0102] init(v)₁, vεV→receive(v)_j,1, vεV→val(j):=v→decide(v)₁, vεV (val(j) cannot be null)→v′=f(val(j))→val(j):=null→discard(v′)₁

Tasks:

[0103] The two tasks that are operating as separate threads are send(v)_1,k and discard(v)_i

[0104] There are no further state changes to val(j) when these tasks are executed (until the next round).

Sample successful execution on an automata (Agent)

• [0105] [ ]represents the packet buffer on the agent
• [0106] ffddeeaabbcc represents the ciphertext block (also works for stream input)
• [0107] hello world! represents the plaintext block
• [0108] λ denotes the empty sequence
• [0109] [λ], receive(ffddeeaabbcc)_1,j,[ffddeeaabbcc],decide(ffddeeaabbcc)_1,j,[hello world!], send[hello world!]_1,k, [λ] . . .
  Sample unsuccessful execution on an automata (Agent)
• [0110] [λ], receive(ffddeeaabbcc)_1,j,[ffddeeaabbcc],decide(ffddeeaabbcc)_1,j,[null], discard[null]_1,k, [λ] . . .

[0111] The synchronization process is a necessary part of the automaton or complete system. The agent is a process running on a distributed system. Each client is a process running in the system. The gateway server is in the middle. FIG. 6 shows these host topology relationships. In order to make the single-to-many connection (1 client, N agents) which is the implementation of the bandwidth “slicing” technique, portions of the bandwidth are equally divided among the N agents for processing.

[0112] Since the server is responsible for authenticating and connecting the parties, it serves as the synchronizer. The agent and client processes have to be synchronized or else they cannot communicate. Encrypting and decrypting can be accomplished without this but it creates a bottleneck at the gateway. This is why current secure network solutions are using hardware to help mitigate that bottleneck. None of those existing hardware solutions deal with high volume traffic by any other means but by adding more hardware into the system. Thus, this software solution overcomes the need for addition encryption/decryption hardware.

[0113] The system architecture maintains certain conditions or restrictions, which must exist to enable optimal composition of automata. The true value in this system is seen when automata are composed. The fact that each automaton exhibits identical characteristics and behavior makes it possible for their I/O executions to be composed thereby increasing the number of operations that can be performed in the same time slice. This conclusion is based on Cartesian multiplication and implemented in this architecture. According to Lynch, reference cited above, there are three restrictions that must be met in order for this type of composition to be valid. These necessary conditions are designed to exist in the system for the wire-speed performance levels.

[0114] First, there must be a total independence among the agents. If one or more agents rely on each other for data transfer and one of them went offline, then the other(s) would stop working. In other words, with respect to the internal actions of any two automata in the system, one cannot affect the other in any way. A violation of this condition would introduce chaos into the system via the avalanche effect if agent after agent began dropping out of the system on the account of one failure. This is shown pictorially in 1320 of FIG. 13.

[0115] Second, the output of one agent cannot become the input of one or more other agents because to do so would introduce feedback into the system which will bring the entire system (and probably the network) to a halt in a short period of time. It will be possible for one agent to forward data to another after it's been decrypted since in that case the receiving agent is treated as an end host instead of an agent. This is pictorially depicted at 1320.

[0116] Thirdly, it is required that a finite number of agents exist in the system. Since infinity is a large number, the practical limitation of this condition is small and at least one agent can be generated to satisfy the zero condition. This is pictorially depicted at 1330. FIG. 14 shows the overall relationship between automata and the automaton.

[0117] The foregoing advantages of the present invention were proved in laboratory performance tests, which show the effectiveness of the present invention as compared to conventional architectures. The test configurations are shown in FIG. 1 and FIG. 2. FIG. 1 shows the Test Case 1, where a configuration employed a gateway server, which decrypted traffic before forwarding to the final destinations. In this configuration client1 110 and client2 120 opened secure communication with destination client1 125 and destination client2 130. Decryption was accomplished on the Main Server 120 and then decrypted packets were sent to final destinations. The average CPU usage, system load and packets per second are shown below for over 100 test runs.

DET Test Case 1

Server Running on Main Server

1 Agent Running on Main Server

3 NT Clients/1 98 Client

[0118]

Date/Time/System	CPU	System Load	Packets/sec
01/11/98 13:54–14:04/SUN 1	99.96	1.28	1133

[0119] FIG. 2 shows The Test Case 2 configuration employed the invention architecture. This configuration performs the network address translation at the main server 220 and hands the decryption work off to the agents running on the final destination hosts 225 and 230. The results given below show a roughly double packet per second throughput performance. The average CPU usage, system load and packets per second are shown below for more than 100 test runs.

DET Test Case 2

Server Running on Main Server

2 Agents Running on NT

3 NT Clients/1 98 Client

[0120]

Date/Time/System	CPU	System Load	Packets/sec
01/11/98 14:53–15:04/SUN 1	78.1	0.808	2206

[0121] While the invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but rather is intended to cover various modifications and equivalent arrangements which are included with the spirit and scope of the following claims.