System and method to communicate sensitive information via one or more untrusted intermediate nodes with resilience to disconnected network topology

[0001] The present inventive concept relates generally to computer-based network communication systems, and more particularly, to a system and method to communicate sensitive information via one or more untrusted intermediate nodes with resilience to disconnected network topology.

[0002] It has become increasingly important to stream large amounts of data in a fault-tolerant, cryptographically-secure manner and to enable redistribution of tasks being performed by a plurality of servers among the plurality of servers, particularly to address situations in which one or more of the plurality of servers fail. Conventional technology is unable to meet the demands for such technology for a variety of reasons, which include complications arising from network topology or directionality and an inability to scale existing systems. Conventional technology includes peer-to-peer transfer technology, which involves distributing and sharing digital documents and computer files using a decentralized, distributed network architecture. Although conventional peer-to-peer transfer technology is operable to transfer data across a network in a secure manner, such technology can only accommodate authenticated data and cannot accommodate encrypted data if the data is broadcast, e.g., sent to a large number of users. This shortcoming essentially renders such conventional peer-to-peer transfer technology no more efficient, and often less efficient, than other transfer technology in terms of traffic and reliability. As such, all data sent via a network using such conventional peer-to-peer transfer technology is available for viewing by all peer subsets associated with the network. Thus, a user who wishes to transfer data while maintaining its confidential nature cannot use conventional peer-to-peer transfer technology.

[0003] Thus, there exists a need for system and method to stream large amounts of data in a fault-tolerant manner, at a large scale, and in a cryptographically-secure environment.

[0004] In response to the aforementioned needs, the present application describes a computerized system and method of communicating secure information between a plurality of computers with an untrusted intermediate and resilience to disconnected network topology.

[0005] In accordance with an aspect of the present disclosure, a system and method are provided that utilize agnostic endpoints that are generalized to be interoperable among various systems, with their functionality based on their location in a network. The system and method enable horizontal scaling on the network and provide complete encryption of data throughout a transfer across the network from end to end. Within this disclosure, horizontal scaling generally refers to a relative increase in the quantity of regionally separated nodes that may be accessed due to a proportional increase in the expanse of the network providing communication services to those nodes. One or more clusters may be set up in a location within a network or series of networks in electronic communication, e.g., in a cloud, a sub-network, a demilitarized zone (DMZ), or perimeter network, residing between a secure area of the network and an unsecure area such as an external network or portion of a network. The system is configured to exchange encrypted information between a backend of the system and an agent by utilizing the clusters. The clusters are configured to store encrypted information in a buffered format and authenticate the agent using public key encryption. The agent is installed via an installation process during which the agent is configured and/or authenticated by a provisioning server of a configuring cluster using public certificate technology. During the installation process, the agent is assigned one or more unique identifiers operable to work with the system and known to the clusters. Horizontal scaling allows the system to utilize a full capacity of the system. That is, the ability of the system to communicate with regionally separated nodes is essentially proportional to the extent of the medium (e.g., the network) providing connectivity to each of the nodes. As long as the agent has connectivity to at least one locale of the network, e.g., one of the clusters, the agent can move data across the system.

[0006] The aforementioned may be achieved in one aspect of the present disclosure by providing a computerized method of communicating using a system configured to exchange encrypted data via at least two servers. The method may include the steps of executing an agent in the system configured with (i) at least one internal domain naming service (DNS) address to point to one of the at least two servers, and (ii) at least one external DNS address to point to another one of the at least two servers, establishing a communication link between one of the at least two servers and the agent, signing a payload containing data using a private key to produce an envelope, and/or encrypting the envelope using a public key associated with one of the at least two servers and the agent. The at least two servers may be configured to authenticate the agent when establishing the communication link between the one of the at least two servers and the agent. An internal DNS address generally refers to an address of a trusted node, while an external DNS address refers to an address of an untrusted node (e.g., an intermediary node). Additionally, the term payload refers to useful data exchanged between at least two nodes, which does not necessarily include other data used for formatting or conveying the data using one or more communication protocols.

[0007] The method may further include the step of compressing the data prior to signing the payload. The public key may be provided by the agent. The step of executing the agent may include configuring and authenticating the agent using a certificate. The step of executing the agent may include assigning one or more unique identifiers to the agent. The at least two servers may utilize a root certificate to communicate with agent.

[0008] The agent may be provided with information associated with the at least two servers. The information may include an IP address and DNS information associated with each of the servers. The at least one of the at least two servers may be a provisioning server configured to certify the agent. The authentication of the agent may include use of a CA certificate and/or a certificate enrollment protocol. The authentication of the agent may be a two-way authentication process between the agent and the at least two servers.

[0009] The method may include the step of initiating a poll, via the agent, by generating and transmitting a signal across the system and to the at least two servers in attempt to identify a task. The signal may be sent to only one of the at least two servers at a time. The poll may be transmitted periodically or aperiodically and encrypted across the system at intervals alternating between the at least two servers. If the task is not identified, the agent may be configured to repeat the step of initiating the poll after a predetermined time period has elapsed. The agent may determine that the task has been identified if the agent receives at least one item detailing the task in response to the poll initiated by the agent. If the task is identified, the agent may be configured to (i) locate and retrieve data associated with the task, (ii) decrypt the data, (iii) generate an output associated with the data, and/or (iv) transmit the output. The output generated by the agent may be associated with a data payload or data envelope and may be compressed, signed, and/or encrypted. If the task is not identified, the agent may be configured to repeat the step of initiating the poll at a predetermined rate that is based on whether the system is experiencing a data breach.

[0010] The rate may be reset so that polls are initiated at a different and quicker rate if the system is experiencing the data breach. One of the at least two servers may be configured to reset the polling rate based on whether the system is experiencing the data breach. The agent may be configured to locate the data associated with the task via a URI. The agent may be configured to (i) retrieve the information associated with the task via a secure socket layer (SSL) connection, and/or (ii) decrypt the information associated with the task via at least one of the public key and the private key.

[0011] The aforementioned may be achieved in one embodiment by providing a computerized communication system. The system may include a processor configured to (i) compress data, (ii) sign the data using a private key, and/or (iii) encrypt data using a public key to form a data payload. The system may further include an agent configured to establish a communication link and transmit the data payload using the communication link and/or a server configured to (i) receive the data payload, and (ii) process metadata associated with the data payload to determine whether an entirety of the data payload was transmitted successfully from the processor to the server. The processor may include (i) a compressor or compression logic configured to compress the data, (ii) an authenticator or authentication logic configured to sign the data, and/or (iii) an encryptor or encryption logic configured to encrypt the data.

[0012] The system may further include a buffered memory associated with the server for use to determine whether an entirety of the data payload was received by the server from the processor. The metadata may include offset/length data related to subsets of the data to enable the data to be reconstructed if a transmission event occurs. The metadata may be compressed, signed, and/or encrypted simultaneously with the data by the processor.

[0013] Additional aspects, advantages, and utilities of the present inventive concept will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present inventive concept.

[0014] The foregoing is intended to be illustrative and is not meant in a limiting sense. Many features and subcombinations of the present inventive concept may be made and will be readily evident upon a study of the following specification and accompanying drawings comprising a part thereof. These features and subcombinations may be employed without reference to other features and subcombinations.

[0020] The drawing figures do not limit the present inventive concept to the specific embodiments disclosed and described herein. The drawings are not necessarily to scale, emphasis instead being placed on clearly illustrating principles of certain embodiments of the present inventive concept.

[0021] The following detailed description references the accompanying drawings that illustrate various embodiments of the present inventive concept. The illustrations and description are intended to describe aspects and embodiments of the present inventive concept in sufficient detail to enable those skilled in the art to practice the present inventive concept. Other components can be utilized and changes can be made without departing from the scope of the present inventive concept. The following detailed description is, therefore, not to be taken in a limiting sense. The scope of the present inventive concept is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled.

[0022] I. Terminology

[0023] In the following detailed description, terminology is used to describe features of the present inventive concept. For example, references to terms “one embodiment,” “an embodiment,” or “embodiments” mean that the feature or features being referred to are included in at least one aspect of the present disclosure. Separate references to terms “one embodiment,” “an embodiment,” or “embodiments” in this description do not necessarily refer to the same embodiment and are also not mutually exclusive unless so stated and/or except as will be readily apparent to those skilled in the art from the description. For example, a feature, structure, process, step, action, or the like described in one embodiment may also be included in other embodiments, but is not necessarily included. Thus, the present inventive concept may include a variety of combinations and/or integrations of the embodiments described herein. Additionally, all aspects of the present disclosure as described herein are not essential for its practice.

[0024] Further, in certain situations, the term “logic” refers to hardware, firmware, software, and/or a combination thereof that is configured to perform one or more functions including, but not limited to, those functions of the present inventive concept specifically described herein or are readily apparent to those skilled in the art in view of the description. Such logic may include circuitry having data processing and/or storage functionality. Examples of such circuitry may include, but are not limited to, a microprocessor, one or more processors, e.g., processor cores, a programmable gate array, a microcontroller, an application specific integrated circuit, a wireless receiver, transmitter and/or transceiver circuitry, semiconductor memory, or combinatorial logic.

[0025] Logic may be in the form of one or more software modules, such as executable code in the form of an executable application, an application programming interface (API), a subroutine, a function, a procedure, an applet, a servlet, a routine, source code, object code, a shared library/dynamic load library, or one or more instructions. These software modules may be stored in any type of a suitable non-transitory storage medium, or transitory storage medium, e.g., electrical, optical, acoustical or other form of propagated signals such as carrier waves, infrared signals, or digital signals. Examples of non-transitory storage medium may include, but are not limited or restricted to a programmable circuit; a semiconductor memory; non-persistent storage such as volatile memory (e.g., any type of random access memory “RAM”); persistent storage such as non-volatile memory (e.g., read-only memory “ROM”, power-backed RAM, flash memory, phase-change memory, etc.), a solid-state drive, hard disk drive, an optical disc drive, or a portable memory device. As firmware, the executable code is stored in persistent storage.

[0026] The term “content” generally refers to information transmitted as one or more messages, where each message(s) may be in the form of a packet, a frame, an Asynchronous Transfer Mode “ATM” cell, or any other series of bits having a prescribed format. The content may be received as a data flow, namely a group of related messages, within ingress data traffic. Content may include one or more types of data such as, but not limited to, text, software, images, audio, metadata and/or other digital data. One example of content may include web content, or any data traffic that may be transmitted using a Hypertext Transfer Protocol (HTTP), Hypertext Markup Language (HTML) protocol, or may be transmitted in a manner suitable for display on a Web browser software application. Another example of content includes electronic mail or email, which may be transmitted using an email protocol such as Simple Mail Transfer Protocol (SMTP), Post Office Protocol version 3 (POP3), or Internet Message Access Protocol (IMAP4). A further example of content includes an Instant Message, which may be transmitted using Session Initiation Protocol (SIP) or Extensible Messaging and Presence Protocol (XMPP) for example. Yet another example of content includes one or more files that are transferred using a data transfer protocol such as File Transfer Protocol (FTP) for subsequent storage on a file share.

[0027] The term “cluster” is used to describe a group of machines or servers grouped or clustered together and all performing a similar function and configured to operate together. For example, a cluster may include eight personal computers all connected via a network. A plurality of clusters, e.g., store and forward clusters, may exist in different locations across a system or network. Within each cluster, its servers are assigned and configured to perform one or more different tasks relative to one another. Each of the clusters may include a provisioning server, a polling server, a transmitting server, and/or a receiving server. Alternatively one or more tasks such as those performed by the provisioning server and the polling server may be consolidated into a single server. For instance, a cluster may refer to one or more elements configured to perform the functionalities necessary to function as an intermediary as described herein. Each cluster may in certain embodiments perform independently of one or more other clusters, with no routing necessary between such clusters.

[0028] The terms “task” and “work” are used interchangeably herein to refer to an act of transmitting and/or receiving data by an agent using one or more clusters.

[0029] The term “computerized” generally represents that any corresponding operations are conducted by hardware in combination with software and/or firmware.

[0030] The term “virtual machine” (VM) refers to a simulation of an electronic device that may be abstract or real, and is usually different from the electronic device conducting the simulation. A VM may be based on specifications of a hypothetical computer or emulate the computer architecture and functions of a real world computer. A VM may be one of many different types such as, for example, hardware emulation, full virtualization, para-virtualization, and/or operating system-level virtualization virtual machines.

[0031] Lastly, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.

[0032] As the present inventive concept is susceptible to embodiments of many different forms, it is intended that the present disclosure be considered as an example of the principles of the present inventive concept and not intended to limit the present inventive concept to the specific embodiments shown and described.

[0033] II. General Architecture

[0034] FIG. 1 illustrates an example secure communication system 100 according to aspects and embodiments of the disclosure. The system 100 includes one or more backend nodes 102a and 102b that are in communication with one or more intermediary nodes 104a-104n via a communication network 106. The backend nodes 102a and 102b exist in a secure area 108 that is formed by a firewall 110 that restricts unsolicited access to the backend nodes 102a and 102b. The intermediary nodes 104a-104n, however, exist in an unsecured area 112 in that they are not restricted from unsolicited access by the firewall 110. As will be described in detail below, at least one node 102a, 102b, or 104a-104n includes an agent 116 that functions as a broker to facilitate communication of one or more data envelopes 118 between at least one backend node 102a or 102b and at least one intermediary node 104a-104n in the unsecure area 108.

[0035] The agent 116 facilitates communication of the data envelope 118 from any one of the backend nodes 102a, 102b or intermediate nodes 104a-104n to any other node in the system. For example, the agent 116 may facilitate communication of the data envelope 118 from the backend node 102a to an intermediate node 104a-104n, from one intermediate node 104a-104n to another intermediate node 104a-104n, and/or from an intermediate node 104a-104n to a backend node 102a, 102b. In a more particular example, the agent 116 may facilitate transmission of the data envelope 118 from backend node 102a to backend node 102b by first transmitting the data envelope 118 from backend node 102b to intermediate node 102a, then from intermediate node 104a to intermediate node 104b, and then from intermediate node 104b to backend node 102b.

[0036] As shown, the agent is stored on, and executed by backend node 102a. However, other embodiments contemplate that the agent 116 may be stored and executed by any node (e.g., backend node or intermediary node) in the system. The backend node 102a comprises a provisioning node that executes the agent 116 and provisions the agent 116 to perform its functions as described herein below. Additionally, although only one agent 116 is shown, other embodiments contemplate that the secure communication system 100 include multiple agents 116 that simultaneously arbitrate and broker communication of data envelopes 118 among the multiple nodes in the system.

[0037] The data envelope 118 generally includes a block of data that has been signed by a private key of the recipient node to receive the data envelope 118. The block of data may include any type of data. For example, the block of data may include big data that is terabytes or event exabytes in size.

[0038] Each backend node 102a, 102b, and intermediate node 104a-104n may include one or more servers, personal computers, mobile computers and/or other mobile devices, and other computing devices. The backend nodes 102a, 102b and/or intermediate nodes 104a-104n may also be referred to as endpoints or computer clusters. For example, one of the backend nodes 102a, 102b and/or intermediate nodes 104a-104n comprises a computer cluster or other type of computing environment having multiple computing devices that function together to perform one or more computing services. The nodes may communicate via wireless, wireline, and/or optical communications. The backend nodes 102a, 102b, and intermediate nodes 104a-104n each have one or more processors and executable instructions stored in volatile and/or non-volatile memory for performing the actions and/or steps described herein.

[0039] The communication network 106 can be the Internet, an intranet, or another wired and/or wireless communication network. In one embodiment, the backend nodes 102a, 102b and intermediate nodes 104a-104n communicate with one another using any suitable protocol or messaging scheme. For example, the backend nodes 102a, 102b and intermediate nodes 104a-104n communicates using a Hypertext Transfer Protocol (HTTP), extensible markup language (XML), extensible hypertext markup language (XHTML), or a Wireless Application Protocol (WAP) protocol. Nevertheless, it should be understood that the backend nodes 102a, 102b and intermediate nodes 104a-104nmay communicate with each other using any suitable networking protocol.

[0040] FIG. 2 illustrates an example backend node 102a or 102b or intermediate node 104a-104n according to one embodiment of the secure communication system 100. The backend node 102a or 102b/intermediate node 104a-104n is a computing or processing device that includes one or more processors 202 and one or more memory units 204. For example, the backend node 102a or 102b/intermediate node 104a-104n can be a computer cluster, a personal computer, such as a laptop or notebook computer, a workstation, or other processing device such as a personal digital assistant or a tablet computer. The backend node 102a or 102b/intermediate node 104a-104n may include a display 206, such as a computer monitor, for displaying data using a graphical user interface 148. The backend node 102a or 102b/intermediate node 104a-104n may also include an input device 208, such as a keyboard or a pointing device (e.g., a mouse, trackball, pen, or touch screen) to enter data into or interact with a graphical user interface 210. The backend node 102a or 102b/intermediate node 104a-104n receives data and/or communications from, and/or transmit data and/or communications to, other backend nodes 102a, 102b/intermediate nodes 104a-104n via the communication network 106.

[0041] The backend node 102a or 102b/intermediate node 104a-104n may include an agent 116 stored in the memory unit 204 and executed on the processors 202 to facilitate transmission of data envelopes from one backend node 102a or 102b/intermediate node 104a-104n to another backend node 102a or 102b/intermediate node 104a-104n. The memory unit 204 also stores the public keys 120, private keys 122, and DNS addresses 124 for each backend node 102a or 102b/intermediate node 104a-104n in the secure communication system 100.

[0042] The memory 204 also stores public keys 212, private keys 214, and domain naming service (DNS) keys 216 for each node used in the system. In one embodiment, the agent 116 performs a discovery process to find nodes in the system, and upon discovery of the nodes, performs an authentication process with each node to generate the public keys 212 and private keys 214, which are subsequently stored in the memory 204.

[0043] FIG. 3 illustrates an example process that may be performed for setting up operation of the secure communication system 100 according to the teachings of the present disclosure. At step 302, the system 100 is initialized by the provisioning node (e.g., backend node 102a) to generate credentials. In this manner, the system 100 is bootstrapped and a certificate authority, e.g., Trust Domain Certificate Authority (TDCA), and certificates are generated. At step 204, the provisioning node assigns certificates (e.g., public keys, and private keys) to each node in the system 100.

[0044] At step 306, the provisioning node installs the agent 116 in the system. That is, the provisioning node loads the agent 116 into a memory 204 of at least one backend node 102a or 102b and/or intermediate node 104a-104n initiates execution of the agent. The provisioning node may also upload the agent with a backend TDCA (i.e., a root certificate) and/or IP/DNS addresses for the backend nodes 102a, 102b and intermediate nodes 104a-104n. The agent 116 is in communication with each of the backend nodes 102a, 102b and intermediate nodes 104a-104n.

[0045] In one embodiment, the provisioning node performs a tasking process, which manages a communication sequence where a data envelope (i.e., signed block of data) is sent from the one node to another node in the system 100. The task may be created outside the backend to cause a data envelope 118 to be sent to the agent 116. In this manner, data may be presented to a backend node 102a, 102b in the secure area 108.

[0046] The provisioning node is configured to respond to a certificate signing request (CSR) by signing the request and injecting into the certificate an agent identifier for the agent 116, that is, the agent identifier is associated with the certificate. The agent identifier may be a base-64 encoded unique identification (UID) case sensitive binary large object or like identifier, e.g., a base-62 encoded universally unique identification version 4 (UUID4). Then, the provisioning node injects a well-defined object identifier (OID) that is registered into the certificate. A new unique identification number or UID is assigned for a predetermined period of time, e.g., a lifetime or lifespan, of the agent 116. In this manner, subtleties related to how a certificate hash, i.e., a mathematical function to convert or abbreviate large data to small data, is calculated are reduced or avoided, and renewal of certificates while maintaining identity over an extended period of time is enhanced. As a certificate is generated and returned to the agent 116, the certificate is pushed to the backend nodes using a subscription mechanism. This enables the system 100 to maintain a mapping of or track an agent identification certificate or a set of certificates.

[0047] In one embodiment, the agent 116 is preconfigured with certain assumptions, knowledge and/or backend logic. For example, the agent 116 may be configured with logic that the intermediate nodes 104a-104n are available and that independent communication between the agent 116 and each of the intermediate nodes 104a-104n is possible. Also, the agent 116 may be configured with logic that at least one communication route or path exists to each one of the intermediate nodes 104a-104n. At least one path between the agent 116 and each of the intermediate nodes 104a-104n may be used for broadcasting data to each of the intermediate nodes 104a-104n upon demand. When data is introduced (e.g., rolled out) to the intermediate nodes 104a-104n of the system 100, it is desirable to broadcast the data to as many of the intermediate nodes 104a-104n as possible to ensure the data remains accessible to the agent 116. When the data is distributed or retrieved across the network 10, the agent 116 is configured to logically route one or more of the intermediate nodes 104a-104n with one or more specific pieces of data. It is foreseen that new ones of the intermediate nodes 104a-104n may be added to the system 100 without updating communication paths by adding information regarding such new intermediate nodes 104a-104n to the backend logic.

[0048] In one embodiment, the agent 116 is configured with one or more DNS addresses to enable hopping of the data envelope 118 between one or more of the intermediate nodes 104a-104n (“DNS hopping”) to reach its destination. The DNS addresses configured in the agent 116 may be determined according to any suitable DNS routing protocol or other DNS accessing technique. For instance, the agent 116 may be configured with an internal DNS address to point to backend node 102a or 102b and/or with an external DNS address to point to an intermediate node 104a-104n. For purposes herein, “internal DNS” refers to one or more Domain Name Service servers behind the firewall 110 in the secure area 108, and “external DNS” refers to one or more Domain Name Service servers in front of the firewall 110 in the unsecured area 112. Such a DNS protocol overcomes a deficiency in conventional systems wherein, among other issues, an agent must be reconfigured every time a new server and/or cluster is rolled out or made accessible to the agent. A backend node 102a, 102b includes instructions regarding how to access a number of real DNS addresses to avoid illicit redirection of the agent 116. Upon obtaining a certificate generated on the backend and authenticating such, the DNS record is changed.

[0049] Upon execution of the aforementioned initializing steps 302, 304, and 306, the system 100 is initialized and ready for use in accordance with the present inventive concept. The initializing steps 302, 304, and 306 need not be repeated unless a user of the system 100 desires to update or reconfigure the system 100.

[0050] At step 308, the system 100 is turned on or booted. Upon booting of the system 100, the agent 116 communicates with the provisioning node using a protocol to encrypt information for transmission over the network, such as the Internet, at Step 310. The protocol used for transmission may be a Secure Sockets Layer (SSL) protocol, a simple certificate enrollment protocol (SCEP), and/or any other suitable communication protocol. The agent 116 acquires and is assigned to a certificate, such as a PKI certificate, at step 312. The certificate assigned to the agent 116 is signed by a trusted domain, such as an Agent Trust Domain, which is controlled by the system 100.

[0051] At this point in the process, the agent 116 has the certificate, and the system 100 is configured to form a secure connection (SSL connection) to one or more of the intermediate nodes 104a-104n based on a list of the intermediate nodes 104a-104n, at step 314. The list of the intermediate nodes 104a-104n may be formed and installed in the agent 116 during any one or more of the initializing steps 302, 304, and 306 of the system 100. The list may include a ranking of the intermediate nodes 104a-104n by priority based on pathway or location within the system 100. For instance, one of the intermediate nodes 104a-104n within a local area network (LAN) or within the secure area 108 may be listed first on the list with a highest priority while other ones of the intermediate nodes 104a-104n outside of the LAN or within the unsecure area 108 may be listed thereafter. The list of the intermediate nodes 104a-104n includes a variety of information associated with each of the intermediate nodes 104a-104n including, but not limited to, communication information such as IP address, DNS information, and the like.

[0052] Each of the intermediate nodes 104a-104n is configured to attempt to authenticate the certificate of the agent 116 by confirming that the certificate of the agent 116 is signed by the Agent TDCA at step 316. If the certificate of the agent 116 has been authenticated, the agent 116 is configured to initiate a process to poll for a new task or new work to be performed by the agent 116, at step 318. The agent 116 will continue to poll for work at predetermined frequencies, as discussed hereafter, until the system 100 is turned off and/or (re)booted, at step 130. If the certificate of the agent 116 is not authenticated, the system 100 will continue to attempt to authenticate the certificate of the agent until, for example, step 308 is repeated, that is, the system 100 is (re)booted.

[0053] FIG. 4 illustrates an example process that may be performed by the system 100 to poll for a new task or new work according to one embodiment of the present disclosure. The agent 116 initiates the poll for work by generating and transmitting a signal across the system 100 in attempt to identify new work, at step 400. For instance, the agent 116 may be configured to open a hypertext transfer protocol secure (HTTPS) connection to each of the intermediate nodes 104a-104n, and poll the intermediate nodes one at a time or simultaneously for work. The polls are performed periodically or aperiodically and are each encrypted, such as in an encryption envelope. Feedback to the poll for work transmitted by the agent 116 is typically received by the agent 116 within seconds, irrespective of whether new work exists or does not exist. At step 402, if no new work exists, processing continues at step 404; otherwise processing continues as step 406 in which the agent 116 is presented with at least one item detailing the new work or a list of new work. The at least one item may be a URI where data associated with the new work may be located via one or more of the intermediate nodes 104a-104n. It is important to note that, with each poll, the agent 116 determines whether the work has already been initiated (by another agent or previously by the agent 116), and whether the work has been marked as aborted. If the work has not been aborted, the agent 116 could be asked to initiate an abort. The agent 116 is configured to regularly check to identify if work has been marked aborted so that any connection associated therewith can be terminated, thereby forcing an immediate poll of the data.

[0054] At step 404, the agent identifies whether a threat or breach in the security of the data occurs. If not, processing continues at step 408 in which the slow polling technique as described above is initiated. However, if a threat or breach in the security of the data has been identified, processing continues at step 410 in which a fast-poll process is implemented.

[0055] In one embodiment, the fast-poll process is implemented whereby a straight HTTP (non-secure) request is issued from the agent 116 to at least one of the backend nodes 102a, 102b and/or intermediate nodes 104a-104n. In this process, an agent 116 presents an AgentID and polls to determine the presence of any work. Unlike that described above, the fast-poll process involves untrusted communication and may not be resilient to compromise. It is, however, convenient in that the speed and efficiency of the polling process may be enhanced. If the agent 116 acquires work following the fast-poll process, the work is obtained via authentic communications and/or a full or complete polling process (a pull for work) according to certain embodiments. It is foreseen that, despite its disadvantages with respect to process security, the fast-poll process may be utilized in relatively low-threat environments, such as when a present threat or breach has not been identified in the system 100. In certain embodiments, the fast-poll process will be performed more frequently than the more secure polling process of other embodiments described herein. The rate of polling directly affects how often data is transmitted across the network; that is, a faster polling rate generally provides an increased rate of data envelope transfer while a lower polling rate generally provides a decreased rate of data envelope transfer. A specific rate of polling may be determined by one or more of the backend nodes 102a, 102b and/or intermediate nodes 104a-104n, which may be, for example, every few seconds. In this manner, a full “handshake” is required less frequently when using fast-polling when the fast-poll process is executed at moderate intervals. For example, a full handshake may be performed approximately every thirty minutes when fast-polling is conducted every one minute. It should be noted that, even if fast-polling is employed, normal and more secure polling by the agent 116 is still performed on a regular basis in some embodiments.

[0056] After processing either of steps 408 or 410, the agent repeats step 400 to poll for work again after a specified period of time elapses.

[0057] The agent 116 performs a search for data associated with the new work by searching each of the intermediate nodes 104a-104n within the system 100 based on the at least one item or the list, at step 406. The agent 116 does not open a connection to any of the intermediate nodes 104a-104n unless one of the intermediate nodes 104a-104n reaches or possesses a specified threshold amount of data to be transmitted. In this manner, a trickle effect is prevented in which low volume data transmissions cause undue congestion, thereby minimizing adverse effects caused due to faults and heightening fault tolerance of the system 100. The search for new work continues until the agent 116 locates data to be transmitted by one of the intermediate nodes 104a-104n, at which point the agent 116 attempts to retrieve the data, at step 412. In one embodiment, the agent 116 only attempts to retrieve data associated with the new work when the agent 116 has sufficient capacity to perform the new work. Additionally, the agent 116 will retrieve the data via an SSL connection or the like and decrypt it with a key and/or code.

[0058] Additionally at step 406, the agent 116 authenticates each certificate presented by each one of the intermediate nodes 104a-104n where data resides and is accessed by the agent 116, such as during an SSL authentication process or handshake, by ensuring that each of the certificates is signed by the backend configuration node. During a handshake, each one of the intermediate nodes 104a-104n where data resides and is accessed by the agent 116, authenticates the agent 116, thereby performing two-way or mutual authentication and ensuring security of the data. In some cases, performing the handshake can be complex and time consuming. Each one of the intermediate nodes 104a-104n contains agent authentication information associated with the agent 116, and in some cases, the intermediate nodes 104a -104n may require a root certificate to communicate with the agent 116. The system 100 is configured to use a variation of cache credentials, which is an SSL feature referred to herein as “client caching.” In a client caching process, the system 100, during an initial handshake, is configured to generate a symmetric key that is known or possessed by both the agent 116 and one or more of the intermediate nodes 104a-104n which are to transmit or receive data via the agent 116. In one embodiment, as part of the handshake, one of the intermediate nodes 104a-104n is configured to return a doubly encrypted binary large object (BLOB) containing symmetric key and resume information, which is retained and presented by the agent 116 during subsequent sign on. In this manner, the binary large object is transmitted from the agent 116 to the one of the intermediate nodes 104a-104n where data resides and is accessed by the agent 116. If the one of the intermediate nodes 104a-104n is able to decrypt the binary large object, the SSL handshake process is short circuited and previously negotiated terms are used.

[0059] It is foreseen that each of the intermediate nodes 104a-104n may include a server configured to perform as the provisioning node, that is, operable to configure the agent 116. Such authentication of the agent 116 may occur using a certified authority (CA) certificate, a SCEP, or the like to initiate or sign the agent 116. The nodes configured to perform as the provisioning node are each configured to issue a specific certificate associated with the agent 116 and also separately and independently issue an agent ID (e.g.,. an OID) that is inserted into the certificate. Thus, the system 100 may utilize two unique identifiers (e.g., a certificate and an agent identifier) so that the certificate can be updated without loss of identification information of the agent 116.

[0060] The agent 116, using code or other instructions, decrypts the data using a private key, at step 414. As part of the decryption process, the agent 116 validates a signature presented by the private key, for example, using the backend configuration node. If the agent 116 determines the signature presented by the private key is valid, the agent 116 checks a certificate revocation list (CR) and, if the certificate is not revoked, parses the data and begins to generate output based on the data in the form of a data payload, at step 416. The data payload is a grouping or collection of data, e.g., related data in a packet or file, that includes an agent certificate and is signed using a private key for later authentication, which results in a signed envelope via, for example, PKCS7 (CMS). The signed data envelope is encrypted using a public key from the agent 116 at a predetermined destination of the data envelope. It should be noted that a cipher is a symmetric cipher having a key that is encrypted using a public key of the agent's certificate. A corresponding private key is not included in the encryption envelope of embodiments of the present inventive concept.

[0061] The output is transmitted by the agent 116 at step 410, which may be performed via one or more streams of data. For example, a backend node 102a or 102b is configured to broadcast the encrypted envelope to one the intermediate nodes 104a-104n. The transmission of the data by the agent 116 may be cryptographically opaque to the intermediate nodes 104a-104n. The agent 116 is equipped with a base uniform resource identifier (URI) (e.g., as part of a binary large object (BLOB)) or in an envelope. The agent 116 may output the data via a write to the base URI, which is an address mechanism. In this manner, each of the intermediate nodes 104a-104n is able to transmit data to a receiving node using a base URI associated with that node. As part of the data output of step 410, the agent 116 is configured to compress the data, sign the data, and encrypt the data, which may be performed by the agent 116 in this order quickly, on the fly, and with little notice.

[0062] The output data is encrypted along with a manifest 120 (e.g., an administrative record) that describes the work performed either partially or completely, and may include metadata associated with the data. The metadata may include a list, cluster identification information, and/or offset/length data associated with each piece of the data payload. The manifest may be used to determine completeness of the data in a buffered memory at each of the intermediate nodes 104a-104n, which may contain metadata and other data. The manifest may also be used to reconstruct or “sew” together pieces of the data in case of any communication interruption or other undesirable event that occurs during transfer of the data. Thus, in preferred embodiments the manifest contains a sufficient amount of metadata to piece together various pieces of the data payload. The manifest is generated and coupled to the work before the agent 116 outputs the data to the intermediate nodes 104a-104n, but after the agent 116 has completed all work associated with the task, e.g., at the generate output step 416.

[0063] Other new tasks may be present and/or identified or identifiable by the agent 116, when the agent 116 generates the manifest. In one embodiment, the manifest may be generated in JavaScript Object Notation (JSON) using a manifest URI. The manifest URI may include a HTTP address to which a manifest could be stored using, for example, a HTTP PUT request. The manifest URI is separate from the base URI previously described. A successful PUT to the HTTP address is interpreted by at least one of the intermediate nodes 104a-104n as a notification of completion in which the notification may be relayed to the backend. The manifest document may be compressed, signed, and encrypted as is the data such that its content is opaque to any of the intermediate nodes 104a-104n utilized by the system 100.

[0064] Using the manifest and the buffered memory at each of the intermediate nodes 104a-104n, the agent 116 is configured to determine completeness of data stored on the intermediate nodes 104a-104n and complete the data, for example, by identifying and moving to a point in the data envelope, and transmitting data from the point in the data envelope until the entire data envelope has been transmitted. For instance, if a connection between one or more of the intermediate nodes 104a-104n and the agent 116 is prematurely terminated, disconnected or dropped, the system 100 may utilize the manifest in attempt to correct any issues associated with such.

[0065] Dropping can occur at any point as the agent 116 performs the aforementioned processes. There are no ideal breaking points in data given that a significant amount of processing may have already been performed. For instance, prior to outputting the data at step 410, the agent 116 has already compressed the data, signed the data, and encrypted the data. If the connection is dropped, it is quite possible that at least a subset of the data may not have arrived at the receiving backend node 102a or 102b or intermediate node 104a-104n. For heightened fault tolerance, the system 100 places the data in a buffer 218 (e.g., a circular buffer) as the data is streamed. Thus, if the connection is dropped by one of the intermediate nodes 104a-104n, the agent 116 is configured to reconnect to the one of the intermediate nodes 104a-104n, e.g., during one or more processes such as but not limited to a HEAD process. A HTTP HEAD request is one of a plurality of defined actions, which supplements an underlying transport protocol (HTTP/1.1) used by the system 100. The HEAD request is used by the system 100 to determine how many “bytes” of a stream were successfully received by the receiving backend node 102a or 102b or intermediate node 104a-104n in case of a disconnect as the data is streamed. In other words, the HEAD request enables the agent 116 to determine how much data successfully arrived at the one or more of the intermediate nodes 104a-104n and to, in reliance on the buffer 218, rewind and start from where it should be based on the determination. In this manner, the system 100 may minimize adverse effects caused by faults, thereby heightening fault tolerance.

[0066] In some instances, overlap of data may exist, which is likely to occur if a connection has been dropped, which may be detected as discussed herein below. If there is such an overlap, decryption of the data may be difficult to accomplish. The system 100 is configured to minimize any overlap by utilizing the buffer 218. With respect to a size of the buffer 218, it may be at least as large as a TCP window, and in some cases set to at least double the size of the TCP window. The buffer size is set via a buffer size setting operation that occurs simultaneously with or immediately after a TCP handshake operation. The buffer size setting operation causes the size of the TCP window to be determined using TCP window size determining algorithm. For instance, if the size of the TCP window is determined to be 64 kb, the algorithm may be applied to determine that a buffer size that is at least 128 kb may be used.

[0067] If the agent 116 cannot perform one or more of the aforementioned functions, the agent may conclude that the task has failed and in turn, attempt to regenerate the data. If the agent 116 cannot regenerate the data, the agent 116 is configured to generate a failure alert to the upstream node. If the agent 116 cannot communicate to the upstream node (e.g., if a node such as a portable laptop has moved from one network to another), the agent 116 is configured to rewind the buffer 218 entirely, that is, rewind to a beginning or starting point of the data transmission, and re-transmit the data to the receiving node.

[0068] The system 100 of the present inventive concept may be configured to achieve network agnosticism. For example, if the agent 116 is actively transmitting data to one of the intermediate nodes 104a-104n while a connection therebetween is terminated and the agent 116 is unable to reconnect to the one of the intermediate nodes 104a-104n, the agent 116 may fall back to the top of the list of the intermediate nodes 104a-104n and will attempt to connect to a next one of the intermediate nodes 104a-104n. Once the agent 116 is able to connect to one of the intermediate nodes 104a-104n, the agent 116 will continue to push data to the one intermediate node 104a-104n.

[0069] The system 100 may cause data for a task to be distributed across multiple intermediate nodes 104a-104n. If the agent 116 is unable to communicate with one of the intermediate nodes 104a-104n, the agent 116 may pause the data transfer and repeatedly attempt to reconnect to the one intermediate node 104a-104n. The system 100 is configured to continue this process until an external instruction is issued, such as an instruction received from a controller of the system 100. If the instruction is not received and the agent 116 is unable to reconnect to the one intermediate node 104a-104n, the system 100 will be in a failed state during the task. Any work in process and being performed by the system 100 does not survive reboot and will need to be restarted in some embodiments. Any work not completed by the agent 116 will still be active and/or associated with and identifiable via one or more of the intermediate nodes 104a-104n, so that the work not completed may be restarted and successfully completed by the agent 116.

[0070] The system 100 may be configured to avoid issues related to virtual machine (VM) duplication. VM duplication typically involves a process of a virtualized environment in which a hypervisor duplicates or clones another VM having most or all of the same credentials of the VM from which it was created. Nevertheless, this behavior may create a security problem in that each VM in the system is no longer uniquely identifiable from a data integrity point of view. In such a scenario, the agent 116, having been previously provisioned by the provisioning node, includes a unique certificate and key to an existing VM, but not to another instance of the existing VM that has been duplicated. To address such, the agent 116 obtains a sequence ID (SEQID) upon initial check-in by targeting an algorithmic solution using a nonce approach. A nonce is an arbitrary number used only once in a cryptographic communication. The agent 116 is configured to present the SEQID every time the agent 116 signs in. The intermediate nodes 104a-104n are configured to record and track the last and next-to-last nonce. If any of the intermediate nodes 104a-104n identify any request that is out-of-sequence, that intermediate node 104a-104n requests that the agent 116 poll more frequently and test for failures with the VM. Using this technique, the duplicate VM will fail, at which point the intermediate nodes 104a-104n may then label the duplicate VM as a clone and instruct the agent 116 to re-provision and obtain a new identifier for the duplicate VM.

[0071] In the foregoing description, the present inventive concept is described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present inventive concept as set forth in the appended claims.

[0072] This description of the present inventive concept is provided to enable any person skilled in the art to make or use the present inventive concept. Various modifications will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied alternatively without departing from the spirit or scope of the present inventive concept. Thus, the present inventive concept is not intended to be limited to the description herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0073] The steps of a method, system, or operation described in connection with the present inventive concept disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0074] Having now described the features, discoveries, and principles of the present disclosure, the manner in which embodiment of the present disclosure are constructed and used, the characteristics of the construction, and advantageous, new and useful results obtained; the new and useful structures, devices, elements, arrangements, parts and combinations, are set forth in the appended claims.

[0075] The following claims are intended to cover all of the generic and specific features of the present disclosure herein described, and all statements of the scope of the present inventive concept, which, as a matter of language, might be said to fall there between.