Data communication service in a trusted execution environment (TEE) at the network edge

[0001] Data communication networks serve their users with data services like media conferencing, machine control, social networking, and content delivery. To support the data services, the data communication networks deploy packet routers in their own network facilities. The packet routers transfer data packets within the data communication network to deliver the data services. For large users like data centers, the data communication networks deploy packet routers across the network edge in the user's own facilities. These large users then connect various devices like application servers, storage drives, and user computers to their on-site packet routers which are integrated into the larger data communication network. For example, an application server in one data center may access data from a storage drive in another data center over the on-site and network packet routers.

[0002] Internet Protocol (IP) is a popular form of packet routing. Multi-Protocol Label Switching (MPLS) is a popular form of data networking that uses IP. MPLS devices wrap IP packets in MPLS wrappers that have routing labels. MPLS routers use the routing labels and the IP addresses to route the MPLS wrapped IP packets. MPLS routers at user facilities are called Customer Edge (CE) servers, and MPLS routers at network facilities are called Provider Edge (PE) servers. User devices are connected to the CE servers, and the CE servers are connected to the PE servers. The PE servers are interconnected to one another. A user may have several sites with CE servers and other user devices like application servers, storage drives, and user computers. The user devices communicate with one another over the on-site CE servers and the network PE servers.

[0003] Data communication networks face numerous security problems like malicious data theft and attacks. Various security mechanisms are used to secure data communication networks like access codes and encryption. Hardware-trust is a security mechanism that uses boot-level control and secret hardware codes. The boot-level control establishes a Trusted Execution Environment (TEE) by strictly controlling the software and data in the hardware from the start of hardware operation. The secret codes are used for hardware authentication by an authentication server that also has a copy of the secret codes. TEEs in different locations may use the authentication server to cross-validate hardware-trust and enhance TEE security. MPLS networks use hardware-trust to significantly improve their data and network security.

[0004] Distributed ledgers handle data transactions like account balances using a blockchain format. A single distributed ledger has multiple geographically-diverse computer nodes that each have their own copy of chain code and their own copy of data blocks. The computer nodes execute the chain code to test and build a consensus on the results of chain code execution. For example, multiple computer nodes each execute chain code to determine a user's new balance after a debit and then build a consensus on the new balance before proceeding. Once a consensus is reached, then the computer nodes each store a new data block in their own blockchain database. The new data block indicates the consensus chain code results. The new data block also has a hash of the previous data block. The node redundancy, consensus requirements, and hash chains make distributed ledgers highly reliable and secure.

[0005] Unfortunately, current data communication networks do not effectively and efficiently use hardware-trust at the network edge. Moreover, current MPLS networks do not effectively and efficiently use distributed ledgers to implement hardware-trust in their PE servers and CE servers.

[0006] A data communication network comprises Provider Edge (PE) circuitry and Customer Edge (CE) circuitry. The PE circuitry boots a trusted PE processor that transfers a hardware-trust hash and receives a hardware-trust certificate. The CE circuitry boots a trusted CE processor that transfers a hardware-trust hash and receives a hardware-trust certificate. The trusted PE circuitry and the trusted CE circuitry exchange and validate the hardware-trust certificates to establish a Trusted Execution Environment (TEE) across the network edge. The trusted PE circuitry and the trusted CE circuitry encrypt and exchange trusted user data in the TEE across the network edge.

[0017] FIG. 1 illustrates data communication network 100 that exchanges trusted user data in a Trusted Execution Environment (TEE) at the network edge. The TEE comprises processors that require hardware authentication and network authorization before interacting with one another. Before interaction in the TEE, the processors first get their hardware identity authenticated and have their network access authorized. The processors in the TEE refuse to interact with other processors that cannot prove their hardware authentication and network authorization.

[0018] On FIG. 1, the TEE is indicated by heavy dotted lines that depict the hardware-trusted processors inside the TEE. The lighter solid lines depict open processors that are outside of the TEE. The open processors do not conform to the TEE but may still use some security measures. The arrows at the bottom of FIG. 1 indicate user facilities and network facilities. The points between the user facilities and the network facilities delineate the network edges. The TEE extends across the network edges from the network facilities into the user facilities. The TEE also interconnects the network edges on different ends of data communication network 100.

[0019] The TEE isolates different groups of users like the different customers of a data center. Along with hardware-trust, the TEE uses unique user IDs and uses unique encryption keys for each unique user ID. The TEE uniquely encrypts the user data for each user ID to isolate the different user groups. The trusted user data for a given user ID remains encrypted in the TEE unless needed by a trusted processor for routing, and some of the trusted data always remains encrypted in the TEE. In some examples, the TEE exchanges the trusted user data using Multi-Protocol Label Switching (MPLS) or Software-Defined Wide Area Network (SD-WAN). The TEE marks the MPLS or SD-WAN packets that transport the encrypted and trusted data with a TEE Identifier (ID) and user ID. The TEE ID and the user ID may be Inter-Planetary File System (IPFS) names.

[0020] Data communication network 100 comprises user processors 101-104, Provider Edge (PE) processors 111-114, Customer Edge (CE) processors 121-124, application server (SRV) processors 131-134, memory (MEM) processors 141-144, and hardware-trust controller 150. User processors 101-104 are linked to CE processors 121-124, server processors 131-134, and memory processors 141-144. CE processors 121-124 are linked to user processors 101-104, server processors 131-134, memory processors 141-144, and PE processors 111-114. PE processors 111-114 are linked to one another and to hardware-trust controller 150. The number of processors has been restricted for clarity on FIG. 1, and data communication network 100 typically has many additional processors.

[0021] Processors 101-104, 111-114, 121-124, 131-134, 141-144 comprise circuitry like Central Processing Units (CPUs), Graphical Processing Units (GPUs), Application-Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), drive controllers, and/or the like. For example, memory processor 141 may be a controller in a flash drive, and CE processor 121 may be a CPU in a packet router. The processors execute software like operating systems, virtual components, network applications, and user applications. Trusted processors 101-102, 111-112, 121-122, 131-132, and 141-142 each have a unique read-only hardware-trust code. The hardware-trust codes are associated with TEE IDs and user IDs in hardware-trust controller 150.

[0022] Hardware-trust controller 150 comprises a computer system with processors, memory, transceivers, bus circuitry, and software. The software includes hardware drivers, operating systems, virtual components, and network applications. Hardware-trust controller 150 stores hardware-trust codes, TEE IDs, user IDs, and encryption keys for trusted processors 101-102, 111-112, 121-122, 131-132, and 141-142. Hardware-trust controller 150 validates the hardware-trust codes to authenticate the processors. Hardware-trust controller 150 identifies the TEE ID, user IDs, and encryption keys for the trusted processors to authorize network access.

[0023] Hardware-trust controller 150 encrypts and transfers the TEE ID, user IDs, and encryption keys to the trusted processors. Hardware-trust controller 150 does not expose the hardware-trust codes externally. Hardware-trust controller 150 does not expose the encryption keys beyond the trusted processors in the associated TEE ID and user ID that use the encryption keys. Hardware-trust controller 150 digitally signs hardware-trust certificates with a private key and transfers the hardware-trust certificates to the trusted processors. The trusted processors use a public key for hardware-trust controller 150 to decode and verify the digitally signed hardware-trust certificates. The hardware-trust certificates typically have a Time-To-Live (TTL) and must be refreshed—possibly on-demand.

[0024] In operation, trusted PE processors 111-112 power-up (boot) and responsively implement hardware-trust. Trusted PE processors 111-112 interact with hardware-trust controller 150 to obtain their hardware-trust certificates, TEE ID, user IDs, and encryption keys. Trusted PE processors 111-112 exchange and verify the hardware-trust certificates to establish the TEE. Trusted PE processors 111-112 may then encrypt and exchange trusted data within the TEE using the TEE ID, user IDs, and encryption keys.

[0025] Trusted PE processors 111-112 trigger the boot of open PE processors 113-114 and trusted CE processors 121-122. Trusted CE processors 121-122 boot and responsively implement hardware-trust. Trusted CE processors 121-122 trigger the boot of open CE processors 123-124. Trusted CE processors 121-122 interact with hardware-trust controller 150 over open processors 123-124 and 113-114 to obtain their hardware-trust certificates, TEE ID, user IDs, and encryption keys. Trusted PE processor 111 and trusted CE processor 121 exchange and verify their hardware-trust certificates to extend the TEE across the network edge. Trusted PE processor 112 and trusted CE processor 122 exchange and verify their hardware-trust certificates to extend the TEE across the other network edge. Trusted PE processors 111-112 and trusted CE processors 121-122 may then encrypt and exchange trusted data within the TEE using the TEE ID, user IDs, and encryption keys.

[0026] Trusted server processors 131-132 boot and responsively implement hardware-trust. Trusted sever processors 131-132 trigger the boot of open server processors 133-134. Trusted server processors 131-132 interact with hardware-trust controller 150 over open processors 133-134, 123-124, and 113-114 to obtain their hardware-trust certificates, TEE ID, user IDs, and encryption keys. Trusted server processor 131 and trusted CE processor 121 exchange and verify the hardware-trust certificates to extend the TEE to the application servers. Trusted server processor 132 and trusted CE processor 122 exchange and verify their hardware-trust certificates to extend the TEE to the application servers. Trusted server processors 131-132, trusted CE processors 121-122, and trusted PE processors 111-112 may then encrypt and exchange trusted data within the TEE using the TEE ID, user IDs, and encryption keys.

[0027] Trusted memory processors 141-142 boot and responsively implement hardware-trust. Trusted memory processors 141-142 trigger the boot of open memory processors 143-144. Trusted memory processors 141-142 interact with hardware-trust controller 150 over open processors 143-144, 123-124, and 113-114 to obtain their hardware-trust certificates, TEE ID, user IDs, and encryption keys. Trusted memory processor 141 and trusted CE processor 121 exchange and verify their hardware-trust certificates to extend the TEE to data storage. Trusted memory processor 142 and trusted CE processor 122 exchange and verify their hardware-trust certificates to extend the TEE to data storage. Trusted memory processors 141-142, trusted server processors 131-132, trusted CE processors 121-122, and trusted PE processors 111-112 may then encrypt and exchange trusted data within the TEE using the TEE ID, user IDs, and encryption keys.

[0028] Trusted user processors 101-102 implement hardware-trust. Trusted user processors 101-102 interact with hardware-trust controller 150 over open processors 123-124 and 113-114 to obtain their hardware-trust certificates, TEE ID, user IDs, and encryption keys. Trusted user processors 101 exchange and verify hardware-trust certificates with trusted CE processor 121, trusted server processor 131, and trusted memory processor 141 to extend the TEE to user devices (computers, robots, phones, data appliances, and the like). Trusted user processors 102 exchange and verify hardware-trust certificates with trusted CE processor 122, trusted server processor 132, and trusted memory processor 142 to extend the TEE to the user devices. Trusted user processors 101-102, trusted memory processors 141-142, trusted server processors 131-132, trusted CE processors 121-122, and trusted PE processors 111-112 may encrypt and exchange trusted data within the TEE using the TEE ID, user IDs, and encryption keys.

[0029] User devices that have user processors 101-104 may communicate over the open portion of data communication network 100. User devices that have user processors 101-104 may access open application servers, open data storage systems, and hardware-trust controller 150 using the open portion of data communication network 100.

[0030] The trusted user devices that have trusted processors 101-102 may communicate over the TEE portion of data communication network 100. The trusted user devices may access trusted application servers and trusted data storage systems in the TEE portion of data communication network 100. Advantageously, data communication network 100 extends the TEE from end-user to end-user across large geographic areas with secure access to data storage and application servers.

[0031] Within the TEE, fresh TEE IDs, user IDs, and encryption keys are distributed to the trusted processors periodically or on-demand. A processor that fails to use fresh hardware-trust certificates, TEE IDs, user IDs, and encryption keys is excluded from the TEE. For example, trusted CE processor 121 would not serve trusted memory processor 141 if memory processor 141 sends an invalid hardware-trust certificate or obsolete TEE ID.

[0032] New trusted processors may be added to the TEE by loading hardware-trust controller 150 with their hardware-trust codes, TEE ID, user IDs, and encryption keys. Old trusted processors may be removed by deleting their hardware-trust codes, TEE ID, user IDs, and encryption keys while also refreshing the TEE ID, user IDs, and encryption keys for the remaining trusted processors. In some examples, hardware-trust controller 150 comprises a distributed ledger that requires peer consensus for a new processor before admitting the new processor into the TEE.

[0033] The TEE IDs are unique within data communication network 100. In some examples, data communication network 100 is comprised of multiple sub-networks that have their own network domains. The TEE can readily span these different sub-network domains. In some examples, a distributed file system generates unique TEE IDs across the different sub-network domains. For example, an InterPlanetary File System (IPFS) may be used to generate the TEE IDs and user IDs.

[0034] Advantageously, data communication network 100 effectively and efficiently uses hardware-trust at the network edge.

[0035] FIG. 2 illustrates PE circuitry 105 and PE circuitry 106 that exchange trusted user data in the TEE between different network edges. PE circuitry 105-106 reside in network facilities. PE circuitry 105 comprises trusted PE processor 111 and open PE processor 113. Trusted PE processor 111 is linked to CE circuitry 115, open PE processor 113, trusted PE processor 112, and hardware-trust controller 150. Open PE processor 113 is linked to CE circuitry 115, trusted PE processor 111, open PE processor 114, and hardware-trust controller 150.

[0036] Open PE processor 113 comprises open PE OS 212, open PE application 214, and open data. Trusted PE processor 111 comprises trusted PE OS 211, trusted PE application 213, trusted data, and a secret hardware-trust code (HWT #). The hardware-trust code uniquely identifies trusted PE processor 111 and is physically embedded within in a read-only memory in processor 111. Trusted PE processor 111 executes trusted PE OS 211 and trusted PE application 213 in response to boot. Trusted PE application 213 is configured to hash the hardware-trust code of trusted PE processor 111, communicate with hardware-trust controller 150, exchange and validate hardware-trust certificates, and isolate/encrypt trusted user data based on TEE ID and user ID.

[0037] Initially, PE circuitry 105 boots trusted PE processor 111. Trusted PE processor 111 responsively executes trusted PE OS 211 and trusted PE application 213. Under the control of trusted PE application 213, PE circuitry 105 boots open PE processor 113 which responsively executes open PE OS 212 and open PE application 214. Open PE application 214 then exchanges open user data for linked processors. Trusted PE application 213 may direct PE circuitry 105 to stop open PE processor 113. Thus, trusted PE application 213 and processor 111 control the ability of open PE processor 113 to serve other processors.

[0038] PE circuitry 106 comprises trusted PE processor 112 and open PE processor 114. Trusted PE processor 112 is linked to CE circuitry 116, open PE processor 114, trusted PE processor 111, and hardware-trust controller 150. Open PE processor 114 is linked to CE circuitry 116, trusted PE processor 112, open CE processor 123, and hardware-trust controller 150.

[0039] Open PE processor 114 comprises open PE OS 252, open PE application 254, and open data. Trusted PE processor 112 comprises trusted PE OS 251, trusted PE application 252, trusted data, and a secret hardware-trust code. The hardware-trust code uniquely identifies trusted PE processor 112 and is physically embedded within in a read-only memory in processor 112. Trusted PE processor 112 executes trusted PE OS 251 and trusted PE application 213 in response to boot. Trusted PE application 213 is configured to hash the hardware-trust code of trusted PE processor 112, communicate with hardware-trust controller 150, exchange and validate hardware-trust certificates, and isolate/encrypt trusted user data based on TEE ID and user ID.

[0040] PE circuitry 106 boots trusted PE processor 112. Trusted PE processor 112 responsively executes trusted PE OS 251 and trusted PE application 253. Under the control of trusted PE application 253, PE circuitry 106 boots open PE processor 114 which responsively executes open PE OS 252 and open PE application 254. Open PE application 254 then exchanges open user data for linked processors. Trusted PE application 253 may direct PE circuitry 106 to stop open PE processor 114. Thus, trusted PE application 253 and processor 112 control the ability of open PE processor 114 to serve other processors.

[0041] FIG. 3 illustrates the operation of PE circuitry 105-106 to exchange trusted user data in the TEE between different network edges. Trusted PE processor 111 boots and executes trusted PE OS 211 and trusted PE application 213. Trusted PE processor 111 hashes its hardware-trust code with a random number. Trusted PE processor 111 transfers the hash and random number to hardware-trust controller 150. Hardware-trust controller 150 validates hardware-trust for trusted PE processor 111 and determines TEE ID, user IDs and encryption keys for trusted PE processor 111. Hardware-trust controller 150 signs a hardware-trust certificate with a private key. Hardware-trust controller 150 encrypts and transfers the hardware-trust certificate, TEE ID, user IDs and encryption keys to trusted PE processor 111.

[0042] Contemporaneously, trusted PE processor 112 boots and executes trusted PE OS 251 and trusted PE application 253. Trusted PE processor 112 hashes its hardware-trust code with a random number. Trusted PE processor 112 transfers the hash and random number to hardware-trust controller 150. Hardware-trust controller 150 validates hardware-trust for trusted PE processor 112 and determines TEE ID, user IDs and encryption keys for trusted PE processor 112. Hardware-trust controller 150 signs a hardware-trust certificate with a private key. Hardware-trust controller 150 encrypts and transfers the hardware-trust certificate, TEE ID, user IDs and encryption keys to trusted PE processor 112.

[0043] Trusted PE processor 111 and trusted PE processor 112 exchange their hardware-trust certificates. The exchange may occur over open PE processor 113 and open PE processor 114. Trusted PE processor 111 and trusted PE processor 112 validate the hardware-trust certificates by using the public key for hardware-trust controller 150 to decode and read the hardware-trust certificates.

[0044] Trusted PE processor 111 initiates the boot of open PE processor 113. Trusted PE processor 112 initiates the boot of open PE processor 114. CE circuitry 115 and open PE processor 113 exchange open user data. Open PE processor 113 and open PE processor 114 exchange open user data. Open PE processor 114 exchanges open user data with other systems like CE circuitry 116. CE circuitry 115 and trusted PE processor 111 encrypt and exchange trusted data. Trusted PE processor 111 and trusted PE processor 112 encrypt and exchange the trusted data. Trusted PE processor 112 encrypts and exchanges trusted data with other systems like CE circuitry 116. Trusted PE processors 111-112 isolate and encrypt the trusted user data using the specific encryption keys for the TEE ID and user IDs.

[0045] FIG. 4 illustrates PE circuitry 105 and CE circuitry 115 that exchange trusted user data in the TEE at the network edge of data communication network 100. CE circuitry 115 resides in user facilities and PE circuitry 105 resides across the edge in network facilities. CE circuitry 115 comprises trusted CE processor 121 and open CE processor 123. Trusted PE processor 111 is linked to open PE processor 113, trusted CE processor 121, and hardware-trust controller 150. Open PE processor 113 is linked to trusted PE processor 111, open CE processor 123, and hardware-trust controller 150. Trusted CE processor 121 is linked to open CE processor 123 and trusted PE processor 111. Open CE processor 123 is linked to trusted CE processor 121 and open PE processor 113.

[0046] CE circuitry 115 comprises trusted CE processor 121 and open CE processor 123. Open CE processor 123 comprises open CE OS 222, open CE application 224, and open data. Trusted CE processor 121 comprises trusted CE OS 221, trusted CE application 223, trusted data, and a secret hardware-trust code (HWT #). The hardware-trust code uniquely identifies trusted CE processor 121 and is physically embedded within in a read-only memory in processor 121. Trusted CE processor 121 executes trusted CE OS 221 and trusted CE application 223 in response to boot. Trusted CE application 223 is configured to hash the hardware-trust code of trusted CE processor 121, communicate with hardware-trust controller 150, exchange and validate hardware-trust certificates, and isolate/encrypt trusted user data per TEE IDs and user IDs.

[0047] CE circuitry 115 boots trusted CE processor 121. Trusted CE processor 121 responsively executes trusted CE OS 221 and trusted CE application 223. Under the control of trusted CE application 223, CE circuitry 115 boots open CE processor 123 which responsively executes open CE OS 222 and open CE application 224. Open CE application 224 may then exchange open user data for processors. Trusted CE application 223 may direct CE circuitry 115 to stop open CE processor 123. Thus, trusted CE application 223 and processor 121 control the ability of open CE processor 123 to serve other processors.

[0048] FIG. 5 illustrates the operation of PE circuitry 105 and CE circuitry 115 to exchange trusted user data in the TEE at the network edge. Trusted CE processor 121 boots and executes trusted CE OS 221 and trusted CE application 223. Under the control of trusted CE processor 121, open CE processor 123 boots and responsively executes open CE OS 222 and open PE application 224. Open user processors 103 and open CE processor 123 then exchange open user data. Open CE processor 123 and open PE processor 113 exchange open user data. Open PE processor 113 and other systems exchange open user data.

[0049] Trusted CE processor 121 hashes its hardware-trust code with a random number. Trusted CE processor 121 transfers the hash and random number to hardware-trust controller 150 over open CE processor 123 and open PE processor 113. Hardware-trust controller 150 validates hardware-trust and determines TEE ID, user IDs, and encryption keys for trusted CE processor 121. Hardware-trust controller 150 signs a hardware-trust certificate with its private key. Hardware-trust controller 150 encrypts and transfers the hardware-trust certificate, TEE ID, user IDs, and encryption keys to trusted CE processor 121.

[0050] Trusted PE processor 111 and trusted CE processor 121 exchange their hardware-trust certificates over open PE processor 113 and open CE processor 123. Trusted PE processor 111 and trusted CE processor 121 validate the hardware-trust certificates by using a public key for hardware-trust controller 150 to decode and read the hardware-trust certificates. Trusted user processors 101 and trusted CE processor 121 then exchange encrypted and trusted data. Trusted CE processor 121 and trusted PE processor 111 exchange encrypted and trusted data. Trusted PE processors 111 and CE processor 121 isolate/encrypt the trusted user data using the specific keys for the TEE ID and user IDs.

[0051] FIG. 6 illustrates application server (SRV) circuitry 125 and CE circuitry 115 that exchange trusted user data in the TEE at the network edge. Server circuitry 125 and CE circuitry 115 reside in user facilities. Server circuitry 125 comprises trusted server processor 131 and open server processor 133. Trusted server processor 131 is linked to open server processor 133 and trusted CE processor 121. Open server processor 133 is linked to trusted server processor 131 and open CE processor 123. Trusted CE processor 121 is linked to open CE processor 123, trusted server processor 131, and PE circuitry 105. Open CE processor 123 is linked to trusted CE processor 121, open server processor 133, and PE circuitry 105.

[0052] Open server processor 133 comprises open server OS 232, open server application 234, and open data. Trusted server processor 131 comprises trusted server OS 231, trusted server application 233, trusted data, and a secret hardware-trust code (HWT #). The hardware-trust code uniquely identifies trusted server processor 131 and is physically embedded within in a read-only memory in processor 131. Trusted server processor 131 executes trusted server OS 231 and trusted server application 233 in response to boot. Trusted server application 233 is configured to hash the hardware-trust code of trusted server processor 131, communicate with hardware-trust controller 150, exchange and validate hardware-trust certificates, and encrypt trusted user data per TEE ID and user IDs.

[0053] Server circuitry 125 boots trusted server processor 131. Trusted server processor 131 responsively executes trusted server OS 231 and trusted server application 233. Under the control of trusted server application 233, server circuitry 125 boots open server processor 133 which responsively executes open server OS 232 and open server application 234. Open CE application 224 and open server application 234 may then exchange open user data for processors. Trusted server application 233 may direct server circuitry 125 to stop open server processor 133. Thus, trusted server application 233 and processor 131 control the ability of open server processor 133 to serve other processors.

[0054] FIG. 7 illustrates the operation of application server circuitry 125 and CE circuitry 115 to exchange trusted user data in the TEE at the network edge. Trusted server processor 131 boots and executes trusted server OS 231 and trusted server application 233. Under the control of trusted server processor 131, open server processor 133 boots and executes open server OS 232 and open server application 234. Open user processors 103 and open server processor 133 then exchange open user data. Open server processor 133 and open CE processor 123 exchange open user data. Open CE processor 123 and other systems like PE circuitry 105 exchange open user data.

[0055] Trusted server processor 131 hashes its hardware-trust code with a random number. Trusted server processor 131 transfers the hash and random number to hardware-trust controller 150 over open CE processor 123 and open PE processor 113. Hardware-trust controller 150 hashes its copy of the hardware-trust code for trusted server processor 131 with the random number to match results and validate hardware-trust for trusted server processor 131 Hardware-trust controller 150 identifies TEE ID, user IDs, and encryption keys for trusted server processor 131. Hardware-trust controller 150 signs a hardware-trust certificate for trusted server processor 131 with a private key. Hardware-trust controller 150 transfers the digitally-signed hardware-trust certificate, TEE ID, user IDs, and encryption keys to trusted server processor 131.

[0056] Trusted CE processor 121 and trusted server processor 131 exchange their hardware-trust certificates over open CE processor 123 and open sever processor 133. Trusted CE processor 121 and trusted server processor 131 validate the hardware-trust certificates by using a public key for hardware-trust controller 150 to decode and read the hardware-trust certificates. Trusted server processor 131 and trusted user processors 101 may also exchange and validate their hardware-trust certificates over open CE processor 123. Trusted user processors 101 and trusted server processor 131 then exchange encrypted and trusted data. Trusted server processor 131 and trusted CE processor 121 exchange encrypted and trusted data. Trusted server processor 131 and CE processor 121 isolate/encrypt the trusted user data using the specific keys for the TEE ID and user IDs.

[0057] FIG. 8 illustrates memory (MEM) circuitry 135 and CE circuitry 115 that exchange trusted user data in the TEE at the network edge. Memory circuitry 135 and CE circuitry 125 reside in user facilities. Memory circuitry 135 comprises trusted memory processor 141 and open memory processor 143. Trusted memory processor 141 is linked to open memory processor 143 and trusted CE processor 121. Open memory processor 143 is linked to trusted memory processor 141 and open CE processor 123. Trusted CE processor 121 is linked to open CE processor 123, trusted memory processor 141, and PE circuitry 105. Open CE processor 123 is linked to trusted CE processor 121, open memory processor 143, and PE circuitry 105.

[0058] Open memory processor 143 comprises open memory OS 242, open memory application 244, and open data. Trusted memory processor 141 comprises trusted memory OS 241, trusted memory application 243, trusted data, and a secret hardware-trust code (HWT #). The hardware-trust code uniquely identifies trusted memory processor 141 and is physically embedded within in a read-only memory in processor 141. Trusted memory processor 141 executes trusted memory OS 241 and trusted memory application 243 in response to boot. Trusted memory application 243 is configured to hash the hardware-trust code of trusted memory processor 141, communicate with hardware-trust controller 150, exchange and validate hardware-trust certificates, and isolate/encrypt trusted user data per TEE ID and user IDs.

[0059] Memory circuitry 135 boots trusted memory processor 141. Trusted memory processor 141 responsively executes trusted memory OS 241 and trusted memory application 243. Under the control of trusted memory application 243, server circuitry 135 boots open memory processor 143 which responsively executes open memory OS 242 and open memory application 244. Open CE application 224 and open memory application 244 may then exchange open user data for processors. Trusted memory application 243 may direct memory circuitry 135 to stop open memory processor 143. Thus, trusted memory application 243 and processor 141 control the ability of open memory processor 143 to serve other processors.

[0060] FIG. 9 illustrates the operation of memory circuitry 135 and CE circuitry 115 to exchange trusted user data in the TEE at the network edge. Trusted memory processor 141 boots and executes trusted memory OS 241 and trusted memory application 243. Under the control of trusted memory processor 141, open memory processor 143 boots and executes open memory OS 242 and open memory application 244. Open user processors 103 and open memory processor 143 then exchange open user data. Open memory processor 143 and open CE processor 123 exchange open user data. Open CE processor 123 and other systems like PE circuitry 105 exchange open user data.

[0061] Trusted memory processor 141 hashes its hardware-trust code with a random number. Trusted memory processor 141 transfers the hash and random number to hardware-trust controller 150 over open CE processor 123 and open PE processor 113. Hardware-trust controller 150 hashes its copy of the hardware-trust code for trusted memory processor 141 with the random number to match results and validate hardware-trust for trusted memory processor 141 Hardware-trust controller 150 identifies TEE ID, user IDs, and encryption keys for trusted memory processor 141. Hardware-trust controller 150 signs a hardware-trust certificate for trusted memory processor 141 with a private key. Hardware-trust controller 150 transfers the digitally-signed hardware-trust certificate, TEE ID, user IDs, and encryption keys to trusted memory processor 141.

[0062] Trusted CE processor 121 and trusted memory processor 141 exchange their hardware-trust certificates over open CE processor 123 and open memory processor 143. Trusted CE processor 121 and trusted memory processor 141 validate the hardware-trust certificates by using a public key for hardware-trust controller 150 to decode and read the hardware-trust certificates. Trusted memory processor 141 and trusted user processors 101 may also exchange and validate their hardware-trust certificates over open CE processor 123. Trusted user processors 101 and trusted memory processor 141 then exchange encrypted and trusted data. Trusted memory processor 141 and trusted CE processor 121 exchange encrypted and trusted data. Trusted memory processor 141 and CE processor 121 isolate/encrypt the trusted user data using the specific keys for the TEE ID and user IDs.

[0063] FIG. 10 illustrates Multi-Protocol Label Switching (MPLS) communication network 1000 comprising a hardware-trust distributed ledger 1150 to support the exchange of trusted user data in a TEE at the MPLS network edge. MPLS communication network 1000 is an example of data communication network 100, although network 100 may differ. In the user facilities, MPLS communication network 1000 comprises MPLS Customer Edge (CE) servers 1101, application servers 1125, data storage 1135, and user computers 1145. In the network facilities, MPLS communication network 1000 comprises network data centers 1060.

[0064] Network data centers 1060 comprise multiple geographically diverse computer systems that each comprise hardware, hardware drivers, operating systems/hypervisors, virtual layers, and network applications. The network applications include Provider Edge (PE) servers (including PE server 1105) and hardware-trust distributed ledger 1150. Data centers 1060 may comprise Network Function Virtualization Infrastructures (NFVIs), and the network applications may comprise Virtual Network Functions (VNFs). The data center hardware comprises Network Interface Cards (NICs), CPUs, RAM, disk storage, and data switches (SWS). The CPUs each have a unique hardware-trust code (HWT #). The virtual layers comprise virtual NICs (vNIC), virtual CPUs (vCPU), virtual memory (vMEM), and virtual Switches (vSW). The NICs of data centers 1060 are linked to each other and to MPLS CE servers 1115. For clarity, the links between individual data centers 1060 are not shown on FIG. 10. Hardware-trust distributed ledger 1150 comprises clients, endorser nodes, orderer nodes, peer nodes, chain code, and blockchain databases. The chain code has hardware-trust hash algorithms, and the databases store hardware-trust codes, TEE IDs, user IDs, encryption keys, and hardware-trust transaction data.

[0065] In operation, PE server 1105 boots and responsively implements hardware-trust. PE server 1105 hashes the hardware-trust code in its CPU and transfers the hash and random number to the ledger client. The ledger client transfers the proposed transaction (hardware-trust authentication and network authorization) to one or more endorser nodes. The endorser nodes execute their chain code to hash the hardware trust code for PE server 1105. The endorser nodes match their own result with the received hash from PE server 1105 to authenticate PE server 1105. The endorser nodes retrieve TEE ID, user IDs, and encryption keys from the database to authorize network service for PE server 1105. The endorser nodes generate a hardware-trust certificate for PE server 1105. The endorser nodes compare their chain code result (trust certificate, TEE ID, user IDs, and encryption keys) to an endorser rule set, and if the chain code result passes the endorsement rules, then the endorser nodes transfer a transaction endorsement back to the ledger client.

[0066] The ledger client then transfers the endorsed transaction (hardware-trust authentication and network authorization) to one or more orderer nodes. The orderer nodes transfer the endorsed transaction to the peer nodes. The peer nodes execute their chain code to hash the hardware trust code for PE server 1105. The peer nodes match their hash result with the hash result from PE server 1105 to authenticate PE server 1105. The peer nodes retrieve TEE ID, user IDs, and encryption keys for the hardware trust code from the database to authorize PE server 1105. The peer nodes generate a hardware-trust certificate for PE server 1105. The peer nodes share their chain code results (trust certificate, TEE ID, user IDs, and encryption keys) to form a consensus on the result. If a consensus is formed, then the peer nodes record the transaction in their databases in the blockchain format. In the blockchain format, a new data block stores new data for a current transaction, and the new block also stores a hash of the previous data block for the previous transaction. The block hashes chain the data blocks together in a time sequence.

[0067] One or more peer nodes transfer the chain code results (hardware trust certificate, TEE ID, user IDs, and encryption keys) to the ledger client. The ledger client transfers the chain results to PE server 1105. PE server 1105 shares the hardware trust certificate with other trusted devices to form a TEE. PE server 1105 encrypts user data using the encryption keys per the user IDs. PE server 1105 wraps the encrypted user data in an MPLS wrapper that has the TEE ID and user ID. Only other devices that use the same TEE ID and user ID have the keys to decrypt and process the MPLS wrapper. The other PE servers in data centers 1060 also establish hardware trust, encrypt trusted data using keys for the TEE ID and user ID, and route the encrypted and trusted data per the MPLS labels and corresponding network addresses.

[0068] In a like manner, CE servers 1115 boot and responsively implement hardware-trust. CE servers 1115 hash their hardware-trust codes and transfer the hashes to the ledger clients in network data centers 1060. The ledger clients transfer the proposed transactions (hardware-trust authentication and network authorization) to the endorser nodes. The endorser nodes execute the chain code to hash the hardware trust codes for CE servers 1105. The endorser nodes match the results with the hashes from CE servers 1115 to individually authenticate CE servers 1115. The endorser nodes retrieve TEE IDs, user IDs, and encryption keys from the database. The endorser nodes generate hardware-trust certificates for CE servers 1105. The endorser nodes compare their chain code result (trust certificates, TEE IDs, user IDs, and encryption keys) to an endorser rule set, and if the chain code result passes the endorsement rules, then the endorser nodes transfer a transaction endorsement back to the client.

[0069] The client then transfers the endorsed transaction (hardware-trust authentication and network authorization) to one or more orderer nodes. The orderer nodes transfer the endorsed transaction to the peer nodes. The peer nodes execute the chain code to hash the hardware trust codes for CE servers 1115. The peer nodes match their results with the hashes from CE servers 1115 to authenticate CE servers 1115. The peer nodes retrieve TEE IDs, user IDs, and encryption keys for the hardware trust codes from the database. The peer nodes generate hardware-trust certificates for CE servers 1115. The peer nodes share their chain code results (trust certificates, TEE ID, user IDs, and encryption keys) to form an individual consensus on the individual results. When consensus is formed for a given transaction, the peer nodes record the transaction in the database in the blockchain format and transfer the chain code results to the client. The client transfers the chain results to CE servers 1115. CE servers 1115 share their hardware trust certificates with other devices in the TEE like PE server 1105. CE servers 1115 encrypt user data using the specific encryption keys for the TEE ID and user ID. CE servers 1115 wrap the encrypted user data in MPLS wrappers that have the TEE ID appropriate user ID. Only devices that use the same TEE ID and user ID have the proper keys to decrypt and process the MPLS wrapper.

[0070] Application servers 1125, data storage 1135, and user computers 1145 each perform the above hardware-trust procedure to obtain their hardware-trust certificates, TEE IDs, user IDs, and encryption keys. Application servers 1125, data storage 1135, and user computers 1145 each share and verify hardware-trust certificates. Application servers 1125, data storage 1135, and user computers 1145 each use their TEE ID, user IDs, and encryption keys to exchange trusted data in MPLS wrappers.

[0071] For example, a trusted user computer in a first user site may use their hardware-trust certificate to retrieve trusted financial data from a trusted company database in a second user site and process the trusted financial data in trusted application servers in a third user site to generate a secret financial presentation. The trusted user computer may then encrypt and transfer the secret financial presentation to another trusted user computer in a fourth user site. During these operations, the secret financial presentation remains encrypted when outside of the trusted user computers and application servers. The trusted CE servers and PE servers only decrypt the MPLS wrapper using the keys for the TEE ID and user ID to obtain the MPLS routing label and network addressing.

[0072] Within the TEE, fresh hardware-trust certificates, TEE IDs, user IDs, and encryption keys can be distributed to the trusted devices periodically or on-demand. A device that fails to use fresh hardware-trust certificates, TEE IDs, user IDs, and encryption keys is excluded from the TEE. For example, trusted PE server 1105 would not serve one of trusted CE servers 1115 that failed to provide a hardware-trust certificate or that used an obsolete user ID.

[0073] New trusted devices may be added by loading the databases in hardware-trust distributed ledger 1150 with the appropriate hardware-trust codes, TEE ID, user IDs, and encryption keys. Old trusted devices may be removed by deleting their hardware-trust codes, TEE ID, user IDs, and encryption keys from the databases and refreshing the TEE ID, user IDs, and encryption keys for the remaining trusted devices.

[0074] In some examples, at least some of the peer nodes are configured to obtain human consent before joining a consensus when adding a new device and/or user ID to the TEE. For example, the peer nodes may require human consent from personnel that currently use the TEE before admitting a new device or user ID to the TEE. Unanimous consent from all peers may be required. The peer nodes can implement various access rules based on their chain code, databases, and consent requirements.

[0075] The TEE IDs are unique within data communication network 1000. In some examples, data communication network 1000 is comprised of multiple sub-networks that have their own IP addressing and MPLS labeling domains. The TEE can readily span these different sub-networks. In some examples, the unique TEE IDs are generated using a distributed file system like InterPlanetary File System (IPFS). In some examples, unique user IDs are generated using a distributed file system.

[0076] Advantageously, MPLS communication network 1000 effectively and efficiently uses distributed ledger 1150 to implement hardware-trust in PE server 1105, CE servers 1115, application servers 1125, data storage 1135, user computers 1145, and other MPLS devices.

[0077] Alternatively, SD-WAN could be used in a like manner instead of MPLS.

[0078] The wireless data network circuitry described above comprises computer hardware and software that form special-purpose network circuitry to support a TEE at the network edge. The computer hardware comprises processing circuitry like CPUs, DSPs, GPUs, transceivers, bus circuitry, and memory. To form these computer hardware structures, semiconductors like silicon or germanium are positively and negatively doped to form transistors. The doping comprises ions like boron or phosphorus that are embedded within the semiconductor material. The transistors and other electronic structures like capacitors and resistors are arranged and metallically connected within the semiconductor to form devices like logic circuitry and storage registers. The logic circuitry and storage registers are arranged to form larger structures like control units, logic units, and Random-Access Memory (RAM). In turn, the control units, logic units, and RAM are metallically connected to form CPUs, DSPs, GPUs, transceivers, bus circuitry, and memory.

[0079] In the computer hardware, the control units drive data between the RAM and the logic units, and the logic units operate on the data. The control units also drive interactions with external memory like flash drives, disk drives, and the like. The computer hardware executes machine-level software to control and move data by driving machine-level inputs like voltages and currents to the control units, logic units, and RAM. The machine-level software is typically compiled from higher-level software programs. The higher-level software programs comprise operating systems, utilities, user applications, and the like. Both the higher-level software programs and their compiled machine-level software are stored in memory and retrieved for compilation and execution. On power-up, the computer hardware automatically executes physically-embedded machine-level software that drives the compilation and execution of the other computer software components which then assert control. Due to this automated execution, the presence of the higher-level software in memory physically changes the structure of the computer hardware machines into special-purpose network circuitry to support a TEE at the network edge.

[0080] The above description and associated figures teach the best mode of the invention. The following claims specify the scope of the invention. Note that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the invention. Thus, the invention is not limited to the specific embodiments described above, but only by the following claims and their equivalents.