Assessing subjective trust

[0001] The disclosure relates to computer networks and to trust levels attributed to entities in a network.

[0002] Devices or network entities in a network are typically relied upon to perform various operations on behalf of users of the network. Some network entities are more reliable than others. Some network entities are more secure than others. Zero-trust networking is an approach to network security in which network administrators assume that network entities should not be trusted by default, even if those network entities are in a private network or previously verified. Example types of network entities may include network nodes (e.g., endpoint devices, intermediate network devices, etc.), network services, or other types of real or virtual entities that can be identified on a computer network. Because network entities are not trusted by default in zero-trust networking, identity and integrity checks are routinely performed on network entities.

[0003] This disclosure describes techniques that include assessing trust for devices (i.e., network entities) in a system using subjective information. In general, subjective information can, in at least some situations, help provide a more holistic assessment of a network or network entity's trustworthiness, and can help to ensure that decision-making aligns with the organization's priorities.

[0004] As described herein, aspects of trust assessment may be customized for different stakeholders (e.g., users, operators, organizations). Since different stakeholders may have different priorities and values, it may be appropriate to customize trust assessments to reflect these differences and provide, for different stakeholders, corresponding different views of the trust assessment for a network entity. For example, one stakeholder might prioritize the reputation of the manufacturer of a network entity, while another might prioritize the level of innovation and investment in research and development of the manufacturer of the network entity.

[0005] To customize trust assessments for different stakeholders, trust factors can be assigned a weight that reflects the importance of a given factor to a relevant stakeholder. For example, for a stakeholder that values sustainability, higher weights may be applied to factors pertaining to sustainability. In some examples, such weights may be derived from information received from stakeholders and used to determine the needs and priorities of stakeholders sharing a common role. Different trust scores may therefore be presented to stakeholders depending on preferences of the individual stakeholder, the identity of the stakeholder, and/or the stakeholder's role.

[0006] In some examples, this disclosure describes operations performed by a computing system in accordance with one or more aspects of this disclosure. In one specific example, this disclosure describes a method comprising determining, by the computing system, a prerequisite score for a network entity in a computer network, wherein the prerequisite score is based on one or more prerequisite factors, and wherein each of the one or more prerequisite factors is a condition to be satisfied; determining, by the computing system, a subjective score for the network entity based on one or more subjective factors for the network entity, wherein each of the one or more subjective factors for the network entity corresponds to a subjective characteristic of the network entity; determining, by the computing system and based on the prerequisite score and the subjective score, a trust score for the network entity; and modifying, by the computing system, operation of the computer network based on the trust score for the network entity.

[0007] In another example, this disclosure describes a system comprising processing circuitry and a storage device, wherein the processing circuitry has access to the storage device and is configured to: determine a prerequisite score for a network entity in a computer network, wherein the prerequisite score is based on one or more prerequisite factors, and wherein each of the one or more prerequisite factors is a condition to be satisfied; determine a subjective score for the network entity based on one or more subjective factors for the network entity, wherein each of the one or more subjective factors for the network entity corresponds to a subjective characteristic of the network entity; determine, based on the prerequisite score and the subjective score, a trust score for the network entity; and modify operation of the computer network based on the trust score for the network entity.

[0008] In another example, this disclosure describes a computer-readable medium comprising instructions that, when executed, configure processing circuitry of a computing system to: determine a prerequisite score for a network entity in a computer network, wherein the prerequisite score is based on one or more prerequisite factors, and wherein each of the one or more prerequisite factors is a condition to be satisfied; determine a subjective score for the network entity based on one or more subjective factors for the network entity, wherein each of the one or more subjective factors for the network entity corresponds to a subjective characteristic of the network entity; determine, based on the prerequisite score and the subjective score, a trust score for the network entity; and modify operation of the computer network based on the trust score for the network entity.

[0016] Incorporating subjective factors into assessments of trustworthiness, rather than relying on only objective factors, can improve the value of trustworthiness assessments. As described herein, trust assessments, which are often expressed or summarized through a numerical trust score, can be a function of both objective and subjective factors. Use of both objective and subjective factors in trust assessments tends to lead to a more complete and multifaceted result. Trustworthiness assessments that consider both objective and subjective factors may more accurately reflect the views of different stakeholders.

[0017] Different stakeholders (e.g., network users, administrators, or organizations) may place different levels of importance on factors contributing to trustworthiness, so trustworthiness assessments could change based on the stakeholder for which the trustworthiness assessment is made. Trustworthiness assessments that take into account the stakeholder for which the assessment is made are more likely to better reflect the values and priorities of that stakeholder.

[0018] Assessing both objective and subjective factors is also likely to improve the relevance and usefulness of trustworthiness. For example, where the trust calculation is used to inform decision-making or guide actions, it will be more valuable or effective if it accurately reflects the values and priorities of the people or organizations using the trust assessment. Subjective factors may help improve the relevance and usefulness of the calculation by ensuring that it is aligned with the needs and goals of the stakeholders.

[0019] Further, to the extent that generating subjective scoring information involves soliciting the opinions and views of stakeholders, trust assessments based on the subjective opinions and views of stakeholders might be more likely to be accepted and supported by the people or organizations that are using the trust assessments. This could be particularly important if a trust assessment is being used to inform important decisions or to guide actions that will have significant consequences.

[0020] Use of subjective information in trust assessments, as described herein, might be considered “subjective” in at least two independent respects. In one respect, the factor values themselves may be subjective. For example, subjective factors as described herein may include attributes such as reliability, flexibility, scalability, maintainability, robustness, technical support availability, and others. At least to the extent that these and other factors are based on opinions or feedback solicited or otherwise received from humans (e.g., one or more stakeholders, users, administrators, owners, or other reviewers), the factor values may be considered “subjective.”

[0021] The weights applied to the subjective or other factors may also be independently considered “subjective.” For example, as described herein, to calculate a trust score, the subjective factors may be weighted differently in trust score calculations for different stakeholders or consumers of the trust score. How the factors are weighted may be based on each stakeholder's expressed view as to which of the factors are considered important to that stakeholder. In some examples, factor weights may be based on responses to a survey or other means of prompting the stakeholder to express views about how the subjective factors should be weighted. In other examples, the factor weights may be based on the role of the stakeholder, and may be determined by an organization or network administrator. Accordingly, how each factor is weighed when calculating the trust score may also be considered independently “subjective,” at least to the extent that the trust score calculation is based on a stakeholder's (or an organization's) subjective opinion about which subjective factors are considered more important, for a given context, than other factors.

[0022] FIG. 1A is a conceptual diagram illustrating an example computer network 100 in accordance with one or more aspects of this disclosure. Computer network 100 may itself include multiple computer networks, and such computer networks may include private networks (e.g., internal corporate or government networks), virtual private networks, residential networks, or public networks (e.g., cellular communication networks, Internet backbone networks, etc.).

[0023] In the example of FIG. 1A, computer network 100 includes a set of network entities 102A through 102G, where “G” may be any number. This disclosure may refer to network entities 102A through 102G collectively as “network entities 102.” In general, each of network entities 102 is individually addressable within computer network 100 and can send and receive data on computer network 100. One or more of network entities 102 may be computing nodes, such as real or virtual endpoint devices (e.g., personal computers, smartphones, tablet computers, server devices, laptop computers, Internet of Things (IoT) devices, wearable devices, etc.) or real or virtual intermediate devices (e.g., routers, switches, firewall devices, virtualized appliances, etc.). In some examples, one or more of network entities 102 may be network services, such as web servers, certain application programming interfaces (APIs), media streaming services, and so on. Other types of network entities may include components, controllers, and routes through computer network 100.

[0024] In some examples, each of network entities 102 may have accompanying metadata, such as a device or port description (e.g., which may be configured on the device by an administrator). Such metadata may also include profile names, which may include descriptions of the device as labeled by an administrator or provided by a customer. Metadata associated with one or more of network entities 102 may include website or site names or descriptions of such entities, which may include geographic locations corresponding to where the entity is housed or located.

[0025] Also shown in FIG. 1A are input systems 103A through 103N (collectively “input systems 103”). Each of input systems 103 may correspond to or include one or more systems or devices often found on a network. Such systems may include devices that support diagnostic processes implemented within a network (e.g., “trouble ticketing” systems, customer feedback systems, and alarm applications), user devices, and systems to support email, messaging, chat, and/or collaboration applications (e.g., Microsoft Teams or Salesforce.com's Slack platform).

[0026] One or more of such input systems 103 may be considered, in some examples, to be “source systems” or “source services” that are responsible for reporting factor values that are used in the calculation of a trust score for one or more network entities 102. For instance, one or more of input systems 103 may operate as a source service that reports the severity of known vulnerabilities applicable to a network device or network entity 102. Although each of input systems 103 are illustrated as being distinct from network entities 102 in FIG. 1A, some input systems 103 may be part of or integrated into a network entity. For example, where an input system 103 monitors metrics for a network entity, a source service or input system 103 might be implemented as a module or other logic that operates within that network entity. Further, in some cases, one or more network entities 102 may monitor and report information about its own operation, and such an example, that network entity may be considered to be acting as a source service or input system 103.

[0027] FIG. 1A also illustrates router 191, firewall 192, access control system 193, asset management system 194, and alarm system 195. Router 191 may represent any network device operating as a router on network 100. Firewall 192 may perform functions relating to monitoring incoming and outgoing network traffic for network 100. Access control system 193 may perform functions relating to regulating or limiting access to one or more network entities 102 within network 100 or limiting access to other resources provided by network 100. Asset management system 194 may perform functions relating to managing, administering, and/or accounting for resources within network 100 or even outside of network 100. Alarm system 195 may perform functions relating to detecting conditions for, generating, and/or managing alarms that may be triggered by conditions within network 100.

[0028] Like input systems 103, each of router 191, firewall 192, access control system 193, asset management system 194, and alarm system 195 are illustrated in FIG. 1A as being distinct from network entities 102. However, in other examples, one or more of router 191, firewall 192, access control system 193, asset management system 194, and alarm system 195 could be implemented as a network entity 102, or implemented using one or more network entities 102. Also, for ease of illustration, one router 191, firewall 192, access control system 193, asset management system 194, and alarm system 195 is shown in FIG. 1A. Techniques described herein, however, may apply to environments in which any number of routers 191, firewalls 192, access control systems 193, asset management systems 194, and alarm systems 195 are deployed.

[0029] As mentioned above, computer networks such as computer network 100 may be designed to operate using a zero-trust networking architecture. The concept of trust arises in not just a security context, but other contexts as well, including contexts involving risk management associated with risks that an organization or business assumes when it relies on a network or devices within the network. However, in a security context, when a computer network operates using a zero-trust networking architecture, numerous authentication and security checks are performed (e.g., each time a service is requested from a network entity in the computer network). Similar processes and checks may be performed in other contexts as well. Performing such checks may be resource intensive and may increase network latency.

[0030] Accordingly, to increase network performance in computer networks that operate using trust architectures (for security purposes and otherwise), network administrators may reduce the intensity of checks with respect to specific network entities on an ad hoc basis. While doing so may increase network performance, doing so may undermine the overall security and/or reliability of the network to the extent that there is no systematic methodology for reducing the intensity of checks with respect to individual network entities. Rather, such reductions in the intensity of checks may be based on a network administrator's experience and intuition. This may lead to inconsistencies and misconfigurations in the computer network, e.g., because knowledge of the reductions in the intensity of checks might not be transferred between network administrators. Ultimately, this may lead to adverse outcomes, such as security breaches and network outages.

[0031] This disclosure describes techniques that may address one or more of these issues. For instance, as shown in the example of FIG. 1A, computer network 100 includes a trust controller 104. Trust controller 104 is a computing system that comprises one or more computing devices, such as one or more server devices, personal computing devices, or other types of computing devices. In some examples, computer network 100 may be a Wide Area Network (WAN), and trust controller 104 may be part of a WAN controller that manages the WAN. In some examples described herein, trust controller 104 determines a level of trust for network entities 102 in computer network 100. In some examples, a level of trust may be expressed as a number or a trust “score.” In other examples, a level of trust may be expressed as a category of trust values that indicate a granular continuum of trust values (e.g., “highly trusted,” “trusted,” “untrusted,” “trust unverified”), or categories of trust values for certain operations (“trusted for data transfer,” “trusted for testing,”). In still other examples, a level of trust may be expressed as a binary value (e.g., “trusted” or “untrusted”).

[0032] Although some examples of this disclosure are described in the context of calculating a “trust score,” such examples may alternatively be implemented using a “level of trust.” Similarly, examples described in terms of a “level of trust” could also be implemented using a trust score. In that sense, therefore, the terms “trust score” and “level of trust” may, depending on the context, be used herein interchangeably. Often, trust scores can be represented in terms of a percentage, with 100% representing complete trust, and 0% representing no trust.

[0033] Additionally, trust controller 104 may use the trust scores for network entities 102 to perform various actions with respect to computer network 100. In some examples, one or more computing systems other than trust controller 104 may determine trust scores and/or use the trust scores for network entities 102 to perform actions with respect to computer network 100. However, for ease of explanation, at least some examples herein describe trust controller 104 as calculating trust scores and performing actions. In such examples, because trust controller 104 determines the trust scores in a predictable, reproducible way, the trust scores may be applied consistently throughout network entities of computer network 100. Thus, inconsistencies associated with ad hoc adjustments to authentication, security, and/or other checking may be reduced and there may be less need for documentation to pass knowledge of security procedures between network administrators.

[0034] Trust controller 104 may be implemented as a single trust controller or as (or within) a system (e.g., a hierarchy) of two or more trust controllers. In examples where trust controller 104 is implemented as a system of trust controllers, the trust controllers of trust controller 104 may correspond to specific classes of network entities, different network entities in sub-networks of computer network 100, or other groupings of network entities.

[0035] Trust controller 104 may determine trust scores for network entities 102. The trust score for a network entity indicates a level of trust in the network entity. In general, there is greater trust in a network entity if the network entity has previously performed in a manner that is desired by administrators of the network. Conversely, there is less trust in a network entity if the network entity has not performed in a manner that is desired by administrators of network. For example, the trust score for a network entity may indicate low trust if the network entity is known to be compromised by malware, if the network entity is known to generate corrupted data, if the network entity is known to have frequent service outages, if the network entity has known unpatched security flaws, and so on.

[0036] To determine a trust score for each of network entities 102, trust controller 104 may use one or more trust score plans 400. Each of network entities 102 in FIG. 1A is shown with an accompanying trust score plan 400. For example, trust score plan 400A is shown adjacent to network entity 102, which is intended to indicate that trust score plan 400A is a trust score plan that can be used to calculate a trust score for network entity 102A. Similarly, trust score plan 400B is shown adjacent to network entity 102B, and trust score plan 400C is shown adjacent to network entity 102C. Some trust score plans 400 may be applicable or used by multiple network entities 102. For example, as suggested in FIG. 1A, trust score plan 400D is associated with and can be used by network entities 102D, 102E, and 102F. Such sharing of trust score plan 400D may be appropriate if each of network entities 102C, 102D, and 102E have common characteristics (e.g., are identical devices, are produced by the same manufacturer, perform similar functions, and/or are otherwise similar).

[0037] In addition, some entities may have multiple trust score plans 400. In the example of FIG. 1A, multiple trust score plans 400G are shown adjacent to network entity 102G. Multiple plans for a given entity are often appropriate, particularly if there are reasons to calculate a trust score for different purposes or in different contexts (e.g., for a normal operating environment, for an environment where an alarm is active in the network, or for use in a demonstration or training exercise).

[0038] Entities may have default trust score plans, which may be based on common or typical usage patterns for the given entity. For instance, for entities 102 that represent a physical network device, the manufacturer of the network device may ship the device with a default trust score plan that is intended to be appropriate for typical uses of the network device. Such a default trust score plan may be based on publicly-available standards documents that specify rules pertaining to how the device should be configured to meet the needs of typical or even specific uses of the network device. In some examples, similar network devices manufactured by different vendors may be associated with different default trust score plans.

[0039] In some cases, there may be multiple standardized or default trust score plans for a network device, depending on how the network device is to be used. For example, one default plan may be based on a set of standards that generally apply in a secure networking context (e.g., based on standards published Center for Internet Security or “CIS”). Another default plan may be based on a different set of standards that might apply to governmental deployment of the device (e.g., Defense Information Systems Agency or “DISA” requirements). A third default plan may be based on a different set of standards that apply to use of the device in the healthcare industry (e.g., Health Insurance Portability and Accountability Act or “HIPAA” requirements). In some cases, trust score plans 400 generated based on one or more of such standards may be manually created by the manufacturer of the network device, and included with the network device or otherwise made available to users. It may be possible, however, to generate such trust score plans 400 automatically based on parsing the requirements, rules, and/or specifications published by standards organizations.

[0040] In general, each of trust score plans 400 may define a different way to calculate a trust score for given entity. Since trust can be used in a variety of decision-making processes involving entities (e.g., network entities 102 of network 100), enabling a flexible way to calculate trust scores can be advantageous, since trust scores can differ based on context. For example, a specific network entity 102 operating in network 100 in FIG. 1A might have a high trust score (and be afforded a high amount of trust) based on its hardware and software characteristics. However, if that same network entity 102 was deployed in a different network, where that different network might be used for a different purpose in a different environment, those same hardware and software characteristics of network entity 102 might not be sufficient to confer any significant trust. Accordingly, the concept of a trust score should be flexible, since not all networks are the same, not all uses and purposes for trust scores are the same, and not all contexts are the same.

[0041] As described herein, a trust score plan enables an ability to define how to calculate a trust score for a given entity. A trust score plan may define and/or describe a list of trust factors within each of a number of trust factor categories. The trust score plan may also define how each of those factors contribute to an overall trust score. A different trust score plan may be used to customize how trust scores can be calculated for different environments, contexts, purposes, customers, deployments, network segments (e.g., core network, edge network, metro deployment), and/or service types (IP v4, IP v6, streaming, Voice Over IP). In some examples, trust score plans may be used to help meet quality of service standards for particular environments, customers, or deployments. Still further, trust score plans may be used for different customer classes, so that customers operating in a governmental context can use trust score plans that adhere to DISA standards, customers in healthcare industries can use trust score plans that adhere to HIPAA standards, and customers in other industries can use trust score plans that adhere to applicable standards in such other industries.

[0042] As described herein, trust scores may be calculated by trust controller 104, and trust score plans 400 may be stored at a location enabling convenient access by trust controller 104. Therefore, although some trust score plans 400 are illustrated in FIG. 1A next to illustrations of various network entities 102, such trust score plans are not necessarily stored at each such network entity 102. Instead, such trust score plans 400 may be stored in a data store within trust controller 104. Therefore, in FIG. 1A, trust score plans 400 shown near a given network entity 102 are illustrated with dotted lines, which is intended to indicate that such trust score plans 400 are merely associated with a given network entity 102, rather than necessarily stored at such network entities 102.

[0043] In accordance with one or more aspects of the present disclosure, trust controller 104 may collect information about one or more network entities 102. For instance, in an example that can be described with reference to FIG. 1A, each of input systems 103 collects information, which may include information pertaining to the operation of each network entity 102, and may include metrics, information about configurations of each network entity 102, information about operations performed by each of network entities 102, and information about various subjective characteristics or ratings of network entities 102.

[0044] Information about operations performed may include, for example, amount of traffic processed, errors logged, reboots required, and other information. Information about subjective characteristics may include information about reliability, uptime ratings, whether issues tend to be quickly and effectively resolved, scalability, flexibility, interoperability with other systems, ease of maintenance, robustness, availability of technical support and/or customer support, privacy and security ratings, and cost effectiveness. Such characteristics might be generated by soliciting opinions or ratings information from users, network operators, or other stakeholders, so in that sense, the information might be relatively qualitative and/or subjective, rather than quantitative or objective. In any event, such characteristics might be expressed in terms of a score (e.g., from 0 to 100), enabling use of the subjective characterizations in a quantitative algorithm to generate a trust assessment or trust score.

[0045] Such input systems 103 send signals that include the collected information to trust controller 104. Trust controller 104 stores the collected information for later use. Alternatively, or in addition, one or more of network entities 102 independently report information about their operation to trust controller 104. Such information may also include metrics, information about configurations, and/or information about operations performed. Trust controller 104 stores any information received from network entities 102.

[0046] Trust controller 104 may prepare to calculate a trust score. For instance, continuing with the example being described in the context of FIG. 1A, trust controller 104 detects a signal (i.e., a “trigger”) that it determines corresponds to a command or request to calculate a trust score for one or more of network entities 102. In response to the signal, trust controller 104 identifies a network entity 102 for which a trust score is to be calculated. In one specific example, trust controller 104 determines that the signal includes a request to calculate a trust score for network entity 102B. Trust controller 104 identifies trust score plan 400B as an appropriate plan to use to calculate a trust score for network entity 102B. In some examples, trust score plan 400B might be a default trust score plan for network entity 102B. In other examples, trust score plan 400B might be a trust score plan that has been customized by an administrator to meet the specific use context of network entity 102B within network 100. Trust controller 104 accesses trust score plan 400B in order to calculate a trust score for network entity 102B.

[0047] In some examples, the trigger corresponding to a command or request to calculate a trust score may identify a stakeholder (e.g., network user or operator or organization) requesting the trust score. In such an example, trust controller 104 may identify the stakeholder based on the credentials (e.g., username and password) used by the stakeholder when accessing the network. Trust controller 104 may use the identity of the stakeholder to select weights to apply to each of a number of subjective factors when calculating a trust score for that stakeholder. The weights may apply the stakeholder specifically, or more broadly to the role of the stakeholder.

[0048] Trust controller 104 may perform calculations to generate trust score result 500B for network entity 102B. For instance, still referring to FIG. 1A, trust controller 104 interprets trust score plan 400B and applies trust score plan 400B to information stored at trust controller 104. Such stored information may include the information previously obtained by trust controller 104 from input systems 103 and/or network entities 102. In most cases, the stored information used when applying trust score plan 400B to generate trust score result 500B will primarily pertain to network entity 102B, since trust controller 104 is seeking to calculate a trust score for network entity 102B. However, it is possible that information about other network entities 102 could be used. Based on the stored information and the specification outlined in trust score plan 400B, trust controller 104 generates trust score result 500B. In some examples, trust score result 500B may include multiple supporting documents and/or reports underlying the trust score calculations. Included within trust score result 500B may be a specific composite trust score value or percentage value that represents the trust score for network entity 102B.

[0049] Trust controller 104 may adjust the operation of network 100 based on the trust score value for network entity 102B. For instance, again referring to FIG. 1A, trust controller 104 may determine that trust score result 500B (or the composite trust score value included within trust score result 500B) suggests that network entity 102B cannot be trusted in the context defined or represented by trust score plan 400B. In such an example, trust controller 104A may take action to limit use of network entity 102B within network 100, given its lack of trustworthiness. In another example, trust score result 500B may suggest that network entity 102B can be trusted, and in such an example, trust controller 104 may enable network entity 102B to play a larger role or process additional data within network 100. In still other examples, trust controller 104 may, based on a trust score result, directly or indirectly reconfigure a network entity under test, or other network entities to modify behavior with respect to that network entity (e.g., route traffic to avoid a given network entity). In further examples, trust controller 104 may output directives to a network controller or other system to reconfigure aspects of a network.

[0050] Trust score plans may describe how to calculate a trust score in terms of contribution classes and factors within those contribution classes. Contribution classes are classes of information that contribute to the trust score for a network entity 102. In some examples, the contribution classes include one or more of a set of prerequisites for the network entity 102, a set of variable factors for the network entity 102, a set of reputational factors for the network entity, and/or a set of subjective factors for the network entity 102.

[0051] In some examples, a trust score calculation may be a function of separate sub-scores for each of the contribution classes. For instance, there may be a first sub-score for the prerequisites for the network entity, a second sub-score for the variable factors for the network entity, a third sub-score for the reputational factors for the network entity, and a fourth sub-score for the subjective factors for the network entity. In some examples, the sub-score for a contribution class is expressed as a percentage of a highest possible sub-score for the contribution class. In other examples, sub-scores for contribution classes are expressed in other ways. Trust controller 104 may use the sub-scores for the contribution classes (e.g., the first, second, third, and fourth sub-scores) for the network entity 102 to determine the trust score for the network entity. Thus, the trust score for the network entity may be the result of a mathematical computation that is based on one or more of the prerequisites for the network entity, the variable factors for the network entity, and/or the reputational factors for the network entity.

[0052] Each of the prerequisites for the network entity 102 is, in general, a condition that must be satisfied for the trust score for the network entity to have a value greater than a minimum value (e.g., zero). In this disclosure, a trust score for a network entity having the minimum value generally means that the lowest trust level (e.g., no trust) is ascribed to the network entity. In some examples, the minimum value may be equal to zero. However, values other than zero may be used to represent a trust score indicating no trust.

[0053] The variable factors for a network entity 102 correspond to current characteristics of the network entity. Example characteristics of a network entity may include a current configuration of the network entity, a version of the network entity, active features of the network entity, sentiment values or polarity associated with a network entity, and so on. The variable factors may be weighted such that some of the variable factors contribute more to the variable factors sub-score than other ones of the variable factors.

[0054] Each of the reputational factors for a network entity 102 may be a function of specified historical events involving the network entity. For example, trust controller 104 modifies the values of reputational factors for a network entity based on behavior of the network entity over time. For example, trust controller 104 may modify the value of one reputational factor based on the number of times that the network entity spontaneously reboots. In another example, trust controller 104 may modify the value of a reputational factor based on the number of times the network entity has been reconfigured.

[0055] Each of the subjective factors for a network entity 102 may be a function of certain subjective or qualitative assessments of a network entity. Such subjective assessment may pertain to any of a number of factors including subjective assessments of reliability, uptime, flexibility, ability to integrate with other systems, ease of maintenance, robustness, availability of technical support, privacy and security, quality of the network or network entity, level of customer support, or overall value relative to costs. Information about these subjective factors may be collected from user and/or stakeholders through focus groups, interviews, online forums, customer feedback forms, surveys, ratings agencies, ranking organizations, and/or user experience (UX) research. Trust controller 104 may collect, from input systems 103, information that has been assembled about such factors. Trust controller 104 may use the information to arrive at a subjective sub-score, which may involve applying a weight to each of such factors as may be appropriate for a given user. To the extent that such information received (e.g., from input systems 103) is originally sourced from administrators, stakeholders, or other human sources, such human sources are preferably encouraged (by policy or otherwise) to provide detailed and specific feedback about their experience with a given network or system.

[0056] Trust score plans enable decoupling of factor values and factor contributions from the manner in which a trust score is calculated. Such decoupling simplifies the process of trying different trust score calculations or applying different rules for calculating a trust score. When a source service or input system 103 provides a factor value, it could be used in different score plan calculations with different contributions and/or weights. Such a factor value might even be used in different factor categories. For example, the same factor might be used as a prerequisite factor in one trust score plan but as a variable factor in another trust score plan. In some examples, trust score plans are initially constructed from a small set of coarse-grained factors and later extended to include a larger set of fine-grained factors. Such factors may be based on a standards-based benchmark specification, such as one published by CIS or DISA, as mentioned above. Such a specification may include a number of “benchmark rules” organized into a number of “benchmark rule groups.” These rules may outline appropriate requirements and/or configurations for ensuring a specific entity complies with the objective underlying the specification (e.g., network security, governmental standards, health industry requirements). In such an example, a coarse-grained factor might be the aggregate score of a benchmark rule group or even of a complete benchmark. A fine-grained factor might be an individual benchmark rule.

[0057] As described herein, trust controller 104 may determine a trust score for a network entity based on several factors. For instance, trust controller 104 may determine a trust score for a network entity based on factors included within one or more contribution classes. In some examples, such factors may include operating characteristics, metrics, sentiment information derived from data collected by one or more of input system 103, network topology, information about subjective assessments of network entities 102, and/or other information.

[0058] Where sentiment information is used, such sentiment information may be derived from text associated with one or more of input systems 103, which may include information derived from diagnostic data associated with one or more of network entities 102, such as notes generated by an administrator or comments provided by a user when addressing an issue arising with a network entity 102 in a trouble ticketing system implemented within network 100. In other examples, sentiment information may be derived from text or other information associated with metadata associated with one or more of network entities 102.

[0059] Where subjective information is used, information about each different type of subjective assessment may be collected from one or more input systems 103, and then used to calculate a subjective score or subjective sub-score. One way to use such subjective assessments is to quantify the assessment (e.g., as a quantity from 0 to 100) and then apply a weight to each of a selected number of different types of subjective assessments. The weight may be chosen based on a user or stakeholder's own values or priorities as to which types of subjective assessments are important for their purposes. For example, one user may value reliability and flexibility traits in a network entity, so those subjective factors may be weighted more heavily when calculating a trust score than other subjective factors when calculating a trust score. A different user may more highly value ease of maintenance and/or availability of customer support than other factors, so those subjective factors may be weighed more heavily for that user.

[0060] When calculating a trust score, subjective factors could, as suggested above, be applied on a per-user basis, so that a subjective sub-score for one user may be different than a subjective sub-score for another user that has different values or priorities. In other examples, subjective factors could be applied based on a user's role, so that subjective factors for two users having the same role (e.g., network administrator) may be applied the same way, but differently than a user having a different role (e.g., field technician).

[0061] Using subjective factors in a trustworthiness calculation in this way could improve the relevance and usefulness of the resulting trust score by ensuring that it takes into account the values and priorities of the stakeholders who will be using the calculation. In other words, subjective factors could potentially improve the relevance and usefulness of a trustworthiness calculation for different types of stakeholders, including network operations center (NOC) staff, network planners, field technicians, end users, and others.

[0062] For example, network operations center staff are generally responsible for monitoring and managing the performance of networks and ensuring that any issues are promptly addressed. For stakeholders in this role, trustworthiness might be primarily determined by factors such as reliability, uptime, and the ability to quickly and effectively resolve issues. Trust scores that incorporate subjective factors that reflect the priorities of NOC staff may be more relevant and useful in informing decisions about which entities to deploy and how to prioritize maintenance and upgrades.

[0063] Network planners are responsible for designing and implementing network architectures and strategies. Such stakeholders may value factors such as scalability, flexibility, and the ability to integrate with other systems, and accordingly, trustworthiness might be appropriately based on such factors. Trust scores that incorporate subjective factors that reflect the priorities of network planners may be helpful to inform decisions about which entities to deploy and how to design and implement networks.

[0064] Field technicians are responsible for installing, configuring, and maintaining network equipment in the field. For these stakeholders, trustworthiness might be determined by factors such as ease of maintenance, robustness, and the availability of technical support.

[0065] End users, stakeholders that use the network and its services, may have varied values and priorities. For example, an end user who relies on the network for critical business operations may prioritize uptime and reliability, while an end user who uses the network for personal communication may prioritize privacy and security.

[0066] In calculating a trust score, trust controller 104 may also consider attributes of entities that separate one entity from another entity. For example, if network entity 102A trusts network entity 102C, and network entity 102C trusts network entity 102E, network entity 102A may trust network entity 102E to some extent, since network entity 102A trusts an entity that also trusts network entity 102E. In such an example, trust controller 104 may increase the trust score for network entity 102E for operations that network entity 102A may perform with network entity 102E (e.g., a transfer of data). Network entity 102A might not have any direct interactions with network entity 102E, but network entity 102A might be able to infer a trust relationship with network entity 102E, based on the trust network entity 102A has in intervening entities (e.g., network entity 102C).

[0067] On the other hand, if network entity 102A trusts network entity 102C, but network entity 102C does not trust network entity 102E, then network entity 102A might not trust network entity 102E, since network entity 102C (an entity that network entity 102A trusts) does not trust network entity 102E. In this example, trust controller 104 may decrease the trust score for network entity 102E for operations that network entity 102A may perform with network entity 102E. Network entity 102A may therefore determine a trust inference with respect to network entity 102E that is based on its level of trust with network entity 102C and network entity 102C's level of trust with network entity 102E.

[0068] Incorporating subjective factors into the calculation of trustworthiness can inform decision-making in several ways. Subjective factors can reflect the values, priorities, and goals of the organization and its stakeholders, and by taking these factors into account, the calculation of trustworthiness can be more closely aligned with the needs and priorities of the organization. This can help ensure that trust score calculations are more relevant and useful for guiding decision-making, as such calculations take into account the perspectives and priorities of the stakeholders.

[0069] Accordingly, incorporating subjective factors has a number of benefits. For example, use of subjective factors tends to prioritize use of network devices based on their alignment with the operator's values and goals. For example, if the operator values sustainability, it may prioritize (e.g., select) network devices that have a low environmental impact.

[0070] In addition, subjective factors such as the level of customer satisfaction or the quality of the services provided by the network device could be used to inform the evaluation of its performance. For example, if a network device consistently receives high ratings from customers, this could be considered a positive subjective factor in its performance evaluation.

[0071] Further, subjective factors such as the reputation of the manufacturer or the level of transparency in their business practices could be used to inform the evaluation of the trustworthiness of a network device. For example, if a manufacturer has a history of producing faulty or unreliable products, this could be considered a negative subjective factor in the trustworthiness evaluation of their network devices.

[0072] Further, a manufacturer or vendor that is responsive and provides good support can be considered more trustworthy, as it suggests that they are committed to maintaining the quality of their products and addressing any issues that may arise. Accordingly, the level of support provided by the manufacturer or vendor may affect trust score calculations.

[0073] Further, a manufacturer or vendor that is willing to work closely with the operator and provide regular updates on the status of their products can be considered more trustworthy, as such practices suggest a commitment to maintaining a strong partnership. Accordingly, the level of collaboration and communication with the operator may affect trust score calculations.

[0074] Still further, a manufacturer or vendor that is actively investing in research and development and bringing new, innovative products to market can be considered more trustworthy, as such activities suggest a commitment to staying ahead of industry trends and evolving with the needs of the operator. It may be appropriate for innovation and investment in research and development to affect trust score calculations.

[0075] Although subjective factors can be useful in evaluating the trustworthiness of network devices, objective criteria, as described herein, can be used to evaluate subjective factors.

[0076] Further details relating to trust scoring, particularly in the context of computer networks, are available in U.S. patent application Ser. No. 16/949,996, filed Nov. 23, 2020, entitled “Trust Scoring of Network Entities in Networks,”. Details relating to inferring trust, particularly in the context of computer networks, are available in U.S. patent application Ser. No. 17/474,871, filed Sep. 14, 2021, entitled “Inferring Trust in Computer Networks,”. Details relating to assessing trust by performing a sentiment analysis for an entity or device within a system (e.g., a network) are available in U.S. patent application Ser. No. 17/644,555, filed Dec. 15, 2021, entitled “Use of Sentiment Analysis to Assess Trust in a Network,”. Details relating to use of trust score plans are available in U.S. patent application Ser. No. 17/655,140, filed Mar. 16, 2022, entitled “Use of a Trust Score Plan”. Details relating to remediation of trust are available in U.S. patent application Ser. No. 18/148,762, filed Dec. 30, 2022, entitled “Remediation Work Score for Network Trust Applications”. The entire content of all of these applications is hereby incorporated by reference.

[0077] FIG. 1B, FIG. 1C, and FIG. 1D are conceptual diagrams illustrating various types of entity specializations, in accordance with one or more aspects of the present disclosure. For example, FIG. 1B illustrates that an entity can take the form of a functional entity, such as one or more routers 121 or one or more switches 122. As also illustrated in FIG. 1B, an entity could take the form of an equipment entity or a network entity, and represent an aggregation of equipment (e.g., a specific type of equipment, such as a set of reconfigurable optical add-drop multiplexers or “ROADMs”) or networks (e.g., subnets or sub networks). The types of entities illustrated in FIG. 1B may generally correspond to the types of entities described in connection with FIG. 1A.

[0078] Although FIG. 1A is described in terms of entities that take the form of network entities 102 (e.g., network devices, computing devices, computing services, and the like), the term “entity” may, in other examples, encompass broader concepts. For example, FIG. 1C and FIG. 1D show other types of entities or at least different classifications of entities, such as quantitative entities 160 and qualitative entities 170. Quantitative entities 160 may include users 161, services 162, power attributes 163, cooling attributes 164. Qualitative entities 170 may include other types of entities, perhaps non-physical entities.

[0079] In the example of FIG. 1D, qualitative entities 170 include one or more instances of vendor support 171, contractor experiences 172, and/or contractor skillsets 173. Techniques described herein may apply to both quantitative entities 160 and qualitative entities 170 having a trust score. Such a trust score may be defined to some extent based a trust score plan selected for the entity, based on sentiment analysis, and other factors. Where sentiment analysis is used, sentiment can be derived from text or other data associated with information or other data associated with one or more instances of vendor support 171, data associated with contractor experiences 172, and/or data associated with or describing assessments of contractor skillsets 173. Such qualitative entities 170 may be particularly susceptible to a sentiment analysis, given that text describing such entities may be readily available through customer reviews or text-based communications among vendors, contractors, customers, and other personnel.

[0080] In addition, a component of a trust score in certain situations may be spatial in nature. For example, as described herein, a trust score may also be defined to some extent based on the level of trust among neighboring entities in a graph, diagram, table, or other arrangement of data.

[0081] FIG. 2 is a block diagram illustrating example components of trust controller 104 in accordance with one or more aspects of this disclosure. FIG. 2 illustrates one example of trust controller 104, and this disclosure encompasses any other appropriate configurations of trust controller 104.

[0082] As shown in the example of FIG. 2, trust controller 104 includes one or more processors 202, one or more communication units 204, one or more input devices 208, one or more output devices 210, zero or more display screens 212, one or more power sources 214, one or more storage devices 216, and one or more communication channels 218. Trust controller 104 may include other components. For example, trust controller 104 may include physical buttons, microphones, speakers, communication ports, and so on. Communication channel(s) 218 may interconnect each of components 202, 204, 208, 210, 212, and 216 for inter-component communications (physically, communicatively, and/or operatively). In some examples, communication channel(s) 218 may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data. Power source(s) 214 may provide electrical energy to components 202, 204, 208, 210, 212 and 216.

[0083] Storage device(s) 216 may store information required for use during operation of trust controller 104. In some examples, storage device(s) 216 have the primary purpose of being a short-term and not a long-term computer-readable storage medium. Storage device(s) 216 may include volatile memory and may therefore not retain stored contents if powered off. In some examples, storage device(s) 216 includes non-volatile memory that is configured for long-term storage of information and for retaining information after power on/off cycles. In some examples, processor(s) 202 of trust controller 104 may read and execute instructions stored by storage device(s) 216.

[0084] Trust controller 104 may include one or more input devices 208 that trust controller 104 uses to receive user input. Examples of user input include tactile, audio, and video user input. Input device(s) 208 may include presence-sensitive screens, touch-sensitive screens, mice, keyboards, voice responsive systems, microphones, motion sensors capable of detecting gestures, or other types of devices for detecting input from a human or machine.

[0085] Communication unit(s) 204 may enable trust controller 104 to send data to and receive data from one or more other computing devices (e.g., via a computer network, such as a local area network or the Internet). For instance, communication unit(s) 204 may be configured to receive data from network entities 102. In some examples, communication unit(s) 204 may include wireless transmitters and receivers that enable trust controller 104 to communicate wirelessly with the other computing devices. Examples of communication unit(s) 204 may include network interface cards, Ethernet cards, optical transceivers, radio frequency transceivers, or other types of devices that are able to send and receive information. Other examples of such communication units may include BLUETOOTH™, 3G, 4G, 5G, and WI-FI™ radios, Universal Serial Bus (USB) interfaces, etc. Additionally, trust controller 104 may use communication unit(s) 204 to communicate with one or more other devices.

[0086] Output device(s) 210 may generate output. Examples of output include tactile, audio, and video output. Output device(s) 210 may include presence-sensitive screens, sound cards, video graphics adapter cards, speakers, liquid crystal displays (LCD), light emitting diode (LED) displays, or other types of devices for generating output. Output device(s) 210 may include display screen 212. In some examples, output device(s) 210 may include virtual reality, augmented reality, or mixed reality display devices.

[0087] Processor(s) 202 may include processing circuitry configured to perform various actions. Processor(s) 202 may be configured to read instructions from storage device(s) 216 and may execute instructions stored by storage device(s) 216. Execution of the instructions by processor(s) 202 may configure or cause trust controller 104 to provide at least some of the functionality ascribed in this disclosure to trust controller 104 or components thereof (e.g., processor(s) 202). As shown in the example of FIG. 2, storage device(s) 216 include computer-readable instructions associated with a data collection system 220, a scoring system 222, an action system 224, trust plan system 227, tracking system 226, analysis system 231, and inference system 221. In the example of FIG. 2, storage device(s) 216 may also include source data 228, trust plan data store 237, and scoring data 230. Furthermore, as shown in the example of FIG. 2, the computer-readable instructions associated with scoring system 222 may include computer-readable instructions associated with a prerequisite system 232, a variable factor system 234, a reputation system 236, and a calculation system 238. In other examples, storage device(s) 216 may include computer-readable instructions associated with other systems or modules.

[0088] Data collection system 220 is configured to collect data regarding network entities 102. For example, data collection system 220 may query log data of network entities as part of collecting the data regarding network entities 102. In some examples, data collection system 220 may be configured to collect configuration and performance data regarding the network entities. The configuration and performance data may include snapshots of configuration data, snapshots of alarm data, and logs. Data collection system 220 may also collect information about network topology, enabling a determination of which network entities 102 are neighbors (i.e., are adjacent to each other in a network topology), and enabling collection and storage of other information about the network (e.g., network paths and routes through the network). Source data 228 may include the data collected by data collection system 220.

[0089] Data collection system 220 may also be configured to collect data that can be used to perform sentiment analysis. For example, data collection system 220 may integrate with one or more of input systems 103 (e.g., a trouble ticketing system) to collect the data relating to diagnostic or troubleshooting performed for one or more of network entities 102. In some examples, data collection system 220 may be configured to collect information from email, messaging, chat, and/or collaboration applications or other subjective material sources. Data collection system 220 may also collect information from one or more user devices, where such user devices are configured to present a user interface and collect comments from users about one or more of network entities 102. In one example, such a user interface may present a simplified free form comment box through a user interface, and prompt or enable users to input any comments they want regarding the trust of one or more of network entities 102. In such an example, data collection system 220 may collect such input and make the data available for subsequent sentiment analysis as described herein.

[0090] Data collection system 220 may store (as source data 228) sentiment information having a form other than simply text and user comments. For instance, in some examples, sentiment information could include information about a status or a change of state. In one such example, a user might provide a comment like “physical security in this site is badly maintained.” If this comment is later removed (or otherwise nullified), sentiment information (e.g., “state information”) having a sentiment polarity that is opposite to that of the original comment might be generated and stored within data collection system 220 as source data 228.

[0091] Remediation system 239 may perform functions related to determining the friction metric and the inertia metric, weighing the friction metric and the inertia metric, and determining the remediation metric from the weighted friction metric and the weighted inertia metric. In some examples, remediation system 239 receives a trust score for a network entity from the trust plan system 227. The remediation system 239 may perform one or more of: identifying at least one weakness of the network entity based on the received trust score; determining a set of remediation actions for addressing the at least one weakness; determining, for each remediation action of the set of remediation actions, an expected amount of work associated with the remediation action; selecting a remediation action from the set of remediation actions, based on the determining, for each remediation action, the expected amount of work associated with the remediation action; and performing an operation to execute at least a portion of the selected remediation action.

[0092] Scoring system 222 may use source data 228 to determine trust scores for network entities 102. Scoring data 230 may include the trust scores determined by scoring system 222 for network entities 102. In the example of FIG. 2, prerequisite system 232 may determine prerequisite sub-scores for network entities 102. Variable factor system 234 may determine variable factor sub-scores for network entities 102. Reputation system 236 may determine reputation sub-scores for network entities 102. Subjective system 233 may determine subjective sub-scores for network entities 102. Calculation system 238 may use one or more of the prerequisite sub-scores for a network entity, the variable factor sub-score for the network entity, the reputation sub-score for the network entity, or the subjective sub-score for the network entity to determine the trust score for that network entity.

[0093] Prerequisite system 232 may determine prerequisite sub-scores for network entities 102. That is, prerequisite system 232 may determine the trust score for a network entity based one or more prerequisites. Each of the prerequisites is a condition that must be satisfied for the trust score for a network entity not to have the minimum value. Prerequisite system 232 may evaluate the conditions to determine the prerequisite sub-score for a network entity. Prerequisite system 232 may store the prerequisite sub-scores for network entities 102 as scoring data 230. Example conditions may include whether the network entity is using a certified software release, whether the network entity is using a certified hardware release, and so on. Other prerequisites may include the network entity not using software versions that are known to be vulnerable, the network entity using only supported software or hardware versions, the network entity having the correct network time settings, the network entity having correct certificates for transport layer security (TLS) loaded, and so on. The conditions may be user-defined.

[0094] Variable factor system 234 may determine variable factor sub-scores for network entities 102. In other words, variable factor system 234 may assess sub-scores for each of several characteristics of a network entity. Each of the one or more variable factors for a network entity corresponds to one or more of the characteristics of the network entity. Variable factor system 234 may store the variable factor sub-scores for network entities 102 as scoring data 230. The characteristics may be user-defined. Examples of characteristics may include:

• • [0095] Use of only approved applications on the network entity.
  • [0096] Use of only signed applications on the network entity.
  • [0097] Whether a release version of the network entity is a known-good version.
  • [0098] A state or configuration of a network entity, e.g., if a recommended configuration is applied or not.
  • [0099] Whether a software release version of a software application of the network entity is within a given number of releases of a current version of the software application.
  • [0100] Rate limits for requests to be processed by a routing engine of a router.
  • [0101] Limits on rates of possible network management interactions per time-period.
  • [0102] Password strength requirements.
  • [0103] Numbers and types of open network ports,
  • [0104] Whether cryptographic keys are renewed more frequently than a threshold time limit.
  • [0105] Sentiment analysis associated with a network entity, which may be determined by analysis system 231, as further described below.

[0106] Subjective system 233 may determine subjective factor sub-scores for network entities 102. In doing so, subjective system 233 may assess scores for each of several subjective characteristics of a network entity. Each of the one or more subjective factors may correspond to a characteristic of the network entity, where the assessment might be somewhat subjective in nature. Subjective system 233 may store the subjective factor sub-scores for network entities 102 as scoring data 230. As with variable factors, the subjective characteristics may be user-defined. Examples of subjective characteristics may include:

• • [0107] Reliability
  • [0108] Uptime Rating
  • [0109] Ability to quickly and effectively resolve issues
  • [0110] Scalability
  • [0111] Flexibility
  • [0112] Ability to integrate with other systems
  • [0113] Ease of maintenance
  • [0114] Robustness
  • [0115] Availability of technical support
  • [0116] Privacy and security
  • [0117] Quality of the network
  • [0118] Level of customer support
  • [0119] Overall value relative to network entity cost

[0120] Trust controller 104 may determine a trust score for a network entity based on several factors, each of which may be defined in a trust score plan. For instance, trust controller 104 may determine a trust score for a network entity based on factors included within one or more contribution classes. In some examples, such factors may include operating characteristics, metrics, sentiment information derived from data collected by one or more of input system 103, or other information.

[0121] Contribution classes are classes of information that contribute to the trust score for a network entity. In some examples, the contribution classes include one or more of a set of prerequisites for the network entity, a set of variable factors for the network entity, a set of reputational factors for the network entity, and a set of subjective factors for the network entity.

[0122] In some examples, there may be a separate score or sub-score for each of the contribution classes. For instance, there may be a first sub-score for the prerequisites for the network entity, a second sub-score for the variable factors for the network entity, a third sub-score for the reputational factors for the network entity, and a fourth sub-score for the subjective factors. In some examples, the sub-score for a contribution class is expressed as a percentage of a highest possible sub-score for the contribution class. In other examples, sub-scores for contribution classes are expressed in other ways. Trust controller 104 may use the sub-scores for the contribution classes (e.g., the first, second, third, and fourth scores) for the network entity to determine the trust score for the network entity. Thus, the trust score for the network entity may be the result of a mathematic computation that is based on one or more of the prerequisites for the network entity, the variable factors for the network entity, the reputational factors for the network entity, and/or the subjective factors for the network entity.

[0123] Each of the prerequisites for the network entity is, in general, a condition that must be satisfied for the trust score for the network entity to have a value greater than a minimum value (e.g., zero). In this disclosure, a trust score for a network entity having the minimum value means that a lowest trust level (e.g., no trust) is afforded to the network entity. In some examples, the minimum value may be equal to zero. However, values other than zero may be used to represent a trust score indicating no trust.

[0124] The variable factors for a network entity correspond to current characteristics of the network entity. Example characteristics of a network entity may include a current configuration of the network entity, a version of the network entity, active features of the network entity, sentiment values or polarity associated with a network entity, and so on. The variable factors may be weighted such that some of the variable factors contribute more to the variable factors sub-score than other ones of the variable factors.

[0125] Where sentiment information is used to calculate or modify a trust score, such sentiment information may be derived from text associated with one or more of input systems 103, which may include information derived from diagnostic data associated with one or more of network entities 102, such as notes generated by an administrator or comments provided by a user when addressing an issue arising with a network entity 102 in a trouble ticketing system implemented within network 100. In other examples, sentiment information may be derived from text or other information associated with metadata associated with one or more of network entities 102.

[0126] Analysis system 231 may perform functions relating to sentiment analysis for a given entity. Sentiment analysis information may be used to determine a variable factor sub-score for one or more network entities 102, as described in connection with FIG. 3A and FIG. 3B. Analysis system 231 may apply this analysis as a component of trust. In examples described herein, analysis system 231 may be integrated into scoring system 222 so that variable factor system 234 and calculation system 238 calculate a trust score by considering sentiment analysis associated with network entities or amongst network entities (see calculations associated with Table 1, below). In other examples, however, analysis system 231 may be used to adjust (and thereby improve) trust scores that have already been determined by a scoring system, such as scoring system 222. In such examples, analysis system 231 may apply an adjustment to a trust score for an entity based on attitudinal information or sentiment analysis.

[0127] Each of the reputational factors for a network entity may be a function of specified historical events involving the network entity. For example, trust controller 104 modifies the values of reputational factors for a network entity based on behavior of the network entity over time. For example, trust controller 104 may modify the value of one reputational factor based on the number of times that the network entity spontaneously reboots. In another example, trust controller 104 may modify the value of a reputational factor based on the number of times the network entity has been reconfigured.

[0128] Each of the subjective factors for a network entity may be a function of specified subjective assessments relating to that network entity. For example, trust controller 104 may collect information about such subjective assessments from one or more input systems 103. Trust controller 104 may also access information about how a given user (or consumer of a trust score) values various types of the subjective assessments. In some examples, a given user may value some subjective assessments but not others, or value some more than others. Trust controller 104 may apply a weight to each subjective assessment corresponding to the value or priority that the user or other consumer of a trust score places on each subjective assessment. In some examples, such weights may apply on a user-specific basis, but in other examples, such weights may apply to different groups of users. For example, trust controller 104 may apply one set of weights when calculating a subjective score or sub-score for users having one role (e.g., a network operations staff), but apply a different set of weights when calculating a score for other users (e.g., field technicians).

[0129] In calculating a trust score, trust controller 104 may also consider attributes of entities that separate one entity from another entity. For example, if network entity 102A trusts network entity 102C, and network entity 102C trusts network entity 102E, network entity 102A may trust network entity 102E to some extent, since network entity 102A trusts an entity that also trusts network entity 102E. In such an example, trust controller 104 may increase the trust score for network entity 102E for operations that network entity 102A may perform with network entity 102E (e.g., a transfer of data). Network entity 102A might not have any direct interactions with network entity 102E, but network entity 102A might be able to infer a trust relationship with network entity 102E, based on the trust network entity 102A has in intervening entities (e.g., network entity 102C).

[0130] On the other hand, if network entity 102A trusts network entity 102C, but network entity 102C does not trust network entity 102E, then network entity 102A might not trust network entity 102E, since network entity 102C (an entity that network entity 102A trusts) does not trust network entity 102E. In this example, trust controller 104 may decrease the trust score for network entity 102E for operations that network entity 102A may perform with network entity 102E. Network entity 102A may therefore determine a trust inference with respect to network entity 102E that is based on its level of trust with network entity 102C and network entity 102C's level of trust with network entity 102E.

[0131] As part of calculating a trust score, variable factor system 234 may apply different weights to different characteristics to determine the variable factor sub-score for a network entity. Thus, specific characteristics may make greater contributions to the variable factor sub-score than other characteristics. Table 1, below, illustrates example contributions of specific choices (possible values) for specific characteristics of a network entity. As shown in the example of Table 1, software applications (apps) may be categorized into a plurality of classes (e.g., Class 1, Class 2, Class 3, etc.). The different classes of apps may correspond to different levels of risk. For instance, Class 1 apps may be limited to apps approved by a system vendor, signed applications, applications with security audits, etc., Class 2 apps may be limited to apps approved by a supplier, unsigned applications, applications with regular security updates, etc., Class 3 apps may be unapproved apps, and so on. In Table 1, the contribution column indicates a score that would be contributed if the corresponding choice in the “choices” column applies. In other examples, one or more choices may be associated with negative values, but the final variable factor sub-score may be limited to a minimum of zero or other predefined minimum value.

[0132] In Table 1, the “Sample Score” column provides an example of how a variable factor sub-score may be calculated for a specific network entity. Thus, in the example of Table 1, if the network entity used only Class 1 apps, variable factor system 234 may determine that the contribution for this characteristic (“Software”) is 30. If a specific attribute of the network entity (e.g., software) is not a known good release, variable factor system 234 may determine that the contribution for the second characteristic is zero. If the network entity uses a software version that is within 3 releases of the most recent release of the software, variable factor system 234 may determine that the contribution for this third characteristic (e.g., “Software version<=last version—3”) is 10. If the sentiment analysis for this network entity indicates a positive polarity, then variable factor system 234 may determine that the contribution for the “Sentiment” characteristic is 20. Based on such assessments, variable factor system 234 may determine that the variable factor sub-score is 60 (30+0+10+20). See Table 1, below.

[0133]

TABLE 1
	Characteristics	Contribution	Choices	Sample Score
Contributions	Software (max	30	Class 1 apps only	30
within variable	50%)	20	Class 1 and Class 2 apps only
trust		10	Class 1, Class 2 and Class 3 apps
	Is a known-good	40	Yes
	release	0	No	0
	Software version <=	10	Yes	10
	last-3	0	No
	Sentiment (max	20	Positive	20
	20%)	5	Neutral
		0	Negative
	Total (max 100)			60

[0134] Reputation system 236 may determine reputation sub-scores for network entities 102. The reputation sub-score for a network entity may be based on one or more reputational factors for the network entity. Each of the reputational factors for the network entity is a function of specified historical events involving the network entity. The reputational factors for a network entity may be user-defined. The historical events may be defined to be those events that occur within a specific time period (e.g., 5 weeks, 10 weeks, 6 months, etc.). Reputation system 236 may store the reputation sub-scores for network entities 102 as scoring data 230. Example types of reputational factors may include:

• • [0135] A number of temporal intervals (e.g., weeks, days, etc.) without a reset of the network entity.
  • [0136] A number of temporal intervals (e.g., weeks, days, etc.) without a reboot of the network entity.
  • [0137] A number of failed log-ins.
  • [0138] Log-in frequency.
  • [0139] Configuration frequency.
  • [0140] A number of protocol timeouts or other runtime data

[0141] Table 2, below, illustrates an example of how reputation system 236 may determine the reputation sub-score for a network entity. For instance, in Table 2, reputation system 236 may increase a reputation sub-score for the network entity by 5 for each month in which the network entity did not experience a reset. Furthermore, in Table 2, reputation system 236 may increase the reputation sub-score for the network entity by 15 for each month in which the network entity did not experience a reboot. Because the considered time period is 5 months, the maximum reputation sub-score, in this example, is 100.

[0142]

TABLE 2
	Factor	Choices	Contribution
Relative weights for	Reputation per period	1-5	5
items within	without reset
reputation	Reputation per period	1-5	15
	without reboot
	Interval	Month
	Length of history	5
	considered
	Maximum		100

[0143] Table 3, below, illustrates an example of how subjective system 233 may determine the subjective sub-score for a network entity. For instance, in Table 3, subjective system 233 may determine, based on information accessed from input systems 103, information about various subjective factors (e.g., examples listed in Table 3 include reliability, uptime, and flexibility). Such information may correspond to a score representing a rating for relevant network entity for each factor (e.g., the network entity addressed in Table 3 has a reliability rating of 90 out of 100). subjective system 233 may also determine the weights that are to be applied to each of the subjective factors, with the contribution for each factor to the subjective sub-score corresponding to the weight multiplied by the rating. For example, in Table 3, since the “reliability” factor is (for a given stakeholder or stakeholder role) is 60%, the reliability factor contributes 54 to the overall subjective sub-score. Note that factors having a weight of 0% effectively are removed from consideration when calculating the subjective sub-score.

[0144]

TABLE 3
		Score	Weight
	Factor	(0-100)	(0-100%)	Contribution
Contributions	Reliability	90	60%	54
within subjective	Uptime	30	0%
trust	Flexibility	50	40%	20
	Total (max 100)		100%	74

[0145] Determining the weights applied in FIG. 3 may be the result of capturing information that reflects the priorities of the stakeholder that is expected to use or apply the resulting trust score. One way to capture such information may be through a stakeholder survey. A non-limiting example of such a survey is outlined below:

• • [0146] 1. What is your role within the organization (e.g., Network Operations Centre (NOC) staff, network planner, field technician, end user, customer) ?
  • [0147] 2. In your opinion, which of the following factors are the most important in determining the trustworthiness of a network or system?(Select all that apply)
    • [0148] Reliability
    • [0149] Uptime
    • [0150] Ability to quickly and effectively resolve issues
    • [0151] Scalability
    • [0152] Flexibility
    • [0153] Ability to integrate with other systems
    • [0154] Ease of maintenance
    • [0155] Robustness
    • [0156] Availability of technical support
    • [0157] Privacy and security
    • [0158] Quality of the network
    • [0159] Level of customer support
    • [0160] Overall value for money
  • [0161] 3. Please rank the factors you selected in the previous question in order of importance, with 1 being the most important and [number of factors selected] being the least important.
  • [0162] In doing this consideration would need to be given to how scores would be assigned against factors.
  • [0163] 4. Are there any other factors that you consider important in determining the trustworthiness of a network or system that were not listed above?Please describe.
  • [0164] 5. How do you currently assess the trustworthiness of a network or system?
  • [0165] 6. In what ways do you think a trustworthiness calculation that takes into account subjective factors would be useful or beneficial for your role within the organization?
  • [0166] 7. Is there anything else you would like to add about the importance of trustworthiness or how it is assessed in your role within the organization?

[0167] Other techniques or processes could be used to collect information about appropriate weights to apply to subjective factors for various stakeholders. Such techniques may include focus groups, interviews, online forums, customer feedback forms, and user experience (UX) research. Preferably, such techniques enable stakeholders to openly discuss their values, priorities, and concerns, and to provide detailed and specific feedback about their experience with the network or system.

[0168] An iterative approach to gathering subjective factors may involve the output of a baseline trustworthiness calculation. By comparing the trustworthiness calculation, and the factors which were scored in previous survey responses, the stakeholders can further refine the importance of certain factors when presented with the impacts of them, as laid out in the trustworthiness report.

[0169] Calculation system 238 may use one or more of a prerequisite sub-score for a network entity, a variable factor sub-score for the network entity, a reputation sub-score for the network entity, and/or a subjective factor sub-score to determine the trust score for the network entity. When calculating a trust score for a network entity, calculation system 238 may determine that the trust score for the network entity indicates no trust if one or more of the prerequisite conditions are not satisfied (e.g., if the prerequisite sub-score has the minimum value, such as zero). In some examples, calculation system 238 determines the trust score for a network entity as a sum of the variable factor sub-score and the reputation sub-score. In some examples, calculation system 238 determines the trust score for a network entity as a weighted sum of the variable factors sub-score, the reputation factors sub-score, and the subjective factors sub-score. For instance, calculation system 238 may apply a 60% weight to the variable factors sub-score, a 30% weight to the reputation factors sub-score, and a 10% weight to the subjective factors sub-score. In some examples, the weights applied to the variable, reputation, and subjective factors sub-scores are user-defined. Thus, in such an example, because the weights applied to each sub-score are user-defined, the ultimate trust score may be dependent on a user's priorities and preferences. In some examples, calculation system 238 may multiply the resulting sum (or weighted sum) by the prerequisite sub-score to determine the trust score for the network entity. In some examples, rather than trust controller 104 calculating a prerequisite sub-score, calculation system 238 may simply set the trust score for a network entity to the minimum value (e.g., zero) if any of the prerequisite conditions are not satisfied. Calculation system 238 may store the trust scores for network entities 102 as scoring data 230. In some examples, the trust scores for network entities 102 may include one or more trust score results 500. Such trust score results 500 may also be stored as scoring data 230.

[0170] Preferably, and to provide full traceability for trust score calculations, trust controller 104 may maintain a record of any subjective (or other) factor values that contributed to a trust calculation. This may include references to each factor, the value and the collection instance (for example a stakeholder survey) which may, in turn, include the date and a reference to the stakeholder. To reflect the changing needs of the customer, the stakeholders may be asked to provide updated responses to the survey (or select from pre-defined criteria), thereby enabling trust score assessments to remain relevant and useful.

[0171] In some examples, scoring system 222 may aggregate the trust scores for two or more of network entities 102 to determine a trust score for a higher-order network entity. Example types of higher-order network entities may include networks, services, routes, and/or other collections of one or more network entities. Scoring system 222 may aggregate the trust scores in one of a variety of ways. For example, scoring system 222 may aggregate the trust scores for two or more of network entities 102 to determine a trust score for a higher-order network entity by taking the lowest trust score of the two or more network entities as the trust score for the higher-order network entity. In another example, scoring system 222 may aggregate the trust scores for two or more of network entities 102 to determine a trust score for a higher-order network entity as an average or sum of the trust scores for the two or more network entities.

[0172] Action system 224 may perform one or more actions based on the trust scores determined for network entities 102 by scoring system 222. For example, action system 224 may modify a traffic pattern of computer network 100 based on the trust score for the network entity. For instance, in this example, based on the trust score for the network entity, action system 224 may change the traffic pattern in computer network 100 to divert network traffic away from or direct the network traffic to the network entity. For instance, if the trust score for a network entity is below a specific threshold, action system 224 may change the traffic pattern in computer network 100 to divert network traffic away from the network entity. In some examples, if the trust score for a network entity is above a specific threshold, action system 224 may change the traffic pattern in computer network 100 to direct network traffic to the network entity. In such an example, based on a trust score of a network entity being greater than a particular threshold, action system 224 may change the traffic pattern by sending instructions to one or more routers in computer network 100 that cause the routers to change routes in computer network 100, and thereby control the operation of such routers. For example, such control may cause those routers to route packets so that packets are directed along routes that lead to or through the network entity.

[0173] In another example, action system 224 may determine that there is a conflict of information between two sensors (e.g., in the context of an airplane, an angle of attack sensor and a nose pitch sensor). In this example, action system 224 may perform one or more actions based on the trust scores for the sensors (i.e., network entities) in response to determining such a conflict. For instance, action system 224 may generate a notice to an operator recommending use of information from the sensor with a greater trust score. In another example, action system 224 may disable the sensor with the lower trust score. In some examples, action system 224 may recommend a probe or obtain a combination of other sensor data to determine which of the two sensors is more likely to be generating accurate data. In an example where the network entities include multiple clock sources (e.g., Precision Time Protocol (PTP) clock sources), action system 224 may change a master clock source from a current clock source to a back-up clock source due to the back-up clock source having a greater trust score than the current clock source. In some examples, action system 224 may identify routes through network 100 based on trust scores for network entities in network 100. Thus, action system 224 may configure routers (and thereby control the operation of such routers) to route network traffic along routes that pass-through network entities having high trust scores.

[0174] Action system 224 may also perform other types of actions based on the trust scores determined for network entities 102 by scoring system 222. For instance, action system 224 may determine or adjust configurations associated with firewall 192 based on a trust score for network entity 102A that is below a specified trust threshold. Such an adjustment may minimize or limit ports on firewall 192 that enable interaction with network entity 102A. In another example, action system 224 may interact with one or more of access control system 193 to limit access to devices with trust scores below a threshold. Action system 224 may interact with asset management system 194 to mark one or more devices as unavailable when a trust score is below a certain threshold, or where the trust score is trending downward. Action system 224 may also raise alarms to indicate that trust scores associated with one or more of network entities 102 are low, are trending lower, and/or have trended below a given threshold.

[0175] Trust plan system 227 may perform functions relating to identifying, interpreting, and/or applying one or more trust score plans 400 when calculating trust scores and/or generating trust score results 500. Trust plan system 227 may use one or more trust score plans 400 in order to determine how to calculate a trust score. Trust plan system 227 may be capable of parsing trust score plans 400 to identify contribution classes, trust factors, contribution values, and contribution factors. Trust plan data store 237 may be primarily maintained by trust plan system 227.

[0176] Tracking system 226 may track changes in trust scores and/or sub-scores and reevaluate the level of trust associated with one or more entities. Tracking system 226 may collect information from source data 228 and determine changes in trust scores and/or sub-scores associated with one or more network entities 102. Tracking system 226 may determine, based on historical information about trust scores for each of network entities 102, which of network entities 102 need service, such as an upgrade, audit, review, test, or other evaluation. Tracking system 226 may monitor sentiment associated with one or more network entities 102, and adjust trust scores as that sentiment changes. In some examples, tracking system 226 may also identify one or more network entities 102 having consistently high trust scores and implement a tendency to use such network entities 102 for high priority, high importance, or mission-critical tasks.

[0177] Inference system 221 may perform functions relating to inferring trust scores for a given entity based on a distance, separation, or locality of that entity from the perspective of another entity. Inference system 221 may apply this inferred trust as a component of trust or as a degree of trust. In some examples, inference system 221 may be used to adjust (and thereby improve) trust scores determined by scoring system 222. In such an example, inference system 221 may apply an adjustment to a trust score for an entity that is based on a degree of separation between an entity and another entity whose trust is being assessed, where the degree of separation corresponds to a distance, a count of intervening entities, or a locality associated with the entity being assessed.

[0178] For example, in FIG. 1A, if network entity 102E has a trust score of 70, network entity 102C, which is adjacent to network entity 102E, may view network entity 102E as having a trust score of 70. If network entity 102C has trust score 80, then network entity 102A, which is adjacent to network entity 102C, may view network entity 102C as having a trust score of 80. However, the trust score that network entity 102A associates with network entity 102E may be a function of the trust scores for network entity 102C, network entity 102E, and the number of intervening network entities between network entity 102A and 102E. Such a function may be described as follows:

• • [0179] trust score of 102A=function(trust score of 102C, trust score of 102E, count of intervening entities between 102A and 102E)

[0180] Where the count of intervening entities is high, the trust score may be reduced. Where the count is low, the trust score may be higher.

[0181] In other examples, inference system 221 may be integrated into scoring system 222 so that calculation system 238 calculates a trust score by considering inferences about trust amongst network entities 102.

[0182] In an example where inference system 221 or aspects of inference system 221 are integrated into scoring system 222, variable factor system 234, reputation system 236, and/or subjective system 233 may use trust inferences or information derived from trust inferences to make variable and reputational assessments, respectfully. Variable factor system 234 may recalculate trust scores, for example, as new entities get added and new trust inferences are determined. Reputation system 236 may assess whether trust inferences between entities improve or deteriorate over time. Subjective system 233 may change assessments of subjective factors as collective subjective scores vary overtime. Such assessments may normalize subjective scores if, for example, subjective for various network entities 102 tend to increase over time (e.g., as devices become more reliable over time, such normalization may enable subjective reliability scores from different time periods to be effectively compared). In such an example, Tables 1, 2, and 3, described above, might be modified and relabeled as Tables 1A, 2A, and 3A as follows:

[0183]

TABLE 1A
	Characteristics	Contribution	Choices	Sample Score
Contributions	Software (max	30	Class 1 apps only	30
within variable	50%)	20	Class 1 and Class 2 apps only
trust		10	Class 1, Class 2, and Class 3
			apps
	Is a known-good	40	Yes
	release	0	No	0
	Software version <=	10	Yes	10
	last-3	0	No
	Sentiment (max	20	Positive	20
	20%)	5	Neutral
		0	Negative
	Change in degree	0-N	Range	4
	of separation
	Total			64

[0184]

TABLE 2A
	Factor	Choices	Contribution
Relative weights for	Reputation per period		5
items within	without reset
reputation	Reputation per period		15
	without reboot
	Interval	Month
	Length of history	5
	considered
	Change in neighbor	3	4
	score
	Maximum		100

[0185]

TABLE 3A
		Score	Weight
	Factor	(0-100)	(0-100%)	Contribution
Contributions	Reliability	90	60%	54
within subjective	Uptime	30	0%
trust	Flexibility	50	40%	20
	Change in	0	0%	0
	neighbor score
	Total (max 100)		100%	74

[0186] Scoring system 222 may use trust inferences or information derived from such trust inferences to supplement existing metrics, thereby improving accuracy of scoring system 222. In general, inference system 221 may receive information from and output information to one or more other systems or modules within storage devices 216, and may otherwise interact with and/or operate in conjunction with one or more other systems and/or modules of trust controller 104.

[0187] It is important to ensure that the calculation of trustworthiness is objective and unbiased, even when subjective factors are being considered. A few strategies can be employed to address potential biases in the calculation:

• • [0188] 1. Use a diverse set of subjective factors: By considering a diverse range of subjective factors, the calculation can better reflect the views and values of different stakeholders. This can help reduce the influence of any one individual or group and minimize potential biases.
  • [0189] 2. Use a transparent and fair process for collecting and incorporating subjective factors: Clearly defining and communicating the process for collecting and incorporating subjective factors will help ensure that the process is transparent and fair, and that stakeholders understand how their input is being used.
  • [0190] 3. Regularly review and revise the calculation: As the organization's needs and priorities change, it may be necessary to review and revise the calculation to ensure that it remains relevant and useful. This can also be an opportunity to assess and address any potential biases in the calculation.
  • [0191] 4. Use multiple sources of data: Incorporating multiple sources of data can help provide a more complete and balanced view of trustworthiness and reduce the influence of any one source of data. This can include both objective and subjective data sources.

[0192] If subjective factors are defined incorrectly or inappropriately, prompt and effective remediation may be important. The first step may be to clarify the definition of subjective factors and ensure all stakeholders agree on the definition. This may include providing additional training or documentation.

[0193] Once the correct definition of subjective factors has been established, any previous evaluations that the incorrect definition may be reviewed and corrected. Going forward, transparent processes for evaluating subjective factors may be employed. This may include creating checklists or guidelines to ensure that all relevant factors are being considered. A peer review process may help ensure that multiple perspectives are being taken into account.

[0194] It may also be appropriate to occasionally, periodically, or continuously monitor the evaluation process to ensure that subjective factors are evaluated consistently and fairly. This may involve conducting regular audits or assessments to identify any biases or inconsistencies in the evaluation process and taking steps to address them.

[0195] Creating a culture of openness and transparency around the evaluation process may also foster appropriate and productive subjective evaluations, and may involve encouraging feedback from all stakeholders, including customers, employees, and vendors, and being willing to change the evaluation process based on that feedback.

[0196] To the extent that a stakeholder or other actor intentionally seeks to skew subjective factors to influence any decision-making process, it may be appropriate to address such attempts by ensuring transparency and accountability in the decision-making process. This could involve establishing clear criteria for evaluating network devices and documenting the factors considered in making a decision.

[0197] Another approach is to involve multiple stakeholders in the decision-making process. This could include representatives from different departments within the telecom operator and external experts or consultants. Bringing in diverse perspectives makes it more difficult for any one person to manipulate the decision-making process. Establishing procedures to enable report of any suspected instances of bias or unethical behavior may also be appropriate. This could involve establishing a hotline or reporting mechanism allowing stakeholders, users, or employees to anonymously report concerns. Finally, ongoing training and education can promote awareness and understanding of the importance of objective decision-making processes.

[0198] Various systems illustrated in FIG. 2 (e.g., data collection system 220, inference system 221, scoring system 222, action system 224, tracking system 226, prerequisite system 232, variable factor system 234, reputation system 236, calculation system 238) may be implemented as modules or other logic. In such an example, such modules illustrated or described elsewhere in this disclosure may perform operations described using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and/or executing at one or more computing devices. For example, a computing device may execute one or more of such modules with multiple processors or multiple devices. A computing device may execute one or more of such modules as a virtual machine executing on underlying hardware. One or more of such modules may execute as one or more services of an operating system or computing platform. One or more of such modules may execute as one or more executable programs at an application layer of a computing platform. In other examples, functionality provided by a module could be implemented by a dedicated hardware device.

[0199] Although certain modules, data stores, components, programs, executables, data items, functional units, and/or other items included within one or more storage devices may be illustrated separately, one or more of such items could be combined and operate as a single module, component, program, executable, data item, or functional unit. For example, one or more modules or data stores may be combined or partially combined so that they operate or provide functionality as a single module. Further, one or more modules may interact with and/or operate in conjunction with one another so that, for example, one module acts as a service or an extension of another module. Also, each module, data store, component, program, executable, data item, functional unit, or other item illustrated within a storage device may include multiple components, sub-components, modules, sub-modules, data stores, and/or other components or modules or data stores not illustrated.

[0200] Further, each module, data store, component, program, executable, data item, functional unit, or other item illustrated within a storage device may be implemented in various ways. For example, each module, data store, component, program, executable, data item, functional unit, or other item illustrated within a storage device may be implemented as a downloadable or pre-installed application or “app.” In other examples, each module, data store, component, program, executable, data item, functional unit, or other item illustrated within a storage device may be implemented as part of an operating system executed on a computing device.

[0201] FIG. 3 is a conceptual diagram illustrating application of one or more trust score plans to calculate a trust score result for a specific network entity, in accordance with one or more aspects of this disclosure. In FIG. 3, multiple trust score plans 400 are shown, each of which describe information about the plan, including how a trust score is to be calculated for the entity pursuant to that plan. In general, each trust score plan 400 includes descriptions of trust score factors and trust score contributions that are relevant and/or used by that plan. Each of trust score plans 400 may be used by trust controller 104 to perform a trust score calculation for an entity, resulting in trust score result 500. To perform the trust score calculation, trust controller 104 uses trust score values associated with the network entity, where that information is supplied by one or more source services 303. In some examples, source services 303 may correspond to or be examples of input systems 103 illustrated in FIG. 1A.

[0202] In some examples, for each entity, such as for each of network entities 102 in FIG. 1A, a default trust score plan 400 may be available that is used when performing trust score calculations for that entity. Where the network entity 102 is a network device, for example, such a default trust score plan 400 might be provided by the manufacturer of the network device. Such a default plan may be designed by the manufacturer for expected use cases and/or may be based on specifications, rules, and/or “workbench” documents published by a standards organization.

[0203] In other examples, rather than using a default plan, a customized trust score plan 400 may be used that more accurately meets the needs of the network or the user's environment. For example, for the network entities 102 illustrated in FIG. 1A, a network administrator may choose specific trust score plans 400 for one or more of network entities 102 of FIG. 1A. By customizing the trust score plans 400 that are applied to each of network entities 102A in FIG. 1A, the trust score calculations can be appropriately tailored to a given network, context, or environment. In some examples, a customer classification value may control which trust score plan is used. For example, if a customer is classified as a “health” type of customer, a first trust score plan may be used, while different trust score plans may be used for customers classified as “payment systems” or “government.” Such a trust score plan may specify the type of subjective assessments are associated with a given network entity 102. In some examples, various subjective assessments (reliability, flexibility, privacy) may apply to some network entities 102 but not others.

[0204] In accordance with one or more aspects of the present disclosure, trust controller 104 may associate one or more trust score plans 400 with one or more network entities 102. For instance, in an example that can be described with reference to FIG. 1A, FIG. 2, and FIG. 3, input device 208 detects input and outputs information about the input to trust plan system 227. Trust plan system 227 determines that the input corresponds to information identifying a trust score plan 400 that is to be associated with one or more specific network entities 102. For example, the input may indicate that trust score plan 400A may apply to network entity 102A, potentially overriding any default trust score plan that might otherwise be used for network entity 102A. The same input or other input may indicate that a different trust score plan 400 may apply to network entity 102B (again, overriding any default trust score plan 400). Trust plan system 227 stores, within trust plan data store 237, information about which of trust score plans 400 should be applied to various network entities 102 when generating trust score results 500 for such network entities 102.

[0205] In general, a different trust score plan 400 could be associated with each of network entities 102 in FIG. 1A. However, as suggested in connection with FIG. 1A, one trust score plan 400 could also apply to multiple devices, particularly devices that are similar, have a similar function, and/or are from the same manufacturer (e.g., routers supplied by the same manufacturer may use a common trust score plan 400). Further, certain network entities 102 may have multiple trust score plans 400 that are applied in various or different contexts, as further described below.

[0206] Trust controller 104 may collect data. For instance, still with reference to FIG. 1A, FIG. 2, and FIG. 3, each of input systems 103 or source services 303 collect data for one or more specific network entities 102 in network 100, where, each of input systems 103 or source services 303 collect information that corresponds to factor values that might be used by trust score plans 400 when calculating trust scores for one or more of network entities 102. Communication unit 204 of trust controller 104 detects input and outputs information about the input to data collection system 220. Data collection system 220 determines that the input corresponds to information (e.g., factor values, sentiment information, subjective assessment information, and/or compliance, vulnerability, and integrity data). Data collection system 220 evaluates the data to identify which of the network entities 102 the information applies to, and which of input systems 103 or source services 303 is supplying the data. Data collection system 220 may also determine, based on the data, the set of contribution values being reported, along with the identity of the contribution factor associated with the values. In some examples, trust controller 104 may expose an application programming interface that each of source services 303 can use to report factor values. Such an API might include a method for posting factor values that has the form:

[0207]

	// PostFactorValues posts one or more Factor values defined by
	values for the entity identified by entityID, from the system
	identified by sourceID.
	PostFactorValues(entityID string, sourceID string, values
	[ ]*ValuePost)
	// ValuePost associates a Factor with a value.
	type ValuePost struct {
	// FactorRef identifies a trust factor.
	FactorRef string
	// Value holds the value to be posted for the Factor.
	Value interface{ }
	}

[0208] In general, data collection by input systems 103 and source services 303 and reporting of such data to data collection system 220 of trust controller 104 may occur independently of any association of a trust score plan 400 with each of network entities 102. In other words, source services 303 may collect data prior to, during, or after a trust score plan 400 for a given network entity 102 is selected, changed, or customized.

[0209] Trust controller 104 may receive a trigger to generate trust score result 500. For instance, again with reference to FIG. 1A, FIG. 2, and FIG. 3, input device 208 of trust controller 104 detects a signal that scoring system 222 determines corresponds to a command, trigger, or request to calculate a trust score for a specific network entity 102, such as network entity 102A. In some examples, the command, trigger, or request may result from an interaction with a UI (e.g., by a user or administrator). In other examples, the command, trigger, or request may result from a scheduled request, or an event occurring elsewhere within network 100 of FIG. 1A or from a different network.

[0210] Where the trigger results from a UI interaction, scoring system 222 may receive an indication of a role associated with the user or administrator performing the UI interaction. scoring system 222 may use that role to determine a set of default weights that can be used to calculate a subjective factor sub-score. In some examples, the set of default weights may indicate the weights that are to be applied to each of the subjective factors when calculating the subjective sub-score (see, e.g., Tables 3 and 3A).

[0211] In other examples, rather than receiving information about a role that can be used to identify subjective factor weights, scoring system 222 may receive information about the weights that should be used for a specific user, which may override some or all of the default weights that may be associated with the user's role. In such an example, the weights may be derived from input received from the user during the UI interaction, or during an earlier time period. For example, when a new user account is created, trust controller 104 may present a Stakeholder Survey on initial login, gather subjective factors from the user or stakeholder, and thereafter use this information to tailor the score specifically for the user. Alternatively, or in addition, the user might select from a pre-defined set of criteria relevant to their role.

[0212] Trust controller 104 may select an appropriate trust score plan 400 to use when generating trust score result 500 for network entity 102A. For instance, still referring to FIG. 1A, FIG. 2, and FIG. 3, trust plan system 227 of trust controller 104 accesses information within trust plan data store 237. Based on the accessed information, trust plan system 227 identifies a specific trust score plan 400 for network entity 102A. In some examples, trust plan system 227 may determine that a default trust score plan 400 is appropriate for use in generating trust score result 500. In other examples, trust plan system 227 may determine that a customized trust score plan 400, such as one generated in response to administrator input, is more appropriate for use in generating trust score result 500. To determine which of trust score plans 400 to use for network entity 102, trust plan system 227 may evaluate contextual information included within the command, trigger, signal, or request received by trust controller 104. Such contextual information may identify the context or circumstances in which the trigger was generated, and trust plan system 227 may use this information when identifying an appropriate trust score plan 400 to use when generating trust score result 500 for network entity 102A.

[0213] In addition, trust controller 104 may, in some cases, use historical information to evolve the content of the trust score plan used to calculate trust score result 500. Alternatively, or in addition, trust controller 104 may use such information to evolve, adjust, or configure the manner in which an appropriate trust score plan is selected. Machine learning techniques may be applied to train a model to edit an existing trust score plan, generate a new trust score plan, or to modify which of an available set of trust score plans is selected in varying circumstances.

[0214] Trust controller 104 may calculate sub-scores for use when generating trust score result 500 for network entity 102A. For instance, continuing with the example being described, trust plan system 227 outputs information about the selected trust score plan 400 to scoring system 222 of trust controller 104. Scoring system 222 uses the information about the selected trust score plan 400 to identify trust score factors relevant to the selected trust score plan 400. Scoring system 222 accesses information about the relevant trust score factors within source data 228. The information accessed within source data 228 may include information about trust score values reported to trust controller 104 by one or more source services 303, as described above.

[0215] Prerequisite system 232 of scoring system 222 assesses the prerequisites specified in the selected trust score plan 400. Prerequisite system 232 determines whether source data 228 associated with network entity 102A satisfies the prerequisites identified in the selected trust score plan 400. If the data indicates that network entity 102A does not satisfy the prerequisites, scoring system 222 may conclude that the trust score for network entity 102A is zero, and may generate trust score result 500 reflecting such a score. If the data indicates that network entity 102A does satisfy the prerequisites, scoring system 222 may perform further calculations to generate trust score result 500, including determining variable reputational, and subjective factor sub-scores.

[0216] To calculate a variable factor sub-score, variable factor system 234 identifies the contribution factors specified in the variable factor category of the selected trust score plan 400. For instance, as described in connection with FIG. 2, variable factor system 234 may apply appropriate weights (as specified in the selected trust score plan 400) to different trust factors (e.g., software characteristics, sentiment—also specified in trust score plan 400) to determine the variable factor sub-score for network entity 102A. Variable factor system 234 may store information about the variable factor sub-score for network entity 102A in scoring data 230.

[0217] To calculate a reputational factor sub-score, reputation system 236 identifies the contribution factors specified in the reputational factor category of the selected trust score plan 400. Reputation system 236 may apply the specified weights (specified within the selected trust score plan 400) to the reputational factors identified within the selected trust score plan 400 to determine the reputational factor sub-score for network entity 102A. Reputation system 236 may store information about the reputational factor sub-score for network entity 102A in scoring data 230.

[0218] To calculate a subjective factor sub-score, subjective system 233 identifies the subjective factors specified in subjective factor category of the selected trust score plan 400. For instance, as described in connection with FIG. 2, subjective system 233 may apply appropriate weights (as specified by the user's role, or by a specific user's preferences) to different subjective scores (e.g., reliability, flexibility, ease of maintenance, level of customer support) to determine the subjective factor sub-score for network entity 102A. Subjective system 233 may store information about the subjective factor sub-score for network entity 102A as scoring data 230. In some examples, information about weights that could apply to various roles may bin included within trust score plan 400.

[0219] Trust controller 104 may generate trust score result 500 for network entity 102A. For instance, continuing again with the example being described in the context of FIG. 1A, FIG. 2, and FIG. 3, calculation system 238 of scoring system 222 accesses information about the variable factor sub-score, the reputational factor sub-score, and the subjective factor sub-score within scoring data 230. Calculation system 238 uses these sub-scores to generate trust score result 500. Calculation system 238 stores information about trust score result 500 within scoring data 230 of trust controller 104.

[0220] In generating trust score result 500, calculation system 238 may calculate a specific numerical trust score value based on the sub-scores and information included in the selected trust score plan 400. For example, as described herein, the contribution values included within the selected trust score plan 400 are associated with the trust scores identified in the selected trust score plan 400. The contribution values are used to define the contribution of each factor to the calculated trust score. How the contribution values are used may depend on the type of trust factor with which they are associated. For example, a trust factor may have an implied maximum contribution and an actual contribution. A percentage score for a factor category is determined as follows.

[0221] $\mathrm{factor}\mathrm{category}\mathrm{sub}‐\mathrm{score}=\left(\mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{actual}\mathrm{contributions}\mathrm{for}\mathrm{each}\mathrm{of}\mathrm{the}\mathrm{factors}\mathrm{in}\mathrm{the}\mathrm{category}/\mathrm{sum}\mathrm{of}\mathrm{the}\mathrm{maximum}\mathrm{contributions}\mathrm{for}\mathrm{each}\mathrm{of}\mathrm{the}\mathrm{factors}\mathrm{in}\mathrm{the}\mathrm{category}\right)*100$

[0222] A simplified version of a trust score calculation that uses factor category sub-scores may have the form of:

[0223] $\mathrm{trust}\mathrm{score}=\left(\mathrm{prerequisites}\mathrm{sub}‐\mathrm{score}\right)*\left(\mathrm{variable}\mathrm{factor}\mathrm{sub}‐\mathrm{score}/100*\mathrm{plan}.\mathrm{VariableWeighting}\right)+\left(\mathrm{reputational}\mathrm{factor}\mathrm{sub}‐\mathrm{score}/100*\mathrm{plan}.\mathrm{ReputationalWeighting}\right)+\left(\mathrm{subjective}\mathrm{factor}\mathrm{sub}‐\mathrm{score}/100*\mathrm{plan\_SubjectiveWeighting}\right))$

[0224] In the calculation above, the overall percentage score for each of the variable and reputational categories may be adjusted according to the category contributions (e.g., as defined in the selected trust score plan 400). Also in the above calculation, the prerequisite sub-score will be either zero (prerequisites not satisfied) or one (prerequisites satisfied). Where the prerequisites sub-score for network entity 102A is zero, the trust score for entity network entity 102A will generally be zero. If the prerequisites are satisfied, however, the trust score for network entity 102A will be the above-described function of the variable, reputational, and subjective factor sub-scores, as defined by the selected trust score plan 400.

[0225] Trust score result 500 may include trust score report 501, trust score supporting document 502, and one or more factor value reports 503. Trust score report 501 may include the calculated trust score (e.g., calculated as described above), as well as information about the variable, reputational, and subjective sub-scores. Trust score supporting document 502 may identify the trust score plan used to calculate the trust score result and/or may include information about how the trust score was calculated, enabling an administrator to determine and/or identify the underlying data that was used to calculate the trust score. In some examples, trust score supporting document 502 includes references to one or more factor value reports 503. Factor value reports 503 may be from different sets of source services 303, which may correspond to compliance, integrity, and/or vulnerability source services 303. In some examples, each of factor value reports 503 may provide the factor values used to calculate the trust score and generate trust score result 500. The applicable trust score plan 400 along with one or more of these factor value reports 503 may enable an administrator to trace or determine how the trust score is calculated, which may be useful for adjusting trust score plans 400 for one or more of network entities 102, for debugging purposes, for troubleshooting, and/or for verifying system integrity.

[0226] Trust controller 104 may act on trust score result 500. For instance, again referring to the example being described in the context of FIG. 1A, FIG. 2, and FIG. 3, action system 224 accesses trust score result 500 from within scoring data 230. Action system 224 determines that based on trust score result 500, adjustments should be made to certain aspects of network 100. For example, action system 224 may cause communication unit 204 of trust controller 104 to output a signal over network 100. One or more devices or system on network 100 detect the signal. In one example, firewall 192 detects the signal and determines that the signal corresponds to a command to adjust settings associated with firewall 192. Firewall 192 adjusts its settings as specified by information included within the signal, or as appropriate based on other information available to firewall 192. In another example, alarm system 195 may detect the signal and make adjustments to the sensitivity of one or more alarms that are configured for network 100.

[0227] FIG. 4 illustrates an example trust score plan, in accordance with one or more aspects of the present disclosure. Trust score plan 400, illustrated in FIG. 4, represents one possible format for information that may be included in a trust score plan, and might be an example of any of trust score plans 400A through 400G illustrated in FIG. 1A, or trust score plans 400 described in connection with FIG. 2 and/or FIG. 3. Although trust score plan 400 of FIG. 4 is shown using a YAML representation of a trust score plan, other formats are possible, including JSON (JavaScript Object Notation), XML (eXtensible Markup Language), any other domain-specific language, or any other suitable format. In general, any appropriate method of formatting, representing, or otherwise storing information pursuant to a trust score plan may be used. A trust score plan could be represented in a format that is primarily machine-readable (e.g., a binary format) or in a format that is convenient for human review (e.g., as in the YAML text-based format shown in FIG. 4). Formats that are text-based, or that are otherwise easily edited by a human user or administrator, may be preferable, since those formats may facilitate changes to the trust score plan (e.g., by an administrator) in response to varying conditions, needs, or environments.

[0228] In some examples, trust controller 104 may provide the capability for administrators or other uses to edit trust score plans and customize trust score plans for specific environments. Trust controller 104 may, in some examples, enable the underlying text (e.g., YAML or JSON text) to be accessible to an administrator for editing the domain specific language in which the trust score plan is written. Trust controller 104 may, in other examples, present a graphical or other user interface to such an administrator. In some cases, such a user interface (or an API to such a user interface) might not require knowledge of the specifics of the domain specific language in which the trust score plans are written, thereby enabling a more accessible way to create a trust score plan or customize an existing trust score plan.

[0229] The example trust score plan 400 of FIG. 4 defines how to calculate a trust score for a specific entity (e.g., a “Junos” device), which may be a network device, such as a router, switch, compute node or other device or system. Included within trust score plan 400 shown in FIG. 4 is a listing of the names of various trust factors (e.g., “vul:junos:advisory_severity” or “periods_since_restart”) each categorized into appropriate trust factor categories (“prerequisite,” “variable,” “reputational,” and “subjective”). For each factor, a factor “contribution” value is provided.

[0230] Trust factors may vary according to the type of data required to express a value for the factor. For example, trust factors may be Boolean, an enumeration, an integer, decimal, or other form.

[0231] The value of a Boolean Factor may be expressed as either true or false. For example, the value of a factor may express whether an entity is using a certified or approved software version, or whether the entity is using a version that falls within the most recent three versions of the software. Factors in the prerequisite category generally would consist of Boolean factors, and any false value for a trust factor categorized in the prerequisite factor category would result in trust score of zero. When used in the variable factor category, a Boolean trust factor would generally add its contribution value to the trust score if the trust factor value is true. The maximum contribution value of a Boolean Factor is its contribution value.

[0232] The value of an enumeration factor may be expressed as one of a fixed set of predefined values. An example of an enumeration trust factor value might relate to an advisory label for software or for a device. Such labels might include “none,” “low,” “medium,” “high,” and “critical” labels, which might correspond to severity levels. A trust score plan that includes a trust factor expressed as an enumeration may specify a contribution value for each predefined value (e.g., in the example above, such values might be 100, 75, 50, 25, and 0, corresponding to “none” through “critical,” respectively). The maximum contribution value of an enumeration factor is the maximum contribution value associated with the predefined values (e.g., the maximum contribution value would be 100 in the example being described).

[0233] The value of a sub-score factor may be expressed as a percentage and may represent a score calculated by a contributing source service (e.g., one of source services 303). The percentage is used to adjust the magnitude of the value that contributes to the trust score. For example, a sub-score factor might be generated by a compliance source service 303 to reflect the results of assessing a group of rules relating to a specific area, such as a rule specified in a standards-based benchmark document. If the compliance source service 303 reports a factor value of 66, and the contribution value for that factor is 10, then the factor would contribute a value of 6.6 to the trust score. The contribution of a sub-score factor value is the sub-score value divided by 100 multiplied by the maximum contribution (e.g., sub-score/100*max_contribution). The maximum contribution value of a sub-score factor is its contribution value.

[0234] Reputational factors are used in the reputational factor category, and often, the value of a reputational factor is a count of the time periods in which an entity meets a condition defined by the factor. For example, a reputational factor may relate to the number of time periods during which a device has not been reset or has not rebooted, or the number of periods without excessive shell access. Generally, reputational factors define the amount of time for each time period (e.g., hourly, daily, weekly, monthly, etc.), the maximum number of periods to consider, and the contribution value for each period during which the condition was met. For example, of one source services 303 reports value of 3 for a specific reputational factor that has a per-period contribution value of 5, the contribution to the trust score will be 15. The maximum contribution value of a reputational factor is the product of the number of periods to consider and the per-period contribution value. In some reputational factor schemes, including at least some of those described herein, source services 303 may need to know the time period interval and the number of periods for which the count should be reported. In other schemes, one or more source services 303 might merely report events that would satisfy (or violate) the condition defined by the trust factor. Using that information, scoring system 222 of trust controller 104 could determine the number of qualifying periods, and use the information provided by each of source services 303 to determine the appropriate contribution to the trust score.

[0235] Subjective factors are used in the subjective factor category, and may correspond to assessments of subjective information, ratings, and other information that is used to inform the trust assessment. Such information may be specified in trust score plan 400 in a manner that parallels variable and reputational factors.

[0236] FIG. 5 illustrates an example trust score result, in accordance with one or more aspects of the present disclosure. Trust score result 500, illustrated in FIG. 5, may be an example of trust score results 500 referenced in FIG. 1A, FIG. 2, and FIG. 3. Trust score result 500 may be composed of a number of different documents, including trust score report 501, trust score supporting document 502 (not specifically shown in FIG. 5), and factor value report 503 (not specifically shown in FIG. 5). A trust score result 500 can, of course, be represented in other ways and in other formats.

[0237] Trust score report 501 presents a summary of the prerequisite, variable, reputational, and subjective factor category sub-scores for a specific entity. In the example of FIG. 5, trust score report 501 shows, for a device identified as “ex11,” factor category sub-scores having values of 1.00, 39.19, 0.00, and 11.00. Those sub-scores result in a trust score of 25.21.

[0238] A trust score supporting document may be a JSON document that includes references to three factor value reports (one each from the compliance, integrity, and vulnerability source services 303). In such a trust score supporting document, a unique code may identify a specific value report produced by a source service 303. A factor value report may also be a JSON document based on information reported by a compliance source service 303, and may identify the source service class and an identifier of the event that generated the factor value report (i.e., the identifier of the compliance scan). The information provided in trust score report 501, the trust score supporting document, and the factor value report can be used to determine the low-level details of the scan that accounted for the trust score calculated from the factor values.

[0239] FIG. 6 is a flow diagram illustrating operations performed by an example trust controller 104 in accordance with one or more aspects of the present disclosure. FIG. 6 is described below within the context of network 100 and trust controller 104 of FIG. 1A. In other examples, operations described in FIG. 6 may be performed by one or more other components, modules, systems, or devices. Further, in other examples, operations described in connection with FIG. 6 may be merged, performed in a difference sequence, omitted, or may encompass additional operations not specifically illustrated or described.

[0240] In the process illustrated in FIG. 6, and in accordance with one or more aspects of the present disclosure, trust controller 104 may determine a prerequisite score for a network entity in a computer network (601). For example, in FIG. 1A, trust controller 104 collects information from one or more input systems 103 about a specific network entity 102A. The information collected pertains to a condition to be satisfied in order for network entity 102A not to be a minimum value. Trust controller 104 uses the information to determine whether each of the conditions is met. Trust controller 104 calculates, based on the determination, a prerequisite sub-score for network entity 102A, which in some examples, is either zero or one.

[0241] Trust controller 104 may determine a subjective score for the network entity based on one or more subjective factors for the network entity (602). For example, trust controller 104 collects information from one or more input systems 103, where the information pertains to subjective characteristics of network entity 102A. Trust controller 104 applies weights to each of the characteristics, where the weights are customized to the consumer of the trust score to be calculated (e.g., a stakeholder). In some examples, the weights are determined based on the role of the stakeholder. Trust controller 104 calculates, based on the applied weights, a subjective sub-score.

[0242] Trust controller 104 may determine a trust score for the network entity based on the prerequisite score and the subjective score (603). For example, trust controller 104 may multiply the prerequisite sub-score and the subjective sub-score to determine a trust score. In some examples, such as where trust controller 104 also calculates variable and reputational sub-scores, trust controller 104 may multiply the prerequisite sub-score and the sum of the variable, reputational, and subjective sub-scores. Trust controller 104 generates trust score result 500B.

[0243] Trust controller 104 may modify operation of the computer network based on the trust score for the network entity (604). For example, in FIG. 1A, trust controller 104 evaluates trust score result 500, and based on the evaluation, adjusts configurations for a router or other network device. Alternatively, or in addition, trust controller 104 may re-route traffic, limit access to network entity 102A, or adjust configurations associated with access control system 193 and/or asset management system 194.

[0244] For processes, apparatuses, and other examples or illustrations described herein, including in any flowcharts or flow diagrams, certain operations, acts, steps, or events included in any of the techniques described herein can be performed in a different sequence, may be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of the techniques). Moreover, in certain examples, operations, acts, steps, or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially. Further certain operations, acts, steps, or events may be performed automatically even if not specifically identified as being performed automatically. Also, certain operations, acts, steps, or events described as being performed automatically may be alternatively not performed automatically, but rather, such operations, acts, steps, or events may be, in some examples, performed in response to input or another event.

[0245] The disclosures of all publications, patents, and patent applications referred to herein are each hereby incorporated by reference in their entireties. To the extent that any such disclosure material that is incorporated by reference conflicts with the instant disclosure, the instant disclosure shall control.

[0246] For ease of illustration, only a limited number of devices (e.g., network entities 102, trust controller 104, entities 110, quantitative entities 160, qualitative entities 170, as well as others) are shown within the Figures and/or in other illustrations referenced herein. However, techniques in accordance with one or more aspects of the present disclosure may be performed with many more of such systems, components, devices, modules, and/or other items, and collective references to such systems, components, devices, modules, and/or other items may represent any number of such systems, components, devices, modules, and/or other items.

[0247] The Figures included herein each illustrate at least one example implementation of an aspect of this disclosure. The scope of this disclosure is not, however, limited to such implementations. Accordingly, other example or alternative implementations of systems, methods or techniques described herein, beyond those illustrated in the Figures, may be appropriate in other instances. Such implementations may include a subset of the devices and/or components included in the Figures and/or may include additional devices and/or components not shown in the Figures.

[0248] The detailed description set forth above is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a sufficient understanding of the various concepts. However, these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in the referenced figures in order to avoid obscuring such concepts.

[0249] Accordingly, although one or more implementations of various systems, devices, and/or components may be described with reference to specific Figures, such systems, devices, and/or components may be implemented in several different ways. For instance, one or more devices illustrated in the Figures herein (e.g., FIG. 1A and/or FIG. 2) as separate devices may alternatively be implemented as a single device; one or more components illustrated as separate components may alternatively be implemented as a single component. Also, in some examples, one or more devices illustrated in the Figures herein as a single device may alternatively be implemented as multiple devices; one or more components illustrated as a single component may alternatively be implemented as multiple components. Each of such multiple devices and/or components may be directly coupled via wired or wireless communication and/or remotely coupled via one or more networks. Also, one or more devices or components that may be illustrated in various Figures herein may alternatively be implemented as part of another device or component not shown in such Figures. In this and other ways, some of the functions described herein may be performed via distributed processing by two or more devices or components.

[0250] Further, certain operations, techniques, features, and/or functions may be described herein as being performed by specific components, devices, and/or modules. In other examples, such operations, techniques, features, and/or functions may be performed by different components, devices, or modules. Accordingly, some operations, techniques, features, and/or functions that may be described herein as being attributed to one or more components, devices, or modules may, in other examples, be attributed to other components, devices, and/or modules, even if not specifically described herein in such a manner.

[0251] Although specific advantages have been identified in connection with descriptions of some examples, various other examples may include some, none, or all the enumerated advantages. Other advantages, technical or otherwise, may become apparent to one of ordinary skill in the art from the present disclosure. Further, although specific examples have been disclosed herein, aspects of this disclosure may be implemented using any number of techniques, whether currently known or not, and accordingly, the present disclosure is not limited to the examples specifically described and/or illustrated in this disclosure.

[0252] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored, as one or more instructions or code, on and/or transmitted over a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (e.g., pursuant to a communication protocol). In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media, which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0253] By way of example, and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, or optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection may properly be termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a wired (e.g., coaxial cable, fiber optic cable, twisted pair) or wireless (e.g., infrared, radio, and microwave) connection, then the wired or wireless connection is included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to non-transient, tangible storage media.

[0254] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the terms “processor” or “processing circuitry” as used herein may each refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described. In addition, in some examples, the functionality described may be provided within dedicated hardware and/or software modules. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0255] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, a mobile or non-mobile computing device, a wearable or non-wearable computing device, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a hardware unit or provided by a collection of interoperating hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.