Fraud score heuristic for identifying fradulent requests or sets of requests

[0001] This application incorporates by reference for all purposes the full disclosure of co-pending U.S. patent application Ser. No. 14/981,722, filed concurrently herewith, entitled “ALLOCATING DEFECTIVE COMPUTING RESOURCES FOR HONEYPOT SERVICES” and co-pending U.S. patent application Ser. No. 14/981,737, filed concurrently herewith, entitled “HONEYPOT COMPUTING SERVICES THAT INCLUDE SIMULATED COMPUTING RESOURCES.”

[0002] A honeypot, in the context of computer network security, can include software and hardware resources that are intended detect, deflect, or counteract attempts at unauthorized use of information systems. In some examples, honeypots can include simulated network resources such as simulated virtual machines, simulated storage, and the like, but in other examples, real network resources can be a part of a honeypot. Some honeypots are designed to trick malicious users into believing that they are using or have access to legitimate resources or important data, when in reality the resources are simulated or the data that the malicious user has access to is not real. Additionally, honeypots can act as a decoy for malicious users. For example, a honeypot can provide a seemingly easy or attractive intrusion point into a network that acts as a distraction from other network vulnerabilities and locations of sensitive information.

[0003] Accordingly, with malicious users believing that they have been undetected and have access to resources and information of interest, the actions of these malicious users can be tracked and neutralized. For example, by allowing malicious users to act within honeypots can allow network administrators to learn about potential security risks of a network and gain information about malicious users that can be provided to law enforcement or can otherwise be used to stop these malicious users. However, while honeypots can be valuable tools for learning about, containing and deflecting malicious users, generating honeypot computing resources for malicious users can tie up valuable computing resources that could otherwise be used by legitimate users that deserve the availability of high quality computing resources that are able to scale as necessary. Moreover, a service provider that services customers by providing such computing services will often not receive payment for computing resources used by malicious users because many malicious users will not pay their bill or will pay using fraudulent payments that will eventually need to be reimbursed or voided.

[0021] The following disclosure relates to determining whether a network user is malicious and provisioning honeypot computing services to malicious users so that they can be tracked, contained, and stopped. In various examples, providing computing services to legitimate users includes making high quality resources available to the user when the user desires and also allowing quickly scaling of resources when the need arises. Accordingly, to provide the best service to legitimate users, various techniques are employed to reserve computing resource capacity for the legitimate users by providing lower-quality computing resources (e.g., computing resources that have degraded performance as a result of needing service) to malicious users instead of allowing these malicious users to tie up precious resources that would be better allocated to legitimate users who deserve the best service.

[0022] In a virtual computing environment, for example, physical hosts are used to instantiate virtual machines for users. Where these physical hosts experience some performance degradation due to hardware faults, or the like, these defective resources can be allocated to a pool of defective resources that can be used to provide honeypot computing services to malicious users. Accordingly, cost-effective honeypots can be generated to observe and stop malicious users while maintaining a pool of top-performing resources that can be reserved for use by legitimate users.

[0023] Allocating a resource to a defective pool of resources can be triggered by receiving notification of a hardware fault associated with the resource and then determining that the resource is eligible for allocation to the defective pool. For example, the resource may not be eligible if it is so degraded that it is inoperable, or it may still be suitable for use in the general resource pool if the resources' degradation is minor. If appropriate, the resource can be allocated to the defective pool and an effective time to repair can be associated with the resource. For example, an effective time to repair can include a predicted time that it will take for a repair technician to diagnose an issue, receive and install replacement parts, or the like.

[0024] When a resource allocation request is received, such as a request to instantiate one or more virtual machine, the request can be scored for maliciousness and if the maliciousness score exceeds a threshold, then the system can determine if there are any eligible resources in the defective resource pool available to service the request and, when there are sufficient eligible resources in the defective resource pool, can allocate these defective resources to the malicious user.

[0025] Allocating these resources can include provisioning honeypot computing services that comprise real and/or simulated computing resources, including virtual hosts and memory services. Determining whether eligible resources are available to provision to the malicious user can include determining whether there are resources in the defective pool that meet the requirements of the request and whether the allocation would allow the resources to be released and available in time for scheduled for diagnosis or repair.

[0026] As a result of a malicious user having been detected, honeypot credentials (i.e., credentials tagged as honeypot credentials) can be provisioned to the user. In some examples, these honeypot credentials can be different than normal user credentials, or the credentials can be the same and the user can be flagged for honeypot credentials at the account level. Honeypot and normal (non-honeypot) credentials can distinguishable to a system that authenticates using the credentials and, as a result, the system can determine how to process a request made using the credentials. As discussed, honeypot credentials can provide an indication that some or all of a set of requested set of computing resources be providing using resources from a defective pool and/or provided in the form of a simulation where such resources are not actually provided. In various examples, regular or non-honeypot credentials can provide an indication to a system to allocate real computing resources from a general resource allocation pool, which can be the best resources available.

[0027] In some examples, various computing services such as virtual machines can be simulated with a simulation state database. Accordingly, instead of running computing services such as virtual machines when requested by the user, a simulation state database can be used to generate responses to user requests that simulate interactions with real computing resources (i.e., computing resources that perform requested/commanded operations as opposed to simulating performance of the requested/commanded operations). For example, where the user sends a mutating request while provisioned with honeypot services, the simulation state database can be updated and a response can be provided to the user that simulates a response that would be provided by real resources, such as virtual machines.

[0028] Mutating requests can include various requests to change or mutate a computing resource in some way, including instantiating the resource, updating the resource, reconfiguring the resource, terminating the resource, and the like. Similarly, where the user provides a query request, the simulation state database can be queried and a response can be provided to the user that simulates a response that would be provided to a query of real resources such as virtual machines. Query request can include various requests that do not change or mutate a computing resource in some way and instead include a request for information about a computing resource, such as the running status of the resource, name of the resource, memory allocation of the resource, and the like.

[0029] In some example techniques, user requests or sets of user requests (e.g., application program interface (API) calls), are evaluated to determine whether they are malicious or fraudulent by a fraud score heuristic. In various examples, fraud score heuristics can be a simple or complex algorithm for analyzing requests or sets of user requests and can be implemented on a hardware and/or software system. For example, a fraud score heuristic can use a pattern that corresponds to characteristics of malicious requests or sets of malicious requests.

[0030] Accordingly, in various examples, a fraud score heuristic is trained by extracting characteristics from training data that includes requests or sets of requests from malicious users. In one example, characteristics are extracted from a request or series of requests and a k-means algorithm can be used to identify one or more clusters of the characteristics, which can be represented in multidimensional space.

[0031] In addition to using known malicious requests to train a pattern or fraud heuristic, data or characteristics from non-malicious users can used as training data or as a filter for patterns that identify requests or sets of requests from malicious users. In one example, patterns for malicious and non-malicious requests or sets of requests can be identified, and the patterns for non-malicious requests or sets of requests can be subtracted from or otherwise used to bias or weight the pattern for malicious users so as to reduce the occurrence of false positives, when uncategorized user requests or sets of requests are analyzed.

[0032] As discussed in this disclosure, fraud, fraudulent use, a fraudulent user, maliciousness, malicious use, a malicious user, and similar terms can encompass actions and/or one or more users performing actions that are intended to perpetrate computer fraud, abuse, misuse, waste, access without authorization or exceeding authorized access, or related activities. Users can include one or more human operators and/or software programs such as bots, or the like. As discussed, fraud scores and fraud heuristics can relate to any such activities, users, or the like.

[0033] In the preceding and following description, various techniques are described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of possible ways of implementing the techniques. However, it will also be apparent that the techniques described below may be practiced in different configurations without the specific details. Furthermore, well-known features may be omitted or simplified to avoid obscuring the techniques being described.

[0034] FIG. 1 illustrates an example operating environment 100 including one or more legitimate users 102 and one or more malicious user 104 that are communicating with a web service system 106 that includes a general resource pool 108 and a defective resource pool 110. In various examples, the legitimate user 102 and/or malicious user 104 can include a user operating any suitable user device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like. Users may also be automated processes in some embodiments.

[0035] In various examples, the general resource pool 108 and defective resource pool 110 can include one or more computing resources 112. Some of these resources can be classified as defective resources 112D or classified as non-defective resources 112A. The computing resources 112 can include a server, physical host, or the like, configured to instantiate virtual or non-virtual machines. For example, the computing resources 112 can be configured to instantiate virtual computing services 114, honeypot computing resources 116, or the like.

[0036] Such computing resources 112 can be associated with, or in a given pool 108, 110 logically or physically. For example a set of computing resources 112 such as a plurality of servers can be present together in a server warehouse, or the like, and association with a given pool 108, 110 can be defined by a metadata tag or other identifier associated with each resource 112, with resources 112 of various pools being intermingled. Alternatively, resources 112 in a given resource pool 108, 110 can be physically located together.

[0037] As illustrated in FIG. 1, defective resources 112D can be transferred from the general resource pool 108 to the defective resource pool 110. In various examples, transferring defective resources 112D from the general resource pool 108 to the defective resource pool 110 can be a logical operation. In other words, transferring defective resources 112D from the general resource pool 108 to the defective resource pool 110 (or vice versa) can include changing an identifier, tag, metadata, or the like, associated with a given computing resource 112 such that the computing resource 112 is identified as being part of the defective resource pool 110. Alternatively, transferring resources 112 between pools 108, 110 can be a physical operation. For example, a human operator or robot can move a computing resource 112 form one location to another, which effectuates the transfer of the resource 112 from one pool to another.

[0038] Computing resource 112 can be identified as being defective for various suitable reasons, including one or more hardware components of the computing resource 112 not operating above a performance threshold. For example, a processor, hard drive, power supply, cable connections, network card, or the like, that is part of a computing resource 112 can have a performance issue that makes the computing resource 112 perform in a manner that not in conformity with quality assurance parameters of a service provider. However, in various examples, although defective resources 112 of a defective resource pool 106 may not be performing above a certain quality threshold, the defective resources 112 of a defective resource pool 106 may still be capable of instantiating honeypot computing services within certain defined parameters as discussed in more detail below.

[0039] As shown in FIG. 1, the general resource pool 108 of non-defective computing resources 112A can provide virtual computing services 114 to one or more legitimate user 102, which can include instantiation of one or more virtual machines, or the like. To maintain top performance for the one or more legitimate users 102, it can be beneficial to provide honeypot computing services 116, and the like, to malicious users 104 by using defective resources 112D from the defective resource pool 106 instead of from the general resource pool 108. In doing so, the activity of malicious users 104 may not affect the availability of top-performing computing resources 112 that may be assigned to the general resource pool 108.

[0040] As discussed in this disclosure, honeypot computing services 116 can include computing services that can allow a service provider to monitor actions of a malicious user 104, limit actions of a malicious user 104, quarantine a malicious user 104, track a malicious user 104, and the like. For example, honeypot computing services can include a malicious user monitoring service, a malicious user tracking service, a malicious user detection service, or the like. Such services can log actions performed by the malicious user and/or throttle computing resources being used by the malicious user. Additionally, as discussed below, honeypot computing services 116 can include the provisioning or use of simulated computing services, including a simulation state database.

[0041] FIG. 2 is a block diagram illustrating an operating environment 200 and example steps of a method of allocating computing resources 202 from a general pool 204 to a defective resource pool 206 that includes a set of defective computing resources 208 that can be used for providing honeypot services. In various examples, the resources 202 can comprise the non-defective resources 112A of FIG. 1 and the general pool 204 can comprise the general resource pool 108 of FIG. 1, or the like. The defective resources 208 can comprise the defective resources 112D of FIG. 1 and the defective pool 206 can comprise the defective resource pool 106 of FIG. 1, or the like.

[0042] The system 200 of FIG. 2 can also include a hardware monitor 210, a resource manager 212, a repair assessor 214, a user device 216, a provisioning service 218, and a threat assessor 220. In some examples, the user device 216 can comprise or be associated with the legitimate user 102 and/or malicious user 104 of FIG. 1, or can comprise any suitable device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like.

[0043] One or more of the hardware monitor 210, resource manager 212, repair assessor 214, provisioning service 218, and threat assessor 220, can be embodied in any suitable hardware and/or software. In one example, the hardware monitor 210, resource manager 212, repair assessor 214, provisioning service 218, and threat assessor 220, can be a part of the web services system 106 of FIG. 1. In further examples, the hardware monitor 210, resource manager 212, repair assessor 214, provisioning service 218, and threat assessor 220, need not be associated and may or may not be physically proximate or running on hardware that is physically proximate.

[0044] As illustrated in FIG. 2, the hardware monitor 210 can receive 222 a notification from the general pool 204 and/or resources 202 of the general pool 204 that indicates that one or more resources 202 and/or portion of a resource 202 has an issue that has caused degradation of the resource 202 or portion of the resource 202. In some examples, such a notification can include an indication of a specific issue or piece of hardware that has failed or assumed a degraded state. In other examples, such a notification can include a general indication that a resource 202 or portion of the resource 202 has failed or assumed a degraded state.

[0045] The received 222 issue from the general pool 204 and/or resources 202 of the general pool 204 can be reported 224 to the resource manager 212, which calls 226 the general pool 204 and/or specified resource(s) 202 at issue and transfers 228 the one or more identified resources 202 at issue to the defective pool 206 of defective resources. As discussed above, in various examples, such a transfer 228 can comprise a logical transfer where an identifier, flag, metadata, tag, or the like, is changed to indicate that a given resource is part of the defective pool 206 instead of being part of the general pool 204.

[0046] In some examples, there can be any suitable number of computing resource pools and such pools may or may not be overlapping. For example, a defective resource pool can be split into a light-degradation pool and a heavy-degradation pool. In other words, in some examples, there can be any suitable number of computing resource pools that include resources of varying levels of degradation, varying types of degradation, and the like. For example, resources can be tagged to identify one or more defects present in the resource or a level of defectiveness, which may be used to select which resources are used to satisfy a user request and/or used to select one or more resource pools that a given resource should be associated with. Additionally, there can also be a pool of resources that are deemed inoperable or degraded to a level where they are effectively inoperable. In further examples, old or outdated computing resources can be allocated to a separate pool, including a defective resources pool (even if such resources are not strictly defective, but only old or outdated).

[0047] Returning to FIG. 2, the one or more defective resources at issue 208 and/or the defective pool 206 can report to a repair assessor 214, which generates a repair schedule or action item, which is reported 232 to the resource manager 212. For example, the repair assessor 214 can comprise a computer system that executes an automated process, perhaps in conjunction with input of a human operator received into one or more input devices, that assesses issues with the one or more defective resources 208 at issue and generates a plan for fixing the defective resource(s) 208 or otherwise diagnosing a problem with the defective resource(s) 208. Such a repair and/or diagnosis plan can include an estimated time until a technician can work on the resource(s) 208, an estimated time until replacement parts for the defective resource(s) 208 become available, and the like.

[0048] Sometimes separate and/or in parallel to the steps discussed above, the user device 216 sends 324 a request to the provisioning service 218. In various examples, the provisioning service 218 can authenticate the request, authorize the request, and otherwise determine whether the request from the user device 216 appears to be legitimate. For example, a determination can be made whether the request is associated with a correct password, is a request related to a correct type of computer instance, and the like.

[0049] Where the provisioning service 218 determines that the request sent 324 from the user device 216 initially meets basic criteria for being classified as a legitimate request (e.g., has valid use credentials), the provisioning service 218 passes 236 the request to the resource manager 212, which can score the threat level of the request by querying 238 the threat assessor 220. For example, a threat level can be determined based on criteria such as whether the user has ever spun up a virtual computing instance in the requested region before, whether there is an indication that the user account is compromised, instantiation of virtual machines in a manner that is uncharacteristic of behavior associated with the user account, an indication that the user will likely default on payment, previous history of abuse by the user, or the like.

[0050] Where a threat level score of the request is above a defined threat level threshold, the user request can be marked as malicious. A threat level threshold can be based on various suitable factors and can change based on various suitable factors. The threat level threshold can be set based on capacity of the general resource pool 204, number of malicious transactions being detected, and the like. For example, where resources 202 of the general resource pool 204 are extremely limited the threat level threshold can be reduced so that more malicious requests are detected to reduce the stress on the general resource pool 204. In another example, where a large number of malicious transactions are being detected, it may indicate that an attack or surge of malicious user is occurring and the threat level threshold can be reduced so that more malicious requests are detected.

[0051] With the request being marked as malicious by the threat assessor 220 or other suitable element, the resource manager 212 can determine whether there are defective resources 208 available in the defective pool 206 that are eligible to service the malicious user request. For example, a determination can be made whether there are defective resources 208 available in the defective pool 206 that have the capability of meeting the specifications of the request up to a desired quality threshold. In other words, a determination can be made as to whether defective resources 208 are available in the defective pool 206 that will satisfy the user's request without any diminished service indicating that the request is being serviced by defective resources 208.

[0052] This determination can also be based on a repair assessment made by the repair assessor 214 and can include a determination of whether the defective resources 208 in their degraded state would be suitable to handle the request and whether the time of repair associated with such defective resources 208 would allow such defective resources 208 to be allocated to service the malicious request without interrupting fixing or diagnosing such defective resources 208. Accordingly, defective resources 208 can be put to use servicing malicious users where such defective resources 208 would otherwise remain unused because such defective resources 208 are deemed unfit for allocation to legitimate users.

[0053] For example, if a given defective resource 208 is scheduled for repair in one week and would be capable of servicing the malicious user request, a determination can be made to allocate such a defective resource 208 if such an allocation would not likely impact the scheduled repair time. In other words, if the malicious user was likely to use the defective resource 208 for less than a week, then the defective resource 208 could be used to service the malicious request and still be available for scheduled repair.

[0054] The where defective resources 208 are available in the defective pool 206, the resource manager can call 240 the desired resources 208, which can be allocated 242 to the user 216 to fulfill the malicious user request. In various examples, servicing the user request can include full servicing of the request (aside from allocation of defective resources 208), or can include provisioning of honeypot computing services as described in more detail below. For example, in some situations, various computing services can be simulated, but may not be. Additionally, as discussed in more detail in this disclosure, the permissions, capabilities, or the like, of the user can be limited where the user's request has been classified as malicious. Additionally, in various examples, provisioning defective resources 208 to satisfy a request mean marking the resource 208 as unavailable for use while it is provisioned as a honeypot resource. For example, such resources may then be ineligible for repair, diagnosis or provisioning to satisfy other requests.

[0055] By providing malicious users with functioning but defective resources 208, it can be possible to observe and learn from the actions of the malicious users so as to train a threat assessment heuristic, gain evidence that can be used by law enforcement, and otherwise learn the behaviors of malicious users. In other words, the malicious uses may be unable to determine, or immediately determine, that they have been provided with defective resources 208 which may lead the malicious users to believe that they have not been detected. Such malicious users may then believe they are continuing to perpetrate their malicious acts under the assumption that they are undetected, which can allow for learning and prevention of real harm to others.

[0056] Additionally, as the user 216 makes further requests, or other information becomes available, the threat score associated with the user or one or more user requests can be modified. For example, where a request has initially be marked as malicious, but further requests from the user clearly indicate that the user is not malicious, the defective resources 208 can be replaced with non-defective resources 202 of the general pool 204.

[0057] FIG. 3 is a block diagram illustrating an example method 300 of allocating defective computing resources to a defective resource pool and repairing the defective computing resources. In various examples, the method 300 can be performed by, or in part by, the resource manager 212 of FIG. 2, or any other suitable hardware and/or software element.

[0058] The method 300 begins where a notification is received 302 that a hardware fault has been associated with a shared computing resource. For example, such a shared computing resource can be the resources 202, 112 of FIGS. 1 and/or 2, or any other suitable resources. At 304, a determination is made whether the computing resource is eligible to be used in the defective hardware pool. For example, where the hardware fault caused the resource to be completely inoperable or so degraded that it is effectively inoperable, then such a resource may not be eligible for inclusion in the defective hardware pool. Alternatively, where the hardware fault does not indicate a substantial degradation of the resource that would impact the resource's use in service, then the resource may not be eligible for the defective hardware pool because it can still be effectively used in a general hardware pool of non-defective resources.

[0059] Where the resource is eligible to be used in the defective hardware pool, the shared computing resource is reassociated 308 from a general availability pool to a defective hardware pool and an effective time to repair is associated 310 with the shared computing resource. However, if the resource is not eligible for use in the defective hardware pool, at 306, the resource is removed from service pools or maintained in the general availability pool. For example, as discussed above, if the resource is completely or effectively inoperable, it can be removed from service. Alternatively, where the hardware fault is non-substantial, the resource can remain in the general availability pool.

[0060] For resources that have been assigned to the defective resource pool, a determination is made, at 312, whether the resource is eligible for use in the general resource pool. For example, such determination can include whether repairs have been made to the resource, whether a diagnosis has been made of the resource, and whether such a diagnosis and/or repair indicates that the resource is in a state where it is suitable for use in the general hardware pool. Where resources are eligible, such resources are returned 314 to the general resource pool.

[0061] FIG. 4 is a block diagram illustrating an example method 400 of servicing a malicious user request using shared computing resources that are assigned to a defective hardware pool. In various examples, the method can be performed by the provisioning service 218 of FIG. 2, or any other suitable hardware and/or software element.

[0062] The method 400 begins where a resource allocation request is received 402 from a user by a provisioning service and the resource allocation request is scored for maliciousness. At 406, a determination is made whether the maliciousness score exceeds a threshold for servicing the resource allocation request from a general resource pool. In various examples, such a general resource pool can include one of the general pools 108, 204 of FIGS. 1 and 2, or the like. If the maliciousness score does not exceed the threshold, then the request is handled 412 with one or more resources from the general availability pool.

[0063] However, if the maliciousness score does exceed the threshold for servicing from the general resource pool, then at 408, a determination is made whether there are available resources in the defective resource pool to satisfy the request. For example, such a determination 408 can include determining whether one or more resources of the defective resource pool meet or substantially meet the requirement of the user request and/or whether use of such resources would allow the resources to be unused at a time when the resources are scheduled to be repaired or diagnosed for issues that have put the resource in a degraded state.

[0064] For example, an effective time to repair can be calculated for the defective resources and an estimated time of allocation can be calculated for the request, and if the estimated time of allocation is less than the effective time to repair or within a range of the effective time to repair, then a given resource can be eligible for allocation for the request. An estimated time of allocation can be based on historical data, including historical data for the user making request. For example, average historical time of resource allocation for a user can be used to calculate an estimated time of allocation for a new request.

[0065] If no resources are available to satisfy the request, then the request is handled 412 with one or more resources from the general availability pool. Alternatively, where no resources are available to satisfy the request, then the request can be declined and the user may not be provided with the requested computing resources or information. However, if resources are available to satisfy the request, then the request is fulfilled 410 using one or more shared computing resources in the defective hardware pool.

[0066] FIG. 5 illustrates an example operating environment 500 where a malicious user 502 provides requests 504, 506 to a honeypot computing service 508 instantiated by a resource pool 510 and where responses 512 are provided to the malicious user 502 based on a simulation state database 514. In various examples, the malicious user 502, can include the malicious user 104 of FIG. 1, the user device 216 of FIG. 2 or can include a user operating any suitable user device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like. The honeypot computing service 508 can include the honeypot computing services 116 of FIG. 1 or any other suitable hardware and/or software element. The computing resource pool 510 can include the defective resource pool 106 and/or general resource pool 108 of FIG. 1, the general pool 204 and/or defective pool 206 of FIG. 2, or the like. The simulation state database 514 can include any suitable hardware and/or software element such as a virtual or non-virtual memory, which may or may not be a part of systems such as the web services system 100 of FIG. 1.

[0067] In various examples, the simulation state database 514 can indicate and track the state of various simulated computing resources. For example, in the case of virtual computing, when a malicious user requests instantiation of a virtual machine, instead of instantiation a virtual machine using real assets, a virtual machine can be simulated in a simulation state database. Accordingly, instead of instantiating the virtual machine on a physical host, an entry can be made in the simulation state database that a (simulated) virtual machine has been instantiated, and the malicious user can receive a notification that of instantiation of the (simulated) virtual machine that appears to be a notification of actual instantiation of the virtual machine in a physical host as requested. If the malicious user then sends a request to pause the virtual machine, an entry can be made in the simulation state databases that indicates the (simulated) virtual machine as being paused, and the malicious user can receive a notification that make it appear as if his real virtual machine has been paused. If the user then sends a request for the status of his virtual machine, the simulation state database can be queries and the status of paused can be identified. This pauses status of the (simulated) virtual machine can be sent to the malicious user and made to appear as if the status is from a real virtual machine.

[0068] Returning to FIG. 5, the resource pool 510 can comprise a plurality of defective resources 516 that can be used to instantiate a honeypot computing service 508. In some examples, the honeypot computing service 508 can comprise a computing service that mimics or simulates some or all of a set of web services, which can include virtual computing services. For example, the honeypot computing service 508 can be configured to simulate some or all of aspects of virtual computing services including instantiating one or more virtual machines; running of one or more virtual machines, stopping one or more virtual machines; configuring one or more virtual machines; querying one or more virtual machines, and the like.

[0069] Additionally, the honeypot computing service 508 can allow a system administrator to observe, track and contain the actions of the malicious user 502 so that any harmful actions that the malicious user 502 commits or attempts to commit are contained, stopped, and/or recorded for purposes of intervention by law enforcement or other parties.

[0070] The malicious user 502 can interact with the honeypot service 508 via various requests (e.g., application program interface (API) calls, and the like). As illustrated in FIG. 5, the malicious user 502 can communicate with the honeypot computing service 508 via one or more mutating requests 504 and/or one or more query requests 506. In some examples, a mutating request 504 can include a request to modify various aspects of one or more virtual or non-virtual machines, including attachment of one or more virtual or physical drive, physical host hardware preferences, physical host location, security group settings, type of machine image, instance size, cryptography settings, launching a virtual machine, pausing a virtual machine, taking a snapshot of a virtual machine, stopping a virtual machine, launching a non-virtual machine, pausing a non-virtual machine, taking a snapshot of a non-virtual machine, and the like.

[0071] In some examples, a query request 506 can include a request for status of, or data related to, various aspects of one or more virtual or non-virtual machine, including attachment status of one or more virtual or physical drive, physical host hardware status, physical host location, security group settings, type of machine image, instance size, cryptography settings, state of a virtual machine (e.g., running, stopped, paused), memory space usage, and the like.

[0072] In various examples, the malicious user 502 can submit requests 504, 506 via a user interface, which can comprise a webpage presented on user device 502, or the like. Additionally, the malicious user 502 can receive responses 512 to submitted requests 504, 506, and such responses can be based on a simulation state database. For example, where the malicious user submits requests 504, 506 via a user interface, the malicious user can also receive responses to the requests 504, 506 via the user interface. In some examples, some or all of the responses can come from live computing resources and/or some or all of the responses can be based on simulated computing resources.

[0073] In various examples, where it is possible to simulate some or all portions of a honeypot computing service 508, the simulation state of the simulated honeypot computing service 508 can be stored in the simulation state database. In one example, a malicious user 502 can request to instantiate one hundred virtual machines, each having multiple processing cores, but instead actually providing the malicious user 502 access to such resources, a simulation of such resources can be created such that the malicious user 502 will believe that the requested services have been provisioned based on responses 512 in a user interface that are generated in response to mutating and/or query requests 504, 506. Accordingly, the malicious user 502 can be tracked and monitored without having the malicious user 502 tying up resources that could be put to better use elsewhere, such as tracking other malicious users 502 and/or servicing legitimate users.

[0074] In other words, while a malicious user can request the provisioning of computing resources having certain specifications, some or all off the requested computing resources can be simulated. For example, the malicious user can request a set of computing resources, and no computing resources are provisioned to the user, but instead a simulation state database is used to simulate all of the requested computing resources. In such an example, a mutating request may not configure any real computing resources and may only trigger updating of a simulation state database to simulate the requested resources.

[0075] In other words, fulfilling a mutating request made by a malicious user can include simulating a response to the mutating request based on the current state of the simulated resource in the simulation state database. For example if the request was to terminate a first virtual machine instance, the database may indicate one of several things: the first instance is active; first instance has been terminated; there is no first instance; and the like etc. The response provided to the malicious user for such a request (and the actual update made to the simulation state database) depends on what the current state of this simulated is as defined by the simulation state database. For instance, if the simulation state database indicates that first instance is active when the request is received, the response would be successful and the simulation state database would be updated to indicate that the first instance has been terminated. However, if the simulation state database lacked an entry for first instance when the request to terminate was received, the response may be a failure and no update to the simulation state database may be made.

[0076] In another example, the malicious user can request a set of computing resources, and non-conforming resources can be provisioned to the user (e.g., resources from a defective resource pool, or the like), but the malicious user can receive an indication that conforming resources have been provisioned. Some or all of the functionalities that such non-conforming computing resources are unable to perform can be simulated by a simulation state database, or the like.

[0077] In a further example, a user can request a set of computing resources, and a portion of the requested resources can be simulated and a portion of the resources can be provided by provisioning real computing resources (including resources from a defective resource pool and/or a general resource pool). In other words, part of a malicious user request can be fulfilled with real resources, and a part can be simulated, but the malicious user receives and indication that all requested resources have been provisioned and receives responses to queries and mutations of these resources that correspond to responses that would come from the requested set of resources.

[0078] FIG. 6 is a block diagram illustrating an operating environment 600 and example steps of a method of identifying a malicious user and generating a honeypot computing service using a simulation state database 602. The operating environment 600 is shown including a service 604 that includes the simulation state database 602 and a plurality of resources 606 that are operably connected to a resource manager 608. The operating environment 600 also includes an authentication service 610, an identity service 612 and a user device 614.

[0079] In some examples one or more of the simulation state database 602, resources 606, resource manager 208, authentication service 610, and identity service 612 may or may not be part of a system such as the web services system 100 of FIG. 1.

[0080] Additionally, the resources 606 can include the computing resource pool 510, defective or general resource pools 106, 108 of FIG. 1, the defective pool 206 of FIG. 2, or the like. The user device 614 can comprise or be associated with the malicious user 104 of FIG. 1, the user device 216 of FIG. 2, malicious user of FIG. 5, or can include a user operating any suitable user device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like. The simulation state database 602 can include the simulation state database 514 of FIG. 4 or can include any suitable hardware and/or software element such as a virtual or non-virtual memory. The resource manager 608 can include the resource manager 212 of FIG. 2 or can include any suitable hardware and/or software element.

[0081] As illustrated in FIG. 6, the identity service can provision 614 honeypot credentials to the user device 614 and can report 616 these honeypot credentials to the authentication service 610. In various examples, the honeypot credentials can be provisioned such that the user device 614 is unaware that the honeypot credentials are any different than normal credentials that a legitimate user would receive. In other words, a malicious user associated with the user device 614 would not be able to distinguish legitimate user credentials from honeypot user credentials. For example, the honeypot credentials may outwardly have the same type of identifiers as legitimate credentials and indicators such as “Honeypot” and the like may be absent from the credentials. Additionally, a user interface presented on the user device may not provide any indications that are different from a session using legitimate user credentials. Accordingly, malicious users may not be tipped off that they have been flagged, categorized or branded as a malicious user.

[0082] In other examples, a honeypot credentials may have a clear indication that they are different than legitimate credentials. For example, such credentials or keys can include strings such as “Honeypot,” “Malicious User,” “Hacker,” or the like. Providing such credentials can be used to identify how sophisticated a malicious user is and/or whether the malicious user is actually a bot or other automated program that would not notice a clear indication of honeypot credentials that are different from legitimate user credentials. Alternatively, a honeypot indicator can include any suitable tag, string, identifier, code, serial number, or the like. In further examples, a honeypot indicator can be an implicit indicator, for instance, when a mathematical property of the credential indicates whether the credential is a honeypot credential or not, or where credentials are honeypot credentials if the sum of digits in the credential has a particular value mod N, or a checksum has a particular value, or the like.

[0083] The user device 614 can send a request to the resource manager 608 using the provisioned honeypot credentials, which can be authenticated 620 by the authentication service 610. The authentication service 610 can provide an indication to the resource manager 608 that the request is being made with honeypot credentials, which the resource manager 608 can use to provision real and/or simulated resources accordingly.

[0084] In one example, resources 606 can include general pool resources that are only used for legitimate users and the simulation state database 602 can be used for malicious users or users having honeypot credentials. In other words, the resource manager can selectively provision 622 resources 606 for legitimate users and can provision 624 simulated resources via the simulation state database 602 for malicious or honeypot users.

[0085] In further examples, a combination of simulated resources via the simulation database 602 and real resources 606 can be provisioned 622, 624 for malicious or honeypot users as necessary. For example, where it is determined that the honeypot user will not be sufficiently fooled by only simulated resources, then some or all of the resources provided to the honeypot user can be real resources 606, which can include defective and/or non-defective resources as discussed above.

[0086] FIG. 7 is a block diagram illustrating an example method 700 of determining whether a user is malicious and provisioning normal or honeypot credentials for the user's account. In some examples, the method 700 can be performed in whole or in part at an identity service 612 and/or authentication service 610 of FIG. 6. The method 700 begins where a user login request is received 702, and at 704, a determination is made whether the user is malicious.

[0087] If the user is determined to be malicious, the user can be provisioned 706 with honeypot credentials. However, if a user is determined to not be malicious, the user can be provisioned 708 with normal credentials. Honeypot and normal (non-honeypot) credentials can distinguishable to a system that authenticates using the credentials and, as a result, the system can determine how to process a request made using the credentials. As discussed, honeypot credentials can provide an indication that some or all of a set of requested set of computing resources be providing using resources from a defective pool and/or provided in the form of a simulation where such resources are not actually provided. In various examples, regular or non-honeypot credentials can provide an indication to a system to allocate real computing resources from a general resource allocation pool, which can be the best resources available.

[0088] In one example, as discussed in this disclosure, a user request can be scored for maliciousness and if the score is higher than a threshold, the user can be deemed malicious and provisioned 706 with honeypot credentials. In another example, a fraud score heuristic can be used to generate a fraud score and if the score is higher than a threshold, the user can be deemed malicious and provisioned 706 with honeypot credentials.

[0089] Although some examples include provisioning different credentials (e.g., keys, or the like) based on whether a user is identified as being malicious or not, in further examples, an existing user credential or account can be flagged as being malicious so that honeypot services can be provided to that malicious user (e.g., have metadata associated with the user account indicating the account to be malicious or fraudulent). In other words, the same credentials or key can be used by the user, but treatment of and response to these credentials can be changed based on determined maliciousness. Additionally, treatment of and response to a given credential, key or account can be different based on the user device that sent a request. For example, a first user device that uses a given user credential can be provisioned with honeypot services whereas a second user device that uses the same credential can be provisioned with legitimate services.

[0090] FIG. 8 is a block diagram illustrating an example method 800 of a service (e.g., the service 604 of FIG. 6) provisioning and running a honeypot computing service based on the types of requests provided by a user having honeypot credentials. The method 800 begins where a request using honeypot credentials is received 802, and at 804, a determination is made regarding what type of request has been received.

[0091] If the request is a mutating request, at 806, a determination is made whether a simulation state database (SSDB) is associated with the user account. If not, a simulation state database is associated 808 with the user account, the simulation state database is updated 810 based on the request, and a response is generated and provided 812 to the user. However, if a simulation state database is associated with the user account, the simulation state database is updated 810 based on the request and a response is generated and provided 812 to the user.

[0092] For example, a malicious user can provide a mutating request to instantiate a virtual machine, but instead of actually instantiating the virtual machine and allowing the malicious user to tie up valuable resources that could otherwise be used by a legitimate user, the virtual machine can be simulated with a simulation state database as discussed in detail above. Where a simulation state database is not already associated with the malicious user's account, one can be associated, and the simulation state database can be updated to reflect instantiation of a (simulated) virtual machine that meets the specifications of the malicious user request. Responses to the request, such as a confirmation of instantiation of the virtual machine, can be provided to the malicious user via a webpage or other suitable user interface.

[0093] Returning to the method 800, if the request type is determined to be a query request, at 804, a determination is made, at 814, whether a simulation state database is associated with the user account. If not, then a response is generated and provided 812 to the user. However, if a simulation state database is associated with the user account, then the simulation state database is queried 816 and a response is generated and provided 812 to the user.

[0094] For example, where no simulation state database is associated with the user account, this can indicate that there are no simulated running virtual machines associated with the user account, and therefore there would be no state to present to the user. Accordingly an error message, or the like, can be provided in response to a query request if no simulation state database is associated with the user account. On the other hand, where a simulation state database is associated with the user account and the malicious user provides a query request, the simulation state database can be queried to identify the simulated state of the resources and a response indicating the state of the simulated resources can be provided to the user.

[0095] In further examples, where honeypot computing services are provided to the malicious user that includes both real and simulated resources, then in addition to querying and/or updating a simulation state database, actual resources can be queries and/or mutated as necessary. Additionally, providing real and/or simulated resources to the malicious user can change over time. For example, where the malicious user makes requests that require inexpensive or simple assets, then a decision can be made to provide real resources instead of simulated resources. However, if the malicious user makes requests that then require expensive, complex or limited resources, then a decision can be made to simulate such resources instead of actually providing them to the malicious user. Additionally, determining whether to use simulated and/or real resources can be based on a determined fraud score, maliciousness score, or the like.

[0096] In some examples, the simulation state database can provide simulated computing resources to assist real computing resources that cannot perform certain actions due to being in a degraded state. For example the simulation state database can track a state for activities that a resource cannot do because the resource is defective. Accordingly, some mutating requests can affect only the simulation state database whereas other mutating requests can affect both the simulation state database and the defective resource.

[0097] FIG. 9 illustrates an example operating environment 900 where a fraud score heuristic is trained 902 by requests 904 provided by malicious users 906 and the fraud score heuristic 908 being used to determine whether uncategorized requests 910 from other users 912 are malicious. In various examples, the malicious users 906 and/or uncategorized user(s) 912 can include any suitable user device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like. Additionally, the malicious user 906 can include and/or be associated with the malicious user 104 of FIG. 1, the user device 216 of FIG. 2, malicious user 502 of FIG. 5, and/or the user device 614 of FIG. 6. Also, the uncategorized user(s) 912 can include and/or be associated with the malicious or legitimate user 102, 104 of FIG. 1, the user device 216 of FIG. 2, malicious user 502 of FIG. 5, and/or the user device 614 of FIG. 6.

[0098] The fraud score heuristic 908 can include any suitable simple or complex algorithm implemented on a hardware and/or software element. For example, the fraud score heuristic 908 can comprise, user and/or be modified based on one or more value, a k-means clustering algorithm, a neural net, a support vector machine, a Bayesian network, a genetic algorithm, a regression algorithm, an instance-based algorithm, a clustering algorithm, a decision tree algorithm, or the like. In some examples, the fraud score heuristic can comprise and/or use supervised, unsupervised or semi-supervised machine learning.

[0099] As illustrated in the example of FIG. 9, heuristic fraud score training 902 can include a set of training data that includes a set of requests 904 coming from one or more known malicious users 906. These known malicious requests 904 can be used to train the fraud score heuristic 908. In other words, by analyzing a set of known malicious requests 904, patterns, characteristics and/or identifiers of malicious requests can be identified so that un-categorized requests 910 can be analyzed to determine whether these requests are malicious or legitimate. In further examples, requests can be re-categorized or have a category confirmed. For example, a request or set of requests can initially be categorized as non-malicious, but the fraud score heuristic 908 can re-categorize the request or set of requests as being malicious or confirm the categorization of non-malicious. A set of malicious requests 904 may be known as or defined as being malicious in various suitable ways. For example, where a user account has been provisioned with malicious user credentials as discussed, requests made from this user account can be defined as known malicious requests.

[0100] In further examples, a set of training data for a fraud score heuristic 908 can include requests from both malicious users 906 and legitimate users. In such an example, the fraud score heuristic 908 can be trained to identify patterns, characteristics and/or identifiers of legitimate and malicious requests so that un-categorized requests 910 can be classified as being either legitimate or malicious. In still further examples, any suitable training data, from or associated with one or both of malicious and legitimate users, can be used to train a fraud score heuristic 908. For example, such data can include user profile data, user request history, user internet protocol (IP) address, user location, user device configuration, and the like.

[0101] Additionally, in some examples, more than one fraud score heuristic 908 can be created and trained. In one example, a fraud score heuristic 908 can be trained for individual users or groups of users that are known to be legitimate, and the fraud score heuristic 908 can be used to determine whether future requests from the user or group of users indicates that the user has become malicious or that the user account has been compromised and taken over by a malicious user. A fraud score heuristic 908 can be trained for groups of users based on one or more user characteristic, including user type (e.g., individual, enterprise, small company, or the like), user location, user payment type, user credit history, or user computing service type (e.g., website hosting, data storage, data processing, supercomputing, machine learning, business intelligence, application development, application testing, internet of things, and the like).

[0102] FIG. 10 is a block diagram illustrating an operating environment 1000 and example steps of a method of training a fraud score heuristic and using the fraud score heuristic to determine whether a request is malicious. The operating environment 1000 includes an identity service 1002, an authentication service 1004, a service provider 1006, a call log 1008, a characteristic extractor 1010, a fraud service 1012 and one or more users 1014.

[0103] In various examples, the users 1014 can include and/or be associated with the malicious or legitimate user 102, 104 of FIG. 1, the user device 216 of FIG. 2, malicious user 502 of FIG. 5, the user device 614 of FIG. 6, and/or the uncategorized user(s) 912 of FIG. 9. The user 1014 can also include any suitable user device including a laptop computer, desktop computer, smart phone, tablet computer, gaming device, heads-up display, wearable computer, home automation computer, or the like. In various examples, the identity service 1002, authentication service 1004, service provider 1006, call log 1008, characteristic extractor 1010, and/or fraud service 1012 can be any suitable hardware and/or software element and may or may not be part of a system such as the web services system 100 of FIG. 1.

[0104] As illustrated in FIG. 10, the identity service 1002 can provision 1016 honeypot credentials to one or more users 1014, which can be reported 1018 to the authentication service 1004. As discussed, honeypot credentials can be different than legitimate user credentials or can be indistinguishable from legitimate user credentials, with the user account being flagged for honeypot services at the authentication service 1004, or the like. The one or more users 1014 can use the provisioned honeypot credentials to request 1020 services at the service provider 1006, which authenticates 1022 the credentials at the authentication service 1004 and indicates to the service provider 1006 that the credential is a honeypot credential and/or that the user account has been flagged for honeypot services.

[0105] The service provider 1006 can record 1024 the request in the call log 1008, which can include an indication of a geographical location of a user device that generated the request, that the request was associated with honeypot credentials, that the request was associated with a honeypot account, parameters of the request, an operation associated with the request, a query associated with the request, IP address, and the like. In some examples, the call log 1008 can be a honeypot-specific call log that is separate from a general call logging service that tracks calls from all users and/or legitimate users. The call log 1008 can be specific to a single user account or can track multiple user accounts.

[0106] The characteristic extractor 1010 can be running 1026 off the call log 1008 and can examine requests that are stored in the call log 1008. Extracted characteristics of the requests can be provided 1028 to a fraud service 1012 which can be configured to generate a fraud score for the request and/or train a fraud score heuristic that evaluates whether one or more requests should be classified as fraudulent. In some examples, the fraud service 1012 can comprise or be associated with the fraud score heuristic 908 of FIG. 9, or the like.

[0107] The fraud service 1012 can provide 1030 fraud patterns, a fraud score service, or the like to authentication service 1004, which can use such information to evaluate requests or other activity associated with the users 1014 to determine whether such requests or activities should be classified as malicious or fraudulent behavior.

[0108] FIG. 11 is a block diagram illustrating an example method 1100 of a service training a fraud score heuristic and handling user requests based on whether the user has normal or honeypot credentials. For example, the web services system 100 of FIG. 1, or the like, can perform the method 1100. The method 1100 begins where a user request is received, and at 1104, a determination is made whether the user credentials are honeypot or normal credentials. In some examples, such a determination can be based on the credentials themselves. In other words, the credentials can include an indicator that the credentials are honeypot credentials or non-honeypot credentials. Alternatively, authentication of the credentials can include referencing a database that includes an identifier of whether the credentials are flagged as being honeypot credentials or normal credentials.

[0109] If the credentials are determined to be normal credentials, then the request is handled 1110 normally. For example, normal handling 1110 can include provisioning, modifying or querying requested computing services without any observation, limitation or simulation that may be associated with honeypot services as discussed. Additionally, normal handling 1110 can include provisioning computing resources from a general pool of resources instead of a defective pool of resources as discussed in this disclosure.

[0110] On the other hand, if the credentials are determined to be honeypot credentials, then characteristic information of the request are extracted 1106. For example, such extraction can be performed by the characteristic extractor 1010 of FIG. 10. Extraction of characteristics of a request can include extraction of any suitable data, information, characteristics, or the like that is a part of or associated with the request, a user account associated with the request and/or computing resources associated with the request. For example, an extracted characteristic can include, user location, user IP address, time from last request, physical host location, user agent, request clock skew, user language, user device type, user device operating system, machine type being requested, number of machines being requested, operating system of machine being requested, type or content of any previous or subsequent request, or the like.

[0111] With the characteristic information extracted 1106, a fraud score heuristic is updated 1108 based on the extracted characteristic and the method cycles back to 1102 where further requests are received 1102. Accordingly, a fraud score heuristic can be iteratively updated based at least on honeypot requests received over a period of time.

[0112] FIG. 12 illustrates a request set 1202 being used to identify clusters 1204 that can be used as predictors of malicious user requests. The request set 1202 can comprise one or more requests that include one or more parameters. Request parameters can include extracted characteristics of one or more request a discussed. In this example, each request includes parameters A-L, but any suitable number of parameters can be present in further examples. The parameters A-L of the request set 1202 can be plotted in any suitable number of dimensions to identify clusters 1204 that are indicative of a pattern among the parameters of the request set 1202.

[0113] The example plot 1200 shown in FIG. 12 is shown in two dimensions for simplicity, but in further examples, plotting in more than two dimensions can be applied. For example the request set 1202 illustrates requests having twelve parameters A-L. Accordingly in such an example, plotting in twelve dimensions corresponding to each of the parameters can be desirable.

[0114] As shown in the simplified graph 1200 of FIG. 12 a centroid 1206 for each of the clusters 1204 can be identified, and the centroids 1206 can be used to define a pattern that is characteristic of malicious or fraudulent requests. For example, where an uncategorized request is plotted in the same space, proximity to the one or more centroid 1206 (e.g., according to a Euclidean or other metric) can be used to generate a confidence score that the uncategorized request is either a malicious request or a legitimate request.

[0115] Clustering and identification of centroids of clusters can be done in any suitable way, including a k-means clustering algorithm, a k-medioids clustering algorithm, a k-medians clustering algorithm, a distribution-based clustering algorithm, a density-based clustering algorithm, and the like.

[0116] Additionally, various types of filtering and/or manipulation of the parameters A-L can be performed at various suitable times during the process of clustering and bucket generation. For example, a pre-processing step for the parameters A-L can be selective inclusion and/or exclusion based on black-listing or white-listing. Additionally, in some examples, parameters A-L can be genericized and/or scaled in any desirable way before or after a clustering operation.

[0117] In one example a call log comprising a set of API calls can be filtered to reduce the fidelity of detail in the set of API calls until a suitable clustering is generated among the parameters of the API calls. Such a filtering can be performed one or more times with different sequence likes. Each parameter can be taken individually and a determination can be made whether a cluster exists for the parameter using k-means clustering, or the like. In other words, a similarity score between the parameters can be identified which can be used to determine whether a cluster is present for that parameter or not. Clusters or buckets can be identified where a similarity score is above a certain defined threshold. A centroid or other center of such a cluster or bucket can then be used as a pattern or a portion of a pattern for identifying malicious requests.

[0118] Genericizing one or more request, or a set of requests, can be done in any suitable way. For example, one or more request parameter can be considered to be unimportant or have a range of values, and that value can be weighted accordingly. Clustering and bucketing can be performed again once such a genericizing abstraction is made and the number of requests that fall into that bucket can be determined to inform further genericizing and/or filtering can be performed to further define a pattern for recognizing malicious requests.

[0119] In further examples, a pattern for recognizing malicious requests can be based on groups or series of requests. For example, a pair of sequential requests, three sequential requests, four sequential requests, five sequential requests, or any suitable number of sequential requests can be used to generate a pattern for recognizing malicious requests or malicious behavior of a user based on a series of requests. In other words, the characteristics of a series or set of requests can be used to identify a pattern instead of a pattern being based on a single request.

[0120] Filtering of one or more pattern, request and/or parameters or characteristics of one or more request can be done in various suitable ways. For example, to guard against false positives where a legitimate user request or set of requests is categorized as being malicious, an anti-pattern can be generated that identifies patterns of legitimate user requests or sets of requests. In other words, the methods of generating patterns to identify malicious requests discussed can be used to generate patterns to identify characteristics that define legitimate user requests or sets of legitimate user requests. With identified patterns for both legitimate and malicious requests or sets of requests, a pattern for legitimate requests or can be subtracted from or used to filter or weight a pattern for malicious requests so that the pattern for malicious requests is less likely to create false positives when testing uncategorized requests or sets of requests.

[0121] FIG. 13 is a block diagram illustrating an example method 1300 of a service denying or approving a user request based on whether the user request exceeds a fraud score threshold or not. The method 1300 begins where a user request is received 1302 and a fraud score is calculated 1304 using a fraud score heuristic. In one example, a fraud score can be calculated based on how close the characteristics of the request, or a set of requests, matches a pattern for fraudulent requests or sets of requests. Referring to the example above, closeness to a pattern can include a calculation of the distance of subject request or request set characteristics from one or more cluster centroids that define a pattern for malicious requests.

[0122] At 1306 a determination is made whether the fraud score exceeds a fraud threshold, and if so, the user request is denied 1308 based on the fraud score exceeding the fraud threshold. On the other hand, if the fraud score does not exceed the fraud threshold, then the user request is approved 1320 based on the fraud score not exceeding the fraud threshold.

[0123] Setting a fraud threshold can be done in various suitable ways. For example, where a certain false-positive, false-negative, positive-positive or negative-negative rate is desired, historical data can be used to tune the fraud threshold to generate performance of fraud detection with desired parameters such that the system correctly classifies user requests without allowing too many fraudulent requests to go undetected or denying legitimate users service due to incorrect classification of their requests.

[0124] As noted above, a customer may interact with a virtual computing system service to launch a virtual machine instance within one a physical host. Accordingly, FIG. 14 shows an illustrative example of an environment 1400 in which one or more virtual machine images 1408 are instantiated within one or more virtual hosts 1412 upon customer request in accordance with at least one embodiment. The virtual computer system service 1402 may provide customers with a customer interface 1404 that may enable a customer to access the virtual computer system service 1402.

[0125] A customer may utilize the customer interface 1404 through one or more communications networks, such as the Internet. The customer interface 1404 may contain certain security safeguards to ensure that the customer has authorization to access the virtual computer system service 1402. For instance, in order to access the virtual computer system service 1402, a customer may need to provide a username and a corresponding password or encryption key when using the customer interface 1404. Additionally, requests (e.g., API calls) submitted to the customer interface 1404 may require an electronic signature generated using a cryptographic key such that the electronic signature is verifiable by the virtual computer system service 1402, such as by an authorization system (not shown).

[0126] Once the customer has gained access to the virtual computer system service 1402 through the customer interface 1404, the virtual computer system service 1402 may allow the customer to interact, through the customer interface 1404, with a management sub-system 1406. For instance, the management sub-system 1406 may enable a customer to remotely provision a virtual machine instance, and the one or more physical host that will instantiate the virtual machine instance can be selected by a resource manager 1418. A customer may use the customer interface 1404 and the management sub-system 1406 to generate a virtual machine instance that includes an operating system and a variety of applications suited to the customer's needs.

[0127] The operating system and the various applications may be maintained in data storage in the form of machine images 1408. The virtual computer system service 1402 may maintain a variety of machine images 1408 based on specific customer preferences, as specified in the management sub-system 1406. When a customer submits a request for provisioning a virtual machine instance through the management sub-system 1406, the virtual computer system service 1402 may identify the machine image the customer has requested and allocate the resources necessary (e.g., the processors and random-access memory (RAM) required to operate the components of the machine image) to process the machine image. The machine image may be instantiated on one or more physical hosts 1412 that is operably connected to a virtual disc service 1420 that can provide object level and/or block level memory storage for one or more physical host. 1412.

[0128] In one example, a customer, through the customer interface 1404, submits a request to the virtual computer system service 1402 to provision and/or reserve one or more physical host 1412 for his/her virtual machine instances. This request may be submitted through an “AcquireServer” web service or other API call to the service 1402 and may include various parameters such as an account identifier for the customer, virtual machine instance types, information such as a number of processors, amount of memory amount of networking capacity, etc. Additionally, if the customer wishes to reserve a server, the request may be submitted through a “PurchaseReservedServer” web service or other API call to the service 1402.

[0129] In response to the request, the management sub-system 1406 may access a server data store 1414 to identify, from a pool of physical hosts 1412 having free capacity, one or more physical host 1412 that may be reserved for the customer to fulfill his/her request, or the like. In some examples, the physical hosts 1412 can be a part of a general pool of resources or a pool of defective resources as discussed above.

[0130] In some examples, the virtual computer system service 1402 may maintain a general pool of physical hosts 1412 that the management sub-system 1406 may utilize to identify one or more physical hosts, or a portion thereof, that may be allocated to the customer for use. Once the management sub-system 1406 has identified the one or more physical hosts 1412 to be allocated to the customer, the management sub-system 1406 may access the server data store 1414 to specify, within a database, which physical hosts 1412, or portions thereof have been allocated for use of the customer.

[0131] In various examples, the server data store 1414 can store any suitable information about physical hosts 1412, including a physical host identifier 1422, a resource pool identifier 1424 (e.g., whether the resource is part of a general resource pool, a defective resource pool, or the like), a network location 1426, and a slot status 1428.

[0132] Once one or more physical hosts 1412 have been allocated for the customer, the customer, through the interface 1404, can submit a request (e.g., “RunInstances” web service or other API call) to the virtual computer system service 1402 to instantiate one or more virtual machine images 1408 onto one of these allocated servers 1412 and/or a portion thereof. For instance, in an embodiment, the customer, through the interface 1404, submits a request to the management sub-system 1406 (e.g., “DescribeServers” web service or other API call) to obtain an identifier for each physical host 1412 allocated for the customer or otherwise being utilized by the customer.

[0133] Once the customer has obtained these one or more server identifiers from the management sub-system 1406, the customer may generate a request that may include an identifier corresponding to the physical host(s) 1412 that should be used for instantiating the virtual machine image 1408. The management sub-system 1406 may instantiate the virtual machine image 1408 on the selected physical host(s) 1412, if the physical host(s) 1412 have sufficient capacity to support the virtual machine instance. In some embodiments, the management sub-system 1406 may include a placement server, which may be configured to select from the plurality of servers 1412 a server for the virtual machine instance in a manner that respects a plurality of dedications in the server data store 1414.

[0134] FIG. 15 illustrates a virtual computer system service in accordance with at least one embodiment. The virtual computer system service 1508, which can comprise physical hardware 1540, can be used by a computing resource service provider for providing computation resources for customers. The physical hardware 1540 may include one or more physical hosts 1542. The physical hosts 1542 may be any device or equipment configured to execute instructions for performing data computation, manipulation, or storage tasks, such as a computer or a server.

[0135] A physical host 402 may be equipped with any needed processing capability including one or more processors, such as a central processing unit (CPU), a graphics processing unit (GPU) or a digital signal processor (DSP), memory, including static and dynamic memory, buses, and input and output ports that are compliant with any handshaking, communications or data transfer protocol. The physical hardware 1540 may also include storage devices, such as storage disks and tapes, networking equipment and the like and/or can be coupled to virtual disc service 1550.

[0136] A virtualization layer 1544 in the computing resources service provider can enable the physical hardware 1542 to be used to provide computational resources upon which one or more virtual machines 1546 may operate. The virtualization layer 1544 may be any device, software, or firmware used for providing a virtual computing platform for the virtual machines 1546. The virtual computing platform 1508 may include various virtual computer components, such as one or more virtual CPUs, virtual memory and the like. The virtual machines 1546 may be provided to the customers of a computing service resource provider and the customers may run an operating system or an application on one or more virtual machines 1546. Further, the computing service resource provider may use one or more of its own virtual machines 1546 for executing its applications. Examples of the virtualization layer 1544 include a hypervisor.

[0137] The virtualization layers 1540 can provide a host activity report associated with the physical hardware 1540, one or more physical hosts 1542 or the one or more virtual machines 1546. The host activity report may include utilization, activity or operational information or statistics for the physical hardware 1540, physical hosts 1542, virtual machines 1546 or applications that are executed on the physical hosts 1542 or the virtual machines 1546. The information included in the host activity report may be gathered or compiled over an interval of time or taken at one or more snapshots in time.

[0138] FIG. 16 illustrates aspects of an example environment 1600 for implementing aspects in accordance with various embodiments. As will be appreciated, although a web-based environment is used for purposes of explanation, different environments may be used, as appropriate, to implement various embodiments. The environment includes an electronic client device 1602, which can include any appropriate device operable to send and/or receive requests, messages, or information over an appropriate network 1604 and, in some embodiments, convey information back to a user of the device. Examples of such client devices include personal computers, cell phones, handheld messaging devices, laptop computers, tablet computers, set-top boxes, personal data assistants, embedded computer systems, electronic book readers, and the like. The network can include any appropriate network, including an intranet, the Internet, a cellular network, a local area network, a satellite network or any other such network and/or combination thereof. Components used for such a system can depend at least in part upon the type of network and/or environment selected. Many protocols and components for communicating via such a network are well known and will not be discussed herein in detail.

[0139] Communication over the network can be enabled by wired or wireless connections and combinations thereof. In this example, the network includes the Internet and/or other publicly addressable communications network, as the environment includes a web server 1606 for receiving requests and serving content in response thereto, although for other networks an alternative device serving a similar purpose could be used as would be apparent to one of ordinary skill in the art.

[0140] The illustrative environment includes at least one application server 1608 and a data store 1610. It should be understood that there can be several application servers, layers or other elements, processes or components, which may be chained or otherwise configured, which can interact to perform tasks such as obtaining data from an appropriate data store. Servers, as used herein, may be implemented in various ways, such as hardware devices or virtual computer systems. In some contexts, servers may refer to a programming module being executed on a computer system. As used herein, unless otherwise stated or clear from context, the term “data store” refers to any device or combination of devices capable of storing, accessing and retrieving data, which may include any combination and number of data servers, databases, data storage devices and data storage media, in any standard, distributed, virtual or clustered environment. The application server can include any appropriate hardware, software and firmware for integrating with the data store as needed to execute aspects of one or more applications for the client device, handling some or all of the data access and business logic for an application. The application server may provide access control services in cooperation with the data store and is able to generate content including, but not limited to, text, graphics, audio, video and/or other content usable to be provided to the user, which may be served to the user by the web server in the form of HyperText Markup Language (“HTML”), Extensible Markup Language (“XML”), JavaScript, Cascading Style Sheets (“CSS”), JavaScript Object Notation (JSON), and/or another appropriate client-side structured language. Content transferred to a client device may be processed by the client device to provide the content in one or more forms including, but not limited to, forms that are perceptible to the user audibly, visually and/or through other senses. The handling of all requests and responses, as well as the delivery of content between the client device 1602 and the application server 1608, can be handled by the web server using PHP: Hypertext Preprocessor (“PHP”), Python, Ruby, Perl, Java, HTML, XML, JSON, and/or another appropriate server-side structured language in this example. Further, operations described herein as being performed by a single device may, unless otherwise clear from context, be performed collectively by multiple devices, which may form a distributed and/or virtual system.

[0141] The data store 1610 can include several separate data tables, databases, data documents, dynamic data storage schemes and/or other data storage mechanisms and media for storing data relating to a particular aspect of the present disclosure. For example, the data store illustrated may include mechanisms for storing production data 1612 and user information 1616, which can be used to serve content for the production side. The data store also is shown to include a mechanism for storing log data 1614, which can be used for reporting, analysis or other such purposes. It should be understood that there can be many other aspects that may need to be stored in the data store, such as page image information and access rights information, which can be stored in any of the above listed mechanisms as appropriate or in additional mechanisms in the data store 1610. The data store 1610 is operable, through logic associated therewith, to receive instructions from the application server 1608 and obtain, update or otherwise process data in response thereto. The application server 1608 may provide static, dynamic, or a combination of static and dynamic data in response to the received instructions. Dynamic data, such as data used in web logs (blogs), shopping applications, news services, and other such applications may be generated by server-side structured languages as described herein or may be provided by a content management system (“CMS”) operating on, or under the control of, the application server. In one example, a user, through a device operated by the user, might submit a search request for a certain type of item. In this case, the data store might access the user information to verify the identity of the user and can access the catalog detail information to obtain information about items of that type. The information then can be returned to the user, such as in a results listing on a web page that the user is able to view via a browser on the user device 1602. Information for a particular item of interest can be viewed in a dedicated page or window of the browser. It should be noted, however, that embodiments of the present disclosure are not necessarily limited to the context of web pages, but may be more generally applicable to processing requests in general, where the requests are not necessarily requests for content.

[0142] Each server typically will include an operating system that provides executable program instructions for the general administration and operation of that server and typically will include a computer-readable storage medium (e.g., a hard disk, random access memory, read only memory, etc.) storing instructions that, when executed (i.e., as a result of being executed) by a processor of the server, allow the server to perform its intended functions.

[0143] The environment, in one embodiment, is a distributed and/or virtual computing environment utilizing several computer systems and components that are interconnected via communication links, using one or more computer networks or direct connections. However, it will be appreciated by those of ordinary skill in the art that such a system could operate equally well in a system having fewer or a greater number of components than are illustrated in FIG. 16. Thus, the depiction of the system 1600 in FIG. 16 should be taken as being illustrative in nature and not limiting to the scope of the disclosure.

[0144] The various embodiments further can be implemented in a wide variety of operating environments, which in some cases can include one or more user computers, computing devices or processing devices which can be used to operate any of a number of applications. User or client devices can include any of a number of computers, such as desktop, laptop or tablet computers running a standard operating system, as well as cellular, wireless and handheld devices running mobile software and capable of supporting a number of networking and messaging protocols. Such a system also can include a number of workstations running any of a variety of commercially available operating systems and other known applications for purposes such as development and database management. These devices also can include other electronic devices, such as dummy terminals, thin-clients, gaming systems and other devices capable of communicating via a network. These devices also can include virtual devices such as virtual machines, hypervisors and other virtual devices capable of communicating via a network.

[0145] Various embodiments of the present disclosure utilize at least one network that would be familiar to those skilled in the art for supporting communications using any of a variety of commercially available protocols, such as Transmission Control Protocol/Internet Protocol (“TCP/IP”), User Datagram Protocol (“UDP”), protocols operating in various layers of the Open System Interconnection (“OSI”) model, File Transfer Protocol (“FTP”), Universal Plug and Play (“UpnP”), Network File System (“NFS”), Common Internet File System (“CIFS”), and AppleTalk. The network can be, for example, a local area network, a wide-area network, a virtual private network, the Internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network, a satellite network, and any combination thereof. In some embodiments, connection-oriented protocols may be used to communicate between network endpoints. Connection-oriented protocols (sometimes called connection-based protocols) are capable of transmitting data in an ordered stream. Connection-oriented protocols can be reliable or unreliable. For example, the TCP protocol is a reliable connection-oriented protocol. Asynchronous Transfer Mode (“ATM”) and Frame Relay are unreliable connection-oriented protocols. Connection-oriented protocols are in contrast to packet-oriented protocols such as UDP that transmit packets without a guaranteed ordering.

[0146] In embodiments utilizing a web server, the web server can run any of a variety of server or mid-tier applications, including Hypertext Transfer Protocol (“HTTP”) servers, FTP servers, Common Gateway Interface (“CGI”) servers, data servers, Java servers, Apache servers, and business application servers. The server(s) also may be capable of executing programs or scripts in response to requests from user devices, such as by executing one or more web applications that may be implemented as one or more scripts or programs written in any programming language, such as Java®, C, C#, or C++, or any scripting language, such as Ruby, PHP, Perl, Python or TCL, as well as combinations thereof. The server(s) may also include database servers, including without limitation those commercially available from Oracle®, Microsoft®, Sybase and IBM® as well as open-source servers such as MySQL, Postgres, SQLite, MongoDB, and any other server capable of storing, retrieving, and accessing structured or unstructured data. Database servers may include table-based servers, document-based servers, unstructured servers, relational servers, non-relational servers, or combinations of these and/or other database servers.

[0147] The environment can include a variety of data stores and other memory and storage media as discussed above. These can reside in a variety of locations, such as on a storage medium local to (and/or resident in) one or more of the computers or remote from any or all of the computers across the network. In a particular set of embodiments, the information may reside in a storage-area network (“SAN”) familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers, servers or other network devices may be stored locally and/or remotely, as appropriate. Where a system includes computerized devices, each such device can include hardware elements that may be electrically coupled via a bus, the elements including, for example, at least one central processing unit (“CPU” or “processor”), at least one input device (e.g., a mouse, keyboard, controller, touch screen, or keypad) and at least one output device (e.g., a display device, printer, or speaker). Such a system may also include one or more storage devices, such as disk drives, optical storage devices, and solid-state storage devices such as random access memory (“RAM”) or read-only memory (“ROM”), as well as removable media devices, memory cards, flash cards, etc.

[0148] Such devices also can include a computer-readable storage media reader, a communications device (e.g., a modem, a network card (wireless or wired), an infrared communication device, etc.), and working memory as described above. The computer-readable storage media reader can be connected with, or configured to receive, a computer-readable storage medium, representing remote, local, fixed, and/or removable storage devices as well as storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information. The system and various devices also typically will include a number of software applications, modules, services, or other elements located within at least one working memory device, including an operating system and application programs, such as a client application or web browser. In addition, customized hardware might also be used and/or particular elements might be implemented in hardware, software (including portable software, such as applets) or both. Further, connection to other computing devices such as network input/output devices may be employed.

[0149] Storage media and computer readable media for containing code, or portions of code, can include any appropriate media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and/or transmission of information such as computer readable instructions, data structures, program modules or other data, including RAM, ROM, Electrically Erasable Programmable Read-Only Memory (“EEPROM”), flash memory or other memory technology, Compact Disc Read-Only Memory (“CD-ROM”), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other medium which can be used to store the desired information and which can be accessed by the system device. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and/or methods to implement the various embodiments.

[0150] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope of the invention as set forth in the claims.

[0151] Other variations are within the spirit of the present disclosure. Thus, while the disclosed techniques are susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the invention, as defined in the appended claims.

[0152] The use of the terms “a” and “an” and “the” and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. The term “connected,” when unmodified and referring to physical connections, is to be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein and each separate value is incorporated into the specification as if it were individually recited herein. The use of the term “set” (e.g., “a set of items”) or “subset” unless otherwise noted or contradicted by context, is to be construed as a nonempty collection comprising one or more members. Further, unless otherwise noted or contradicted by context, the term “subset” of a corresponding set does not necessarily denote a proper subset of the corresponding set, but the subset and the corresponding set may be equal.

[0153] Conjunctive language, such as phrases of the form “at least one of A, B, and C,” or “at least one of A, B and C,” unless specifically stated otherwise or otherwise clearly contradicted by context, is otherwise understood with the context as used in general to present that an item, term, etc., may be either A or B or C, or any nonempty subset of the set of A and B and C. For instance, in the illustrative example of a set having three members, the conjunctive phrases “at least one of A, B, and C” and “at least one of A, B and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of A, at least one of B and at least one of C each to be present.

[0154] Operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. Processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs or one or more applications) executing collectively on one or more processors, by hardware or combinations thereof. The code may be stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory. In some embodiments, the code is stored on set of one or more non-transitory computer-readable storage media having stored thereon executable instructions that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause the computer system to perform operations described herein. The set of non-transitory computer-readable storage media may comprise multiple non-transitory computer-readable storage media and one or more of individual non-transitory storage media of the multiple non-transitory computer-readable storage media may lack all of the code while the multiple non-transitory computer-readable storage media collectively store all of the code. Further, in some examples, the executable instructions are executed such that different instructions are executed by different processors. As an illustrative example, a non-transitory computer-readable storage medium may store instructions. A main CPU may execute some of the instructions and a graphics processor unit may execute other of the instructions. Generally, different components of a computer system may have separate processors and different processors may execute different subsets of the instructions.

[0155] Accordingly, in some examples, computer systems are configured to implement one or more services that singly or collectively perform operations of processes described herein. Such computer systems may, for instance, be configured with applicable hardware and/or software that enable the performance of the operations. Further, computer systems that implement various embodiments of the present disclosure may, in some examples, be single devices and, in other examples, be distributed computer systems comprising multiple devices that operate differently such that the distributed computer system performs the operations described herein and such that a single device may not perform all operations.

[0156] The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate embodiments of the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.

[0157] Embodiments of this disclosure are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate and the inventors intend for embodiments of the present disclosure to be practiced otherwise than as specifically described herein. Accordingly, the scope of the present disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the scope of the present disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0158] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.