US 8,832,839 B2Grant
Assessing system performance impact of security attacks
Issue Date:2014-09-09
•17 Claims
•5 Drawing Sheets
Abstract
A method for assessing an impact of a security attack on a system includes defining a system affecting metric for an observation period as a fraction of time the system satisfies a defined specification, defining a resource failure based model and a resource usage based model for the system, obtaining results for each of a plurality of states of the resource failure based model and the resource usage based model, solving the resource failure based model and the resource usage based model and obtaining a term fraction of time each model spends on each of the plurality of states, obtaining a state probability according to the term fraction, and obtaining a measure of the system affecting metric according to the state probability.
Metadata
Assignee
- Siemens Aktiengesellschaft
Inventor
- Alberto Avritzer
Application Information
Application Number:US 13/178,581
Filing Date:2011-07-08
Priority Date:2011-01-04
Art Unit:2492
Classifications
IPC:
G06F11/00G06F12/14G06F12/16G08B23/00G06F21/00G06F11/34
Patent Drawings (5 sheets)
Description
Cross-Reference to Related Application
[0001] This application claims the benefit of Provisional Application No. 61/429,518 filed on Jan. 4, 2011 in the United States Patent and Trademark Office, the contents of which are herein incorporated by reference in its entirety.
Background
[0002] 1. Technical Field
[0003] The present disclosure relates to methods for assessing system performed, and more particularly to assessing system performance during a security attack.
[0004] 2. Discussion of Related Art
[0005] Deterministic State Testing (DST) is a load test generation and execution approach for the generation and execution of performance tests. DST uses a high level state definition, and an analytical approximation to identify most likely states in continuous-time Markov chain representation of a system under test. The most likely states represent states to be exercised by performance tests because these states contain the problems that are most likely to be encountered in production. The current mode of operation of DST is based on ranking by usage and by failures related to security vulnerabilities.
Summary
[0006] According to an embodiment of the present disclosure, a method for assessing an impact of a security attack on a system includes defining a system affecting metric for an observation period as a fraction of time the system satisfies a defined specification, defining a resource failure based model and a resource usage based model for the system, obtaining results for each of a plurality of states of the resource failure based model and the resource usage based model, solving the resource failure based model and the resource usage based model and obtaining a term fraction of time each model spends on each of the plurality of states, obtaining a state probability according to the term fraction, and obtaining a measure of the system affecting metric according to the state probability.
[0007] According to an embodiment of the present disclosure, a method for assessing an impact of a security attack on a system is embodied as a non-transitory computer readable medium storing instructions executable by a processor.
[0008] According to an embodiment of the present disclosure, a system for assessing an impact of a security attack includes a processor configured to obtain a measure of a system affecting metric for an observation period as a fraction of time the system satisfies a defined specification according to a state probability by defining a resource failure based Markov chain and a resource usage based Markov chain for the system, obtaining results for each of a plurality of states of the resource failure based Markov chain and the resource usage based Markov chain, solving the resource failure based Markov chain and the resource usage based Markov chain and obtaining a term fraction of time each Markov chain spends on each state, obtaining a Markov state probability; and obtaining a measure of the system affecting metric according to the state probability, and a memory configured to store the measure of a system affecting metric.
Brief Description of the Drawings
[0009] Preferred embodiments of the present disclosure will be described below in more detail, with reference to the accompanying drawings:
[0010] FIG. 1 is a flow diagram of a method according to an embodiment of the present disclosure;
[0011] FIG. 2A is resource usage Markov chain according to an embodiment of the present disclosure;
[0012] FIG. 2B is resource failure Markov chain according to an embodiment of the present disclosure; and
[0013] FIG. 3 is a diagram of a computer system for automatic measurement of morphometric and motion parameters according to an embodiment of the present disclosure.
Detailed Description of Exemplary Embodiments
[0014] According to an embodiment of the present disclosure, an impact of security attacks on a system performance may be assessed to reveal information about system performance survivability. More particularly, the assessment is a stochastic assessment of the likelihood of performance attacks and the impact of security attacks on system performance. An exemplary method may be applied at a back end of a software development process.
[0015] The security attacks may include, for example, denial of service (DoS) attacks, buffer overflow attacks, access control discovery attacks and other types of security attacks that impact system performance. In the case of a DoS attacks computational resources are consumed, disrupting configuration information, state information and physical network components, and obstructing the communication media. Different attacks may affect system performance differently.
[0016] According to an embodiment of the present disclosure, a method for assessing an impact of security attacks on a system includes building a resource failure based model and a resource usage based model. Markov chains may be used as the model to evaluate system reliability, e.g., a probability of successful operation when the system is subjected to security attacks. Likely security attacks are simulated and an assessment is made of the performance during the security attacks.
[0017] According to an embodiment of the present disclosure, a security test suite is created and applied to a system under test. At this time, system performance is measured and Markov modeling may be used to assess a security testing metric.
[0018] The security test suite may include a set of likely attacks. More particularly, the security test suite includes tests cases derived from known types of security attacks.
[0019] An exemplary method may be used to stochastically determine a conditional reliability of a system under test using test case results. More particularly, the security testing metric may capture an indication of a time that the system is operating correctly (e.g., within a defined performance/reliability specification). The system performance metric may be determined by solving a failure based Markov chain and a resource usage based Markov chain and incorporating the results of the security test suite into the system performance metric.
[0020] According to an embodiment of the present disclosure, a method for assessing an impact of security attacks on a system (FIG. 1 ) includes defining a system performance/reliability affecting metric for a given observation period as a fraction of time the system satisfies a defined performance/reliability specification 101 and defining a security based resource failure based Markov chain and a resource usage based Markov chain for the system under test 102. Exemplary component measurements in the specification include processor usage (e.g., percentage of total), processor interrupt rate, interrupt handling time, TCP (Transmission Control Protocol) throughput, system availability, average queue length (e.g., in the case of applications such as email), etc.
[0021] The system under test can take a variety of different system resource configurations. Several Markov resource usage based Markov chains may be generated, wherein one Markov chain may be associated with each resource configuration. System resource configurations may vary because of resource failures and/or security attacks.
[0022] The method may further include using security test case results to obtain the pass/fail results for each state of the resource failure based Markov chain and the resource usage based Markov chain 103, wherein the pass/fail results convey the impact of the failure on the customers and solving the Markov chains using known techniques and obtain long term fraction of time each Markov chain spends on each state 104.
[0023] The stochastic nature of security attacks may be obtained by conditioning each Markov chain on the occurrence of the given security attack. The Markov state probability may be obtained using numerical solution techniques, such as Uniformization.
[0024] Uniformization is a known method for determining transient solutions of continuous-time Markov chains, e.g., involving the transformation of a continuous time Markov chain to an analogous discrete time Markov chain.
[0025] At block 105, the system reliability metric conditioned on the occurrence of the known security attacks may be obtained by summing-up the states for each of the system performance requirements met using the pass/fail criteria obtained from the system under test. That is, according to an exemplary approach, results from a transient Markov chain analysis may be used to estimate reliability at a given system execution time.
[0026] More particularly, at block 105, the system under test may be modeled as a resource failure based Markov chain X for system reliability. This Markov chain may lead to C possible distinct configurations and C different resource usage-based Markov chains Y(i) for configuration ci. The resource usage based Markov chains may be solved to obtain a long-term fraction of time in each state of the individual chains, where yj(i) is obtained, the fraction of time during (0, t) that Y(i) spends in state sj(i). The resource failure-based Markov chain is solved to obtain the fraction of time during (0, t) that X spends in each state φi. For the resource failure based Markov chain generate as a test suite the list of states from X such that the sum of the total fraction of time in these states is above a given threshold value, in order to cover an appropriate number of cases during the observation interval, e.g., a threshold value maximizing the number of cases. It should be understood that other system reliability metrics may be used.
[0027] FIGS. 2A and 2B are exemplary resource usage and failure based Markov chains, respectively. The Markov chains codify changes in transitions from one state to another; according to an exemplary embodiment of the present disclosure, states may be modeled as the number of resources N used or available.
[0028] FIG. 2A is an exemplary resource usage Markov chain for a given resource configuration with N-K total number of resources. In FIG. 2A , rightward pointing arrows correspond to resource allocation due to work arrival, e.g., escalating from 0 resources used 201 to 1 resource used 202. Leftward pointing arrows correspond to resource release due to work completion, e.g., decrementing from 1 resource to 0 resource.
[0029] FIG. 2B is an exemplary resource failure Markov chain producing C different resource configurations. In FIG. 2B , rightward pointing arrows correspond to resource failure due to a security attack, e.g., decrementing from N resources 203 to N−1 resources 204. Leftward pointing arrows correspond to resource repair, e.g., making resources available after repair.
[0030] According to an embodiment of the present disclosure, the system reliability assessments may be conditioned on the occurrence of security attacks. Performance of the system reliability metric may be measured as accuracy and/or efficiency, for example, as compared to observed real-world data.
[0031] Referring to accuracy, pass/fail criteria may be determined for each state of the resource usage Markov chain for each defined failure configuration. Data related to the known security attacks is collected periodically (e.g., attack trees and attack patterns or profiles). The system reliability metric conditioned on security attacks is evaluated each time a new system or a modified system goes through a performance testing phase.
[0032] Referring to efficiency, the derivation of a security test suite based on known security attacks allows the security manager to focus on a significantly shorter list of attacks as compared to the full list of vulnerabilities published at the standard security web sites. When new attack types become popular, the security test suite needs to be updated to account for the impact of these new attacks on system performance.
[0033] Exemplary methods may be generalized to automatically change the network, system and software configuration to increase the available capacity of the system under test, when the system reliability metric conditioned on security attacks e.g., the probability of not meeting performance requirements when the system is under a certain security attack, crosses a pre-defined threshold. Here, exemplary methods may be executed in real-time on a live production system to detect a performance impacting security attack and to initiate counter-measures.
[0034] It is to be understood that embodiments of the present disclosure may be implemented in various forms of hardware, software, firmware, special purpose processors, or a combination thereof. In one embodiment, a method for low complexity signal analysis may be implemented in software as an application program tangibly embodied on a computer readable medium. The application program may be uploaded to, and executed by, a processor comprising any suitable architecture.
[0035] Referring to FIG. 3 , according to an embodiment of the present disclosure, a computer system 301 for implementing a method for assessing an impact of security attacks on a system performance can comprise, inter alia, a central processing unit (CPU) 302, a memory 303 and an input/output (I/O) interface 304. The computer system 301 is generally coupled through the I/O interface 304 to a display 305 and various input devices 306 such as a mouse and keyboard. The support circuits can include circuits such as cache, power supplies, clock circuits, and a communications bus. The memory 303 is non-transitory and can include random access memory (RAM), read only memory (ROM), disk drive, tape drive, etc., or a combination thereof. Embodiments of the present disclosure can be implemented as a routine 307 that is stored in memory 303 and executed by the CPU 302 to process the signal from the signal source 308. As such, the computer system 301 is a general-purpose computer system that becomes a specific purpose computer system when executing the routine 307 of the present disclosure.
[0036] The computer platform 301 also includes an operating system and micro-instruction code. The various processes and functions described herein may either be part of the micro-instruction code or part of the application program (or a combination thereof) which is executed via the operating system. In addition, various other peripheral devices may be connected to the computer platform such as an additional data storage device and a printing device.
[0037] It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures may be implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the methods described herein are programmed. Given the teachings of the present disclosure provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of embodiments described herein.
[0038] Having described embodiments for assessing an impact of security attacks on a system performance, it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in embodiments of the present disclosure that are within the scope and spirit thereof.
Claims
What is claimed is:
1. A computer-implemented method for assessing an impact of a security attack on a software system, the method executed by the computer comprising the steps of:
defining a system performance/reliability affecting metric for an observation period as a fraction of time the system satisfies a defined performance/reliability specification;
defining a resource failure based Markov model and a plurality of resource usage based Markov models for the system, wherein each resource usage based model is associated with a corresponding resource configuration;
using results of security test cases performed on said software system to obtain pass/fail results for each of a plurality of states of the resource failure based model and the resource usage based model for each corresponding resource configuration;
solving the resource failure based Markov model and the plurality of resource usage based Markov models and obtaining a long term fraction of time each model spends on each of the plurality of states;
and
obtaining a measure of the system performance/reliability affecting metric by summing the states for a plurality of system performance requirements met using the pass/fail results obtained from the plurality of states of the resource failure based model and the resource usage based model for each corresponding resource configuration, and incorporating the long term fraction of time each model spends on each of the plurality of states.
2. The method of claim 1, wherein solving the Markov models for the resource failure model and the plurality of resource usage models includes conditioning each Markov model on an occurrence of a security attack on the software system.
3. The method of claim 1, wherein the resource usage based model monitors a number of resources being used in each resource configuration of the software system.
4. The method of claim 1, wherein the resource failure based model monitors a number of resources in the software system that fail due to a security attack.
5. The method of claim 1, wherein the system reliability metric measures performance as accuracy.
6. The method of claim 5, wherein accuracy includes determining a pass/fail criteria for each state of the resource usage models for a each defined failure configuration.
7. The method of claim 1, wherein the system reliability metric measures performance as efficiency.
8. The method of claim 7, wherein efficiency includes deriving the security test cases based on known security attacks, and updating the security test cases when new attacks are discovered.
9. A system for assessing an impact of a security attack on a software system, comprising:
a processor configured to obtain a measure of a system performance/reliability affecting metric for an observation period as a fraction of time the system satisfies a defined performance/reliability specification by defining a resource failure based Markov chain and a plurality of resource usage based Markov chains for the system, wherein each resource usage based chain is associated with a corresponding resource configuration,
using results of security test cases performed on said software system to obtaining pass/fail results for each of a plurality of states of the resource failure based Markov chain and the resource usage based Markov chain,
solving the resource failure based Markov chain and the resource usage based Markov chain for each corresponding resource configuration and obtaining a long term fraction of time each Markov chain spends on each state,
and
obtaining a measure of the system affecting metric according to the state probability; and
a memory configured to obtain a measure of the system performance/reliability affecting metric by summing the states for a plurality of system performance requirements met using the pass/fail results obtained from the plurality of states of the resource failure based model and the resource usage based model for each corresponding resource configuration, and incorporating the long term fraction of time each model spends on each of the plurality of states.
10. The system of claim 9, wherein solving the Markov chains for the resource failure model and the plurality of resource usage models includes conditioning each Markov chain on an occurrence of a security attack on the software system.
11. The system of claim 9, wherein resource usage based model monitors a number of resources being used in each resource configuration of the software system.
12. The system of claim 9, wherein the resource failure based model monitors a number of resources in the software system that fail due to a security attack.
13. The system of claim 9, wherein the system reliability metric measures performance as accuracy.
14. The system of claim 13, wherein accuracy includes determining a pass/fail criteria for each state of the resource usage models for a each defined failure configuration.
15. The system of claim 9, wherein the system reliability metric measures performance as efficiency.
16. The system of claim 15, wherein efficiency includes deriving the security test cases based on known security attacks, and updating the security test cases when new attacks are discovered.
17. A non-transitory computer readable medium storing instructions executable by a processor to performed method for assessing an impact of a security attack on a software system, the method comprising:
defining a system performance/reliability affecting metric for an observation period as a fraction of time the system satisfies a defined performance/reliability specification;
defining a resource failure based Markov model and a plurality of resource usage based Markov models for the system, wherein each resource usage based model is associated with a corresponding resource configuration;
using results of security test cases performed on said software system to obtain pass/fail results for each of a plurality of states of the resource failure based model and the resource usage based model for each corresponding resource configuration;
solving the resource failure based Markov model and the plurality of resource usage based Markov models and obtaining a long term fraction of time each model spends on each of the plurality of states;
and
obtaining a measure of the system performance/reliability affecting metric by summing the states for a plurality of system performance requirements met using the pass/fail results obtained from the plurality of states of the resource failure based model and the resource usage based model for each corresponding resource configuration, and incorporating the long term fraction of time each model spends on each of the plurality of states.
Patent Citations (23)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US7013395(B1) | 2006-03-01 | Swiler et al. | Examiner |
| US7203622(B2) | 2007-04-01 | Pan et al. | Examiner |
| US7219049(B2) | 2007-05-01 | Prowell | Examiner |
| US7266484(B2) | 2007-09-01 | Lombardo et al. | Applicant |
| US7284274(B1) | 2007-10-01 | Walls et al. | Applicant |
| US7792779(B2) | 2010-09-01 | Dash | Examiner |
| US7818797(B1) | 2010-10-01 | Fan et al. | Examiner |
| US7992192(B2) | 2011-08-01 | Maher et al. | Examiner |
| US8079080(B2) | 2011-12-01 | Borders | Examiner |
| US8108929(B2) | 2012-01-01 | Agrawal et al. | Examiner |
| US8135661(B2) | 2012-03-01 | Olsson | Examiner |
| US2003/0014735(A1) | 2003-01-01 | Achlioptas et al. | Examiner |
| US2005/0060357(A1) | 2005-03-01 | Prowell | Examiner |
| US2006/0253839(A1) | 2006-11-01 | Avritzer et al. | Examiner |
| US2007/0240207(A1) | 2007-10-01 | Belakhdar et al. | Examiner |
| US2009/0044256(A1) | 2009-02-01 | Moyle et al. | Examiner |
| US2009/0094669(A1) | 2009-04-01 | Savadi et al. | Examiner |
| US2009/0144827(A1) | 2009-06-01 | Peinado et al. | Examiner |
| US2011/0185422(A1) | 2011-07-01 | Khayam et al. | Examiner |
| US2011/0208681(A1) | 2011-08-01 | Kuecuekyan | Examiner |
| US2012/0078823(A1) | 2012-03-01 | Suyama et al. | Examiner |
| US2012/0137367(A1) | 2012-05-01 | Dupont et al. | Examiner |
| US2013/0111588(A1) | 2013-05-01 | Agrawal et al. | Examiner |
Non-Patent Literature (8)
- Dacier, Marc, Yves Deswarte, and Mohamed Kaâniche. “Quantitative assessment of operational security: Models and tools.” Information Systems Security, ed. By SK Katsikas and D. Gritzalis, London, Chapman & Hall (1996): 179-86. [retrieved from Citeseer database on Dec. 16, 2012 ].Examiner
- Kim, Dong Seong, Khaja Mohammad Shazzad, and Jong Sou Park. “A framework of survivability model for wireless sensor network.” in Availability, Reliability and Security, 2006. ARES 2006. The First International Conference on, pp. 8-pp. IEEE, 2006. [retrived from IEEE database on Dec. 16, 2012].Examiner
- Ortalo, Rodolphe, Yves Deswarte, and Mohamed Kaâniche. “Experimenting with quantitative evaluation tools for monitoring operational security.” Software Engineering, IEEE Transactions on 25, No. 5 (1999): 633-650. [retrieved from IEEE database on Dec. 16, 2012].Examiner
- Vijayaraghavan, V., and Sanjoy Paul. “iMeasure security (iMS): A novel framework for security quantification.” In Networks and Communications, 2009. NETCOM'09. First International Conference on, pp. 414-421. IEEE, 2009. [retrieved from IEEE database on Dec. 16, 2012].Examiner
- Pimentel, Andy D., L. O. Hertzbetger, Paul Lieverse, Pieter van der Wolf, and E. E. Deprettere. “Exploring embedded-systems architectures with Artemis.” Computer 34, No. 11 (2001): 57-63. [retrieved from IEEE database on Dec. 16, 2012].Examiner
- Lili, Shi, Yang Shoubao, Guo Liangmin, and Wu Bin. “A Markov chain based resource prediction in computational grid.” in Frontier of Computer Science and Technology, 2009. FCST'09. Fourth International Conference on, pp. 119-124. IEEE, 2009. [retreved from IEEE database on Dec. 16, 2012].Examiner
- Lings et al., “A Reference Model for Successful Distributed Development of Software Systems,” Proc. Second IEEE International Conference on Global Software Engineering, Munich, Germany, Aug. 2007, 130-139.Applicant
- Houmb et al, “Estimating Impact and Frequency of Risks to Safety and Mission Critical Systems Using CVSS,” WDSE 2008, Seattle, Nov. 2008.Applicant