Actions

Difference between revisions of "Risk Based Testing"

(Created page with "'''Content Coming Soon'''")
 
Line 1: Line 1:
'''Content Coming Soon'''
+
'''Risk Based Testing''' is an approach that takes a scientific approach when accounting for [[Risk|risk]]. It is mainly based on the factors of the business impact and the likelihood of failure, although there could be more.<ref>Defining Risk Based Testing [https://www.stickyminds.com/article/risk-based-testing-test-only-what-matters-0 Rajnish Mehta]</ref>
 +
 
 +
Risk based testing uses risk to prioritize and emphasize the appropriate tests during test execution. In simple terms – Risk is the [[Probability Theory|probability of occurrence]] of an undesirable outcome. This outcome is also associated with an impact. Since there might not be sufficient time to test all functionality, Risk based testing involves testing the functionality which has the highest impact and probability of failure. Risk-based testing is the idea that we can organize our testing efforts in a way that reduces the residual level of product risk when the system is deployed.
 +
*Risk-based testing starts early in the project, identifying risks to system quality and using that knowledge of risk to guide testing planning, specification, preparation and execution.
 +
*Risk-based testing involves both mitigation – testing to provide opportunities to reduce the likelihood of defects, especially high-impact defects – and contingency – testing to identify work-arounds to make the defects that do get past us less painful.
 +
*Risk-based testing also involves measuring how well we are doing at finding and removing defects in critical areas.
 +
*Risk-based testing can also involve using risk analysis to identify proactive opportunities to remove or prevent defects through non-testing activities and to help us select which test activities to perform.
 +
The goal of risk-based testing cannot practically be – a risk-free project. What we can get from risk-based testing is to carry out the testing with best practices in risk management to achieve a project outcome that balances risks with quality, features, budget and schedule.<ref>Explaining Risk Based Testing [http://tryqa.com/what-is-risk-based-testing/ Try QA]</ref>
 +
 
 +
 
 +
'''The Various Approaches to Risk Based Testing<ref>The Various Approaches to Risk Based Testing [https://www.researchgate.net/profile/Michael_Felderer/publication/259635976_Integrating_risk-based_testing_in_industrial_test_processes/links/00463531e0fc070d61000000/Integrating-risk-based-testing-in-industrial-test-processes.pdf Michael Felderer,  Rudolf Ramler]</ref>'''<br />
 +
Bach (1999) presents a pragmatic approach to risk-based testing grounded on a heuristic software risk analysis. Bach distinguishes inside-out risk analysis starting with details about a situation and identifying associated risk, and outside-in risk analysis starting with a set of potential risks and matching them to the details of the situation.
 +
Amland (2000) defines a risk-based testing approach that is based on Karolak's risk management process (Karolak 1995) comprising the following steps and the corresponding risk management activities: planning (risk identification and risk strategy), identification of risk indicators (part of risk assessment), identification of the cost of a failure (part of risk assessment), identification of critical elements (part of risk assessment), test execution (risk mitigation), and estimation of completion (risk reporting and risk prediction). Amland was one of the first in introducing a systematic risk-based testing approach. As such, it is not aligned with the standard test process.
 +
Redmill (2004; 2005) provides a thorough discussion of risk-based testing (Redmill 2004) as well as a proposal for practical application suggesting a single factor risk assessment, either on probability or on impact, or a two-factor risk assessment, in which probability and impact are combined (Redmill 2005). The application of the resulting risk classification in the test process, e.g., for test planning, design, execution or reporting, is beyond the scope of the approach.
 +
Felderer et al. (2012) propose a risk-based test process including the phases risk identification, test planning, risk assessment, test design as well as static and dynamic evaluation. The focus of their work is on the risk assessment model defined on the basis of an industrial project. In this model the risk coefficient is assigned to features and determined by impact, probability and time factors. Each factor is determined by
 +
criteria which are defined by metrics. Metrics are determined manually, semiautomated or automated.
 +
The Practical Risk-Based Testing Approach (PRISMA) (van Veenendaal 2012) distinguishes business and technical risks determined by weighted criteria to calculate the overall risk of the risk items. Additionally, PRISMA defines a process consisting of concrete activities, i.e., initiating, planning, kick-off meeting, extended risk identification, individual preparation, processing individual scores, consensus meeting, and define differentiated risk-based testing approach. The activities are defined in a very concrete way with detailed instructions.
 +
 
 +
In addition to generic risk-based testing approaches, also several model-driven approaches to risk-based testing have been introduced.
 +
Chen et al. (2002) use activity diagrams to represent requirements and use a risk analysis to select test cases for regression testing purposes. The risk of a test case is determined by its cost, measured based on the cost of the requirements the test covers, and its severity probability, measured based on the severity-weighted number of defects uncovered by the test case. The approach is intended to support regression testing of requirements, and does not consider a broad spectrum of arbitrary risk items as in our approach. Chen et al. focus on the phases risk assessment
 +
and test execution but not consider the overall risk-based test process.
 +
Stallbaum and Metzger (2007; 2008) introduce a model-driven risk-based system testing approach that is based on the Factor-Criteria-Metrics model (Cavano and McCall 1978). The focus of their approach is the annotation of risk assessment data in UML-based models and the automated generation of risk-based test suites, however, without a standard-aligned risk-based testing methodology and without an approach for its introduction in existing test processes.
 +
Finally, the model-driven risk-based testing approach of Wendland et al. (2012) formalizes requirements as integrated behavior trees, augments the integrated behavior tree with risk information, identifies for each risk in the integrated behavior tree an appropriate test directive, and finally passes both the risk-augmented integrated behavior tree and the test directive definition into a test generator.
 +
 
 +
 
 +
=== See Also ===
 +
 
 +
 
 +
===References ===
 +
<references/>
 +
 
 +
 
 +
=== Further Reading ===
 +
*A taxonomy of risk-based testing [https://arxiv.org/pdf/1912.11519.pdf Michael Felderer, Ina Schieferdecker]

Revision as of 20:52, 20 March 2020

Risk Based Testing is an approach that takes a scientific approach when accounting for risk. It is mainly based on the factors of the business impact and the likelihood of failure, although there could be more.[1]

Risk based testing uses risk to prioritize and emphasize the appropriate tests during test execution. In simple terms – Risk is the probability of occurrence of an undesirable outcome. This outcome is also associated with an impact. Since there might not be sufficient time to test all functionality, Risk based testing involves testing the functionality which has the highest impact and probability of failure. Risk-based testing is the idea that we can organize our testing efforts in a way that reduces the residual level of product risk when the system is deployed.

  • Risk-based testing starts early in the project, identifying risks to system quality and using that knowledge of risk to guide testing planning, specification, preparation and execution.
  • Risk-based testing involves both mitigation – testing to provide opportunities to reduce the likelihood of defects, especially high-impact defects – and contingency – testing to identify work-arounds to make the defects that do get past us less painful.
  • Risk-based testing also involves measuring how well we are doing at finding and removing defects in critical areas.
  • Risk-based testing can also involve using risk analysis to identify proactive opportunities to remove or prevent defects through non-testing activities and to help us select which test activities to perform.

The goal of risk-based testing cannot practically be – a risk-free project. What we can get from risk-based testing is to carry out the testing with best practices in risk management to achieve a project outcome that balances risks with quality, features, budget and schedule.[2]


The Various Approaches to Risk Based Testing[3]
Bach (1999) presents a pragmatic approach to risk-based testing grounded on a heuristic software risk analysis. Bach distinguishes inside-out risk analysis starting with details about a situation and identifying associated risk, and outside-in risk analysis starting with a set of potential risks and matching them to the details of the situation. Amland (2000) defines a risk-based testing approach that is based on Karolak's risk management process (Karolak 1995) comprising the following steps and the corresponding risk management activities: planning (risk identification and risk strategy), identification of risk indicators (part of risk assessment), identification of the cost of a failure (part of risk assessment), identification of critical elements (part of risk assessment), test execution (risk mitigation), and estimation of completion (risk reporting and risk prediction). Amland was one of the first in introducing a systematic risk-based testing approach. As such, it is not aligned with the standard test process. Redmill (2004; 2005) provides a thorough discussion of risk-based testing (Redmill 2004) as well as a proposal for practical application suggesting a single factor risk assessment, either on probability or on impact, or a two-factor risk assessment, in which probability and impact are combined (Redmill 2005). The application of the resulting risk classification in the test process, e.g., for test planning, design, execution or reporting, is beyond the scope of the approach. Felderer et al. (2012) propose a risk-based test process including the phases risk identification, test planning, risk assessment, test design as well as static and dynamic evaluation. The focus of their work is on the risk assessment model defined on the basis of an industrial project. In this model the risk coefficient is assigned to features and determined by impact, probability and time factors. Each factor is determined by criteria which are defined by metrics. Metrics are determined manually, semiautomated or automated. The Practical Risk-Based Testing Approach (PRISMA) (van Veenendaal 2012) distinguishes business and technical risks determined by weighted criteria to calculate the overall risk of the risk items. Additionally, PRISMA defines a process consisting of concrete activities, i.e., initiating, planning, kick-off meeting, extended risk identification, individual preparation, processing individual scores, consensus meeting, and define differentiated risk-based testing approach. The activities are defined in a very concrete way with detailed instructions.

In addition to generic risk-based testing approaches, also several model-driven approaches to risk-based testing have been introduced. Chen et al. (2002) use activity diagrams to represent requirements and use a risk analysis to select test cases for regression testing purposes. The risk of a test case is determined by its cost, measured based on the cost of the requirements the test covers, and its severity probability, measured based on the severity-weighted number of defects uncovered by the test case. The approach is intended to support regression testing of requirements, and does not consider a broad spectrum of arbitrary risk items as in our approach. Chen et al. focus on the phases risk assessment and test execution but not consider the overall risk-based test process. Stallbaum and Metzger (2007; 2008) introduce a model-driven risk-based system testing approach that is based on the Factor-Criteria-Metrics model (Cavano and McCall 1978). The focus of their approach is the annotation of risk assessment data in UML-based models and the automated generation of risk-based test suites, however, without a standard-aligned risk-based testing methodology and without an approach for its introduction in existing test processes. Finally, the model-driven risk-based testing approach of Wendland et al. (2012) formalizes requirements as integrated behavior trees, augments the integrated behavior tree with risk information, identifies for each risk in the integrated behavior tree an appropriate test directive, and finally passes both the risk-augmented integrated behavior tree and the test directive definition into a test generator.


See Also

References

  1. Defining Risk Based Testing Rajnish Mehta
  2. Explaining Risk Based Testing Try QA
  3. The Various Approaches to Risk Based Testing Michael Felderer, Rudolf Ramler


Further Reading