Acceptance Testing - Revision history

User at 21:45, 17 July 2023

2023-07-17T21:45:20Z

User at 15:11, 23 November 2022

2022-11-23T15:11:34Z

Show changes

User: The LinkTitles extension automatically added links to existing pages (https://github.com/bovender/LinkTitles).

2021-02-06T13:27:58Z

The LinkTitles extension automatically added links to existing pages (https://github.com/bovender/LinkTitles).

User: Acceptance Testing is a type of testing done by users, customers, or other authorised entities to determine application/software needs and business processes.

2018-11-27T17:18:50Z

Acceptance Testing is a type of testing done by users, customers, or other authorised entities to determine application/software needs and business processes.

New page

Acceptance Testing is a type of testing done by users, customers, or other authorised entities to determine application/software needs and business processes. Acceptance testing is the most important phase of testing as this decides whether the client approves the application/software or not. It may involve functionality, usability, performance, and U.I of the application. It is also known as user acceptance testing (UAT), operational acceptance testing (OAT), and end-user testing. It is one of the final stages of the software’s testing cycle and often occurs before a client or customer accepts the new application. Acceptance tests are black-box system tests. Users of the system perform tests in line with what would occur in real-time scenarios and verify whether or not the software/application meets all specifications.<ref>Definition - What is Acceptance Testing? [https://economictimes.indiatimes.com/definition/acceptance-testing Economics Times]</ref>

Acceptance Testing is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery.

[[File: Acceptance Testing.jpg|300px|Acceptance Testing]]<br />
source: [ https://www.istqb.org/ ISTBQ]

The acceptance testing process, which is also known as end-user testing, operational acceptance testing or field testing, acts as a form of quality control to identify problems and defects while they can still be corrected relatively painlessly. It is one of the final stages of the software’s testing cycle and often occurs before a client or customer accepts the new application — and encourages close collaboration between developers and customers. Acceptance tests are designed to replicate the anticipated real-life use of the product to verify that it is fully functional and complies with the specifications agreed between customer and manufacturer. They may involve chemical tests, physical tests or performance tests, which may be refined if needed. If the actual results match the expected results for each test case, the product will pass and be considered adequate. It will then either be rejected or accepted by the customer.<ref>Explaining Acceptance Testing [https://www.investopedia.com/terms/a/acceptance-testing.asp Techopedia]</ref>

'''Origins of Acceptance Testing'''<ref>History of Acceptance Testing [https://www.agilealliance.org/glossary/acceptance/#q=~(filters~(postType~(~'page~'post~'aa_book~'aa_event_session~'aa_experience_report~'aa_glossary~'aa_research_paper~'aa_video)~tags~(~'acceptance*20test))~searchTerm~'~sort~false~sortDirection~'asc~page~1) Agile Alliance]</ref>
*1996: Automated tests identified as a practice of [^Extreme-Programming-XP|Extreme Programming], without much emphasis on the distinction between unit and acceptance testing, and with no particular notation or tool recommended
*2002: Ward Cunningham, one of the inventors of Extreme Programming, publishes Fit, a tool for acceptance testing based on a tabular, Excel-like notation
*2003: Bob Martin combines Fit with Wikis (another invention of Cunningham's), creating FitNesse
*2003-2006: Fit/FitNesse combo eclipses most other tools and becomes the mainstream model for Agile acceptance testing

'''Who Performs Acceptance Testing'''<ref>Who performs Acceptance Testing? [http://softwaretestingfundamentals.com/acceptance-testing/ Software Fundamentals]</ref>
Acceptance Testing is the fourth and last level of software testing performed after System Testing and before making the system available for actual use.
*Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales and/or Customer Support.
*External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
- Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. (This is in the case of the software not being owned by the organization that developed it.)
- User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.

'''Key Points for Acceptance Testing'''<ref>Key Points for Acceptance Testing [https://www.testimate.fi/en/services/acceptance-testing/ Testimate]</ref>

*Test that the entire system works according to specifications and serves its purpose
Acceptance testing ensures that the supplied system meets its requirements and other contract details. It is important to check the relevant business processes in an end-to-end manner in order to ensure that the system fully meets the business needs.
*Independence from the supplier/developer
Objective, non-biased acceptance testing requires a testing party that is independent from the system supplier or developer.
*Consider the end user needs
The system’s end users are in a key role during test planning and while testing the business processes. However, experienced testing professionals are needed to discover issues that are overlooked by normal users. Testimate specialises in testing business and technical requirements from the end user’s point of view.

'''Types of Acceptance Testing'''<ref>What are the Different Types of Acceptance Testing? [https://simplicable.com/new/acceptance-testing Simplicable]</ref>

Acceptance testing is the practice of confirming that a product, service, system, process, practice or document meets a set of requirements. As the term suggests, acceptance testing is the process of accepting a project release or change request for launch. The following are common types of acceptance testing.
*Non-Functional Testing: Non-Functional testing is a broad term for any test designed to confirm a non-functional requirement. This includes areas such as accessibility, quality, security, operations and design that aren't considered a functional requirement by a particular business unit.
*Operational Testing: Testing conducted by teams who are responsible for support, maintenance and day-to-day operations of a process, service or system. Operational testing confirms requirements such as performance, interoperability, backup, recovery and maintenance procedures.
*Performance Testing: Performance testing confirms that user interfaces and automation remain responsive under anticipated workload, volumes and stresses.
*Regression Testing: Regression tests confirm that changes didn't break existing functionality or non-functional qualities.
*Smoke Testing: A basic test that a build is successful. Typically performed by developers before announcing that a release is ready for testing. The term extends from initial product development tests that switch on an electronic device for the first time in the hopes that it doesn't begin smoking and catch on fire.
*Security Testing: The practice of testing information technology for security vulnerabilities using known threats and attack patterns.
*User Acceptance Testing: Testing of user interfaces by stakeholders or test analysts representing stakeholders.

'''Acceptance Testing Process'''<ref>Acceptance Testing Process [https://www.softwaretestinghelp.com/what-is-acceptance-testing/ Software Testing Help]</ref>

[[File: Acceptance Testing1.jpg|300px|Acceptance Testing 1]]<br />
source: [https://www.softwaretestinghelp.com/what-is-acceptance-testing/ Software Testing Help]

*Business Requirements Analysis: Business requirements are analyzed by referring all the available documents within the project.
**System Requirement Specifications
**Business Requirements Document
**Use Cases
**Workflow diagrams
**Designed data matrix

*Design Acceptance Test Plan: There are certain items to be documented in the Acceptance Test Plan.
**Acceptance Testing strategy and approach.
**Entry and exit criteria should be well-defined.
**The scope of AT should be well-mentioned and it has to cover only the business requirements.
**Acceptance test design approach should be detailed so as anyone writing tests can easily understand the way in which it has to be written.
**Test Bed set up, actual testing schedule/timelines should be mentioned.
**As testing is conducted by different stakeholders, details on logging bug should be mentioned as the stakeholders may not be aware of the procedure followed.

*Design and Review Acceptance Tests
Acceptance tests should be written at a scenario level mentioning what has to be done (not in-detail to include how to do). These should be written only for the identified areas of scope for business requirements, and each and every test has to be mapped to its referencing requirement.

All the written acceptance tests have to be reviewed to achieve high coverage on business requirements.

This is to make sure that any other tests apart from scope mentioned are not involved so that testing lies within the scheduled timelines.

*Acceptance Test Bed Set up
Test Bed should be set up similar to a Production environment. Very high-level checks are required to confirm on environment stability and usage. Share the credentials to use the environment only with a stakeholder who is performing this testing.

*Acceptance Test Data Set Up
Production data has to be prepared/populated as test data in the systems. Also, there should be a detailed document in such a way that the data has to be used for testing.

Do not have the test data like TestName1, TestCity1, etc., Instead have Albert, Mexico, etc. This gives a rich experience of real-time data and testing will be up-to-the-point.

*Acceptance Test Execution
Designed Acceptance tests have to be executed on the environment at this step. Ideally, all the tests should pass at the first attempt itself. There should be no functional bugs arising out of Acceptance testing, if any, then they should be reported at a high priority to be fixed.

Again, bugs fixed have to be verified and closed as a high priority task. Test execution report has to be shared on a daily basis.

Bugs logged in this phase should be discussed in a bug-triage meeting and has to undergo Root Cause Analysis procedure. This is the only point where acceptance testing assess whether all the business requirements are actually met by the product or not.

*Business Decision
There comes out a Go/No-Go decision for the product to be launched in Production. Go decision will take the product ahead to be released to the market. No-Go decision marks the product as Failure. Few factors of No-Go Decision:
**Poor Quality of the product.
**Too Many open Functional Bugs.
**Deviation from business requirements.
**Not up to the market standards and needs enhancements to match the current market standards.

'''Acceptance Testing in Extreme Programming'''<ref>Acceptance Testing in Extreme Programming [https://en.wikipedia.org/wiki/Acceptance_testing Wikipedia]</ref>

Acceptance testing is a term used in agile software development methodologies, particularly extreme programming, referring to the functional testing of a user story by the software development team during the implementation phase.

The customer specifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, whatever it takes to ensure the functionality works. Acceptance tests are black-box system tests. Each acceptance test represents some expected result from the system. Customers are responsible for verifying the correctness of the acceptance tests and reviewing test scores to decide which failed tests are of highest priority. Acceptance tests are also used as regression tests prior to a production release. A user story is not considered complete until it has passed its acceptance tests. This means that new acceptance tests must be created for each iteration or the development team will report zero progress

'''Benefits and Pitfalls of Acceptance Testing'''<ref>What are the Benefits and Pitfalls of Acceptance Testing [https://www.agilealliance.org/glossary/acceptance/#q=~(filters~(postType~(~'page~'post~'aa_book~'aa_event_session~'aa_experience_report~'aa_glossary~'aa_research_paper~'aa_video)~tags~(~'acceptance*20test))~searchTerm~'~sort~false~sortDirection~'asc~page~1) agilealliance.org]</ref>

'''Expected Benefits'''

Acceptance testing has the following benefits, complementing those which can be obtained from unit tests:
*encouraging closer collaboration between developers on the one hand and customers, users or domain experts on the other, as they entail that business requirements should be expressed
*providing a clear and unambiguous "contract" between customers and developers; a product which passes acceptance tests will be considered adequate (though customers and developers might refine existing tests or suggest new ones as necessary)
*decreasing the chance and severity both of new defects and regressions (defects impairing functionality previously reviewed and declared acceptable)

'''Common Pitfalls'''

Expressing acceptance tests in an overly technical manner. Customers and domain experts, the primary audience for acceptance tests, find acceptance tests that contain implementation details difficult to review and understand. To prevent acceptance tests from being overly concerned with technical implementation, involve customers and/or domain experts in the creation and discussion of acceptance tests. Acceptance tests that are unduly focused on technical implementation also run a the risk of failing due to minor or cosmetic changes which in reality do not have any impact on the product's behavior. For example, if an acceptance test references the label for a text field, and that label changes, the acceptance test fails even though the actual functioning of the product is not impacted.

===References===
<references/>

===Further Reading===
*Acceptance Test Engineering Guide [https://archive.codeplex.com/?p=testingguidance Grigori Melnik, Gerard Meszaros]
*Automated Acceptance Testing: A Literature Review and an Industrial Case Study [https://www.computer.org/csdl/proceedings/agile/2008/3321/00/3321a027-abs.html B. Haugset and G. K. Hanssen]
*User Acceptance Testing - A step-by-step guide [https://www.safaribooksonline.com/library/view/user-acceptance-testing/9781780171678/ Brian Hambling, Pauline van Goethem]

@@ Line 22: / Line 22: @@
 *Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales, and/or Customer Support.
 *External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
-   - Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. (This is in the case of the software not being owned by the organization that developed it.)
+- Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. (This is in the case of the software not being owned by the organization that developed it.)
-   - User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.
+- User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the software. They can be the customers themselves or the customers’ customers.

@@ Line 1: / Line 1: @@
-Acceptance Testing is a type of testing done by users, customers, or other authorised entities to determine application/software needs and business processes. Acceptance testing is the most important phase of testing as this decides whether the client approves the application/software or not. It may involve functionality, usability, performance, and U.I of the application. It is also known as user acceptance testing (UAT), operational acceptance testing (OAT), and end-user testing. It is one of the final stages of the software’s testing cycle and often occurs before a client or customer accepts the new application. Acceptance tests are black-box system tests. Users of the system perform tests in line with what would occur in real-time scenarios and verify whether or not the software/application meets all specifications.<ref>Definition - What is Acceptance Testing? [https://economictimes.indiatimes.com/definition/acceptance-testing Economics Times]</ref>
+Acceptance Testing is a type of testing done by users, customers, or other authorised entities to determine [[application]]/software needs and [[business]] processes. Acceptance testing is the most important phase of testing as this decides whether the client approves the application/software or not. It may involve functionality, [[usability]], performance, and U.I of the application. It is also known as user acceptance testing (UAT), operational acceptance testing (OAT), and end-user testing. It is one of the final stages of the software’s testing cycle and often occurs before a client or [[customer]] accepts the new application. Acceptance tests are black-box [[system]] tests. Users of the system perform tests in line with what would occur in real-time scenarios and verify whether or not the [[software]]/application meets all specifications.<ref>Definition - What is Acceptance Testing? [https://economictimes.indiatimes.com/definition/acceptance-testing Economics Times]</ref>
-Acceptance Testing is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system’s compliance with the business requirements and assess whether it is acceptable for delivery.
+Acceptance Testing is a level of software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system’s [[compliance]] with the business requirements and assess whether it is acceptable for delivery.
@@ Line 8: / Line 8: @@
-The acceptance testing process, which is also known as end-user testing, operational acceptance testing or field testing, acts as a form of quality control to identify problems and defects while they can still be corrected relatively painlessly. It is one of the final stages of the software’s testing cycle and often occurs before a client or customer accepts the new application — and encourages close collaboration between developers and customers. Acceptance tests are designed to replicate the anticipated real-life use of the product to verify that it is fully functional and complies with the specifications agreed between customer and manufacturer. They may involve chemical tests, physical tests or performance tests, which may be refined if needed. If the actual results match the expected results for each test case, the product will pass and be considered adequate. It will then either be rejected or accepted by the customer.<ref>Explaining Acceptance Testing [https://www.investopedia.com/terms/a/acceptance-testing.asp Techopedia]</ref>
+The acceptance testing [[process]], which is also known as end-user testing, operational acceptance testing or field testing, acts as a form of [[quality]] [[control]] to identify problems and defects while they can still be corrected relatively painlessly. It is one of the final stages of the software’s testing cycle and often occurs before a client or customer accepts the new application — and encourages close [[collaboration]] between developers and customers. Acceptance tests are designed to replicate the anticipated real-life use of the [[product]] to verify that it is fully functional and complies with the specifications agreed between customer and manufacturer. They may involve chemical tests, physical tests or performance tests, which may be refined if needed. If the actual results match the expected results for each test case, the product will pass and be considered adequate. It will then either be rejected or accepted by the customer.<ref>Explaining Acceptance Testing [https://www.investopedia.com/terms/a/acceptance-testing.asp Techopedia]</ref>
@@ Line 15: / Line 15: @@
 *2002: Ward Cunningham, one of the inventors of Extreme Programming, publishes Fit, a tool for acceptance testing based on a tabular, Excel-like notation
 *2003: Bob Martin combines Fit with Wikis (another invention of Cunningham's), creating FitNesse
-*2003-2006: Fit/FitNesse combo eclipses most other tools and becomes the mainstream model for Agile acceptance testing
+*2003-2006: Fit/FitNesse combo eclipses most other tools and becomes the mainstream [[model]] for Agile acceptance testing
 '''Who Performs Acceptance Testing'''<ref>Who performs Acceptance Testing? [http://softwaretestingfundamentals.com/acceptance-testing/ Software Fundamentals]</ref>
 Acceptance Testing is the fourth and last level of software testing performed after System Testing and before making the system available for actual use.
-*Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the organization that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product Management, Sales and/or Customer Support.
+*Internal Acceptance Testing (Also known as Alpha Testing) is performed by members of the [[organization]] that developed the software but who are not directly involved in the project (Development or Testing). Usually, it is the members of Product [[Management]], Sales and/or Customer Support.
 *External Acceptance Testing is performed by people who are not employees of the organization that developed the software.
     - Customer Acceptance Testing is performed by the customers of the organization that developed the software. They are the ones who asked the organization to develop the software. (This is in the case of the software not being owned by the organization that developed it.)
@@ Line 30: / Line 30: @@
 *Test that the entire system works according to specifications and serves its purpose
 Acceptance testing ensures that the supplied system meets its requirements and other contract details. It is important to check the relevant business processes in an end-to-end manner in order to ensure that the system fully meets the business needs.
-*Independence from the supplier/developer
+*Independence from the [[supplier]]/developer
-Objective, non-biased acceptance testing requires a testing party that is independent from the system supplier or developer.
+[[Objective]], non-biased acceptance testing requires a testing party that is independent from the system supplier or developer.
 *Consider the end user needs
 The system’s end users are in a key role during test planning and while testing the business processes. However, experienced testing professionals are needed to discover issues that are overlooked by normal users. Testimate specialises in testing business and technical requirements from the end user’s point of view.
@@ Line 38: / Line 38: @@
 '''Types of Acceptance Testing'''<ref>What are the Different Types of Acceptance Testing? [https://simplicable.com/new/acceptance-testing Simplicable]</ref>
-Acceptance testing is the practice of confirming that a product, service, system, process, practice or document meets a set of requirements. As the term suggests, acceptance testing is the process of accepting a project release or change request for launch. The following are common types of acceptance testing.
+Acceptance testing is the practice of confirming that a product, [[service]], system, process, practice or document meets a set of requirements. As the term suggests, acceptance testing is the process of accepting a project release or change request for launch. The following are common types of acceptance testing.
-*Non-Functional Testing: Non-Functional testing is a broad term for any test designed to confirm a non-functional requirement. This includes areas such as accessibility, quality, security, operations and design that aren't considered a functional requirement by a particular business unit.
+*Non-Functional Testing: Non-Functional testing is a broad term for any test designed to confirm a non-functional [[requirement]]. This includes areas such as accessibility, quality, security, operations and [[design]] that aren't considered a functional requirement by a particular business unit.
-*Operational Testing: Testing conducted by teams who are responsible for support, maintenance and day-to-day operations of a process, service or system. Operational testing confirms requirements such as performance, interoperability, backup, recovery and maintenance procedures.
+*Operational Testing: Testing conducted by teams who are responsible for support, maintenance and day-to-day operations of a process, service or system. Operational testing confirms requirements such as performance, [[interoperability]], backup, recovery and maintenance procedures.
 *Performance Testing: Performance testing confirms that user interfaces and automation remain responsive under anticipated workload, volumes and stresses.
 *Regression Testing: Regression tests confirm that changes didn't break existing functionality or non-functional qualities.
-*Smoke Testing: A basic test that a build is successful. Typically performed by developers before announcing that a release is ready for testing. The term extends from initial product development tests that switch on an electronic device for the first time in the hopes that it doesn't begin smoking and catch on fire.
+*Smoke Testing: A basic test that a build is successful. Typically performed by developers before announcing that a release is ready for testing. The term extends from initial product development tests that switch on an electronic [[device]] for the first time in the hopes that it doesn't begin smoking and catch on fire.
 *Security Testing: The practice of testing information technology for security vulnerabilities using known threats and attack patterns.
 *User Acceptance Testing: Testing of user interfaces by stakeholders or test analysts representing stakeholders.
@@ Line 54: / Line 54: @@
-*Business Requirements Analysis: Business requirements are analyzed by referring all the available documents within the project.
+*[[Business Requirements]] Analysis: Business requirements are analyzed by referring all the available documents within the project.
 **System Requirement Specifications
 **Business Requirements Document
 **Use Cases
-**Workflow diagrams
+**[[Workflow]] diagrams
-**Designed data matrix
+**Designed [[data]] matrix
 *Design Acceptance Test Plan: There are certain items to be documented in the Acceptance Test Plan.
-**Acceptance Testing strategy and approach.
+**Acceptance Testing [[strategy]] and approach.
 **Entry and exit criteria should be well-defined.
 **The scope of AT should be well-mentioned and it has to cover only the business requirements.
@@ Line 77: / Line 77: @@
 *Acceptance Test Bed Set up
-Test Bed should be set up similar to a Production environment. Very high-level checks are required to confirm on environment stability and usage. Share the credentials to use the environment only with a stakeholder who is performing this testing.
+Test Bed should be set up similar to a Production environment. Very high-level checks are required to confirm on environment stability and usage. Share the credentials to use the environment only with a [[stakeholder]] who is performing this testing.
 *Acceptance Test Data Set Up
@@ Line 89: / Line 89: @@
 Again, bugs fixed have to be verified and closed as a high priority task. Test execution report has to be shared on a daily basis.
-Bugs logged in this phase should be discussed in a bug-triage meeting and has to undergo Root Cause Analysis procedure. This is the only point where acceptance testing assess whether all the business requirements are actually met by the product or not.
+Bugs logged in this phase should be discussed in a bug-triage meeting and has to undergo [[Root Cause]] Analysis procedure. This is the only point where acceptance testing assess whether all the business requirements are actually met by the product or not.
 *Business Decision
-There comes out a Go/No-Go decision for the product to be launched in Production. Go decision will take the product ahead to be released to the market. No-Go decision marks the product as Failure. Few factors of No-Go Decision:
+There comes out a Go/No-Go decision for the product to be launched in Production. Go decision will take the product ahead to be released to the [[market]]. No-Go decision marks the product as Failure. Few factors of No-Go Decision:
 **Poor Quality of the product.
 **Too Many open Functional Bugs.
@@ Line 119: / Line 119: @@
 '''Common Pitfalls'''
-Expressing acceptance tests in an overly technical manner. Customers and domain experts, the primary audience for acceptance tests, find acceptance tests that contain implementation details difficult to review and understand.  To prevent acceptance tests from being overly concerned with technical implementation, involve customers and/or domain experts in the creation and discussion of acceptance tests. Acceptance tests that are unduly focused on technical implementation also run a the risk of failing due to minor or cosmetic changes which in reality do not have any impact on the product's behavior.  For example, if an acceptance test references the label for a text field, and that label changes, the acceptance test fails even though the actual functioning of the product is not impacted.
+Expressing acceptance tests in an overly technical manner. Customers and domain experts, the primary audience for acceptance tests, find acceptance tests that contain implementation details difficult to review and understand.  To prevent acceptance tests from being overly concerned with technical implementation, involve customers and/or domain experts in the creation and discussion of acceptance tests. Acceptance tests that are unduly focused on technical implementation also run a the [[risk]] of failing due to minor or cosmetic changes which in reality do not have any [[impact]] on the product's behavior.  For example, if an acceptance test references the label for a text field, and that label changes, the acceptance test fails even though the actual functioning of the product is not impacted.