This policy is in compliance with BSP MORB 148 IT Risk Management, Appendix 76 Chapter 8 (Systems Testing) - https://morb.bsp.gov.ph/appendix-76/
Purpose and Objectives
The purpose of the Test Policy document is to represent the testing philosophy and to provide a direction that the QA team should adhere to and follow.
The objective of the test is to verify that the functionality of the SaFi Mobile App works according to the specifications.
The test will execute and verify the product features, fix and all high and medium-severity defects per the entrance criteria, prioritize lower-severity defects for future fixing via CR (Change Request). The final product of the test is twofold:
EFPS-ready and MVP-ready software;
A set of stable test scripts that can be reused for Functional and UAT test execution.
Scope
This policy contains information about how to ensure that our testing process is effective and that our software applications meet the business requirements we have developed.
This policy describes the high-level principles, approach, and major objectives that will be undertaken towards software testing.
Policy
Guiding Principles
Testing will be focused on meeting the business objectives, cost efficiency, and quality.
There will be common, consistent procedures for all teams supporting testing activities.
Testing processes will be well-defined, yet flexible, with the ability to change as needed.
Testing activities will build upon previous stages to avoid redundancy or duplication of effort.
Testing environment and data will emulate a production environment as much as possible.
Testing will be a repeatable, quantifiable, and measurable activity.
Testing will be divided into distinct phases, each with clearly defined objectives and goals.
There will be entrance and exit criteria.
Types of Testing
Functional Test
Functional testing is a type of testing that checks the functions of the application to ensure that it is doing exactly what it is meant to do. Functional testing is carried out by feeding the input and validating the output from the application.
Smoke Test
This test verifies the important features are working and there are no showstoppers in the build that is under testing.
Sanity Test
It is a quick and basic test (or set of tests) to ensure that the code changes made are working properly without any bugs. It is a subset of Regression testing and is usually executed after the software product has passed the Smoke test.
Regression Test
This type of software test re-runs functional tests to ensure that a software application works as intended after any code changes, updates, revisions, improvements, or optimizations.
Exploratory Test
In exploratory testing, testers do not work on the basis of previously created test cases. They check a system without a plan in mind to discover bugs that users may face when navigating a website or app without a specific aim or direction.
Non-Functional Test
Non-Functional Testing is a type of testing used to evaluate a software application’s performance, usability, dependability, and other non-functional characteristics.
Load Test
This checks how systems function under a large number of concurrent virtual users performing transactions over a certain period of time.
Levels of Testing
Unit Test
Unit tests are at the bottom of the pyramid with comparatively larger test suites in size. They are written for each microservice and for all of their internal modules/components. The execution time is considered very low, but the scope of testing is higher with granular coverage.
Owner: Development Team
Component Test
Component tests are second to Unit Tests in size as this is nothing but the full‒functional test suite for all microservices. You will cover the tests for all possible cases, boundary values, edge cases, positive and negative cases, etc.
Owner: Development Team
Integration Test
Integration tests usually are only a handful by just writing 2 or 3 tests for each integration among the internal modules and external APIs. The size is considerably small.
Owner: Development Team
API Test
API testing is a set of quality assurance actions that include sending calls to the API, getting output, and validating the system’s response against the defined input parameters, in particular, the accuracy of data and data’s format, HTTP status codes, and error codes.
Owner: QA Team
System Test
System tests are very small‒sized suites with test scenarios concentrating on the major workflows and user journeys. The execution time is high as it may involve GUIs, but the scope is limited.
Owner: QA Team
Software Testing Life Cycle
Software Testing Life Cycle refers to a testing process which has specific steps to be executed in a definite sequence to ensure that the quality goals have been met. In the STLC process, each activity is carried out in a planned and systematic way.
Requirement Analysis
Requirement Analysis is the first phase of STLC. During this phase, the test team studies and analyzes the requirements from a testing perspective.
Test Planning
A test plan outlines the strategy that will be used to test an application, the resources that will be used, the test environment in which testing will be performed, the limitations of the testing, and the schedule of the testing activities.
Test Case Design
The test case development activity is started once the test planning activity is finished. This is the phase of STLC where the testing team writes down the detailed test cases. Along with test cases, the testing team also prepares the test data if any is required for testing. Once the test cases are ready, these test cases are reviewed by peer members or the QA lead. Also, the Requirement Traceability Matrix (RTM) is prepared. The Requirement Traceability Matrix is an industry-accepted format for tracking requirements where each test case is mapped with the requirement. Using this RTM we can track backward & forward traceability.
Test Environment Setup
Setting up the test environment is a vital part of the STLC. Basically, the test environment decides on which conditions software is tested. This is an independent activity and can be started parallel with Test Case Development. Test environment configuration must mimic the production environment in order to uncover any environment/configuration related issues.
Test Execution
The test team starts executing the test cases based on the planned test cases. If a test case result is Pass/Fail then the same should be updated in the test cases. A defect report should be prepared for failed test cases and should be reported to the Development Team through the bug tracking tool for fixing the defects. Retesting will be performed once the defect has been fixed.
Test Closure
The testing team will be called out for a meeting to evaluate cycle completion criteria based on Test Coverage, Quality, Time, Cost, Software, and Business Objectives. Discuss what all went well, and which area needs to be improved & take the lessons from the current STLC as input to upcoming test cycles, which will help to improve bottleneck in the STLC process. Once the test cycle is completed, then a test closure report & test metrics will be prepared.
Execution Strategy
Entry and Exit Criteria
The entry criteria refer to the desired conditions in order to start test execution; only the migration of the code and fixes need to be assessed at the end of each cycle.
The exit criteria are the desirable conditions that need to be met to proceed with the implementation.
Entry and exit criteria are flexible benchmarks. If they are not met, the QA team will assess the risk, identify mitigation actions and provide a recommendation. All this is input to the project manager for a final “go-no go” decision.
Test Cycles
There will be four cycles for functional testing. Each cycle will execute all the scripts.
The objective of the first cycle is to identify any blocking and critical defects in all major functionalities.
The object of the second cycle is to identify the remaining blocking, critical defects and most of the high defects. It is expected to use some workaround on high defects in order to proceed with the tests.
The objective of the third cycle is to identify remaining high and medium defects, remove the workaround from the second cycle, and correct gaps in the scripts.
The objective of the fourth cycle is to meet the 95% pass rate of the test cases and 95% of medium severity defects have been closed.
Validation and Defect Management
It is expected that the testers execute all the scripts in each of the cycles. However, it is recognized that the testers could also do additional testing if they identify a possible gap in the scripts. This is especially relevant in the second cycle when the Product Team joins the QA team in the execution of the test, since the Product Team has a deeper knowledge of the business processes. If a gap is identified, the scripts and traceability matrix will be updated, and then a defect logged against the scripts.
The defects will be tracked through JIRA only. The Squad teams will gather information on a daily basis from JIRA, and request additional details regarding the defects. The Squad teams will work on fixes.
It is the responsibility of the tester to open the defects, link them to the corresponding test case, assign an initial severity and status, retest, and close the defect.
It is the responsibility of the Squad Lead to review the severity of the defects and facilitate with the Squad team the fix and its implementation, communicate with testers when the test can continue or should be halted, request the tester to retest, and modify the status as the defect progresses through the cycle.
It is the responsibility of the Squad teams to review the defects in JIRA on a daily basis, ask for details if necessary, fix the defect, communicate to the Squad Lead that the fix is done, and implement the solution.
It is the responsibility of the Project Manager to communicate with the Product Owner and Business Owner about all the NON-critical defects, especially HIGH severity defects, and whether those bugs need to be fixed immediately or can wait for the next cadence.