This document intends to cover different levels of testing we need. And we’ll cover the testing objective, scope and metrics in other documents.
Unit Testing
Why
Unit testing checks the functionalities of the 'smallest' part of the system. It helps developers build confidence in features they implemented, and make changes quickly. Good unit cases serve as project documentation and are newbie/beginner friendly. Unit testing often can cover corner cases that cannot be covered in other levels of testing.
How
Use such libraries as JUNIT and TestContainers to make it automatic
Ideally, every function/method should be covered
Ideally, every branch, decision and condition of a function/method should be covered
Ideally, functions/methods that deal with input/output should be tested with both valid and invalid inputs
Ideally, every state and branch of Termporal.IO workflows should be covered if possible
Test cases dealing with BigDecimal values should be handled with special attention
Replace Mock with TestContainer if it is applicable.
Integration Testing
Why
Software modules are designed and implemented by different developers whose understanding and programming logic may differ from each other. Therefore, integration becomes necessary to verify that these modules can work together in unity. Plus, interactions with database and external services could be erroneous, and the system may fail to catch some exceptions.
How
Use such tools as docker, docker-compose, TestContainer, JUNIT to make it automatic
For module A, only integrate A’s dependent modules(database, broker and other modules) and neglect modules that depend on A
Test cases should checked the connectivities between the depending module and its dependent modules
All interfaces(Restful API, Message Queue) between the depending module and its dependent modules should be tested
Test cases should focus on the data flow among modules
Both happy/normal path and unhappy/exception path(invalid interface parameters, timeout and retries) should be covered
End-to-End Testing
Why
//TODO
How
//TODO
Smoking Testing
// we may not include it at this stage, but will have it later when the system is production ready
System Testing
Why
We need to make sure that all the modules, and external dependencies(database, third-party systems and other services) can work together as expected.
How
QA engineers should be responsible for the system testing
QA engineers should prepare related documents detailing test cases(precondition, input, output, expected result, actual result), these documents should be reviewed and approved by all business owners to make sure that the cases cover all the business requirements. These documents can be part of the acceptance documents
QA engineers should discuss the non-functional requirements(e.g: monitoring metrics) with business owners and tech leaders and other stakeholders to prepare corresponding test cases.
Use or create tools to make it automatic
Stress Testing
// we’ll have performance test at the later stage
Regulatory Conformance Testing
Why
Since we deal with financial and banking business, it’s very important to conduct regulatory conformance testing to check if our product complies with corresponding laws and regulations. Governments often have regulations regarding data storage, access and disclosure as well as security aspects. So, it’s very important to have this kind of testing here
How
// TODO
Penetration Testing
Why
Cyber security is very important for financial sectors. To avoid possible vulnerabilities, we need to invite security experts/white hat hackers to do penetration tests to make sure that our system is secure and our customer’s money is safe here.
How
// TODO
Acceptance Testing
Why
We need to evaluate our vendor’s deliveries when the contract is to complete. We assume that we can do that stage by stage, and don’t want to check everything at the final stage. Such things as performance requirements and non-functional requirements, should we include these items? Let’s figure out.
How
// TODO