Test Pyramid Importance in test suite design across STLC
Testing pyramid is a framework for designing a test suite in order to gain speed and confidence in test coverage and quality of software released. It is one of the ways test lead decides the number of tests and its distribution across unit test, integration test, system or component test, e2e i.e. end to end test and user acceptance test level.
It is crucial to decide how many tests are sufficient for each different types, It is essential to consider how to maximize test coverage, time spent in test execution, feedback received from particular tests, frequency of test case modification needed to overcome pesticide paradox.
Test pyramid benefits us in balancing total number of tests in different types of testing, speed up test suite execution and control the cost incurred to modify functionality of software application.
To gain maximum benefit from test pyramid framework, we have to start testing from Unit tests which is at bottom of the pyramid to integration tests at the middle followed by system tests then User acceptance tests and User interface tests at the top of the pyramid, as it is easier at the unit test level and becomes complex as we go upwards.
It is advised to cover most of the tests in unit testing phase as it is less costly and easier to fix any defect at unit testing level for maximum test coverage.
More appropriate number of integration tests that involve connection with other components or modules with the unit, in addition to API or web services tests, and database connection tests, which confirms that software components works as single unit as expected.
Component or system tests are carried out to evaluate system’s compliance with the specified requirements.
Finally E2E i.e. end to end testing, sometimes known as functional or usability testing, which involve opening a browser, accessing a particular page, entering the inputs, validating the valid inputs, sending HTTP requests to the server along with valid data, connecting with database, updating the values in respective tables, viewing the updates and exporting the results as desired.
We arrange types of testing to be carried out in triangle and maximum number of tests at the bottom of pyramid and fewer at the top of pyramid.
Unit tests are designed such that it can catch defects and able to pin point root cause of it, making it quick to identify. The tests such as null pointer or business logic or handling data types say from integer to float, can be easier to locate in unit testing stage rather than in integration tests.
Time to execute unit test if take 1 millisecond then component test or integration test would take around 1 second to be carried out, whereas end to end functionality tests or acceptance GUI test would take about 1 minute. So if there are plenty of GUI tests that are required to get executed will take hours to get completed, making it costlier, hence it is desired that tests that cover logic and business requirements should get covered in unit tests.
GUI testing using automation scripts requires more time, preconditions to be met, takes time and configuration for running such tests, more complex to design and therefore costlier to implement and execute.
In case of mobile testing the bugs or defects can occur in terms of layout issues, OS issues, Network issue, authentication issue, and device incompatibility. Testing pyramid in case of mobile testing will have browser at the bottom, emulator / simulator will be middle layer, and real devices will be at the top of pyramid.
Browser related testing will include cross platform compatibility, functional system testing, responsive design of browser based on size of desktop, mobile, smart tab. Integration or simulation/emulator based testing for mobile will include native API integration, version or platform limitation of simulation that can be tested on. Real device testing in mobile include ease in use, performance, native API integration, change in OS and compatibility with mobile versions.
Conclusion
Testing pyramid is a concept or a framework that decides balancing of distribution of tests among various testing stages like unit tests where maximum tests should be carried out that covers business logic and requirements, integration tests which verifies the connectivity with database, API or HTTP request verification and finally end to end testing where software should be verified from start to end requirement, which is complex and takes most of test execution time.
Mobile test pyramid include browser at the bottom, simulator and emulator in the middle and real device at the top of the pyramid. Browser should be tested for cross platform compatibility, responsive to device size. Simulator/emulator is tested for native API integration, OS and mobile version compatibility. Real devices are tested for functionality, performance, API integrations.
Please leave your questions/comments/feedback below.
Happy Learning 🙂
Connect to me on Linked In (Click here)
On a mission to contribute to the Software Testing Community in all possible ways.
