An edge case refers to a problem or situation that occurs only at an extreme (maximum or minimum) operating parameter. In software testing, edge cases are scenarios that arise when a system is pushed to its limits, such as handling the largest or smallest possible inputs, or dealing with unusual or unexpected inputs that fall at the extreme ends of the spectrum. These cases are important to identify as they may lead to unanticipated behavior and potentially expose issues in the system’s handling of boundary conditions. Edge cases are not the most common situations but they must be tested to ensure that a system is robust and behaves correctly under every scenario that could reasonably occur in the real world.
Catching edge cases requires a deeper domain knowledge and many times these edge cases are not covered in the published test plan. The domain knowledge that is needed could be knowledge of the production hardware, third-party software knowledge, or industry knowledge. For example, if working on an application in the insurance industry, knowing the ins and outs of insurance policies would help you test edge cases.
Edge Case Example
I want to go over one example from a project I tested a few years ago:
Background: A Windows Service in the application is responsible for extracting data from a SQL database to generate Excel and PDF reports according to user-defined parameters. Although there are restrictions limiting reports to a one-year timeframe, the volume of data can still be substantial.
Scenario: High-Volume Report Generation
Description: The system is designed to handle large amounts of data, but an edge case arises when users request several high-volume reports simultaneously. Each report can potentially contain hundreds of thousands of records, leading to individual report files that are multiple gigabytes in size.
Edge Case:
- Users generate multiple extensive reports concurrently, which are queued up for processing by the Windows Service.
- The size of these reports accumulates, significantly increasing memory usage on the server.
- The server is equipped with 64 GB of memory, which is substantial, yet there’s a risk of memory exhaustion due to the cumulative size of the queued reports.
Potential Issues to Investigate:
- Does the service effectively manage memory when processing and queuing large reports to prevent resource depletion?
- Is there a proper mechanism to queue the report generation tasks to prevent simultaneous processing of multiple large reports that could lead to server crashes?
- Are there fallback or recovery procedures in place for scenarios where the memory consumption approaches critical limits?
- What performance degradation occurs as the memory usage increases, and how does this affect the overall system stability?
Objective: This edge case scenario aims to test the system’s resilience and robustness in managing and processing extremely large datasets without compromising the server’s performance or stability. The goal is to identify any potential weaknesses in resource handling and to ensure that the system can recover gracefully from such demanding conditions without data loss or service interruption.
Edge Cases to Test Cases
The edge cases listed above were not necessarily test cases when testing started. In fact, after executing some initial test cases during the testing phase, I noticed the size of the PDFs and the slowness of the system when working with large datasets. I concluded that having too many of these reports running at the same time could potentially run the server out of memory.
Once I verified we had memory issues when running multiple large reports, these edge cases became test cases. These tests would always need to be run to verify the system always performed correctly when generating large reports.
Conclusion
Testing edge cases is an important part of software testing. These tests will not always be thought of while testers are writing test cases. Make sure management understands that this type of testing is important and that you cannot only execute the test cases and consider the system working.