Every time we’re testing Happy Path scenarios, we should expect the app to work as specified and nothing else. This is the way most users will probably use the app and it should be working pretty much flawlessly.
When we start to deviate from the Happy Path, we’ll start to encounter several scenarios. These might not be exactly what we expect our users to do, but are totally valid and should work as well.
Going back to “posting a photo to Instagram”, the Happy Path would be to do all the processes from within the app… But, Instagram also allows us to post photos from the gallery app on our phones. This is not the Happy Path and is not what most users would do, but is a totally valid action that should be supported and handled gracefully.
After exploring our positive scenarios, we’ll know most of the things we should be able to do with our app. But this won’t give us the test coverage we need. We also need to know what our app can’t do and how it should behave in those situations.
Again taking a look at our “posting a photo to Instagram” use case, what comes to your mind that shouldn’t be possible? What about unsupported file types? Should the app support photos with huge file sizes? What about funky aspect ratios like 21:9?
These are a mix of scenarios where the app shouldn’t allow us at all to proceed, and scenarios where the app should adjust what we input to something that’s valid. This should all be documented and tested.
After clearing all the functional tests, we can dive deep into the non-functional side of things. The non-functional testing is a pretty deep rabbit hole that can go as deep as the team is willing to. The type of non-functional tests applied to a product is completely up to the team, their priorities, and the budget. My favorites are Compatibility and Stress testing. Let’s take a look at each of them.
When doing compatibility testing, all we are doing is checking how our system behaves with other systems. Things like the OS version of our app, the version of our browser, the browser itself, or even the device we’re using at the time can affect our application.
Testing a mobile application on an iPhone is definitely not the same as doing it in a Samsung. But at the same time, testing it in a Samsung is not the same as testing it in a Google Pixel. So these are all things that need to go into consideration when designing your test plan. Usually, the best bet is to go for what your user base uses the most to ensure that your product works as expected for the majority of your clients.
In stress testing, the goal is to test how robust our application is. This type of testing is even more important in “mission-critical” apps where a crash could mean a huge loss for the company or for the user. This type of testing puts emphasis on how the app responds to heavy load, its availability of it, and how it handles errors when they do come up.
Ideally, a system should never stop and if an error does happen, the app needs to handle it gracefully and give the user an option to keep using it. This type of testing is usually confused with Load Testing. The key difference is that with Load Testing, the goal is to test how the application handles the normal expected workload in an environment that simulates the real-world usage it will get. While with Stress Testing, the tester goes beyond this point to test the absolute limit under sub-optimal conditions.
In future articles, we’ll go over how to mix and match these types of testing to build a good Test Plan.