With the evolution of test automation, the focus has shifted from testing to automation. Almost everyone in the industry aspires to automate processes, aiming to reduce manual efforts. Many have gone a step further by incorporating various buzzing technologies such as AI, ML, and Data Science into automation. Consequently, there is a prevailing tendency among QAs and other decision-makers involved in the process to consider exploratory testing very late in the process, or perhaps not consider it at all. This raises a question for QAs: Is exploratory testing still worthwhile in 2024?
This blog is an attempt to address the same question from my personal experience. I will be touching upon on different risks involved when you ignore the exploratory testing and completely rely on automated tests.
[Read what are few fundamental questions for QA to ask before they test]
Understanding Exploratory Testing Briefly
The literal meaning of Explore is “to try to discover; learn about“.
Exploratory testing involves systematic learning of the system. During the process of learning the system, you discover how system behaves when input provided to it was never thought before.
Tweet
Systematic Learning
When testers interact with the system, they dynamically approach the situation based on what they learn from interacting with the system.
Below is an example where S #1 is the ideal scenario to reach to the state where you can verify the total amount for products added in the basket. While the variations (S #2 – S #4) represent different paths to reach the same state. This knowledge is acquired only through interaction with the system.
In the below example the variations may be simple and easy to predict in advance. For a real world system, the variations would be challenging to anticipate in advance and can be learnt only while exploring the system.
Input and Output for Exploratory Testing
In S #2, #3, and #4, the input to the system varies, but the expected output remains the same. This is a fundamental aspect of the exploratory testing process: you alter inputs based on your interactions and expect output in accordance with the defined requirements of the system.
In summary, exploratory testing scenarios cannot be predicted in advance. They are driven by creativity and a dynamic approach to evaluating the system. Another crucial factor in exploratory testing is curiosity to discover new states within the system. It’s important to note that during exploratory testing, you don’t deviate much from the way real users interact with the system.
[Read here, What you should know before you transition to automation role]
The real question, is it still worth doing exploratory testing?
Well, this question arises because you may have a comprehensive suite of automated tests. You believe that we have sufficient test coverage and it will report if anything breaks during the code change process. That’s true, but it does not discover any new issue that have not been predicted yet.
In automated test coverage, you are overlooking the fact that these tests are scripted and lack the ability to explore the application to create variations. Your automated tests are the tests which you know in advance and you believe these tests should always pass as they are critical to the system.
These automated tests interact with the system in the same way all the time, it does not learn any thing about how real users may interact with the system. Most of the time the automated tests are very generalised approach to interact with the system.
On the other hand, exploratory testing involves variations, creativity and dynamic approach to test the system. You do not only test the happy paths of the system, rather you try to reach to a state by altering the path to ensure that what ever path is taken to reach to that state, the system behaves exactly the same as it is expected to be.
Additionally, the intent of exploratory testing is not to discover the most critical bugs. The critical bugs should be identified as early as possible by automated test. Instead, the exploratory testing is done to discover the bugs which are not very straightforward and mostly creates problem for users who are interacting with the system in unconventional way.
Exploratory testing assesses robustness of the system. It ensures that system behaves as expected even when an unconventional approach was used to interact with it.
Many times, I do exploratory testing to simulates users who do not know how to use the system or an user who may have a destructive mindset and trying to exploit the system.
Automated tests lack these capabilities.
So, is it still worth doing?
Absolutely. It’s the only way of uncovering those unexpected glitches and providing freedom to users to interact with system as they wish to do. It ensures that system is robust to accomodate every type of users, whether they are unconventional in their approach.
So, next time you’re in doubt whether to explore or stick to the automation, remember: exploration along with automation makes the system more robust and user friendly.
Subscribe to our newsletter to receive such articles directly to your inbox.
Thank you for reading this post, don't forget to subscribe!
