Is retrying failed automated tests a good idea?
Jul 12, 2023
Sam
Retrying failed automated tests can be useful for diagnosing flaky tests, but should not be a method to ignore or dismiss them.
Imagine a scientist whose experiment didn't produce the expected results.
Would it be wise to simply repeat the experiment without investigating why it failed?
Likely not. The scientist would take note of the failure, examine the factors that could have led to it, then modify the experiment accordingly.
Simply repeating the failed experiment without understanding the reason for the failure wouldn't contribute to scientific progress.
When an automated test fails, it's crucial to investigate why it failed. In some cases, it could be due to 'flaky' tests - tests that intermittently pass and fail without any changes to the code. In this case, retrying can help diagnose the issue.
However, simply retrying a failed test without addressing the underlying cause of the failure doesn't improve the software's quality or reliability.
Having said that, manual testing or user behaviour testing plays a significant role in quality assurance. These testers approach the software from an end-user's perspective, identifying potential usability issues that might be missed by automated tests.
Complementing this with codeless and low-code automation can further enhance the speed and reliability of testing.
Thus, while retrying failed automated tests can be part of the strategy, it should be balanced with understanding the reasons for failure and employing diverse testing methods to ensure comprehensive software quality assurance.
Retrying failed automated tests can be useful for diagnosing flaky tests, but should not be a method to ignore or dismiss them.
Imagine a scientist whose experiment didn't produce the expected results.
Would it be wise to simply repeat the experiment without investigating why it failed?
Likely not. The scientist would take note of the failure, examine the factors that could have led to it, then modify the experiment accordingly.
Simply repeating the failed experiment without understanding the reason for the failure wouldn't contribute to scientific progress.
When an automated test fails, it's crucial to investigate why it failed. In some cases, it could be due to 'flaky' tests - tests that intermittently pass and fail without any changes to the code. In this case, retrying can help diagnose the issue.
However, simply retrying a failed test without addressing the underlying cause of the failure doesn't improve the software's quality or reliability.
Having said that, manual testing or user behaviour testing plays a significant role in quality assurance. These testers approach the software from an end-user's perspective, identifying potential usability issues that might be missed by automated tests.
Complementing this with codeless and low-code automation can further enhance the speed and reliability of testing.
Thus, while retrying failed automated tests can be part of the strategy, it should be balanced with understanding the reasons for failure and employing diverse testing methods to ensure comprehensive software quality assurance.
Retrying failed automated tests can be useful for diagnosing flaky tests, but should not be a method to ignore or dismiss them.
Imagine a scientist whose experiment didn't produce the expected results.
Would it be wise to simply repeat the experiment without investigating why it failed?
Likely not. The scientist would take note of the failure, examine the factors that could have led to it, then modify the experiment accordingly.
Simply repeating the failed experiment without understanding the reason for the failure wouldn't contribute to scientific progress.
When an automated test fails, it's crucial to investigate why it failed. In some cases, it could be due to 'flaky' tests - tests that intermittently pass and fail without any changes to the code. In this case, retrying can help diagnose the issue.
However, simply retrying a failed test without addressing the underlying cause of the failure doesn't improve the software's quality or reliability.
Having said that, manual testing or user behaviour testing plays a significant role in quality assurance. These testers approach the software from an end-user's perspective, identifying potential usability issues that might be missed by automated tests.
Complementing this with codeless and low-code automation can further enhance the speed and reliability of testing.
Thus, while retrying failed automated tests can be part of the strategy, it should be balanced with understanding the reasons for failure and employing diverse testing methods to ensure comprehensive software quality assurance.
