Best practices #3: Speeding up test suite

Every developer and tester knows that it’s fundamental for every automation test to be fast and stable. But how to achieve the first of these two characteristics?

Here you can find 10 advices:

  1. First of all we need to realize that we are more interested in execution time of entire test suite rather than single test case. Therefore first thing we can do is to execute our tests in parallel. This is definitely the most effective approach among any other advices on the list. Most of modern test frameworks give such ability. Just check the documentation. For example when you use grade and testng you can set maxParallelFork parameter. But be careful, if your tests are not well designed and they have dependencies one on another probably some of them will start failing.
  2. Check parallelization strategy of your framework. Some of them run test cases in parallel and some classes. In the second approach when you have big test classes you may consider splitting them into smaller ones.
  3. If you have an ability of setting order of execution always put the most time consuming first. According to greedy algorithms sorting test cases from the lowest one to the fastest will give the best results.
  4. For UI and e2e tests consider using API of your application to perform test setup. Even if your application doesn’t provide any api you may try to use raw HTTP calls for this purpose. You may save a lot of time this way.
  5. If you are able to save state of your system consider not setting up everything before each test. Rather use already prepared environment.
  6. Use reports generated by test framework or CI server ( ex. Jenkins test plugin) for detecting the slowest test cases and consider refactoring.
  7. Never use hardcoded sleeps in your test code. Better approach is waiting for some event or polling the system with small time intervals.
  8. Keep your code clean. Legacy code tend to do much more then it is needed. Every action take some time. So focus on your code structure and coding standards.
  9. Avoid redundancy in your tests. Often couple of tests do the similar checks. In general there is no need of testing the same thing on many different levels. Lower level tests are faster so always test in the lowest possible one.
  10. Think of replacing external input/outputs of your application in test environment with mocks and introduce contract testing. Here you can find interesting talk about strategy for testing microservices in very efficient way by cucumber creator:
  11. Last, extra advice. Ask yourself if you truly need all of tests you have. Remember that even the most extensive test suite is useless when it is never or hardly ever executed. Fast feedback is very important so check if you are not having tests protecting you against defects that are not even a risk and consider switching them off.

Hope this short list will help you faster your test suites and make your life better 😉