Good Practices


Automated tests progress is nearly similar to the one of manual tests. Nevertheless, automation has specificities conditioning the way which automated tests are designed and realized.


Simplicity

An automated test must be as simple as possible : easy to write, easy to read and easy to understand. The writing of complex tests increases the risk of error and so of false positive (test fail although the tested application isn’t involved).


Functionnal Unicity

In the case of a manual acceptance test, it’s common to follow long test procedures who are verifying a lot of functionalities of the tested system in one scenario.

It’s necessary due to the specifics constraints of manual acceptance tests : in order to test the application, the tester has to set-up pre-requisites and verify post-conditions. Several system functions are tested like this with only one test.

Automated tests allow getting rid of those constraints because the setting-up of pre-requisites and verifications can be done without the tested application.

Each automated test can test only one function of the SUT (System Under Test).

This method has many advantages :

  • Tests are shorter and easier.
  • The qualification of a function doesn’t depend on the good functioning of another function.
  • During a campaign test, it’s possible to test only parts of function of the SUT.

Independence

The two steps of setting-up the pre-requisites and cleaning-up the environment must assure that test cases are strictly independant from each other.

The execution of a test case should never depend of the previous test case’s result.

This rule is essential for those reasons :

  • Campaign layout : for each campaign, it must be possible to choose which tests and in which sequence they must be executed.
  • Results interpretation : when tests depend on each other, it’s more complicated to interpret causes of failure. The failure of one test can lead to the failure of the next test. It’s impossible to know if they failed because of the first test’s failure or because of real issues.

Static Data Test

An automated test must be able to be replayed as many times as necessary and obtain each time the same result.

To make it possible, the simplest solution is to use identic data from one execution to another. This is particularly true for non-regression tests which are valid only if they are executed in strictly identical conditions.

This is possible thanks to the setting-up and cleaning-up steps of the environement.


Dynamic Data Test

There are 2 exceptions to the previous rule. Some data tests can’t be determined a priori because they depend on the context in which the test case is executed. Among those data, there are dates and data generated by the application.

1. DATES

All dataset containing dates is subject to expiration. For example, a contract which was active when tests were realized can expire after a certain period of time. This can lead to the failure of the tests who are using this data set.

To handle this problem, 2 strategies are possible :

  • The first one is to upgrade frequently test cases. This solution requires to set-up a follow-up procedure of datasets maintenance which could be expensive.
  • It’s preferable to set-up a mechanism which allows to define the dates at the moment of the test execution, relatively to the day date (for example, the first Monday of the month, the first open-day of the year, the previous month…).

2. DATA GENERATED BY THE APPLICATION

Some data generated by the tested application can’t be determined a priori, for example, identifiers or timestamps generated at the execution. Sometimes data in output of a test case must be used as input of the next test case. When it happens, it’s necessary to store those data in order to use them later.


Maintenance

One of the main brakes to automation tests stands in the need of maintaining them. That’s why automated tests concern stable functions of the tested system which are little set to expand.

Despite those precautions, features of the SUT are going to need maintenance. So we need to anticipate the features during the realization of the tests in order to minimize the maintenance charge.

1. DATA TEST CENTRALIZATION

Sometimes, because of the evolution of the data model for example, a test case must be revalued.

To minimize the maintenance charge, the data of a test must be centralized. Concretely, it means that the data of a test are replaced by parameters whose values are saved in one or several parameters file(s).

In the case of a test case reevaluation, only these parameters files are modified and not the totality of the test cases.

2. COMMON PARTS POOLING

The common steps of several test cases must be shared. So if a modification of the SUT affects a common step to several cases, corrections must be made at only one place. This implies :

  • To create shared modules from which test cases will be built.
  • To configure data of the shared parts in order to valorize them differently according to test cases.