Agile Test Automation is Incomplete Without Continuous Integration

Test Automation and Continuous Integration (CI) go hand in hand. In Agile world quick feedback is critical. Test Automation provides feedback about the quality of developed code and CI can accelerate that feedback!

What is Continuous Integration?

Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily, leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. Martin Fowler

Continuous integration is an ongoing and rapid process. This means that teams need to be able to react quickly when a build breaks or new tests need to be built. In order to integrate a quality code into existing features, it is important to validate it against existing tests. If the tests fail, the code (build) should be rejected until either the bugs are fixed or the test cases are updated.

Below are the steps to deploy a quality code using automated tests and CI:

1. Use source code repository (Version Control System) – It is the first and required step to start using continuous integration. For e.g. Github, CVS, Perforce, SVN

  • Introduce check-in policy – Check in policy may be introduced after establishing source code repository. This depends on the team and length of the sprints. Firstly it involves storing all necessary files in repository. Secondly, each developer should check-in as often as possible completed part of work. Frequently integration show problem early and either solution is easy or rollback is necessary for only small part of solution.

2. Automate build – Next step is to perform build automatically – no more than one action should be required to start it. Either developer should start it manually or the build is automatically triggered after each code check-in (commit). The build process could be scripted using Ant, CruiseControl etc.

3. Create auto-deployable test environment and use automated test suite – After each build new version of binaries should be automatically deployed to test server. After each deployment, kick off automated tests to validate the committed code against existing features. The automated tests act as gate keeper for new code before it is integrated. It is time consuming to run source code validation but saves time during manual/regression testing and improves over all code quality. A solid automation test suite acts as a safety net and filters out the buggy code to some level.

4. Report and integrate – Report the status of automated testing. Having a pass/fail label can help in rejecting the buggy code.

  • Introduce re-submission policy: Developers need to investigate the failures with the highest priority. They either should fix the bugs or update the test cases and go back to step 1 for re-submission. Once all the tests pass, the quality code should be integrated to the master repository and deployed to staging for further testing.