Gradle tries to be very clever about finding out whether a task is up-to-date, or whether it needs to rerun. This is quite a good concept for all the tasks which convert something from input directories to output directories (e.g. compiling), but it is quite complicated and non-intuitive when tests are executed. Actually, I expect that all the tests are executed when I run a test task, independent of whether the test sources have changed and whether the tests have already be executed. To do this is easy, just add some configuration to the respective test task.

For the ordinary test task, do this:

test {
  outputs.upToDateWhen { false }
}

When using the testset plugin (org.unbroken-dome.test-sets), you can do the same for all your test levels:

acceptanceTest {
 outputs.upToDateWhen { false }
}
systemTest {
 outputs.upToDateWhen { false }
}
integrationTest {
 outputs.upToDateWhen { false }
}

more information: