All continuous builds have a common point: when a project fails we have to find out why. There can be several reasons:

• Reason 1: a change in the project itself is causing the failure
• Reason 2: a change in one of the project's dependencies is causing the failure

Solving case 1 is relatively easy. However, solving case 2 is usually quite difficult. How could we make it easier?

Here's a proposal for improving continuous build tools:

When a project goes from a "success" build state to a "failure" build state, perform a source diff (using CVS or whatever SCM used) between the 2 dates on both the project itself + all its dependencies. Then generate a report showing all the changes.

This should allow to get a clear view of all the things that changed and that led to the build failure. This strategy seems especially well suited to continuous build systems as these builds are executed often and thus the differences between 2 builds should be small enough to get a clear picture.

Has anyone done this already? Does it work?

Note: I have suggested this idea to the Gump project. I would be very eager to see how it works out.

Implementation details:

• If your SCM is too slow to perform large diffs over several revisions there is still hope! Save the last build checkout and perform a file diff against the current checkout and the last saved one. If you're using CVS or an SCM that supports putting metainformation in source file (like $Id:$) then you'll also get to know who made the change. Otherwise, for each file change you'll need to query your SCM to know who made the change.
• I think it might be interesting to target the build failure emails to the list of people who have made changes since the last build + at least one person from the failed project's team.

Limitations:

• Say we have project A which depends on project B which depends on project C. If project A fails because of a change in project C, then we won't know it unless we're doing diff on transitive dependencies. But that might be too long. I think a good strategy would be to only generate diff on the first level dependencies. Then possibly provide a feature to generate transitive dependencies on demand (but not at at every build failure - it'll be too costly I think).