Is anybody aware of a tool that takes two jars A and B and determines if A is a drop-in replacement for B, i.e. checks binary compatibility of a jar with an older version?

Would be so cool to catch these kind of problems in continuous integration...

I'm not familiar with such as tool... It would be indeed very cool. I guess it can be done with simply JUnit:

Steps:
1- write a test suite that exercises the public API only
2- compile it and create a jar
3- execute the tests by including the test jar + the framework jar (the code being tested for compatibility) in the CP
4- save the test jar in the project CVS
5- for any existing test jar, execute the tests by including the old test jars (compile with a previous version) + the new framework jar in the CP

What do you think? Has anyone done this before?

-Vincent

Not exactly what I was aiming at, but I guess this is what happens when you can't stop thinking about JUnit in Action ;-)

What I had in mind was an Ant task that takes two jars, "latest release" and "compilation result", and compares their public API on the byte code level, using a library like BCEL.

Upsides of this approach:

The "latest release" does not have to be in CVS, it can be a file or downloaded from any URL, e.g. from a server like the maven ibiblio repository.

Writing Unit tests is not required. That's a big plus for projects like GUI libraries where it's hard to write unit tests (or maybe it isn't - can't wait to read your book).

The build process is much easier. Just fire up an additional Ant task and provide the location of your latest release. No need to recompile with old jar, no need to manage any class path issues.

I follow the IDEA plugin development list, it seems that Timur Zambalayev has a tool to monitor API changes, see http://www.intellij.net/forums/thread.jsp?forum=23&thread=36605&tstart=0&trange=15

Don't know which tool he uses, if it can handle this problem, or if it can be used in CI and break the build for incompatible changes. If we can't find out or if his tool doesn't have a reasonable license, I guess this could be a nice new little project on sourceforge.

Just thought I'd let you know that I have started writing that thing and simple stuff is working:
- detect removed class
- detect gender change (Interface Class)

I could use some help, though - feel free to contact me if you're interested.

Hi Lars,

I think it's an excellent idea. It looks very similar to jdiff (http://jdiff.sourceforge.net/). Actually I can only see 2 main differences:
- JDiff works with source code whereas your solution works with BCEL (which I prefer).
- JDiff only generates HTML and not XML (which I'm sure your solution will use)

I'm sure you've seen that Timur has posted his source code (on the same thread).

The main difference with your idea and what I had in mind when I started this topic is that I was talking about binary compatibility whereas you are talking about API compatibility.

But I do agree, API compatbility is already a nice step forward and having a check for API compatibility as part of the build process looks very promising.

Actually I can envision a process where any API change would have to be explicitely documented possibly as nested items in the Ant task you're envisioning. Then some processing could generate an XML/HTML file describing the API changes. The idea would be that anything not described would be considered a build error.

Cool stuff!
-Vincent

Honestly, it will be way easier to implement such comparison tool using ASM.