What sets it apart from other such tool IMO is the UI and speed/quality of search. I especially like the ability to see who's sending the most mails to a list and the nice syntax coloring display of emails (in addition to the thread view). Another nice feature is that emails are indexed a few seconds after they have been received by the list (compared to several hours with other tools). I love it :)

My slides are available online.

If you're interested to meet up register on the Facebook event.

We have also organized a joint XWiki and Maven tech meetup on the 9th of October, at Terracotta. Indeed Terracotta has been extermely kind not only to lend us a room but also to cater for some pizzas and snacks :) Thanks Terraccotta!

The address is:

Terracotta Inc.
650 Townsend St. Suite 325
San Francisco, CA 94103 USA
+1 415 738 4059

Jason Van Zyl, creator of Maven will be there too!

I hope to see a lot of you there!

I'm also happy to report that XWiki (the company and open source project I work for) is a Sonatype Partner. I'd love to see Maven products built on top of XWiki. I think XWiki's flexibility makes it an ideal platform for building those products and at the same time empowering them with collaborative features. In the near future I'll work closely with Jason and his team to integrate XWiki and Maven and hopefully we'll see some cool things out of this.

Well done guys and long live Sonatype!

The conference was very well organized and the training room was superb. Thanks Vincent for setting this up.

I've made my slides available here.

I'm extremely excited on several accounts:

• I'll be coming back to my first love which is software development. Even though I've continued doing development over all those years it was mostly during my free time, working on open source projects (mostly Cactus, Maven and Cargo). I'm glad to be able to bring software development in my day time job.
• I've spent the last 9 years doing consulting (IT Architecture consulting and then Offshore software development consulting). I was keen to build a product and focus on that single job. I'll still do varied jobs of course but it'll be centered around a product.
• I strongly believe wikis are only just entering the general market. They were niche tools so far but I have the feeling they're going to get more mainstream in the coming years. Of course XWiki is not just a simple wiki, it's actually a second generation wiki, on which collaborative web application can be developed (in short, take almost any "Web 2.0 application" - whatever it means! - and XWiki is a good candidate for building it).

I'm now off to Javapolis 2006 where I'll be manning the XWiki booth with Ludovic and Guillaume. Come and say hello to us!

We were lucky to have Jason Van Zyl talk about new Maven2 stuff and especially the Repository Manager and the maven.org development platform. Emmanuel Venisse has done a presentation on Continuum, Fabrice Bellingard has presented a return of experience of implementing Maven in a large company and I have done a quick presentation of new quality features in Maven2. See Jean-Laurent's blog entry for more details.

This tool could be based on Clirr or JDiff for example. It would also have an option to fail the build if there are new methods without a @since tag.

Do you know if such a tool exists?

Here's a method we had (it expands a JAR file):

    public void expandToPath(String path) throws IOException
    {
         File workDir = new File(path);
         JarInputStream inputStream = getContentAsStream();
         
         byte[] buffer = new byte[40960];
         
         ZipEntry entry;
         while ((entry = inputStream.getNextEntry()) != null)
         {
              String entryName = entry.getName();
              entryName = entryName.replace('/', File.separatorChar);
              
              String outFileName = workDir.getPath() + File.separator + entryName;
              File outFile = new File(outFileName);
              
              if (outFileName.endsWith("/") || outFileName.endsWith("\\"))
              {
                   outFile.mkdirs();
               }
              else
              {
                   if (!outFile.getParentFile().exists())
                   {
                        outFile.getParentFile().mkdirs();
                    }
                   
                   if (!outFile.exists())
                   {
                        outFile.createNewFile();
                    }
                   
                   FileOutputStream out = new FileOutputStream(outFile);
                   int read;
                   while ((read = inputStream.read(buffer)) > 0)
                   {
                        out.write(buffer, 0, read);
                    }
                   
                   out.close();
               }
          }
         inputStream.close();
     }

Here's how I've transformed the method by removing all File operations and instead introducing a FileHandler interface with the following methods, equivalent to the File ones:

• append(URI, String): appends a suffix to a URI
• mkdirs(URI): create directories for the URI
• exists(URI): return true if the URI exists
• createFile(URI): create a file
• getOutputStream(URI): get an output stream for the passed URI

    public void expandToPath(URI path) throws IOException
    {
         JarInputStream inputStream = getContentAsStream();
 
         byte[] buffer = new byte[40960];
 
         ZipEntry entry;
         while ((entry = inputStream.getNextEntry()) != null)
         {
              String entryName = entry.getName();
  
              URI outFile = getFileHandler().append(path, entryName);
  
              if (outFile.toString().endsWith("/"))
              {
                   getFileHandler().mkdirs(outFile);
               }
              else
              {
                   if (!getFileHandler().exists(getFileHandler().getParent(outFile)))
                   {
                        getFileHandler().mkdirs(getFileHandler().getParent(outFile));
                    }
   
                   if (!getFileHandler().exists(outFile))
                   {
                        getFileHandler().createFile(outFile);
                    }
   
                   OutputStream out = getFileHandler().getOutputStream(outFile);
                   int read;
                   while ((read = inputStream.read(buffer)) > 0)
                   {
                        out.write(buffer, 0, read);
                    }
   
                   out.close();
               }
          }
         inputStream.close();
     }

The interesting part comes now. Because it was a bit hard to create a unit test for the original expandToPath method nobody had done it. It would have involved passing a test JAR but more difficult it would have involved passing a target directory where the JAR would be expanded. This is not easy as the location of this target dir would depend from where the tests is executed and making it work seamlessly from both a build tool and from your IDE is not trivial. Here comes VFS to help us. By implementing the FileHandler interface using VFS, we can now write the following unit test:

    public void testExpandToPath() throws Exception
    {
         URI jarURI = new URI("ram:///test.jar");
 
         FileObject testJar = VFS.getManager().resolveFile(jarURI.toString());
         ZipOutputStream zos = new ZipOutputStream(testJar.getContent().getOutputStream());
         ZipEntry zipEntry = new ZipEntry("rootResource.txt");
         zos.putNextEntry(zipEntry);
         zos.write("Some content".getBytes());
         zos.closeEntry();
         zos.close();
 
         DefaultJarArchive jarArchive = new DefaultJarArchive(jarURI);
         jarArchive.setFileHandler(new VFSFileHandler());
 
         jarArchive.expandToPath(new URI("ram:///test"));
 
         // Verify that the rootResource.txt file has been correctly expanded
         FileObject rootResource = VFS.getManager().resolveFile("ram:///test/rootResource.txt");
         assertTrue(rootResource.exists());
     }

Notice the use of the "ram:" URI scheme. This one of the many filesystems supported by VFS and it means that all file operations will happen in a virtual file system in memory. Also note that VFS doesn't currently support creating Zip files so we're using the JDK's ZipOutputStream API. The nice thing is that as this test operates in memory there's no need to define a target location on the file system.

The other nice thing is that by introducing VFS to this expandToPath() method it's now possible to expand a JAR to any file system supported by VFS. We could thus expand to a FTP server, to a WebDAV repository, to an HTTP URL, to a remote machine using SSH, etc. All this without changing a line to our code. Nice isn't it?

IntelliJ IDEA has revolutioned the IDE landscape by adding "intelligence" to IDEs. A few days ago I did a thought experiment by asking myself the following question "how feasible would it be to build a project without knowing any meta-data about it?". In other words, is it possible for a build tool to be intelligent enough to build a project without build files nor POMs. Said differently, is it possible to figure out a project's POM automatically? Let's review some required typical meta-data information and see how they could be guessed.

Source locations

It is possible to guess where sources are by looking for *.java files (for Java projects - The same applies for other project types). Now we still need to differentiate main sources from test sources but that's also relatively easy to do. We can check for classes extending JUnit's TestCase for example or the TestNG equivalent, or any other well-known testing framework.

Note: An interesting thing here is that to be intelligent we'd need the help of the community to add new rules to the discovery process. For example imagine that a new testing framework appears; we'd need to add it to the Test Discovery Rules. Thus, this type of intelligent build system would need to rely a lot on the community and thus would need to get its data from an online repository that could be edited by the community.

Dependencies

How do we detect project dependencies? One relatively way is to parse the sources that we have found above and find all external imports. Then query ibiblio to find matching package names (this information is present in Maven POMs on ibiblio). Now for guessing the version, there's no easy magic. A first approach would be to get the latest released version of the dependencies we've found.

Project type

Project types can easily be guessed by looking at some files. For example if a web.xml file is present then it's a WAR project, if an application.xml one is found then it's an EAR project, if a jnlp file is found then it's a JNLP project, etc.

SCM

SCM can easily be guessed by looking for special files on the filesystem of the project. For example we would look for .cvs directories for SCV and for .svn files for Subversion, etc

Developers

Once we got the SCM URL we can then query the SCM to get the list of all developers.

Project name

The project name could be the name of the top level directory and the version could be set arbitrarily to 1.0. Actually we could even check ibiblio to see if the project is already on ibiblio, get the latest version there and increase the minor number by one as a first order guess. Another strategy would be to query the SCM and look for tags and deduce existing versions by parsing those tags (there are some usual conventions for naming tags so it should be possible to make a good guess).

Modules and artifacts

Discovering the different modules of a project is probably one of the hardest thing to do. If you look at different projects in the wild I believe there are not that many directory structures out there. Maybe 10-15. Thus it should be possible to register knowledge of these structures and let the tool discover which ones matches the closest with the project at hand. This would also allow to deduce the different artifacts that have to be generated. Of course it won't be perfect as there are projects which generate several artifacts and which may be in the same module. Again it's a question of doing 80% of the job and leaving 20% to be done manually.

Additional information

Of course, the information found above are just guesses. In most cases they could be correct but of course we would need to offer a way for the user to edit them and to add any missing information.

Conclusion

I believe it should be possible to create such an intelligent meta-build project which could be used to generate files for one of the existing build system such as Maven, Ant, etc. For example it could create an internal POM file on which Maven could then be executed to produce the build results. At a minimum such a tool could be used to convert existing projects to Maven. I wonder how intelligent it could be but I guess it could go pretty far.

Disclaimer: Of course, such a tool would be bad from a conventions stand point. One of the great strength of Maven has been to standardize the directory structure of projects. I can go to any Maven project and I know exactly where stuff will, what will be generated, etc.

Are there other information which you think could be guessed automatically? Can you think of better algorithms to guess some of the information shown above?

"Better Builds with Maven”"

• is a comprehensive 'How-to' guide for use with Maven 2.0.4 and later
• is available free of charge and includes supporting sample code
• covers how to use Maven 2.0 to better manage the build, test and release cycles associated with software development
• is written for beginning and intermediate Maven users
• is authored by Maven experts
• Jason Van Zyl, Chief Architect and founder of Maven
• Vincent Massol, author of "Maven: a Developers Notebook"
• with chapters and key content and code contributions from leading Apache Software Foundation Maven Project members: Brett Porter, John Casey and Carlos Sanchez.
• is published by Mergere, Inc

Content Includes:

• An introduction to Maven 2.0
• Creating, compiling and packaging your first project
• Best practices and real-world examples
• Creating J2EE builds and using J2EE models
• Extending builds through plugins
• Monitoring source code, testing, dependencies and releases
• Leveraging repositories, Continuum for continuous integration and transitive dependencies
• Converting existing Ant builds to Maven

Download a free copy at http://library.mergere.com.

I hope this book will help boost Maven2's adoption.

There were 2 main goals for this conference:

• Reduce "FUD" spread by influential non-MS technology users by showing them what MS is actually working on. It's harder to spread FUD when you know the details...
• Get feedback from competitive technology users to improve MS products/technologies

I applaud MS for having the foresight (and the money) to do this. It takes some vision and courage to set this up and allow everyone to blog freely about it.

Here are the topics we were shown during these 3 days:

• Scripting and dynamic languages by Jim Hugunin
• .NET CLR 2.0 Reliability by Allesandro Catorcini
• GotDotNet and Communities by Korby Parnell
• Open Source at Microsoft by Bill Hilf
• Microsoft Research by Rick Rashid
• WCF & WS-* by Don Box
• Internet Explorer by Dean Hachamovitch
• LINQ/C# by Luca Bolognese, Anders Hejlsberg
• Executive General Session by Sanjay Parthasarathy
• XBOX Extensibility by Brian Keller
• WPF – next generation UI by Chris Anderson
• Windows Server “Longhorn” as an App Server by Doug Purdy
• ASP.NET/IIS7/Atlas by Scott Guthrie
• InfoCard by Mike Jones
• Windows Mobile/Embedded by Mike Hall and David Karle
• eScience – the Next Decade. Lessons learned and the path forward from TerraServer, SkyServer and bio-informatics by Jim Gray
• Software Factories by Jack Greenfield

What was the outcome? Was it effective?

It was certainly good to be invited (many thanks to Steve for that). The presentations were of mixed quality and I felt that the topics were too broad. There were some that were of interest to me but lots of others were not in my area of expertise/interest. I've also felt that there were not enough participation to meet the original goal defined by MS.

I'd like to believe Microsoft was serious about the feedback we gave them but I'm not sure how much I can believe this... The reason I have some doubt is because the sessions were not meant to gather feedback but rather to explain how things are done at Microsoft. If the goal of this conference is to get feedback then I think the format of the presentations could be much improved. Here are some ideas for next year (in case there's a MTS07):

• Mix formal presentations with round tables sessions. Get 10 persons per table with each table having a Microsoft coach to drive discussions. Have different topics per table and let attendees pick the topics that interest them.
• Use the SPA conference session formats as exemples. This idea is to have more workhop sessions than formal presentations to get everyone's participation and to get deliverables as part of the session's outputs.
• Feedback from everyone was provided at different point in time and to different Microsoft employees. The feedback was received verbally and there's currently no guarantee that the feedback will be remembered and acted upon. I'd suggest to have a large white board that is used throught the conference to record all ideas for improvement. This will also stimulate feedbacks.
• Report on all feedback submitted during the previous conference at the begining on the conference to show what impact the feedback has had.

Note that I have received an email from MS pointing to survey where MS is asking for feedback. I'd still prefer giving the feedback during the conference and directly to the concerned people but with a way of ensuring that this feedback will be tracked (whether it is used or not doesn't matter that much, what's important to me is that it is considered and that I know of the outcome).

All in all a very good week and in addition were had some nice treats: a welcome basket of eatable goodies and ... a one-year MSDN subscription! Now that's very nice and I'm pretty sure last year's participant must be jealous by now ;-). BTW, on this topic of presents, there were some real disapointments when MS announced that we would have a 120$ voucher to buy stuff at the MS company store ... and when we later learned that this voucher was only the right to spend up to 120$ at the store! I'd suggest to remove this next year as most around me (including me) have found this more negative than positive. Of course the announcement of the MSDN subscription the day after helped a lot overcome this negative feeling ;-). We're too spoiled for sure...

Once more, thanks Steve and Microsoft for this very nice week!

"During this Vincent Massol interview you'll receive more information on the status, philosophy and strenghts of Maven 2.0. "What were the shortcomings in Maven 1 and how do we now write maven 2 plugins ?" are just a few questions Dion Almaer asked. Other topics discussed are Continuum, Cargo and Agile outsourcing and offshoring... check it out."

The video is available here.

It appears that Microsoft is serious about Java and Open Source after all. They contacted me for my work on Maven. As a matter of fact, their own MSBuild system is not working as well as they would have liked and they are interested in reusing Maven as a build scaffolding and develop numerous Java and .Net plugins on top of it and integrate it all in the next version of Visual Studio (which will sport lots of new features for Java development too). This is all part of their new 2007 plans for implementing Software Factories. I can't tell you much more at this stage.

The best news is that I won't be working alone on this as they are setting up a full scale Java Team. Yeah that rocks! Some friends from our Open source group in Paris have also been recruited (check Ludovic's , Jeremi's , Didier's and Francois's blogs).

I'll be blogging more about it, stay tuned...

Update (3rd April 2006): Of course this was an April's fool. However there's some truth in it. I'm indeed going to Redmond the week of the 10th of April. Microsoft is inviting 70 people using competing technologies from all over the world. The idea seems to be twofolds: Microsoft will present what they are working on and we are expected to critique/provide feedback on what they're doing compared to what we are doing using our technologies. Sounds pretty fun. It seems this is something similar to the event that Matt Raible attended (more here, here and here). I'll blog more on this when I know more.

• Moderated wikis. Right now there are only two choices for a wiki: either they are open and anyone can edit a page or they are closed wikis and you need to register and get the rights to make modifications. For example most spaces on the Codehaus wiki are closed. They were initially open but vandalism was too high and we had to close them. This is hampering documentation contributions. A moderated wiki would alleviate this: when the page is saved, an email would be sent to a list of moderators for the space for approval of rejection (either by responding to a certain email address as for mailing list moderation or by clicking on a link in the email). Ideally, clicking on the validation link in the email would open the page in a browser with the modifications highlighted so that the moderator could make some changes before clicking on the save button.
• Anonymous edits. Although this feature already exists, I'd like wikis to add 2 fields when anonymously editing a topic: a user name and an email address. The idea is make it even easier to contrinute to a wiki. If the wiki is moderated as explained above, moderators would receive an email. The idea of the username and email is to allow the moderator/community to discuss with the contributor if need be and to give him credits. These 2 fields would obviously be optional and there should be a text on the page explaining that the email will not get displayed on the wiki and that filling the fields will allow credits/acknowledgment to be given.
• Diff notifications. Most wikis allow some form of space watch but the wikis I have used still do not offer the possibility to send notifications in a text diff format (wiki markup diff is good enough). For a development wiki, the idea is to send diff notifications to the development mailing list so that all developers are aware of wiki page modifications.
• Daily notifications. This is also supported in some wikis but what I would like is the ability to watch a single space and to aggregate changes in that space (using the diff notification format mentioned above). Please note that Confluence does not support this as it requires you to modify all other spaces permissions so that the user doing the watch has no view rights on the other spaces, which is not usable for example on wikis such as the one on Codehaus which have hundreds of spaces.
• In place comments. The idea is again to lower contribution by allowing wiki users to highlight a portion of text in their browser and to associate a comment with it (like a post-it). There would be an option to turn on/off these comments. It's easier for a user to highlight a line and put a comment like "I don't understand this sentence" or fix a typo rather than have to use current the type of comments at the bottom of a page. Note that this is similar to how word processors such as Word allow adding comments to a document.
• Patch handling. I'd like the ability to make modifications to a page and then instead of saving, have the ability to click on a "Generate patch" button which would generate a text file in wiki markup diff format. Then there would need to be a "Apply patch" action that can be done on a page. This would allow using wikis for project development web sites and allow contributors to provide documentation patches along with code patches. This is currently a big pain when using a wiki as a project development web site.

I have quite a few other suggestions for improvements but I feel those are the major ones when it comes to using a wiki as a project development wiki. Let's hope wiki vendors are listening... :-). Are these also on your wish list?

I guess it depends on the definiton of "successful". My definition of a successful OSS project is:

1. A project that has a big user community and mindshare
2. A project that has a big development community (i.e. committers and contributors). That is the development mailing list should be quite active with lots of contributions, improvements, ideas being discussed. All those submitted by a large number of different persons
3. A project that lives on if the initial or main contributor leaves the project

If we go by point 1, I think that the main 2 projects I have started (Cactus and Cargo) are not doing too bad. Cargo is still new but it seems to be on track and its user base is growing very quickly.

However on points 2 and 3, I'm not so sure I'm doing well. Mind you, Cargo has quite a lot of committers (growing every week!) and they're all doing a great job. I'm just thinking about the level beyond (look at the activity on the Maven project, Spring or other projects as an example of what I mean). Cactus is a bit different as it's now a mature project, so let's focus on Cargo. Of course it could be that these project domains are narrow and thus do not interest lots of developers. This is probably true but I don't think this is the only issue. I have the feeling that some of the reasons could be:

• Unconsciously I may be "driving" the project too much. For example if someone proposes something, I'm probably going to argue with this person in order that it fits Cargo's quality crtieria and direction. It's possible that by doing this, I'm cutting some creativity from this person and thus even if he implements the thing, maybe he'll feel it has not completely come from him and won't identify with it enough to maintain and improve it in the future. Or maybe by arguing I'm just making the life of that person harder and as this person is doing this in the little free time he has, he may not pursue it...
• Maybe I'm answering emails asked on the list too fast, thus preventing any other contributor to answer. This is setting the tone and maybe as a consequence people are expecting me to answer all emails. Everyone "knows" that I'll answer them anyway...
• The same could be said for applying patches and implementing things. I think this is less true though as I also have a day job and there are lots of JIRA issues accumulating in the Cargo project for example, so there's room for takers
• Documentation of Cargo seems good at first glance (although I know there are lots of holes). I wonder if, as a consequence Cargo users who look at the web site think that Cargo is stable, mature, etc and thus may feel less inclined to participate. Who wants to participate to a project that is mostly "done"! (Note: For those reading this and interested in Cargo, it is far from true and there are tons of things to design and implement!)

On the other hand, I feel that not doing any one of these points will hamper Cargo's user adoption...

Or maybe I'm completely wrong on all points above and this is just me fantasizing!

I'm really interested to know what you think and if this is something other OSS committers/contributors have noticed too. I've given the example of Cargo but really this is a general discussion on how to best manage an OSS project.

• When you're developing a framework, i.e. a piece of software meant to be used at an API level by other developers. In that case, breaking binary compatibility is not something to do lightly.
• When working in a large team it's common to define "interface" projects that represent the contracts to be followed by the different teams. In that case breaking the binary compatibility in an "interface" project is something that has to be planned and organized.

Enforcing binary compatibility in the build

The automated build is a nice place to enforce binary compatibility as the build is something executed by the indiviudal developers before checking-in and it's also executed by the continuous integration build. Thus any binary incompatibility can be quickly discovered. Or course this doesn't replace tests which can also help discover breakages. However the problem is that with all the nice refactoring IDEs we have now, it's easy to refactor the tests at the same time as the code and thus introducing a binary incompatibility is not always noticed.

A good strategy to discover an incompatibility is to compare the current code with the latest released code. This is what Clirr is doing. Clirr currently sports an Ant and Maven1 integration. The good news is that there's a Maven2 plugin in the work (more on that when it's released). However using a tool is only good if there's a strategy behind it.

Strategy for using Clirr

Here is what I believe can be done to automate binary compatibility checks in the build:

• Start by organizing your packages so that you clearly demarcate the user-public API from the SPI from the internal implementations. You'll probably want to fail the build only on the user-public API (and possibly on the SPI too but that's probably a lower severity).
• Use Clirr to make your build fail upon violation on the user-public API.
• After discussing with the team and possibly with users, decide whether you wish to allow the binary incompatibility. Always consider going for a deprecation cycle. If you choose to allow the incompatibility, register it in an exception file that you pass to Clirr so that it builds without choking on those errors (Note: I believe Clirr needs to be improved to better support exceptions not only at the file level but at the violation level).
• When the release time comes, you'll have a nice file listing all the binary incompatibilities. Include it in the release notes so that your users know what to expect and even better, for each incompatibility add a description that explains how to modify the user code to use the new version of the API.

Note: On the Cargo project we've tried to do this, even though there's still room for lots of improvement. Actually our main issue on Cargo is not detecting binary incompatibilites but rather deciding to release a 1.0 version which would mean that from then forward we would aways look for a deprecation solution rather than break binary compatibility. We've always pushed back this 1.0 release because our API has been changing quite frequently but we're now nearing a 1.0 version. When that comes we'll turn Clirr on to fail the build upon breakage. I'll let you know how it goes...

The way I work goes like this: I start writing about a topic. If it's taking too long to explain it, I consider that something is wrong about the code. I modify the source code so that the document I'm writing has the minimal required size to explain the topic.

The good thing with a book is that what you're explaining has to be simple and not convulted which leads to this nice effect of improving usability of your code. I get a bit of the same result when I write project documentation but not to the same level. This is probably simply because writing a book is a more involved process, you dedicate more time to it and thus you want it to be as perfect as possible (and thus as readable as possible).

I guess nothing here is new. This is all about having a user of your code. Tests are "users" of your code and thus leads to better design. I guess documentation can also be a "user" of the code and thus help improving it.

If you're writing some framework/tool, consider writing a book for it and if you're diligent in your writing your code will end up being better! As an added benefit your users will love you... :-)

Note: Adding a new container is very easy and you only have to implement a small interface and your container will be made available automatically through all the existing end user APIs - Make sure to contact us if you're interested in adding a new container support.

I'd like to quickly demonstrate how to use the new Maven 2 plugin on 2 use cases (more samples can be found here):

• Use case 1: Deploying a WAR to Tomcat 5.x and starting the container
• Use case 2: In-place webapp deployment with Jetty 4.x

Deploying a WAR and starting Tomcat 5.x

Create a Maven 2 project and put the following configuration in your pom.xml file:

To generate your WAR, start Tomcat and deploy the WAR in it, simply type: mvn package cargo:start . That's all! Do you want to do the same in, say, Orion 2.0.5? Simply change tomcat5x with orion2x and the home element to point to where you have installed Orion 2.0.5. You don't have Orion on the machine running the build? No issue, simply replace <home> with:

Orion 2.0.5 will then be automatically downloaded and installed the first time you run your build.

Inplace webapp development with Jetty

Let's imagine you're using the same project as above but this time you'd like to start Jetty and make it point to your webapp directory (i.e. src/webapp ) so that whenever you make a change to your webapp's sources Jetty automatically picks it up and serves it. See Brett's nice blog post on this. Simply modify your pom.xml as follows:

Then open a shell prompt and type mvn cargo:start . Jetty will start and monitor your webapp's dir for any change (Note that the same can be achieved using other containers too).

If you're interesting in learning more, check the documentation and join us on the Cargo mailing lists.

I was lucky to be presenting in Room 1 (the big one) and it was well packed. Here are the questions I asked the audience last year:

"The room was packed (I'd say around 400 to 600 people). Before starting with my session I've asked how many people are already using Maven and I've counted about 20 (but at that time the room was only half-packed), so I'd say it was about 3-5%. My second was "How many are planning to use Maven" and I got a resounding 3/4th of the people raising their hand. That shows that Maven is still in it's early adoption phase and that it has some great potential."

This year when I asked the audience who is using Maven 1, I got about 30%-35% hands raised. Wow! This is a huge boost from the 3%-5% of last year! There were also about 5% people already using Maven 2 so that's great too. It seems Maven's adoption is well on its way

Thanks everyone for the warm welcome during the presentation.

Enjoy the slides while waiting for Javapolis to broadcast the videos.

read (photo);
photoContainsHuman = callMechanicalTurk(photo);
if (photoContainsHuman == TRUE) {
   acceptPhoto;
}
else {
   rejectPhoto;
}

This is really like the Matrix except that the humans get paid a little bit of money (but in the end that's close to getting fed) and it's other humans that controlling the programs... until we have web services using other web services using the Mechanical Turk. Then who's controlling who is going to be hard to decide :-)

Source: Google Blogoscoped.

1. load a database schema in the instance
2. load data in the instance
3. start/stop a database instance
4. ability to create an instance from scratch

The ideal situation would be to find an existing Java framework that would already perform all or some of those steps. Then I could easily create a Maven 2 plugin wrapping it. So far I haven't been able to find such a tool. If you know any please suggest them!

Here's what I have found so far below. Please note that I have probably made mistakes while filling this table and I'd be happy to be corrected...

	Load schema	Load data	Start/stop instance	Create instance	Comments
DBunit
DDLUtils					I think DDLUtils is the old commons-sql project.
Derby ij					ij 10.1.1.0 requires the db2jcc.jar which is not on Ibiblio. I need to check the license to see if it could be uploaded.

Again, let me know if you know some tools that are not listed here.

If no such tool exist, an idea I have would be to add support for databases in Cargo. Indeed Cargo is meant for manipulating any kind of containers. It happens that the first type of container we've implemented are J2EE containers but it should work for any other type and the interfaces should remain the same.

WDYT?

This is a sad moment. Our friend and colleague Nicolas Chalumeau has departed this world on the 4th of October at the age of 27.

The Apache Software Foundation would like to express its condolences to Nicolas' family and friends.

Nicolas had been developing open source software at Apache for several years. He was participating in several projects including Jakarta Cactus and Apache Maven. But even more importantly than helping those projects, what made Nicolas stand apart was his kindness and his generosity. He was always open to discussion and was constantly helping others. On the Cactus project he was regularly the first to answer questions from newcomers trying to use Cactus.

Recently he had taken up the important task of helping migrate Cactus to use Cargo and he had been working diligently towards that task. Actually Nicolas had been doing such a great job on the Cactus project that the Cactus committers had unanimously voted him in to become a committer. Nicolas was voted an Apache committer on the 1st of August 2005. I had several discussions with Nicolas about this new role and he was extremely happy to be part of the Apache Foundation.

Nicolas, we'll miss you, but the work you have done for the community will be remembered forever and will live on. All our thoughts and good wishes are with you and your loved ones.

-Vincent on behalf of the Apache community

(we have also created a special farewell page for Nicolas on the Cactus website).

The Google Summer of Code is now over. I've had the pleasure of being a mentor for Codehaus. More specifically I've mentored the following projects:

• JBoss 3.x and 4.x support in Cargo. This project was successfully implemented by Nyoman Winardi (a.k.a. Win). Win started sending patches and over the course of the programme Win became a committer proper. The full support of JBoss 3.x and 4.x will be available in the next release of Cargo (version 0.7).
• JSR-88 support for Cargo. This project was successfully implemented by Lev Olkhovich who also became a Cargo committer. Lev implemented deployment of J2EE archives using JSR-88. In the process he started a conversation to refactor Cargo to support the notion of remote containers. There's still some refactoring going on and we need to add some more tests but we should be able to have support for remote containers and JSR-88 in Cargo 0.7.
• Refactor Cactus to use Cargo. One more project revolving around Cargo! This one was implemented by Xuan Thang Nguyen. Xuan sent several patches and thanks to Felipe we've applied some of those. However, I have to admit I have personally not been available enough to fully help Xuan apply his patches. We still have some patches in JIRA that haven't been applied yet. Actually the plan was to have Nicolas Chalumeau to help Xuan and apply Xuan's patches. Nicolas has been working and helping on Cactus for a long time now and was voted a committer on Cactus at the beginning of the SOC programme. However, the Apache Software Foundation (ASF) is extremely slow when it comes to adding a committer to a project (it can take more than 2 months) and we've not been able to give right access to Nicolas. Thus he's not been able to apply his own patches nor Xuan's... This is really an issue that the ASF has to solve quickly lest it'll see people leaving to create their project somewhere else.
• Faqbot project. There were 2 students on this project: Jie Tang and Harsh Puri. Harsh has had to resign from the programme because of the tragic flooding that happened in the region of Mumbai. Jie has continued alone and has done some good work. Unfortunately he's not been able to fully complete the project (which was probably the most ambitious of all the SOC projects I've mentored). The hardest part was probably starting a project from scratch. Everything had to be done. Hopefully Jie and others will continue the project and make it release-ready. I was very excited by this project and I still am.
• Real-time collaboration editing (Oxyd). This one was implemented by Jeremi Joslin of XWiki fame. Even though it was a project started from scratch Jeremi was able to complete it and have a first usable release ready. Well done Jeremi!

Learnings

• Open source is about collaboration with others. I don't think the SOC emphasis was enough on this point. For example it was "fordbidden" for students to work together and the main focus was to produce a working piece of software.
• Open source is not bound by time. People do it in their free time (at least most people) and as such they can't be expected to be bothered by strong release pressure. The SOC students had to work on a given date which caused some friction as the students were not always aligned with the project's timeline. Let me give you one example; It happened that some student needed to do a refactoring to the existing code to progress. This needs to be reviewed and possibly voted by project committer. There could easily be a delay of 1 week before we get everyone's agreement/ideas, etc. In the meantime the student is under pressure to quickly progress.
• I have taken too many students. I felt I did not do the best possible job when it came to mentoring them. Some were not autonomous enough and would have required more mentoring. I'll take fewer students next time. The hardest is really to mentor students on a new open source project started from scratch. As I'm already involved with several open source project, I did not have enough bandwidth to help on all aspects required to set up a new project.
• Several students had not enough time to participate. Some were still passing exams, others had some other summer job. This, combined with the deadline and the nature of open source did not mix well together. I think students should have an open source project to complete on a much longer timescale (possibly with milestones to monitor progress). This would also help in having them really integrated into an open source team. In addition it'll show their real commitment over time which is really what "professional" open source is about. There's nothing worse that someone who contributes big portion of code and then leaves some time after. Then, all the bug fixing and maintenance falls on the shoulders of the committers who were not the ones with the itch in the first place...

Parting words

The SOC was good. It has boosted the open source community quite remarkably (even though it has also probably put some strain on it...). Out of all the students I've mentored I think 2 or 3 of them (out of 6 initially) will continue to work on the open source project they've participated to. That's a 30%-50% ratio and I'm very happy about it. Thanks Google for making this happen!

I have presented From Maven 1 to Maven 2 which went well. There were about 60-70 people in the room, all Maven 1 users (to be expected for such a talk) and a few (about 5) Maven 2 users.

Update 2005-09-28: Kito has blogged about JavaZone 2005 and has uploaded some nice pictures.

Here are my findings after one day of using CDS. Please note that this is definitely not long enough to have a definitive opinion on the topic but I thought I'd still share what I've learnt today.

General opinion

CDS is a very good desktop search. I was very impressed. It seemed perfect at first and then slowly I started finding some little flaws compared to YDS. Still it is extremely good. It has all the features you'll find in YDS and Google Desktop Search (GDS).

Pros of CDS vs YDS

• Integration with Windows taskbar
• Low resource for indexing. It is not slowing my laptop when indexing. That's very good!
• Immediate scanning of new resources. If you receive an email for example, it is immediatly available for searching. No need to wait for the next indexing.

Cons of CDS vs YDS

• No vertical layout for views (as there is in YDS). This means that you cannot fully the message being previewed
• No "All" categories search. You have to choose the category you wish to search (emails, files, contacts, etc)
• No as-you-type results
• No possibility to choose the columns to display (for exemple email folders or email size). There are only a few basic columns
• Slower to search and display items than YDS. It was very fast initially and it quickly became slow and very slow as indexed items increased
• XML preview is using IE engine on Windows and thus there are lots of XML files that don't display correctly

Some minor details:

• Delete key does not work to suppress an email
• Cannot select different emails (to suppress them for example)

Conclusion

If only it could have a better view layout and be faster to display results it would be perfect. Its killer features are really its CPU-friendly indexing for me and the immediate availability of new resources in searches.

I've just noticed that YDS has released verson 1.2beta yesterday and I'm installing it. For now, I'll still keep using YDS which is still my favorite. YMMV.

It addresses different types of concern: from culture to communications to technical infrastructure to development practices.

Now just before you rush to it, you should know that it's written in French. I'm really sorry about that. I would have much preferred to write it in English but Pivolis' first market is currently France.

I hope you'll enjoy it.

Note: On the Pivolis web site, don't click on the English flag. The English version of the site is not yet up to date and you won't find the White Paper there.

In preparation for the launch of Maven: A Developer's Notebook, Tim and I have created the Mavenbook.org web site. By using it you'll be able to track the progress of the book but more importantly it'll allow you to download the book's source code, get additional samples, read tips and tricks on using Maven, etc. All of this through a blog interface.

BTW, we're not web designers so we've used a skin found on OSWD. I really like this web site which has both free skins and premium ones.

Last, the icing on the cake, the web site has been done using XWiki. Yeah, you heard it right, there is a wiki under the hood, which allow us to easily edit any page online. Each news item is actually a wiki page to which we have associated an Article object. The home page has a simple macro that does a search for all pages with articles and matching some criteria. Cool, no?

Enjoy!

For example, imagine that you have some integration tests in your project's test directory and that you need, say, Tomcat 5.0.30 to run them. You'll need to start the Tomcat container before the test:test goal kicks in. Do this by writing a preGoal in your maven.xml :

You'll need to provide Cargo configuration data in your project.properties or build.properties file. For example a minimal configuration would be:

cargo.containers = tomcat
cargo.container.tomcat.containerKey = tomcat5x
cargo.container.tomcat.homeDir = C:/apps/tomcat-5.0.30
cargo.container.tomcat.config.hint = standalone
cargo.container.tomcat.config.dir = ${maven.build.dir}/tomcat
cargo.container.tomcat.config.standalone.servlet.port = 8180

Alternatively you can ask the plugin to automatically download and install Tomcat for you (it'll download it only once), by specifying:

cargo.containers = tomcat
cargo.container.tomcat.containerKey = tomcat5x

cargo.zipUrlInstaller.tomcatinstaller.installUrl = http://www.apache.org/dist/jakarta/tomcat-5/v5.0.30/bin/jakarta-tomcat-5.0.30.zip
cargo.zipUrlInstaller.tomcatinstaller.installDir = ${maven.build.dir}/installs
cargo.container.tomcat.zipUrlInstaller = tomcatinstaller

cargo.container.tomcat.config.hint = standalone
cargo.container.tomcat.config.dir = ${maven.build.dir}/tomcat
cargo.container.tomcat.config.standalone.servlet.port = 8180

As you may have noticed, in our example above we've reused the existing Maven Test plugin which looks for test sources in ${pom.build.unitTestSourceDirectory} . With this strategy you'll need to create a separate subproject for running your integration/functional tests in order not to interfere with pure unit tests that you may already have in ${pom.build.unitTestSourceDirectory} .

If there's a strong demand, we may consider adding a cargo:test goal in the future that you look for tests in, say, src/test/cargo by default (leaving src/test/java for unit tests).

Please note that there's also an alternative which is to start the Container directly from your unit tests.

If you find Cargo interesting, please come and help us on the Cargo mailing lists. There are lots of different ways you can help: trying Cargo on containers, implementing new containers (for example, JBoss, JOnas, WebSphere, etc), discussing new ideas, letting us know what new features you'd love to see, etc.

The major changes from Cargo 0.4 to 0.5 are:

• New Maven plugin
• Added support for hot deployments
• Added support for the jo! container

Detailed release notes are available on the download page.

Here's an example of how to use Cargo from Java:

Container container = new Resin3xContainer();
container.setHomeDir("c:/apps/resin-3.0.8");

Deployable war = container.getDeployableFactory().createWAR("path/to/simple.war");
container.getConfiguration().addDeployable(war);

container.start();
// Here you are assured the container is started.

container.stop();
// Here you are assured the container is stopped.

Here's an example using the provided Ant tasks:

And here's an example using the Maven plugin:

// To run it:
maven cargo:start

// To configure it, add the following in a Maven properties file:
cargo.containers = myTomcat
cargo.container.myTomcat.containerKey = tomcat5x
cargo.container.myTomcat.homeDir = c:/apps/jakarta-tomcat-5.0.30
cargo.container.myTomcat.config.hint = standalone
cargo.container.myTomcat.config.dir = ${maven.build.dir}/myTomcat/config
cargo.container.myTomcat.config.standalone.servlet.port = 8180

Enjoy!

Here's the kind of output Clirr gives (the full output is available here):

ERROR: 8001: org.codehaus.cargo.deployment.DefaultJarArchive: Class org.codehaus.cargo.deployment.DefaultJarArchive removed
INFO: 8000: org.codehaus.cargo.module.DefaultJarArchive: Class org.codehaus.cargo.module.DefaultJarArchive added
ERROR: 7002: org.codehaus.cargo.container.Container: Method 'public void addDeployable(org.codehaus.cargo.container.deployable.Deployable)' has been removed
INFO: 7011: org.codehaus.cargo.ant.ConfigurationElement: Method 'public void addConfiguredEar(org.codehaus.cargo.ant.EARElement)' has been added
INFO: 4000: org.codehaus.cargo.container.jetty.JettyStandaloneConfiguration: Added org.codehaus.cargo.container.configuration.StandaloneConfiguration to the set of implemented interfaces
ERROR: 7005: org.codehaus.cargo.container.Container: Parameter 1 of 'public void setConfiguration(org.codehaus.cargo.container.Configuration)' has changed its type to org.codehaus.cargo.container.configuration.Configuration
ERROR: 7006: org.codehaus.cargo.ant.ConfigurationElement: Return type of method 'public org.codehaus.cargo.container.Configuration createConfiguration(org.codehaus.cargo.container.Container)' has been changed to org.codehaus.cargo.container.configuration.Configuration
ERROR: 4001: org.codehaus.cargo.container.jetty.JettyStandaloneConfiguration: Removed org.codehaus.cargo.container.Configuration from the set of implemented interfaces
INFO: 7003: org.codehaus.cargo.container.spi.AbstractConfiguration: Method 'public void configure()' has been removed, but an inherited definition exists.
ERROR: 5001: org.codehaus.cargo.container.deployable.EAR: Removed org.codehaus.cargo.util.MonitoredObject from the list of superclasses
INFO: 5000: org.codehaus.cargo.container.deployable.EAR: Added org.codehaus.cargo.util.monitor.MonitoredObject to the list of superclasses
ERROR: 7012: org.codehaus.cargo.container.Container: Method 'public java.io.File getOutput()' has been added to an interface
INFO: 7010: org.codehaus.cargo.container.spi.AbstractContainer: Accessibility of method 'protected java.io.File getOutput()' has been increased from protected to public
INFO: 6000: org.codehaus.cargo.container.property.GeneralPropertySet: Added public field JVMARGS

Even though we're using JIRA with an Iteration-Driven Development strategy (IDD) it was still a very interesting exercise to verify that we had not missed any issue by running Clirr on the source code. In addition, it provides a more detailed view of what exactly has changed in term of API which our JIRA report does not provide.

The next step would be to use it to fail our build whenever someone introduces a public API break. It would be quite easy for us because we've cleanly separated non-public API from public APIs by using internal packages (see the Cactus API design rule to see what it means). Of course sometimes, you want to voluntariy add a breaking change. That's legitimate but it has to be controlled. The strategy would be to have the build fail and then if the change is voluntary to exclude it from Clirr.

Well done Lars!

I think this is really where the value of Ant is. All those years of implementing the base building block for a portable OS Java API have created a very useful Task set. I think every Java application that needs to do copying, deleting a directory, spawning a Java application, etc should use these tasks. There's no point in reinventing the wheel!

For example, you may think that deleting a directory is simple. But it's not so easy. Have a look at the Delete Ant task source code. You'll find portion of code like this one:

/**
 * Accommodate Windows bug encountered in both Sun and IBM JDKs.
 * Others possible. If the delete does not work, call System.gc(),
 * wait a little and try again.
 */
private boolean delete(File f) {
     if (!f.delete()) {
          if (Os.isFamily("windows")) {
               System.gc();
           }
          try {
               Thread.sleep(DELETE_RETRY_SLEEP_MILLIS);
           } catch (InterruptedException ex) {
               // Ignore Exception
           }
          if (!f.delete()) {
               if (deleteOnExit) {
                    int level = quiet ? Project.MSG_VERBOSE : Project.MSG_INFO;
                    log("Failed to delete " + f + ", calling deleteOnExit."
                        + " This attempts to delete the file when the ant jvm"
                        + " has exited and might not succeed."
                        , level);
                    f.deleteOnExit();
                    return true;
                }
               return false;
           }
      }
     return true;
}

Would you have thought about this? Probably not and you would have been right not to as this only happens in some rare occasions. But when one of your users reports it, it's going to be darn difficult to identify and fix. Personally I'd rather depend on a stable and well tested library rather than recode it myself.

The problem is that the Ant tasks are a bit too much linked to the execution engine (the XML scripting engine). For example reusing an Ant tasks requires you to create a Project object. This in turn drags loggers, the Ant classloader (in some cases) and possibly other objects. I know it's possible to use Ant from Java (I've been doing it for a long time now) but I'd love it be even easier to do so.

Instead of writing:

Project project = new Project();
Expander expander = project.createTask("unzip"); 
expander.setSrc(new File(zipfile)); 
expander.setDest(new File(destdir)); 
expander.execute();

I'd like to be able to write:

Expand expand = new Expand();
expand.setSrc(new File(zipfile));
expand.setDest(new File(destdir));
expand.setLogger(myLogger);
expand.execute();

I don't want to see the get/setLocation, get/setTaskName(), get/setDescription() and in general all methods from Task.java.

What I'd love to see is Ant moving in the direction of providing completely reusable Tasks that have 0% dependencies on the Ant engine. This means that loggers, classloaders would be passed to the Ant task by the program who uses it.

I'd like to see Ant provide 2 distributable jars: one containing the XML scripting engine only and one containing all the pure java beans Ant tasks that can be reused in any Java application.

I'd like to see Ant separate into 2 subprojects: one for the XML scripting engine (let's call it engine) and one for the Ant tasks (let's call it tasks). The reason for the 2 projects is to ensure there's no dependency in the direction tasks->engine.

I'd like to see Maven2 use those completely reusable Ant tasks instead of recreating them (this is a wish I'm addressing to both projects, not just Ant! :-)).

I'd like to see those Ant tasks being a JSR and incorporated in a future version of the JDK, thus providing a higher level API that the best classes from the JDK.

Is that where Ant is heading today?

Indeed, the barrier to participation is often due to the fact that a user who wants to participate will need to understand the whole design, how the different classes are entangled, what effect a change here will have on the rest of the project, etc. Thus, if we make the project more modular a contributor who wants to participate will only need to understand the design of a given 'module'.

A 'module' would need to have some good-to-have characteristics:

• Very loose coupling with other modules
• Clearly defined and *published* interfaces. There should be some documentation on the project's web site explaining them and a tutorial showing how to implement new modules (or swapping a module implementation by another one) for example.
• Separate builds so that it's easy to build only the module (this can be alleviated if the master build is easy to use (i.e. no property tweaking necessary, it just builds - As it's the case with good Maven builds... ;-))
• Separate documentation on the web site, so that the website itself is modular and the complexity of each module is hidden in that module's web site. Thus the top level web site would be quite simple only listing what the project does as a whole and listing the different modules

Interestingly one way to implement the 'very loose coupling with other modules' characteristic is by using a Service Architecture. This can be done for example by using the Dependency Injection pattern and/or using a lightweight container - PicoContainer, Spring, etc).

This is probably obvious stuff but I've just realized that it's not only good design practices but that it'll also help open source projects attract more contributions. Of course that leads to another topic which is when to accept contributions and how to maintain them in the long run but that would be another discussion...

So what is the need for a distributed build?

I can see several use cases:

• building on several JDKs
• building on different OS platforms
• building with different environment setups (for example building with different application servers, different browsers, different databases, etc) to validate that a product integrates well with various environment setups
• delegating the build load on several machines when the build starts to take too long (of course, the first solution should be to try to lighten your build as much as possible)

A proposed architecture

Disclaimer: this only ONE potential solution. There are lots of other solution probably even more valid than this one. Please feel free to add your ideas as comments to this post.

It could work as follows:

1. The central build machine (aka the build orchestrator) decides to start a build. The orchestrator can be one existing continuous integration tool like CruiseControl, DamageControl, etc. They can trigger a build on anything they want: time-based, change-based, manual, continuously, etc. The orchestrator sends a build request to the space. The request contains all the information about the requested build (e.g. JDK to run on, OS to run on, App.Sever/DB/etc to run on)
2. The space holds all requests. It chould be a good idea to provide a browser to see pending requests (preferably using a simple HTTP browser so that people who wish to contribute can see what type of builds are required). In any case it's important that the space be transactional (Note: I'm not sure about the word "transactional". What I mean is that a request cannot be read by several build agents at the same time)
3. Build agents listen on space build requests objects that match their capabilities. Using Jini/Javaspace would be nice here because (among other things) agents would be able to easily listen to requests with Jini attributes (OS, JDK, etc). Once they read a request they start a local build and publish the result to the space as a Result object
4. The build orchestrator listens to Results object, and generate result reports, aggregating all results. Build results could contain anything required: result of the build, logs, generated artifacts, etc. The orchestrator gets the data from the Result object and perform usual build operations (publishing, build result notification).

Of course there would be several details to sort out, like should we send 2 Requests object for each build need so that we can compare the results and only accept the result if they match, etc.

Conclusion

I think this type of distributed build could be especially interesting for open source projects in order to build an active community around a project. This would be yet another way in which people can contribute to an open source project: by lending some of their machine CPU to perform continuous integration builds of this project. This usually makes sense as open source projects may be low on hardware resources and lending some would help. Of course it also bring its challenge of security issues that would also need to be implemented...

Would you like such a distributed build system? I personally prefer this architecture over one where the orchestrator directly sends build requests to build agents as I find it more scalable and more flexible.

I've also written 2 Maven quizzes for JavaBlackBelt.com. If you want to see if your Maven knowledge is up to the par, go and take them! And let me know your comments. If you register on the site you can even propose new questions!

For IntelliJ IDEA

• Start by downloading the latest javasvn.jar from the TMate JavaSVN web site. You need version 0.8.0 or later (I've used 0.8.0). The reason is that there is a new property named javasvn.ssh2.key that has been added as a hack for getting the Subclipse plugin for Eclipse to work with svn+ssh... Drop the jar in [IDEAHOME]/plugins/svn4idea/lib , replacing the existing jar of the same name there.
• Modify the [IDEAHOME]/bin/idea.bat file to add the javasvn.ssh2.key system property: IF "%IDEA_JVM_ARGS%" == "" set IDEA_JVM_ARGS= [...] -Djavasvn.ssh2.key=/path/to/your/private/key .
• Make sure you use an openSSH-compatible private key. The Putty format is NOT supported by JSch. If you have a Putty private key, use Puttygen to export it as an OpenSSH key.
• Make you sure you specify a valid username in the IDEA subversion setting, leaving the password field blank.

For Eclipse

• Install Subclipse using the provided Eclipse update site
• Install the JavaSVN Subclipse Extension, also using the provided Eclipse update site
• Modify the way you start Eclipse as mentioned on the JavaSVN Subclipse Extension web page
• Go the SVN Repository Exploring perspective, right-click on your SVN Repository, click on "properties" and make sure you enter a valid username (leave the password blank).

Enjoy! That should please all the Codehaus hausmates... :-)

Enjoy! (I hope...;-))

The typical local builds that developers run on their machines work by building the subproject they're working on but also all the dependent subprojects it requires. Usually, as building all dependent subprojects takes a lot of time, the developer infrequently checks-out other project sources and build them on demand. His focus is on his subproject that he's making modifications to (and rightly so!). This strategy has the following drawbacks:

• Setting up the build on a new fresh machine is complex and takes time. Indeed you have to check out all the top level project sources and build all projecets one by one until you reach the subproject you're concerned with.
• It doesn't scale too well. Your local build starts taking tens of minutes which does not encourage running it that often. And if you do, you don't rebuild all the subprojects even though there are probably lots of changes that have been made by other coworkers. Thus, you're increasing the possibility of an integration break (breaking your other coworkers when they integrate your changes).
• When someone from another team inadverently breaks your project's build, you'll have to switch context (i.e. stop what you're doing) and help out to restore the master build. If this happens unfrequently, it's probably fine and even positive (as it increases team collaboration ;-)). However when it happens frequently (which is bound to happen as the team grows), you'll start suffering from it...

Because of all these problems, I have been using a different approach on my current project for the past 2 years. This was mostly motivated by the fact that the project is a big project (close to 100 developers) and we were hitting the issues mentioned above. I have called this strategy "Binary Dependencies Build". If you're interested this is an approach I have presented both at TSSS2004 and at Javapolis 2004.

Here is how it works (click on image for a larger picture):

Imagine that you have a "trading" subproject that depends on 2 other subprojects ("partners" and "referenceData"). The idea is that your local build will NOT build them from sources but instead will download their latest version that work from a remote artifact repository (a location where the result of the subproject build is located). In order to accelerate even further the build, the versions downloaded are stored locally. In our example, the latest "partners" jar is already available locally and is thus not downloaded but the "referenceData" one is not. It is downloaded and then stored locally. The "trading" subproject is built using these binary dependencies.

This is all fine but there is a burning question: How do I do continuous integration with such a system? Won't the binary dependencies be old versions when I get them? The solution to this is to have a continuous build server that continuously build subprojects and puts their artifacts in the remote repository. Note that there are put in the repository only if their build passes with no errors. This ensure that there are always fresh versions available and that they are as "good" as they can get.

Doing it with Maven

The good news is that this feature is built in Maven. Maven implements this support of artifact repositories (local and remote) and it supports the process of automatic download of artifacts not available in the local repository. Usually Maven will verify first in the local repository if the artifact's version exists and if so will use it. However, if an artifact's version contains the "SNAPSHOT" keyword, Maven will always check if there's is a more uptodate artifact in the remote repository. This allows implementing easily the strategy defined above.

Conclusion

We've been very happy with this solution so far. I think there are 2 key points in making this work:

• A good build that provides assurance that the binary artifacts are working. Indeed we've experienced that our subproject build was not always good enough to qualify how "good" was a jar artifact. This was usually caused by the non-existence of automated functional tests which meant that even though the build was passing the jar was not working when executed on the developer's machine. The solution is of course to include integration/functional tests in the build (at least the master CI build).
• A quick master build. It's important that it generates fresh jar artifacts as quickly as possible so that CI can happen as often as possible.

1. Build takes too long to execute (and thus it is executed less often)
• Fix the build by having more subprojects with binary dependencies and/or streamline the build to ensure that only important build steps are run. Optimize it (f.e. offer different goals/targets: one for a clean build and another one that does not perform a clean).
2. Local build not executed
• Build coaches to increase build awareness. Complementary idea: "unbreakable builds".
3. Public API breakage in dependent project without warning
• Better team communication. Set up a deprecation policy. Complementary idea: "unbreakable builds".
4. Not enough continuous commits (all packed up at end of iteration)
• Team meetings to explain more the importance of continuous integration. Complementary idea: "unbreakable builds". The idea is that if you keep your changes to yourself and accumulate them, whenever you'll want to commit them, the unbreakable build will likely reject your changes as they will break some other part of the code. Thus you'll need to spend several days to talk to other developers to not only fix your code but also fix theirs. Normally after doing this several times, you should understand that it is in your best interest to commit frequently.
5. No functional/integration automated tests (f.e. no local verification of ejb-jar deployments)
• Automated functional tests! Build a suite slowly over time, improving it at each iteration. And maintain it! Decide on a good data handling strategy (this is usually the main issue). Ensure that your data strategy keeps everyone in sync WRT DB data.
6. Commit problems (Forget to commit some files, Pb due to SCM tool - Starteam: new directory do not appear in Starteam view!)
• Coaching + change the SCM tool! Complementary idea: "unbreakable builds".
7. Devs “building” with IDE but forgetting to use the automated build
• Coaching. Complementary idea: "unbreakable builds".
8. Checkstyle errors failing the build
• Coaching. More team meeting to decide what checkstyle errors we want to fail the build or not. Get a strong team buy-in. Complementary idea: "unbreakable builds".
9. Failing unit tests
• It probably means that the unit tests are actually integration tests depending on database data. Ensure that unit tests are quick and fast and independent of the environment. Complementary idea: "unbreakable builds".
10. [Maven] project.xml not up to date and missing dependency
• SCM diff emails on check-ins (team by team) in order for everyone to have the knowledge of what's happening. Complementary idea: "unbreakable builds".
11. Database data modifications (vol