And I'll start blogging again. Promise.

This is because these systems only offer an ugly form based interface for releases. No web services, no scp. Just these bloody forms. -And most people interact with them manually, uploading files, pasting in release notes and so forth. There are probably some odd, unknown, project-specific scripts lying around in random places that can automate this, but I haven't seen anything reusable.

In fact, I hadn't seen anything at all, so 4 months ago I wrote my own script to do this. Except I couldn't get the HTTP multipart posts (attachments) to work properly. So I left it fermenting for a while...

But two days ago I felt the pain again, and I came across DHH's Rake script which does something very similar. -And it had the multipart code I couldn't grok myself.

So I decided to take the best parts of my old attempt and DHH's code from Rails' Rake script and make a proper library, so that others can use it. -Not only on RubyForge, but hopefully also on SourceForge and other sites running SourceForge clones.

So if you're as lazy as me, check out XForge.

XForge is released using XForge (of course!), so it works fine on RubyForge. Please let me know if you get it to work anywhere else, or if you have bugs or improvements.

In HTML programming, common template techniques are JSP, PHP, ERB and ASP. By passing some extra data to a template engine, a template can be turned into an HTML page that displays the data, using the layout from the template.

In OO programming, the most common concept of a template is a Class. A piece of a program can supply some data to a class' constructor and get a new object.

Sometimes, instantiating a new object by simply calling a constructor with some extra data is not the optimal way to create an object though. Consider the common situation where the desired object is composed of an aggregation of several other objects. -A complex graph of objects.

One way to approach this is to design classes (aka templates) to honour Dependency Injection (DI), and use a DI container supporting a configuration file (in XML or other format). NanoContainer or Spring are two frameworks that use this approach to instantiate complex graphs of objects composed of "variable data" from a configuration file.

Assuming you want the ability to create object graphs that are configured in an external file, and your language is Java, then it doesn't get much simpler than with NanoContainer or Spring.

But in Ruby...

A similar effect that is radically simpler and extremely easy to use can be achieved with a dash of Ruby magic using YAML and eval under the covers. Consider this object template:

codehaus_project_template = Project.new
codehaus_project_template.home_page = "http://\#{unix_name}.codehaus.org"

jabber_publisher = JabberPublisher.new
jabber_publisher.id_resource = "damagecontrol@jabber.codehaus.org/#{unix_name}"

codehaus_project_template.publishers << jabber_publisher

project = codehaus_project_template.dupe("unix_name" => "mooky")      
assert_equal("http://mooky.codehaus.org", project.home_page);
assert_equal("damagecontrol@jabber.codehaus.org/mooky", project.publishers[0].id_resource);

module ObjectTemplate
  def dupe(variables)
    template_yaml = YAML::dump(self)
    b = binding
    variables.each { |key, value| eval "#{key} = variables[\"#{key}\"]", b }
    new_yaml = eval(template_yaml.dump.gsub(/\\#/, "#"), b)
    YAML::load(new_yaml)
  end
end

In an Ajax web app the server sends back XML, and the XML is transformed to HTML in the browser. There are several ways to do this transformation:

One approach is to use XSLTProcessor as described here.

However, XSLT is problematic for a couple of reasons. First, not all browsers support it, and more importantly, most web ui designers don't speak XSL.

A second approach is to use XPath in JavaScript to pull out bits of pieces of the XML and modify small pieces of the DOM to fill them in.

A third (and new?) approach could be to use the familiar template approach (as in JSP, ASP, ERB, PHP templates) and generate bigger chunks of DHTML. This would make it much easier to develop nice looking views, since they can now be designed in a regular web authoring tool such as DreamWeaver or NVU.

But wait - the template engine now has to sit inside the browser! And this is where I discovered JavaScript Templates.

JST templates could theoretically be rendered (and tested) in a build system, using Rhino. Being able to test view rendering is something I consider very important, especially when systems start to get complex (which they do with Ajax).

I'm eager to try this out.

Because I like simple things I also like to keep things in one place (I don't want to maintain this metadata in a separate RHTML template).

Ruby doesn't support annotations out of the box, so I opened up the Class class and added support for annotations. Now I can do:

require 'rscm/annotations'

  class EmailSender
    ann :description => "IP address of the mail server", :tip => "Use 'localhost' if you have a good box, sister!"
    attr_accessor :server
  end

The class' annotations can then be accessed like this:

  EmailSender.server[:description] # => "IP address of the mail server"

It took the Java community forever to do stuff like this. First a couple of years of XDoclet, then JSR175. And tons of code too. In Ruby it's 1 hour's work and 25 lines of code. Go figure.

For more info see http://rscm.rubyforge.org/classes/Class.html

And oh, here's the code:

class Class
  def ann(anns)
    @@anns ||= {}
    def self.method_missing(sym, *args) #:nodoc:
      @@anns[sym]
    end
    $attr_anns ||= {}
    $attr_anns.merge!(anns)
  end

  alias old_attr_reader attr_reader #:nodoc:
  def attr_reader(*syms) #:nodoc:
    syms.each do |sym|
      @@anns[sym] = $attr_anns
    end
    $attr_anns = nil
    old_attr_reader(*syms)
  end

  def attr_accessor(*syms) #:nodoc:
    attr_reader(*syms)
    attr_writer(*syms)
  end
end

Java has been my language of choice for many years. Learning Ruby has opened up my eyes about what OO, agile development and extreme productivity should be like.

I'm not "converting" from Ruby to Java - I always try to use the best tool for the job at hand. Sometimes it's Java, sometimes it's .NET, sometimes it's Excel or PHP. More often than anything else it's not even about programming. But when it is, Ruby turns out to be the best choice for me more often than any other language. Probably because I'm an XP head. (And because I still don't know Objective-C or Smalltalk - it's on this year's TODO list).

Ruby - because I can get the job done fast and end up with super readable and maintainable code. And I'm worth it.

Now if those big corps that I tend to consult for would dare to do more things in Ruby... I reckon it will take a few years before we see them changing.

I must say I think that's exactly what StarTeam deserves. Who knows, maybe they'll even pull a Darwin Award one day?

Excel 2003 VBA Programmer's Reference

Professional Xdoclet 2.0

It is true that Ara Abrahamian, Erik Hatcher and I were thinking about writing a book about XDoclet a couple of years ago, but it never happened. But claiming that I have written a book about Excel VBA is just plain hilarious. (The Amazon page for this VBA book doesn't have this incorrect information).

Hmm wait a minute, can I claim royalties for this?

If you hate notes like me (no, i said HATE), feel free to let it all out right here, right now. In your own language if it makes you feel better. Like this.

"How about a tool that deletes all code that isn't tested?".

I am still not sure whether Chris was entirely serious or not, but it was an intriguing idea that I have been thinking about quite often since then.

Now, the idea has been materialised in a new little tool in the Extreme XP Tools family: Guantanamo!

Guantanamo is eating its own dog food, and as a result, it has 100% coverage. I still haven't tried to run it over any other codebases, so it remains to see whether it is useful or not. -And whether it is something anyone would actually want to use.

I'll give it a try at work :-)

After a longish session pairing with Paul we finally released PicoContainer 1.0 today.

This has been a great journey so far. Paul and I started it right after I moved to London to start working for ThoughtWorks. It all started when I asked Paul one night whether he had heard of Inversion of Control. He got this weird expression on his face when I asked, and I couldn't quite figure out whether he was excited or depressed.

"It's my favourite pattern dood." (Dood pronounced in his native British)

So he got all excited and told me everything about IoC (I didn't have that much of a clue), and about Avalon - a project Paul had been with for a while, but was drifting away from.

My initial reaction (a couple of hours later when Paul didn't have any more breath) was:

"Why does it have to be that complicated?"

We quickly decided to do a spike on a simple piece of code that could do IoC style dependency management via constructors rather than via setters and ugly container lookups from within the components. It's kind of blurry to me how we decided to go for constructors, as we were both drunk when we started TDDing the first lines. I think it was some kind of eureka moment where Paul was thinking about a comment made by Rachel Davies while she was reviewing our colleague Joe Walnes et al's Java Open Source Programming, and I was just being new to this and thought about simplicity. The way I remeber it we unisonely burst out:

"Let's do it with constructors!" (Me left, Paul right)

This was also my first real journey into TDD land. I had known about TDD for a couple of years, but hadn't yet had that epiphany where it fundamentally changed the way I think about code - and religiously follow the practice. I think the first test looked something like this:

public class ContainerTestCase extends junit.framework.TestCase {
     public static class Fred {
          private final Wilma wilma;
  
          public Fred(Wilma wilma) {
               this.wilma = wilma;
           }
  
          public Wilma getWilma() {
               return wilma;
           }
      }
 
     public static class Wilma {}
     
     public void testShouldInstantiateClassesAndPassDependencies() {
          Container container = new Container();
          container.registerComponent(Wilma.class);
          container.registerComponent(Fred.class);
          Fred fred = (Fred) container.getComponent(Fred.class);
          assertNotNull(fred.getWilma());
      }
}

This was our first requirement - a container that can instantiate classes and understand what arguments to pass to the constructors. We alternated the keyboard rapidly and made the test pass. This involved some simple reflection logic and was implemented in a couple of classes.

The following days we added more features. Occasionally I would happily try to implement code without writing the test (requirement) first. It always resulted in a slap from Paul and a

"You're doing up-front design again. How do you know whether that is needed or even if it's going to work?"

So I gradually got the hang of TDD, thanks to Paul. (A couple of months later I was greatful for that. I was ruthelessly refactoring the entire codebase to improve the design. That would have been absolutely impossible without the extensive test suite we had built up by then.)

This was back in the days when Codehaus was a small place only hosting a handful of projects. About a week after we started Bob was kind enough to lend us a space for our new little baby, and that's when we named it PicoContainer.

PicoContainer quickly got Jon Tirsén's attention (who I didn't know back then - he has later become a good friend and also a colleague). Jon quickly joined the project and has since been an important contributor. Much of the nice modular design we have today is owed to him. Several other people (Joe Walnes, Chris Stevenson and Mike Hogan to name a few) also helped us in the early days, and Rickard Öberg blogged about it. PicoContainer was now a well-established project. This was late June 2003.

Half a year later my colleague Martin Fowler wrote an article about the concepts behind IoC and PicoContainer, and this is how the term "Dependency Injection" was introduced. (Martin initially coined it "Service Injection", but Rod Johnson refined the term during a Geek Night in the ThoughtWorks offices in London).

Lately we have invited some carefully selected contributors, and we will hopefully invite more in the future.

PicoContainer now has pretty good documentation, we have fixed some 180 feature requests and bugs. It's simple to use, and the jar is no more than 50kb with no external dependencies. There is a .NET and a Ruby port, and more in the pipe. NanoContainer is also an interesting sister project that adds scripted configuration using a multitude of script languages, as well as integration with WebWork, Hibernate and much more. (It's not that well documented yet, but we'll get there).

The best part of this story isn't PicoContainer itself. It is how many people have learned about dependency injection as a way to decouple their code (both internally and from the container itself) and make it testable and mockable. At the end of the day that will bring us code that is easier to maintain. It's your code that counts, and you should be able to choose what container you want to deploy it in - if you need a container at all!

The nice thing about EJB3 is that the spec committee is trying to address the limitations of the EJB2 component model. The EJB3 component model is based on the following concepts and goals:

1) POJOs/POJIs (plain old java objects/interfaces - nothing to extend)
2) Dependency Injection (setter flavour)
3) Testability outside the container
4) Annotations
5) CMP becomes Hibernate

Linda mentioned 24 times during her presentation that Gavin and Hibernate had been an important source of inspiration for the EJB3 persistence model (it's true, Marcf was counting and nearly creamed his pants). That's great though. Gavin and Hibernate rock.

Those of you who are a little bit uptodate in Java OSS land may have recognised that the 3 first points have been popularised by three lightweight containers, namely Spring, Hivemind and PicoContainer.

Linda was talking about as much about POJO/DI/Testability as she was about persistence, so when she didn't mention once where the rest of the spec (the non-persistence related stuff) was inspired from, it struck me as rather strange.

Here are my speculations about why:

a) The EJB3 spec committee have invented these concepts independently and have not been inspired by the current lightweight containers.
b) Other projects only get mentioned if one of its contributors are on the spec committee.
c) The EJB3 spec committee is trying to hide the fact that they were inspired from other projects and sell it as their own invention.

Cedric utters his enthusiasm about the new spec. And expresses how sick he is of lightweight containers.

Well done folks!

DamageControl supports this of course, but it also supports some interesting historic reports. You can see an example here:

DamageControl stats for PicoContainer

This kind of information will tell you wether your build process is heading in the right direction (build times going up or down). Seeing this will make it harder to ignore problems related to build time. -And hopefully encourage the development team to do something about it before it becomes too big a hassle.

Other similar historic reports that I'm planning to write are
o JIRA activity
o Download activity
o Mailing list activity

Especially a historical JIRA activity graph can be used to determine the responsiveness of a project:

o Bug lifetime distribution
o Open bug counts

Combine all these graphs in a single page, and you get a pretty good picture of the health and activity of a project, both historically and currently.

Unfortunately, (s)he doesnt't mention *what* of the ideas are not new. (There is just "the idea"). The only thing that the authors of PicoContainer might have claimed as newness is to have a *framework* that supports constructor based dependency injection.

As far as I can tell, there is no prior art to that. And for the IoC stuff: We never claimed it to be our invention (there are heaps of pre-pico IoC frameworks of course); We just came up with an framework that makes DI (which is just a part of the broader pattern/principle/technique referred to as IoC) simpler than previous DI oriented frameworks/techniques.

Please be more specific about what isn't new (because there is more to PicoContainer than just C-DI) and blog it where people can leave comments. Oh well, leave'em here then.

A mock is an object that acts as a "dummy" placeholder for a "real" object. Its class is generally an implementation of an interface or a subclass of some class, either generated dynamically or statically coded.

For a detailed explanation of what Mocks are and what they are intended to do, see MockObjects and JMock.

Right, the problem with the codebase developed by this team I'm referring to is that many of the "mocks" replace the wrong classes. The result of this is overly complicated tests, in addition to a false perception of what is being tested. Many of the "mocks" in this codebase are not really proper mocks, apart from having the word "Mock" in their name.

So where and how are these misunderstood "mocks" used? They are being used as substitutes for the classes that are supposed to be tested(!) In short, the unit tests are testing the "mocks".

The system consists of WebWork Actions that talk to Hibernate via DAOs (Data Access Objects). -A fairly common combination of technologies and patterns.

Most actions are following this kind of style:

public class CheeseAction extends ActionSupport {
     private String cheese;
     public void setCheese(String cheese) { this.cheese = cheese; }
     public String getCheese() { return cheese; }
 
     public String execute() {
          try {
               // CheeseDao is a concrete class.
               saveCheese(new CheeseDao());
               return SUCCESS;
           } catch (Exception e) {
               return ERROR;
           }
      }
 
     protected void saveCheese(CheeseDao dao) {
          dao.saveCheese(cheese);
      }
}

The CheeseDao class is where the Hibernate stuff happens. Calling the CheeseDao.saveCheese method will hit the database.

Hitting the database when testing an Action is very time consuming. The team therefore decided to mock out the hibernate DAOs in the tests.

The tests that test the actions follow this pattern:

public class CheeseActionTest extends TestCase {
 
     public void testAddCheese() {
          // Yikes, we're testing the "mock"
          MockCheeseAction action = new MockCheeseAction();
          action.setCheese("stilton");
          assertEquals(ActionSupport.SUCCESS, action.execute());
          assertEquals("stilton", action.getSavedCheese());
      }
 
     public class MockCheeseAction extends CheeseAction {
          private String savedCheese;
  
          protected void saveCheese(CheeseDao dao) {
               CheeseDao mockDao = new CheeseDao() {
                    public void saveCheese(String cheese) {
                         savedCheese = cheese;
                     }
                };
               super.saveCheese(mockDao);
           }
  
          public String getSavedCheese() {
               return savedCheese;
           }
      }
}

(The development team's primary motivation for using mocks seems to have been to reduce the test execution time. While this is a noble goal, it is not the only good reason to use mocks. Another good reason to use them - and TDD in general - is to let good designs emerge more easily. Codebases developed with TDD and Mocks generally become a lot more decoupled than others).

It must have been when someone was thinking about how to have the actions use a different DAO (a mock DAO) that this mocking antipattern was invented.

(If my assumption about how this "mocking" technique was conceived is correct, it is also likely that the tests were written after the actions themselves. At least at the time of the invention of the antipattern. If the test had been written before the action, the likelyhood of having the action so tightly coupled with the Hibernate DAO would have been reduced).

There is something fundamentally wrong about the "mocking" approach used in the test above:

1) The test is not testing the *real* action, but a subclass that only exists in the test codebase. The team's established practice is to extend the action and override methods that use objects that we want to mock out. Then, a mock object is created for the DAO (this is actually more like a proper mock), and the superclass' method is called with that object instead.

In this trivial example it may not look so bad. After all, the overridden saveCheese method in the "mock" calls the superclass' method, right?

But in the real codebase the tests and overridden actions are often so complicated (with many overridings) that this easily gets out of control. Specifically, if the superclass is refactored, there is no guarantee that the test is still valid. The derived "mock" implementation within the test may not be overriding anything in the superclass anymore, and the test may fail to execute the methods originally intended.

2) The Action class is not tested through its public interface, which is how it will be accessed when assembled in the application.

3) The test becomes overly complicated because both the class under test AND its dependencies have to be "mocked".

4) The expectations that are supposed to verify that the action invokes the dao's methods are clunky, as they require additional fields and getters on the subclass "mock".

Mocks are substitutes for real classes. It is quite common to misunderstand what classes they are supposed to be substituted with mocks. Mocks are supposed to replace the objects that the class under test depends on. You should never substitute the class you intend to test with a Mock. If you do, you're not testing the real thing.

Here is what the code should have looked like: (Also note the new name of the test method. I'll cover that in a new blog entry about "Unit Test Intents").

public class CheeseAction extends ActionSupport {
     private final CheeseDao dao;
     private String cheese;
     public void setCheese(String cheese) { this.cheese = cheese; }
     public String getCheese() { return cheese; }
     
     // Yes, you can do this. See http://wiki.opensymphony.com/space/PicoContainer+Integration
     public CheeseAction(CheeseDao dao) {
          this.dao = dao;
      }
 
     public String execute() {
          try {
               dao.saveCheese(cheese);
               return SUCCESS;
           } catch(Exception e) {
               return ERROR;
           }
      }
}

And the test (which should ideally be written before the action):

public class CheeseActionTest extends TestCase {
 
     public void testExecuteCallsSaveCheeseOnDaoAndSucceedsWhenNoDaoException() {
          Mock mockDao = new Mock(CheeseDao.class);
          mockDao.expect("saveCheese", C.eq("stilton"));
  
          CheeseAction action = new CheeseAction((CheeseDao)mockDao.proxy());
          action.setCheese("stilton");
  
          assertEquals(ActionSupport.SUCCESS, action.execute());
          mockDao.verify();
      }
}

There are several benefits doing Mocking like this (the proper way):

1) The tests get a lot shorter.

2) It's more readable.

3) We're testing the right thing.

4) TDD buys us nice decoupled designs for free. This action is now a PicoComponent, honouring constructor based dependency injection.

However, in order to be able to do things properly like this, the dependencies to be mocked out (in this case, the CheeseDao) should ideally be interfaces. This can incur some percieved overhead in the codebase, as the developer will now have to maintain both a CheeseDao and a HibernateCheeseDao. But it's better than having to maintain a CheeseAction and a MockCheeseAction.

Recent additions to MockObjects (and soon JMock?) allow dynamic mocking of concrete classes too, using CGLib.

And last, but not least. If you write your tests first, using mocks, you will end up with well-designed, nicely decoupled code. Post-testing and post-mocking is actually quite evil. It tends to open up and cripple the classes even more than they were before.

"You'll spend more time writing tests than code."

While in some cases this is true, or close to true, it is not a bad thing. Here is why:

1) You end up with less (functional) code than a with a non-TDD approach. This is because with TDD you don't write code you "think you might need". You write exactly what makes the test pass.

2) You know when you're done. Knowing when you're done saves time.

3) You gain confidence for refactoring. Being able to refactor is essential in order to obtain a maintainable and well designed codebase. It is impossible (or very time consuming at best) to refactor if you don't have tests. This is because you can't know whether your refactorings will break something without tests. Having a well designed codebase saves time.

So while there might be some percieved overhead in doing TDD, the design and maintainability of your codebase will improve drastically. You will catch bugs sooner rather than later. This will in the long run lead to SHORTER development time.

Don't get me wrong - I don't do TDD because I want to save time. I just wanted to argue that TDD does not slow you down.

"You'll spend ages writing hugely complicated test cases."

The TDD practioneers stress that unit tests shall be simple. The test fixture should be simple and the assertions few. If your tests are simple, so will your code will be.

Systems developed without TDD are always more complicated than the TDD ones. If your test is too complicated, you're not doing it right.

"You'll spend ages bug fixing the test harnesses that sometimes are more complicated than the code they're testing".

This is like saying that cooking is crazy because there are bad chefs. There are good and bad ways to do TDD. The good one is to keep the tests simple. So do that.

TDD is not easy. It is very hard to do properly until you pair with someone who knows how to do it. -Even for seasoned developers (who often have a lot of pride to swollow and a lot to unlearn).

Many people think they are doing TDD while they are not. There is more to it than writing the tests first. -Like always doing the simplest thing and continuosly refactor the code. Mocking is also an essential part of the whole TDD concept.

Piotr JFDI'ed the long-awaited thing. An easy to use and powerful attributes package for the Java platofrm.

It draws the best from Ara's XRAI, Mark's Attrib4J, Jon's Nanning and Jakarta's Commons-Attributes. All of which failed to deliver a usable product, but (tribute) all contributed to Piotr's new brainchild via their ideas and code.

And if you don't care about attributes - just read the code and doco. It's nice. Very nice. The package scope is even com.thoughtworks even if Piotr is not a Thoughtworker.

This begs for an interview :-)

IntelliBrains: Can you write an IdeaMock for us? So we can mock IDEA in IDEA? Or is there a way to launch IDEA from IDEA?

Puleeese!

:-)

Flamers: It's only 5 weeks old, so don't pick too many holes in it. Use our JIRA instead.

My old blogs were at freeroller and blog-city. Rest in peace.

I've written a Maven plugin for it. Have a look at a sample report.

With a tool like this, you quickly see candidates for refactoring.