Jonas, the article shows a complete (perhaps purposeful?) misunderstanding of how clustering works, and how it should work. Teracotta's approach will simply re-invent the same fundamental mistakes that JBoss AOP Cache and Gemstone/J already discovered.

Our Coherence product is very popular because it is proven (sites with zero downtime now for over three years), it scales (to hundreds of machines, and not just on paper), it is very natural for a developer to use (simple, non-ambiguous APIs) -- and it promotes the use of good patterns and approaches to domain modeling.

Peace.

Hi Cameron.

What do you mean with "fundamental mistakes"?

My intention was not to pick on Coherence, which is a great product. Could you please explain what Coherence does better than Terracotta's DSO (and JBossCache and Gemstone) and what they are doing wrong...

Thanks.

Hi Cameron,

I worked with every single version of GemStone/J starting with 0.81 up to 4.1 and I can't see the "fundimential mistake" that you're talking about.

Care to explain yourself or is this marketing FUD.

Hi Cameron. Thanks for the response.

Coherence provides a solution that obviously is a good fit for a lot of users. But as is always the case, there are tradeoffs to be made, and I think there is room to debate "how [clustering] should work."

In my article, I'm simply trying to highlight the tradeoff that has to be made around object identity. I'd really like to better understand the fundamental mistakes you see on the other side of that tradeoff.

I'm also very curious about your statement that the distributed hashmap model promotes good domain modeling. My experience has been exactly the opposite: because it does not preserve object identity, the hashmap tends to force one to use relational rather than object domain types.

Not to pick nits, but the title is "Time to Throw Out Your Distributed HashMap?" and the post reads "DSO might just as well be the death of the the distributed hashmap, e.g. JCache". Since the JCache API is directly based on our Coherence product, and Coherence has the vast majority of the market, I could not help but to notice this post ;-). I didn't complain about FUD though, and I am happy to respect another's opinion even when I disagree with it.

Regarding the "fundamental mistake," as I would explain it, it is actually a weakness of the JVM design which makes implementing a fully transparent OODBMS in Java impossible. Regardless of what it is being called, jboss and Terracotta are both attempting to implement Java OODBMS's.

(Coincidentally, this topic came up last week at the EAS summit, and I've been blogging on a different conceptual approach to the VM that I feel could address this. See http://www.jroller.com/page/cpurdy .. but note that I haven't gotten very far in the discussion).

I'm also quite aware of Jonas' work, and that of his current employer. Regarding technology for transparently weaving functionality into Java software, it is something that we (Tangosol) invested heavily in, and did some pioneering work in. We were, for example, able to dynamically weave mixins and aspects into clustered WebLogic applications back in early 2001, using our own commercial AOP IDE and runtime weaving engine. (Yes, I said 2001. That was a year and a half ahead of the infamous Xerox patent, for example.)

So it is an area of technology that I have great interest in, for a variety of reasons, and one that I have opinions on, but most of these opinions are formed by experience: Our Coherence software powers production applications in many of the world's most successful organizations, including some of the busiest websites and most of the leading financial institutions, so when someone says that something else "Seems to be the only sane way of doing clustering in Java," you can be sure that I have solid reasons (e.g. very happy customers) to disagree.

Peace.

You can't just assert that DSO and Gemstone/J are attempting to implement Java OODBMS's and make some vague statements about there being a fundamental mistake in the JVM. One could argue, as I have in the past, that all distributed caches are just disguised OODBMS's with poor querying but that really doesn't get us any further either.

What DSO does isn't that different from Coherence I imagine though with a different development and deployment model. Basically what Jonas is saying is that it is easier to deploy a distributed caching solution that you don't have to develop for up front. Just like its easier to deploy logging into a system with AOP than it is to change all the classes that need logging.

My main point is that it is unfair to walk in with all your customers (we all know you have been around the longest) and claim that they prove you do things the best way and all other ways won't work. If that were the case you would have never had the chance to replace my in-memory session replication that we did in 1998 in the WebLogic server.

(disclaimer: I know and like all the parties involved in this discussion.)

Jonas, I didn't realize that you had gone to work for Terracotta, so I should have been offering congratulations instead of arguing in your blog comments ;-)

Sam, you were (are?) an advisor for Terracotta and were (are?) being paid by one of their investors, so I'd hope that you would be supporting their efforts. I think if you were to stop and re-read the post from my point of view, you'd understand my natural reluctance to agree that "DSO might just as well be the death of the the distributed hashmap, e.g. JCache etc. Seems to be the only sane way of doing clustering in Java." Obviously, now that I know that Jonas works for Terracotta, what he said doesn't bother me at all.

Also, I just wanted to point out that we have not been around the longest, by a long shot. There were at least three significant vendors in the space before we arrived; maybe we were just the luckiest. It's also not nice to put words in my mouth, such as "all other ways won't work". What I said (and I do believe) is that there are flaws in Java that prevent transparent OODBMS implementations. If you examine jboss AOP cache (I assume someone did that for due dilligence?), you'll see that the conceptual problem that Terracotta is trying to solve is the same conceptual problem that jboss AOP cache is trying to solve. (Am I wrong? I can only tell you from what I have read.)

At any rate, I'm glad to have the discussion on the technical merits. I think the various technologies in discussion are going to be interesting and relevant for years to come. Eventually we'll solve these problems, one way or another.

Peace.

Cameron:

As you know, I ran the JRockit JVM group at BEA for the last 4 years. I would be interested to hear precisely what you mean when you refer to the "flaws in Java." Would you care to elaborate? Also, what do you think the conceptual problem that Terracotta is trying to solve is exactly? I am mystified by your comparison of Terracotta to JBoss AOP cache - I'd like to hear your technical explanation of that comparison.

Bob, from a brief reading of the linked article, one would conclude that the Terracotta approach is to automatically replicate updates to objects by observing those updates as they occur (e.g. by byte-code modifying the classes of the objects to observe). That's why I referenced the jboss AOP cache, since it has the same conceptual approach. Feel free to correct my misunderstanding, and please don't assume that I'm suggesting more similarities than a shared conceptual approach. Peace.

"Bob, from a brief reading of the linked article, one would conclude that the Terracotta approach is to automatically replicate updates to objects by observing those updates as they occur (e.g. by byte-code modifying the classes of the objects to observe)"

I have also being thinking about this because surely there is much more overhead (network latency) in sending the updates, even with the synchronized block support that I have learned about on the TSS posting. In a J(*)EE application which I would assume is the target platform for the solution the synchronized block is far too fine grain and that ideally it is the transaction scope that is much more important.

Does DSO provide a JCA implementation that would allow for more coarse grain and natural scoping of updates?

Does the solution require all instances of a particular class be clustered as I can imagine that there are instances of a type that are clustered or not? Is the solution contextual aware? I hope we can agree that the Map interface makes this much more explicit and understandable rather than trying to formulate some complex pointcut expression after the fact to exclude data instances that are not relevant within the cluster.

I have lots of questions about inheritance and object identity that I will defer until I have read the documentation.

By the way do you have any extension points in the communication system for distributed tracing solutions to integrate with. I could see customers being very interested in seeing the updates across a cluster in the scope of a transaction. JXInsight 4.0 Preview release will be shipping with distributed tracing for JMS (tracking of message across hops) and WorkManager (BEA/IBM commonj spec) next week. I hope that we can apply the same approach to your solution. I am looking forward to when Cameron releases his own WorkManager implementation.

http://www.jinspired.com/products/jdbinsight/corbatracing.html

Regards,

William Louth
JXInsight
"J2EE tuning, testing and tracing with JXInsight"

William,

All excellent questions. Let me take a stab at a few answers here:

1. Synchronize vs. transaction-control. Terracotta is api-less so there is no JCA support. Did you mean JCA as in a way to bundle up the messages and send them over the DSO-plumbing as if it were JMS or JTA and a way to define transaction boundaries in the code? Anyways, the important thing here is that DSO offers a coarser grained mechanism than Synchronize wherein a developer, at configuration time, can specify methods that demarcate the transaction. We talk up synchronize as it is an auto-locking tool whereas method-locks are for more advanced users.

2. UNCLUSTERED INSTANCES OF CLASSES. DSO has the notion of "roots." Steve Harris defines them quite well in various places including the FAQ on our site. Briefly: a root is the top level object you wish to share such that all object references that join that root (visualize a tree of objects) will automatically be shared as well. As a result of this design, one would tend to share the data model, the collection acting as a cache in the single VM, the singleton, the global, anything stateful that the app might need to share when clustered or when failing over. Sticking to the cannonical session example, you wouldn't ask DSO to "cluster sessions" in the config as much as you would "cluster the session manager" (which hold references to all the sessions). This mechanism is pretty standard to all the clustering tools.

3. OBJECT IDENTITY. Please fire away. You might want to reach out to Steve Harris via the DSO forum at http://www.terracottatech.com/forums instead of here. (I would give you his e-mail address but it would just cause spam. His e-mail address is just his first name AT our domain.

4. Customers are already VERY interested in being able to see AND STEP THROUGH transactions across the cluster. Would love to talk more!

Thanks for the questions.

--Ari

William:

Good to hear from you. It's been a while. I'm quite sure that there are a number of areas we can cooperate with you. Drop me a line (bob AT our domain name) and we'll schedule a time to talk.

Bob Griswold