Common source parsing storage format wanted
[ vmassol ] 11:32, Thursday, 8 April 2004

As part of my day work, I've met CAST Software in France. They are really nice guys. And they have a very good product suite that analyzes application source code and produces different kinds of reports (quality and reverse-engineering/browsing).

The really nice part is their open architecture:

It's nice for the following reasons:

  • There can be as many parsers as there are languages, script languages, etc. All data goes to the same database and this data is available to all analyzers. This means that it's possible to define analyzers that act at the level of the full application. For example, you can define some architecture pattern validation. This is the same idea I had some time ago with the PatternTesting project, using AOP to write these rules (I was relying on the Javac parser to perform the source code parsing for me).
  • It allows to write browsing tools that show the full flow of the application, from configuration files to database tables

Compare this to existing open source tools (or not open source for some) like Checkstyle, Clirr, PMD, Findbugs, jcoverage, Simian, Clover, etc. They all use their own parsers and more important their own persistence format for storing the parsed data. Even though they almost all offer an XML report format, it's already processed data (as opposed to raw parsing data). Thus it is very hard to reuse the information from these tools to offer a higher level integrated view. It's also a lot of work for these tools to develop all the necessary parsers.

What would be really nice is the creation of an open source project that does the following:

  • Define a common storage model (database schema, XML schema, other). It should allow to store historical data too.
  • Define a common API to store parsing data
  • Define a common API to query parsing data

If such open source project existed, we could have parser projects which could parse Java code, .Net code, Hibernate configuration + code, Struts configs + code, AspectJ Aspects, etc. These parsers would user the common API to store parsed data in the common storage format, thus making the data available to other projects, such as the one mentionned above (Checkstyle, PMD, etc). These projects woud be analyzers and would use the common API to query parsed data and generate reports for their domain of work.

With this, we could build nice analyzers that could completely check for full architecture best practices, across technologies and across frameworks.

Do you think such a project would work?

Note: Maybe CAST should open source this part (storage model) and keep as closed source their added value Parsers and Analyzers? :-)


Comments

Have you looked at ATerm?

http://www.program-transformation.org/Tools/ATermFormat

--Carlos E. Perez, April 8, 2004 01:02 PM

Or maybe this may be more flexible:

http://www.sable.mcgill.ca/jil/

--Carlos E. Perez, April 8, 2004 01:05 PM

Hi Carlos,

Thanks for the links. I guess the only issue is finding some solution that can be heavily queried and remains performant (that's why a database is nice I think). I don't know if this can be achieved with XML and XQuery for example.

Thanks!

--Vincent Massol, April 8, 2004 01:08 PM

Oh, is that what your looking for?

Then take a look at this list http://www.manageability.org/blog/stuff/knowledge-management-open-source-java/view

In particular, OpenShore seems to be used in the context similar to yours.

--Carlos E. Perez, April 8, 2004 02:40 PM

Hey, thanks for mentioning Clirr together with all those other well respected tools :-) I hope that I'll soon find the time to work on it again...

Right now I'm a bit sceptical that what you suggest is feasible (perfomance is only one of the reasons). If CAST opens their storage model, I'll definetely have a look, though.

--Lars Kühne, April 17, 2004 12:11 PM

what about MOF?
it's OMG standard I believe. The netbeans guys want to build the refactoring and code analysis on the MOF "metadata repository".. I'm not expert, but maybe google has more details.

--mkleint, April 20, 2004 08:27 PM

Check out XRadar: http://sourceforge.net/projects/xradar

--Casco, June 8, 2004 11:49 AM

At JavaZone 2004, John Crupi (Sun) made a talk and they have something like that in the works called SALSA:

http://www.javazone.no/web/show.do?page=92&articleid=2021

--jerome, October 13, 2004 10:13 PM
Post a comment









Remember personal info?