Alex Rivera | Logout

How do commercial Java static analysis tools compare with the free ones?

Asked 2008-10-16T07:10:02.620
33

I'm familiar with a handful of the free static analysis tools available for Java, such as FindBugs and PMD. What I'd like to know is how the commercial products such as Klocwork and Coverity stack up against these. What are their strengths and weaknesses?

Edit
Report

4 Answers

18

We use a suite of open source and commercial static analysis tools. The different tools find different kinds of bugs and some are tuned for lower false positive rates, at the expense of possibly missing some real problems.

In my experience, Findbugs does a good job of finding real problems, especially if you focus on Correctness errors as their team suggests. Recently the developers of Findbugs have added some basic security vulnerability checks as well. Coverity has a low false positive rate especially if you don't turn on their experimental checkers, and Coverity Prevent includes a good tracking database for trend/cluster analysis. I am not convinced yet that their threading checkers (static or dynamic) work - at least they haven't found anything interesting for us. Klocwork Developer for Java returns higher false positives, but we find they have the strongest security checking of these tools. So it depends on whether your priority is quality checking (Findbugs, Coverity) or security vulnerability analysis (Klocwork, or Fortify). Some of our developers also use PMD to support source code reviews, as it helps with general code cleanup.

A recent project conducted with NIST called "SATE: Static Analysis Tool Exposition" reviewed a wide variety of different tools and their underlying approaches. https://samate.nist.gov/index.php/SATE.html and other references to this project such as at OWASP. The general finding is that different tools have different strengths and weaknesses, so use more than one if you want to do a thorough job.

answered 2008-10-31T22:06:36.630
6

The one feature you will most certainly find in a commercial static analysis tool (and that you will not find easily in a freeware analysis tool, at least in 2008, at the time of the OP) is
Reporting: Measures software quality trends over time

As explained in this question about code metrics, any static code analysis in itself in not always meaningful, because you could have:

  • too many "defects" to fix
  • too many categories of defect reported

You need the ability to do some triage, and you need to check if a particular defect is occurring less and less over time or not, in order to help you prioritize what to fix.

This is especially true on legacy project with thousands of classes: you do not fix defect on many files just like that, without having a good reason. That reason can be deduced from a good reporting and trend analysis you will not find with freeware tools.


Update: from 2012 (4 years later), Sonar (Now in 2018 named "SonarQube") "Historical Information" (aka "Time Machine") in its 4.x and 5.x series.
Note those project dashboards were dropped in SonarQube 6.1 (Sept. 2016): see this thread.
Those dashboard would need to be re-created manually through a cust

answered 2008-10-16T07:36:33.050
6

I'll suggest you to try SONAR an open source software quality management tool, dedicated to continuously analyze and measure source code quality. This soft take the result from code analysis tool, consolidate that results and give you access to an user friendly interface.

answered 2008-10-16T08:04:35.117
1

here's a list of commercial analysis tools : http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis#Java_2

coverity has several tools :
http://www.coverity.com/html/coverity-readiness-manager-java.html : this should be on par with findbugs and PMD but with better presentation

prevent : http://www.coverity.com/html/prevent-for-java.html : low FALSE POSITIVES.

thread analyzer : http://www.coverity.com/html/coverity-thread-analyzer-java.html : this is what is absent in most open source tools.

answered 2008-10-16T07:18:56.660

Your Answer