Automation for the people: Continuous Inspection

Free yourself from mundane, manual inspections with software inspectors

Paul Duvall , CTO, Stelligent Incorporated

Paul Duvall is the CTO of Stelligent Incorporated , which helps companies address software quality with effective developer testing strategies and Continuous Integration techniques that enable teams to monitor and improve code quality early and often. He is a contributing author to the UML™ 2 Toolkit and currently co-authoring Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley).

Summary:   When starting new projects, most of us plan to review code before actually releasing it into production; however, when delivery schedules supersede other factors, reviews tend to be the first practice thrown out. What if you were able to perform a portion of these reviews automatically? In this first article of the new series Automation for the people , development automation expert Paul Duvall begins with a look at how automated inspectors like CheckStyle, JavaNCSS, and CPD enhance the development process and when you should use them.

 

http://www.ibm.com/developerworks/java/library/j-ap08016/index.html?S_TACT=105AGX02&S_CMP=EDU

 

Code inspections can take different forms. Some organizations use formal peer reviews, in which a developer's peers critique the code and offer ideas for improvements; other organizations exercise pairing, where one person types and the other is considering high-level design decisions and suggesting code improvements. Furthermore, some teams have another developer review their code in the form of a "desk check" before committing it to the version control repository.

Regardless of how an organization chooses to perform a code inspection, one thing is for sure: they're implicitly manual processes . And as I've found over the years, manual processes are error prone and tend to be forgotten when the going gets rough. Software inspections, however, don't always have to be manual; in fact, an entire genre of open source and commercial tools, which I dub software inspectors , are available to facilitate statically analyzing code (also known as static analysis tools).

With software inspectors, code inspections become automated through build tools like Ant or Maven. And it's through this automation that low-level source code details like coding standards, complexity, and duplication (to name a few) become the responsibility of a machine. This responsibility shift has the tendency to enhance in-person manual inspections by focusing on more high-level aspects of development, such as design and long term maintenance issues.

About this series

As developers, we work to automate processes for users; yet, many of us overlook opportunities to automate our own development processes. To that end, Automation for the people is a series of articles dedicated to exploring the practical uses of automating software development processes and teaching you when and how to apply automation successfully.

Software inspectors to the rescue

There are a host of software inspectors ranging from sophisticated commercial tools to simple open source alternatives; however, I've found three, in particular, quite useful: CheckStyle, CPD, and JavaNCSS, (see Resources ):

• CheckStyle reports deviations from a project's predefined coding standards.
• CPD reports code duplication.
• JavaNCSS can help teams focus on areas of higher code complexity.

All these tools are freely available and easy to incorporate into a build platform like Ant or Maven and, consequently, they are extremely easy to integrate into Continuous Integration (CI) systems.

What is Continuous Integration?

Continuous Integration (CI) is a practice that enables teams to receive feedback and make improvements on a continual basis rather than waiting until later in the development cycle to find and fix defects. Tools (like CruiseControl) run in the background polling a version control repository looking for changes. When a change does occur, the tool executes a predefined build script through Ant, for example. Continuous Inspection is facilitated by the practice of Continuous Integration.

Inspector style standards

At several companies, I've participated in committees where we spent countless hours defining coding standards. In one case, we defined a 25-page coding standards document that almost no one followed. Consequently, our code reviews were painful as teams spent time quibbling over whether a developer was using the two-spaces rule (as opposed to the four-spaces rule) and whether or not a parameter should be declared as final . We could have saved valuable time simply by offloading low-level inspections to a software inspector.

Using an automated source code inspection tool such as CheckStyle provides an easy way to execute a configurable set of rules, which can be based on a project's coding standards. Moreover, you can run CheckStyle inside an IDE, such as Eclipse through a CheckStyle plug-in, and CheckStyle straightforwardly integrates into build scripts through its Ant task or Maven plug-in. Any rule deviations CheckStyle discovers are displayed in report form that clearly delineates which files were in violation. What's more, you can configure Ant and Maven in concert with CheckStyle such that builds can fail if any rules are violated.

For example, Listing 1 demonstrates a snippet from an Ant build script that runs CheckStyle to generate an HTML report. Note the failOnViolation property, which in this case is set to false . If this attribute is set to true , the build fails upon the discovery of any source code violations.

Listing 1. Using CheckStyle with Ant

<taskdef resource="checkstyletask.properties" classpath="${checkstyle.jar}"/> <checkstyle config ="${basedir}/cs-rules.xml" failOnViolation ="false"> <formatter toFile ="${checkstyle.data.file}" type="xml" /> <fileset casesensitive="yes" dir ="${src.dir}" includes="**/*.java" /> </checkstyle> <xslt taskname="checkstyle" in="${checkstyle.data.file}" out="${checkstyle.report.file}" style="${checkstyle.xsl.file}" />

 

In Listing 1, the config attribute, which is set to cs-rules.xml , identifies the rules file to run recursively against the source directory (as in the fileset dir attribute). The xslt task accepts the file generated from the formatter toFile attribute. This task transforms the XML generated in Listing 1 into a readable HTML file using the XSL file checkstyle.xsl, which is included with the CheckStyle installation.

Listing 2 is an example of a portion of a CheckStyle rules file. Out of the box, CheckStyle can run over 120 rules on a code base.

Listing 2. CheckStyle Rules defined in the cs-rules.xml file

<module name="Checker"> <module name="TreeWalker"> <property name="cacheFile" value="target/checkstyle.cache"/> <property name="tabWidth" value="4"/> <module name="ImportOrder"> <property name="ordered" value="true"/> <property name="separated" value="true"/> </module> <module name="LineLength "> <property name="max" value="120 "/> </module> <module name="FileLength"> <property name="max" value="400"/> </module> <module name="UnusedImports"/> </module> </module>

 

Each CheckStyle rule is part of a module. For instance, the LineLength module establishes the rule that all lines must be 120 characters or fewer or CheckStyle produces an error.

You can extend CheckStyle with custom rules, too, and coding standards are easily enforced as well. What's more, it's automated!

Figure 1 shows an example of a CheckStyle report generated with Ant using the xslt task demonstrated in Listing 1 :

Figure 1. CheckStyle HTML report

What about my IDE?

A common response I hear is that these inspections can be run in IDEs. While this is true and highly recommended , the benefit of having a separate machine running a common set of rules (using a tool like Ant) against source code is simply better consistency.

Duplication Inspection with CPD

One of the basic goals of object-oriented programming is to promote reuse by creating reusable objects that can adapt to different contexts. Yet, all too often, copy-and-paste programming is used and masked as "reuse." Sometimes this happens because a developer is not aware of a reusable component and other times, it's the result of providing a "quick fix." Consequently, if you need to make a modification, you must apply it to all of the copies and its variations (which, of course, without tools becomes quite difficult). This manual approach results in maintenance headaches and the strong possibility of missing and even introducing defects.

CPD, which is part of the popular open source static analysis tool PMD, reports the number of duplicate lines in a code base. What's more, CPD's token threshold is configurable, meaning that you can vary how CPD suggests duplicate lines of code. For example, by setting the threshold at 100 tokens, CPD shows only instances where at least 100 tokens have been duplicated. Keep in mind, however, that source code always contains some duplication; CPD simply provides teams with a means to investigate areas of considerable code duplication where corrective actions, such as targeted refactoring, can be taken.

Listing 3 demonstrates CPD in action via Ant. One of the first options required by the CPD task is the minimumTokenCount attribute, which identifies the number of tokens to compare (the minimum value is one token). Additionally, you can set CPD to ignore certain code options, such as identifiers and literals. In this example, the Ant fileset type delineates which files CPD should analyze and which it should ignore.

Listing 3. Using CPD with Ant

<taskdef name="cpd" classname="net.sourceforge.pmd.cpd.CPDTask" classpathref="pmd.classpath" /> <cpd minimumTokenCount="100" format="xml" language="java" ignoreIdentifiers="true" ignoreLiterals="true" outputFile="${basedir}/cpd.xml" > <fileset dir="${src.dir}"> <include name="**/*.java" /> <include name="**/*.jsp" /> <include name="**/*.xml" /> <exclude name="**/*Test*.java" /> </fileset> </cpd>

 

As shown in Figure 2, the cpd.xml is output using the outputFile attribute in Listing 3. In the near future, I'd like to add an XSL to CPD to generate an HTML report.

Figure 2. CPD XML output displaying code duplication violations

Finding duplicate code manually can be quite challenging in all but the most obvious copy-and-paste violations. Using a tool like CPD, however, quickly reveals duplicated code, which you can then improve.

What are paths?

A source code path is also known as a control flow statement, which indicates code that changes the flow in a method. Examples include conditionals, such as an if statement, or looping constructs like while and for .

Inspecting complexity with JavaNCSS

Many studies have confirmed that as the number of paths within a method increases, so does the difficulty of understanding and maintaining the same method. Consequently, the more difficult code is to understand and maintain, the more likely it is that defects can be easily introduced.

JavaNCSS is a free tool that provides various code measurements such as the number of non-commenting source statements (or lines of code) and the cyclomatic complexity number of all analyzed methods. Listing 4 demonstrates the use of JavaNCSS, which, as you can see, isn't very challenging. Simply point JavaNCSS to a directory containing source code and let it rip!

Listing 4. Using JavaNCSS with Ant

<path id="javancss.classpath"> <fileset dir="${lib.dir}" /> </path> <taskdef name="javancss" classpathref="javancss.classpath" classname="javancss.JavancssAntTask"> </taskdef> <javancss srcdir="${src.dir}" generateReport="true" outputfile="${javancss.report.dir}/javancss_metrics.xml" format="xml"/>

 

As with most software inspectors, an XML file is generated that you can easily transform into an HTML report through XSLT, as I've done in Figure 3:

Figure 3. JavaNCSS HTML report

Understanding an application's code complexity is by no means a panacea. In fact, in some cases, complexity values may be misleading. For example, some tools give switch statements high values. I've found, however, that using a tool like JavaNCSS helps channel efforts in reducing areas of higher complexity, which ultimately improves code understandability and maintainability.

---

Back to top

An inspection at every change keeps defects in range

Every time a team member commits modifications to a version control repository, the code has changed. But how did it change? Was the modified code the victim of a copy-and-paste job? Did the complexity increase? The only way to know is to run a software inspector at every check-in . Moreover, receiving feedback on each of the risks discussed thus far on a continuous basis is one sure-fire way to keep a code base's health in check automatically !

Cruise'n with CI

CruiseControl is one of the most widely used open source CI tools in the Java™ community. The tool is configured to run in the background and polls a version control repository, like CVS for example. When source code modifications are detected (i.e., someone has checked in code), CruiseControl executes a checkout of the source and runs a predefined build script.

As a result, teams can run software inspectors such as CheckStyle, CPD, and JavaNCSS whenever a change is made to the version control repository. This ability allows teams to monitor and perform inspections over a period of time, and as with most CI tools, teams can generate feedback in the form of reports, e-mail, or even by using devices like the Ambient Orb (see Figure 5 ).

Listing 5 demonstrates the use of the CruiseControl configuration file (typically named config.xml) to display the results of the CheckStyle reports on the CruiseControl dashboard. The other software inspectors use a similar syntax to incorporate results into CruiseControl's dashboard.

Listing 5. Logging CheckStyle with CruiseControl in config.xml

<listeners> <currentbuildstatuslistener file="logs/${project.name}/status.txt"/> </listeners> <modificationset quietperiod="30"> <svn RepositoryLocation="http://www.qualitylabs.org/svn/ambientorb" username="[username]" password="[password]" /> </modificationset> <schedule interval="60"> <ant anthome="apache-ant-1.6.5" buildfile="build-${project.name}.xml"/> </schedule> <log> <merge dir="merge dir="checkout/${project.name}/_reports/checkstyle" /> </log>

 

Figure 4 shows an example report generated using CruiseControl and CheckStyle. Note that you can configure CruiseControl to publish the reports of other tools like CPD and JavaNCSS, and by running these reports with every source code change , your team can actively improve code in real-time instead of in latter cycles.

Figure 4. CheckStyle report integrated with CruiseControl

Ambient feedback

One of the coolest things about continuously running software inspectors in a CI tool is that teams have myriad mechanisms for notifications at their disposal. Sometimes the build may not fail, but something has changed in such a way that corrective action may need to be taken sooner rather than later. For example, it's actually quite easy to configure a device, like the Ambient Orb, to change colors if code complexity increases or even if a certain number of coding standards are violated.

Listing 6 uses the Ambient Orb Ant task and a Ruby script to change the color and animation of an Orb if 20 or more classes exceed 300 Source Lines Of Code (SLOC). In this case, I chose to change the color of the orb to magenta and the animation to crescendo if the classes meet this condition.

Listing 6. Delegating Ant build file for CruiseControl and Ambient Orb

<target name="checkSloc" > <exec dir="${basedir}" executable="cmd.exe"> <arg line="/c ${config.dir}/javancss/SlocThreshold.rb ${reports.javancss.dir}/javancss_metrics.xml 20 ${javancss.file}"/> </exec> <available file="${basedir}/${javancss.file}" property="sloc.exceeded"/> <antcall target="notifyOrb" /> </target> <target name="notifyOrb" if="sloc.exceeded"> <taskdef classname="org.qualitylabs.ambientorb.ant.OrbTask" name="orb" classpathref="orb.class.path"/> <orb query="http://myambient.com:8080/java/my_devices/submitdata.jsp" deviceId="AAA-9A9-AA9" colorPass="green" colorFail="magenta" animationPass="none" animationFail="crescendo" commentFail="SLOC+Exceeded" /> </target>

 

Figure 5 demonstrates what the orb looks like when there are too many large classes in the code base:

Figure 5. The Ambient Orb

With this Ambient Orb notification judo, I don't receive endless e-mails; I just glance at the SLOC Threshold Orb to quickly understand how many large classes I have, which alerts me to redirect my refactoring efforts. Of course, it goes without saying that combining ambient orbs with software inspectors (and a little creativity) creates endless possibilities!

---

Back to top

Automate when it's appropriate

Keep in mind that there are quite a few different software inspectors to choose from. While the three profiled here happen to be some of my favorites, I've also been impressed with tools like JDepend, PMD, and FindBugs (the latter two inspectors have been discussed in detail in developerWorks already). Once a Continuous Inspection process is in place, plugging in new inspectors is a matter of minutes.

Continuous Inspection will not and should not eliminate manual software inspections; however, by running a suite of automated inspection utilities with every change to a version control repository, those manual inspections you do perform are much more productive and efficient. Moreover, Continuous Inspection has the added benefit of making it more likely that risks can be reduced at every step along the way, rather than waiting until the very end of a project.

Next month, I'll take a deeper look at some of the more interesting Continuous Integration tools available, such as CruiseControl, Luntbuild and Continuum and let you determine which one is best for you. Automation has never been this fun!

 

---

Back to top

Download

Description	Name	Size	Download method
Software Inspection examples in Ant	j-ap08016.zip	2MB	HTTP

Information about download methods

 

Resources

Learn

• "Realizing Continuous Integration " (Kevin Lee, developerWorks, September 2005): Learn the concept and practices of Continuous Integration.
• "Zap bugs with PMD " (Elliotte Rusty Harold, developerWorks, January 2005): Get started with PMD, a static code analysis tool similar to CheckStyle.
• "Monitoring cyclomatic complexity " (Andrew Glover, developerWorks, March 2006): This article, part of the In pursuit of code quality series, addresses how to monitor code complexity.
• Ambient Orb Ant task : Learn more about this handy task on the author's blog entry at testearly.com.
• The Java technology zone : Hundreds of articles about every aspect of Java programming.

Get products and technologies

• JavaNCSS : An excellent tool for spotting code complexity.
• CheckStyle : Avoid those costly, low-level manual inspections with CheckStyle.
• CruiseControl : Continuous Inspection starts with a Continuous Integration tool like CruiseControl.
• CPD : A freely available code duplication tool from the PMD team.
• Ambient Orb Ant task : Reduce your e-mail bloat!