Over the coming weeks/months I’ll be posting a series of product reviews here. Starting that process is SSW Code Auditor. As many of you know, I have an ongoing relationship with SSW however I’m not getting paid off for this series of posts. Those who know me better would also know that even if I was getting paid … that wouldn’t stop me from publicly bashing something I don’t like.
For anyone who’s hasn’t come across SSW in their time, you might understand why this tool exists if you take a look at their almost limitless number of standards. They range from the infamous Rules to Better Email to the SQL Server Stored Procedure Naming Standard. With more writing time being spent on these standards than the average magazine publishing house it quickly becomes hard to difficult to enforce and track their implementation; thus Code Auditor, SQL Auditor and Lookout were born.
The Code Auditor sales pitch touts the product as “[using] the power of Regular Expressions to audit your code”. I’d like to rephrase that to something like this:
Code Auditor is a regex runner on steroids combining a regex editor, tester, pre-built rules databases, VS.NET integration and lots of other wacky features like being able to process MDB files.
A pre-built rules database is provided with the product that follows along the lines of SSW’s published standards, however this can be easily customized if you’d like to name your exception variable something other than ex or want to enforce standards that your organization has already developed.
So far it sounds like the dream product … and I’m sure the product homepage could do an even better job of selling it… but I still only see Code Auditor as part of a wider toolset. With me spending 99% of my time in the .NET space, I’ve always been a heavy promoter of FxCop. With the release of VS2005 we now have this functionality in the OOBE which is even better. SSW addresses this, however I’d like to rebut some of their comments:
FxCop has a good set of rules however there are some problems:
FxCop works against compiled assemblies and uses metadata to get information about the code. SSW Code Auditor on the other hand looks at the source code itself.
I never realized you could do a code review by reading the versioning or security information of an assembly. I guess they meant MSIL instead… however it helps to understand a product before comparing it. (Update: this has been fixed on their site)
FxCop only works against .NET assemblies. SSW Code Auditor on the other hand works against any plain text files.
Correct – but I don’t have a problem with this. I personally don’t use Code Auditor for anything other than my C# code. If two pieces of C# code compile to the same IL then I don’t really have a problem with what the source code was. The runtime runs IL not source code.
Many of the rules of FxCop are obscure. SSW Code Auditor on the other hand has rules that are more 'general developer' oriented.
In my opinion an FxCop rule like “Review unused parameters” is more useful than “Times New Roman should not be used”. Unused parameters generally indicate either an incomplete method implementation or a badly designed method signature. Code Auditor is unable to check for this, although can happily tell me that I’m not allowed to use Times New Roman.
Many people consider FxCop’s rules to be obscure (somewhat understandably) when they get an error thrown on something like “Static holder types should be sealed”. This is a pretty simple rule though – if all of your members are static then you have no instance members. If you have no instance members, you have nothing to override if you inherit from it and it just becomes a marker rather than a base class, which is a perfect situation for attributes. (PS: This rule is now redundant in C# 2.0 with the addition of a “static” keyword for classes.)
FxCop rules are fixed. SSW Code Auditor can easily be extended via new rules (using regular expressions).
FxCop rules are anything but “fixed”. David Kean has written and released a great set of custom FxCop rules. (Update: this has been fixed on their site)
FxCop only provides one type of report. SSW Code Auditor on the other hand provides a report that is flexible and shows the data in different views.
SSW Code Auditor is designed to be scheduled so you keep your code projects clean all the time.
Granted – this has been seen as big limitation with the .NET 1.1 version of FxCop as it would required team managers to have an updated copy of the source code on their machine before being able to run a scan. With Code Auditor you could have the scan running on a schedule on some backroom server and just get it emailed to you. For me though, this argument falls apart rapidly: you’d expect a team manager who is reviewing code to have a copy of the code anyway; FxCop can be automated by integrating it into a continuous build solution like CC.NET.
Code Auditor sits well as one part of a wider auditing and review process. If I was short of time but had to perform a review, I’d be doing things in this order:
- Get experienced developer to look at the code
- Run FxCop
- Run Code Auditor (with a rule set tailored to the organization, not the packaged one)
I would not be performing the following unless I was incredibly bored for an extended period of time:
- Run NUnit (this should be happening on a regular basis by the developer, and in some form of CI scenario)
I continue to rely on ReSharper’s extremely powerful abilities to check and fix my code formatting, and thus I disable all of Code Auditor’s rules in this area (but at least I can do that). (Update: I reported this and ReSharper integration is now on its way)
The consensus is:
- Code Auditor is one part of a wider auditing and review process.
- It’s in my top 3, but experienced developers and FxCop still rank higher.
- The regex builder is great – leading many people to consider buying the product just for this functionality.
- In comparison to other tools, heavier upfront customization is required resulting in higher implementation costs. This is a result of the rules being more specific that most – so it’s really about upfront pain for long term gain – still something that needs to be considered though.
Rocky H has also posted a review of Code Auditor on his blog which includes a screenshot based walkthrough and some interesting discussion in the comments.
Tatham Oddie 