Finding Defects in C#: Coverity vs. FxCop

Finding Defects in C#

Key Considerations

• Does it find critical defects?

• What is the false positive rate?

• Is it actionable?

• Is it accurate?

• Is it integrated to my workflow?

• How do I manage persistency

Selecting the Right Solution

• Compiler warnings: verifies a program is type safe

• Byte code analysis: identifies defects in the intermediate language and tries to map it back to the source code

• Source code analysis: understanding the meaning and intention of the program – produces the most accurate results

Varying Levels of Static Analysis Exist

if (x == 0)

do_something(x);

x = 1;• Source code analysis solution can infer the developer’s intent:

“x=1” to happen in the same block as “do_something” call

• Developer is warned because “x==0” block does not actually include both statements

Indentations Don’t Match Boundaries:

Source vs. Byte Code Analysis (Example)

Coverity and FxCop Case Study

Complementary Solutions

Stand-alone FxCop is good; FxCop + Coverity is better

Coverity Makes FxCop Enterprise-Grade

• Find more critical defects • Improve accuracy of FxCop analysisAnalysis

• Manage all quality and security issues in one workflow

• Improved defect managementEfficiency

• Improve visibility into quality and security trends over time and across the supply chain

Governance

Case Study• Analysis of paint.net project (formerly open

source)• Version 3.22

• 100K lines of code

• Analysis done using• Coverity 7.0

• Microsoft Visual Studio 2013/FxCop 12.0

• Coverity and FxCop look for different things• Coverity Static Analysis looks for code defects using:

• Bug Pattern Matching, Sophisticated Inter-procedural Dataflow Analysis, Abstract Interpretation, False Path Pruning, Boolean Satisfiability, Design Pattern Intelligence, Change Impact Analysis

• FxCop checks conformance to Microsoft’s .NET Framework Design Guidelines

• Difference in depth vs. breadth

• No issues found by both Coverity and FxCop• Numbers in orange indicate number of findings

Coverity FxCop

978

0

128

Critical Defects

Coding style & standard

issues

Different Solutions for Different Things

Critical Defects vs. Coding Style Defects

Type Coverity 7.0

FxCop Shared defects

Resource leaks 75 0 0

Concurrency problems 20 4 0

Logic errors 4 2 0

Hierarchy problems 5 2 0

Unhandled exceptions (incl. NULL deref)

21 0 0

Critical Defect Subtotal 125 8 0

Coding Standards, Best Practices, Other

3 970 0

Total Bugs 128 978 0

The “Big 3” Classes of Defects in C#

1. Null references

2. Resource issues

3. Threading issues

Resource Leaks• Database connection leaks• Resource leaks• Socket & Stream leaks

API usage errors• Use of freed resources

Concurrent data access violations• Values not atomically updated• Data race conditions

Performance inefficiencies• Unnecessary synchronization

Program hangs• Thread deadlock• Infinite loop

Logic Errors• Dead code

Error handling issues• Unchecked return value

Code maintainability issues• Static set in non-static method

Class hierarchy inconsistencies• Failure to call base.close() or

base.dispose()• Missing call to base class

Control flow issues• Suspicious extraneous semicolon• Inconsistent comparison usage• Comparison of incompatible types

Null pointer dereferences• Dereference after null check• Dereference before null check• Dereference null return value

Suspicious code• Copy/paste errors• Significant indentation anomalies• Swapped arguments

Arithmetic errors• Incorrect shift operation• Incorrect expressions• Overflow while evaluating expression

Issues You Can Find via Source Code Analysis

• Different analysis tools often find different but complementary issues

• Use the right solution to find the issues that are important to you

Conclusion

Want to try Coverity on your code?

For a free trial visit:

www.coverity.com