Copyright 2010 TCloud Computing Inc. Java Unit Test and Coverage Introduction Alex Su 2010/07/21
May 10, 2015
Copyright 2010 TCloud Computing Inc.
Java Unit Test and Coverage IntroductionAlex Su2010/07/21
Agenda
Trend Micro Confidential
• Test-driven development• Junit
• Ant Integration• Mock Object
• Mockito• Code Coverage Analysis• Coverage Tool
• Emma• Cobertura
• Case Study
Test-driven development
Trend Micro Confidential
Test-driven development (TDD) is a software development technique that relies on the repetition of a very short development cycle
• Add a test• Run all tests and see if the new one fails• Write some code• Run the automated tests and see them succeed• Refactor code
Test-driven development - Scenario
Trend Micro Confidential
Given/When/Then
Example 1: New lists are emptyGiven a new listThen the list should be empty.
Example 2: Lists with things in them are not empty.Given a new listWhen we add an objectThen the list should not be empty.
Test-driven development - Scenario
Trend Micro Confidential
class ListTest { @Test public void shouldKnowWhetherItIsEmpty() { List<String> list1 = new ArrayList<String>(); assertTrue(list1.isEmpty()); List<String> list2 = new ArrayList<String>(); list2.add("element"); assertFalse(list2.isEmpty()); } }
Test-driven development
Trend Micro Confidential
JUnit
Trend Micro Confidential
• JUnit is a unit testing framework for the Java programming language.
• JUnit is linked as a JAR at compile-time; the framework resides under packages junit.framework for JUnit 3.8 and earlier and under org.junit for JUnit 4 and later.
JUnit
Trend Micro Confidential
• Test RunnerA Runner runs tests. You will need to subclass Runner when using RunWith to invoke a custom runner.
• Test FixtureA test fixture represents the preparation needed to perform one or more tests
• Test CaseA test case defines the fixture to run multiple tests.
• Test SuiteA Test Suite is a Composite of Tests. It runs a collection of test cases.
Junit – Test Result
Trend Micro Confidential
• Success
• FailureA failure is when one of your assertions fails, and your JUnit test notices and reports the fact.
• ErrorAn error is when some other Exception occurs--one you haven't tested for and didn't expect, such as a NullPointerException or an ArrayIndexOutOfBoundsException.
Junit – Annotation
Trend Micro Confidential
• @BeforeClass• @AfterClass• @Before• @After• @Test• @Ignore
JUnit
Trend Micro Confidential
public class DummyTest { private static List<String> list;
@BeforeClass public static void beforeClass() { list = new ArrayList<String>(); } @AfterClass public static void afterClass() { list = null; } @Before public void before() { list.add("Alex"); } @After public void after() { list.remove("Alex"); } @Test public void getElement() { String element = list.get(0); assertEquals(element, "Alex"); } @Ignore("Not Ready to Run") @Test public void notRun() { assertEquals(list.size(), 2); } @Test(expected = IndexOutOfBoundsException.class) public void getElementWithException() { list.get(1); }}
JUnit
Trend Micro Confidential
@RunWith(Suite.class)@SuiteClasses({DummyTest1.class, DummyTest2.class})public class DummyTestSuite {}
JUnit
Trend Micro Confidential
@RunWith(SpringJUnit4ClassRunner.class)@ContextConfiguration(locations = { "classpath:appContext.xml" })public class DummyTest { @Resource private DummyService dummyService;
@Test public void createDummy() { Dummy dummy = new Dummy(); dummy.setId(ID); dummy.setName("Hermione Granger"); dummy.setDescription("Gryffindor");
dummyService.createDummy(dummy);
dummy = dummyService.getDummy(ID); assertNotNull(dummy); }}
Junit – Ant Integration
Trend Micro Confidential
<junit showoutput="${build.test.dir}" printsummary="on" fork="yes" haltonfailure="false>
<classpath refid="classpath" /><formatter type="xml" />
<batchtest todir="${build.test.dir}"><fileset dir="${test.dir}">
<include name="**/*Test.java" /></fileset>
</batchtest></junit>
<junitreport todir="${doc.test.dir}"><fileset dir="${build.test.dir}">
<include name="**/TEST-*.xml" /></fileset> <report format="frames" todir="${doc.test.dir}" /></junitreport>
Mock Object
Trend Micro Confidential
• supplies non-deterministic results (e.g. the current time or the current temperature);
• has states that are difficult to create or reproduce (e.g. a network error);
• is slow (e.g. a complete database, which would have to be initialized before the test);
• does not yet exist or may change behavior;• would have to include information and methods
exclusively for testing purposes (and not for its actual task).
Mock Object - Mockito
Trend Micro Confidential
• Argument matchers • Verifying exact number of invocations / at least x /
never• Verification in order• Shorthand for mocks creation - @Mock annotation• Stubbing consecutive calls • Spying on real objects• Aliases for behavior driven development
Mock Object - Mockito
Trend Micro Confidential
public class ListTest { private List<String> mockedList;
@Before public void initMocks() { mockedList = mock(List.class);
given(mockedList.get(0)).willReturn("first", "second"); given(mockedList.get(1)).willThrow(new IndexOutOfBoundsException()); }
@Test public void getList() { assertEquals("first", mockedList.get(0)); assertEquals("second", mockedList.get(0)); }
@Test(expected = IndexOutOfBoundsException.class) public void getListWithException() { mockedList.get(1); }}
Code Coverage Analysis
Trend Micro Confidential
• Finding areas of a program not exercised by a set of test cases.
• Creating additional test cases to increase coverage.• Determining a quantitative measure of code
coverage, which is an indirect measure of quality. • Identifying redundant test cases that do not
increase coverage.
Code Coverage Analysis
Trend Micro Confidential
• Basic Metrics• Statement Coverage(Line)• Basic Block Coverage• Decision Coverage(Branch)• Condition Coverage • Multiple Condition Coverage • Condition/Decision Coverage• Modified Condition/Decision Coverage • Path Coverage
Code Coverage Analysis
Trend Micro Confidential
• Statement Coverage(Line)• Basic Block Coverage• Decision Coverage(Branch)• Condition Coverage • Multiple Condition Coverage
boolean a = true, b = true;if(a && b) { System.out.println("true");} else { System.out.println("false");}
Code Coverage Analysis
Trend Micro Confidential
• Condition/Decision Coverage(C/DC)• Modified Condition/Decision Coverage(MC/DC)
boolean a = true, b = true;if(a && b) { System.out.println("true");} else { System.out.println("false");}
if(a || b) { System.out.println("true");} else { System.out.println("false");}
Code Coverage Analysis
Trend Micro Confidential
• Path Coverage
X = 0
A>1
A=2 or
X > 1
X = 20
Y
Y
N
N
A = 3 and X = 5A = 0 and X = 3
A = 0 and X = 0A = 2 and X = 0
print x
Code Coverage Analysis
Trend Micro Confidential
• Other Metrics • Function Coverage• Data Flow Coverage • Loop Coverage • Race Coverage • Relational Operator Coverage
Coverage Tool
Trend Micro Confidential
• Instrumentation• Source code instrumentation• Bytecode instrumentation• Runtime instrumentation
Coverage Tool
Trend Micro Confidential
Possible feature Runtime Bytecode Source code
Gathers method coverage yes yes yes
Gathers statement coverage
line only yes yes
Gathers branch coverage indirectly yes yes
Can work without source yes yes no
Requires separate build no no yes
Requires specialised runtime
yes not accurate no
View coverage data inline with source
not accurate yes yes
Source level directives to control coverage gathering
no no yes
Runtime performace high impact variable variable
Container friendly no not accurate yes
Coverage Tool
Trend Micro Confidential
• Reports are filterable so you can tell what needs to be evaluated for code coverage.
• offline and on-the-fly instrumentation.• Ant integration.• JUnit integration.
Coverage Tool - Emma
Trend Micro Confidential
• Last update : 2005-06-13• Stats on both class and method coverage• Partial/fractional line coverages is unique trait -
shown in yellow when there are multiple conditions in a conditional block
• Stricter code coverage.• Integration with Eclipse available -
http://www.eclemma.org/• Better documentation than cobertura.• Instrumentation process is faster than cobertura.• Standalone library and does not have any external
dependencies.• Common public license 1.0 friendlier that GPL.
Coverage Tool - Cobertura
Trend Micro Confidential
• Last update : 2010-03-03• GPL'd version of JCoverage (which is commercial). • Prettier reports.• Branch/block and line coverages only - no class or
method level coverage.• How many times a line has been executed• <cobertura-check> where one can specify percentage of
coverage that's a MUST or else build fails.• Data merge feature - good for QA labs... for merging
coverage data to prepare historical trend graphs.• Depends on other third party libraries.• Integration with Eclipse available -
http://ecobertura.johoop.de/
Coverage Tool – Comparsion(Exception)
Trend Micro Confidential
• Source CodeString str = "abc";System.out.println(str.substring(5));
• Decompile ResultString str = "abc";System.out.println(str.substring(5));
Coverage Tool - Comparsion(Exception)
Trend Micro Confidential
• Emmaboolean aflag[] = ($VRc != null ? $VRc : $VRi())[3];
String s = "abc";
System.out.println(s.substring(5));
aflag[0] = true;
Coverage Tool - Comparsion(Exception)
Trend Micro Confidential
• Cobertura boolean flag = false; int __cobertura__branch__number__ = -1; TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 25); String str = "abc"; TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 26); System.out.println(str.substring(5)); TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 27);
Coverage Tool - Comparsion(Exception)
Trend Micro Confidential
• Clover__CLR3_0_217a17agbtkjkdv.R.inc(1571);__CLR3_0_217a17agbtkjkdv.R.inc(1572);String str = "abc";__CLR3_0_217a17agbtkjkdv.R.inc(1573);System.out.println(str.substring(5));
Coverage Tool – Comparsion(Loop)
Trend Micro Confidential
• Source CodeList<String> list = new ArrayList<String>();for(String element : list) { System.out.println(element);}
• Decompile resultString element;for(Iterator iterator = list.iterator(); iterator.hasNext(); System.out.println(element))
element = (String)iterator.next();
Coverage Tool - Comparsion(Loop)
Trend Micro Confidential
• Emmaboolean aflag[] = ($VRc != null ? $VRc : $VRi())[4];
List list = new ArrayList();
Iterator iterator = list.iterator();
aflag[0] = true;
do
{
aflag[2] = true;
if(iterator.hasNext())
{
String s = (String)iterator.next();
System.out.println(s);
aflag[1] = true;
} else
{
break;
}
} while(true);
aflag[3] = true;
Coverage Tool - Comparsion(Loop)
Trend Micro Confidential
• Coberturaboolean flag = false;int __cobertura__branch__number__ = -1;TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 36);List list = new ArrayList();TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 37);Iterator iterator = list.iterator();do{
TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 37);__cobertura__line__number__ = 37;__cobertura__branch__number__ = 0;if(iterator.hasNext()){
if(__cobertura__branch__number__ >= 0){
TouchCollector.touchJump("tcloud.dummy.service.CoverageTestMain", __cobertura__line__number__, __cobertura__branch__number__, false);
__cobertura__branch__number__ = -1;}String element = (String)iterator.next();TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 38);System.out.println(element);
} else{
if(__cobertura__line__number__ == 37 && __cobertura__branch__number__ == 0){
TouchCollector.touchJump("tcloud.dummy.service.CoverageTestMain", __cobertura__line__number__, __cobertura__branch__number__, true);
__cobertura__branch__number__ = -1;}TouchCollector.touch("tcloud.dummy.service.CoverageTestMain", 40);return;
}} while(true);
Coverage Tool - Comparsion(Loop)
Trend Micro Confidential
• Clover __CLR3_0_2145145gbu63tpg.R.inc(1462); __CLR3_0_2145145gbu63tpg.R.inc(1463); List list = new ArrayList(); __CLR3_0_2145145gbu63tpg.R.inc(1464); String element; for(Iterator iterator = list.iterator(); iterator.hasNext();
System.out.println(element)) { element = (String)iterator.next(); __CLR3_0_2145145gbu63tpg.R.inc(1465); }
Coverage Tool - Comparsion(Loop)
Trend Micro Confidential
• Clover with decompile resultIterator iterator;__CLR3_0_2166166gbu71moi.R.inc(1535);__CLR3_0_2166166gbu71moi.R.inc(1536);List list = new ArrayList();__CLR3_0_2166166gbu71moi.R.inc(1537);__CLR3_0_2166166gbu71moi.R.inc(1538);iterator = list.iterator();
_L4:if(iterator.hasNext()){
__CLR3_0_2166166gbu71moi.R.iget(1539);} else{
__CLR3_0_2166166gbu71moi.R.iget(1540);return;
}if(true) goto _L2; else goto _L1
_L1:break; /* Loop/switch isn't completed */
_L2:__CLR3_0_2166166gbu71moi.R.inc(1541);String element = (String)iterator.next();System.out.println(element);if(true) goto _L4; else goto _L3
_L3:
Trend Micro Confidential
Case Study
Trend Micro Confidential
THANK YOU!