Java Library Evolution Puzzlers

Java Library Evolution Puzzlers

Jens Dietrich1

1Massey UniversitySchool of Engineering and Advanced Technology

Palmerston North, New Zealandhttps://sites.google.com/site/jensdietrich/

Email: j.b.dietrich /at/ massey.ac.nz

August 31, 2014

1

https://sites.google.com/site/jensdietrich/

Revision History

Revision Date Remarks

1.0 13 Sept 13 initial version2.0 10 Feb 14 added bridge (synthetic methods

generated by compiler)3.0 14 Feb 14 added generics3 (changing the order

of multiple type parameter bounds)4.0 31 Aug 14 added static* (static vs non-static)

2

Table of Contents

Introduction

Modifying Interfaces

Modifying Method Signatures

Static vs Non-Static

Primitive vs Wrapper Types

Using Generic Parameter Types

Changing the Values of Constants

Modifying Exceptions

Miscellaneous

Summary

References

3

Introduction - Deploying Java Programs

I Java programs are usually built (ant,maven,gradle,..) with alllibraries they use, and then deployed

I if the program or a library changes, the program is rebuilt andredeployed

I the build step includes V&V: compiling and (automatedregression) testing

I partial library upgrades are becoming more and more popular,example: OSGi bundle updates

I the puzzlers described here show the difference between thesetwo deployment modes

4

Introduction - Source vs Binary Compatibility

I a program is source compatible with a library lib.jar if theprogram uses the library, and compilation succeeds:javac -cp ..,lib.jar,.. ...

I source compatibility is checked by the compiler,incompatibility results in compilation errors

I a program is binary compatible with a library lib.jar if itlinks and runs with this library:java -cp ..,lib.jar,.. .. [JLS, ch. 13]

I binary compatibility is checked by the JVM, incompatibilityresults in (linkage) errors

5

The Limitations of Binary Compatibility

I in the JLS, a very narrow definition of binary (in)compatibilityis used: “ A change to a type is binary compatible withpre-existing binaries if pre-existing binaries that previouslylinked without error will continue to link without error.”[JLS, ch. 13.2]

I binary compatibility is defined w.r.t. to what the linker candetect by means of static analysis, failure results in errors(not exceptions)

6

Introduction - Evolution Problems

I assume that a program references code defined inlib-1.0.jar

I assume that the program can be compiled successfully andcan be executed with lib-1.0.jar without causing an erroror exception

I then the library evolves to lib-2.0.jar

7

Introduction - Evolution Problems

Questions:

1. does the program link with lib-2.0.jar - i.e., is it binarycompatible with lib-2.0.jar?

2. does replacing the library change the behaviour of the program- i.e., is the library change binary behavioural compatible?

3. does the program compile against lib-2.0.jar - i.e., is itsource compatible with lib-2.0.jar?

4. does recompiling the program against the changed librarychange the behaviour of the program - i.e., is the librarychange source behavioural compatible?

8

Introduction - Running Experiments

I check out code:hg clone https://bitbucket.org/jensdietrich/

java-library-evolution-puzzlers

I each example has a program with a main classaPackage.Main, and two versions of classes defined in aseparate library

I cd to folder and run ant as follows:ant -Dpackage=aPackage

I this will do the following:

1. compile the two versions of the library and build lib-1.0.jar

and lib-2.0.jar

2. compile and run the program with lib-1.0.jar

3. compile the program with lib-1.0.jar , but run it withlib-2.0.jar

4. re-compile and run the program with lib-2.0.jar

9

https://bitbucket.org/jensdietrich/java-library-evolution-puzzlers

https://bitbucket.org/jensdietrich/java-library-evolution-puzzlers

Adding a Method to an Interface

lib-1.0.jar

package lib.addtointerface;public interface Foo {

public void foo();}

⇓lib-2.0.jar

package lib.addtointerface;public interface Foo {

public void foo();public void bar();

}

program

package addtointerface;import lib.addtointerface.∗;public class Main implements Foo {

@Override public void foo() {System.out.println(”foo”);

}public static void main(String[] args) {

new Main().foo();}

}

I the interface Foo is extendedby adding bar()

I but the client classimplements the old interface

I is this still binary compatiblewith lib-2.0.jar?

10

Adding a Method to an InterfaceSolution

I running the program with library version 2.0 succeeds - theclient program is not using the method added to the interface!

I i.e., the program (compiled with lib-1.0.jar) is binarycompatible with lib-2.0.jar

I but recompilation fails as Main does not implement bar()

I i.e., the program is source incompatible with lib-2.0.jar

11

Removing a Method from an Interface 1

lib-1.0.jar

package lib.removefrominterface1;public interface Foo {


}

⇓lib-2.0.jar


public void foo();}

program

package removefrominterface1;import lib.removefrominterface1.∗;public class Main implements Foo {

@Override public void foo() {System.out.println(”foo”);

}@Override public void bar() {

System.out.println(”bar”);}public static void main(String[] args) {

new Main().foo();new Main().bar();

}}

I the method bar() is removedfrom the interface Foo

I but the client classimplements the old interface

12

Removing a Method from an Interface 1Solution

I running the program with library version 2.0 succeeds !

I i.e., the program (compiled with lib-1.0.jar) is binarycompatible with lib-2.0.jar

I but recompilation fails as Main.bar() does not override amethod!


13


lib-1.0.jar



}

⇓lib-2.0.jar


public void foo();}

program


public void foo() {System.out.println(”foo”);

}public void bar() {


new Main().foo();new Main().bar();

}}

I this is almost identical to theprevious example

I but this time the @Override

annotation is not used

14


I as before, the program (compiled with lib-1.0.jar) isbinary compatible with lib-2.0.jar

I but recompilation also succeeds as the compiler does notcheck whether Main.bar() overrides a method

I i.e., the program is also source compatible withlib-2.0.jar

15


lib-1.0.jar



}

⇓lib-2.0.jar


public void foo();}

program



}public void bar() {


Foo f = new Main();f.foo();f.bar();

}}

I this is similar to the previousexample

I note the declaration of f inmain

16


I this time the program is binary incompatible withlib-2.0.jar: a linkage error (NoSuchMethodError) occursas the linker now tries to find bar() in Foo (the declared typeof f), not in Main (the actual type)

I compilation against lib-2.0.jar fails for the same reason -the compiler also fails to find bar() in Foo


as well

17

Specialising Return Types 1

lib-1.0.jar

package lib.specialiseReturnType1;public class Foo {

public static java.util.Collection getColl() {return new java.util.ArrayList();

}}

⇓lib-2.0.jar


public static java.util.List getColl() {return new java.util.ArrayList();

}}

program

package specialiseReturnType1;import lib.specialiseReturnType1.Foo;public class Main {

public static void main(String[] args) {java.util.Collection coll = Foo.getColl();System.out.println(coll);

}}

I return type is replaced by asubtype

I i.e., postconditions arestrengthened (methodguarantees more)

I program should run withlib-2.0.jar !

18

Specialising Return Types 1Solution

I running the program with library version 2.0 fails !

I inspecting byte code (javap -c Main.class) shows thatmain references getColl asgetColl()Ljava/util/Collection; - and this descriptorhas changed

I the result is a linkage error (NoSuchMethodError)

I recompiling (and then running) the program withlib-2.0.jar succeeds

I i.e., the program (compiled with lib-1.0.jar) is binaryincompatible but source compatible with lib-2.0.jar

19


lib-1.0.jar


public static long getAnswer() {return 42L;

}}

⇓lib-2.0.jar


public static int getAnswer() {return 42;

}}

program

package specialiseReturnType2;import lib.specialiseReturnType2.Foo;public class Main {

public static void main(String[] args) {long i = Foo.getAnswer();System.out.println(i);

}}

I return type is narrowed fromlong to int

I similar to specialisingreference types

20


I again, this is binary incompatible, but source compatible

I i.e., the problem can easily be fixed through recompilation

I clients can safely widen the int to a long

21


lib-1.0.jar

package lib.specialiseReturnType3;import java.util.∗;public class Foo {

public Collection getColl() {return new ArrayList();

}}

⇓lib-2.0.jar


public List getColl() {return new ArrayList();

}}

program

package specialiseReturnType3;import lib.specialiseReturnType3.Foo;import java.util.∗;public class Main extends Foo {

public static void main(String[] args) {Foo f = new Main();Collection c = f.getColl();System.out.println(c);

}@Override public Collection getColl() {

return new HashSet();}

}

I return type Collection isreplaced by subtype List

I but getColl() is nowoverridden in Main !

22


I as before, the program is binary incompatible withlib-2.0.jar: java.lang.NoSuchMethodError:

lib.specialiseReturnType3.Foo.getColl()

Ljava/util/Collection

I recompilation with lib-2.0.jar fails as well: “compilererror: return type Collection is not compatible with List”

I i.e., the program is neither binary nor source compatiblewith lib-2.0.jar

I when overriding a method, the return type can only bespecialised (co-variant return types [JLS, 8.4.5]), but this doesnot apply here as the overridden method itself has specialisedits return type

23


lib-1.0.jar


public Collection getColl() {return new ArrayList();

}}

⇓lib-2.0.jar


public List getColl() {return new ArrayList();

}}

program

package specialiseReturnType4;import lib.specialiseReturnType4.Foo;import java.util.∗;public class Main extends Foo {

public static void main(String[] args) {Main f = new Main();Collection c = f.getColl();System.out.println(c);

}@Override public Collection getColl() {

return new HashSet();}

}

I minor change: f is nowdeclared as Main, not Foo

I what impact does this have?

24


I the program runs but does not compile with lib-2.0.jar !

I i.e., the program is binary compatible but sourceincompatible

I to find out why, inspect byte code

I Specialising Return Types 4: getColl() is referenced asgetColl:()Ljava/util/Collection;

- reference to local method that hasn’t changed

I Specialising Return Types 3: getColl() is referenced aslib/specialiseReturnType3/Foo.getColl:()

Ljava/util/Collection;

- reference to inherited method that has changed

25

Generalising Parameter Types 1

lib-1.0.jar

package lib.generaliseParamType1;public class Foo {

public static void doIt(java.util.List coll) {System.out.println(coll);

}}

⇓lib-2.0.jar


public static void doIt(java.util.Collection coll) {System.out.println(coll);

}}

program

package generaliseParamType1;import lib.generaliseParamType1.Foo;public class Main {

public static void main(String[] args) {Foo.doIt(new java.util.ArrayList());

}}

I param type List is replacedby supertype Collection

I this can be seen as weakenedprecondition (expects less)

I should be compatible !

26

Generalising Parameter Types 1Solution


I similar to changing return types, the descriptor changes,resulting in a linkage error (NoSuchMethodError)

I recompiling (and then running) the program with libraryversion 2.0 succeeds

I i.e., the program (compiled with lib-1.0.jar) is binaryincompatible but source compatible with lib-2.0.jar

27


Class1 Class2

Interface1 Interface2

implements

28


lib-1.0.jar


public static void doIt(Class1 c) {System.out.println(”C1”);

}public static void doIt(Interface2 c) {

System.out.println(”I2”);}

}

⇓lib-2.0.jar


public static void doIt(Interface1 c) {System.out.println(”I1”);

}public static void doIt(Interface2 c) {

System.out.println(”I2”);}

}

program

package generaliseParamType2;import lib.generaliseParamType2.∗;public class Main {

public static void main(String[] args) {Foo.doIt(new Class1());

}}

I doIt is overloaded

I can the compiler select amethod after generalising theparameter type?

29



I the descriptor changes, resulting in a linkage error(NoSuchMethodError)

I recompiling fails as well - the compiler cannot select the mostspecific method [JLS, 15.12]: Error: reference to doIt isambiguous, both method doIt(Interface1) in Foo and methoddoIt(Interface2) in Foo match.

I i.e., the program (compiled with lib-1.0.jar) is neither binarynor source compatible with lib-2.0.jar

30


lib-1.0.jar


public static boolean isEven(int i) {return i%2==0;

}}

⇓lib-2.0.jar


public static boolean isEven(float i) {return i%2==0;

}}

program

package generaliseParamType3;import lib.generaliseParamType3.Foo;public class Main {

public static void main(String[] args) {int n = Integer.MAX VALUE;System.out.println(Foo.isEven(n));

}}

I is the program binary andsource compatible?

I what is printed on theconsole?

31


I the program is not binary compatible with lib-2.0.jar,but seems to be source compatible - it can be recompiledand then executed

I however, the output changes: while the original programprints true, the recompiled program prints false

I the type parameter change changes the semantics of theprogram - although the method body is not changed !

I the change is source compatible, but source behaviouralincompatible

I the problem is that the widening conversion from int tofloat results in loss of precision [JLS, ch. 5.1.2]

32

Change a Method from Static to Non-Static

lib-1.0.jar

package lib.static1;public class Foo {

public static void foo() {System.out.println(”foo”);

}}

⇓lib-2.0.jar



}}

program

package static1;import lib.static1.Foo;public class Main {

public static void main(String[] args) {Foo.foo();

}}

I remove the static modifierfrom foo()

33

Change a Method from Static to Non-StaticSolution

I the change is source incompatible: a non-static methodcannot be referenced from a static context

I an instance must be created to invoke a non-static method

I but what about binary compatibility ?

I lets consider the reverse scenario first

34

Change a Method from Non-Static to Static

lib-1.0.jar



}}

⇓lib-2.0.jar


public static void foo() {System.out.println(”foo”);

}}

program

package static2;import lib.static2.Foo;public class Main {

public static void main(String[] args) {new Foo().foo();

}}

I add a static modifier to foo()

35

Change a Method from Static to Non-StaticSolution

I the change is source compatible

I many IDEs will generate a warning: static methods should bereferences using static context (Foo.foo())

I but the change is still binary incompatible: ajava.lang.IncompatibleClassChangeError is thrown

I the same happens in the previous scenario

36

Static vs Non-StaticSolution

I the reason is the use of different byte code instructions:

I static methods are invoked using invokestatic, fornon-static methods, invokevirtual is used instead

I the JVM checks the method type during linking, and createsan IncompatibleClassChangeError if a unexpected type isencountered [JVMS, ch. 5.4]

I the same applies for field (read and write) access: there aredifferent byte code instructions for accessing static andnon-static fields: getfield, putfield, getstatic,putstatic

37

Primitive vs Wrapper Types 1

lib-1.0.jar

package lib.primwrap1;public class Foo {

public static int MAGIC = 42;}

⇓lib-2.0.jar


public static Integer MAGIC = new Integer(42);}

program

package primwrap1;import lib.primwrap1.Foo;public class Main {

public static void main(String[] args) {int i = Foo.MAGIC;System.out.println(i);

}}

I the field type int is replacedby its wrapper type Integer

I is this transparent to theclient program?

38

Primitive vs Wrapper Types 1Solution


I the descriptors have types, resulting in a linkage error(NoSuchFieldError)

I recompiling (and then running) the program withlib-2.0.jar succeeds - the compiler applies unboxing[JLS, 5.1.8]

I i.e., the program is binary incompatible but sourcecompatible with lib-2.0.jar

39

Primitive vs Wrapper Types 2

lib-1.0.jar


public static Integer MAGIC = new Integer(42);}

⇓lib-2.0.jar


public static int MAGIC = 42;}

program

package primwrap2;import lib.primwrap2.Foo;public class Main {

public static void main(String[] args) {Integer i = Foo.MAGIC;System.out.println(i);

}}

I the field type Integer isreplaced by the respectiveprimitive type int

I is this transparent to theclient program?

40

Primitive vs Wrapper Types 2Solution


I the descriptors have different types, resulting in a linkage error(NoSuchFieldError)

I recompiling (and then running) the program withlib-2.0.jar succeeds - the compiler applies boxing [JLS,5.1.7]

I i.e., the program is binary incompatible but sourcecompatible with lib-2.0.jar

41

Generics 1

lib-1.0.jar

package lib.generics1;import java.util.∗;public class Foo {

public static List<String> getList() {List<String> list = new ArrayList<String>();list.add(”42”);return list;

}}

⇓lib-2.0.jar


public static List<Integer> getList() {List<Integer> list = new ArrayList<Integer>();list.add(42);return list;

}}

program

package generics1;import lib.generics1.∗;public class Main {

public static void main(String[] args) {java.util.List<String> list = Foo.getList();System.out.println(list.size());

}}

I generic type parameter inmethod return type ischanged

I does this matter?

42

Generics 1Solution

I this is binary compatible due to type erasure in Java

I however, this is not source compatible - the compiler cannotassign a list of integers to a variable declared as a list ofstrings

43

Generics 2

lib-1.0.jar


public static List<String> getList() {List<String> list = new ArrayList<String>();list.add(”42”);return list;

}}

⇓lib-2.0.jar


public static List<Integer> getList() {List<Integer> list = new ArrayList<Integer>();list.add(42);return list;

}}

program

package generics2;import lib.generics2.∗;public class Main {

public static void main(String[] args) {java.util.List<String> list = Foo.getList();for (String s:list) {

System.out.println(s);}

}}

I note that only the way thegeneric type is used haschanged

I the program iterates over thestrings in the list

44

Generics 2Solution

I this is binary compatible acc. to the JLS

I when the elements are accessed inside the loop, a castinstruction (checkcast) is inserted by the compiler

I this cast fails when the list is changed to a list of integers, anda runtime exception is thrown

I the change is therefore binary behavioural incompatible

I this is not source compatible either

45

Generics 3

lib-1.0.jar

package lib.generics3;import java.io.Serializable;public class Foo<T extends Serializable & Comparable> {

public void foo(T t) {t.compareTo(””);System.out.println(t);

}}

⇓lib-2.0.jar

package lib.generics3;import java.io.Serializable;public class Foo<T extends Comparable & Serializable>{

public void foo(T t) {t.compareTo(””);System.out.println(t);

}}

program

package generics3;import lib.generics3.∗;public class Main implements java.io.Serializable {

public static void main(String[] args) {Main m = new Main();new Foo().foo(m);

}}

I Main only implementsSerializable, but notComparable

I can Main even be compiled ?

I what is the impact ofchanging the order of theinterfaces defining the boundsof the type parameter?

46

Generics 3Solution

I the program compiles and links with lib-1.0.jar, despitenot implementing both interfaces!

I however, executing the program with lib-1.0.jar leads to aClassCastException

I the reason for this is how erasure works: only the leftmostbound is used [JLS, ch. 4.6]

I i.e., foo(T) is referenced as foo(Serializable)

I before compareTo is invoked (at runtime!), the parameter iscast to Comparable, and this fails

47

Generics 3Solution ctd

I changing the order of the interfaces in lib-2.0.jar is binaryand source incompatible

I now the leftmost bound is Comparable, i.e., foo(T) isreferenced as foo(Comparable)

I this incompatibility is detected by both the compiler and thelinker

48

Constants 1

lib-1.0.jar

package lib.constants1;public class Foo {

public static final int MAGIC = 42;}

⇓lib-2.0.jar


public static final int MAGIC = 43;}

program

package constants1;import lib.constants1.∗;public class Main {

public static void main(String[] args) {System.out.println(Foo.MAGIC);

}}

I now the question is: whatdoes this program print?

49

Constants 1Solution

I the program prints 42 when it is executed with lib-1.0.jar

as expected

I but the program still prints 42 when executed withlib-2.0.jar !

I the compiler inlines the constant value into the client class

I this is binary compatibility but binary behaviouralincompatible

50

Constants 2

lib-1.0.jar


public static final String MAGIC = ”42”;}

⇓lib-2.0.jar


public static final String MAGIC = ”43”;}

program



}}

I inlining is applied toprimitive data types, butwhat about strings?

I what will be printed to theconsole?

51

Constants 2Solution

I the program still prints 42 when it is executed withlib-2.0.jar

I constant inlining is still applied when strings are used

I strings are immutable objects, and in many cases can betreated like primitive types


52

Constants 3

lib-1.0.jar


public static final int MAGIC = 40+2;}

⇓lib-2.0.jar


public static final int MAGIC = 40+3;}

program



}}

I now the constant value isdefined by an expression


53

Constants 3Solution

I the program still prints 42 when it is executed withlib-2.0.jar

I the compiler applies constant folding

I this does not seem to be specified in [JLS] and mighttherefore be a compiler-specific optimisation

I i.e., the expression is evaluated at compile time


54

Constants 4

lib-1.0.jar


public static final Integer MAGIC = 42;}

⇓lib-2.0.jar


public static final Integer MAGIC = 43;}

program



}}

I now the constant is definedusing the wrapper type

I note that assignment is safedue to autoboxing


55

Constants 4Solution

I now 43 is printed as expected when the program runs withlib-2.0.jar !

I i.e., simply by using wrapper types, constant inlining can beprevented (at least using the current version of the compiler)

I A variable of primitive type or type String, that is final andinitialized with a compile-time constant expression (15.28), iscalled a constant variable. [JLS, ch. 4.12.4]

I it is unclear why wrapper types are excluded, they areimmutable as well!

I this may explain why many constants in projects like velocityare defined using wrapper types - to prevent inlining

56

Exceptions

I methods can declare exceptions

I for checked exceptions, the compiler forces callers to handle orrethrow the exception

I at runtime, when an exception occurs the JVM searches theinvocation chain of the method (stack) for a suitableexception handler [JVMS, ch. 2.10]

I the compiler treats exceptions as part of the methoddeclaration - what about the JVM?

I i.e., are certain changes to exceptions (removing or specialisingexceptions) binary incompatible but source compatible?

57

Adding a Runtime Exception

lib-1.0.jar

package lib.exceptions1;public class Foo {

public static void foo() {}}

⇓lib-2.0.jar


public static void foo()throws UnsupportedOperationException {throw new UnsupportedOperationException();

}}

program

package exceptions1;public class Main {

public static void main(String[] args) {lib.exceptions1.Foo.foo();

}}

I in lib-2.0.jar , anUnsupportedOperation-

Exception is declared andthrown

I note that this is a runtime(unchecked) exception

58

Adding a Runtime ExceptionSolution

I the program is source compatible with lib-2.0.jar:UnsupportedOperationException is a runtime exception,and whether it is declared or not makes no difference

I however, while the program is binary compatible, it is binarybehavioural incompatible - but this is only because theexception is actually thrown in lib-2.0.jar

I if the throw statement was removed, the program wouldbecome binary behavioural compatible although the exceptionis still declared

I it would still be possible to compile the modified program -the compiler cannot figure out that a declared uncheckedexception is never thrown

I this makes sense - runtime exceptions are thrown implicitly(null pointers, failed casts etc) and it is too difficult for thecompiler to check this

59

Adding a Checked Exception

lib-1.0.jar


public static void foo() {}

}

⇓lib-2.0.jar

package lib.exceptions2;import java.io.IOException;public class Foo {

public static void foo() throws IOException {throw new IOException();

}}

program


public static void main(String[] args) {lib.exceptions2.Foo.foo();

}}

I in lib-2.0.jar , anIOException is declared orthrown

I this is a checked exception

60

Adding a Checked ExceptionSolution

I not surprisingly, the change is source incompatible

I however, the program is binary compatible withlib-2.0.jar

I i.e., the declared exception is not detected during linking asthis is not part of the method descriptor

I the change is binary behavioural incompatible as theexception is thrown but not caught

61

Generalising a Checked Exception

lib-1.0.jar



}}

⇓lib-2.0.jar


public static void foo() throws Exception {throw new Exception();

}}

program

package exceptions3;import java.io.IOException;public class Main {

public static void main(String[] args) {try {

lib.exceptions3.Foo.foo();}catch (IOException x) {

System.err.println(”Caught it”);}

}}

I in lib-2.0.jar , theIOException is replaced byits super type Exception

I the client program onlyhandles the IOException

62

Generalising a Checked ExceptionSolution

I the program is again source incompatible but binarycompatible with lib-2.0.jar

I but as before, the program behaviour changes as theexception is not caught, i.e. the change is binary behaviouralincompatible

63

Specialising a Checked Exception

lib-1.0.jar


public static void foo() throws Exception {throw new IOException();

}}

⇓lib-2.0.jar



}}

program



lib.exceptions4.Foo.foo();}catch (Exception x) {


}}

I in lib-2.0.jar , theException is replaced by itssub type IOException

I this is similar to specialisingthe return type

64

Specialising a Checked ExceptionSolution

I the program is source and binary compatible withlib-2.0.jar

I this is (surprisingly) different to specialising return types

I the exceptions are not part of the method descriptor used toreferences method when linking

65

Removing a Checked Exception 1

lib-1.0.jar



}}

⇓lib-2.0.jar



}

program



lib.exceptions5.Foo.foo();}catch (IOException x) {


}}

I in lib-2.0.jar , theIOException is removed fromthe method

I but note the exceptionhandler in main

66

Removing a Checked Exception 1Solution

I the program is binary compatible but source incompatiblewith lib-2.0.jar

I the compiler infers that the catch statement is not reachablebecause the updated foo() does not throw an exception:exception IOException is never thrown in body ofcorresponding try statement [JLS, ch. 14.21]

67

Removing a Checked Exception 2

lib-1.0.jar


public static void foo() throws Exception {throw new Exception();

}}

⇓lib-2.0.jar



}

program



lib.exceptions6.Foo.foo();}catch (Exception x) {


}}

I in lib-2.0.jar , theException is removed fromthe method

I but note the exceptionhandler in main

68

Removing a Checked Exception 2Solution

I surprisingly, the program is binary compatible and sourcecompatible with lib-2.0.jar

I the compiler still considers the catch clause as reachable

I this makes sense, as Exception includes runtime exceptions

69

Removing a Checked Exception 2 ctdSolution

I however, it seems to contradict the reachability rules: A catchblock C is reachable iff both of the following are true: Eitherthe type of C’s parameter is an unchecked exception typeor Throwable; or some expression or throw statement in thetry block is reachable and can throw a checked exceptionwhose type is assignable to the parameter of the catch clauseC. [JLS, ch. 14.21] but RuntimeException and all itssubclasses are, collectively, the runtime exception classes. ..The unchecked exception classes are the runtime exceptionclasses and the error classes. [JLS, ch. 11.1.1].

I i.e., one would expect a reachability compiler error!

I this has been report as a bug in the JLS, and will be fixed inJLS-8 (email communication with Alex Buckley)

70

Ghost

lib-1.0.jar

package lib.ghost;public class Foo {

public static class Bar {public static void foo() {

System.out.println(”foo”);}

}}

⇓lib-2.0.jar

package lib.ghost.Foo;

public class Bar {public static void foo() {

System.out.println(”foo”);}

}

program

package ghost;public class Main {

public static void main(String[] args) {lib.ghost.Foo.Bar.foo();

}}

I now the question is: whatdoes this program print?

71

GhostSolution

I the program is not binary compatible with lib-2.0.jar:java.lang.NoClassDefFoundError: lib/ghost/Foo$Bar

I however, the program is source compatible

I the problem is that the reference lib.ghost.Foo.Bar caneither refer to an inner class Bar within the classlib.ghost.Foo, or a top-level class Bar in the packagelib.ghost.Foo

I the byte code representation however differs:lib/ghost/Foo$Bar vs lib/ghost/Foo/Bar

72

Bridge

lib-1.0.jar

package lib.bridge1;public class Foo {

private int foo = 0;public class Inner {

@Overridepublic String toString() {

return ”Inner[foo=”+foo+”]”;}

}}

⇓lib-2.0.jar

package lib.bridge1;public class Foo {

int foo = 0;public class Inner {

@Overridepublic String toString() {

return ”Inner[foo=”+foo+”]”;}

}}

program

package bridge1;import lib.bridge1.∗;import java.lang.reflect.Method;public class Main {

public static void main(String[] args) {Method[] mm = Foo.class.getDeclaredMethods();for (Method m:mm) {

System.out.println(m);}

}}

I reflection is used to find outhow many methods Foo has

I can this be changed by onlychanging the access modifierof the field foo from private

to default?

73

BridgeSolution

I the program is binary and source compatible withlib-2.0.jar

I however, the behaviour changes

I in lib-1.0.jar, a synthetic bridge method static int

lib.bridge1.Foo.access$000(lib.bridge1.Foo) isgenerated by the compiler to enable access to the private fieldby the inner class

I this method is not necessary if access to the field in changedto non-private

I synthetic methods [JLS, ch. 13.1] are widely used, forinstances when overriding methods with generic parametertypes, and co-variant return types

74

Summary

I binary compatibility does not imply source compatibility(example: addtointerface)

I binary compatibility does not imply binary behaviouralcompatibility (example: generics2)

I source compatibility does not imply binary compatibility(example: specialiseReturnType1)

I source compatibility does not imply source behaviouralcompatibility (example: generaliseParamType3)

75

Ongoing Research and Open Questions

I Empirical study on Qualitas Corpus on whether and how oftenthese problems occur in real-world programs. ProceedingsIEEE CSMR-WCRE 2014. Preprint:https://sites.google.com/site/jensdietrich/

publications/preprints

I Quiz developers to find out whether they are aware of this.Over 400 developers have responded, first results here:https://sites.google.com/site/jensdietrich/

java-developer-survey-2013, preprint on arxiv:http://arxiv.org/pdf/1408.2607v1.pdf

I Build better tools (better than clirr) to check librarycompatibility, infer semantic versioning info. Some ongoingwork (Uni of Western Bohemia), more planned for later 2014.

76

https://sites.google.com/site/jensdietrich/publications/preprints

https://sites.google.com/site/jensdietrich/publications/preprints

https://sites.google.com/site/jensdietrich/java-developer-survey-2013

https://sites.google.com/site/jensdietrich/java-developer-survey-2013

http://arxiv.org/pdf/1408.2607v1.pdf

Acknowledgements

I would like to thank Kamil Jezek who contributed Generics 1and Static 1 and 2, Hussain Al Mutawa who pointed me to ghostreferences used in Ghost, and Alex Buckley for his comments andfor contributing Adding a Method to an Interface.

This work was inspired by Java Puzzlers by Joshua Bloch andNeal Gafter [PUZZ].

77

References

JAPI Jim des Rivieres: Evolving Java-based APIs.http://wiki.eclipse.org/Evolving_Java-based_APIs

JLS James Gosling, Bill Joy, Guy Steele, Gilad Bracha and AlexBuckley: The JavaTMLanguage Specification 7th Edition.

JVMS Tim Lindholm, Frank Yellin, Gilad Bracha, and Alex Buckley.The JavaTMVirtual Machine Specification - JavaTMSE 7Edition.

PUZZ Joshua Bloch, Neal Gafter. Java Puzzlers. Addison-Wesley2005.

78

http://wiki.eclipse.org/Evolving_Java-based_APIs