Towards a new world of Java - Java 7 and Harmony Select Lv Jing Apache Harmony PMC.

Towards a new world of JavaTowards a new world of Java- Java 7 and Harmony Select- Java 7 and Harmony Select

Lv JingApache Harmony PMC

AgendaAgenda• Java7 Highlights• Harmony Select Overview• Java Modularization• NIO2 – advanced new io• New GC algorithm

2

HighlightHighlight : VM : VM• JSR 292: VM support for non-Java languages

(InvokeDynamic)

o extending the JVM with first-class architectural support for languages other than Java, especially dynamic languages

o can run non-Java languages efficiently, with a performance level comparable to that of Java itself

o Java language itself will not be dynamic

3

the Da Vinci Machine Project

HighlightHighlight : VM : VM

• Garbage-First GC (G1)o Idea from the Paper ISMM 2004o For “server-style” JVMo Merged into Java6 (hotspot14)

o *Only supported after signing theo contract with Sun/Oracle?

4

HighlightHighlight : VM : VM

• JSR 294: Language and VM support for modular programmingo Tightly bind to Modularization (Project Jigsaw)

• Compressed 64-bit object pointerso Save memory for 64-bit machineo At cost of performance

5

Highlight : LanguageHighlight : Language

• JSR 308: Annotations on Java typeso The Java 6 annotation syntax is useful but limited.

• The Type Annotations syntax permits annotations to be written in more places

• Programmers can use type annotations to write more informative types, can detect and prevent more errors.

e.g: List<@NonNull Object>

6

Highlight : LanguageHighlight : Language• JSR TBD: Project Lambda (Closure)

o The goal of this Project is to formulate a proposal to add first-class functions, function types, and lambda expressions (informally, "closures") to Java,

o to implement a prototype suitable for inclusion in JDK 7 so as to enable broad experimentation.

o key features: • A literal syntax, for writing closures, and • Function types, so that closures are first-class citizens in the type system. • Closure conversion, so that a closure of appropriate type can be used where

an object of a single-method interface or abstract class is required, and • Extension methods, so that closure-oriented bulk-data methods can be

retrofitted onto existing libraries

7

Highlight: CoreHighlight: Core

• JSR 203: More new I/O APIs for the Java platform (NIO.2)o This Project's mission is to produce the implementation of the (New)

New I/O APIs being defined by JSR 203 as well as related work in the JDK.

o Improved filesystem interfaceo Complete socket-channel functionalityo Support asynchronous I/O

8

Highlight: CoreHighlight: Core

• SCTP (Stream Control Transmission Protocol)o The goal of this Project is to develop an API for the Stream

Control Transport Protocol (SCTP) and a corresponding OpenJDK prototype.

• SDP (Sockets Direct Protocol)

9

Highlight: CoreHighlight: Core

• Modularization (Project Jigsaw)o An implementation-specific, simple, low-level module system focused

upon the goal of modularizing the JDK, and the application of that system to the JDK itself

o Language and classlib support for JDKo Focus on future support for modularization

10

Highlight: CoreHighlight: Core

• Concurrency and collections updates (jsr166y)• Unicode 5.1• Upgrade class-loader architecture• Method to close a URLClassLoader• Elliptic-curve cryptography (ECC)

11

Highlight : otherHighlight : other

• Webo Update the XML stack

• client o XRender pipeline for Java 2Do Forward-port 6u10 deployment featureso Create new platform APIs for 6u10 graphics featureso Nimbus look-and-feel for Swingo Swing JLayer component

12

13

Introduction to Harmony ProjectIntroduction to Harmony Project

• Apache Harmony is the Java SE project of the Apache Software Foundation. o created since 2005o http://harmony.apache.org/

• J2SE 5 – milestone 14• J2SE 6 – milestone 2

• V.S OpenJDKo JCK issues

1414

Sun JDK (Treemap)

IBM JDK (10% of J6)

Java ME

Harmony JDK 6

OSGi Profiles

(Adoption of innovation has no borders)

Harmony SelectIBM JDK (30% of J7)

MIWG

The Harmony Adoption and The Harmony Adoption and

CommunityCommunity

Galileo plan to include it as supported execution environment

15

Harmony Select overviewHarmony Select overviewWhat is Harmony SelectA Componentized Runtimewith sufficient capabilities to run server

middleware

Current progress

Applications tested• Geronimo build, tests and samples –pending• Apache Ant unit tests – passed• Apache Maven unit tests – passed • Eclipse – passed

Applications to be tested• Apache Hadoop

16

2009 2010 2011 2012 2013 2014 2015 2016

Community Development: The Open Component RuntimeTechnology evolution, platform development, validation Foster Commercial Industry Ecosystem

Including fit for purpose communities (verticals)Industry positioned as “saving Java”Innovation into Java community

Standards Based specificationsCoordination via IWG or constellation or “dot-org”

Open Source AdoptionEclipse, Apache, Linux distros

Initial commercial supportAlpine, Virtuoso, sMash, p8, XS, XTP,

“Java redefined phase”

Broad adoption of SelectEnterprise Platform

Uber scalable Select platform exploitation of the IBM processor roadmap, “Power 8” heterogeneous (co)processor support - Prism *.*

Component Enterprise IndustryFit for purpose “de rigueur” De facto delivery model for enterprise componentsStandards efforts expand platforms via component definitionOpen source innovation delivered via components to enterprise platforms – “eTunes” Major industry uptake phase

Products available on Select, not “Java”

“Primary Enterprise Platform phase”

IBM Deliveries: GA levelInitial products add Server Profile “right sized” configs

XS, XTP and Alpine stack, sMash, Datapower appliancesNotes, Sametime,

Server Profiles with improved QoSVirtualization integration, suspend, restart, mobilityLarge-Heap, Scalability to TBNRRReal-timeSecurity

Initial Industry uptakeIndustry adopts vision as their own

Patform innovations Multicore performance focused innovations Cloud Java evolution

“Early adoption, platform innovation phase”

We are here

Harmony Select RolloutHarmony Select Rollout

17

Futures – What can we do?Futures – What can we do?GoalsSmaller• Reducing the memory

footprint• Decoupling more code

Faster• Tuning to better

performance• Leveraging IBM

hardware

Smarter• Running on OSGi

based framework

Fit for use• Customizing for IBM

products

Innovations• Fine-grained performance

configuration• Multi-core, huge memory

scalability• Multi-tenancy• Cluster• JVM hibernate• Realtime• Dynamic language• Functional language• Your thoughts?

ObstaclesNo alternatives• JMX 1.4• JAXB• Security providers• Kerberos• SSL

Not fully tested• Modules not in Java7

Java ModularizationJava Modularization• Goal

o An implementation-specific, simple, low-level module system focused upon the goal of modularizing the JDK, and the application of that system to the JDK itself

o Including JSR294 ， project Jigsaw

• Historyo OSGi becomes the standard o Harmony: the first modularized JREo JSR277(Java Module System) raised in JCP, but no much progresso JSR294(Improved Modularity Support in the Java Programming Language) :

simplize the plano Jigsaw becomes one part of the openjdk (independent to JCP)o Harmony OSGi fully implementedo Jigsaw: a new feature in java7?

18

Different DesignsDifferent Designs• Harmony design

o Based on OSGio OSGi jars o No language support for modularization o How to use is a big question

• JSR294/Jigsawo No compatible with OSGi o “simple module system”o simple view modal o single versiono key word: moduleo No “black hole”

19

Jigsaw worksJigsaw works

• At the cost of inside-java-modularizationo Language changeso Classfile changeso VM changeso Compiler/javadoco Module-file format

20

Language changesLanguage changes

• New key word: module

• Version support with annotation

21

Other changesOther changes• The module-info.class file

o A module compilation unit (module-info.java) is compiled to a ClassFile structure like any other compilation unit.

o Optimized for streamed reading, from beginning to end. Random access should not be required when reading a module file, though it can be required when writing.

o Content-specific compression: Pack200+gzip/lzma for classes, bzip2 for native code, etc.

o Independent of any specific installed-module format or target-filesystem capability.

• Introduce new APIso a module system whose core can be implemented directly within the Java virtual

machine, since otherwise the central classes of the JDK could not themselves be packaged into meaningful modules.

o java.lang.moduleo java.lang.reflect.module

22

Current statusCurrent status

• Working on module class file format

• Current version divided classlib intoo jdk.boot, jdk.base, jdk.awt, jdk.swing, and jdk.tools.

• Compared to Harmony:

23

NIO2 – advanced new NIO2 – advanced new io io

• NIO is far from completion

• NIO2 – new New IOo Add new network/io programming modelo Add complete file system support

24

Why use need new Why use need new model?model?

• Several modelso Blocking (java.net/io)o block on real operationso set a timeout before real operationso Non-blocking (nio)o try once, return immediatelyo select (nio)o select on sockets, operate afterwards

• Disadvantage of selecto Can operation only a limited file descriptorso 2048 by default, can re-compile kernelo or use multi-processo Slow when fds increased

• C/C++ can do, java cannot!

25

New model and APIsNew model and APIs• SIGIO

o No query, trigger by signal

• Asynchronization IO o Returns immediately, kernel help to make it doneo Can check if operation is doneo Zero-Copy

• New networking/io APIso Epoll (linux)o aio (POSIX)o Overlapped API (windows)o Pollset (AIX)

26

epoll on linux (v2.6)epoll on linux (v2.6)

• Advantage of epollo Much larger fds to be used ( > 100,000 on 1G mem machine)o High performance (kernel mmap)o No performance loss while fds increases

• All advantage due to its implementation• much better than poll/select on real apps

• file system?

27

Overlapped API on Overlapped API on windowswindows

• Windows kernel do the real operationo Read/write -> getCompletetionPort

• 2 ways to checko Signal when it’s done (callback)o user query

• Support both network and file

28

Posix aioPosix aio

• Follow standard aio model

• Support by various of systemso Linux (slower than epoll)o AIX (support only file system)o Zos (support only network system)

29

Pollset on AIXPollset on AIX

• AIX does not have aio on socket, nor epoll

• Use pollset_poll

• +10% throughput in benchmarks

30

Strategy of Strategy of implementationimplementation

• Use faster API?

• Customer requirement?

• More research required

31

File system : why need File system : why need

improvementimprovement

• io/nio lacks:o No symbolic links o No simple remove/copyo Directory search and iteratoro Bad filterso Limited support for file attribute access

32

New support for File New support for File systemsystem

• Various of features for different systemso Keep an eye on file system

• File iterator

• File Watch service

• File type detector

• aio

33

Garbage-First Garbage Garbage-First Garbage Collector Collector

• G1 is a “server-style” GC and has the following attributes. o Parallelism and Concurrency. o Generational. o Compaction. o Predictability.

• Best fito multi-coreso Big memory

34

Standard GCStandard GC

• Current normal GC

35

Modern GCModern GC

• Serial Collector • Parallel Collector • Parallel Compact Collector (ParallelOld) • *Concurent Mark-Sweep(CMS) Collector

36

New challengingNew challenging

• Multi-core/Multi-thread• VLH• Real time/soft real time• Clouding • …

37

G1 Design and G1 Design and advantageadvantage

• Generationo no physical separationo single contiguous heap which is split into same-sized regions.

• Pauseso Collection in G1 takes place through evacuation pauses, evacuation

pauses are done in parallel, with ALL available CPUs participating

• Compact

• Predictabilityo Compacted heap is easy to malloc

38

Q&AQ&A

• Http://harmony.apache.org

39