Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)

Steve Poole IBM

Fast Talking Java Meets Native Code

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.

WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.

ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S

CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.

IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS

Important Disclaimers

!Works at IBM’s Hursley Laboratory in the UK

Involved in IBM Java VM development since before Java was 1

Currently leading IBM’s OpenJDK technical engagement

Steve Poole

“Compact Off-Heap Structures in the Java Language”

“Packed Objects”

aka

What this talk is about

!This technology is being developed to help address important pressures on Java !This talk focuses on how to improve Java interop to Non Java applications

“Compact Off-Heap Structures in the Java Language”

You’ll learn

1. An overview of this technology 2. Why we need your input and support 3. How to get started with using this technology yourself

Part 1Native interoperability Why it’s so important

C

Interoperability - why is it important?

Java is built on talking native !

Operating Systems Processor Architectures File Systems User interfaces Network Interfaces !

Java keeps the WORA monsters away !

!

WORA - the components

Your Java application

JDK Java code

JVM JIT Class Library Native code

Your

JNI c

ode

native

2013 - Not so Java-centric

Your Java application

JDK Java code

JVM JITClass

Library Your

JNI c

ode

native

Your

Nat

ive

code

2013 - clusters and new processors

High speed memory channels (RDMA etc)

Specialised Processors (GPUs etc)

non Java Runtime

2013 - Can we survive with JNI 1.1?

Your Java application

JDK Java code

JVM JITClass

Library Your

JNI c

ode

native

Your

Nat

ive

code

High speed memory channels (RDMA etc)

Specialised Processors (GPUs etc)

Oth

er r

untim

e

?

??

• 1997 - JNI 1.1

• Designed for independence

• Version independence

• Platform Independence

• VM Independence

• Java at the centre of the world

• Not performance focused

2013 - we can’t survive with JNI 1.1

• What we need is

• To be able to share data between Java and other runtimes with minimal overhead

• To not require a Java centric data view

• and share data off-heap too!

• Can we do that?

• Time for a demo.

Packed Objects to the rescue

• Screensaver style

• Rectangles moved around the screen, bounce off the sides…

A simple demo

A simple demo

!15

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

! typedef struct { int red,green,blue; float vx,vy; SDL_Rect rec; float x,y; ! } RECT;

RGB colour elements

position adjustment vectors

SDL structure contains location , height and width

float version of location - makes it easier to handle different h/w speeds

A simple demo

!16

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

!17

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines

Mixed C and JNI. Java updates the location of each element

Mixed C with PackedObjects - Java updates the location of each element

More than one demo

Demo 1 - all C

!18

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

!19

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines 20 0

Mixed C and JNI. Java updates the location of each element

Mixed C with PackedObjects - Java updates the location of each element

Results

Demo 2- Mixed JNI and C

!20

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

!21

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines 20 0

Mixed C and JNI. Java updates the location of each element

Mixed C with PackedObjects - Java updates the location of each element

Results

!22

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines 20 0

Mixed C and JNI. Java updates the location of each element 11 200000

Mixed C with PackedObjects - Java updates the location of each element

Results

200 000 JNI calls per frame. Copying data, validating data everytime.

AND ensuring there is no chance of optimization by the JIT.

(~4MB per frame)

(~4MB per frame)

Demo 2- Details

!23

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

Demo 2- Details

!24

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!for every rectangle…

C function calls Java method via JNI passes in x,y, vx and vy (by copy) Java method updates the rectangles new position Java native calls native method passing in new x,y, vx and vy values (by copy) C native method returns Java method returns C function returns

Demo 3 - Packed Objects

!25

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

!26

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines 20 0

Mixed C and JNI. Java updates the location of each element 11 200000

Mixed C with PackedObjects - Java updates the location of each element

Results(~4MB per frame)

(~4MB per frame)

!27

Demo Frames per second number of JNI calls per frame

Standard - all C code calling the SDL graphics routines 20 0

Mixed C and JNI. Java updates the location of each element 11 200000

Mixed C with PackedObjects - Java updates the location of each element

18 1

Results(~4MB per frame)

(~4MB per frame)

(4 bytes per frame)

Demo 3 - Packed Objects

!28

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides

!

Initialise

Per frame

Update rectangle locations

Draw rectangles with SDL

terminate

Demo 3 - Packed Objects

!29

Draw 100 000 random sized rectangles using SDL !Move them around the screen and get them to change direction if they hit the sides !once per frame..

Java method updates each rectangle’s position directly !

for(Rectangle r: rectangles) { r.x=r.x+r.vx; r.y=r.y+r.vy; … }

Today Java only speaks Java

• Getting data into and out of Java has always required some form of marshaling or serialization process

!• Interaction with native data structures in memory is particularly problematic

• JNI is the slowest but safest – But you need good C / C++ knowledge

• Unsafe and NIO are faster but more challenging to use – They both have their own programming ‘model’

• If the Java side is in control of storage layouts it’s easier

• When mapping existing native structures it’s much, much, much more difficult

!• Ironically the JVM is completely aware of native data formats

Tomorrow, with Packed Objects..

! typedef struct { int red,green,blue; float vx,vy; SDL_Rect rec; float x,y; ! } RECT;

Java will let you reference data stored like this

C structure

Tomorrow, with Packed Objects..

! typedef struct { int red,green,blue; float vx,vy; SDL_Rect rec; float x,y; ! } RECT;

! public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; ! }

as if it looked like this

C structure Java structure

Tomorrow, with Packed Objects..

! typedef struct { int red,green,blue; float vx,vy; SDL_Rect rec; float x,y; ! } RECT;

! public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; ! }

C structure Java structure

without copying

!!!!!!!!!!!‘C’ runtime

!!!!!!!!!!!Java heap

Tomorrow, with Packed Objects..

! typedef struct { int red,green,blue; float vx,vy; SDL_Rect rec; float x,y; ! } RECT;

! public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; ! }

And off heap

Part 2Under the covers

http://www.flickr.com/photos/mwichary/

! public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; ! }

Let’s look at these classes from a JVM point of view..

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!Java heap

!37

int red int green int blue float vx float vy SLDRect ptr float x float y

short x short y int width int height

Java structure

! public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; ! }

public class SDLRect { short x,y int width,height } J

Under the covers

!!!!!!!!!!!Java heap

!38

int red int green int blue float vx float vy SLDRect ptr float x float y

short x short y int width int height

Questions !1 - How does a JVM ‘know’ the class of these objects? !!2 - What are the data format rules for these types? !!3 - When synchonizing on a object - where is the monitor stored?

!!!!!!!!!!!Java heap

!39

int red int green int blue float vx float vy SLDRect ptr float x float y

short x short y int width int height

Answers !1 - How does a JVM ‘know’ the class of these objects?

Additional internal ptr !2 - What are the data format rules for these types?

JVM impl specific (including ordering)

3 - When synchonizing on a object - where is the monitor stored?

Additional internal field

class ptr monitor

class ptr monitor

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

!!!!!!!!!!!Java heap

!40

int red int green int blue float vx float vy SLDRect ptr float x float y

short x short y int width int height

class ptr monitor

class ptr monitor

No space for the extra Java fields - what do we do?

No space for the extra Java fields - what do we do?

How to add the metadata

!!!!!!!!!!!Java heap

!41

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

!!!!!!!!!!!Java heap

!42

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

Rectangle r=PackedObject.newNativePackedObject(Rectangle.class,0x******);

!!!!!!!!!!!Java heap

!43

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

class ptr monitor data ptr

“r”

Constructor was not called!

!!!!!!!!!!!Java heap

!44

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

class ptr monitor data ptr

“r”

Rectangle r=PackedObject.newNativePackedObject(Rectangle.class,0x******);

SDLRect s=r.rect; !

!!!!!!!!!!!Java heap

!45

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

class ptr monitor data ptr

“r”

Rectangle r=PackedObject.newNativePackedObject(Rectangle.class,0x******);

SDLRect s=r.rect; !

class ptr monitor data ptr

“s”

!!!!!!!!!!!Java heap

!46

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

class ptr monitor data ptr

“r”

Rectangle r=PackedObject.newNativePackedObject(Rectangle.class,0x******);

SDLRect s=r.rect; SDLRect t=r.rect;

class ptr monitor data ptr

“s”

!!!!!!!!!!!Java heap

!47

public class Rectangle { ! int red,green,blue; float vx,xy; SDLRect rec; float x,y; }

public class SDLRect { short x,y int width,height }

Under the covers

!!!!!!!!!!!‘C’ runtime

[4] red [4] green [4] blue [4] vx [4] vy [2] rect.x [2] rect.y [4] rect.width [4] rect.height [4] x [4] y

class ptr monitor data ptr

“r”

Rectangle r=PackedObject.newNativePackedObject(Rectangle.class,0x******);

SDLRect s=r.rect; SDLRect t=r.rect;

class ptr monitor data ptr

“s”

class ptr monitor data ptr

“t”

Part 3Truth and Consequences

• Off heap Packed Objects are considered to already exist so constructors are not called.

• Since there is limited opportunity to store the vital metadata it is constructed when needed.

• The assumption is that creating new objects when accessing embedded data is confusing and dangerous.

• How does ‘==‘ work?

• What about synchronization?

• The idea of ‘identityless’ objects is being discussed.

• Something that PackedObjects has in common with ValueTypes

• Could you handle Objects of this style?

Design considerations

• This approach offers Java a way forward in the multi environment world

• Almost zero cost data interop will benefit Java when exploiting new processors and memory systems.

• There are additional use cases that ‘fall out’ from this design.

• Low cost r/w of files

• Instant application startup

• Basic unmanaged memory capabilities

• Under the covers heap usage benefits through better control for ‘locality of reference’

The way forward?

• IBM and Oracle have been discussing these sorts of concepts for some time

• We need your input. How important are these use cases to you? What’s missing, not needed? What do you hate?

• We are trying to keep clear of syntax discussions at this point - it’s too early and just distracting.

Is this the right or only way?

• Google for “IBM Java 8 Beta”

You too can use Packed Objects

The current prototype implementation surfaces a lot of the internals that will eventually just disappear

Hard hats must be worn…

This approach makes it easier for us to quickly revise the

design but means users of the prototype have to work harder.

!

AND !

it avoids syntax wars :-)picture courtesy http://www.flickr.com/people/karen_od/

“Compact Off-Heap Structures in the Java Language”

Wrap up -hopefully you now:

1. Have a basic understanding of this technology 2. Know why we need your input and support 3. Know how to get started with using this technology

Thank You