Mario BodemannAndroid and Java Evangelist@mariobodemann@contentful
Virtual Reality in Android
API driven CMS, emphasis on content creation and delivery.
Virtual Reality in AndroidUsing Cardboard
What is Cardboard?
● Easy to produce● Available from
different companies and in different shapes
● can have a button/lever/hole for interaction
● only needs a smart phone (Android/ios) and an App
How does it work?
● Use 3D data and movement to draw two images: left and right eye
● Distort images for lenses
● Combine image to scene in brain
Programming Cardboard
How do I program Cardboard?
https://developers.google.com/cardboard/android/
● SDKs available for ● Java
○ OpenGL○ Pure Android
● Unity Plugin○ Unity Engine (C# /JS)○ Platform independent○ No reliable linux development available
● Demo app○ Find a box and
press the button○ On github
● Library○ jar distributed
with demo repository
○ no maven repository
How To: Android Demo
https://github.com/googlesamples/cardboard-java
Setup of the demo
Activity
Model Repository
OpenGL
Interaction
● All code in one Activity○ OpenGL code ○ Model creation
● Hard to expand on
Expanding on the demo
● App using above code as base
● Expanding it to new use cases
● Having fun exploring
Contentful Cardboard Idea
https://github.com/contentful-labs/contentful-cardboardhttps://play.google.com/apps/testing/com.contentful.cardboard
MVPR
Presenter
Model Repository
OpenGL
Interaction
Interactor
Activity (View)
VR Renderer
● Split Activity into several parts
● Improve modularity● Improve testability● Improve readability● Improve line number
count ;)
Interactor
● Repository abstraction
● Uses registered Listener to inform Presenter
● Live time of the Presenter
public interface Interactor { interface Listener { void onModelReceived(ProtoModel model); void onError(Throwable throwable); }
void setListener(Listener listener); void requestModels();}
Presenter
● Coordinates Interactor and View
● Enriches data from Interactor
● Tells View to update its models
public class Presenter implements Interactor.Listener {
// … setup
private int received = 0;
@Override public void onModelReceived(ProtoModel model) { model.position[0] += received * 4.0f; received++; view.addProtoModel(model); }
View
● Connects Renderer and Presenter
● Takes models and enqueues them in the renderer
● Startup point for Android app
● Deals with interaction
public class MainActivity extends CardboardActivity implements CardboardView.StereoRenderer, View {
@Override public void onCreate(…) {
renderer = new CardboardRenderer( getApplicationContext()); presenter = new Presenter( new ContentfulInteractor(this)); presenter.bind(this); }
@Override public void onCardboardTrigger() { vibrator.vibrate(25); userInteracted(); }
View (cont.)
● Listens to Cardboard Library callbacks
● For setup of a frame● And rendering every
eye, redirecting it to the renderer
@Overridepublic void onSurfaceCreated(…) { renderer.surfaceCreated();}
@Overridepublic void onNewFrame( HeadTransform headTransform) { renderer.prepare();}
@Overridepublic void onDrawEye(Eye eye) { float[] perspective = eye.getPerspective(Z_NEAR, Z_FAR); float[] eyeView = eye.getEyeView(); renderer.render(perspective, eyeView);}
Renderer public class CardboardRenderer implements Renderer { @Override public void surfaceCreated() { addFloorAndCeiling(); }
@Override public void prepare() { addPendingModels(); glEnable(GL_DEPTH_TEST); glEnable(GL_BLEND); // … More gl* calls }
● Abstracts from OpenGL rendering
● Generates OpenGL GPU Models
● Listens to View
@Overridepublic void render(float[] perspective, float[] eyeView) { multiplyMM(view, 0, eyeView, 0, camera, 0);
for (final Model model : backgroundModels) { renderModel(perspective, model); }
for (final Model model : foregroundModels) { renderModel(perspective, model); }}
Renderer (cont.)
● Render all models added
● Starting with background objects, since foreground objects contain transparency
Advantage of new system
Sample Challenge: How to change a texture?
● Change file● Rerun tooling● Re releasing new version in store● Convince user to update
● Need technical person● Potentially through several iterations
Purposal:Using A CMS
● Change a asset online
● Restart app to resync data (done at startup?)
● User only needs network
● Can be done by non technical person
Using A CMS
● Everything except for the grid comes from CMS
● Changing it with drag and drop
● Changing VR content from outside
Future
● Unit tests?● Provide different Renderer VR/classic/engine code?● CMS for Shader?● A/B testing of in assets?● Your ideas?
Future
Recap
Recap
● Cardboard is relatively easy to program and use● Expanding from Google Demo is the key to success● Using a CMS for VR
Questions and Answers