Augmented Reality with Flex, FlarToolkit and Papervision3D Roman Protsyk 20.12.2010
Jan 28, 2015
Augmented Reality with Flex,
FlarToolkit and Papervision3D
Roman Protsyk20.12.2010
Agenda
• Augmented reality definition and examples
• Augmented Reality in web browser with FlarToolkit
• Augmented reality demo creation using Flex, FlarToolkit and Papervision3D demo
What is Augmented Reality? Augmented Reality (AR) term referring to all the projects to complement the reality of any virtual
elements
The term was proposed by Tom Caudell, a researcher of Boeing Corporation,in 1990 . Ronald Azuma in 1997 defined the augmented reality as a system that 1. Combines the virtual and real. 2. Interact in real-time3. Works in 3D.
Nintendo – Wii
Augmented Reality Examples : Game Consoles
Sony PlayStation 3 – Controller + Eye Cam
Sony PlayStation 3 – Eye Pet
XBox – Project Natal
1. Mobilizy – Wikitude2. Nokia – Conceptwith Image Comparison
3. Whiteapp – AR iPhone apps
Augmented Reality examples: Location Based Mobile Apps
Augmented Reality examplesThere are AR applications outside of console
games, including military and consumer products, too. Night-vision goggles and targeting-assistance technology help marksmen in battle, and children's stories come to life with AR-enhanced books. The uses are vast.
Augmented reality in web browser
• The world’s first Flash based augmented reality library ported from NyARToolkit (Java ported version of ARToolKit) by Tomohiko Koyama aka Saqoosha.
• First version was released in May 2008.• Porting was completed in about 1 week.• FLAR is a part of Spark project -
Flash/ActionScript open source community. http://www.libspark.org/wiki/saqoosha/FLARToolKit/en
FlarToolkit examplehttp://ge.ecomagination.com/smartgrid/?c_id=Matter#/augmented_reality
FlarToolkit example• http://www.livingsasquatch.com/
Basic usage• FLARToolKit doesn’t render the 3D objects.• Calculates only the orientation of the marker.• Use other 3D engines to render 3D objects.
- Papervision3D
- Away3D (both 2.x and 3.x)
- Sandy
- Alternativa3D• Start-up guide for FLARToolKit.
http://saqoosha.net/en/flartoolkit/start-up-guide/
Demo
Let’s create demo application and look inside FLARToolkit in parallel
Creation of sample application
• Create marker image “pattern16.pat”:• http://flash.tarotaro.org/ar/MarkerGeneratorOnli
ne.swf
• Open Flex Builder• Create sample ActionScript “FlarDemo” project• Download FlarToolkit:
http://www.libspark.org/svn/as3/FLARToolKit/ • Download Papervision3d:
http://code.google.com/p/papervision3d/downloads/list
Inside FLARToolKit
Marker pattern specification: Needs to be square. Only 50% of center area is used in matching process. Marker pattern is 16 x 16 dots bitmap by default. Size of the pattern can be lager, but it requires more recognition time.
Creation of sample application
• Get “camera_para.dat” file which contains default properties for camera from: http://www.libspark.org/svn/as3/ARToolKitMarkerGenerator/trunk/
• Camera calibration - is the problem of obtaining internal and external camera parameters from the available photographs or video filmed with it
Inside FLARToolKit• Capture the image from webcam. (Step 1 of 7)
– Simply use Camera and Video classes.– then draw video instance onto the BitmapData.
Inside FLARToolKit• Binarize input image. (Step 2 of 7)
Inside FLARToolKit
• Labeling. (Step 3 of 7)
Inside FLARToolKit• Find squares. (Step 4 of 7)
Inside FLARToolKit
• Matching with patterns.
(Step 5 of 7)Extract the images from detected rectangles using homography transform.
Inside FLARToolKit• Render the 3D objects. (Step 6 of 7)
- FLARToolKit includes support classes which converts FLAR’s transform matrix to each 3D engine internal matrix classes.
Inside FLARToolKit• Calculate transform matrix. (Step 7 of 7)
- It’s calculated by square’s vertices of the detected marker.
Inside FLARToolkit
Demo applicationpublic function FlarDemo(){setupFlar();setupCamera();setupBitmapData();setupPV3D();addEventListener(Event.ENTER_FRAME, loop);}
private function setupFlar():void{fparams=new FLARParam();fparams.loadARParam(new params() as ByteArray);mpattern=new FLARCode(16, 16);mpattern.loadARPatt(new pattern());}
private function setupCamera():void{vid=new Video(640, 480);cam=Camera.getCamera();cam.setMode(640, 480, 30);vid.attachCamera(cam);addChild(vid);}
private function setupBitmapData():void{bmd = new BitmapData(640, 480);bmd.draw(vid);raster = new FLARRgbRaster_BitmapData(bmd);detector = new FLARSingleMarkerDetector(fparams,mpattern, 80);}
Demo applicationprivate function setupPV3D():void{scene = new Scene3D();camera = new FLARCamera3D(fparams);container = new FLARBaseNode();scene.addChild(container);
var pl:PointLight3D = new PointLight3D();pl.x = 1000;pl.y = 1000;pl.z = -1000;
var ml:MaterialsList = new MaterialsList({all: new FlatShadeMaterial(pl)});
var cube1:Cube = new Cube(ml,30,30,30);var cube2:Cube = new Cube(ml,30,30,30);cube2.z = 50;var cube3:Cube = new Cube(ml,30,30,30);cube3.z = 100;
container.addChild(cube1);container.addChild(cube2);container.addChild(cube3);
bre = new BasicRenderEngine();trans = new FLARTransMatResult();
vp = new Viewport3D();addChild(vp);}private function loop(e:Event):void{bmd.draw(vid);try{if(detector.detectMarkerLite(raster, 80) && detector.getConfidence()>0.5){vp.visible = true;detector.getTransformMatrix(trans);container.setTransformMatrix(trans);bre.renderScene(scene, camera, vp);}else{vp.visible = false}}catch(e:Error){}}
http://protsyk.org.ua/flartoolkitdemo/FlarDemo.zip
Demo applicationLoading of different Papervision3D objects
http://protsyk.org.ua/flartoolkitdemo/Flar3DEarth.zip
Future of FLARToolKit
Alchemy version of FLARToolKit
• Done by Ryo Iizuka (aka nyatla)
- 2 times faster than AS3 version.
- You can use more polygons to create 3D models.
- It runs on more slower PCs.
- Committed at alchemy branch.
http://www.libspark.org/svn/as3/FLARToolKit/branches/alchemy/
Links• FLARToolkit project web page by Saqoosha
http://www.libspark.org/wiki/saqoosha/FLARToolKit/en
• Google group: FLARToolKit-userz.
http://groups.google.com/group/flartoolkit-userz • FlarToolkit/Flash Augmented Reality Getting
Started by Mikko Haapoja
http://www.mikkoh.com/blog/2008/12/flartoolkitflash-augmented-realitygetting-started/