Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)

Virtual Dart – An Augmented Reality Game on Virtual Dart – An Augmented Reality Game on Mobile DeviceMobile DeviceSupervised by Prof. Michael R. LyuSupervised by Prof. Michael R. Lyu

LYU0604LYU0604 Lai Chung Sum (04523002)Lai Chung Sum (04523002) Siu Ho Tung (04559983)Siu Ho Tung (04559983)

2

OutlineOutline

• Background Information & Objective• Summary of Works Done in Semester 1• Works in Semester 2• Game Flow• Programming Tricks• Difficulties• Conclusion• Q & A

3

ObjectivesObjectives

• Demonstrate how a game can Demonstrate how a game can “recognize” its external environment for “recognize” its external environment for Augmented RealityAugmented Reality

• Develop a mobile game, Virtual Dart, Develop a mobile game, Virtual Dart, which illustrates the proposed which illustrates the proposed methodologymethodology

4

DefinitionDefinition

Augmented Reality is . . .Augmented Reality is . . .• a combination of real world and computer generated data• the use of video and which is augmented by addition of computer

graphics

5

Background InformationBackground Information

Programming

Sybmian C++

C#

Target Mobile Phone

Nokia N90 (S60 2rd Edition FP3)

Nokia N80 (S60 3rd Edition)

(Nokia 6600)

6

Summary of Works Done in Semester 1Summary of Works Done in Semester 1

Feature Selection

Feature Recognition

Motion Tracking on the featue

7

Summary of Works Done in Semester 1Summary of Works Done in Semester 1

Program CoreProgram Core1. Implementation of Feature

Selection Algorithm2. Implementation of Initial Feature

Recognition Algorithm3. Implementation of Enhanced

Feature Recognition Algorithm4. Modification of mVOTE Motion

Tracking Engine5. Implementation of mobile game,

Virtual Dart, in prototype form

Virtual DartVirtual Dart

Studied MaterialsStudied Materials 1. Existing mVOTE Engine & its

related papers2. Symbian C++ &

Nokia S60 3rd edition programming guides

3. Corner Detection Algorithms such as FAST Corner Detector and Harris Corner Detector

4. Block Matching technique

8

Problem Discovered in Semester 1Problem Discovered in Semester 1

Problem

Enhanced Feature Recognition Algorithm takes more than 30 seconds to recognize the saved features

Cause

1. Nokia N80 has only limited computational power

2. The algorithm maybe too general to recognize features

Solution

Design a new and specific algorithm (To be discussed later)

9

What does the program need to What does the program need to recognize?recognize?

LED Recognition

Bar Code Recognition

Common in indoor environment

Can emit light+

Goal: Use the simplest method to build the recognizer

10

How to locate LED?How to locate LED?

Filter out the surrounding environment

1. Locate a block which contains LED

2. If the block contains more than a threshold of non-black pixels

1. Filter for LED 2. Find LED

11

LED Recognizer – FilterLED Recognizer – Filter

Use Green / Red LED as target object

Goal: Separate the target object from its surrounding environment

12

Add constraint to other color channelsResult image of G > 230 and B < 204

LED Recognizer – FilterLED Recognizer – Filter

Set a lower bound for Green color channel of the pixel

13

LED Recognizer – Filter ExtensionLED Recognizer – Filter Extension

• Use similar technique to build a filter for Fluorescent Light, Light Bulb

14

LED Recognizer – Version 1LED Recognizer – Version 1

Check every Check every block in the block in the

screenscreen

Prevent block Prevent block overlappingoverlapping

15

Limitations of LED Recognizer – Version 1Limitations of LED Recognizer – Version 1

Problem

1.Slow

2.Target object at the corner of the block

3.Select more than one blocks for same object

16

LED Recognizer – Version 2LED Recognizer – Version 2

Search the Search the center region center region of the blockof the block

onlyonly

Distance Distance checkcheck

Stop after Stop after finding 2 finding 2 feature feature blocksblocks

17

Comparison of 2 versions of Comparison of 2 versions of LED RecognizerLED Recognizer

Version 1 Version 2

18

LED Recognizer – Final VersionLED Recognizer – Final Version

LED Recognizer – Version 2LED Recognizer – Version 2LED Recognizer – Version 2LED Recognizer – Version 2

RemoveRemoveDistanceDistance

CheckCheck

Remove Remove Block Block

Overlapping Overlapping CheckCheck

Stop After Stop After Finding Finding

First First FeatureFeature

Simplify by

Reduce the number of tracking point to 1Reduce the number of tracking point to 1Reduce the number of tracking point to 1Reduce the number of tracking point to 1

+

Final Version of LED RecognizerFinal Version of LED RecognizerFinal Version of LED RecognizerFinal Version of LED Recognizer

19

Motion Tracking by mVOTE EngineMotion Tracking by mVOTE Engine

Problem

The motion tracking of the LED would be lost very easily

Cause Solution

1. Detect if the motion tracking of the LED is lost or not

2. Relocate the LED if the motion tracking is lost

Areas around the LED are flat regions. mVOTE Engine cannot work well in flat region

20

Virtual Dart ModelingVirtual Dart Modeling

Dart Throwing Preview Scene

21

Virtual Dart ModelingVirtual Dart Modeling

X39 pixels0.279 Meter

4 pixels0.0286 Meter

1.6 + 0.279 = 1.879 m

1.6 - 0.0286 = 1.5714 m

1.6 m

22

Game FlowGame Flow

23

Programming TricksProgramming Tricks

A lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile Motion

Found a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John Carmack

Newton Newton Approximation Approximation

MethodMethod

24

Programming TricksProgramming Tricks

25

DifficultiesDifficulties

Build 3D model of dart images

Insufficient support from emulator

Colors in images captured change in different platforms

Unclear explanations from Nokia S60 documentations

“Active Object” in Symbian platform

26

ConclusionConclusion

• Successfully recognized Green / Red LED for Augmented Reality

• Developed a mobile game with Augmented Reality – Virtual Dart

27

Virtual DartVirtual Dart

EndEnd

28

LED Recognizer – FilterLED Recognizer – Filter

Before Filtering After Filtering

29

LED Recognizer – FilterLED Recognizer – Filter

Before Filtering After Filtering

30

LED Recognizer – FilterLED Recognizer – Filter

• Use similar technique to build filter for Red LED

• Set lower bound for Red color channel and upper bound for Blue color channel

31

Projectile Motion with Init. Velocity = 16m/s

0

0.5

1

1.5

2

2.5

3

3.5

0 2 4 6 8 10

X Displacement (in Meter)

Y D

ispl

acem

ent

(in M

eter

)

Angle = 0

Angle = 5

Angle = 10

Angle = 15

Angle = 20

Angle = 25

Projectile MotionProjectile Motion

tVxx ixii ,1

tm

VVBVV ixi

ixix

,2,1,

tVyy iyii ,1

tm

VVBtgVV iyi

iyiy

,2,1,

32

Programming TricksProgramming Tricks

float Cos(int aAngle){

switch(aAngle){

case(0): return 1.0; break;case(1): return 0.99984769516; break;case(2): return 0.99939082702; break;case(3): return 0.99862953475; break;...case(12): return 0.97814760073; break;case(13): return 0.97437006479; break;case(14): return 0.97029572628; break;case(15): return 0.96592582629; break;default: return 1.0; break;

}}