ART: Augmented Reality Table for Interactive Trading Card Game Albert H.T. Lam, Kevin C. H. Chow, Edward H. H. Yau and Michael R. Lyu Department of Computer Science and Engineering The Chinese University of Hong Kong
Jan 15, 2016
ART: Augmented Reality Table for Interactive Trading Card Game
ART: Augmented Reality Table for Interactive Trading Card Game
Albert H.T. Lam, Kevin C. H. Chow,Edward H. H. Yau and Michael R. Lyu
Department of Computer Science and EngineeringThe Chinese University of Hong Kong
Presentation OutlinePresentation Outline
• Introduction
• Objective
• Augmented Reality Table
• System Design
• Implementation
• Demonstration
• Conclusion
Trading Card GameTrading Card Game
• Two players play a match.
• Players play cards to fight with each other. It includes summoning monsters, casting spells, or setting traps, etc.
• We choose “YU-GI-OH”as our implementation
Game MatGame Mat
• Game Mat of YU-GI-OH• Different kinds of card must be put inside
particular card zones
Traditional Card Game Computer Card Game Traditional Card Game Computer Card Game
• Traditional Card Game– Interaction with real people– Lack of sound and visual effect– Require players’ own imaginations
• Computer Card Game– 3D animations– Handle complicated rules– Interaction with computer
ObjectiveObjective
• To develop a generic Interactive Trading Card Game Interface which allows:
– Interaction with people and computer– Playing naturally in the original way– Visual and sound effect– Complicated calculation
Augmented Reality Table (ART)Augmented Reality Table (ART)
• ART is a prototype platform– Employ Augmented Reality technology– Users can play trading card games on it
– Provide an interactive environment and interface
– Realize the game with 3D graphics and sound effect
Hardware SetupHardware Setup
• Overhead camera– The only input device
• Plasma monitor– Act as a table– Display 3D-animation– Produce sound effect
• Computer– Recognize valid inputs– Maintain game rules– Generate outputs
System ArchitectureSystem Architecture
• Perception Module– Calibrate– Extract card and input
commands from the camera
• Database Module– Access card information– Access game and card
rules – Identify cards
• Game Enhancement Module– Display game mat– 3D animation– Sound effect
• GameCore Module– Control game flow– Store game state– Rule-based game
engine
System ArchitectureSystem Architecture
ART Card Game
ART Card Game CoreGeneric Card Game Database
Augmented Reality Perception
CommandDetector
Calibration
Card Detector Game Enhancement
Video Decoder
Card Locator
Calibration
Card Recognizer
3D Animation
Image Database
Input Analyzer
Sound Effect
Game CoreRule Database
DisplayCamera
Card Information
Input Information
Environment Information Output Generator
Game Manager
Rule ManagerCard Database
PerceptionPerception
• Read and process the raw video perceived from the camera
• Detect card inputs and commands
• Some assumptions– Fixed camera, fixed table– Camera is approximately right above the table– Cards can only be placed in predefined
regions called card zones
Card LocatorCard Locator
• Locate search window– The screen is partitioned into some fixed small search
windows as card zones identified in calibration– Select those with significant number of changed
pixels as candidates
– Search windows just stop changing will be processed– This minimizes both search space and frequency
Image difference Binary image difference
Card LocatorCard Locator
• Locate four corner card positions given a search window– Apply Canny Algorithm– Extract Contours– Search for contours which
• contain 4 points• contain nearly right
angle corners• have area within a
certain threshold
Card DetectorCard Detector• Detect the card input including:
– Location (in which card zone)– Orientation (vertical or horizontal)– Unique card ID (to query for card information)
• Compute an undistorted version of card image by geometric transformation followed by bilinear interpolation
Card DetectorCard Detector
• Identify card image by 2 steps:– Image retrieval
• Classify and retrieve cards of the same type as candidates
– Image matching• Use color-based matching since higher level features are lost
due to low resolution of query image (about 40 x 40 pixels)• Find the best match card according to pixel difference:
• Reject the image if the pixel difference is larger than the threshold
• Accept the image with minimum pixel difference• Improve the accuracy by using block matching
w
i
h
jDQDQP jiIjiIIID
1 1
2)),(),((),(
DatabaseDatabase
• The Database Module stores all game information• It provide necessary information for the other
three modules• Image Database
– Contain all card images
• Card Database– Contain all card
information
• Rule Database– Contain all game
rules and card rules
Rule-based Game EngineRule-based Game Engine
• The system is driven by rules• Define a rule:
– Premise-conclusion form– e.g.
“If in main phase and player put a card, then summon the monster.”
– The action may be to load another rule, remove a rule, reset a rule, update game states, or show some outputs, etc.
MainPhase PutCard SummonActionLIstActionLIstDisplay Monster
premise 1 premise 2 conclusionaction list
Main Phase Put Card Summon
Destroyer Summon Destroy Card
premise 1 premise 2 conclusion
Rule-based Game EngineRule-based Game Engine• Maintain a pool of rules which loads and stores
rules dynamically during the game
• Receive and raise appropriate events according to the game rules, thus, maintaining the game logic
• Mechanism: forward checking until no more conclusion is reached
Why Rule-based?Why Rule-based?
• Extensible– New cards can be extended by adding new rules and
new card information to the database
• Flexible– Card games are very flexible and some card effect
are unpredictable, of which brute-force programming cannot handle it
• Generic– New card games can be implemented by using a new
set of game rules
Game EnhancementGame Enhancement
• Provide game mat to place cards
• Provide buttons for command input
• Enhance visual and sound effects
• Display game states and hints
Game MatGame Mat
Game statesCard zones Buttons Battle field
Screenshot during BattleScreenshot during Battle
DemonstrationDemonstration
ConclusionConclusion
• We developed a prototype of the Augmented Reality Table for trading card games
• With AR techniques, the system enhances existing trading card games, while remaining much of original playing style
• The system provided an interactive interface for players and performs player’s effort to maintain game rules and calculations
Q&AQ&A
Improved Block Matching Algorithm
Improved Block Matching Algorithm
• Solution, use an Improved Block Matching Algorithm– Divide the image into 9 squares– Apply Block Matching Algorithm to each
squares