MAF Paolo Quadrani p.quadrani@cineca.itp.quadrani@cineca.it 20-21 giugno 2005 Marco Petrone m.petrone@cineca.itm.petrone@cineca.it.

MAFMAF

Paolo Quadrani p.quadrani@cineca.it

20-21 giugno 2005

Marco Petrone m.petrone@cineca.it

MAF

MAF Architecture

MAF

Software layers

• HAL: vertical applications, custom objects

• LAL: view, visual pipe, widgets, operations, logic

• MFL: VME, data pipe, vtk filters and all additional libraries used inside the framework

HALHAL

LALLAL

MFLMFL

MAF

MAF structure

LogicLogic

View Manager

View Manager

Operation Manager

Operation Manager

Vme Manager

Vme Manager

Interaction Manager

Interaction Manager

AppApp

ViewsViews OperationsOperations VmesVmes Devices + InteractorsDevices

+ Interactors

MAF

Framework communication

Msg send: OnEvent() Call

Command execution

Module separation: who send a message do not know the listener

LogicLogic

Operation Manager

Operation Manager

OpOp

MAF

What do the ‘App’ class?• Create the parent frame of

the application• Make an instance of the

logic• Plug the views and the

operations used inside the application

• Hide/Show the interface elements for the vertical application

• Terminate the application deleting the logic

MAF

Logic: the application’s behavior

• Create and instance of each manager: mafViewManager, mafOpManager, mafVmeManager, mafInteractionManager

• Create the menu’ • Create the toolbar• Create the sidebar• Create the timebar• Create the log area• Listen all the events raised from the managers and

from the interface elements• Call (virtual) methods to mange the events

MAF

mafViewManager

• Create/Delete plugged views

• Manage view selection

• Add/Remove VMEs to/from the views

• Has knowledge about the selected view and selected VME

MAF

mafOpManager

• Manage the operation execution

• Manage the undo stack

• Enable/Disable menu’ items and toolbar buttons according to the VME selected and to the operation execution

MAF

mafVmeManager

• Add/Remove a VME to/from the tree

• Manage the file save, open, new

• Manage the file history

• Set the time to the VME tree

MAF

mafInteractionManager

• Make a default instance of mouse device• Manage interaction devices through

DeviceManager• Make an instance of the Positional Event

Router (PER), used to interact dynamically with a device and the selected VME through VME’s behavior

• Make an instance of the Static Event Router (SER), used to interact statically with a device and a fixed VME

MAF

How do the view work?

• Copy: make an instance of itself, set the listener to the mafViewManager and call method Create()

• Create: create the render window and create the view scene graph giving to it the information of which VMEs are creatable

MAF

How do the view work? (2)

• VmeAdd/VmeRemove: add/remove a VME to the scene graph

• VmeCreatePipe/VmeDeletePipe: create/delete visualization pipeline for the VME that has to be shown/hidden (these are called by the mafSceneGraph on VmeShow call)

MAF

How do the operation work?• Copy: make a copy of itself• OpRun: first method called by the mafOpManager; create the

GUI for the operation• OnEvent: listen the messages sent from the GUI• OpStop: the operation is ending with wxOK or wxCANCEL,

notify it to the OpManager• OpDo: called if the operation is terminated with wxOK

– the op is pushed inside the undo stack (if the m_canundo flag is true)

• OpUndo: called only when UnDo is required– The op is popped out from the undo stack– All the objects must be alive to establish previous situation

MAF

VME

VMEVME

CurrentDataPose MatrixVoid *

ClientDataClientData

mafVmeData class, created by Logic

mflDataPipe: static vtk data or dynamically generated by a custom data pipe

MAF

VME 4D data structure

• 4D data, positioned in space with a hierarchical organization

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

VME 4D data structure

• The kind of data stored inside a node:

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

VME 4D data structure

• Output: 4x4 matrix and VTK dataset

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

VME 4D data structure

• VME consumers

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

VME 4D data structure

• VME output changes according to input time

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

VME 4D data structure

• Data manipulation by means of Operations

VME

ParentVME

Matrix Array

Data Array

Attributes

CurrentTime

MAFOperation

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

MAF

Data Storage

ROOT

R-FemurL-Femur

L-Tibia

Storage

<MSF Name="legs">

<VME Name="L-Femur">

...

<VME Name="L-Tibia">

...

</VME>

</VME>

<VME Name="L-Femur">

...

</VME>

</MSF>

VME Tree

XML Hierarchy

• The VME tree is stored inside a folder as a single XML file (MSF file) storing hierarchy, pose and metadata, plus single data files of node’s data (VTK Datasets).

• Hierarchy

• Matrix Array

• Data Array

• Attributes

MAF

VME Derived: Output Pipes

New VME

ParentVME

Attributes

VME GUI

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

CurrentTime

MatrixPIPE

DataPIPE

Matrix Array

Data ArrayMAF

Operation

• VMEs can be customized to generated different output data and matrix using custom pipes.

MAF

VME Derived: Custom GUI

New VME

ParentVME

Attributes

VME GUI

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

CurrentTime

MatrixPIPE

DataPIPE

Matrix Array

Data ArrayMAF

Operation

• VMEs GUI can also be customized to set specific parameters…

MAF

VME Derived: VME Links

• Links to other VME can be added to extend the tree into a Graph.

VME Derived

ParentVME

Attributes

MAFOperation

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

VME 2

VME 1

VME 3

Link

Link

Link

CurrentTime

MatrixPIPE

DataPIPE

MAF

VME Derived: VME Links (2)

• VME can listen to changes to linked VMES.

VME Derived

ParentVME

Attributes

MAFOperation

4x4 posematrix

3Ddataset

VisualPIPE

RenderWindow

MAF View

VME 2

VME 1

VME 3

Link

Link

Link

CurrentTime

MatrixPIPE

DataPIPE

MAF

VME Derived: VME Meter

• An example of VME Derived is the VME Meter: it’s a VME continuously computing the distance in between linked VMEs.

• VME Derived are sort of persistent operations which are saved in the data tree.

MAF

MAF Interaction and Events

• Interaction inside MAF is accomplished by processing input events coming from GUI or input devices

MAF

Discrete and Continuous

• Discrete Events: low frequency and typically causing application state change. Typically coming from GUI or discrete I/O device.

• Continuous Events: rised when user interacts inside views using a continuous input device, like a mouse or a tracker.

MAF

MAF Interaction Model

• The typical MAF interaction model is composed by an event source (the device), an interpreter (the interactor) and a logic module (operations, logic or others)

Interactor OperationDevice

MAF

Events from Device

• Events coming from input devices are synchronized with the main application thread by Device Manager:

Speech

Mouse

Tracker

Haptic De

vice

Man

age

r

S.E

.R.

Inter. Manager

Async Events Sync Events

MAF

Device Manager

• Responsible for devices:– Creation– Storing/Restoring of device setup– Input events synchronization

MAF

Device Setup & Config

MAF

Device Setup & Config

• Default devices• User adds newdevices

MAF

Device Setup & Config

• Selected devicesettings

• Settings can bestored and restored

MAF

Interaction with MAF

• Direct Interaction with devices:– Static Event Routing

• Interaction with objects in the scene:– Positional Event Routing

MAF

Static Event Routing

• Interaction with devices is called “Static Event Routing”, where devices are assigned directly to “actions” by user, and interactors are connected to “actions”by application:

StaticInteractor

Inter. Manager

Logic

Op Manager

Running Op

Static EventRouting

Tracker Action

S.E.R.

MAF

SER Example

Head TrackAction

S.E.R.

CameraInteractor

Current ViewTracking device assigned to changing the point of view in the scene.

MAF

Assign Device to Actions

• Static bindings ofthe deviceto the actions

• Actions are declaredby application modules

MAF

Actions declaration (for Ops)

• Define a GetActions() function in the operation returning the list of actions required by operation.

MAF

• Attach interactor to Action:

Event ID Action Name

Attach Interactors to Actions

Interactor

Interactor Action NameEvent ID

•Detach interactor from Action:

MAF

Positional Event Routing

• A specialized interactor (P.E.R.), check where user is pointing in the view and routes events toward the interactor connected to the pointed object.

S.E.R.

P.E.R.

View Manager

SelectedView

Inter. Manager

Logic

VME Manager Op Manager

PointedVME

BehaviorRunning

OpPositionalRouting

CameraInteractor

MAF

Positional Event Routing (2)

• P.E.R. is an interactor statically assigned to “PointAndManipulate” action.

• Picking in the scene occurs only in response to “button down” events, therefore to be used with the P.E.R. an input device must be able to rise a button event.

MAF

Attach Interactors to VME

• The interactor should be set in the m_behavior field of the VmeData object connected to the VME, which can be retrieved with GetClientData() function (MAF 1.x):

MAF

Create a Mouse Interactor

• Create a Compositor Mouse interactor, and then use it to create interactors to be assigned to single mouse buttons:

MouseCompositorInteractor

TranslateInteractorInput

Events

RotateInteractor

LeftButton

RightButton

Mouse

MAF

Create a Mouse Interactor (2)

• Header file (Op.h):

Left Button!

• Body file (.cpp)

MAF

Applications

MAF

Applications

MAF

Applications

MAF

Applications

MAF

Applications

MAF

Applications

MAF

Exercise

• Open the ExampleApp project with Visual Studio 6 and:

1. Add visibility for mflVMELandmark type into mafViewCustom.

2. Change the visual pipe used to render the surfaces in mafPipeCustom

MAF

Exercise

• Open the ExampleApp project and the mmoInteractonOp files and uncomment code to:

1. Declare a new Action2. Create a mouse interactor3. Attach mouse interactor to the action4. Attach mouse interactor to the VME

MAF

Ex: declare the new Action

• Uncomment code of GetAction() function inside the .cpp :

• also inside the .h file:

MAF

Ex: Create the Interactor

• Uncomment code to create interactor in .cpp file:

MAF

Ex: Destroy Interactor

• Inside OpStop() uncomment code to destroy interactor:

MAF

Ex: Assign Interactor to Action

• Uncomment code to send event to attach interactor inside OpRun() function:

• And inside OpStop() to detach it:

MAF

Ex: Attach Interactor to VME

• First, comment out lines assigning interactor to action (PASS 3)!

• Set the interactor in the m_behavior field of VME’s mafVmeData inside OpRun():

MAF

Ex: Attach Interactor to VME(2)

• Also uncomment code to detach interaction inside OpStop():