D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0.pdf

8/14/2019 D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0.pdf

1/28

This document presents the results of the experiments carried out at CAR Venue (SantCugat) in the framework of the 3D Acrobatics experiment. Three different runs of theexperiment were carried out between May 2103 and September 2013. In two of these runsthe experiment collaborated with other experiments within EXPERIMEDIA project, namelyCONFetti experiment and CAR experiment. The results collected in those runs are discussedand evaluated. Key conclusions extracted from the results related to the use of FMI in thiscontext are highlighted. This document has been prepared considering particularlyinformation contained in deliverables D2.1.1 (First EXPERIMEDIA Methodology), D4.7.1

(3D Acrobatics: experiment description and requirements), D2.2.1 (EXPERIMEDIABaseline Components), D4.2.1 (CAR Experiment Design and Plan) and D4.7.2. (3D

Acrobatics: Experiment Progress Report).

D4.7.3

Experiment Results and Recommendation

2013-10-29

Jos Manuel Jimnez (STT Ingeniera y Sistemas)

www.experimedia.eu


2/28

EXPERIMEDIA Dissemination Level: PU

Copyright STT Ingeniera y Sistemas and other members of the EXPERIMEDIA consortium 2013 1

Project acronym EXPERIMEDIA

Full title Experiments in live social and networked media experiences

Grant agreement number 287966

Funding scheme Large-scale Integrating Project (IP) Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation

(FIRE)

Project start date 2011-10-01

Project duration 36 months

Activity 4 Experimentation

Workpackage 4.7 EX7 3D Acrobatics

Deliverable lead organisation STT Ingeniera y Sistemas

Authors Jos Manuel Jimnez Bascones, (STT Ingeniera y Sistemas)

Reviewers Sandra Murg (JRS)

Version 1.0

Status Final

Dissemination level PU: Public

Due Date PM24 (2013-09-30)

Delivery Date 2013-10-29


3/28



Table of Contents

1. Executive Summary ............................................................................................................................ 3

2. Introduction ........................................................................................................................................ 4

3. Description of the Experiment ........................................................................................................ 5

3.1. General Description of the 3D Acrobatics Experiment ..................................................... 5

3.2. Overview of the Experiment Design ..................................................................................... 6

3.2.1. System architecture ............................................................................................................... 7

3.3. Components Developed within the 3D Acrobatics Experiment ...................................... 8

3.3.1. Motion capture ...................................................................................................................... 8

3.3.2. Generation of video contents ........................................................................................... 13

3.3.3. Synchronization with metadata ......................................................................................... 13

3.3.4. Visualization of motion capture data for training purposes ......................................... 14

3.3.5. Implementation of VRPN server ..................................................................................... 14

3.4. Equipment ................................................................................................................................ 15

3.5. Ethics and privacy ................................................................................................................... 16

4. Experiment execution ...................................................................................................................... 18

4.1. First experiment run ............................................................................................................... 18

4.2. Second experiment run .......................................................................................................... 19 4.3. Third experiment run ............................................................................................................. 21

5. Results ................................................................................................................................................ 24

5.1. Motion capture experience .................................................................................................... 24

5.2. Lessons learnt from FMI ....................................................................................................... 25

5.3. Dissemination .......................................................................................................................... 26

6. Conclusions ....................................................................................................................................... 27


4/28



1. Executive Summary

This document describes the results obtained in the execution of the 3D Acrobatics experimentduring the development period. A general overview of the whole experiment is reported

describing in detail the technical goals achieved. A detailed explanation of the activitiesperformed on each task within the experiment is also presented. The results obtained in theexecution of the experiment are outlined and the main conclusions collected in the different runsof the experiment are highlighted. The exploitation plan is included in the last section of thedocument.

The report is organized in several sections starting by this executive summary in Section 1. Section 2 includes a brief introduction to the 3D Acrobatics experiment and its relationship withFuture Media Internet products and services. Section 3 presents an overall description of theexperiment. In this section a detailed explanation of the different software modules implementedin this experiment is included. Ethics and privacy aspects of the experiment are also discussed inthis section. Section 4 reports the execution of practical runs of the experiment carried out atCAR Venue. Some of these runs we performed in collaboration with other experiments withinthe EXPERIMEDIA project while other runs were executed exclusively between CAR and STT.Section 5 discusses the results obtained in the different runs of the experiment paying specialattention to the outstanding elements considered in this development such as the use of motioncapture technologies in real training conditions with high performance athletes, the integration ofthe motion capture data with FMI products and services, the impact of these new services in thetarget community, etc. This section ends with a summary of the lessons learnt in the practicalimplementation of this experiment. Section 6 summarizes the conclusions obtained in thisexperiment. Section 0 presents a summary the next actions that STT will undertake in order tofurther develop this product; in addition, the exploitation plan for the resulting new product isoutlined.

This document completes the information included in documents D4.7.1 1 and D.4.7.22, whichare the previous deliverables corresponding to the 3D Acrobatics experiment.

1 D4.7.1 3D Acrobatics: experiment description and requirementshttps://svn.it-innovation.soton.ac.uk/svn/experimedia-docs/Official deliverables/D4.7.1 3D AcrobaticsExperiment Description and Requirements v1.0.pdf2 D4.7.2 3D Acrobatics: experiment progress reporthttps://svn.it-innovation.soton.ac.uk/svn/experimedia-docs/Official deliverables/D4.7.2 3D AcrobaticsExperiment Progress Report v1.0.pdf


5/28



2. Introduction

EXPERIMEDIA is a collaborative project aiming to accelerate research, development andexploitation of innovative Future Media Internet (FMI) products and services through test beds

that support experimentation in the real world which explore new forms of social interaction andexperience in online and real world communities. Within this framework 3D Acrobaticsexperiment focuses on high quality content production for assessment and improvement ingymnastic exercises by the use of motion capture technologies. The goal of this experiment is torecord training sessions of gymnastics at CAR Venue and automatically generate assessment datafor helping the athletes improve their performance. These 3D motion capture data is also usedto compute metadata which is synchronized and saved with the athletes motion in order toprovide a valuable 3D graphics and augmented reality experience. The experiment makes use ofthe connectivity and storage facilities available at CAR Venue.

3D Acrobatics integrates 3D graphics as well as augmented reality and synchronization withexternal video. The experiment carried out research on synchronization of 3D motion capture,

video and metadata. The experiment makes emphasis on the quick delivery of data to theathletes and trainers. To this end it makes use of mobile devices (such as tablets, laptops, etc.) inorder to collect data from the inertial sensors (as well as from other motion capture devices andtechnologies) and provides feedback to the users in real-time on almost any platform(computers, tablets, laptops, smartphones, etc.). The information gathered by the motion capturedevices is transferred to a local computer, laptop or tablet. This device stores locally the motioncapture data and connects through a Wi-Fi connection to the server. A data manager (softwaremodule) manages this information in the cloud making it available to the community. In this waythe athlete has the possibility of sharing her/his data with trainers, colleagues and mates whomight be geographically distributed, thus enlarging her/his experience in training and gymnastics.

Recording sessions have been run at CAR Venue in order to evaluate the developments done within the 3D Acrobatics experiment. Figure 1 shows examples of the training sessions recordedat CAR during the different runs of the experiment.

Figure 1. Experiment runs at CAR.


6/28



3. Description of the Experiment

3.1. General Description of the 3D Acrobatics Experiment The 3D Acrobatics experiment focuses on high quality content production for the gymnastictraining sessions including 3D motion capture based on inertial sensors and 3D biomechanicalanalyses.

Figure 2. Inertial sensor STT-IBS.

The use of inertial sensors for motion capture in training sessions for gymnastics and othersports can be an important improvement for the assessment and training of athletes. Thanks to

its reduced size (see Figure 2 ) inertial sensors can be easily attached to the athletes body withoutcompromising her/his mobility in anyway.

Figure 3. Inertial sensors attached to the athlete's body.

Another important characteristic of these sensors is that each one includes its own Bluetoothantenna which allows it to connect directly to the device hosting the antenna. This featurefacilitates dramatically the task of fixing the sensors on the athletes body since no wires arerequired at any moment (see Figure 3).


7/28



3D Acrobatics carried out research on synchronization of motion capture data gathered from theinertial sensors, video obtained from the cameras available at the CAR Venue and metadata. Theexperiment aimed at focussing on the use of all those elements in the training process and on theimprovement of the athlete's technique.

In the event that the training session was recorded using stereoscopic (3D) HD cameras thesecontents is also synchronized with the motion capture data. To this end a VRPN 3,4 (VirtualReality Peripheral Network) server has been implemented allowing the communication betweenthe software controlling the stereoscopic cameras and the motion capture data. This feature hasbeen successfully tested in collaboration with the EXPERIMEDIA experiment CONFetti.

As the experiment runs took place in the CAR Venue the interaction between athletes, trainersand other professionals involved in the preparation of the athletes was very effective during thepractical execution of the experiment.

Mobile devices such as tablets or laptops were used to collect the data from the inertial sensors.In addition laptop computers were used to collect motion capture data using optical systems.

These alternatives are discussed in detail in Section 4. Motion capture data was displayed onthose devices in real-time providing in this way instantaneous feedback to trainers; athletesbenefited as well of this display since they got feedback immediately after the exercise wasperformed. Examples of the use of the motion capture devices are reported in Section 4.

3D motion data was collected by a computer connected to motion capture devices. Thiscomputer included a data manager which allowed uploading the motion capture information to arepository in the server. This data was retrieved and analysed in order to generate metadata thatcould be used for improving the technique of the athletes by the use of advanced 3D graphics.

Finally the 3D motion capture data and metadata were stored in the cloud. The data managermodule allows the athlete to access her/his data and share it with trainers, colleagues and friends

who might be geographically distributed. In this way the athlete is able to share the experience with a large and distributed community. The athlete can use this way to get advice from othertrainers as well as remarks from other athletes.

3.2. Overview of the Experiment Design

The implementation of the 3D Acrobatic Sports experiment required the integration of a motioncapture system (developed by STT) with the SOA 5 (Service-Oriented Architecture) implementedby ATOS to manage the data recorded at CAR Venue. In addition, 3D video and motion capturedata have been synchronized using a VRPN server; this task has been carried out incollaboration with CONFetti experiment.

3 Russell M. Taylor II, Thomas C. Hudson, Adam Seeger, Hans Weber, Jeffrey Juliano, Aron T. Helser, "VRPN: ADevice-Independent, Network-Transparent VR Peripheral System", VRST 01, November 15 -17, 2001, Banff,

Alberta, Canada. Copyright 2001 ACM 1-58113-427-4/01/00114 VRPN: Virtual Reality Peripheral Network, http://www.cs.unc.edu/Research/vrpn/ 5 David Sprott and Lawrence Wilkes, "Understanding Service-Oriented Architecture", CBDI Forum, January 2004.
http://www.cs.unc.edu/Research/vrpn/http://www.cs.unc.edu/Research/vrpn/http://www.cs.unc.edu/Research/vrpn/http://www.cs.unc.edu/Research/vrpn/


8/28



As indicated in the previous paragraph, 3D Acrobatics experiment also interacts with thesoftware developments made by ATOS in the CAR experiment. The same SOA is used tomanage the data which is collected in the motion capture sessions as well as the metadatagenerated when the motion capture sessions are reviewed. Therefore an effective collaboration

with the CAR experiment has also been carried out.

3.2.1. System architectureFigure 4 depicts graphically the system architecture implemented in the 3D Acrobaticsexperiment.

Figure 4. System architecture.

The functional building blocks of this architecture are the following:

Inertial (or other technology) motion capture system Generation of video contents Synchronization of motion capture data with video and metadata Visualization of motion capture data for training purposes VRPN server

The actual implementations and/or modifications of these building blocks are described in detailin Section 3.3. From the AVCC component the implemented by ATOS the SOA functions havebeen used in the 3D Acrobatics experiment. For the generation of metadata and synchronizationa new software application has been developed by STT.


9/28


10/28



athlete being analysed. If inertial sensors are used the raw data is collected using the triaxialsensors integrated on the sensor's board. These sensors are a triaxial magnetometer, a triaxialaccelerometer and a triaxial gyroscope. These triaxial sensors provide a set of nine degrees offreedom (raw data) which are converted into three-dimensional rotations by using a Kalmanfilter.If optical technology is used the raw data collected from the cameras is composed of a set of2D coordinates which represent the projection of the optical markers on the camera sensor.By using the camera calibration data (extrinsic and intrinsic parameters) the set of 2Dcoordinates are transformed into a set of 3D points. These are the coordinates of themarkers in the 3D space. As long as either the inertial sensors or the optical markers arelocated in predefined anatomical positions of the athlete the raw data can be used tocompute biomechanical parameters by using the corresponding biomechanical models.

2) Biomechanical models . The second function of the motion capture application is thebiomechanical module. This software component is in charge of analysing the data collected

from either the inertial sensors or the optical markers and translating it into a coherentkinematic skeleton of the human body. This kinematic model is in turn used to compute thebiomechanical parameters relevant for the type of analysis that the trainer or athlete wants toperform.

3) Motion analysis . The third and last function of the motion capture application is a softwaremodule which automatically saves the motion data file and generates a report for thecaptured motion. This module generates a 3D animation of the captured motion using askeleton-like avatar. In addition this module can show the computed biomechanical data inthe form of graphs and reports.

The 3D Acrobatics experiment required the implementation of particular biomechanical modelsin the inertial motion capture solutions developed by STT. In particular protocols for pommelhorse analysis and jump analysis have been implemented in this application. The details of thebiomechanical models implemented in this software application are fully described in thedeliverable D4.7.2.

The detailed analysis of the results collected in the experiment runs carried out at CAR Venue inMay 2013 and July 2013 revealed some limitations on the use of inertial motion capturetechnologies in pommel horse analysis. For this reason it was decided to include in theexperiment the use of optical motion capture technologies in order to overcome the identifiedlimitations. These limitations are described in detail in Section 4.

This decision imposed the need to implement in the software application for optical motioncapture modifications similar to those implemented in the inertial motion capture application inthe first half of the experiment. Therefore the optical motion capture system was modified inorder to implement protocols which allowed data collection, included biomechanical models andperformed motion analysis of the gymnastic exercises (pommel horse and jumps) using opticalmarkers.

In addition, cycling analysis was also considered in order to allow a comparative analysis with thecycling experiment driven by CERTH at CAR Venue.


11/28



Data collection

For the motion capture approach based on the use of inertial sensors one the maindevelopments made within this project has been the implementation of the Kalman filter in theprocessor built-in the printed board circuit (PCB). The advantages derived from thismodification are the optimization on the use of the bandwidth for data transmission as well asthe possibility to increase the sample rate of the sensor up to 250 Hz.

A secondary action developed within this task is the development of a library which allows theconnection of the inertial sensor with a Bluetooth antenna running on Linux or Androidoperating systems. This library is currently under testing. This task is encompassed in the futureactions and developments being undertaken by STT after completion of the 3D Acrobaticsexperiment.

Finally the calibration of the internal sensors in the inertial sensor has also been improved inorder to take into account the exigent conditions imposed by the motions in the exercisesexecuted in pommel horse and other gymnastic specialities.

For the motion capture approach based on the use of optical markers the data collection modulehas been modified in order to include a marker set which allows capturing optical raw data forgymnastic motions such as pommel horse exercises or jump analysis.

Analogously a marker set for cycling analysis has also been implemented. The aim of this markerset was to perform comparative analysis with the Kinect experiment driven by CERTH at CAR

Venue.

Biomechanical models

The goal of this section was the implementation of a set of biomechanical models which providethe data required for the assessment of the training process in acrobatic sports. The focus of thistask was the analysis of pommel horse exercises as defined in deliverable D4.7.1. The datacollected by the biomechanical models describe in an objective and quantifiable manner theperformance of the athlete carrying out some exercises. After analysis and review of these datathe trainers are able to propose and recommend to the athlete the best practices that will lead toa significant improvement in the execution of the exercises. In addition the athletes are able to

visualize these objective and quantifiable data by using 3D graphics which helps themdramatically to understand the explanations and recommendations made by the trainers.

Within 3D Acrobatics experiment two solutions have been implemented for quantitative analysisof the athlete's motion. On one hand a solution based on the use of the inertial sensors wasproposed. The biomechanical parameters used to evaluate the athlete's performance arecomputed from the measurements collected by the inertial sensors taking into account thetopological information provided by the biomechanical model. The inertial sensors used on eachbiomechanical model must be located on the right anatomical positions of the athlete's body inorder to allow computing of the required data. Inertial sensors are light and comfortable to wearby the athlete. They do not impose any limitation to the motions of the individual beinganalysed.


12/28



Figure 5. Relative angles measured in anatomical joints using two inertial sensors.

In biomechanics, the magnitudes of interest are very often defined as relative angles measured inthe anatomical joins. In order to compute those relative angles, the absolute rotations of twoinertial sensors placed at the limbs joined by the anatomical articulation are used. From theabsolute rotations of the two inertial sensors the relative orientation between them can be easilycomputed. This relative orientation allows computing relative angles at the joint such as

internal/external rotation, flexion/extension and abduction/adduction (see Figure 5).

In the scope of the 3D Acrobatics experiment the main gymnastic exercise selected for theexperiment validation was pommel horse. The pommel horse is one of the six pieces ofequipment in men's artistic gymnastic competitions. A contemporary pommel horse exercise ischaracterized by different types of circles with the legs apart or joined in a variety of supportpositions on all parts of the horse. The performance, as assessed by the judges who judge theexercise and determine the sum of the technical and positional execution errors, depends on thegymnast's ability to perform the circle with maximum amplitude. For angles two main variablesare identified: the angle formed by the truck and arms and the angle formed by the trunk and

legs (see Figure 6).

In addition to pommel horse a protocol for jump analysis was implemented.

Figure 6. Two positions during pommel horse exercise.

The analysis of the specialized literature led to the definition of a set of protocols for the analysisof pommel horse exercises using inertial sensors. These protocols are described in detail indeliverable D4.7.2.

On the other hand a second solution for quantitative motion analysis was proposed based on the

use of optical motion capture technology. The need of use of optical motion capture requiredthe implementation of protocols similar to those described in previous paragraphs using optical


13/28



markers. Optical motion capture technologies differ from the technology based on inertialsensors in the type of data captured by the motion capture devices. While inertial sensorsprovide 3D rotations of the body segments they are attached to, optical motion capturetechnology gives the 3D coordinates of the optical markers. These are small light spherescovered by reflective tape. The basic principle behind optical motion capture systems is the factthat the markers reflect the infrared light emitted by the LEDs located around the camera lenses.

This reflected light is seen by the motion capture cameras which transmit to the computer the2D coordinates of the reflections in the reference frame of each image sensor. Using thecalibration parameters these 2D coordinates are used to compute the 3D coordinates of themarkers in the global (calibration) reference frame.

In order to compute the position and orientation of a body segment in the space it is requiredthat the segment has at least three optical markers attached to it. In the sought of keeping thetotal number of optical markers moderate for a given protocol the optical markers are usually

located at the body joints. In this way a given optical marker is shared between two contiguoussegments and its coordinates are used to compute the positions and orientations of the twosegments linked together by the anatomical joint. Figure 7 depicts an example of an opticalmarker set for cycling analysis.

Figure 7. Optical marker set for cycling analysis.

Optical markers are very light and small. The use of optical markers in practical training sessions

does not impose any restriction to the athlete who can perform her/his exercises without anyproblem. Wearing the markers on the body is not uncomfortable.

During the development of the 3D Acrobatics experiment adequate protocols have beendeveloped in order to allow using optical motion capture technology during practical trainingsessions.

Motion analysis

The module of motion analysis has been modified according to the characteristics of the

biomechanical models implemented in the previous section. This module allows representing themotion of the body segments captured by the described models using a skeleton-like avatar. In


14/28



addition the biomechanical parameters computed by the implemented models are shown in formof 2D plots in a separate window.

These modifications have been implemented in both the inertial and the optical motion capturesolutions.

3.3.2. Generation of video contents The motion capture applications (both inertial and optical) include a 3D render engine which isused to visualise the motion executed by the athlete using a skeleton-like avatar. A new function

was implemented in this 3D graphics engine aimed at generating a video sequence for theanimated avatar.

This function allows the use of the 3D engine to select a point of view in the 3D window and togenerate a video sequence of the selected time interval of the exercise being analysed. This videois saved in a file in the hard drive in H.264 format.

The file containing the video generated can be uploaded to the repository using the data managerapplication developed for this purpose using the SOA implemented by ATOS. Therefore therepository can contain video files as well as the motion capture files. All these files can beretrieved for further analysis either by the athlete or the trainer.

3.3.3. Synchronization with metadata The main goal of this task was the implementation of a software module which allows thesynchronization of the metadata with the motion capture data and video (generated or recorded).

Originally, it was foreseen that this software was going to use the software tools developed by ATOS for the CAR experiment. These tools included both the SOA for handling files and agraphic user interface with functions aimed at creating annotations (metadata) in the videoframes. The software module for synchronization implemented by STT makes use of the SOAbut implements a set of functions for video annotation totally implemented by STT. Figure 8depicts a view of the user interface of the application which allows defining metadata on videogenerated by the motion capture applications.

In order to synchronize motion capture data and video (either recorded by GigE cameras ofstereoscopic 3D HD cameras) a trigger function was implemented in both the optical and the

inertial motion capture applications. This trigger function was implemented to support twodifferent operation modes. On one hand the motion capture applications are able to send atrigger signal when the motion capture process is started. This signal can be captured by thesoftware running the recording process with the cameras and at this moment the cameras canstart recording video. On the other hand the motion capture applications have been modified soas to accept a trigger signal launched by another component. This signal is captured by themotion capture applications and the motion capture process is launched when the signal isreceived.


15/28



Figure 8. Application for synchronization of video and metadata.

The implemented functions can be used by ATOS for the purpose of synchronizing videorecorded by the GigE cameras available at CAR and motion capture data recorded using theinertial sensors. Analogously PSNC can make use of this function within the CONFettiexperiment for a similar purpose with the stereoscopic video recording.

3.3.4. Visualization of motion capture data for training purposes The motion analysis function of the motion capture applications are used as visualization toolsfor the purpose of visualization of motion capture data previously recorded. The data manager

software allows downloading the motion files from the repository to the local machine runningthe motion capture data. The motion files can be open and reviewed using these applications.

In addition the data manager software also allows downloading the raw videos and/or theannotated videos for review.

3.3.5. Implementation of VRPN serverIn the 2012 open call of the EXPERIMEDIA project two experiments were approved to bedeveloped in the CAR Venue: CONFetti conducted by PSNC and 3D Acrobatic Sportdeveloped by STT. In the kick-off meeting of these experiments it was seen that there were

some areas of joint interest for both experiments. Therefore it was agreed to establishcollaboration between those experiments in those activities where there is a given degree ofoverlapping.

It was identified an area of collaboration consisting in the combination of the stereoscopic videorecorded by the PSNC cameras with the three-dimensional motion capture data captured by theinertial sensors. The data obtained with the motion capture system can be shared with thesoftware components developed within the CONFetti experiment and the combined result canbe displayed on the computer screen.

The practical implementation of this collaboration required sharing the data collected by themotion capture applications as well as the three-dimensional model representing the different


16/28



elements of the skeleton. In order to implement this collaboration it was agreed upon thedevelopment of a VRPN server.

The Virtual-Reality Peripheral Network (VRPN) is a set of classes within a library and a set ofservers that are designed to implement a network-transparent interface between applicationprograms and the set of physical devices (tracker, etc.) used in a virtual-reality (VR) system. Theidea is to have a computer or other host at each virtual reality station that controls theperipherals (tracker, button device, haptic device, analogue inputs, sound, motion capturesystem, etc.). VRPN provides connections between the application and all of the devices usingthe appropriate class-of-service for each type of device sharing this link. The application remainsunaware of the network topology. Note that it is possible to use VRPN with devices that aredirectly connected to the machine that the application is running on, either using separatecontrol programs or running all as a single program.

VRPN also provides an abstraction layer that makes all devices of the same base class look thesame. This merely means that all trackers produce the same types of reports. At the same time, itis possible for an application that requires access to specialized features of a certain trackingdevice (for example, telling a certain type of tracker how often to generate reports), to derive aclass that communicates with this type of tracker. If this specialized class were used with atracker that did not understand how to set its update rate, the specialized commands would beignored by that tracker. Each of these abstracts a set of semantics for a certain type of device.

There are one or more servers for each type of device, and a client-side class to read values fromthe device and control its operation.

Within the 3D Acrobatics experiment the implementation of the VRPN server for both theinertial and the optical motion capture applications was accomplished. This software was madeavailable to PSC for integration in CONFetti experiment. The 3D models corresponding to theskeletons were delivered to PSNC.

3.4. EquipmentDuring the experiment runs carried out at CAR Venue the following equipment was used.

Inertial motion capture system

o Set of 6 inertial sensors running at 250 Hz.

o Bluetooth antennao Tableto Laptop


17/28



Figure 9. Motion capture system with inertial sensors.

Optical motion capture systemo Set of 6 infrared cameras with resolution 832x832 pixels running at 250 Hzo Set of 6 tripods and 6 camera supportso Set of camera cableso Ethernet switch PoEo Laptop

Figure 10. Motion capture system with inertial sensors.

3.5. Ethics and privacy The 3D Acrobatics experiment recorded actual motion data from real athletes in the CAR Venue. Although the data recorded by the motion capture systems do not include any kind ofpicture or video it can be understood that a given person can be identified by her/his motioncharacteristics. This situation was discussed during the EXPERIMEDIA General Assembly heldin Madrid in January 2013. It was agreed that the current Data Protection Act (DPA) used at

CAR was valid for the development of the 3D Acrobatics experiment.


18/28



After the meeting in Madrid a questionnaire was distributed by the partner responsible forPrivacy Impact Assessment in the EXPERIMEDIA project including some questions about thePersonal Data Flows regarding the experiments at CAR. This questionnaire was filled andsubmitted by CAR as well as the experimenters. As a result the partner responsible for PIAproposed an action plan composed of several points. The most relevant suggestion related to 3D

Acrobatics experiment was the proposal to sign a controller-processor contract between CARand STT.


19/28



4. Experiment execution

Three different runs of the experiments were performed at CAR Venue in the following dates:

First experiment run: May 15-16, 2013, Second experiment run: July 24th, 2013 and Third experiment run: September 16-18, 2013

4.1. First experiment run The first experiment run made use of the protocols implemented for the pommel horse.Previous tests were carried out at the biomechanical laboratory available in CAR premises. Theresults obtained in these early tests were satisfactory.

After the previous tests, the experiment run with athletes was carried out in the main gymnasiumavailable at CAR Venue premises. The run was executed in parallel with CONFetti experiment.

The execution of this run revealed two different problems with the inertial motion capturesystem. On one hand the inertial sensors experienced problems to be connected to the hostcomputer using Bluetooth protocol; on the other hand, the relative angles measured at the hipsof the athlete showed a lack of accuracy.

In order to identify the source of the connectivity problems new experiments were run a fewdays after by CAR staff. In these tests no electronic equipment different from the inertial sensors

was present in the gymnasium. The output of these experiments revealed no problems with thesensor connectivity to the host computer if no additional electronic equipment was present inthe gymnasium. The conclusion was that no corrective action should have to be undertaken.

Figure 11. Training session in pommel horse; motion capture data and stereoscopic video recordedsimultaneously.

The lack of accuracy was examined in detail. It was concluded that the Kalman filter requiredmore precise data from the triaxial gyroscope. The corrective action proposed was theimprovement of the calibration procedure applied to the gyroscopes. This action required the re-


20/28



calibration of the available sensors as well as a change in the firmware of the device. This changein the firmware required in turn a modification in the inertial motion capture software.

Despite the two problems identified during this run of the experiment valid motion capture data were recorded during the training session. This data was recorded simultaneously with thestereoscopic video of the CONFetti experiment. As a result some augmented reality videos werecreated combining the images recorded by the stereoscopic cameras and the motion capture datastreamed by the VRPN server. Figure 11 shows an athlete performing his exercises in thepommel horse and wearing a set of four inertial sensors. The purpose of these sensors was themeasurement of the relative angles in the hip joints and lumbar spine. Figure 11 depicts a frameof an augmented reality content generated with the stereoscopic cameras by PSCN and themotion capture data collected by STT's sensors. The clip with the augmented reality contents

was generated by PSCN combining the images recorded with the 3D cameras and the 3D modelstreamed using the VRPN server.

The athlete participating in this run did not find any problem while wearing the inertial sensors. These sensors did not generate any problem in performing his exercises. He expressed that he was feeling comfortable wearing the sensors.

4.2. Second experiment run The second run of the experiment was conducted by STT technicians and CAR technical staff. This experiment was carried out using a new set of inertial sensors which incorporated theimproved firmware for gyroscope calibration. All the tests in this run were carried out in thegymnasium available at CAR premises.

The experiment was performed in three steps. The first set of tests was conducted with thepurpose of verifying the connectivity between the inertial sensors and the host computer.Different connectivity tests were carried out connecting the sensors and transmitting data fromdifferent places of the gymnasium. No connectivity problems were detected even if the sensors

were placed at up to 40 meters from the host computer.

Figure 12. Position of the inertial sensors for jump analysis.

The second set of tests focussed on the measurement of knee flexion motions and jumps. Forthis purpose two different athletes were appointed. Four inertial sensors were used placed infemur and tibia in both right and left legs simultaneously (see Figure 12). Different tests wereperformed with the two athletes. The results obtained in all the tests were very precise and


21/28



provided the expected feedback. Athletes were satisfied with the results and expressed noobjection in the use of the inertial sensors. Figure 13 shows the results obtained in one of theseexperiments; on the left side a skeleton showing the bones in the legs are depicted; on the right agraph with the flexion/extension angles in the knees is shown.

Figure 13. Results of jump analysis test.

Figure 14. Results of flexion/extension analysis test.

The third and final set of tests was performed on the same two athletes on the pommel horse. Inthis case the motion capture data obtained from the inertial sensors was not accurate. Noconnectivity problems were experienced but the accuracy of the relative angles in the hip joins

was poor once the exercise started. The environment was examined and any potential source ofmagnetic distortion was discarded.

As a consequence of these results a detailed revision of the theory, algorithms andimplementation of the firmware and software in the inertial sensors was accomplished. Differentexperiments were designed in order to identify the source of poor accuracy in the motion


22/28



performed by an athlete in the pommel horse. The results of this research revealed limitations inthe implementation of the Kalman filter which computes the orientation of the inertial sensor. It

was concluded that this control algorithm fails if the motion to be measured contains acomponent of permanent rotation around a given axis. In order to validate this conclusion all theexperiments were performed with inertial sensors manufactured by different vendors. In bothcases similar results were obtained.

Figure 15. Accuracy of relative angles measured with inertial sensors by different manufacturers.

Figure 15 shows the results obtained in one experiment performed with inertial sensors bydifferent manufacturers. In this experiment three inertial sensors were located on a rigid body

which rotated around the vertical axis with an angular speed of 360/s. This is approximately thesame angular speed of an athlete working on the pommel horse. The graphs in Figure 15 depictthe relative angle between two consecutive sensors in the rigid body. According to the conditionsof the experiment this relative angle must be kept constant. However it can be noticed adifferent of about 30 in the left graph (which corresponds to STT sensor) and a difference ofabout 80 in the right graph (which corresponds to an inertial sensor by a manufacturer differentfrom STT).

The lack of accuracy of the motion capture system with inertial sensors for the pommel horseexercises forced the need of using an optical motion capture system for this type of exercises.

The optical motion capture system did not impose any constraint neither to the athletes nor tothe trainers. The optical markers are lighter and smaller than inertial sensors; they are notintrusive (as the inertial sensors) and allow the athletes to perform the exercises withoutlimitation. The main drawback derived from this decision was the need to replicate in thesoftware of the optical motion capture system the modifications that were already implementedin the software of the inertial motion capture system. This decision was taken by the end of

August.

4.3. Third experiment run The third run of the 3D Acrobatics experiment made use of an optical motion capture system inorder to overcome the limitations identified in the other two runs with the inertial sensors.During the third run several training sessions of pommel horse exercises were successfullyrecorded with two different athletes. Figure 16 and Figure 17 show side by side pictures with the

athlete performing the exercise and the 3D avatar rendered in the computer screen by the optical


23/28



motion capture application. These training sessions were recorded in two consecutive days andtwo different athletes participated in the experiment.

Figure 16. Training session in the pommel horse: circles and flairs.

Figure 17. Training session in the pommel horse: swings and scissors.

Figure 18. Training session in cycling.

The third day was devoted to recording training sessions in cycling. The purpose of thisexperiment was to record simultaneously the motion capture system with the Kinect cyclingexperiment driven by CERTH in the CAR Venue.


24/28



In addition some of the training sessions in gymnastics and cycling were simultaneously recordedusing the stereoscopic HD cameras in the CONFetti experiment.

The collaboration with the CONFetti experiment is materialized by the implementation of a VRPN server as well as the sharing of the 3D model to be used in the combination of theresults. This VRPN server was made available for both the inertial and optical motion capturesolutions. This collaboration allowed to create augmented reality contents by combining the

video recorded in the CONFetti experiment with the 3D models and motion data recorded usingthe motion capture systems.

Figure 19. Collaboration between three experiments within EXPERIMEDIA.

Figure 19 shows the teams of three different experiments working together in the sameenvironment during the third run of the 3D Acrobatics experiment.

The experience of the athletes using the optical motion capture system was very positive. Theuse of the optical markers did not impose any restriction to their motions during the exercises.

The output generated by the motion capture software provided useful insight about the

execution of the exercises being a value tool for improving the athlete performance.


25/28



5. Results

5.1. Motion capture experience The use of motion capture systems in real training sessions has been very positive. Both trainersand athletes have adopted the technology without any problem. Athletes felt comfortable usingthe motion capture devices (either inertial sensors or optical markers). The use of those devicesdid not impose any restriction during the execution of the exercises.

The main goal of trainers and athletes is to improve the athlete's technique in order to achievethe optimal preparation for the competition. During the execution of the 3D Acrobaticsexperiment was found that this community is very open minded and are ready to adopt newelements which can potential help them to achieve a significant improvement in performance.

Trainers and athletes are not particularly interested in the principles or the operation of thetechnological assets that are made available by the training centre. Instead they are moreconcerned about the practical results that can be achieved by the use of the technology. Keepingthese concepts in mind the main goals derived from the use of the motion capture systems

within the 3D Acrobatics experiment are described in the following points:

It has been proven that the motion capture devices (inertial sensors or optical markers)are comfortably used by the athlete performing a training session.

It has been verified that the results delivered by the motion capture systems provideadditional value to the current training methods; trainers and athletes corroborated that

the data provided by the motion capture systems are reliable and useful for improvingthe athlete's technique and the trainer's approach to training. It has been learnt that an outstanding value provided by the use of motion capture

systems is the fact the evaluation is done in an "objective way" thanks to the quantitative values provided by the motion analysis software.

It has been confirmed that the use of 3D avatars gives a better understanding of theexercise as well as the perception of the own body and motion; a more detailed analysiscan give a better insight about the potential improvements than athletes can made inproprioception by using this kind of technologies.

It has been shown that the athlete's understanding about her/his own motion andtechnique is significantly increased.

The number of training sessions carried out during the execution of the 3D Acrobaticsexperiment has been reduced. Therefore it has not been possible to verify that the use of motioncapture systems significantly reduces the time required to improve the athlete's behaviour withrespect to a given exercise or technique.

In term of testing FMI products and services the experiment has totally reached the expectedresults. Nevertheless as the number of athletes using this system has been reduced the potentialimpact of this technology in FMI has only been tested in a reduced scope. It would be advisableto select a bigger group of users so as to cover a few tens of athletes in order to better evaluatethe impact of FMI products and services.


26/28



The overall evaluation of the results as well as the interviews with athletes, trainers and technicalstaff of CAR is that the tools used in the 3D Acrobatics experiment are a valuable tool fortraining and performance improvement in sports.

5.2. Lessons learnt from FMI The experimenters were interested in the development of the 3D Acrobatics experiment in orderto look into the application of inertial motion capture systems in the training methods of highperformance athletes as well as to explore the possibilities of FMI in sports training. FMIproducts and services are seen as a tool to reach a large community of users who are sport fansbut not necessarily high performance athletes. The following list summarizes the most relevantlessons learnt by the experimenters:

Inertial sensors are practical training sessions. Reliability of the motion capture data recorded in a practical training session has to be

improved. Inertial sensors can be used under certain circumstances and for someexercises (for instance jumping or physical evaluation).

Inertial as well as optical motion capture systems can be successfully applied in thetraining facilities of a high performance centre.

The results provided by motion capture systems are relevant for both the trainer and theathlete for the improvement of the athlete's individual technique.

Time required for the application of the motion capture results for the improvement of apractical training session is totally acceptable.

It is very helpful to have the option of sharing the motion capture data, video and

metadata through FMI products and services in order to improve athlete's individualtechnique.

During the execution of the 3D Acrobatics experiment the use of FMI products and services hasbeen limited to sharing motion data files as well as video files. In addition although the size ofthe observation group was adequate for the scope of the analysis proposed in the experiment itdoes not reveals trends in the potential impact of the used tools in a larger social network.

From the perspective of the FMI functions it would be advisable to enlarge the scope of theexperiment in order to encompass a larger community of athletes and trainers. In order to

accomplish this goal it would be recommended to select another type of sports speciality such assoccer, basketball or handball. These sports are extremely popular in most of Europeancountries and the number of sport schools managing young teams has a tremendous potential toperform the analysis outlined in this paragraph. For this application the results of the analysistools should be organized in a different way. A suitable approach would be creating a socialnetwork tool for athletes, trainers and sports enthusiasts.

Following these suggestions it would be possible to carry out a more detailed exam regarding thepotential impact that objective physical evaluation and training programs based on the use ofmotion capture systems can have in today's society.


27/28



5.3. DisseminationSTT has devoted efforts to the dissemination of the 3D Acrobatics results among its commercialpartners and customers. To this end a logo was designed in order to provide a visual image ofthe new services and products (see Figure 20).

Figure 20 3D Acrobatics logo

As a result of these dissemination activities, the following entities have showed interest in theoutcome of the 3D Acrobatics experiment:

Motion Sports Institute ( http://www.motionsportsinstitute.com/SitePages/Home.aspx ).

This company develops a product called web-locker which is aimed at sharing usefulinformation for athletes over Internet. The FMI products and services in combination

with the motion capture systems for objective physical evaluation has attracted theattention of this company.

Atlantic Health ( http://www.atlantichealth.org ). This company is a potential partner ofSTT in USA which is looking an expanding its activities in the field of physical evaluationon sports and rehabilitation. The FMI tools would allow sharing the informationbetween health professionals and patients/athletes in a very efficient way.

Indiana University ( http://www.indiana.edu ). STT introduced the results of 3D Acrobatics experiment to managers of Indiana University. This university has establisheda strong partnership program in sports with colleges and high schools. The availability oftools such those implemented in this experiment are of tremendous interest in apartnership program like the one implemented by Indiana University.
http://www.motionsportsinstitute.com/SitePages/Home.aspxhttp://www.motionsportsinstitute.com/SitePages/Home.aspxhttp://www.motionsportsinstitute.com/SitePages/Home.aspxhttp://www.atlantichealth.org/http://www.atlantichealth.org/http://www.atlantichealth.org/http://www.indiana.edu/http://www.indiana.edu/http://www.indiana.edu/http://www.indiana.edu/http://www.atlantichealth.org/http://www.motionsportsinstitute.com/SitePages/Home.aspx


28/28


6. Conclusions

The execution of the 3D Acrobatics experiment led to the conclusion that the proposedtechnology provides a tremendous added value to the sports community. The combination of

motion capture systems with FMI products and services represents an important contribution tothe sports community in the area of sports training and performance improvement.

The use of motion capture technologies has proven to provide a useful feedback both to athletesand trainers. It gives an instantaneous view of the exercise performed by the athlete offering abetter understanding of the training and execution techniques.

The runs of the experiment also highlighted some limitations on the use of inertial sensors forcertain types of evaluation. It has been proven that the inertial motion capture technology can besuccessfully applied in some exercises. This fact together with the portability of the system (can

be used outdoors, can be used in the field, there are no limitations imposed by the environment,etc.) makes this technology an interesting alternative for carrying out analysis in the field. This isan affordable that can be used not only with professional teams but also with amateur teams,schools, etc.

Considering the conclusions extracted of the experiment execution, STT is considering threedifferent action lines to continue with the activities started with the 3D Acrobatics experiment.

These lines are organized in different levels: technical, evaluation and verification andexploitation. At the technical level, STT considers the improvement of the algorithms andmethods implemented for the fusion algorithm. This development will enlarge the area of

potential application of the technology used in the 3D Acrobatics experiment. At the evaluationlevel, STT is aiming at continuing the collaboration with CAR in the application of the inertialsensors in the analysis and evaluation of knees in soccer teams. This area has been identified asan application area in which this technology may have a great impact. Finally, at exploitationlevel, STT will pursue the dissemination activities started with its partners and customers in USA.

D4.7.3 3DAcrobatics Experiment Results and Evaluation v1.0.pdf

Documents