D4.6.3 CONFetti Experiment Results and Evaluation v1.0.pdf

8/14/2019 D4.6.3 CONFetti Experiment Results and Evaluation v1.0.pdf

1/23

This document presents the results of the CONFetti experiment which aimed to investigate

the possible applications of FMI (Future Media Internet) technologies in the improvement of

the sports training process. The experiment was performed in the CAR (Centre d'Alt

Rendiment) venue and employed technologies such as videoconferencing, stereoscopy,

augmented reality and motion tracking.

D4.6.3

CONFetti Experiment Results and Evaluation

2013-10-29

Sergiusz Zieliski(Pozna Supercomputing and Networking Center)

www.experimedia.eu


2/23

EXPERIMEDIA Dissemination level: PU

Copyright Pozna Supercomputing and Networking Center and other members of the EXPERIMEDIAconsortium 2013 2

Project acronym EXPERIMEDIA

Full title Experiments in live social and networked media experiences

Grant agreement number 287966

Funding scheme Large-scale Integrating Project (IP)Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation

(FIRE)

Project start date 2011-10-01

Project duration 36 months

Activity 4 Experimentation

Workpackage 4.6 Remote and collaborative training for acrobatic sports

Deliverable lead organisation Pozna Supercomputing and Networking Center

Authors Sergiusz Zieliski(Pozna Supercomputing and Networking Center)

Reviewers Diego Esteban (ATOS)

Version 1.0

Status Final

Dissemination level PU: Public

Due date PM24 (2013-09-30)

Delivery date 2013-10-29


3/23



Table of Contents

1. Executive summary ............................................................................................................................ 4

2. Introduction ........................................................................................................................................ 5

3. Experiment architecture .................................................................................................................... 7

3.1. Architecture overview .............................................................................................................. 7

3.1.1. HD videoconferencing ......................................................................................................... 7

3.1.2. 3D ............................................................................................................................................ 7

3.1.3. Motion tracking ..................................................................................................................... 7

3.1.4. Visualisation ........................................................................................................................... 8

3.1.5. Integration with the Kinect cycling experiment ............................................................... 8

3.1.6. Experiment Content Component ...................................................................................... 9

3.1.7. Audio Visual Content Component .................................................................................... 9

3.2. Equipment .................................................................................................................................. 9

3.3. Ethics and privacy ................................................................................................................... 10

4. Experiment execution ...................................................................................................................... 11

4.1. First experiment run ............................................................................................................... 11

4.2. Second experiment run .......................................................................................................... 12

5. Results ................................................................................................................................................ 14

5.1. Questionnaires ......................................................................................................................... 14

5.2. Impact of QoS on QoE ......................................................................................................... 14

5.3. Dissemination .......................................................................................................................... 15

6. Conclusions ....................................................................................................................................... 16

Appendix A. Questionnaire ............................................................................................................... 17


4/23



1. Executive summaryThis document presents the results of the CONFetti experiment which aimed to investigate the

possible applications of FMI (Future Media Internet) technologies in the improvement of the

sports training process. As the relationship between the coach and the athletes is irreplaceable,the goal is not to substitute it, but to augment the availability, convenience and effectiveness of

the process basing on this relationship. The technologies that were utilised include HD

videoconferencing, stereoscopy, motion tracking and augmented reality. The experiment was

held at the CAR (Centre d'Alt Rendiment) venue, allowing the experimenters to evaluate their

system in a realistic target environment.

Section 2, which is an introduction, explains what the experiment's premise was and what the

different stakeholders wanted to obtain through performing it. Section 3 describes the

experiment architecture in terms of software and hardware as well as the legal issues associated

with it. Section 4 provides information about the two runs of CONFetti, while Section 5summarises its results. Section 6 gives a conclusion of the document and experiment.


5/23



2. IntroductionThe CONFetti experiment aimed to investigate the possible applications of FMI technologies in

the improvement of the sports training process employed at the CAR venue. The technologies

that were utilised included HD videoconferencing, stereoscopy, motion tracking and augmentedreality. The experiment employed a system allowing the coach to hold a training session without

the need for his presence in the training hall. He connected remotely with his protgs located in

the hall using a high definition (HD) videoconferencing tool. The system enabled him to observe

the athletes performing their gymnastics routines and, afterwards, to give them advice while

reviewing an instant replay with slow motion capabilities (Fig. 1). Additionally, the coach has

been able to review and present archival footage of athletes recorded with stereoscopic (3D) HD

cameras. This footage also contained a 3D model of a human body reflecting the athlete's

movements collected via motion tracking. The stream with the superimposed model was treated

as one of the sides of the videoconference for both the athletes and the coach to view.

Figure 1: A coach and gymnast reviewing a pommel horse performance stereoscopic replay

The experiment's goals from the venue's point of view were to evaluate the possible benefits

offered by the various FMI technologies used. The training process could get improved in thefollowing ways:

The availability of coaches for sessions is incremented as they can be held even when thecoach is not present at the venue.

The convenience for the coaches is raised thanks to the possibility of holding sessionsfrom remote locations.

The coach has access to additional motion tracking and sensor data concerning theathletes' performance, on which he can base his decisions.

The transfer of knowledge and instructions is more efficient with the use of archivalfootage and 3D model teaching aids.


6/23



The use of new technologies like augmented reality or stereoscopy may appeal to youngathletes and make the sessions more interesting and, in turn, more beneficial.

The experimenters carried out the experiment in order to investigate the following issues:

The feasibility of installing and utilising a setup for stereoscopic HD videoconferencingand augmented reality in a real-life training facility

User acceptance level (concerning both coaches and athletes) of the FMI functionalitiesprovided by the system deployed in a sport training setting

Performance of a platform combining HD videoconferencing with stereoscopic videoand remotely rendered 3D models generated basing on motion tracking


7/23



3. Experiment architectureThe CONFetti experiment design involved constructing the software platform, complementing

it with a number of hardware elements and solving some organisational, scheduling, logistics and

legal issues.

3.1. Architecture overviewThe CONFetti platform combines a number of FMI technologies and was integrated with a few

external components, all of which are mentioned below.

3.1.1. HD videoconferencingThis component's main task is to enable a stereoscopic HD videoconferencing connection

between the athletes in the training room and the remote coach. Additionally, it allows the coach

to review and present the archival footage of previous athletes' performances. There are two

instances of the videoconferencing client, one running on a machine at the coach's site, the other

in the training hall at CAR. The videoconferencing solution adopted for the CONFetti

experiment is based on HDVIPER, an open and scalable HD videoconferencing platform

developed in the course of the CELTIC initiative with a substantial participation of PSNC. The

aim of the HDVIPER project was to provide a converged platform for High Definition

videoconferencing (point-to-point and multi-point transmissions), to be integrated in a PC

environment (pre-commercial prototype) and to be tested in residential VC, healthcare, business

VC and distance education scenarios. Thanks to its extendable architecture the platform's

functionality can easily be broadened with additional services apart from HD video and audio

connectivity. The architecture is based on SIP (Session Initiation Protocol), which is verywidespread in Internet multimedia applications. Its popularity makes possible the platform's

compatibility with a wide array of solutions including software phones, smart phones and

videoconferencing terminals. HDVIPERs architecture is compatible with various client

applications, but for the purposes of the experiment one solution had to be selected and

appropriately extended. The chosen software is Minisip, an open source SIP User Agent

developed by the Royal Institute of Technology (KTH) in Stockholm and a community of

developers including PSNC employees.

3.1.2. 3DThe 3D functionality utilised in CONFetti is based on stereoscopy. A number of dual-lens

cameras are used to acquire footage of the athlete performing the routine, the coach and the

athletes gathered by the training hall VC terminal. The stereoscopic signal is transmitted through

the videoconference in the Side-by-side format and displayed on 3D enabled TVs. The

experiment participants are required to wear dedicated glasses in order to view the 3D video

feed. Additionally, a 3D transcoder component was developed with the ability to convert the 3D

signal to other formats than SbS.

3.1.3. Motion trackingThe motion tracking functionality in the CONFetti experiment was realised in cooperation withSTT, the EXPERIMEDIA partner responsible for the 3D Acrobatic Sports experiment. The


8/23



first run of the experiment was used as an opportunity to record gymnastic performances and, in

parallel, gather motion tracking data of the same performance, allowing to create 3D models of

the athletes bodies animated using this data. During CONFettis first run inertial Bluetoot h

sensors were utilised, whereas for the second run they were replaced by reflective markers (Fig.

2).

Figure 2: The capturing of a pommel horse routine with motion capture

3.1.4. VisualisationThe visualisation component is used to render the 3D models that are superimposed on the

archival video material. The input data for the animation rendering process are models andsensor events acquired by motion tracking done by the STT experiment.

The software used for this component is the Vitrall visualisation system designed and

implemented at PSNC. Vitrall is a distributed, web based visualisation system that is capable of

remote rendering of complex 3D content coming from various sources, e.g. scientific data or

multiple 3D model formats. It is designed to efficiently utilise multi-GPU and CPU-GPU hybrid

server installations. Thanks to highly efficient remote server rendering even a less capable client

is able to efficiently display and manipulate large 3D datasets in real-time. The system can also be

used in collaborative environments, where many users can simultaneously interact, modifying the

3D content in real-time. Vitralls high-performance remote visualisation capabilities have beensuccessfully demonstrated at various international conferences.

3.1.5. Integration with the Kinect cycling experimentThe CONFetti platform was integrated with the system utilised by CERTH in the Kinect cycling

experiment. This allowed to hold a 3D HD videoconference between the cycling training spot

and a remote location with statistical data from CERTH's system superimposed onto the

stereoscopic videoconference stream (Fig. 3).


9/23



Figure 3: Stereoscopic videoconference held during the CERTH cycling experiment

3.1.6. Experiment Content ComponentThe ECC's role is to gather monitoring QoS data from the other components that register as

sources of such data. Since all the software components of the CONFetti setup are written in

C++, a C++ client for the Experiment Content Component was needed. An API delivered by

the component's creator enabled the experimenters to implement the required client

functionality in the system and use it to gather usage statistics during the second run of the

experiment.

3.1.7. Audio Visual Content ComponentThe role of AVCC in the CONFetti experiment was to serve as a video repository. Videos ofathletes performing the gymnastic routines with 3D models superimposed were uploaded to an

AVCC instance and can be streamed to the Minisip videoconferencing client in order to review

them during the training session, giving the coach a possible source of training aids.

3.2. EquipmentApart from the software platform, the CONFetti experiment utilised a set of hardware

components that had to be transported to CAR for each experiment run. The main hardware

elements were:

3 stereoscopic cameras for capturing footage of the athlete performing the trainingroutine, the coach and the athletes gathered by the VC terminal

2 PCs with Blackmagic Decklink capture cards for receiving the HD stereoscopic signal A NewTek 3Play replay machine for instant replay and frame by frame analysis

functionality

2 3D TVs for displaying the videoconferencing signal in the training hall and in thecoach's room

A video recorder capable of outputting stereoscopic materials from flash cards


10/23



3.3. Ethics and privacyThe key ethics and privacy issues to consider for the CONFetti experiment were:

Obtaining consent for participation in the experiment from coaches and athletes

Storing and processing video material including the participants Storing and processing animations of the athletes' bodies Storing and processing questionnaires filled by the participants

The consent from the participants was obtained by CAR representatives. As for the body

animations and questionnaires, they are anonymous and as such do not pose a problem in regard

to privacy. The key issue proved to be the video materials used both for the creation of training

aids and for the production of the promotional videoclip.

Discussions between representatives of the venue, the experimenters and the consortium

partners responsible for Privacy Impact Assessment have been held during a number of projectmeetings and followed up with an e-mail exchange during which a questionnaire was filled by

both the experimenters and the venue representatives in order to complement the PIA-related

experiment information that was delivered in D4.6.1.

After gathering the necessary information representatives of the partner responsible for PIA

proposed an action plan regarding personal data flows in the CONFetti experiment. The main

point of this plan was the need to sign a controller-processor contract between the venue and the

experimenting institution (CAR and PSNC). Such an agreement has been drafted to resolve the

mentioned issues.


11/23



4. Experiment executionTwo runs of the CONFetti experiment were performed, in May and September 2013. The

original schedule assumed the second run would be done between June and August, however

June would be too soon after the experiment's first run and the availability of participants at thevenue during July and August was problematic.

4.1. First experiment runThe first run of CONFetti, done on May 15-16th 2013, included an HD 3D videoconference

with instant replay and frame-by-frame functionality. Archival materials recorded in January 2013

were also available for the coach to use, although they did not yet contain 3D models of the

athletes' bodies. The opportunity was used to record gymnastic performances in parallel with

tracking the athletes' motions performed by STT. This allowed to subsequently render the 3D

models for use in the future. Additionally, all the experiment's participants were asked to fill

questionnaires concerning their views on the system, its different functionalities and the

difference it made in the training session.

Figure 4: The coach room during a CONFetti experiment run

The equipment involved was installed in two rooms in CAR: the training hall and an office

located in a different building, which played the role of the remote coach's post (Fig. 4). This

post was equipped with a 3D camera on a tripod, a 3D-capable TV, a recorder capable of

outputting stereoscopic materials from flash cards and a PC with two Blackmagic DeckLink

capture cards. The Minisip videoconferencing client was installed on this computer, with the

camera and the aforementioned recorder connected as sources of video streams. The recorder

played back performances of the gymnasts recorded in January. The videoconferencing client

installed on the coach's terminal maintained a connection with the training hall, where the setup

was a little more complicated. It consisted of two 3D cameras on tripods, two 3D TVs, a

NewTek 3Play replay machine and a PC with two DeckLink cards and Minisip installed. The

signal sources for Minisip were the cameras - one of them was directed at the athletes andcoaches taking part in the videoconference from the training hall, whereas the other was directed


12/23



at the gymnast performing his routine on the pommel horse. The signal from the latter camera

was transformed on the way by the replay machine, which made it possible to have instant

replays with fluent speed control and frame-by-frame analysis as well as straight forwarding of

the live feed to the videoconference. The VC signal was simultaneously displayed on two 3D

TVs, one for the on-site coach and one for the athletes. Altogether, four stereoscopic HD

streams were simultaneously sent through the videoconference: the coach and the archival

materials from one side; the athletes discussing with the coach and the athlete performing the

pommel horse routine (or the replay) from the other side. The choice of displayed signal was

independent and free on both sides.

During a single training session an athlete performed the routine on the pommel horse, which

was observed by two coaches: one present in the training hall, the other one remote.

Subsequently, the gymnast joined his colleagues gathered by the VC terminal, where the replay of

his performance was displayed. The coaches consulted each other, commented on the routine

observed in the replay and gave advice to the athlete. Additionally, the remote coach had thepossibility of presenting archive video materials with performances by the same gymnasts from 4

months before.

4.2. Second experiment runThe second run of the CONFetti experiment was performed on 16-18th September 2013. This

time, apart from exposing the participants to the functionalities present in May, the 3D models

rendered basing on motion tracking data collected by STT were used. The animated models

rendered by the Vitrall system were superimposed onto footage of athletes performing that was

recorded during the first run. The animation of the model corresponded with the athlete's

movements to give the viewers a more comprehensive overview of the performance (Fig. 5).

Figure 5: Footage of a pommel horse session with the skeleton model superimposed

Additionally, the CONFetti platform was integrated with the system utilised by CERTH in the

Kinect cycling experiment. The biomechanical data generated by the system was superimposed


13/23



on the stereoscopic videoconferencing signal allowing the remote coach to observe it alongside

the feed of the cyclist. Thanks to the videoconferencing connection they were able to

communicate and exchange remarks regarding the training session.

The opportunity was also taken to record high quality 4K video materials of the venue and

experiments in order to produce a videoclip for the purpose of promoting the experiment and

project. A short interview was also recorded with the CTO of CAR, Josep Escoda.


14/23



5. Results5.1. Questionnaires

The experiment was received very enthusiastically by both the coaches and the athletes, as well as

other CAR employees who visited both posts very often. The participants of the experimentwere asked to answer questionnaires prepared in cooperation with CAR psychologists, which

helped in obtaining information on usability of the CONFetti system in this scenario as well as

ideas for possible improvements. Analysis of the answers given shows that the technology might

prove useful in this scenario. Both coaches and athletes stated that the instant replay and 3D

features allowed them to evaluate the performances in a more detailed way. The

videoconferencing ability was appreciated by the coaches who would gain the benefit of more

flexible scheduling from it. It became apparent that a person overlooking the athletes in the

training room is still necessary, as one of them expressed concern what would happen if he got

injured with no coach around. Another concern was that the 3D glasses needed to view the 3Dsignal were a slight annoyance, as they had to be put on and taken off constantly. This might be

solved with the perfection and popularisation of glassless 3D technologies. The experiment has

also shown other limitations of the technology in this scenario. The sports discipline needs to be

fairly static in terms of space in order to fit in the frames of all the cameras. Also, 3D technology

requires a steady distance between the camera and the main plane of the scene, at least in its

more basic form. Those obstacles can be overcome with more staff and more expensive

equipment, but if the essential setup is to be used it requires some preparation beforehand and

puts some limitations on the usage areas.

The most interesting results drawn from the questionnaires were:

The vast majority of participants (12 out of 14) strongly agreed that the system was easyto use

All of them found the video quality to be good The fluency was described as smooth enough to observe the athletic performance,

although the answers were not as strong as in the case of the question about quality

3 people noticed the delay in the videoconferencing signal 5 people thought it was disturbing to use 3D glasses

The audio quality was described as good The instant replay and frame by frame functionality was found to be useful 1 person thought that the equipment setup disturbed the training session The 3D video allowed the participants to review the performance more thoroughly and

better visualise the remarks of the coach

5.2. Impact of QoS on QoETo facilitate a deeper evaluation of the stereoscopic technology during CONFetti runs, a small

part of the sessions was held in 2D for comparison. All the signals in the experiment were sent

in the 1080p25 format. To get the coaches' opinion on this matter, materials in the 720p50,

1080p50, 1080p25 (Full HD) and 576p50 (SD) formats were also presented to them. Results of


15/23



this study show that the difference between the Full HD and 720p resolutions is not noticeable

in these circumstances, whereas the difference between 720p and SD was noticed, as well as the

one between 50 and 25 frames per second. The conclusion would be that in the case of the setup

that was used in CONFetti (specifically the screen sizes and viewing distances), the 720p

resolution provides a high enough quality. The fluency of the movement provided by the 50 fps

framerate is more important in this scenario than the additional details provided by Full HD.

This is especially significant because some equipment components do not allow to combine

both, effectively forcing the users to choose between resolution and framerate.

5.3. DisseminationDisseminating the results of research projects is a crucial factor in being part of the scientific

community. That is why the experimenters have used a number of opportunities to spread

information about CONFetti:

Figure 6: Visitors of Researchers' Night 2013 at the CONFetti stand

A paper titled "Stereoscopic videoconferencing with augmented reality in technologyenhanced sports trainingwas submitted to the eChallenges 2013 conference's

Technology Enhanced Learning thematic area and a presentation was given at the

conference itself

EXPERIMEDIA and CONFetti were mentioned in presentations during the i3'2013conference as well as the FI-PPP InfoDay and Brokerage Event in Pozna

The CONFetti platform was used in a stand at Researchers Night 2013in Pozna(Fig.6)

A promotional videoclip of CONFetti and EXPERIMEDIA was produced


16/23



6. ConclusionsThe analysis of the questionnaires and the QoS data measured during the experiment runs shows

that the system is suitable for application in a training venue, both performance and functionality

wise. The feedback that was gathered also contains useful remarks concerning possibleenhancements and fine-tuning of the systems parameters.

The experiments results show that the deployment of a system offering CONFettis

functionalities in CAR would be beneficial for all the stakeholders involved. The athletes found it

fun and interesting to use and also stated that the instant replays in slow motion and 3D help

them review their performance and understand the coachs remarks. For coaches, the additional

functionalities, the high quality of the VC signal and the stereoscopy make the remote training

session a valid alternative to personal interaction, allowing them to save time and be more

flexible in scheduling. For the venue, the system could allow to decrease the costs of the training

program and increase the availability of coaches, which is especially important in the times ofausterity.

The system created during the CONFetti experiment proved to be useful for sports training

applications. The feedback gathered from both the athletes and the coaches verifies that, while

additionally giving some interesting remarks about possible enhancements. The venue also found

it beneficial to make use of the systems possibilities in the training process. The tests have also

proven that it is feasible to set up such a system in a real-life training hall and that its

performance is satisfying for the purposes evaluated in the scenario.


17/23



Appendix A. QuestionnaireThe below questionnaire was used to gather participant feedback during after the CONFetti

experiment runs. It was presented to the coaches and athletes in Spanish.

The following statements could be rated using a scale of 1-5, where:

1I strongly disagree

2I disagree

3I don't know

4I agree

5I strongly agree

A

1. It was easy to understand how the system works.

2. The video quality was good.


18/23



3. The high definition gives me more details that are important for evaluation of the

performance.

4. The fluency of the video was smooth enough for me to observe the athletic

performance.

5. The video signal from the other side was delayed.


19/23



6. It was a disturbance to use the 3D glasses.

7. The audio quality was good and allowed me to hear the other side of the

videoconference clearly.

8. It was useful to use instant replay and frame by frame functionality.


20/23



9. I think it is useful to have the possibility of holding training sessions through

videoconferencing.

10. The technology setup disturbed my concentration during the training session.

B

1. I think seeing the video is useful for learning or perfecting the technique.


21/23



2. 3D observation helps me to see in more detail the position of my body during the

execution compared with a standard video image.

3. 3D observation allows me to be more aware of the body extension while performingcompared with a standard video image.

4. The 3D observation allows me to be more aware of the inclination of the body relative

to the rack during the performance compared to a standard video image.


22/23



5. 3D observation allows me to better be aware of the action of the arms during execution

(Extension-abduction) compared with a standard video image.

6. 3D observation allows me to be better aware of rhythm and exercise execution speed

compared to a standard video image.

7. 3D observation helps me relive the sensations I felt during the exercise.


23/23



8. Usually I use visualization to improve the execution of technical motions.

9. I think 3D observation I would improve the visualization of the execution of theexercise.

How do you think the system could be improved?

3D without glasses More cameras to add different angles of the performance A coach still has to be around in case of injury

D4.6.3 CONFetti Experiment Results and Evaluation v1.0.pdf

Documents