Virtualizing testbed resources to enable remote experimentation in online telecommunications education

Virtualizing testbed resources to enable remoteexperimentation in online telecommunications

education

Johann M. Marquez-Barja∗, Nicholas Kaminski∗, Francisco Paisana∗, Christos Tranoris‡ and Luiz A. DaSilva∗∗CTVR / the telecommunications research centre

Trinity College Dublin, Ireland.{marquejm,kaminskn,paisanaf,dasilval}@tcd.ie

‡University of Patras, [email protected]

Abstract—In this paper we present an approach towardsempowering online telecommunications engineering education byenabling hands-on remote experimentation over Trinity CollegeDublin’s wireless testbed. Moreover, in order to offer a flex-ible testbed, capable of fulfilling the different and particularrequirements of experimenters, we have created a framework thatallows the virtualization of our testbed resources to create exper-imentation units to be used by remote experimenters/learners.Furthermore, we present the FORGEBox framework that offersan environment and resources to create online material capableto access the virtualized and physical testbed resources for incor-porating experimentation into HTML-based online educationalmaterial.

Keywords—virtualized testbeds, cloud computing, wirelesstestbeds, research experimentation, remote experimentation, elearn-ing.

I. INTRODUCTION

Learning is a key human activity essential for personalwell-being and ensuring a good quality of life. Learning cantake many forms, from informal learning via online resources,workplace learning to update oneself on latest proceduresor protocols, to formal certified learning within establishededucational institutions. Learning is also an important concernwhen considered simply from a budgetary perspective. For ex-ample, in 2009 the European Union (EU) budget on educationwas 6.2% of the European Gross Domestic Product (GDP)1.Moreover, the education budget is currently being reduced ina number of EU countries, for example in Spain and Greece,which is an additional rationale for innovative solutions en-abling the provisioning of cost-effective high quality learning.

Information and Communications Technologies (ICT) haveenabled a wide range of e-learning methods, technologies andtools, varying from asynchronous assisted learning systemsto highly dynamic and interactive learning platforms. Thesetechniques provide new options to improve the learning expe-rience of students under a limited budget. A step forward inthe evolution of the e-learning processes is to enhance themby enabling experimentally-driven e-learning. To achieve thisgoal two main components must come together: remote testbed

1http://epp.eurostat.ec.europa.eu/statistics explained/index.php/Educational expenditure statistics

experimentation and online platforms capable of interactingwith those testbeds.

Experimentation is a key component of engineering ed-ucation. However, physical experimentation is expensive, inparticular for low-budget institutions, difficult to maintain,usually requires specific guidance during the experiment, andthe access to lab facilities are restricted beyond regular workinghours, as emphasized by Bose [1]. Physical experiments aremandatory for most engineering education areas in order toallow learners to fully understand design procedures, practicallimitations, and engineering tradeoffs. The experimentationprocess is widely recognized as vital for developing theseskills in both undergraduate and graduate telecommunicationsengineering education and research [2]. Deploying and main-taining experimental laboratories is costly; offering a labora-tory capable of developing cutting edge telecommunicationstechnologies requires massive effort and budget. Therefore,remote laboratories can alleviate these problems by providingexternal access to experiments and allowing students to accessexperiments without time or location restrictions. Furthermore,these facilities can be packaged with supporting materials thatprovide the necessary guidance to students or are configuredto constrain operation according to the experimenter’s require-ments. Remote experiments can be ready all the time, andthus the remote laboratory concept provides a tool to sustaina learner-centric teaching approach [3]. Moreover, severalstudies demonstrate the benefits of hands-on laboratories andlearning-by-doing approaches [4], [2], [5]. Additionally, thereare programmes leveraging the wide deployment of experi-mental platforms such as Fed4FIRE2, Planetlab3 [6], Orbit4and Forging Online Education through FIRE (FORGE)5 [3]for remote experimentation with communication networks.

Testbed facilities, in order to fulfil the different users’needs and to offer customized solutions to each user or typeof experiment, can apply virtualization techniques. Virtual-ization of computational resources offers numerous benefitsthat have been recognized in several fields. These benefits arebased on the abstraction that is fundamental to virtualization;

2http://www.fed4fire.eu3http://www.planet-lab.org4http://www.orbit-lab.org5http://www.ict-forge.eu

virtualization separates operating system components fromhardware resources. The nature of this abstraction and theadditional functionality enabled depends largely on the specificcomponents used to compose the overall testbed.

This paper presents our initiative on enhancing engineeringeducation by providing an online course empowered by hands-on experimentation on top of our high-performance testbedthat combines virtualized cloud-computing and advanced radiohardware. This effort provides an example of the successfulintegration of a virtualized wireless testbed into universityinstruction.

II. RELATED WORK

Online available labs, depending on their features, havebeen grouped into several categories. Depending on the meth-ods used to access and to trigger the equipment at the back-end facility and the technology used in the front-end graphicalinterface, from three to six different categories have beendefined [7], [8], [1]. We can summarize those taxonomies intothree categories:

• Virtual labs, which are software-based laboratories,empowered by simulation tools.

• Remote labs, based on remote experimentation on reallab equipment.

• Hybrid labs, which combine the above two by pro-cessing output data from real measurements into sim-ulation tools.

As mentioned before, experimentation is required to trainstudents and enhance their skills in higher education pro-grams, in particular in engineering fields. There are severalworks that describe the approaches that different universitiesand/or projects have applied to enable engineering-relatedonline laboratories. Most of the approaches rely on simulation,providing virtual labs for teaching robotics [9], electroniccircuits [10], control systems [7] or a broad list of engineeringdisciplines [1].

Nevertheless, there are particular areas that require realexperimentation, and where remote-based labs bring an addedvalue, as is the case of the lab presented by Fidalgo et al.[11] for remotely designing and testing circuits in physicalequipment.

Regarding remote labs for teaching telecommunicationsrelated content, few approaches have been publicly proposed.Bose and Pawar [12] have proposed a remote lab for wirelesssignals where students can learn about the foundations of wire-less signal, with concepts such as antenna radiation pattern,gain-bandwidth product of an antenna, cross polar discrimina-tion and SNR. The architecture proposed presents a front-endwebpage to access the back-end equipment. The website isdeployed with Adobe Flash to offer the graphical interface.The back-end equipment relies on LabView to interface to thetelecommunications equipment, including spectrum analyzers,oscilloscopes and signal generators. This lab can be accessedby one student at a time.

Our approach falls into the remote lab category. We haveenabled flexible radio equipment to be accessed through a

cloud-based system, allowing students to perform remote ex-periments from different computing devices.

III. VIRTUALIZED TESTBED FOR EDUCATIONAL USE

In Trinity College Dublin (TCD) we have been successfulin deploying a wireless communications and networks labo-ratory to support telecommunications engineering education.Our laboratory is equipped with advanced radio hardware thatallows us to study, in a flexible manner, a wide range ofwireless technologies. Figures 1 and 2 show the computa-tional resources and specialized radio frequency equipment,respectively, deployed in our laboratory. The wireless networklaboratory allows us to train students on radio hardware -e.g.,Universal Software Radio Peripheral (USRP)- that we employto perform complex experiments in wireless communications,and spectrum analyzers that provide accurate measurementsacross the frequency spectrum. This combination of labora-tory resources has enabled a telecommunication ecosystemthat supports hands-on experimentation not only for TCD’sstudents, but also students from around the globe, since thelaboratory can be accessed both locally and remotely throughthe Internet.

Fig. 1. Wireless laboratory servers

General testbed facilities can be classified into develop-ment, research and instructional oriented [7]. Our testbed wasinitially deployed for research purposes. More recently, wehave applied our expertise in testbed development to alsosupport engineering education, testing and prototyping workby industry partners, and research efforts of several institutionsthroughout Europe. Targeting the educational initiative dis-cussed here, we have designed a wireless communications andnetworks course that is anchored in hands-on experimentationusing our local testbed. Our hands-on lab course relies in twocomponents: the testbed facilities (local and remote) and theeducational material.

Fig. 2. Wireless laboratory radio equipment

A. Testbed architecture and resources

We maintain a wireless testbed for the exploration ofSoftware Defined Radio (SDR) systems, focusing on providingusers the ability to customize the platform to their needs.In pursuit of providing this custom testbed experience, ourfacility is organized into experimentation units, supported byvirtual computational platforms. Experimentation units are alogical organization of resources consisting of a computationalelement, a hardware radio front-end component, and the Irissoftware defined radio package [13]. Each experimentationunit represents the minimum set of resources required by auser to construct a radio element in TCD’s facility. Due tothe capabilities of Iris, these experimental units effectivelyrepresent the potential to realize any arbitrary radio systemthat a user may require. This organization of TCD’s testbedfacility allows users to deploy a customized radio system asneeded.

User customization is underpinned by the application ofa cloud-computing based management system. This paradigmcombines virtualized computational resources, loaded with ahighly configurable (and reconfigurable) SDR package, and

widely flexible radio hardware to support experimentationbased research. Virtualization of computational elements pro-vides each user complete control over an isolated computa-tional environment. Since each user controls their own virtu-alized environment, users are free to completely individualizetheir computational platform, including configuration of globallibraries or loading of additional software. Users may savetheir configurations, including all details of the computationalsystem, for later use, at any point. The management systemstreamlines coordination of virtual computers, handling thedeployment of these environments and their connection toradio hardware at the request of the user. The result ofcloud-computing based management is that each user gets theexperience of a personal testbed, yet the resources are sharedamong many separate users and projects.

Figure 3 displays the architecture of TCD’s virtualizedtestbed. Each experimentation unit initially provides users witha virtual computer running a Linux-based operating system andloaded with the Iris SDR package. An array of servers, referredto as host servers, provide the computational power to runthese experimentation units. Each virtual machine is connectedto a USRP mounted on our ceiling grid within our dedicatedtesting space. These connections are made through dedicatednetwork ports; i.e., each experimentation unit is provided witha dedicated network interface for connection to a USRP. Thefront-end server coordinates and controls virtual machines,handling the deployment of computational environments andtheir connection to USRPs. The front-end server also providesusers access to experimentation units, currently through SSH.Centralized control also allows the front-end to handle thescheduling of user access.

Fig. 4. Support architecture

Figure 4 displays the support structure for the virtualizationmanagement system. A gateway machine provides a centralpoint for remote testbed access, data access, and documenta-tion. This gateway machine also runs the FORGEBOX userinterface, see Section III-B. The dedicated documentationserver provides users with a wealth of information on howto use the testbed. The data server holds user data separatelyfrom virtual machines, providing a more permanent storage lo-

Fig. 3. Testbed architecture

cation and allowing the collection of information from severalvirtual machines. This support system allows users to take fulladvantage of the cloud-based management structure.

B. FORGEBox framework

To deploy this course and make it available to learnersthrough web access, a framework called FORGEBox6 is used.This framework technology is provided by the FORGE EUFramework Programme 7 project, which aims to transform theFuture Internet Research and Experimentation (FIRE)7 testbedfacilities into a learning resource for higher education. FORGEprovides an educational layer over the FIRE facilities, enablingexperiment-based learning resources. FORGE specifies devel-opment methodologies and best practices for offering testbedexperimentation facilities to learners.

This section gives a brief overview of the FORGEBoxframework technology, which is used to support interactivecourses using resources of remote labs. FORGEBox is definedand implemented within the FORGE project. The frameworkdefines an architecture, user roles, core entities and necessarycomponents to support these courses. There are three definedcore entities:

• FORGEBox course module, which consists of coursepresentation parts and interactive parts. The presenta-tion parts contain text, images and video content. Theinteractive parts contain widgets that help the learnerto interact with the underlying remote lab resources.

6http://www.forgebox.eu7http://ict-fire.eu

• FORGEBox widgets are small web-based standaloneapplications that expose the remote lab functionalityand enable a learner to perform actions on remoteresources. These can be embedded in web pages oreBooks.

• FIRE adapter/FORGEBox services are backend ser-vices that either support widget functionality or sup-port a lab course assistant/designer to setup a remotelab for a course.

The framework defines also the following main actors:

• Learner: utilizes FORGEBox tools and services, usesdifferent means to access the courses and manipulatesremote lab resources through widgets. The meansto access the courses include electronic books orLearning Management System (LMS) web pages.

• Lab course designer: designs a course and imple-ments it by using learning material (e.g., text, figures,videos), widgets and FIRE adapters.

• Lab course assistant: responsible for the normalexecution of a course and has different responsibilities,such as creating accounts for lab learners, schedulingand reserving remote lab resources (if not carriedout automatically when a Learner starts a course),delegating control, etc.

• Widget provider: develops and maintains widgets(usually for web consumption) providing a user in-terface for learners to manipulate the experimentationenvironment.

• FIRE adapter/service provider: develops and main-tains FIRE adapters, deployed into FORGEBox ser-vices.

FORGEBox also defines a platform that hosts all the abovenecessary artefacts to enable an interactive course. It is an ag-gregation of services able to support all FORGE concepts andrequirements, learning widgets, and FIRE adapters/services.FORGEBox is delivered as a middleware solution, deployedin institutions executing courses or into a cloud infrastructure,bridging the interfaces between learning means and FIRE toolsand facilities.

FORGEBox includes tools and services that target bothlearners and lab course designers. Learners can easily accessFIRE facilities and perform small experiments through theweb enabled interface of widgets. Lab course designers havea collection of tools to create the course content, and toprepare and configure the target FIRE testbed that supportsthe interactive course.

Figure 5 presents a detailed view of the FORGEBoxarchitectural framework. The figure displays also the conceptof a FORGE repository that hosts any shared published itemssuch as lab courses, widgets and FIRE adapters to be used bythe learning community and by other organizations hostinga FORGEBox instantiation. At its simplest form the coreconsists of services that make some tasks easier such ascreating, managing and operating lab courses and their contentas well as widgets and adapters. FORGEBox will contain a setof managing services, a widgets layer, a FIRE adapters layerand a local repository of hosted lab courses. FORGEBox isimplemented and provided under open source licenses. A run-ning instance can be found at http://www.forgebox.eu/fb. Theglobal repository for FORGEBox artefacts sharing is calledFORGEStore and its located at http://www.forgestore.eu.

C. SSH

One of the most used means to access machines residingin remote lab facilities is through terminal utilities. The mostknown protocol is Secure Shell (SSH), since it enables anexperimenter to access a resource and fully manage it viathe command line. SSH client tools exist for almost anyoperating system, either natively or via external downloadedtools. In FORGEBox we turned to web-based SSH tech-nology. Web-based SSH makes it possible to access SSHservers through web clients that are based on JavaScript/Ajaxor JavaScript/WebSockets technologies. To enable web-basedSSH clients a server side web application is needed, hosted ina server playing the role of an SSH proxy. Incoming requestsare processed on the web application server. Keyboard eventsare forwarded to a secure shell client communicating with theconnected SSH server. Terminal output is either passed to theclient where it is converted into HTML via JavaScript or itis translated into HTML by the server before it is transmittedto the client. The SSH proxy web application can be hostedin FORGEBox as a service. Figure 6 presents the ssh2webwidget as used inside a course.

The widget is provided by a web application written inJava. Figure 7 displays the architecture of the widgets webapplication. It shows how the communication is accomplishedfrom the ssh2web application towards a remote machine, via

Fig. 6. SSH2web widget embedded in HTML

private/public keys or even username/password pair. The textprovided by the SSH communication is translated from aread/writer thread class into a web JSON object and is passedto the websocket class, and is sent to the web browser viaa javascript web socket opened in the browser. The reverseprocess is applied when the user types a command. Eachkeystroke is passed as a JSON object to the websocket class,where it is transformed again and sent through the SSH sessionclass to the remote machine. Finally, we can audit the sessionfor security purposes, for future reference, or any other kind ofmeasurements, by accessing the history of typed commands.

Fig. 7. SSH2web widget architecture

IV. EDUCATIONAL MATERIAL

As mentioned previously, our testbed is deployed underthe cloud-computing paradigm, having different computationaland USRP radio resources under a single managed cloud, asshown in Figure 8. Such resources are then used to enablespecific requests on the demand of the users of the cloud. Wehave deployed the tools that enable the whole infrastructureto create the ‘experimentation units’ for educational purposes.Moreover, Figure 9 illustrates the concept of deploying a realnetwork from a ‘drag and drop’ widget, where the widgettells the front-end what resources are needed and enables suchresources to be delivered to the cloud. All of this is enabledthrough the FORGEBox framework. Finally, the student canaccess those resources directly in a transparent way, havingreal resources for online experimentation.

The created content can be easily adapted to MassiveOnline Open Courses (MOOCs) and interactive eBooks, where

Fig. 5. FORGEBox framework architecture

Fig. 8. Mapping the Widget-based experiment configuration into the testbedresources

rich multimedia content is combined with interactive pages toset up and run large-scale experiments on top of our wirelesstestbed.

We now briefly provide an overview of a module created inTCD, as well as the type of experimentation involved withinthis module:

Title of the course

Coexistence of Small Cells in Radar Bands

Fig. 9. Mock up of the Experiment creator widget

Summary of the course

Radar spectrum is currently heavily underutilized due pri-marily to the need to avoid interfering with sensitive measuringequipment. Pursuit of such protection has resulted in thecreation of large exclusion zones around radar stations thatisolate them from potential interference. Due to the high degreeof radar directionality and the availability of highly capablewireless systems, existing static exclusion zones representan unnecessary inefficiency. TCD has constructed a RadioCoordinator Module using the Iris software defined radiopackage and the virtualized wireless testbed that allows radiosto identify the features of a radar system and effectively operatewithin that radar’s exclusion zone without causing harmfulinterference. This coordinator module is a prototype for a smallcell controller, designed to open radar spectrum for use in

future cellular networks.

With this course we demonstrate the capability of CognitiveRadios (CRs) or Secondary User (SU) systems to coexist withradar systems, the incumbents, even when the CR is locatedinside the radar exclusion zone defined by the range of itssignal through its antenna main beam. In order to do so, CRshave to perform the following tasks:

• Estimate the parameters of the radar systems in theradio environment, namely, their antenna radiation andscan pattern and rotation periodicity, through spectrumsensing.

• From the perceived radar’s rotation periodicity andradiation and scan patterns, design a mechanism thatpredicts future instances of interference from the CRto the radar and vice-versa. To assess whether theremight be interference, the received signal strength bythe CR is compared with a pre-defined threshold.

• Schedule transmissions in such a way it avoids inter-fering with the radar for the intervals of interferencecalculated in the second step.

• Stay synchronized with the radar antenna rotation.

Online material of the course

The material for this course can be used and reused indifferent elearning platforms and devices, as mentioned beforeand shown in Figure 10. Moreover, the full HTML-basedcourse is shown in Figure 11, where different multimediacomponents, such as widgets, sliders and images, are combinedto offer an enhanced hands-on remote experimentation course.

This course is being taught in the 5C2 - Wireless networksand communications module of the Electrical and ElectronicEngineering (EEE) Master in Engineering (MAI) degree withinTCD. Moreover, it will be soon publicly available through theaforementioned FORGEBox portal.

Fig. 10. The course material shown in different devices

V. DISCUSSION

Within the elearning community, and in particular, theonline labs community, several concerns are being consideredand discussed regarding the educational and technologicaldomains.

1) Education related:

• Pedagogical principles. When deploying the contentlabs and designing the experimentation, several peda-gogical aspects should be taken into account, such asmatching the curricula, inclusion, learner engagement,innovative approaches, and effective learning, amongothers.

• Learning analytics. The labs deployed should aim tocollect data and analyze the learning patterns of thestudents in order to improve the labs [14].

• Evaluation. The labs should have an evaluation modulethat enables the observation of the skills gained by thestudents due to experimentation.

2) Technology related:

• Development methodology and tools. A methodol-ogy should be defined when deploying online labs.Common tools can be used to provide homogeneousfeedback about the students’ performance, enablingcross-referenced information to enrich data for ana-lytic purposes.

• Sustainability models. There is a concern among on-line lab providers and developers about the sustainabil-ity models that should be applied towards maintainingsuch labs and towards improving them [8].

VI. CONCLUSIONS

We have presented our approach for enabling virtualizedcloud-computing and advanced radio hardware for educationalpurposes. The use of our enhanced facilities for hands-on re-mote experimentation and the experience of lecturing the 5C2module allow us to attest to the benefits of blended teaching,combining traditional lectures with hands-on experimentationusing remote high-performance facilities, and its valuableimpact on the students’ learning experience. Moreover, thisapproach enables Self-Regulated Learning (SRL), allowing thestudents to learn at their own pace. Nevertheless, our approachwill continue being developed to tackle the issues discussed inthe paper, to improve the learning experience of the studentsand to provide feedback to instructors.

ACKNOWLEDGMENTS

This work has received funding from the European Union’sSeventh Framework Programme for research, technologicaldevelopment and demonstration under grant agreements no.610889 (FORGE) and 258301 (CREW).

We also acknowledge support of the Fed4FIRE project(’Federation for FIRE’), an integrated project funded by theEuropean Commission through the 7th ICT-Framework Pro-gramme (318389).

REFERENCES

[1] R. Bose, “Virtual Labs Project: A Paradigm Shift in Internet-BasedRemote Experimentation,” IEEE Access, vol. 1, pp. 718–725, 2013.[Online]. Available: http://dx.doi.org/10.1109/access.2013.2286202

[2] G. Jourjon, S. Kanhere, and J. Yao, “Impact of an e-learning Platformon CSE Lectures,” in Proceedings of the 16th annual joint conferenceon innovation and technology in computer science education, 2011, pp.83–87.

[3] J. M. Marquez-Barja, G. Jourjon, M. Alexander, C. Tranoris,J. Domingue, and L. A. DaSilva, “FORGE: Enhancing elearningand research in ict through remote experimentation,” in IEEE GlobalEngineering Education Conference (EDUCON), April 2014, pp. 1157–1163.

[4] A. Sanchez, I. Moerman, S. Bouckaert, D. Willkomm, J. Hauer,N. Michailow, G. Fettweis, L. Dasilva, J. Tallon, and S. Pollin,“Testbed federation: An approach for experimentation-driven researchin cognitive radios and cognitive networking,” in Future Network &Mobile Summit (FutureNetw), June 2011, pp. 1–9. [Online]. Available:http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=6095242

[5] N. Handigol, B. Heller, V. Jeyakumar, B. Lantz, and N. McKeown,“Reproducible network experiments using container-based emulation,”in Proceedings of CoNEXT, 2012.

[6] PlanetLab Consortium, “Planetlab: An open platform for develop-ing, deploying, and accessing planetary-scale services,” http://www.planet-lab.org/.

[7] A. Diwakar, S. Poojary, R. Rokade, S. Noronha, and K. Moudgalya,“Control systems virtual labs: Pedagogical and technologicalperspectives,” in IEEE International Conference on ControlApplications (CCA), Aug. 2013, pp. 483–488. [Online]. Available:http://dx.doi.org/10.1109/cca.2013.6662796

[8] S. Frerich, D. Kruse, M. Petermann, and A. Kilzer, “Virtual Labsand Remote Labs: Practical experience for everyone,” in IEEEGlobal Engineering Education Conference (EDUCON), Apr. 2014, pp.312–314. [Online]. Available: http://dx.doi.org/10.1109/educon.2014.6826109

[9] P. Abreu, M. Romano Barbosa, and A. Mendes Lopes, “Roboticsvirtual lab based on off-line robot programming software,” in 2ndExperimenta International Conference (exp.at’13), 2013, pp. 109–113.[Online]. Available: http://dx.doi.org/10.1109/expat.2013.6703040

[10] D. Bagchi, K. Kaushik, and B. Kapoor, “Virtual labs for electronicsengineering using cloud computing,” in 2013 3rd InterdisciplinaryEngineering Design Education Conference (IEDEC), Mar. 2013,pp. 39–40. [Online]. Available: http://dx.doi.org/10.1109/iedec.2013.6526757

[11] A. V. Fidalgo, G. R. Alves, M. A. Marques, M. C. Viegas, M. C.Costa-Lobo, U. Henandez-Jayo, J. Garcia-Zubia, and I. Gustavsson,“Adapting Remote Labs to Learning Scenarios: Case StudiesUsing VISIR and RemotElectLab,” IEEE Revista Iberoamericana deTecnologias del Aprendizaje, vol. 9, no. 1, pp. 33–39, Feb. 2014.[Online]. Available: http://dx.doi.org/10.1109/rita.2014.2302071

[12] R. Bose and P. Pawar, “Virtual Wireless Lab Concept, design andimplementation,” in IEEE International Conference on TechnologyEnhanced Education (ICTEE), Jan. 2012, pp. 1–7. [Online]. Available:http://dx.doi.org/10.1109/ictee.2012.6208669

[13] P. D. Sutton, J. Lotze, H. Lahlou, S. A. Fahmy, K. E. Nolan, B. Ozgul,T. W. Rondeau, J. Noguera, and L. E. Doyle, “Iris: an architecturefor cognitive radio networking testbeds,” IEEE communicationsmagazine, vol. 48, no. 9, pp. 114–122, Sep. 2010. [Online]. Available:http://dx.doi.org/10.1109/mcom.2010.5560595

[14] L. Rosniak Tibola, “A proposal of engineering education architecture:Improve engineer’s competencies through practice labs and 3Dvirtual worlds,” in 2th IEEE International Conference on IndustrialInformatics (INDIN), Jul. 2014, pp. 791–794. [Online]. Available:http://dx.doi.org/10.1109/indin.2014.6945615

Fig. 11. The HTML-based hands-on remote experimentation based course