* Dissemination Level: PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other program participants (including the Commission services), CO= Confidential, only for members of the Consortium (including the Commission services) ** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other Deliverable D7.1 Title: Training Curricula Dissemination Level: PU Nature of the Deliverable: R Date: 30/09/2019 Distribution: WP5 Editors: UPV Reviewers: PSNI, PPM Contributors: UPV, CBRNE Abstract: The curricula composed by MAGNETO for training of LEA personnel Funded by the Horizon 2020 Framework Programme of the European Union MAGNETO - Grant Agreement 786629 Ref. Ares(2019)5742851 - 13/09/2019
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
* Dissemination Level: PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other
program participants (including the Commission services), CO= Confidential, only for
members of the Consortium (including the Commission services)
** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other
Deliverable D7.1
Title: Training Curricula
Dissemination Level: PU
Nature of the Deliverable: R
Date: 30/09/2019
Distribution: WP5
Editors: UPV
Reviewers: PSNI, PPM
Contributors: UPV, CBRNE
Abstract: The curricula composed by MAGNETO for training of LEA personnel
Funded by the Horizon 2020 Framework
Programme of the European Union
MAGNETO - Grant Agreement 786629
Ref. Ares(2019)5742851 - 13/09/2019
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 2 of 104
Disclaimer
This document contains material, which is copyright of certain MAGNETO consortium parties and may not
be reproduced or copied without permission. The information contained in this document is the
proprietary confidential information of certain MAGNETO consortium parties and may not be disclosed
except in accordance with the consortium agreement.
The commercial use of any information in this document may require a license from the proprietor of that
information.
Neither the MAGNETO consortium as a whole, nor any certain party of the MAGNETO consortium
warrants that the information contained in this document is capable of use, or that use of the information
is free from risk, and accepts no liability for loss or damage suffered by any person using the information.
The contents of this document are the sole responsibility of the MAGNETO consortium and can in no way
be taken to reflect the views of the European Commission.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 3 of 104
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 4 of 104
List of Authors
Partner Author
UPV Francisco Pérez, Victor Garrido
CBRNE Tony Godwin, Nigel hale
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 5 of 104
Table of Contents Revision History ............................................................................................................................................ 3
List of Authors ............................................................................................................................................... 4
Table of Contents .......................................................................................................................................... 5
Index of figures ............................................................................................................................................. 9
Index of tables ............................................................................................................................................. 10
Personal details ................................................................................................................................... 49
Expectations on MAGNETO Training ...................................................................................................... 94
Type of Class ....................................................................................................................................... 94
Figure 6. Biggs and Tang analysis of “Tutor and Student perspectives”..................................................... 28
Figure 8. Evaluation criteria and KPI definition .......................................................................................... 30
Figure 9. Definition of a subject as General and Specific objectives .......................................................... 33
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 10 of 104
Index of tables Table 1. Example of training activities ........................................................................................................ 19
Table 2. Identified Training Tools ................................................................................................................ 20
Table 3. MAGNETO Training Curriculum ..................................................................................................... 35
Table 4. General Knowledge Objectives Session 1 ..................................................................................... 36
Table 5. General Knowledge Objectives Session 2 ..................................................................................... 36
Table 6. General platform Objectives ......................................................................................................... 37
Table 7. Video Analysis Objectives .............................................................................................................. 38
Table 9. Text analysis Objectives ................................................................................................................ 40
Table 10. Learn the features of the scenarios Objectives........................................................................... 40
Table 11. Equality and non-discrimination Objectives ............................................................................... 41
Table 12. Privacy and data protection Objectives ...................................................................................... 42
Table 13. Criminal procedure law Objectives ............................................................................................. 42
Table 11. Training scheduling ..................................................................................................................... 43
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 11 of 104
Glossary
ADL Assisted Distance Learning
DDP Deliverable Development Plan
EAB External Advisory Board
EC European Commission
ETHAB Ethics Advisory Board
GA General Assembly
GDPR General Data Protection Regulation
LEA Law Enforcement Agency
KoM Kick-off Meeting
LMS Learning Management System
MoM Minutes of Meeting
PMC Project Management Committee
PO Project Officer
QPR Quarterly Periodic Report
RTF Result Transferability Framework
SAB Security Advisory Board
STAB Scientific and Technical Advisory Board
TPE Threat Prediction Engine
WP Work Package
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 12 of 104
Executive Summary MAGNETO is a project founded by the European Commission within the H2020 addressing topic SEC-12-
FCT-2017 “Technologies for prevention, investigation, and mitigation in the context of fight against crime
and terrorism”.
Both video or audio recording, and text mining have become a major resource for legal investigations.
Considering the growing complexity, diversity and overlap of the video-surveillance systems, audio
recordings or huge amount of data coming from external databases or digital documents, the MAGNETO
project has the goal of designing and developing a multimedia analysis platform to simplify these
procedures.
In particular, the main goal of the WP7 “MAGNETO’s Training” is to provide MAGNETO end-users, also
known as LEAs (Law Enforcement Agency), the necessary training to go beyond their current multimedia
analysis knowledge and also cover their needs to successfully handle the MAGNETO tools, i.e the
Multimedia Analysis Platform and the different Multimedia tools to successfully follow the MAGNETO
proposed scenarios.
Figure 1. MAGNETO Training Programme user driven approach.
Deliverable D7.1 “Training Curricula” covers the first two steps of the MAGNETO training development,
as seen in the Figure 1. It aims a first step in defining the training methodology for better practice and
knowledge acquisition of each of the skills proposed by the end-users and the technical partners.
Additionally, methods for measuring up to what extent the training goals have been accomplished have
also been defined and will be adapted to the MAGNETO tools.
The second step within the MAGNETO training development is the definition of a training curriculum. This
curriculum will include several subjects covering all the necessary aspects for the end-users in order to
take full advantage of the MAGNETO platform. It will also cover the subjects scheduling and the necessary
prerequisites to access each subject.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 13 of 104
1. Introduction This deliverable is divided in 6 Sections, which include the training questionnaire analysis, the training
methodology, the assisted distance-learning platform, the training evaluation methods and criteria, the
training curriculum definition and finally the training scheduling.
In chapter 2, a study was undertaken to understand current end users training and acquisition
methodologies on multimedia analysis. With the provided questionnaire, the LEAs have been directly
involved in helping identifying and also describing the indicators that allow a relevant assessment in terms
of improving the required skills. Based on the study´s results, an ad hoc training methodology was
definied.
Chapter Σφάλμα! Το αρχείο προέλευσης της αναφοράς δεν βρέθηκε. has been dedicated to defining a
training methodology by analizing/observing different methods and environments. Several learning
methods such us face-to-face workshops, digital manuals, webinars and others, have been studied and
selected to be included in the? MAGNETO Training.
Chapter Σφάλμα! Το αρχείο προέλευσης της αναφοράς δεν βρέθηκε. includes the study of a set of
“Assisted Distance Learning” (ADL) platforms or Learning Management Systems (LMS). As a result, a
platform was selected to host the MAGNETO Training Materials. This platform will provide LEAs members
a training environment according their needs and enable them to use the MAGNETO platform(?) and all
available tools. Moreover, the training will also cover legal, ethical and privacy issues.
Chapter 4 identifies a methodology for measuring the goals of the MAGNETO training and also the learning
progress of the LEAs. A step-wise progressive evaluation process will be incorporate to ensure that end
users acquire the knowledge to improve their skills on multimedia analysis and also on using the
MAGNETO platform and tools.
Chapter 6 contains the definition of the term curriculum in order to better understand the following
sections. Furthermore it includes the description of the key terms “subject”, “objective”, “prerequisite”
and “scheduling”, among others.
Finally, chapter 7 includes the proposal of a first draft for the MAGNETO training schedule divided by
natural days. Each day foresees a six hours of training.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 14 of 104
2. Questionnaire
2.1 Introduction This chapter presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing these questionnaires was to better understand the end-user requirements and needs related to the scope of the training within the MAGNETO Platform. The target group for the questionnaires were analysts since a previous questionnaire presented in Deliverable 2.2 identified them as the key user group for MAGNETO. Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. The questionnaire has assisted the training developers to produce relevant training in formats and ways
that are acceptable to the trainees.
In total there were 31 respondents to the questionnaires from ~10 organizations. Most of them were
national and municipal police agencies.
The Training Questionnaire report is attached to this document and can be found in Appendix B.
2.2 Questionnaire analysis Questions presented to the respondents were largely in the form of binary (yes/no) questions. There were
also a number of questions which required the respondent to select from a 5-point scale ranging from
“very dissatisfied” to “very satisfied” or “unimportant” to “critical”.
In these cases (?), to provide a useful indication of the results obtained across the respondents, answers
were processed into percentages using the simple expediency of calculating the sum of all assigned scores
as a percentage of the theoretical maximum. In the text these are referred to as ‘Compound Scores’.
High Compound scores are interpreted as revealing a consistent and strong affinity to the associated item,
whereas a low percentage score indicates a consistent lack of desire to see the item.
2.2.1 General Questionnaire analysis
Respondents represented a good cross section of age, academic qualification, experience and geographic
location. They had attended a significant number of previous training sessions. The implications of this
being that their(?) feedback is drawn from personal experience and technical knowledge of criminal and
counter-terrorism activities as well as forensics. Since these groups represent the target group of users
of the MAGENTO system, the results are hence important.
2.2.2 Background and Experience of Respondents
Video/Audio Analysis: Although respondents indicated familiarity with topics relating to video and audio
analysis, the examples they quoted were somehow limited.
Data Mining: Respondents indicated good familiarity with these topics and provided a good range of
examples.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 15 of 104
Augmented intelligence: Familiarity with this topic was limited and this was reflected in the small number
of examples quoted.
2.2.3 Technical and Organisational Capabilities
The responses indicate that technical capabilities are important to the respondents – perhaps more so for
in-office systems than mobile systems although there was also interest in these. Organisations are
dependent upon a range of resources including the need for co-operation and the use of open intelligence.
In this respect it should be pointed out that collaborations often occur, however not without difficulties
arising due to differences in terms of procedures and data formats.
2.2.4 Previous Training
Respondents have experience with a broad range of class types. There is however a clear preference for
‘guided’ classes including discussion and co-operative working features. These denotes that such sessions
are more valued when they include a strong practical aspect. There was less experience with on-line
platforms than with some other training types but these were also well received on the whole. Again, the
need for feedback and interaction was also noticed.
Training within the LEA’s own offices was seen as an option taking into consideration the added benefit
of reducing travel costs and time. Some also saw the benefit of being away from the office for such
trainings. In all cases, the need for good quality training materials and equipment as well as good
‘charismatic’ trainers and engaging materials was clearly pointed out. The ideal teacher to pupil ratio was
determined at 10 to 15 pupils per teacher.
The use of pc’s / computers was seen as important as well as the need of to keeping the ratio of users per
computer to no more than 2 pupils.
Legal and ethical training was seen as being important (by some) an at the same time participants pointed
out that in the past these topics have not been delivered as they should have.
In terms of evaluation of training, there is a lower incidence of essays and the like being used? by the
respondents and there is an associated lower affinity for these as well.
2.2.5 Expectations
Respondent’s expect guided simulation or similar, with well-presented reference material and guide-
books. There is a strong focus on training being practically focussed on real LEA work – if possible (?) using
examples from their own experiences. The use of video’s and on-line systems is not precluded, however
it is not the respondent’s preference – at least not for main-line training, perhaps more likely for initial
briefing and updates.
2.3 Conclusions: Questionnaire The questionnaire reported here has specifically focussed on the needs of analysts rather than on all
potential user groups identified in D2.2 since this group is the most likely to use the full range of
capabilities delivered by MAGNETO. Analysts are a largely office based user group. Hence the conclusions
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 16 of 104
presented here are focussed mainly on office usage. Although this excludes some of the potential uses of
MAGNETO, such as on mobile systems for example, it is nevertheless considered to be practicable and
appropriate for a system being developed for TRL 6.
There is clearly a strong interest in a wide and diverse range of training modalities, from technical
documents for self-education to face-to-face training. Participants prefer using Webinars for remote
training rather than videos and also value interaction and feedback features.
Noting again that the system is only intended to be TRL6 and that the training itself can therefore not be
fully developed within the project, a mixed Economy of training methodologies – i.e. one that recognises
the project’s finite budget and the practicalities of getting together groups of analysts from up to ten LEAs
across Europe – is recommended. In all cases there should be a strong focus on practical real-life
examples.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 17 of 104
3. Training methodology
3.1 Introduction Magneto training methodology is developed in order to meet the end users training needs on multimedia
analysis. This methodology will establish suitable training environments for increasing the knowledge
acquisition on the use of Magneto tools.
To have a clear idea about the selected methodologies, it is important to comprehend the process of
learning. The first subsection will cover this topic.
3.2 Learning and Teaching Strategies The old-style seminar lessons where the tutor just explains or lectures to their students are no longer
considered to be effective. This was an important aspect to be considered in the evolution process since
the estimated student attention span for the teaching style, “just sit there and listen” is of less than half
an hour.
Recent studies [1], have demonstrated that learning is a circular process. As David A. Kolb [2] pioneered
the idea of experiential learning, which is learning by reflection on doing experiential learning’s focus is
the individual learning process.
Figure 2. Training user driven approach
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 18 of 104
Figure 2 demonstrated the 4MAT model or the way to label the four types of learners:
• WHY …? Need reason or relevance
• WHAT FOR …? Need additional information and usefulness
• HOW …? Need training to comprehend
• WHAT IF …? Need to investigate it themselves or teach others
As David A. KOLD[2] explains, the best learning (results?) is achieved when performing the displayed
training repeatedly.
Therefore, as a tutor/trainer, one must consider the specific order set out in the diagram while applying
the 4MAT model to follow(?). In summary, in order to keep the attention of the “WHY student”, first the
explanation of “WHY the topic at hand is important” has to first be done. Next, the tutor has to centre on
the “WHAT” and “HOW students”, as they are more patient. The other (who?) between these two comes
from it is also logical to first explain the theory (WHAT) and then the application process (HOW). Finally,
the “WHAT IF students” are rather interested in obtaining more information and therefore need to be
placed at last.
Following on from the topics of communication methods and learning mechanisms, it is necessary to
introduce the Neuro-Linguistics Programming (NLP) models. By using the techniques provided by the NLP
models, it is possible to improve the communication between tutors and students. Therefore, the training
has to implement several communication channels and not just the traditional ones used in seminar
lectures, i.e. auditory word elements but also visuals and practical ones. Considering these aspects, the
classic NLP model proposes the following pyramid for subdividing the learning experience.
Neuro-Linguistics Programming
Students remember
Figure 3. Neuro-Linguistic Programming (NLP) applied to education
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 19 of 104
Figure 3 demonstrates how the NLP can be applied to educations (the education field/ education?). It is
important to notice that seeing (50%) or seeing and hearing (50%) are fairly effective, but doing or
teaching (90%) however is the best experience for the students to learn and remember a specific topic.
Also, when implementing the learning experience pyramid, the teaching procedure should be arranged
as follows: “First see how something is done, then do it yourself, and, finally, teach it to others”.
3.3 Conclusions: Training Activities Considering the knowledge gained from the previous section, it is recommended to adopt a variety of
training activities to reach students, and to do so in a diverse manner. In summary, it is important that
enough TO DO (TEACH), WRITE (SAY), SEE, HEAR and then READ are included in the learning experience.
As such, the number of methods that can be adopted is quite high. The next table shows some examples.
Table 1. Example of training activities
DO (TEACH) WRITE (SAY) SEE HEAR READ
Small Groups X X X X
Simulations X X X
Case Study X
Practice by doing X
Exercises X X
Sharing Experiences X X X
Group Discussions X X X
Brainstorm X X
Teaching others X
Thinking time X
Demos X X
Lectures X
Further Reading X X
Unfortunately, the scope of the MAGNETO project does not give enough room for implementing all the
above-mentioned activities. There is also the requirement of delivering an efficient and cost effective
training programme that can be delivered (applied?) across multiple Law Enforcement Agencies.
Combining the user-selected preferences (see previous chapter Questionnaire
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 20 of 104
3.4 Introduction This chapter presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing these questionnaires was to better understand the end-user requirements and needs related to the scope of the training within the MAGNETO Platform. The target group for the questionnaires were analysts since a previous questionnaire presented in Deliverable 2.2 identified them as the key user group for MAGNETO. Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. The questionnaire has assisted the training developers to produce relevant training in formats and ways
that are acceptable to the trainees.
In total there were 31 respondents to the questionnaires from ~10 organizations. Most of them were
national and municipal police agencies.
The Training Questionnaire report is attached to this document and can be found in Appendix B.
3.5 Questionnaire analysis Questions presented to the respondents were largely in the form of binary (yes/no) questions. There were
also a number of questions which required the respondent to select from a 5-point scale ranging from
“very dissatisfied” to “very satisfied” or “unimportant” to “critical”.
In these cases (?), to provide a useful indication of the results obtained across the respondents, answers
were processed into percentages using the simple expediency of calculating the sum of all assigned scores
as a percentage of the theoretical maximum. In the text these are referred to as ‘Compound Scores’.
High Compound scores are interpreted as revealing a consistent and strong affinity to the associated item,
whereas a low percentage score indicates a consistent lack of desire to see the item.
3.5.1 General Questionnaire analysis
Respondents represented a good cross section of age, academic qualification, experience and geographic
location. They had attended a significant number of previous training sessions. The implications of this
being that their(?) feedback is drawn from personal experience and technical knowledge of criminal and
counter-terrorism activities as well as forensics. Since these groups represent the target group of users
of the MAGENTO system, the results are hence important.
3.5.2 Background and Experience of Respondents
Video/Audio Analysis: Although respondents indicated familiarity with topics relating to video and audio
analysis, the examples they quoted were somehow limited.
Data Mining: Respondents indicated good familiarity with these topics and provided a good range of
examples.
Augmented intelligence: Familiarity with this topic was limited and this was reflected in the small number
of examples quoted.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 21 of 104
3.5.3 Technical and Organisational Capabilities
The responses indicate that technical capabilities are important to the respondents – perhaps more so for
in-office systems than mobile systems although there was also interest in these. Organisations are
dependent upon a range of resources including the need for co-operation and the use of open intelligence.
In this respect it should be pointed out that collaborations often occur, however not without difficulties
arising due to differences in terms of procedures and data formats.
3.5.4 Previous Training
Respondents have experience with a broad range of class types. There is however a clear preference for
‘guided’ classes including discussion and co-operative working features. These denotes that such sessions
are more valued when they include a strong practical aspect. There was less experience with on-line
platforms than with some other training types but these were also well received on the whole. Again, the
need for feedback and interaction was also noticed.
Training within the LEA’s own offices was seen as an option taking into consideration the added benefit
of reducing travel costs and time. Some also saw the benefit of being away from the office for such
trainings. In all cases, the need for good quality training materials and equipment as well as good
‘charismatic’ trainers and engaging materials was clearly pointed out. The ideal teacher to pupil ratio was
determined at 10 to 15 pupils per teacher.
The use of pc’s / computers was seen as important as well as the need of to keeping the ratio of users per
computer to no more than 2 pupils.
Legal and ethical training was seen as being important (by some) an at the same time participants pointed
out that in the past these topics have not been delivered as they should have.
In terms of evaluation of training, there is a lower incidence of essays and the like being used? by the
respondents and there is an associated lower affinity for these as well.
3.5.5 Expectations
Respondent’s expect guided simulation or similar, with well-presented reference material and guide-
books. There is a strong focus on training being practically focussed on real LEA work – if possible (?) using
examples from their own experiences. The use of video’s and on-line systems is not precluded, however
it is not the respondent’s preference – at least not for main-line training, perhaps more likely for initial
briefing and updates.
3.6 Conclusions: Questionnaire The questionnaire reported here has specifically focussed on the needs of analysts rather than on all
potential user groups identified in D2.2 since this group is the most likely to use the full range of
capabilities delivered by MAGNETO. Analysts are a largely office based user group. Hence the conclusions
presented here are focussed mainly on office usage. Although this excludes some of the potential uses of
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 22 of 104
MAGNETO, such as on mobile systems for example, it is nevertheless considered to be practicable and
appropriate for a system being developed for TRL 6.
There is clearly a strong interest in a wide and diverse range of training modalities, from technical
documents for self-education to face-to-face training. Participants prefer using Webinars for remote
training rather than videos and also value interaction and feedback features.
Noting again that the system is only intended to be TRL6 and that the training itself can therefore not be
fully developed within the project, a mixed Economy of training methodologies – i.e. one that recognises
the project’s finite budget and the practicalities of getting together groups of analysts from up to ten LEAs
across Europe – is recommended. In all cases there should be a strong focus on practical real-life
examples.
) with the materials and the identified needs regarding the trained tools (see Table 2), it was decided to
implement the following methodologies for the MAGNETO project.
Table 2. Identified Training Tools
Training tool WP.
Ref
Training required Technical
manual
Webinars F2F workshops
External Interfaces WP6 Y X
Background Processing WP3 N
Distinctive Region and Pattern WP3 Y X x
Heterogeneous Data analysis WP3 Y X
Spatio-temporal search of DROP features
WP3 Y X X
Data Storage and Processing WP3 Y X X
Text Translation WP3 Y X X
Semantic information processing interface
WP4 Y X X
Information fusion WP4 Y X X
Evidence discovery WP4 Y X X
Data interaction WP4 Y X X
Situation awareness WP5 Y X X X
Courtproof-awareness WP5 Y X x X
Taking into account the selected methodologies, the training materials could include PDF technical
manuals, on-line presentations through and on-line platform (Webinar) and face-to-face workshops (F2F).
It is important to notice that in parallel to the development of MAGNETO tools, the MAGNETO Training
contents will be simultaneously generated. Hence some of the selected activities could change in the
future.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 23 of 104
Figure 4. MAGNETO proposed activities
Regarding MAGNETO F2F workshops, locations or dates have not yet been decided. The purpose of these
workshops is to resolve any issues identified, which are not covered by either technical manuals or
webinars.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 24 of 104
4. Assisted Distance Learning platform
4.1 Introduction Due to the international participation in MAGNETO, all the training material will be hosted in an online
learning platform known as a Learning Management System (LMS), with the aim of making easy for the
end-user to access and benefit from it.
Figure 5. MAGNETO Training methodologies and materials
The first fully featured Learning Management Systems (LMS) called EKKO was introduced in 1991. It was
developed and released by Norway’s NKI Distance Education Network. Later on,, New Brunswick’s NB
Learning Network (Rutgers University, NJ, EEUU) presented their own system. In the last 30 years, the
industry has experienced rapid growth, increasing the number of LMS from 15 to more than 700 vendors
on the market nowadays.
4.2 Learning Management System (LMS) LMS platforms are virtual learning spaces created to facilitate the distance training experience, both for
educational institutions and companies. The system chosen allows the creation of “virtual classrooms”,
where the interaction between tutors and students takes place. Users can also undertake evaluations,
exchange files and participate in forums and chats as well as many other additional tools.
Below are listed some of the benefits of LMS platform:
▪ Study at any place and time: this solves the problem of geographic or temporal distances and
offers great freedom in terms of time and pace of learning.
▪ Cheap and flexible way to expand the training.
▪ User friendly application: no great previous knowledge is required in order to use it.
▪ Constant updated learning through the interaction between tutors and students.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 25 of 104
Figure 6. LMS functionalities
The basic components or characteristics of any virtual environment, which must also be strongly linked
and interconnected in such a way that they influence each other and provide feedback, can be
summarized in the following:
▪ Centralization and automation of learning management.
▪ Flexibility: the platform can be adapted to the study plans of the institution as well as to the
contents and pedagogical style of the organization. It also allows the trainer to quickly and easily
organize courses.
▪ Interactivity: the user becomes the protagonist of her/his own learning experience through self-
service(?) and self-guided services.
▪ Standardization: this feature allows using courses made by third parties, personalizing the content
and reusing the knowledge.
▪ Scalability: these resources can work with a variable number of users according to the needs of
the organization.
▪ Functionality: features and features(?) that make each platform suitable (functional) according to
the requirements and needs of each individual user.
▪ Usability: ease with which people can use the platform in order to achieve a specific goal.(?)
▪ Ubiquity: ability of a platform to generate peace of mind for the users and make him certain that
everything he needs will be found in that virtual environment. (?)
▪ Integration: the LMS platform must be able to integrate with other business applications used by
human resources and accounting allowing thereby measuring the impact, efficiency, and above
all, the cost of training activities.
Once the basic characteristics of a LMS have been determined, the next step is to (will be?) decide which
LMS to use from all the available ones. One of the main considerations when choosing a LMS platform is
the cost aspect. According to that, LMS platforms are divided into two main types: licensed LMS and Free
GPL platforms open for educational purposes?. However, there are many other important aspects to take
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 26 of 104
into consideration too when choosing a LMS. Hence deciding which platform best suits the user´s needs
can become a complex task.
Comparing suppliers can be also a demanding task due to the high number of options available. High-end
options do not always guarantee quality and some solutions which offer quick and easy-to-install versions
with basic functionalities, may not meet the organization´s needs in the future.
In summary, although each application is different, there are a number of common factors:
▪ Licensed or free of use
▪ Evaluation of functionality and intention of use
▪ Infrastructure
▪ Contents: activities, configuration of the exercises, interface, interaction, compatibility and
maintenance.
Below, there is a brief description of the LMS platforms that were documented during the research phase.
LMS under license
▪ Blackboard
Blackboard stands out among the proprietary systems. One of the main advantages lies in the great
flexibility it offers due to the many options available for students to learn according to their own style
and rhythm. Blackboard’s work philosophy is very ambitious and its developers team aims at working
together with students and trainers to continuously improve it.
Open LMS
▪ Sakai
Sakai is an international community that collaborates to create technology that improves teaching,
learning and research. The management Committee of the Sakai project is formed by a group of
individuals belonging to various institutions (universities, schools, government, hospitals, etc.) that
provide the necessary leadership for the good direction of the project.
▪ Moodle
Moodle is a project directed and coordinated by an Australian organization of 30 developers, which
is supported financially by a worldwide network of about 60 companies. The learning platform is
designed to provide educators, administrators and students with a unique, robust and secure
integrated system to create personalized learning environments. Moodle can be downloaded on the
web server itself and it offers an assistance service from a Moodle partner. Moodle is the most used
learning platform in the world. It has currently more than 65 million users worldwide.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 27 of 104
4.3 Conclusions: (Moodle) As a result of an exhaustive study on the various LMS, it was concluded that in general both open LMS
(Sakai and Moodle) platforms support the needed features for this project. At the end we decide to choose
Moodle due to its large community of active contributors with the aim of continuously improve and
optimize the platform and also due to its large number of modules available.
As previously explained, Moodle is an open source LMS that supports the creation of fully customizable
learning environment. It was designed to cover the needs of educators, students and administrators. It
is a web-based platform developed in PHP technology and MySQL databases.
Moodle provides a large number of features making it the most used LMS in the world:
▪ Solid and confident: Moodle is used for all type of organizations to manage their learning projects
online.
▪ Easy to use: it provides a simple interface to navigate and management the contents with a drag-
and-drop functionality.
▪ Flexible and customizable: as an open source with modular framework it can be customized and
adapted to the individual needs.
▪ Collaborative tools and activities: students and educators can interact and contribute through
tools like chat, forums, surveys, etc.
▪ Always updated by a large community to be aware the requirements of the users (?).
▪ Ubiquitous and multi-platform: the access to the platform is provided by web and its interface is
responsive. Thus it is possible to connect to it by any device or place.
▪ Robust, safe and private: safety controls of the platform controls are updated constantly in order
to protect the privacy and data integrity. For this purpose, Moodle includes system protection
against unauthorized access, data loss or fraudulent use.
The tools available in the platform can be divided into two large groups:
- Resources: used to present information to students
- Activities: used to build to build community and apply the knowledge learned
The most important tools are:
▪ Blackboard Collaborate Classrooms: is a virtual classroom system that allows for complex online
interaction.
▪ Blog: an online journal with content sorted chronologically. Blogs can be groups or an individual
effort(?).
▪ Calendar: all-in-one calendar that can display all the events needs in the course created by
students, administrators or educators.
▪ Chat: real-time discussions for students where they can directly interact with educators.
▪ Database: students and educators collaborate to build a searchable bank of record entries
relating to any topic.
▪ Feedback: students can give their impressions and opinions through surveys, anonymous or not.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 28 of 104
▪ Lesson: create programmed learning units that allows tutors to compose the different courses.
▪ Quiz: different types of quizzes to evaluate the knowledge learned.
• Reports: a helpful tool to provide educators extra information on the use of their courses through
logs, live logs, activity report, course participation and statistics .
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 29 of 104
5. Training evaluation methods and criteria
5.1 Introduction In the final phase of the Magneto project a gradual evaluation process will be established in order to
ensure that the LEAs have acquired the required knowledge for improving their skills on multimedia
analysis.
The first subsection provides an overview of the different evaluation methodologies and strategies in
order to a better understand the selected evaluation criteria.
5.2 Evaluation methodologies and strategies Evaluation is a topic which, in the past, has generated considerable discussion and caused significant
controversy. However, it seems that there is some consensus: evaluation is necessary in order to estimate
or measure the development of a process whose final result is strongly determined by the assessment of
its own stages. Therefore, the evaluation is an essential instrument in the development of the teaching
function.
With regard to the activities, the evaluation is a good indicator, which guides the path of “know-how”,
pointing out alternative options in the search of consequent and operational solutions.
The main characteristics of a good evaluation are as follows:
▪ Explicit
There should be no doubts about what is being evaluated.
▪ Grounded
The necessary reasons must be gathered, so they support the assessment according to the
purposes and objectives of both, the work and the student.
▪ Transparent
Students must know how they will be evaluated as how it will be done.
▪ Additive
The questions should be preferentially independent and contribute separately to the total
performance.
In conclusion, there are no “right” or “wrong” evaluation methods, but there are better and worse ones,
or at least more useful and less useful ones.
Another important aspect to discuss regarding the evaluation methods is the “assessments-objective
alignment”.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 30 of 104
Figure 7. Biggs and Tang analysis of “Tutor and Student perspectives”
As Biggs and Tang said in 2009 [7], usually the objectives of the students and the tutors are not the same.
As seen in the Figure 7, the first concern of the tutor is to achieve some objectives about the course.
Subsequently, they perform the teaching activities and at the end an evaluation of the student. On the
other hand, students are concerned about their evaluation and how to achieve a score for their tests
through learning activities. Thus, it is necessary to align the assessment with the objectives of the course.
Once the objectives have been aligned with the evaluation, the tutor has to include a feedback system
that allows him to change/ adapt the training accordingly. A continuous evaluation methodology is
adopted for this purpose. Since the introduction of the Bologna Process in 1999, there are many European
universities that have included this revolutionary system. Through Bologna the student’s evaluation in
which they were evaluated through regular tests of knowledge assimilation, was abolished. By this, tutors
assume the role of counsellor and assess the student’s learning process not merely by sanctioning their
results but by continuously monitoring their work, and hence helping them meeting their objectives.
Furthermore, this system also considers the “student’s competences”, that it is to say, that the students
are capable to demonstrate comtpetency in specific areas.(?)
Thus, although final exams may be utilized, they will not be the unique evaluation method. Therefore, the
procedures designed to facilitate monitoring the student´s learning progress during their academic years
were varied(?).
Several mechanisms are found in the education literature to support continuous evaluation:
▪ Tutor Evaluation
Historically, evaluation has been almost the exclusive preserve of the tutor, who, based on his or
her experience, issues an “authorized opinion”, which allows the student to be placed within an
evaluative framework, linked to the “assignment” or previously established request. This is the
most know evaluation method and is still most of the common ones.
▪ Auto-Evaluation
In the last few years, the concepts of self-evaluation and co-evaluation processes were developed.
The idea behind this is for the students to make judgments of value to their own work, as well as
that of their peers. To guide the students in this task usually questionnaires are especially
designed for this purpose. This situation requires the students, among other things, to
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 31 of 104
demonstrate objectivity and equanimity, appealing to conscience and maturity. The same
operation can be implemented with groups, where the evaluation was focused on the teammates.
In most common cases, objectivity is not threatened, and sometimes can result in downgrade
evaluation.
▪ Co-Evaluation
By sharing, the affirmation, that the ultimate function of teaching is to achieve the “autonomy”
of the student, one would also agree that what is relevant within the learning system, would also
be to involve the student through their self-criticism, introducing into the trial process their own
self-assessment.
Finally, the possibility of providing different assessment states for the same work or developing exercises
(self-evaluation, cooperative evaluation and tutor evaluation) adds a greater certainty of appreciation of
the progress achieved and allows crossing data and information.
5.3 Conclusions: Evaluation methods, criteria and KPI The key objective is to incorporate in the Magneto training similar evaluation methods applied to other
universities and professional (i.e. CISCO, Palo Alto, etc.) courses performed in the UPV. The preferred
evaluation method will the Test/Quiz/Exam, but as implied previously the “final exam” will be only one
part of the total evaluation.
At the end of each topic (technical manuals and webinars), the user will have the chance to do a no-graded
small quiz. The answers to the test will be given at the end of it. This quiz can be done several times and
serves as an auto-evaluation mechanism for the student to measure their understanding and learn the
most important content of a subject.
Then, at the end of each chapter there will be a graded test. that will cover only the knowledge of this
chapter. In this case the test can be done twice since the answers will not be provided.
At the end of the training, and once the students have passed the chapter tests, they will have to pass a
final test that will gather questions from all the previous ones. The students will only receive the grade at
the end. This test can be undertaken only once.
The summative evaluation will take into account the results of the chapter tests and final test. Each part
will cover a percentage of the final grade, 40% to 60%, respectively. In addition, each individual test will
need to be passed with a score of at least 50% to successfully complete the course.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 32 of 104
Figure 8. Evaluation criteria and KPI definition
Following the scheme depicted in Figure 8 and considering the available information on the features and
design of the Magneto tools, the KPI can be categorized and defined as follows:
General Objective
Improve the current knowledge on heterogeneous data mining techniques of the MAGNETO LEA’s
▪ Specific Objective
o Gain knowledge to properly use the MAGNETO Distinctive Region and Pattern tool
▪ Configure and manage the basic features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Configure and manage the extended features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Improve the background knowledge on Heterogeneous Data analysis tool
▪ Understand basic heterogeneous data analysis concepts applied to the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Improve data storage knowledge
▪ Learn how to store, modify, and delete files properly on the platform
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Gain knowledge to properly use the MAGNETO Text Translation tool
▪ Configure and manage the basic features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Configure and manage the extended features of the software
KPI: 80% students pass with a grade greater than 6/10
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 33 of 104
General Objective
Improve the current knowledge advanced (?) semantic reasoning of the MAGNETO LEA’s
▪ Specific Objective
o Gain knowledge to properly use the MAGNETO Semantic Information Processing tool
▪ Configure and manage the basic features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Configure and manage the extended features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Gain knowledge to properly use the MAGNETO Information Fusion tool
▪ Configure and manage the basic features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Configure and manage the extended features of the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Improve the background knowledge on Evidence Discovery techniques
▪ Understand evidence discovery techniques applied to the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Improve the background knowledge on Data Interaction
▪ Learn to correctly interpret pre-processed data
KPI: 80% students pass with a grade greater than 6/10
General Objective
Improve the current knowledge on augmented intelligence of the MAGNETO LEA’s
▪ Specific Objective
o Gain knowledge on Situation Awareness
▪ Learn to read an interpret geographic information
KPI: 80% students pass with a grade greater than 6/10
▪ Learn to manage, represent and visualize data from the different sources
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Gain knowledge on managing Court-proof Evidences
▪ Learn how to understand and manage control files
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Gain knowledge on integration and interoperability
▪ Learn how to integrate and interoperate the MAGNETO platform with other
systems and data sources
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 34 of 104
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o Improve the knowledge on the legal procedures, privacy and ethical principles to ensure
it will be accomplished during the MAGNETO project
▪ Understand evidence discovery techniques applied to the software
KPI: 80% students pass with a grade greater than 6/10
▪ Specific Objective
o The training achieves an overall good satisfaction level from the students
▪ Survey passed to the students
KPI: 80% of the students are satisfied with the used training methodology
KPI: 80% of the students consider the materials adequate to the training
KPI: 80% of the students would recommend this training experience
Prior KPIs have been defined at an early stage considering a general approach. However, these KPIs
will be re-evaluated by the time the tools will be implemented and available for analysis.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 35 of 104
6. Training curriculum definition A Curriculum is the subjects comprising a course of study in a training process. The curriculum depicts the
sequence of steps, which will guide the coaches and trainees to the education target (learning goals?).
The curriculum describes the subjects, objectives and the necessary evaluating processes required. In
addition, the interaction between coaches and trainees would (will?) be incorporated, as well as the
materials, content and resources required.
The following items should be included in the curriculum:
• Number of subjects
• Order of the subjects
• Timing of all subjects
• Mentor of each subject
Therefore, the entire curriculum of training will cover several subjects, taking into account the
requirements and the increasing difficulty. In addition, it will be developed according to the necessary
time for each subject to examine all the material, exercises and prepare the evaluation. This is necessary
in order to create a schedule that is fit for purpose. Finally, a subject specialist (Mentor) will be assigned
to each subject in order to be a point of contact for concepts that the main coach could not handle
(during…?). The tutor will be responsible for recording issues to be resolved through weekly dialogue with
the specialist.
Figure 9. Definition of a subject as General and Specific objectives
The subject’s composition is showed in Figure 9. Each subject is divided into several general objectives.
Each of these general objectives could do refer to general topics. There can be as many general objectives
as required. For example, in a Math subject a general objective would be “Learn basic statistics”.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 36 of 104
At the same time, each general objective could (can?) be sub divided into specific objectives. Specific
objectives are created to specify only one single task. Following the same example, in this case, a specific
objective could be “Learn to divide two numbers”.
Specific objectives could have a set of prerequisites. Sometimes it is necessary to gain additional
knowledge in order to understand the topic. For example, in order to resolve the specific objective “Learn
to divide two numbers”, first the student would need to “Learn to add two numbers”.
Finally, each specific objective would have a determinate method to be evaluated. For example, in the
last examples, the best way could be to pass a test.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 37 of 104
7. MAGNETO Training Curriculum
7.1 Subject definition This section will cover the MAGNETO training curriculum. It is important to remember that this curriculum
may change in the following months while the Multimedia Analysis Platform and its key features are
further developed.
Table 3. MAGNETO Training Curriculum
No. Subject Name Prerequisite Mentor (Subject
Specialist)
Mentor
(Company)
Duration
1 General knowledge Not required Victor Garrido UPV 12 h
2 Platform configuration
and management
Subject 1 Francisco Pérez UPV 6 h
3 Video Analysis Subject 1 – 2 Francisco Pérez UPV 6 h
4 Audio Analysis Subject 1 – 2 Francisco Pérez UPV 6 h
5 Text Analysis Subject 1 – 2 Francisco Pérez UPV 6 h
6 Scenarios training Subject 1 – 2 Francisco Pérez UPV 18 h
7 Legal, ethical and
social
Not required Francisco Pérez UPV 12 h
Table 3 shows the seven subjects that have been identified for the MAGNETO training. They can divided
into three parts. The first one will incorporate the necessary background knowledge in order to continue
with the next part of the training (Subject 1). The second part encompasses all the subjects that cover the
Multimedia Analysis Platform and its tools (Subject 2 – 6). Finally, the last part, will provide the necessary
legal, ethical and social considerations (Subject 7).
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 38 of 104
7.2 Subjects objectives The subjects have already been defined, and now we have (we will?) to determinate their objectives, both
general and specific. The consideration (?) will contain objective, prerequisites and evaluation method.
7.2.1 Subject: General knowledge
This subject will include all the minimum necessary knowledge that is required in order to create the first
steps in the platform. For example, it will include the technical vocabulary, the use of encoders and data
formats as well as the most common (most commonly used) audio, video and text processing.
General Objective 1.1. – Learn the basic principles in multimedia tools.
This general objective encompasses the main principles in multimedia technology. For example, technical
vocabulary and different kinds of formats and encoders.
Table 4. General Knowledge Objectives Session 1
No. Specific Objectives Prerequisites Evaluation method
1.1.0 Multimedia Vocabulary Not required Quiz
1.1.1 Standard multimedia formats Not required Quiz
1.1.2 Basic video Not required Exercise
1.1.3 Basic audio Not required Exercise
1.1.4 Basic Text Not required Exercise
1.1.5 Upload files Not required Exercise
1.1.6 Basic graphs Not required Exercise
General Objective 1.2. – Learn the advanced principles in multimedia tools.
After learning the basics, this general objective covers more advanced principles in multimedia
technology. In this part, the student will interact with the multimedia data, common good practices and
format conversions.
Table 5. General Knowledge Objectives Session 2
No. Specific Objectives Prerequisites Evaluation method
1.2.0 Specificity of multimedia recordings General Objective 1.1 Quiz
1.2.1 Map orientation General Objective 1.1 Exercise
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 39 of 104
1.2.2 Basic scenarios General Objective 1.1 Exercise
1.2.3 Advanced functions General Objective 1.1 Exercise/Quiz
1.2.5 Basic architecture interactions 1 General Objective 1.1 Exercise/Quiz
7.2.2 Subject: Platform configuration and management
The MAGNETO Multimedia Analysis Platform is the center of MAGNETO training curricula. This tool covers
the main functionalities and will provide all the analysis tools developed in the project.
General Objective 2.1. – Learn the general functions of the platform.
This general target focuses on the core functionalities of the Multimedia Analysis Platform such as basic
The objective of the ethical and legal considerations module is to provide training and orientation to the
end-users of the MAGNETO Multimedia Analysis Platform and analytics tools regarding to the ethical and
legal limits of their deployment. The classes will teach how the tools(?) (container) can be deployed
respecting ethical and societal values, and their development into the law (?). However, the functionality
and contributions of the tools to law enforcement needs can still be retained (?). This part (of the training
(?) ) will focus on three main areas: non-discrimination, privacy and data protection as well as criminal
procedure law.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 43 of 104
General Objective 7.1. – Equality and Non-Discrimination.
The course on Equality and Non-Discrimination teaches Magneto users on equality and non-discrimination
matters when using the Magneto Multimedia Analysis Platform and analytics tools. Users begin the course
with an overview of the technical notion of equality as a principle of justice and the fundamental rights to
non-discrimination and equality, followed by EU equality principles pertinent to law enforcement
activities. The course also teaches from an equality perspective, how the deployment of analytics modules
can lead to unfair forms of differentiation and decision-making. On the other hand, it also explains how
discrimination can result out of the different stages of the analytics process.
Finally, the course will approach how to avoid ethnic profiling. According to international human rights
law, ethnic profiling is forbidden. Therefore, it is important for LEAs, and law enforcement data analysts
in particular, to gain a strong knowledge base on this topics too. The course will teach how specific
characteristics could be searched within multimedia data bases.
Table 11. Equality and non-discrimination Objectives
No. Specific Objectives Prerequisites Evaluation method
7.1.1 The Fundamental Right to Equality and
Non-Discrimination
Not required Quiz
7.1.2 Law Enforcement and EU Non-
Discrimination Law
Spec. Objective 7.1.1 Quiz
7.1.3 How Algorithms Differentiate Fair and
Unfair Forms of Differentiation
Spec. Objective 7.1.2 Quiz
7.1.4 Ethnic and Discrimination Profiling: How to
Avoid Ethnic Profiling
Spec. Objective 7.1.3 Quiz
General Objective 7.2. – Privacy and Data Protection.
The objectives of the privacy and data protection course are to inform and teach LEAs about the
fundamental rights to privacy and data protection, the General Data Protection Regulation and the Law
Enforcement Directive. This (?) is another key requirement as the collection and processing of personal
data are an integral component in the use of the MAGNETO Multimedia Platform and analytics tools.
The course will also address how these rights have shaped current EU data protection legislation, for
example such as the GDPR.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 44 of 104
The course will primarily focus on the EU legal framework and consider the ethics and illegality of mass-
surveillance.
Table 12. Privacy and data protection Objectives
No. Specific Objectives Prerequisites Evaluation method
7.2.1 The Fundamental Rights to Privacy and Data
Protection and EU Data Protection Law
Not required Quiz
7.2.2 EU data protection law: key definitions Spec. Objective 7.2.1 Quiz
7.2.3 EU Data Protection Law: Data Quality Principles Spec. Objective 7.2.2 Quiz
7.2.4 Specific Data Protection Requirements for LEAs Spec. Objective 7.2.3 Quiz
7.2.5 Biometrics, Automated Decision-Making
(Profiling) and EU Data Protection Legislation
Spec. Objective 7.2.4 Quiz
7.2.6 The ethics and illegality of Mass-Surveillance Spec. Objective 7.2.5 Quiz
General Objective 7.3. – Criminal Procedure Law.
The training module aims to teach the users of MAGNETO modules about the basic principles of criminal
procedure law. Although criminal procedure law is regulated primarily on the national level, these
courses will be provided on a multi-national level. For this reason, the first part will mainly focus on the
basic international principles of criminal procedure law and the right to fair trial in particular. However,
throughout the course and where relevant (?), national examples of criminal procedure law will be
provided. Users are advised to first follow the privacy and data protection course.
Table 13. Criminal procedure law Objectives
No. Specific Objectives Prerequisites Evaluation method
7.3.1 International Principles of Criminal Procedure
Law (The Right to Fair Trial)
Not required Quiz
7.3.2 Data Collection from multimedia sources Spec. Objective 7.3.1 Quiz
7.3.3 The legality of multimedia data as evidence in
court
Spec. Objective 7.3.2 Quiz
* Dissemination Level: PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other
program participants (including the Commission services), CO= Confidential, only for
members of the Consortium (including the Commission services)
** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other
8. MAGNETO Training Scheduling
The MAGNETO training has been scheduled to be completed in less than three working weeks (this is only a first proposal). That means 13 days of 6 work hours
a day. the multimedia analysis Users can skip the first lessons if they have already basic knowledge in multimedia analysis.
Personal details ................................................................................................................................... 49
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 76 of 104
Expectations on MAGNETO Training ...................................................................................................... 94
Type of Class ....................................................................................................................................... 94
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 77 of 104
Introduction This document presents a summary of the results of questionnaires returned to the MAGNETO project by LEA project partners. The purpose of issuing the questionnaires was to better understand the end-user requirements and needs related to the scope of the training in the MAGNETO Platform. The questionnaires were focussed on Analysts as a previous questionnaire presented in Deliverable 2.2 identified that these were perhaps the key user group for MAGNETO. This is an internal report to assist with the development of the Deliverable 7.1 (Training Methodology) and Deliverable 7.2 (Training Tools). Detailed responses from the respondents to the questionnaires are presented in a separate spreadsheet which is available to project partners on an as-needed basis. This summary report will help the training developers to produce relevant training in formats and ways
that are acceptable to the trainees.
In total there were 31 respondents to the questionnaires from ~10 organisations, mostly in the form of
national and municipal police.
The raw results of the questionnaires are summarised below along with brief summaries of the
implications of those results, where such comment is warranted. The request, question or prompt that
was used to solicit a response from the respondents is given at the start of each sub-section. Questions
presented to the respondents were largely in the form of binary (yes/no) or questions but there were also
a number which required the respondents to select from a 5 point scale. These were processed as
described in Section 1.1.
Where text answers were also provided by the respondents, to expand on their multiple choice answers,
key points from these are also presented, but readers are encouraged to read the full responses contained
in the spreadsheet if a fuller understanding is needed.
Occasionally, direct quotes from the respondents are re-produced below. These are shown in italics in
quotation marks e.g. “Thus”.
A note on the Interpretation of Results
Questions presented to the respondents were largely in the form of binary (yes/no) or questions but there
were also a number which required the respondents to select from a 5 point scale;
In respect of aspects of previous training that the respondents had received, the options were
Option
/ Score Satisfaction Explanation
0 Very dissatisfied The training will be improved if removed.
1 Dissatisfied Nice to have, but the training will be fully useful even without it.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 78 of 104
2 Matched Expectations If removed, the training will be only partially useful.
3 Satisfied This improved the training.
4 Very satisfied This was a key aspect of the training.
In respect of aspects of possible future training the options were
Option
/ Score Expectation Explanation
0 Unimportant With or without this, the training is exactly the same.
1 Wish Nice to have, but the training will be fully useful even without it.
2 Important Without this, the training will be only partially useful.
3 Serious Without this, the training will be usable but not useful.
4 Critical Without this, the training will be not useful at all.
In these cases, to provide a useful indication of the results obtained across the respondents, answers have
been processed into percentages using the simple expediency of calculating the sum of all assigned scores
as a percentage of the theoretical maximum. So, for example, if all 31 respondents select option 4 then
this would be converted to 100% (31*4 out of a theoretical maximum of 31*4) and one rated as ‘4’ by 5
of the users, ‘3’ by 15 of the users and ‘2’ by 6 users would be converted to 74%
[(5*4+15*3+6*2)/((5+15+6)*4)]. These are referred to in the text as ‘Compound Scores’
High Compound scores are interpreted as revealing a consistent and strong affinity to the associated item,
whereas a low percentage score indicates a consistent lack of desire to see the item. A mid-range
percentage score represents a mixed response. These scores can be used to set priorities for the design.
Note that not all respondents, responded to all questions, so that the sample size varies with question.
Sufficient data are presented so that these variances are clear.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 79 of 104
Personnel Details and Experience / Knowledge (Questions 1 to 8)
Respondents Age and Experience (Questions 1 to 3)
Request: Please complete the following form with your personal information.
Respondents were asked to indicate their age, relevant levels of academic qualifications, years of
experience and familiarity with the types of technologies likely to be relevant to the MAGNETO system.
These are summarised in the graphics below.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 80 of 104
Previous Training Courses in Forensic Tools (Question 4)
Request: Please indicate the number of previous training courses you have received in the
field of forensic tools.
21 respondents (68%) indicated a variety of previous training courses covering topics such as mobile
phone surveys, forensic computing, pc forensics/ data recovery as well as more general training in criminal
investigation and the use of industry standards such as i2 iBase and i2 Analyst from IBM and mapping
systems such as ArcGIS from esri. It is notable that 10 respondents indicated that they had not received
any previous training in forensic tools, but this does not mean that they have not received training in other
types of tools.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 81 of 104
Familiarity with Digital Video Analysis (Question 5)
Request: Please check bellow the Digital video analysis topics you are familiar with.
Request: Respondents were asked to provide text examples of video coding and video
encapsulation.
10 respondents (32%) provided examples of video coding which included items such as analogue to digital
conversion, metadata analysis, differences in video resolution codecs, frame rates and associated storage
requirements and use of standard software such as NACSPORT. 4 respondents (13%) provided examples
of video encapsulation referring to standards such as .avi, .mov and MPEG.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 82 of 104
Familiarity with Digital Audio Analysis (Question 6)
Request: Please check bellow the Digital audio analysis topics you are familiar with.
Request: Respondents were asked to provide examples of Audio Coding and Audio
Encapsulation.
9 respondents (29%) provided examples of audio coding which included items such as examples of
metadata, references to standards such as MP3, FLAC, M4A and AAC. A number of the respondents
indicated that audio coding was not relevant to their work area. 4 respondents (13%) provided additional
details of audio encapsulation but only ‘Linear pulse Code Modulation’ was mentioned by name.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 83 of 104
Familiarity with Text and Data Mining Topics (Question 7)
Request: Respondents were asked to indicate the Text and Data Mining topics they were
familiar with
Request: Respondents were asked to provide examples of Distributed Storage Platforms and
Text Gathering Techniques.
9 respondents (29%) provided examples of Distributed Storage Platforms which included items such
Microsoft Sharepoint (Microsoft), HedVig and SWARM, Dataminer (Oracle?), I2 (IBM) and Azure
(Microsoft). 9 respondents (29% but not exactly the same group of respondents) provided examples of
Text Gathering Techniques such as Python Web Scraping, ARMITAGE, CeWL and others.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 84 of 104
Familiarity with Augmented Intelligence (Question 8)
Request: Please check below the Augmented Intelligence topics you are familiar with.
Request: Respondents were asked to provide examples of the topics they were familiar with.
6 respondents (19%) provided additional responses to the request for examples but only one response
was directly relevant – mentioning haptic feedback – and 3 of the responses noted that the respondent
had no knowledge about Augmented Intelligence.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 85 of 104
Organization Details
Organization Type (Questions 1 & 2)
Areas of Interest (for your duties) (Questions 3,4,5)
Questions (3 and 4)
Percentage of respondents
answering “Yes”
Question 3: Do CRIMINAL activities belong to the area of your specific interest/duties?
84%
Question 4: Do TERRORIST attacks belong to the area of your specific interest/duties?
68%
Question 5: Do you perform any particular tasks in the field of counter criminal and terrorism activities?
61%
Respondents answering ‘Yes’ to question 5 indicated that they were involved with i) teaching police
officers (including teaching in the field of terrorism and counter crime) (2 respondents), ii) analysis of data
for presentation to other officers (6 respondents) iii) forensic analysis (2 respondents), iv) Threat and Risk
Analysis (2 respondents) and v) searching the internet for criminal activity (1 respondent).
Note that some of the respondents who answered ‘No’ to Question 5 and some of the respondents who
answered ‘Yes’ to Question 5, confirmed that they thought that the MAGNETO research would be
beneficial to their organisation and 1 respondent indicated that they thought that MAGENTO research
would not benefit their organisation.
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 86 of 104
What are the most important capabilities necessary to counter criminal and terrorist
activities? Please select the importance of each activity (Question 6)
Request: Respondents were asked to select the importance of a range of technical and
organizational capabilities.
Technical Capabilities
All capabilities scored highly for importance, the calculated compound scores (see section 1.1) are
Technical Capability
Compound Scores (%)
Analysis Tools 63
Information Systems 66
Technical resources 62
Mobile Platforms 52
Mobile devices 52
Sensors 47
Organizational and Individual Capabilities
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 87 of 104
All capabilities, apart from ‘Luck’ have compound scores at around 65 (see below), indicating significant
importance to the respondents.
Organizational and Individual Capabilities
Compound Scores (%)
Luck 42
Mind-set 63
Pro-activity 63
Awareness 62
Knowledge 63
Open Intelligence 63
Co-Operation 66
Procedures / policing 60
Human resources 59
Did your organization perform field collaborative experience with other organizations
(Question 7)?
27 of 30 respondents indicated that they had performed collaborative exercises and 3 said that they had
not.
In the affirmative case were such (collaborating) organizations from the same country
or different countries (Question 8).
20 of 31 respondents indicated that they had collaborated within the same country and 16 of 31 indicated
that they had collaborated outside of their own country (N.B. some respondents had collaborated both
within and out of their own country).
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 88 of 104
What have been the major issues regarding national and/or international co-
operation / communication (Question 9)?
Request: Respondents were asked to provide further text details where they had selected
‘others’.
One respondent indicated the need to use define an agreed set of specific terms before entering such
joint ventures, one respondent referred to different data protection acts in the single federal states, one
noted issues with knowledge sharing and platforms but a number also noted that many of the difficulties
can be easily overcome.
Which are your current acquisition methodologies on Video, Audio and/or Text
analysis (Question 10)
Respondents provided a range of responses covering bespoke forensic software (2 respondents), open-
source/freeware/’standard’ software (3 respondents) and manual analysis (2 respondents).
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 89 of 104
Questions on previous training that has been received
Which types of classes did you attend and how satisfied were you with them? (Question
4.1)
Respondents were asked to provide example of the worst and best aspects of the types of classes ranked
above.
Seminar (In person) – ‘Best’ and ‘Worst’ Respondent Comments
The best aspects of seminars were noted as being related to having face-to-face training (direct
interaction) and the possibility for attendees to ask questions. But it was also noted that this required
competent and charismatic teachers, a clear practical approach and the ability of the seminar leader to
‘dynamically modulate’ (adapt to attendees questions). Others noted that the ability to exchange ideas
with other attendees was also valuable.
The need to ensure that attendees had the requisite prior knowledge was noted as was the need for a
flexible schedule with sufficient time.
Video recorded lectures (On-Line lectures) – ‘Best’ and ‘Worst’ Respondent Comments
A key benefit of video recorded lectures was noted as being the ability of trainees to watch the teaching
content again (e.g. if missed first time or not fully understood first time). The lower financial and logistical
burden of such training (compared with other types) was also noted as a key benefit. On the negative
side many respondents noted that the lack of the possibility to ask questions and to seek clarification was
an issue. The need for technically good videos (clear language and video quality) was also noted.
Webinars (On-Line Lectures) – ‘Best’ and ‘Worst’ Respondent Comments
D0.0 Training Curricula
H2020-SEC-12-FCT-2017-786629 MAGNETO Project Page 90 of 104
Webinars were noted as being good for getting an initial introduction to a subject or for upgrading existing
knowledge but there was perhaps a suggestion that inflexibility and the ‘inability to ask questions’ was a
problem for ‘new’ learning. The possibility of ‘bringing together a vast and heterogeneous audience’ was
noted as a benefit as were the low financial and logistic burdens but the potential problems with differing
time zones among attendees was also noted. The need for attendees to be susceptible to distraction and
hence ‘The student must be very motivated to be constant’ was also noted. The potential for interaction
was noted as a benefit.
Guided simulations (Practical lectures with exercises) – ‘Best’ and ‘Worst’ Respondent
Comments
Respondents noted that the combination of theory and plenty of practice was the key benefit for this type
of training; especially the ability to try things out with a trainer present to provide guidance. The
importance for these sessions to be focused on ‘how it’s actually done’ was also noted. The need to have
an appropriate student to teacher ratio ‘in order that students get the attention needed e.g if struggling
to understand a particular matter’ was noted as well as the fact that ‘Different learners’ technical
background cause slow down’. The potential relatively high costs of these types of sessions was also noted
as a concern.
Which types of training materials were used and how satisfied were you with them?
(Question 4.2)
Respondents were asked to provide example of the worst and best aspects of the types of materials