How to Assess and Map Employees Competencies1366342/FULLTEXT01.pdf · competencies the company potentially lacks. This thesis investigates how to assess, eval-uate and map employees

Linköpings universitetSE–581 83 Linköping+46 13 28 10 00 , www.liu.se

Linköping University | Department of Computer and Information ScienceMaster thesis, 30 ECTS | Datateknik

2019 | LIU-IDA/LITH-EX-A--19/081--SE

How to Assess and MapEmployees CompetenciesHur man bedömer och kartlägger anställdas kompetenser

Christoffer Johansson

Supervisor : Christer BäckströmExaminer : Peter Jonsson

External supervisor : Jacob Bergvall

http://www.liu.se

Upphovsrätt

Detta dokument hålls tillgängligt på Internet - eller dess framtida ersättare - under 25 år från publicer-ingsdatum under förutsättning att inga extraordinära omständigheter uppstår.Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka ko-pior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervis-ning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annananvändning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säker-heten och tillgängligheten finns lösningar av teknisk och administrativ art.Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning somgod sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentetändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsman-nens litterära eller konstnärliga anseende eller egenart.För ytterligare information om Linköping University Electronic Press se förlagets hemsidahttp://www.ep.liu.se/.

Copyright

The publishers will keep this document online on the Internet - or its possible replacement - for aperiod of 25 years starting from the date of publication barring exceptional circumstances.The online availability of the document implies permanent permission for anyone to read, to down-load, or to print out single copies for his/hers own use and to use it unchanged for non-commercialresearch and educational purpose. Subsequent transfers of copyright cannot revoke this permission.All other uses of the document are conditional upon the consent of the copyright owner. The publisherhas taken technical and administrative measures to assure authenticity, security and accessibility.According to intellectual property law the author has the right to bementionedwhen his/her workis accessed as described above and to be protected against infringement.For additional information about the Linköping University Electronic Press and its proceduresfor publication and for assurance of document integrity, please refer to its www home page:http://www.ep.liu.se/.

© Christoffer Johansson

http://www.ep.liu.se/

http://www.ep.liu.se/

Abstract

The need to manage, map and develop a companies employees competencies is con-stantly increasing in today’s rapidly growing market. It is therefore incredibly importantto have a broad understanding of what competencies one’s employees possess and whichcompetencies the company potentially lacks. This thesis investigates how to assess, eval-uate and map employees competencies in the best possible manner, followed by the po-tential effects this can have on the individuals undergoing the assessment. Lastly the the-sis identifies how a competency management system could be implemented and whichfunctionalities said system should contain. This study began with a systematic literaturereview, for me to get a better understanding of the competency concept and competency as-sessment. The literature study was followed by a questionnaire distributed to all Etteplansemployees in Linköping and Norrköping, and lastly four interviews were conducted withemployees at the office in Linköping. To get a clearer understanding of the employeesthoughts regarding competency and the implementation of a new competency manage-ment system. One of the most important factors to take into consideration when applyinga competency management procedure is that these implementations takes time and mustbe carefully planned and executed to generate the best possible results. All companies areunique and require their own competency management procedure to succeed.

Acknowledgments

First and foremost I would like to express my gratitude to Etteplan for giving me the chanceto write my master thesis with them. I would also like to thank my external supervisor atEtteplan, Jacob Bergvall, for supporting me during this thesis and constantly providing mewith valuable comments and feedback. Furthermore, I would like to thank my supervisorChrister Bäckström and my examiner Peter Jonsson at Linköpings University for the help andfeedback during this thesis. Lastly, I would like to thank my opponent, Sebastian Parmbäck,for both wise and rewarding comments regarding this thesis.

iv

Contents

Abstract iii

Acknowledgments iv

Contents v

List of Figures vii

List of Tables viii

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Aim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Presentation of Etteplan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.6 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Theory 42.1 Competency Management System . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.1 Frequent Itemset Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2.1.1 Frequent Pattern Growth Algorithm . . . . . . . . . . . . . . . 6

2.2.2 Cluster Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Competency Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3.1 Competency Classification . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3.2 Competency Iceberg Model . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Method 103.1 Systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.1.1 The Review Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.1.1.1 Planning the Review . . . . . . . . . . . . . . . . . . . . . . . . 113.1.1.2 Conducting the Review . . . . . . . . . . . . . . . . . . . . . . . 113.1.1.3 Reporting the Review . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2 Empirical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2.1 Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2.1.1 Survey Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2.1.2 Survey Instrument Development . . . . . . . . . . . . . . . . . 133.2.1.3 Survey Execution . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2.1.4 Data Analysis and Reporting Survey Results . . . . . . . . . . 14

3.2.2 Semi-structured Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Results 16

v

4.1 Systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.1.1 Planning the Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.1.1.1 Identifying the Need for a Review . . . . . . . . . . . . . . . . . 164.1.1.2 Defining Research Questions . . . . . . . . . . . . . . . . . . . . 174.1.1.3 Developing a Review Protocol . . . . . . . . . . . . . . . . . . . 17

4.1.2 Conducting the Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.1.2.1 How to Assess and Evaluate Competencies . . . . . . . . . . . 204.1.2.2 Challenges With Competency Assessment . . . . . . . . . . . . 224.1.2.3 Potential Affects On the Person Being Evaluated . . . . . . . . 23

4.1.3 Evaluation of the Review . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Questionnaire results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2.1 Etteplans Employees Thoughts on Competency . . . . . . . . . . . . . . 254.2.2 Etteplans Employees Thoughts on Competency Management System . 28

4.3 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.3.1 Opinions Regarding Competency . . . . . . . . . . . . . . . . . . . . . . 314.3.2 Ethical Aspects to Consider When Mapping Competencies . . . . . . . . 324.3.3 How the Current CMS is Used Today . . . . . . . . . . . . . . . . . . . . 334.3.4 Preferred and Requested Functionality in a New Competency Manage-

ment System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5 Discussion 355.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.2.1 Systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . 375.2.2 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375.2.3 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.3 Proposed Competency Assessment Approach at Etteplan . . . . . . . . . . . . . 40

6 Conclusion 426.1 Future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Bibliography 44

A Appendix A – Questionnaire 48

B Appendix B – List of Primary Studies 50

vi

List of Figures

2.1 The Knowledge Discovery from Databases process . . . . . . . . . . . . . . . . . . . 52.2 Competency iceberg model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.1 Distribution of where respondents are located . . . . . . . . . . . . . . . . . . . . . 244.2 Department distribution of the respondents . . . . . . . . . . . . . . . . . . . . . . . 254.3 Questionnaire respondents amount of years working at Etteplan . . . . . . . . . . . 254.4 Distribution of how important people think it is to map skill based competencies . 264.5 Distribution of how important people think it is to map knowledge based compe-

tencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.6 Distribution of how important people think it is to map behavioural based com-

petencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.7 Respondents definition of competency . . . . . . . . . . . . . . . . . . . . . . . . . . 274.8 Distribution over whether the respondents believe implementing a CMS will ben-

efit their personal development or not . . . . . . . . . . . . . . . . . . . . . . . . . . 294.9 Distribution over whether the respondents believe using a CMS would change the

way they acted when making different career choices . . . . . . . . . . . . . . . . . 304.10 Distribution over which functionalities the respondents wish for in a CMS . . . . . 31

vii

List of Tables

4.1 Keywords used when creating query strings . . . . . . . . . . . . . . . . . . . . . . 174.2 Query strings used when searching for articles . . . . . . . . . . . . . . . . . . . . . 184.3 Inclusion and exclusion criteria for article selection . . . . . . . . . . . . . . . . . . 184.4 Data extraction form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.5 Result from literature search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.6 Primary articles used in the literature review . . . . . . . . . . . . . . . . . . . . . . 20

B.1 Detailed overview of primary studies . . . . . . . . . . . . . . . . . . . . . . . . . . 51

viii

1 Introduction

The term "competence" first appeared in an article written by R.W White [47] in 1959 as a con-cept of performance motivation. However, the term first gained traction in the early 1970’swhen David McClelland wrote a seminar paper with the title "Testing for Competence RatherThan for Intelligence" [7]. In this paper McClelland questions the great amount of power thatintelligence and aptitude tests has when it comes to choosing applicants for college entranceor for jobs. He later discusses if this truly is the correct approach, or if people should bejudged on the basis of some other criteria, such as competence within a field. Yet, it was notuntil 1978 that the concept was used in relationship to individual performance development.When Thomas F. Gilbert used the term in his book entitled "Human competence: engineeringworthy performance" [16].The usage of the term "competence" varies widely, and the definition of the concept differsfrom person to person. Some scholars see competence as a synonym to skills, while someargues that the term is a combination of theoretical and practical knowledge, skills and be-havior. Mapping, analyzing and developing all of these factors can make a huge differencefor the growth of a company. Managing the competencies within the organization in a struc-tured and coordinated manner leads to a certainty that the company consists of employeeswith the correct competencies and skills. Furthermore, mapping each role leads to the hiringand promotion of the correct people.Competency and skills management has for several years been suggested to utilize employ-ees competencies more efficiently. By using competency maps, employers and employeescan identify their skills and competencies, and the level of said competencies. In addition tothis the maps can also be used to present in which fields each person is in need of develop-ment [1, 27, 49]. By having all the employees competencies mapped, and by comparing thesewith the required skills for the position, it becomes easy to see if there is anyone within thecompany that is qualified for the job. Or possibly if anyone is close to having the requiredcompetencies, and would become qualified after some training. Competency mapping hasseveral benefits, it increases employees self-awareness and aids in career management, it be-comes easier to find potential leaders within the organization and it assists in job evaluation.However, competency mapping can be applied to more than just individuals. It can also beused to identify key competencies within an organization, as well as to show required com-petencies to complete a specific job. This could be helpful when there is a vacant position inthe organization that has to be filled.

1

1.1. Motivation

1.1 Motivation

As technology evolves, the importance of hiring people with the right competencies and de-veloping the necessary competencies within your organization increases. With this followsthe need to map and assess each individuals competencies in a correct manner. The use ofcompetency mapping has grown significantly during the last couple of years and is nowused by Toyota [39] in order for all employees to evaluate themselves and to develop theircompetencies. As companies are getting more interested in having multi-skilled employeesthe need to map competencies increases, so that training can be applied to the fields whereimprovement is necessary [49]. Performing competency management within companies andorganizations have become an important process to embrace, in order to keep up in the on-going technological evolution.This thesis is written in collaboration with Etteplan1. Etteplan is currently using a human re-source management system containing some competency management functionalities. Thecurrently used system is not mainly built for competency management, which is the mainreason for why Etteplan are looking into developing their own competency managementsystem. I have decided to exclude the name of the currently used system in this thesis. Thisthesis will be conducted at Etteplans office in Linköping.By starting off with a literature study, the knowledge needed to better understand "compe-tencies" as a concept and competency management in general will be gained. An empiricalstudy will also be conducted, where a survey will be used to gain knowledge of what theemployees at Etteplan thinks about competency in general, competency management andthe potential implementation of a new competency management system. Followed by inter-views with individuals from each department at the office in Linköping. This information canhopefully later be used when deciding on how Etteplan should manage the competency as-sessment and mapping of their employees. In addition to how the competency managementsystem should be used and which functionalities the system should contain.

1.2 Aim

The aim of this thesis is to analyze how one actually assess an individuals competencies, andwhat impact this has on peoples development. Information that will be used to identify howand why a competency management system should be developed, and later to propose howsaid software should be built.

1.3 Research Questions

This thesis will try to answer the following two questions regarding competency develop-ment and competency assessment of a company’s employees. As well as answering how andwhy a new competency management system should be developed. The research questionsare presented below, followed by short motivations of why each question is of interest.

• How do you evaluate an individuals competencies? And how does this affect the per-son being evaluated?

This question is important to answer in order to fully understand the necessity of a compe-tence management system and how this system should be used and developed. To evalu-ate someones competencies might be complicated, and it can be difficult to decide how thisshould be done. Two individuals might have the same area of expertise, but their skill mightbe on different levels. It is also important to understand that different people might react indifferent ways when being evaluated.

1https://www.etteplan.com

2

1.4. Presentation of Etteplan

• How should a competency management system be built, and how should the interac-tion between the system and the employees work?

The systematic literature review and the empirical study should shine light upon how a newsoftware should be developed and which functionalities said system should contain. Thisis important since the development takes time and costs money, and by making a thoroughstudy these two factors could be reduced to a minimum.These questions will be answered through a systematic literature review and by conductinga survey at Etteplans offices in Linköping and in Norrköping. The survey will be followedby interviews conducted at the office in Linköping.

1.4 Presentation of Etteplan

Etteplan is a consulting firm specialized in global engineering. Etteplan was founded in1983 in Finland and have more than 3400 employees spread across seven countries, of whicharound 500 of these are based in Sweden, divided over 15 offices. Etteplan work towards theworld’s leading companies in the manufacturing industry, and provides design engineeringservices, technical documentation solutions and software solutions. I will be a part of thesoftware solutions department during this thesis.Etteplan is currently using a human resource management system. The system also consistsof some competency management and competency development functionalities.

1.5 Delimitations

The term "competency" is very broad and can be interpreted in several different ways. There-fore, this thesis will not answer the question "what is competence?". This thesis will haveits own interpretation of the concept and the study will be conducted considering this. Themethod I will be using will be presented later in this thesis.

1.6 Thesis Outline

The remainder of this thesis will be outlined as followed. Firstly there will be a theoreticsection that will provide the knowledge needed in order to follow along in this paper. Nextfollows a section presenting the methodology used in this thesis. After that there will be asection presenting the results acquired during the study, followed by a section discussingsaid results. Lastly, the result will be concluded and summarized, in addition to answeringthe research questions once again, in a more concise manner.

3

2 Theory

This section feature all the underlying knowledge and terms needed in order to comprehendthe results presented later in the thesis.

2.1 Competency Management System

Competency management systems (CMS) [13] are used by companies in order to map, man-age and develop their employees competencies. Competency management is closely relatedto learning management, and functionality regarding learning is often implemented in com-petency management systems. For example tools to schedule training sessions and courses,and reports covering employees development are some functionalities that developers ofcompetency management systems often implement in their software.A CMS often includes tools such as competency management, succession planning, skill-gapanalysis, in addition to competency analysis and mapping [37, 41]. Competency manage-ment is useful within a company when they want an overview of the competencies and skillsof its employees. This information can later be used to manage the company and fill vacantpositions. However, there might not be anyone employed that have enough knowledge inorder to be promoted, and this is where skill-gap analysis is useful. By identifying how big ofa skill-gap there is, a development plan can be constructed and applied to minimize the gap.Thus making it possible for an employee at the company to grow and take a new step in theircareer, and possibly get one step closer to their career goals.One of the main usages of a CMS is to map employees competencies, discussed in 2.3, in or-der for individuals to easier plan their career and see where they are, and where they want tobe in a couple of years. The goal for the companies using competency management systemsis to expand and develop the expertise and knowledge of their work force. In order to stayrelevant and at the cutting edge of their area of expertise.A typical CMS offers the following features [43]:

• A pre-populated database of competencies, most often categorized by industry.

• A database with employees and their skills and competencies.

• The possibility to search each database.

• A reporting functionality.

4

2.2. Data Mining

• Performance reviews for each employee.

• The possibility to upload and disseminate reports consisting of so called "360-degreefeedback". Which is feedback from everyone that have been in direct contact with theemployee during an assignment, for example the employee oneself, supervisors andcustomers.

Competency management systems are designed to make it easier to grasp and analyze thisinformation. The software should therefore have an interface that is easy-to-use and also easyto master for new users.

2.2 Data Mining

Data mining is the process of discovering patterns in large sets of data. It is most easily under-stood as a technology that allows you to take data as an input, and thereafter extract interest-ing patterns or knowledge from said data [48]. One could say that the goal with data miningis to find and extract knowledge from data [36]. Data mining and the following technologiespresented in this section could be used when initially filling up a new CMS with data and in-formation. Instead of having to manually enter users competencies and knowledge. Fayyad,Piatetsky-Shapiro and Smyth provided one of the earliest and most cited definitions of thedata mining process in 1996 [17]. Where they defined the process as “the nontrivial process ofidentifying valid, novel, potentially useful, and ultimately understandable patterns in data.”The term "data mining" originally referred to the analysis step in the Knowledge Discovery ofDatabases (KDD) process. Where specific algorithms are applied for extracting patterns fromdata. However, since then this definition have been dropped, and these days "data mining"may refer to either the entire process or just the algorithmic step in the procedure.Figure 2.1 depicts how Fayyad et al. [17] envisioned the entire KDD process. The processconsists of the following five stages: Selection, Preprocessing, Transformation, Data mining andlastly Interpretation/Evaluation.The three first steps in the process deals with preparing the data for mining. It is importantthat the relevant data is picked from a large and diverse set of data, as data mining onlycan discover patterns that actually occur in the data set. The data is then preprocessed, i.e.cleaned, where findings containing so called noise and missing data are removed. Thereafterthe data is transformed into a representation that is suitable for the data mining algorithmthat is applied in the fourth step in the process. This step involves the application of datamining algorithms used in order to find patterns in the transformed data sets. The generatedpatterns can take many different forms, depending on what algorithms that were used. Fi-nally the patterns are thoroughly interpreted and evaluated. It is important to not accept thepatterns right away as sufficient. It could potentially be better to search for patterns usingsome other metric, in order to find the patterns that are of most importance.

Figure 2.1: The Knowledge Discovery from Databases process

5

2.2. Data Mining

The data mining process is an iterative process. Meaning that it is possible to redo theentire process once it is completed. After the initial run is completed, the user has to evaluatethe identified patterns thoroughly and make a decision on whether further work is necessary,or if the result is sufficient. As mentioned above it is of most importance that the correct andrelevant patterns are used, in order for the result to be as good as it possibly can be. Theresults from the first run of the data mining process are often not adequate. Either the resultsare not acceptable or the user believes that further improvements can be done. The processis then repeated after some adjustments have been done. These adjustments can be done atany time during the process. For example additional data sets may be acquired or new datamining algorithms may be selected during the data mining step. When the user is satisfiedwith the identified patterns the process is completed.

2.2.1 Frequent Itemset Mining

Frequent itemset mining is a data mining technique where data mining algorithms are usedin order to find sets of information that occur frequently together in data. Frequent itemsetsare patterns that frequently occur. This is often used on shopping websites. When inspectingan item the site often shows items "other shoppers also looked at". This is possible thanksto frequent itemset mining. The algorithms identifies patterns from searches done by otherusers and are therefore able to display items you as a user could possibly be interested in [2].There are several methods that can be used to mine frequent itemsets. Some of the mostwell-known algorithms are [42]:

• Apriori algorithm: The Apriori algorithm uses a so called breadth-first strategy in orderto count the support of each itemset. This method uses a generate-and-test approach,which means that the algorithm generates candidate itemsets and tests if they are fre-quent. However, generating candidate itemsets is expensive, both in time and space.Making this method more or less useless when working with larger data sets.

• Eclat algorithm: Unlike Apriori, the Eclat algorithm is a depth-first search algorithm.This strategy is based on set intersection, meaning it uses groups of itemsets that sharea common prefix.

• FP-growth algorithm: The FP-growth algorithm, where FP stands for frequent pattern,is the most common depth-first approach to date. This algorithm is also a bit morecomplex then the other two, and will therefore be discussed more in depth below.

2.2.1.1 Frequent Pattern Growth Algorithm

Unlike the Apriori algorithm, the frequent pattern growth (FP-growth) algorithm allows fre-quent pattern discovery without generating candidate itemsets. The FP-growth method isessentially composed of two steps: Creating a FP-tree, and extracting frequent patterns di-rectly from the tree.The frequent pattern growth algorithm starts of by counting occurrences of items in the dataset. Only the frequent items are then stored in a "frequent items table". Thereafter the FP-treeis built by inserting instances from the frequent items table. Items in these instances are sortedin descending order, as this makes it quicker to process the tree. The algorithm then checksthe items in each instance and discards the items that does not meet the required minimumthreshold [42, 22].The main benefit with the FP-growth method is that the algorithm searches for shorter fre-quent patterns recursively and removes these, instead of searching for longer ones. Thisreduces the search cost considerably.

6

2.3. Competency Mapping

2.2.2 Cluster Analysis

The term cluster analysis, or clustering, is a term used to describe the process of grouping aset of objects [33, 45]. Where the objects in the same group are more similar to each other thanthey are to those in other groups. These groups are called clusters. The grouping of the dataobjects is only based on information given in the data that describes each object.This technique can be applied in many different kinds of fields, and does not only apply in theworld of data. Clustering is useful in several fields, such as: machine learning, bioinformaticsand information retrieval. Cluster analysis is the main task in exploratory data analysis.Which is an approach of collecting and summarizing data sets.Cluster analysis is related to several other techniques of dividing data into different groups.One could say that cluster analysis is a type of classification as it is easy to label each cluster.Since the objects in each group has something in common.There are several different types of clusterings, where a so called "clustering" is an entiregroup of different clusters. The four most common cluster analysis techniques are:

• Hierarchical clustering: The goal with hierarchical clustering is to build a hierarchy ofclusters. There are two types of strategies when we talk about hierarchical clustering,and these are agglomerative and divisive clustering. Where the former is a a "bottom-up"approach and the latter is a "top-down" approach.

• Partitional clustering: Partitional clustering is used when the goal is to not have anyoverlapping clusters at all. Such that each data object is in only one cluster.

• Fuzzy clustering: Fuzzy clustering is also called soft clustering, where each data objectcan belong to more than one cluster.

• Density-based clustering: Density-based spatial clustering of applications with noise(DBSCAN) groups together data objects that are close to each other. DBSCAN is one ofthe most commonly used clustering algorithms.

2.3 Competency Mapping

Competency mapping is the term for describing the process of identifying strengths andweaknesses of an individual. The goal with mapping individuals competencies is to makeit easier for them to understand themselves and to pinpoint where career development needsto be directed. In order for employees to develop and get to the point in their career wherethey want to be. It is also useful to have mapped the competencies of the employees whenthe company have a job position to fill. Instead of hiring someone new, the company can lookat their own work force to find someone suitable for the job [1, 27, 37].There is no correct way to map competencies and it can be done in several different ways.Although there are some guidelines that could, or possibly should, be followed when com-paring the present and desired levels of knowledge and skills, when trying to perform a taskeffectively. These are [49]:

• Start with identifying the required skills for the job.

• Map the identified skills needed and compare these with the skills of the employees.Identify minimum required level of competencies.

• Map possible skill gaps.

• Create a development plan.

7


2.3.1 Competency Classification

There are two different classifications of competencies: core competencies and profes-sional/functional competencies. Core competencies are competencies that each individualwithin a company is expected to possess. These competencies are seen as crucial for a com-pany to succeed, and the goal with core competencies is for individuals to be able to performin different positions throughout the company. Meaning that each employee should be ableto handle different assignments without additional training. Core competencies are meant tobe flexible and can change as the company changes, evolves and adapts to the ever-changingtechnology [1, 27, 37, 41].Professional competencies and functional competencies are the technical knowledge neededin order to perform a job. The goal with these competencies is to optimize each individualsperformance when performing a specific job. These competencies are grouped for each jobwithin the company, and they are grouped in three different categories [1]:

• Behavioural competencies: These competencies refer to the behavioural knowledgeand traits needed by an employee in order to complete a task.

• Threshold competencies: This refers to the characteristics needed by an employee inorder to perform an assignment efficiently.

• Differentiating competencies: Characteristics that differentiate superior performersfrom average performers are classified as differentiating competencies.

2.3.2 Competency Iceberg Model

N. Anisha [1] recommends visualizing competencies in terms of an iceberg. Technical knowl-edge and skills are at the top, above the waterline. These competencies are visible, and there-fore easy to identify and appraise. While behavioural competencies are beneath the water-line and more difficult to assess. These competencies can also be more difficult to develop.As these competencies constitutes the actual essence of a person. Behavioural competenciesinclude what values the individual has, how the person views him or herself (self-image),how he or she typically behaves (traits) and what motivates a person to perform a specifictask (motives).

8


Figure 2.2: Competency iceberg model

• Skill: An individuals ability to do something well. For example being a great Javadeveloper.

• Knowledge: Information that a person uses in a specific area. This could for examplebe what differentiate an experienced developer from a recently graduated developer.The experienced developer could possess more information and therefore knowledgein the specific area.

• Values: How an individual sees the world and the people around him or her. It couldbe anything from political values to personal values. For example that all individualsare equal.

• Self-Image: How persons view themselves. Identity, personality and worth are alldifferent aspects each individual takes into account when assessing themselves. Forexample seeing yourself as a leader.

• Traits: How a person typically behaves. An example of a trait could be how good anindividual handles stress

• Motives: What motivates a person to perform a specific task. It could be anything froma need of acknowledgement to feeling powerful.

The competencies that lie beneath the waterline directly affects the competencies above thesurface. For example, the things that motivates a person influences how he or she uses theirskills and knowledge in order to complete a task. Either deliberately or instinctively. Further-more, the behavioural competencies could be problematic to assess and map from an ethicalpoint of view. People might feel uncomfortable knowing their characteristics are being as-sessed and evaluated, and not just their practical skills.

9

3 Method

This section presents the methodologies and concepts used in order to conduct the study andwrite this paper.

3.1 Systematic Literature Review

The first part of this thesis will consist of a Systematic Literature Review (SLR) based on pre-vious studies regarding competency and competency management systems. The systematicreview will follow the guidelines presented by Kitchenham and Charters [25]. The reason forchoosing these guidelines is because they are suited for software engineering studies. Whichis a category where this thesis fits perfectly well.Kitchenham et al. discusses the reasons for conducting an SLR, the importance of it, as wellas both advantages and disadvantages with a systematic review. They argue that one of thebiggest advantages is that the results from the review are unlikely to be biased, thanks tothe well-defined methodology of the process. However, the significant disadvantage withsystematic literature reviews is that they require substantially more time and effort than atraditional literature review [25].Kitchenham et al. have summarized several different guidelines for conducting a system-atic review and have concluded that the majority of studies are in an agreement on the mainstages in the review process. These steps are planning the review, conducting the review andreporting the review. Below is a short presentation of the process.

3.1.1 The Review Process

The review process consists of three major parts, where each phase covers the stages per-formed during the SLR. The purpose with the first phase is to do a detailed plan of the re-view. Afterwards, the review is conducted, followed by a presentation of the results in thethird and final phase.

10

3.1. Systematic Literature Review

3.1.1.1 Planning the Review

It is crucial to plan the review in detail, since a thorough planning leads to an increased chanceof ending up with a successful result. The planning phase consists of the following five substages:

• Identify the need for a review.

• Commissioning a review.

• Specifying the research questions.

• Develop a review protocol.

• Evaluate the review protocol.

The process might appear linear. However, many of these steps involve iteration, which alsoapplies to the sub tasks in the following two sections. Firstly it is important to confirm theactual need for a review. It happens from time to time that systematic reviews are commis-sioned, and in that case a commissioning report has to be written. None of the steps in thisphase are mandatory. However, Kitchenham et al. states that the two most important stepsin the planning process are defining the research questions that the review will address, andcreating a plan over the review procedure. An evaluation of the review protocol is needed ifthe review was commissioned. This is otherwise not necessary.I have decided that I will skip the commissioning a review step, since I will conduct this sys-tematic review myself. Furthermore, the last step will partially be skipped seeing that I willskip the commissioning report. However, an evaluation process is constantly ongoing and Iwill revise the review protocol if I find any possible improvements.

3.1.1.2 Conducting the Review

As soon as the protocol has been accepted, the actual review can begin. In this phase theprimary objective is to find as many studies as possible that relate to the research questionsstated during the planning phase. The magnitude of this process is what differentiates asystematic review from a traditional literature review. Kitchenham et al. recommends thatanyone who performs an SLR tries all of the steps presented in this phase. This phase consistsof the following five sub tasks:

• Identification of research.

• Select a primary study.

• Study quality assessment.

• Extraction and monitoring of data.

• Data synthesis.

The first step in this phase is to find as many potentially relevant studies to the research aspossible. It is possible to find relevant studies in multiple ways. However, Kitchenham etal. states how necessary it is to devise and follow a research strategy. Once the preliminarysearch for relevant studies have been completed, the studies have to be assessed and analyzedfurther. Their actual relevance have to be determined. Using a selection criteria, declaredduring the planning the review phase, makes it possible to find studies with a direct connectionto the research questions. Inclusion and exclusion criteria should be based on the researchquestions. These are used when deciding on which papers to include and which to excludein the review. It is considered essential to study the quality of each obtained primary study.

11

3.2. Empirical Study

This is done in addition to the inclusion and exclusion criteria. The next step in the processis to extract and monitor data the researchers obtained in each of the primary studies. This isdone by designing data extraction forms, which are used to collect data that is relevant to theresearch questions. The result from the studies is finally compared and summarized in theData synthesis.

3.1.1.3 Reporting the Review

The final step in the process consists of writing down the results from the review, and dis-seminating it to interested individuals. The following three sub tasks are a part of this phase:

• Specify ways of dissemination.

• Formatting the main report.

• Evaluating the report.

It is crucial to specify how the SLR should be distributed to individuals that potentially couldbe interested in taking part of the review. This could be by publishing a paper in an academicjournal, or presenting the results during a conference. Therefore it is important to format themain report correctly, since journal and conference papers usually have restrictions regardingsize and content. The last step of the entire literature review process concerns evaluating thereport. Journal and conference papers are peer reviewed and PhD theses are reviewed byexperts as a part of the examination process.

3.2 Empirical Study

The results collected from the systematic literature review will then be used to perform anempirical study. I will start off by conducting a survey that will be answered by the employeesat Etteplans offices in Linköping and in Norrköping. Followed by interviews, in order to getsome more detailed answers regarding the employees thoughts around competence and theimplementation of a CMS. This will hopefully give me a better understanding of what theemployees wish to gain from a competence management system. All this information willeventually lead to a proposition on how a CMS should be developed, implemented and used.

3.2.1 Survey

As a first step in this empirical study I will conduct a survey. Thus, getting a better under-standing of the employees thoughts on competence as a concept and their thoughts regardingthe implementation of a competence management system within the company. A major ad-vantage with a survey is that it is possible to collect a considerable amount of answers in ashort period of time. However, the questions in a survey have to be easy to understand andbasic, and it is not possible to ask supplementary questions. Which is a major disadvantage.This will be solved by conducting interviews with a few selected employees. Another dis-advantage is that people in general find it difficult to assess themselves [15], which couldturn out to be an issue. Seeing that the survey is partially about the employees thoughts oncompetence, both as a concept as well as their own. The entire survey can be found in Ap-pendix A.The survey will be distributed through an email list to all employees at Etteplans offices inLinköping and Norrköping, hopefully receiving a response rate of at least 80%. Everyonewill answer the questionnaire anonymously and the only question that possibly could relateanswers to a specific group of people is the question where the respondent has to state inwhich department it works.This survey process will follow the guidelines presented by A. Glasgow [20]. There are sev-eral different guidelines to choose from and they can differ considerably. However, the reason

12


for choosing Glasgow’s guidelines was because they were among the first I found and theyfit perfectly with the study I were to conduct. Glasgow breaks down the process into threephases: survey design, survey instrument development and survey execution. Glasgow also arguesthat data analysis and reporting of results is so important that it should be considered its ownphase in the process. All the phases in the survey process is presented below.

3.2.1.1 Survey Design

I will be designing the questionnaire myself and my supervisors will review the question-naire. Making sure the questionnaire is complete and do not lack any necessary questions inorder to obtain enough knowledge about peoples thoughts concerning competence. The firststep in this phase is to develop a sample plan [20]. This sampling plan is used to describe theapproach in which a sampling group will be selected from a population. The sampling planalso states how a sufficient group size will be determined, as well as in which choice of mediathe survey will be distributed through. Survey media involves approaches like face-to-faceinterviews, telephone interviews and written surveys. This phase also covers how to choosethe best possible target group for your survey and the importance of picking the samplinggroup randomly.Glasgow states the importance of this phase, in order for the survey to stay unbiased. How-ever, this is not a phase that I will spend a lot of time with. Seeing that this thesis is conductedin collaboration with Etteplan, and I am only interested in the opinions of the employees atEtteplan. Making it impossible to stay fully unbiased.I will be conducting a written survey distributed using an email list to all the employees atEtteplans offices in Linköping and Norrköping. 121 employees will receive the questionnaire.Glasgow mentions the importance of choosing a correct sampling size to get a reliable result.I believe that it is reasonable to involve both offices in the survey, since the two are closelyconnected and work closely together. The survey will be distributed to everyone workingat the two offices and not only a few selected departments. Hopefully involving as manydifferent thoughts as possible, increasing the depth of the answers. Involving just a couple ofdepartments could make the answers biased, as there is a possibility that people in the samedepartment have the same views and experiences regarding competence.

3.2.1.2 Survey Instrument Development

The second phase in the survey process discusses the linguistics and what actually catego-rizes a good question. Glasgow states that it is crucial that the survey questions are easy tounderstand and consistent with the educational level of the respondents [20]. The questionsshould be as easy to understand as possible, avoid misinterpretation, avoid any undefinedabbreviations and avoid any double negatives. Another crucial thing to consider when con-ducting a survey is to avoid combined questions. Where the respondent may wish to answerpositively on one question, but negatively on the other. An example of such a question couldbe "I enjoy my job assignments and I like my colleagues." where one might wish to answerdifferently on these two questions. The questions must be ethical as well. It is necessary toavoid questions asking the respondents for information they are not allowed to share, or donot have.Glasgow argues that it is necessary, yet difficult, to avoid questions that could invoke anykind of biased answers. Either formulating a question in a specific way, or even the orderof questions in the survey. An answer on a previous question could influence the answeron the following question. Nevertheless, this is important to avoid and only strengthens theimportance of reviewing the survey before distributing it. The questions in this questionnairewill be a combination of both open-ended and closed-ended questions. Simple closed-endedquestions should provide enough data in order to proceed with the study. However, open-

13


ended questions will provide more detailed answers regarding the employees thoughts andopinions on competence.

3.2.1.3 Survey Execution

The third phase in the process regards the actual execution of the survey. One essential ethicalaspect when executing the survey is to maintain the confidentiality of individual answers. Itshould not be possible to separate answers from one another and associate answers withspecific individuals. Survey results should only be presented in total. Another importantethical aspect to remember is that survey participation is voluntary. Respondents are notobliged to answer the questionnaire, and the researcher should avoid being obstinate when itcomes to receiving answers. Furthermore, respondents should not feel pressed to answer thesurvey, and there should be a reasonable deadline for when the answers has to be submitted.My plan is to give the respondents two weeks to answer the questionnaire. However, thismight change depending on the response rate after two weeks.

3.2.1.4 Data Analysis and Reporting Survey Results

The fourth and final phase includes analyzing the data and reporting the results in a correctmanner. A step that is crucial in order for the survey to be taken seriously and for the resultsto be seen as reliable. The results from the questionnaire will be presented in Section 4.2 andlater discussed and analyzed in Section 5.2.2.

3.2.2 Semi-structured Interviews

One person from each department at the office in Linköping will be participating in a semi-structured interview, in order to collect more detailed answers. The interviews will followthe guidelines presented by Runeson and Höst in [35]. There are several different guide-lines regarding semi-structured interviews, but they are all very similar. Runeson and Höst’sguidelines were chosen because they are adapted for studies within the field of software en-gineering.In a semi-structured interview the interviewer does not follow a strict list of questions anda strict plan. This interview technique can somewhat be seen as a combination of a regularinterview and a conversation. The interviewer writes a couple of open-ended questions, butleaves room for supplementary questions that might emerge during the conversation. Thequestions and interview plan are only used as a guideline, and the interviewer can decide tosteer away from the plan whenever [35]. A semi-structured interview is suitable in this thesisas it is a follow-up on a questionnaire, to gain more detailed answers and thoughts regardingthe topic.The interviews will be qualitative and not quantitative. Meaning that they follow a specifictopic, but the questions are open-ended and might not be asked in the same order or in thesame way to each interviewee [5]. The goal is to hear what the respondents think themselves,which is hard when conducting a quantitative interview. Where the questions have more ofa "yes or no" character.My plan is to interview four individuals at the office in Linköping. One from each of the threedepartments and one executive, since it is interesting to gain as many different thoughts andideas as possible. The three employees will hopefully be interviewed first, followed by theexecutive. Being able to analyze the answers from the employees before talking with the exec-utive hopefully generates some new questions that could be asked during the interview withthe executive. I have chosen to only interview persons from the office in Linköping, eventhough the questionnaire will be sent to the employees in Norrköping as well. The reason forthis is that I find it relevant to talk to people where I am based.As with questionnaires there are some ethical aspects that has to be taken into account whenconducting interviews. Two major factors are confidentiality and anonymity. The interviewee

14


have to be assured that everything that is said during the conversation will stay anonymous,unless the interviewee want otherwise. However, one problem is the number of persons be-ing interviewed. It is difficult to keep everyone anonymous when there will only be fourpersons interviewed, and all the interviews will be conducted at the office. Keeping individ-uals separated from their own answers might be easy, but the interviewees may be groupedtogether and accused of some answers as a group.Another ethical dilemma is how to document the conversations. Hopefully everyone willaccept being recorded, in order to avoid potential breaks in the interview when I have to takenotes. Taking notes may ruin the flow of the conversation, and the interviewee could pos-sibly forget something that he or she was thinking of. Recording the dialogues also ensuresthat no information is lost during the discussion. Therefore, assuring the interviewee that Iam the only one who will be working with the recording is crucial, in order to maintain theconfidentiality and anonymity. Longhurst [30] discusses the importance of transcribing therecordings as soon as possible after the interview. Hearing the taped conversations when itis still fresh in your memory makes transcription easier and reduces the time consumed con-siderably. However, this might be difficult to sustain if more than one interview is scheduledduring a day.Several studies discusses the importance of having a lot of experience when conducting in-terviews [23, 34]. However, I do not have any experience when it comes to conducting in-terviews, but I believe that this may be more important when interviewing people that youhave no kind of connection with. When as in my case interviewing people that I at least havespoken to before makes it much easier to maintain a conversation and a flow in the interview.Without much experience at all.The questions below are the main questions that I will be asking during the interview. How-ever, as mentioned above, the questions may be asked in a different order to each person andin a different way. Depending on in which direction the interview is heading and dependingon potential supplementary questions that have arose during the dialogue.

• What is your opinion on the "competence" concept?

• It is impossible to avoid the ethical aspect when it comes to mapping individuals com-petencies. What are your thoughts regarding mapping the three different competencycategories: skill based, knowledge based and behavioural based competencies? Arethere different ethical aspects to consider for each category?

• How are you using the CMS that is used at Etteplan today?

• Do you have any personal preferences or requests regarding a competence managementsystem? When it comes to how it should work, how it should work, how it should beimplemented etc.

15

4 Results

This section presents the results gathered during this thesis. The first part of this sectionpresents the results from the systematic literature review and the second part presents theresults from the survey and the interviews.

4.1 Systematic Literature Review

The literature review process began with a planning of the review, performing each sub stepthe phase consists of. Followed by the actual execution of the review. Lastly the findings fromthe literature study was recorded and reported.

4.1.1 Planning the Review

The first step in the systematic literature review process was the planning of the entire review.

4.1.1.1 Identifying the Need for a Review

Ever since 1978 when the concept of competence was first used in relationship with individ-ual performance development, the term has been a subject of discussion. No consensus existsregarding how the concept should be defined [40] and researchers define the concept in manydifferent ways [11]. Some sees the concept as a synonym to skills, while others define the con-cept as a combination of theoretical and practical knowledge, behaviour and skills. Possiblythere will never be a mutual definition of the term. However, there is an agreement that theneed to map and develop competence within companies increases, in order to stay relevantand grow as an organization [24, 29].The necessity to map and utilize employees competencies and knowledge have never beenas important as it is today. Competency mapping have grown considerably in recent yearsand the method have become crucial for companies to adapt. This literature study shouldlead to a conclusion regarding how Etteplan should perform the competency mapping andcompetency assessment of their employees.

16


4.1.1.2 Defining Research Questions

The main purpose with this systematic literature review is to obtain a deeper understandingof how you evaluate and analyze an individuals competencies. Along with how this poten-tially could have a negative or positive impact on the people being evaluated. Which will bedone by answering the following research questions:

• How do you assess and evaluate an individuals competencies?

• What are the biggest challenges with assessing and evaluating individuals competen-cies?

• How could this potentially affect the person being evaluated?

Answering these questions should present a clear picture on how to assess and evaluate anindividuals competencies the best way possible. Being aware of the common challenges andhow the assessment potentially can affect individuals will help when deciding upon how theassessment process should work, and later when presenting a proposed approach on how acompetency management system should work and be implemented.

4.1.1.3 Developing a Review Protocol

A review protocol presents the methods which were used when conducting the systematicliterature study. In addition to important decisions that was taken during the construction ofthe review protocol. I will be using three common databases for scientific papers, primarilyin the fields of computer science, to find relevant studies. These three digital libraries are:

• IEEE Xplore Digital Library

• ACM Digital Library

• ScienceDirect

In addition to these databases, Google Scholar and CiteSeerX will be used for so called chainsampling. Which is the term for describing the process of finding additional studies of interestfrom the reference lists of the articles found during the literature study. The plan for the dataextraction is to firstly gather as many different studies and articles as possible. Which willbe done by using different query strings, presented in table 4.2, constructed from a set ofkeywords, presented in table 4.1.

Primary keywords Secondary keywords Supplementary keywordsCompetence assessment Companies CompetenceCompetence evaluation Challenges CompetenciesCompetency classification Behavioural ConsequencesCompetency mapping Threshold Company

DifferentiatingConsequence

Table 4.1: Keywords used when creating query strings

17


Query stringsCompetence AND (assessment OR evaluation) AND (Companies OR Company)Competency AND (classification OR mapping) AND (Companies OR Company)Competence AND (assessment OR evaluation) AND (Behavioural OR Threshold ORDifferentiating)Competency AND (classification OR mapping) AND (Behavioural OR Threshold ORDifferentiating)Competence AND (assessment OR evaluation) AND (Challenges OR Consequences)Competency AND (classification OR mapping) AND (Challenges OR Consequences)

Table 4.2: Query strings used when searching for articles

The articles retrieved during the search process will then be filtered through three phases,where different inclusion and exclusion criteria, presented in table 4.3, are applied. Firstly, thebroader criteria, such as language and type of publication, are applied on the articles. Whenthe articles have gone through the first step and some have been filtered out, a new criteriawill be applied. This criteria will review the titles of the selected studies and filter out thearticles with titles that are not of interest to this thesis. Lastly, the abstracts and introductionsof the articles that got through the first two phases will be read, to decide which of the articlesshould be included in the literature review.

Phase Inclusion criteria Exclusion criteria

1st

It is a conference paperor book chapteror journaland within the scope of competency assessment

Not in Englishis a duplicateis a summary

2nd Title is related to competency assessment

3rd

Abstract and conclusion states that thearticles have studied competencyassessment within companies, organizationsand/or different kinds of professions

Table 4.3: Inclusion and exclusion criteria for article selection

When this process is completed there should be a reduced amount of relevant articles.These articles will then be read and summarized using a data extraction form, presentedin 4.4. Kitchenham et al. [25] mentions the importance of analyzing and evaluating eacharticle in the same way, reducing the possibility for bias. They recommend constructing a dataextraction form to answer a few questions regarding the keywords that have been used, aswell as the research questions. This is done in order to decide if the studies are relevant or not.All studies will be classified based on a wide range of questions, such as type of publication,if they contain the stated keywords and if the articles answer the research questions or not.

18


Question Yes / No CommentName of articleAuthorType of publicationYear of publicationDigital libraryKeywordsHow to assess and evaluate competencies?What challenges was identified?Potential ways the evaluation could affect individuals?

Table 4.4: Data extraction form

4.1.2 Conducting the Review

During the initial search the goal was to collect as many articles related to the subject aspossible. This was done by using the databases advanced search functionalities, and conductsearches using the constructed string queries listed above in table 4.2. Initially the search gen-erated a total of 78,244 articles, which were filtered down to 15 studies when going throughthe three inclusion and exclusion phases. The articles chosen for the systematic literaturereview are presented in table 4.6, and a more detailed presentation is located in Appendix B.

Database Initial Phase 1 Phase 2 Phase 3IEEE 47,453 139 34 6ScienceDirect 28,576 244 48 6ACM Digital Library 2,215 63 19 1Chain Sampling - - - 2Total 78,244 446 101 15

Table 4.5: Result from literature search

19


ID Title ReferenceA1 Supporting competence management in software projects [12]

A2 Competence analysis of IT professionals involved in business services— Using a qualitative method [31]

A3 Effects of Automated Competency Evaluation on Software Engineers’Emotions and Motivation: A Case Study [9]

A4 From Behavioral Indicators to Contextualized Competence Assessment [3]

A5 Assessment of the behavioural component of competencyto substantiate practitioner competency claims [10]

A6 A Semantic Approach for Improving Competence Assessmentin Organizations [19]

A7 Competence assessment as an expert system forhuman resource management: A mathematical approach [4]

A8 Robust competence assessment for job assignment [21]A9 The Assessment of Learning: From Competence to New Evaluation [8]

A10 The factors related to self-other agreement/disagreement in nursingcompetence assessment: Comparative and correlational study [44]

A11 Preceptors’ experiences of using a competence assessment toolto assess undergraduate nursing students [6]

A12 Assessing competencies: An evaluation of ASTD’s CertifiedProfessional in Learning and Performance (CPLP) designation [26]

A13 Evaluating assessment with competency mapping [32]A14 Competencies and Their Assessment [14]A15 Challenges to the assessment of competence and competencies [28]

Table 4.6: Primary articles used in the literature review

All of the studies above discusses the subject competence management/assessment insome way. However, the majority of the articles discusses the subject within a specific field.For example nursing and teaching. This does not provide a general insight into how to as-sess competence, which requires you to make assumptions. Nevertheless, I believe that theprimary studies found during the search will provide enough data in order to answer theresearch questions.

4.1.2.1 How to Assess and Evaluate Competencies

There is no definite way to assess and evaluate individuals competencies. Different methodssuits different occasions. The gathered primary studies present and suggest several differentmethods, all within their specific field. One reoccurring scenario is within an educationalenvironment. Where the students are asked to assess their competencies at the beginning of acourse, and finally at the end of the course. This method is used to evaluate how the studentshave developed, and to see if what they thought of themselves actually corresponded totheir competencies. This method is often used in a educational environment seeing there is alearning element when you have to assess yourself.The majority of the methods presented in the articles originates from self-assessment andassessment of others. Where one popular method, presented in several of the studies, is thecombination of self-assessment and assessment of each other. Essentially, each of the studieshave adapted one or both methods to suit the research conducted in each article.

Self-assessment The self-assessment is most often done through questionnaires, where theemployees are asked to answer several questions to grade themselves and their competen-cies. The results from the questionnaires could be entirely private, where each participantis the only one able to see their own results. These results are then used to follow the in-

20


dividuals progress and development. However, the results could also be accessible to a se-lected group of people, for the individuals answering the questionnaire to get help in theirdevelopment. This method is used by Dorn et al. [12] where a group of university studentsare participating in a software project management course. The students are asked to use acompetency management system to plan and control their future study based on their com-petencies. Primarily, the students answer a questionnaire at the start of the course, wherethey are asked to estimate and judge their own competencies. The results generated fromthese questionnaires are available to both the student and their supervisor, making it possi-ble for the supervisor to guide and observe the development of the students competencies.At the end of the course the students are asked to answer another questionnaire, where thefinal feedback showed which competencies where improved and which new competenciesthe student acquired during the course.Bohlouli et al. [4] also presents a method in which the assessment process begins with theparticipants answering a questionnaire. Each answer is weighted and the results are usedtogether with a mathematical model. This model calculates the persons so called "availablecompetencies" and creates a development plan for each person that answered the question-naire. These plans are then used by each employee in order for them to achieve their goals,and develop their skills the best way possible.

Assessment of Others Interviews and questionnaires are two methods often used whenconducting competency assessments [31]. Another popular method often used in conjunc-tion with competence assessment is so called "360-degrees feedback". This method is a pro-cess where feedback from the employees colleagues, supervisors and subordinates, as wellas self-assessment by the individuals themselves is collected and analyzed [4]. The feed-back can also, when relevant, include feedback from external sources who have interactedwith the employees during a project or a job. This feedback could be relevant when work-ing as a consultant, where you most certainly interact with several different companies andindividuals throughout your career. Making it relevant having your competencies and yourperformance assessed by the persons you come in contact with at these companies and dur-ing these projects.A method often used in educational environments, in combination with students assessingthemselves mentioned above in self-assessment, is for the supervisors to assess their studentsperformance and competencies at the end of the course [12]. This is done through question-naires where the supervisors get the chance to evaluate and grade each students performance.Drisko [14] mentions how important it is that the one assessing another individuals compe-tencies already possesses the competencies being evaluated. Which means that the evaluatorpossesses the knowledge, skill or competency being examined and have significant experi-ence in the field or profession. Another approach is to use tools that are suited for compe-tency evaluation. Therefore, it is important for the supervisors to evaluate each student aswell, since they are experienced and already possesses the competencies.One important thing to consider when assessing and evaluating competencies is to carry outtests relevant for the work where the competencies later will be used. For example whentesting how well a consultant works under pressure, it is important to not put the consultantin a situation where he/she is uncomfortable. Otherwise the results might be tampered, andsomeone who actually has the required competencies to perform the task might appear un-qualified.Competencies can be assessed in several different ways, and can be assessed by several differ-ent individuals. The approaches can be both quantitative and qualitative, and the evaluatorcould be anything from a client to a supervisor. However, Drisko [14] emphasizes how impor-tant it is that the approach is explicit and directly related to the competency being assessed.Furthermore, it is optimal to use several methods at once, to get a result as close to reality aspossible.

21


Burke et al. [6] presents a mixed method where the assessment process contained both aqualitative phase and a quantitative phase. Each phase respectively consisted of group in-terviews and a questionnaire. The interviews were divided into six focus groups where thepurpose was to establish professors experience in using a competency management tool toassess nurses clinical competencies. The questionnaire was used to quantitatively measurethe professors attitude towards the competency assessment system.Kwon et al. [26] also presents a mixed quantitative-qualitative method to identify the relation-ship between a competency model used in an educational environment and the final examtaken by students. The first phase consisted of interviews and a literature study, where theresults later was used to construct a first draft of the competency model. Phase two startedof by constructing a survey using the first draft of the competency model, and these resultswere then used to finalize the model.Collins and Nicholson [10] suggests assessing competencies on a sub-component level.Meaning that instead of assessing and analyzing for example someones leadership compe-tency, you divide it into several competencies needed for someone to be a good leader. Thesecompetencies are then assessed separately.

4.1.2.2 Challenges With Competency Assessment

One major challenge with competency assessment is how difficult it actually can be to as-sess individuals competencies [21]. Especially behavioural competencies that can sometimesbe hard to define. One specific occurrence that verifies how difficult it actually is to assessyour own or someone elses competencies is how often people overestimate or underestimatethemselves [12]. People simply misjudge their competencies and what they are capable ofaccomplishing. Takase et al. [44] states that one reason for this could be individuals person-alities and that it is difficult to be impartial. It is therefore important having as many peopleas possible involved in the assessment and evaluation of someone. Collecting a large amountof opinions and views should give a picture of an individuals competencies as close to realityas possible.Some categories of competencies are easier to assess then others. As mentioned in Section 2competencies can be divided into three categories: skill based, knowledge based and be-havioural based competencies. The first two categories are pretty straight forward to test andassess. Whereas behavioural competencies can be more difficult and somewhat questionableto assess from an ethical point of view.The majority of the studies found in this literature review are specialized in a specific areaof expertise, for example nursing and teaching. Which does not provide an insight into howthese methods could be used generally. Therefore, the only thing one could do is to makeassumptions on how these methods would work in other environments. Furthermore, themajority of the methods presented in the studies are based on data from interviews and sim-ilar approaches of collecting data. The methods are therefore based on peoples opinions andnot very "scientifically proven". “Furthermore, the main challenge and limitation that sim-ilarly addressed in most of the literature is a lack of efficient and generalized competencematching method.” [4].The need for continuous development of competencies with employees have never been asimportant as it is today. Individuals have to develop existing competencies along with newonce. This calls for new ways of training and learning, since current learning materials mostoften consists solely of theory. Furthermore, current guidelines on how and when theoryshould be applied in practice are ambiguous and diffuse [3]. Learning material have to beconnected with the actual work process in a better way. Otherwise it will become harder todevelop competencies, and with that assessing them.Burke et al. [6] discusses the challenges that comes with implementing and using a compe-tency management system. It can be anything from linguistic difficulties to how time con-suming it actually can be to use these tools. Everyone has to commit wholeheartedly for the

22


system to actually be useful. There is a risk that people might prioritize other work relatedthings, leaving the competence assessment and development to come in second. Moreover,it can be difficult to even recognize competencies. It requires well developed and well per-formed tests. In addition to these tests being observed and evaluated by competent individ-uals.Lichtenberg et al. [28] have identified several challenges in terms of competency assessment.Defining competencies in a precise and measurable manner is challenging, and it can bestressful for everyone involved if it is done poorly. Furthermore, they discussed a few pro-fession based challenges that are different depending on in which market your company isspecialized in. Firstly it is important to decide which competencies are relevant to the profes-sion, in addition to deciding an appropriate minimal level of competence. The final step is todecide upon which methods and tools to use when assessing all components of competency,and to establish mechanisms to provide effective evaluative feedback. However, even thoughthese challenges might seem tough to meet, perhaps the single biggest obstacle to overcomewould be convincing those who are skeptical towards the implementation that this is a nec-essary evil, for the company and its employees to develop effectively.Lastly, it can be challenging to immediately find the best suited assessment method and tool.The system has to suit the profession in which it will be used, as well as stimulate the user.The things the user learns has to be utilized in a proper fashion [8]. The process of findinga method that suits the situation is iterative and requires careful testing, making it very timeand resource consuming [19].

4.1.2.3 Potential Affects On the Person Being Evaluated

Finding studies discussing potential effects on people being evaluated proved to be difficult.However, there is an ethical dilemma when it comes to mapping competencies. People mightfeel monitored and find it strenuous and stressful when their competencies, both knowledgebased and behavioural based, are mapped [32].Calikli et al. [9] discusses potential effects on supervisors and employees when using com-petency management systems at a company. Inconsistent data and linguistic problems couldfor example lead to difficulties for supervisors when using a CMS. Leading to frustration andpotentially affect the decision making process. There is a risk that employees and develop-ers feels stressed and feels as if there is a considerable amount of pressure on them whenthey know they are being mapped. Being forced to complete competency profiles when hav-ing other things to focus on can also cause stress. Which can give a negative effect on theirperformance. Research have shown that negative emotions leads to negative developmentresults. Whilst positive emotions leads to positive results [9].

4.1.3 Evaluation of the Review

One important thing to understand when it comes to the validity of this systematic litera-ture review is that the majority of the primary studies focuses on a specific area of expertise.They are not generalized, making it difficult to draw any conclusions. Everything leads toassumptions, and the only thing one can do is trying to draw your own conclusions and rec-ommendations from these assumptions and later on implement them in your own professionor study. However, I believe this literature study have provided enough results to answer theresearch questions stated in Section 4.1.1.2. As well as being able to suggest to Etteplan howthey should proceed in the process of implementing a new CMS.However, no literature review is perfect and there is a potential risk that I have missed somecrucial factors when it comes to competency assessment and how it should be done. Fur-thermore, one recommendation when performing a literature review is having someone peerreviewing the study. Confirming its validity and making sure the process have been per-formed correctly. This have unfortunately not been done in this case, which is a threat to the

23

4.2. Questionnaire results

validity of this review. Lastly, Kitchenham and Charters [25] mentions that one of the primarythreats to literature reviews is that studies tend to be publication biased. Researchers have atendency to not present negative results, as this could shine a negative light on their study.An example of a negative result could be potential affects on individuals while having theircompetencies assessed and mapped. This would explain why I had a difficult time findingarticles regarding this topic.

4.2 Questionnaire results

As mentioned in Section 3 a questionnaire was distributed to all employees at Etteplans of-fices in Linköping and Norrköping. The questionnaire was sent to 121 employees, where 55answered. This is a response frequency of 45% which is quite good.The demography of the questionnaire is presented in Figure 4.1, Figure 4.2 and Figure 4.3.A majority of the respondents are employees located in Linköping (76%), which was ex-pected. Furthermore, the majority of the respondents belong to the mechanic department(44%), where the second largest represented department is software development (31%) andthe remaining questionnaire participants work with technical documentation (20%) and other(5,5%). When it comes to work experience at Etteplan, the result shows that a considerableamount of the respondents began their career at Etteplan and have worked at the companyfor 0-3 years (60%).

Figure 4.1: Distribution of where respondents are located

24


Figure 4.2: Department distribution of the respondents

Figure 4.3: Questionnaire respondents amount of years working at Etteplan

4.2.1 Etteplans Employees Thoughts on Competency

The distributed questionnaire partly consisted of a section regarding competency, where thegoal was to develop a clear picture of how important the employees think it is to map dif-ferent kinds of competencies. Followed by how the employees define competency. As men-tioned in Section 2 competency can be divided into three categories: skill, knowledge andbehavioural based competencies. In the questionnaire, the participants graded how impor-tant they thought it was to map each category with a grade from one to five. As seen inFigure 4.4, Figure 4.5 and Figure 4.6, the majority of the employees find it quite importantto map all three categories. However, it is possible to conclude that people find it more rele-vant to map skill and knowledge based competencies. The opinions are very similar betweenthese two categories, with as many votes on both grade four (20%) and five (50,9%). The onlydifference is that one person (1,8%) considers it to not be that important to map skill basedcompetencies. Whereas the opinions are more evenly divided between grades three (29,1%),

25


four (40%) and five (25,5%) when it comes to behavioural competencies. However, a smallgroup of people (5,5%) do not find it particularly important to map behavioural competenciesat all.

Figure 4.4: Distribution of how important people think it is to map skill based competencies

Figure 4.5: Distribution of how important people think it is to map knowledge based compe-tencies

26


Figure 4.6: Distribution of how important people think it is to map behavioural based com-petencies

Figure 4.7 presents how the respondents define competency, and what competency as aconcept means to them. The respondents had the possibility to chose several alternatives onthis question, seeing as competency is a term with a wide range of definitions that can differfrom person to person. Following this question the questionnaire participants had the chanceto give their own definition of the concept, since competence is such a wide concept. As men-tioned, people have their own definitions of competency and by giving the respondents thepossibility to give their own definition eliminates the risk of missing any valuable informa-tion.A majority of the respondents agree that practical knowledge (90,9%), theoretical knowledge(85,5%) and cognitive skill (85,5%) are three terms describing competency. Furthermore, char-acteristics (36,4%) is the fourth most picked definition. Just over a third of the respondentsconsiders behaviour and how an individual typically behaves to be a competency. Lastly, theremaining three definitions: personal values (16,4%), self-image (14,5%) and personal mo-tives (20%) have an even amount of answers.

Figure 4.7: Respondents definition of competency

27


A couple of respondents gave their own view on competence and what defines the con-cept according to them. Some considers everything to be competencies and you use differentcompetencies within different professions and during different situations. A skill or knowl-edge that might classify as a competency within the medical care might not categorize asa competency within a technology company. Ultimately, one person recognizes EmotionalQuotient (EQ) to be a competency. EQ is the capability of individuals to identify their ownand others emotions, differentiate the emotions and categorizing them correctly. In additionto have relationships with others and being able to show empathy. Furthermore, EQ is theability to use emotional information to manage and adjust thinking and behaviour in order toadapt to a specific situation or environment. Lastly, EQ treats the ability to understand whypeople act the way they do in certain situations [38]. This was an interesting and unforeseenanswer, and only strengthens the fact that everyone have their own definition of competency.

4.2.2 Etteplans Employees Thoughts on Competency Management System

Figure 4.8 depicts how many of the survey participants believe their individual developmentwould, or would not, benefit from using a competency management system. A majority(85,5%) of the respondents believe their personal development would benefit from an im-plementation of a new and well-suited competency management system. The questionnaireparticipants later had the chance to follow up their answer by giving their opinion on howthey think they would benefit from using a CMS, or possibly why they would not benefitfrom it.A popular reflection was that implementing a well functioning CMS would lead to a signif-icant increase in awareness regarding which competencies you possess, and which compe-tencies you need to develop. A competency management system would also bring attentionto personal strengths and weaknesses, making it easier to build development plans in orderto reach career milestones and ultimately your career goals. By having a clear developmentplan, it becomes easier to plan future educations, and find proper ways to develop compe-tencies.Furthermore, one common expectation was that mapping competencies in a proper mannershould increase the chance of receiving a relevant job and ending up at a relevant company.Instead of finding out too late that you have been given a project you either dislike or areunqualified to accomplish. The last common denominator amongst the opinions given bythe respondents, regarding why implementing a CMS could benefit the individual develop-ment was that such a system would help when trying to remember what you have doneand accomplished throughout your career. For example remembering tasks and projects youliked or possibly disliked participating in. Nowadays it is easy to forget these things, whichmight lead to participating in similar projects as before, projects you actually disliked work-ing on. Having a competency management system where you can record old projects andtake notes on each of them should lead to an increased chance of working with somethingyou truly liked and felt motivated performing. Pleased employees leads to increased motiva-tion, which leads to better performances, which ultimately leads to enhanced results for thecompany.The respondents identified a couple of negative consequences that could emerge when im-plementing a CMS as well. Where the most commonly mentioned factor is how mappingemployees could be problematic from an ethical point of view. The two main worries peoplehad considered the type of information shared on the platform, in addition to which peoplewho actually have access to the information. Some respondents did not like the idea of shar-ing "personal information" to unauthorized people. Another possible problem with a CMSis who will be in charge of updating and adding competencies to users profiles. People withlow self-awareness might add competencies they believe they possess, when they in fact donot. However, this is less of a problem if the competencies are easy to prove, such as practi-cal and theoretical competencies. For example certifications gained throughout your career.

28


Lastly, negative feedback might lead to unmotivated employees which could lead to poorperformance. Employees might feel monitored and graded as well, possibly affecting theirmotivation and performance.

Figure 4.8: Distribution over whether the respondents believe implementing a CMS will ben-efit their personal development or not

The questionnaire contained the question "how do you think Etteplan would benefit fromobtaining a clearer picture of their employees competencies?", where the respondents had thechance to answer freely. The most common answer were that it would simplify the processof finding the best suited consultants for a project. Correct person would be assigned to thecorrect job. Furthermore, it would help project leaders to get an overview of the team andeach members competencies, strengths and weaknesses. Implementing a competency man-agement system would also inform Etteplan if they are in need of competency developmentwithin certain technology fields. At the same time, the employees can get a more individualcompetency development, which often is viewed as something positive amongst the employ-ees.Another common opinion given by the respondents was that mapping competencies couldshed light on competencies employees might not use too often. Either during past projects orcurrent project. Giving the employees a chance to work with projects unlike anything theyever worked with previously.Ultimately, it would benefit everyone who ever find themselves in a place where they are inneed of help from someone. By having the possibility to easily find someone who possess therequired competencies and then being able to ask them personally, eliminates the require-ment of asking an executive to find someone capable of helping.Figure 4.9 depicts whether the respondents believe they would act differently or make dif-ferent career choices if they used a system where they could map and follow their personaldevelopment. The distribution was even, with one more respondent believing they wouldact differently (50,9%) if using a CMS.

29


Figure 4.9: Distribution over whether the respondents believe using a CMS would change theway they acted when making different career choices

The questionnaires participants were asked to answer what functionalities they wishedfor in a CMS. Figure 4.10 shows that all propositions are relevant functionalities in a CMS, butsome of them are more significant than others. A clear visual functionality for competencymapping (83,3%), the possibility to update and manage your own profile and competencies(79,6%) and having the alternative to see which competencies that have to be acquired anddeveloped to reach a certain point in your career (75,9%) had a higher priority than the re-maining functionalities. Following up and evaluating completed projects (68,5%) and somekind of learning management system for courses and development of ones competencies in-tegrated within the CMS (61,1%) were two components not as desired as the former three,but still two components with more than 50% of the respondents wishing for them. The func-tionality with the least amount of votes were the possibility for performance managementand reporting (37%) following a completed project.

30

4.3. Interviews

Figure 4.10: Distribution over which functionalities the respondents wish for in a CMS

Finally, the respondents had the chance to leave additional thoughts regarding the im-plementation of a new CMS. The by far most common request is for the system to be user-friendly and for the competencies to be displayed using some visual functionality. Respon-dents suggested some kind of tree structure, inspired by skill trees used in computer andvideo games. The user profiles should be unequivocal with distinct development pathsand course management. Being able to see colleagues chosen courses and career paths wassomething many desired as well. Furthermore, one proposal was to have a divided system.With one "closed" part with personal information that each user wish to keep for themselves.Strength and weaknesses that might not be of interest for colleagues, but still competenciesthe user wish to develop. In addition to an "open" part of the system filled with practicalcompetencies, such as programming skills. Information useful for everyone involved in thecompany.Nevertheless, one respondent indicated that these kinds of systems might be received differ-ently. Some might feel motivated to achieve their personal goals and develop, while othersmight feel monitored by this kind of mapping. Therefore, it is important to be clear on howthe implementation of a new CMS benefits not only the employees but also Etteplan as acompany.

4.3 Interviews

Four employees at Etteplan were interviewed so I could get a clearer and wider picture ofwhat the employees think regarding competency, and their thoughts regarding the imple-mentation of a new competency management system.

4.3.1 Opinions Regarding Competency

The interviewees had several different definitions of competence, but essentially it can all besummarized as the ability to perform a specific task. Using both practical skills and socialskills. Competencies are something measurable you acquire by performing tasks. More ex-perience leads to more competencies, which leads to the ability to undertake more difficulttasks. Behavioural competencies and characteristics are competencies as well according tothe interviewees. Even though you might be an expert in your area of expertise, without

31

4.3. Interviews

social skills and the ability to express your competency you might seem less competent thanwhat you truly are. Social competence also considers how you act while interacting with agroup of people, consisting of anything from friends to colleagues.Some competencies are clearer and easier to identify and classify than others. When apply-ing for a new job there are some requirements one have to fulfil. These are competencies onemust possess and either you have them, or you do not. It is easy for the employer to identifyand test these competencies. However, behavioural competencies might be more difficultto test and identify just through a couple of interviews. These competencies might not berequired, merely competencies the employee wishes the applicant possesses. Competenciesvaries from situation to situation. Some skill or behaviour might classify as a competencywithin a certain profession, but not within another.The interviewees thought personal values could affect competencies and how they appear.However, values and motives are not competencies according to the interviewees. One ex-ample given by one of the interviewees was if a person have values and opinions against thetobacco industry, then this person lacks the required competencies to work with marketingfor cigarettes. Another example could be about having to work with someone who have po-litical opinions you object with. This might interfere with completing a task and could resultin a negative atmosphere. However, if you are capable to leave your opinions at the doorand complete your tasks without any conflicts, then you have the competencies required toperform the job. As said before, if you are capable to complete and carry out your work, youpossess the required competencies.As mentioned above, personal values, motives and backgrounds rather affect how compe-tencies appear. Two individuals with the exact same competencies, but with different back-ground might present their competencies differently. People act different during differentmoments.

4.3.2 Ethical Aspects to Consider When Mapping Competencies

Mapping peoples competencies, skills and possibly even personality traits comes with ethi-cal difficulties. It is not possible to avoid taking this into consideration when assessing andmapping employees competencies. However, the interviewees agree upon behavioural com-petencies being the more questionable category to map from an ethical point of view. Prac-tical skills are easily tested to establish an understanding of the competencies individualspossesses. Behavioural competencies, however, are significantly more difficult to test and in-dividuals might need a considerable amount of time in an environment to apply and showtheir behavioural based abilities.One interviewee mentions the potential ethical difficulties with testing peoples behaviouralcompetencies, not just mapping them. The interviewee sees no difference, from an ethicalpoint of view, in assessing behavioural competencies if it is up to each and everyone to gradeand assess themselves. In comparison to testing how well you manage to complete a pro-gramming task. However, it is difficult to justify exposing individuals for real stress tests,especially if the persons are unaware of the assessment. This could imply some ethical diffi-culties.The interviewees was introduced to the idea of having a divided system with one "open" partand one "closed" part. An implementation idea all four participants agreed would be relevantand reasonable. The open part would consist of information relevant for all employees, suchas practical competencies, courses taken and such. Whereas the closed part of the systemwould consist of information only important for you personally and your executive. For ex-ample personal goals, behavioural competencies one might not want to share, competenciesunder development and development plans. Users could be able to decide for themselveswhat they would like to share or keep secret.One common denominator for all four interviews was that none of the participants personallyhave any issues with mapping or talking about their competencies, nor having their compe-

32

4.3. Interviews

tencies assessed by colleagues and supervisors. Neither knowledge based or behaviouralbased competencies. Everyone considered it to be relevant and most of all interesting to havea map of their competencies. The map can then be used to pinpoint what competencies theyshould develop in order to become as good as possible at their job. Identifying these knowl-edge gaps should increase the chance of creating a well suited development plan togetherwith their executive.As mentioned above, the interviewees claims they understand the risk of mapping and as-sessing employees competencies from an ethical point of view. However, there is a possibilitythat the interviewees might underestimate the potential problems with mapping and assess-ing individuals competencies in such a detailed and thorough manner. People might find itstrenuous being thoroughly assessed and there is a risk employees feel openly questioned, oreven personally attacked.One of the interviewees mentioned how important it is to especially map consultants compe-tencies, seeing that the entire business model is based on selling competencies and competentindividuals. The consultant executives needs to know what skills and knowledge they haveavailable in order to match correct consultant with the correct project. Furthermore, it is inthe consultants own interest being assigned to a project suited for the person. Minimizing therisk of being allocated to a job where the requirements later appears to be way too high. Beingat a workplace where you feel superfluous and as if you can not provide any knowledge isvery destructive.Another potential ethical problem is what information and data you as an employer actuallyis allowed to store, especially considering GDPR. GDPR restrains organizations from storingpersonal data without any legal basis. However, this is not a problem as long as everyonegives their consent and approval for the data to be stored and mapped.

4.3.3 How the Current CMS is Used Today

The opinions concerning the CMS currently in use are quite unanimous, all four intervieweeswould like to see a new system being implemented. The interviewees only use the systemwhen they are required to do so. This is when updating their consultant CV, along withpreparing for individual performance appraisal and when planning career goals for eachemployee. Furthermore, the system is used when hiring new personnel and when writingnew employment contracts. Lastly, the system should also be used when reporting part-timework, parental leave and such. The interviewees try to use the system as rarely as possible.

4.3.4 Preferred and Requested Functionality in a New Competency ManagementSystem

When asked about desired functionalities in a new CMS, the interviewees mentioned a num-ber of different proposals. However, the main request from each of the interviewees was forthe new CMS to be user-friendly and consist of some visual functionality when it comes tofeaturing and displaying competencies. Everything should be defined in a clear and concisemanner, for example what employees are required to do when it comes to adding compe-tencies to their profile. I introduced the interviewees to the idea of displaying competenciesusing a skill tree, which was received positively. Everything has its place in a tree structure,making it clear and easy to find what you are looking for. In addition to adding your owncompetencies to the tree. A tree structure would also make it easier to see where each indi-vidual needs development in order to reach certain points in their career.Another collective request was the possibility to see other users profiles. Simplifying the pro-cess of finding someone with the correct competencies when in need of help. Eliminating theneed to ask someone who might be able to redirect you to a relevant person.One interviewee came with the idea of categorizing and tagging former jobs and projects youhave been involved in. For example if you had the roles project manager and Java developer

33

4.3. Interviews

during a project, you should be able to tag an evaluation report with "project manager" and"Java developer". As a short summary of what you did during the job. This functionalitywould also make it easier to match consultants with jobs, along with finding people who canhelp you when you have encountered a problem. Writing personal evaluation reports fol-lowing a completed project would also result in the consultant having to reflect over possiblecompetencies acquired during the project. In addition to other relevant things learned, forexample things to consider every time you enter a new project. Making sure things are exe-cuted in the correct order.Implementing a functionality to see how your competency tree have expanded and devel-oped over time was also an idea everyone considered exiting and something that wouldincrease your motivation. Being able to see how you have grown and developed as a personduring your career.

34

5 Discussion

This section will analyze and discuss the entirety of this thesis. Mainly focusing on the meth-ods used to conduct this thesis and the collected results from the systematic literature review,the questionnaire and finally the interviews. These two main sections will be critically as-sessed, and potential weaknesses and improvements will be identified and discussed.

5.1 Method

This study is based on three different methods: one systematic literature review where pre-vious studies within the field of competency were analyzed, a quantitative research usinga questionnaire and a qualitative research where four semi-structured interviews were con-ducted. I consider these three methods well suited for this kind of study, since it is importantto adapt to the individuals being evaluated and to take their emotions and thoughts into con-sideration when performing competency assessment, evaluation and mapping. Therefore, itis important to collect as many opinions, thoughts and ideas as possible from the people whowill be taking part in the competency assessment process later. Conducting a questionnaire isthe perfect way to gather as many opinions as possible from many individuals during a shortperiod of time. The interviews could then be used to obtain a deeper understanding withincertain specific questions of interest.A systematic literature review was relevant to perform in order to find similar studies, or atleast studies within the field of competency. In order to get a broader understanding of thesubject and possibly identify angles of approach that I might not have considered. In retro-spect, it probably would have been wise to spend more time on identifying more keywords,thus being able to create better and more detailed query strings. Two words that I shouldhave added are skill and knowledge. Furthermore, a potential disadvantage with systematicliterature reviews is the risk of missing out on qualified studies that possibly would havebeen better than the ones generated by the query strings. This is a potential issue becauseyou can not control which studies you will end up with. The only thing one can do is tocome up with more detailed and well planned keywords. Another solution to minimize therisk of missing any relevant articles is to use chain sampling to search through each primarystudies references to try to find useful articles. Chain sampling was used during this studyand resulted in one additional study.Conducting more interviews would of course generate more data and minimizing the risk

35

5.2. Results

of missing any valuable suggestions and opinions. However, conducting a larger amount ofinterviews would have taken too much time, since I have been working on this thesis by my-self. Furthermore, I believe the collected data is sufficient enough and represents the generalopinion amongst the employees.As mentioned in Section 3.2.1.3 I gave the employees two weeks to answer the questionnaire,and my plan was to extend the time-frame if I found it to be necessary. I sent out a remindera couple of days before deadline which generated additional answers. Enough answers forme to decide to close the survey after two weeks, since I felt that I had received a consider-able amount of data. I do not believe extending the response period would have generatedenough answers for it to be a reasonable decision. People usually respond to a questionnaireat the beginning when they receive a message regarding the survey, or after a reminder. Ex-tending the response period would most certainly have resulted in me having to send out areminder once more, which would have felt repetitive and most likely not resulted in a con-siderable amount of additional answers.As mentioned in Section 3.2.2, Hove and Anda [23] highlights the importance of having a lotof experience with conducting interviews, in addition to having extensive knowledge of thetopic of the interview. However, I lack experience when it comes to conducting interviewsand my knowledge of competency and competency assessment was limited when perform-ing the interviews. I believe my insufficient experience of conducting interviews showedduring the interviews. The first interview was not as semi-structured as I would have liked.It felt more like an actual interview than a conversation and I did not feel as comfortable as Ihad hoped conducting the interview. Furthermore, the amount of supplementary questionswere too few and the quality were somewhat too low. However, I became more comfortablefor each interview and the quality of both the interview as a whole but also the supplemen-tary questions rose for each conducted interview as well. Each interview was longer thanthe previous one, which I interpret as a sign of me becoming more comfortable during eachinterview. Someone with more experience than me would probably have been able to collectmore data from the interviews, but I am satisfied with the results and I have developed mypersonal interviewing abilities as well.Initially the idea was to compare existing competency management systems, to try and iden-tify the best way to build and implement a new CMS. However, it turned out to be ratherdifficult to gain access to demo versions of the systems. This will be discussed further inSection 6.1.

5.2 Results

Using a system for competency management and having a process of assessing, mapping anddeveloping employees competencies at your company is in theory an effective approach forthe company to develop and stay in the cutting edge of technology. However, in reality thereare several underlying factors to take into consideration when implementing a competencymanagement system and adopting a competency assessment process. Merely implementinga CMS and informing your employees to use it will most likely not generate any results, it isnot that easy. It requires thorough and hard work in order for the process to work the bestway possible.Completely satisfying everyone is however more or less impossible, but the goal should al-ways be for as many employees using the system as possible and that they feel motivatedusing it. The users should feel as if they benefit from mapping and assessing their compe-tencies, in addition to creating development plans. It is important to have everyone knowingthat the implementation of a CMS not only benefits them personally, but also Etteplan as acompany.

36

5.2. Results

5.2.1 Systematic Literature Review

Conducting a systematic literature review at the beginning of a theoretical study like this oneis a suitable approach in order to gain a broader and deeper understanding of the studiedsubject. I would however have liked to find more studies where the ones conducting thestudy implemented a CMS within an organization or a company, but I am happy with the re-sults and I consider the generated studies to be adequate both in terms of quantity but also inquality. The primary studies answered the research questions defined during the planning ofthe review. However, it turned out to be difficult to find studies regarding potential effects onindividuals having their competencies assessed. As i mentioned in Section 4.1.3 a potentialreason for this might be that the studies are publication biased. Negative results are seldompublished and someone being affected by competency assessment negatively might be leftout in the final article.The majority of the primary studies are all specified in certain fields, for example the medicalcare and within an educational environment. Meaning that you can not merely read a studyconsidering competency assessment and then just apply the method to their organization.You have to draw your own conclusions and analyze what you have read, and then makeyour own assumptions regarding what works best in your situation and for your organiza-tion or company.The main conclusion one can draw from the literature review is that there are no incorrector correct procedures to follow when it comes to assessing and evaluating individuals com-petencies. It all amounts to finding the proper solution for your situation. All organizationsand companies are unique and they all need their own solution and process for mapping, as-sessing and developing competencies. Although the most common approach when it comesto assessing competencies seems to be self-assessment, in addition to having an executiveassessing your competencies and performances.The wide range of assessment approaches and the lack of a definite assessment process comeswith both its advantages and disadvantages. Advantageous in the sense that you plan the as-sessment process and the implementation of the system however you like, in order to findan approach suitable for your company. This, on the other hand, could be seen as somethingnegative as well. Not having any directives on how to begin the process and proceed withthe work might end up in the developing procedure stagnating.Assessing and evaluating peoples competencies comes with both challenges and potentialrisks. Section 4.1.2.2 presents several challenges that might occur during the assessment pro-cess, where the ethical aspect is one of the most crucial factors to consider when performingcompetency evaluations. Everyone involved should be in the clear on how the mapped com-petencies are used and how it benefits their work and their personal development. Assessingand testing practical competencies are rarely problematic, but when it comes to somehowtesting and assessing behavioural based competencies difficulties might occur. Because youevaluate individuals personalities and characteristics, which is something people might reactnegatively to. Therefore, it is necessary to be careful and plan the assessment and evalua-tion process carefully, in order to eliminate the risk of any ethical dilemmas occurring andreducing the stress and other potential affects on the employees.

5.2.2 Questionnaire

I believe the questionnaire gave a clear picture of what the employees at Etteplan thinks ofcompetency assessment and competency management. In addition to functionalities theywould like to have in a new competency management system. The questionnaire also identi-fied some important problems with the current competency management system that needsto be addressed and changed for the new system to be successful. For example, it is un-clear how all the information everyone is asked to add to the current system actually is used.Which makes people unmotivated to use the software.

37

5.2. Results

I am satisfied with the response rate (45%), even though it turned out to be significantly lowerthan what I hoped for (80%) when designing and planning the survey in Section 3.2.1. In ret-rospect, I understand how naive I was when hoping for such a high response rate. Internalsurveys within an organization or a company generally generates a response rate of 30-40%on average [18]. The generated response rate from the questionnaire conducted during thisstudy was therefore above average.One interesting result gathered from the questionnaire was that the exact same amount ofpeople voted for the grade three and four in figures 4.4 and 4.5. Regarding which competen-cies the respondents consider important to map. This could be a complete coincidence, but Ibelieve the main reason for why people graded skills and knowledge the same is because thequestionnaire was conducted at a consulting company. Where technical competencies are atfocus and the actual selling point for these kids of companies. Furthermore, another possi-bility is that the respondents views skill and knowledge as synonyms and therefore picked thesame grade on both questions. I can see why people might see skill based and knowledgebased competencies as the same thing. For example an individuals programming abilitiescan be viewed as both a skill and a knowledge. I gave the respondents a definition of eachcompetency category in a short paragraph before the questions, but there is always a riskof people ignoring pieces of text when they see that the text itself does not contain a ques-tion. This is something I possibly should have analyzed further. However, it is difficult togather any clear answers regarding this issue since there is no way to contact questionnairerespondents and ask supplementary questions. In retrospect I maybe should have asked therespondents to add their e-mail to their questionnaire answers, so I could contact them if anyfurther questions emerged when reading the answers.As mentioned in Section 4.2.1 and seen in figure 4.7, a clear majority of the respondents definecompetency as practical ability, theoretical ability and cognitive ability. In comparison to personalvalues, self-image, personal traits and personal motives, four definitions that received less thanhalf of the votes as the three former ones. A reason for this could be that practical, theoreticaland cognitive competencies are easier to identify and assess. While behavioural competen-cies, i.e. the four definitions fewer picked, are more difficult to pinpoint and analyze. Thiscorresponds with the theory and what was mentioned in Section 2.3.2. The competenciesabove the waterline are easier to identify and assess, while the competencies below the lineneeds more time to be assessed correctly. Furthermore, the competencies below the waterlinedirectly affects the ones above the line, either deliberately or unintentionally. Two individu-als with the exact same competencies, but who are from two entirely different backgroundsmight present their competencies differently. People who is more quietly spoken and lackscommunication skills might seem more incompetent just because they find it difficult com-municating their competencies. Whilst people who have an easy time talking can communi-cate their competencies.It can clearly be seen that the majority of the respondents have a positive attitude towardscompetency assessment and the implementation of a new competency management system,given that the system is user-friendly and its purpose is apparent. The two main functionali-ties people would like to have in a CMS are: a clear visual functionality of your competenciesand some sort of learning management, where you can manage courses and developmentplans. Visualize an individuals competencies using a skill tree is something that have beenmentioned several times throughout this thesis, and it is something I believe would motivatepeople into using the system more often. A skill tree makes it easy to follow and plan yourdevelopment, in addition to offering a clear picture of what you have to do in order to reach adesired point in your career. A skill tree could be designed in several different ways, but oneidea is to have a picture of the employee as a starting node and have different career paths asbranches.One interesting idea presented by one of the questionnaire participants was the possibilityto have a divided CMS. Where the "open" part of the system contains information and datarelevant to your colleagues, for example practical competencies and skills. This information

38

5.2. Results

is relevant for everyone to see in order to simplify the process of finding help when needed.Instead of having to go to your executive and ask them to find someone suitable. Further-more, the "closed" part of the system could for example consist of a development plan as wellas competencies that currently are under development. This information is only relevant toyou and your executive, since the development plan is created in consort with your closestexecutive. Users might want to add competencies that currently are being developed to thetree, but they are not comfortable enough with their skill level in order to show it to all col-leagues. Making a closed part of the system very relevant and this is something I believewould increase the amount of people using the system regularly, instead of just updatingtheir profiles once a year.Lastly, an area which I had not identified as a competency, but one of the respondents men-tioned was Emotional Quotient (EQ). Possessing the ability to identify your own and othersemotions, and on the basis of this know how to behave can definitely be seen as a compe-tency. Being able to adapt to a situation only by reading peoples emotions could turn outto be very beneficial. This only justifies how wide of a concept competency actually is, andeveryone has their own definitions.

5.2.3 Interviews

One interesting idea that emerged during the interviews was to have some sort of develop-ment history functionality in a new competency management system. Making it possible tosee how you have developed and how your skill tree have grown throughout your career. Ibelieve having the possibility to see how you have developed and what competencies youhave acquired will lead to increased motivation and the will to keep on growing, both as aconsultant but also as a person when developing behavioural competencies. Increased moti-vation leads to enhanced performance which benefits both the consultant and the company.I introduced the interviewees to the idea of presenting competencies using a skill tree, whichonly received positive reactions amongst them. The interview participants agreed on havingsome sort of tree structure in order to visualize competencies would make things very clearand easy-to-understand. Furthermore, they believed having a skill tree would simplify theprocess of finding useful information about each other.Implementing a skill tree makes the concept Gamification come to mind. Gamification is theapplication of game based elements and game principals in non-game contexts [46]. Im-plementing a skill tree in a company environment fits the definition perfectly. The idea of"leveling up" your competencies and choosing a specific route in the skill tree is a conceptoften used in video games, and the method is definitely suitable to apply within a consultantenvironment in order to map employees competencies.Another interesting thought one of the interviewees had that emerged during one of the in-terviews regarded potential ethical dilemmas when performing competency assessments. Itis not only the mapping that could prove to be problematic, but also the actual testing of thecompetencies. Especially when testing behavioural competencies. It could for example besomewhat questionable to test an individuals stress resistance, if the person is unaware of thefact that they are going to be exposed to these tests. This is an opinion I share. I believe thatthe one in charge of the assessment and the person conducting the tests have to be extremelyclear on the entirety of the process. Which competencies that will be tested and how the testswill be executed and conducted. To minimize any risk of someone feeling uncomfortableduring the tests.One thing I find important to highlight is that none of the interviewees have any concernswith their competencies being assessed and mapped. They only view it as something pos-itive and they all agree that this process only would benefit their development and career.After all, it is a part of the job as a consultant and the entire business model is about sellingcompetencies. Making it relevant to have a clear picture of what jobs employees are capa-ble of undertaking. It is in your own interest as a consultant to end up in a project where

39

5.3. Proposed Competency Assessment Approach at Etteplan

you feel as if you can provide knowledge and competency. This might be an opinion manyconsultants share, since all four interviewees said this during the interviews.

5.3 Proposed Competency Assessment Approach at Etteplan

Implementing a competency management system within a company or organization and forit to work in the best possible way is a lot more complex than one might think. I have duringthis study gained a much deeper understanding of how wide of a concept competency actu-ally is, and I realize that you can not just find a solution online and apply it on your companyin hope for it to work properly. It requires much harder work than that. Every company isunique in its own way and needs to find their own process and method suitable for it in orderto be as successful as possible.The company have to be very clear towards their employees regarding how all the informa-tion in a CMS will be used, and how it benefits them and the company. Thus motivatingpeople to use the system more frequently and updating their profiles more often, and not justafter a completed job or project.To my understanding the employees at Etteplan wants a system that is much more clear anduser-friendly than the system used today. The two functionalities most people requested wasa visual functionality for mapping competencies, where a perfect solution for this would beto use a skill tree in order to visualize users competencies. The second functionality peopleasked for was some sort of learning management system for managing courses and develop-ment plans. My advice is to have these two functionalities as the main parts in a new CMS.The users profiles should consist of a skill tree presenting all competencies one possesses,making it clear and simple to identify in which areas an employee is capable of working in.Another proposition is to have some kind of functionality to scroll back in the tree to see howthe skill tree have grown throughout the years. An example for this could be to have a simpleslider control combined with a time line.Furthermore, the system should be divided into two parts, one open part and one closed.The open part should consist of the skill tree and what courses people have taken and areparticipating in, and it should be accessible for all colleagues. If people are in need of helpthere should be a simple way of finding anyone who possess the competency required tosolve the issue. Whilst the closed part only should be available for the users themselves andthe executive who the employees constructs their development plan with. The closed partshould also consist of a skill tree where the user can add competencies under development,and competencies that the user temporarily do not want to share with anyone. However,there is a potential problem regarding behavioural competencies and which part they shouldbelong to. Some behavioural competencies are relevant to display in the open part, for exam-ple an individuals leadership skills, since this competency is relevant to possess for certainjobs. My advice is to differentiate behavioural competencies and have some in the open partof the system and some in the closed part.Filling up the system with all relevant data and everyone’s competencies manually wouldtake forever and would be really tedious. This is something that should be done automat-ically and a possible solution for this could be to use data mining presented in section 2.2.The optimal solution would be if it was possible to mine the system that is used today andtransfer all of the data in this system to the new one. This would be the best approach to solvethis problem since all employees at Etteplan already have a profile in the currently used sys-tem filled with their competencies and skills. Everyone could then complement and changetheir profiles as they wish after the process is completed. Another potential solution could beto mine employees LinkedIn profiles, if possible, since more or less everyone have a profilenowadays showing their competencies, skills and knowledge.Frequent Itemset Mining presented in Section 2.2.1 is a method used to find patterns in datasets. This technique could be used when creating the skill tree and deciding upon required

40

5.3. Proposed Competency Assessment Approach at Etteplan

competencies to acquire in order to reach a certain point in your career. One example could beto data mine and analyze project managers profiles in the currently used CMS to find mutualpatterns in their competencies and courses they have attended to become project managers.These competencies and courses could then be used to draw the "project manager"-branch inthe skill tree, giving a clear picture of what one must do to become a project manager.

41

6 Conclusion

It is difficult to determine how a company should work with competency management, as-sessment and mapping. The process is considerably more complex than what one might thinkand it requires hard and thorough work to find the method and process most suitable for thecompany or organization. It is not possible to merely find a competency management processonline and implement it within your company, hoping for it to work and generate results. It isnecessary to consider employees thoughts and requests for them to feel motivated to use thesystem frequently and not just after a completed project or job. There is always a risk of peo-ple feeling insecure and finding it strenuous when having their competencies assessed andevaluated. Thus, making it important for the executives to be clear towards their employeeson how this assessment process benefits everyone at the company, employee, employer andthe entirety of the company. Furthermore, developing new software always comes with arisk and there is always a possibility for underestimation, regarding both difficulty and cost.The development process requires thorough planning to minimize and prevent any potentialrisks and difficulties that might occur during the process.However, I believe this thesis has been able to identify some guidelines that Etteplan shouldfollow if they ever plan to implement a new competency management system and when as-sessing and mapping their employees competencies. In addition to a few functionalities theCMS should consist of and be built around. A new CMS should consist of two main func-tionalities: a skill tree for mapping competencies and a learning management platform. Theskill tree is a visual representation of an employees competencies and should also displaycompetencies and courses you have to take in order to reach certain points in your career.The tree can only be managed and updated by the employee itself in addition to the closestexecutive. The learning management platform should consist of courses employees are capa-ble of attending and the development plan created together with the executive. Users shouldbe able to see their colleagues skill trees and which courses they are attending. This is impor-tant since employees should be able to locate people who possess certain competencies whenin need of help. Eliminating the need to go to an executive and ask them to find someonecapable of helping.Finally, this study have identified several potential approaches when it comes to assessingand evaluating individuals competencies. There are no right or wrong ways to conduct com-petency assessment. However, the majority of the approaches and methods are based on

42

6.1. Future research

self-assessment and assessment of others. Two procedures I recommend Etteplan using whenimplementing their own competency management process.

6.1 Future research

Competency and competency management is a very wide concept and this study has onlyscratched the surface of the vast quantity of ideas to realize when applying competency man-agement within a company. There are a lot of potential subjects to study in the future, inorder to optimize the competency management process.Given more time and resources to continue this study an idea for future research could beto expand the study and having it involve several other companies that have developedtheir own competency management systems. Analyzing their development process and howthey use their systems today. Furthermore interviewing and studying what their employeeshave to say about the systems and their thoughts on having their competencies assessed andmapped on a daily basis.My plan was to compare and analyze a number of competency management systems usedtoday. However, it turned out to be quite difficult to gain access to systems and you hadto represent a company in order to actually be able to take part of their demos. Therefore,analyzing and comparing several existing competency management systems could be a po-tential future research. Comparing the most popular systems with each other and identifyingthe factors for why these systems have the most users. In addition to identifying how thesesystems are developed. This data should lead to a better and more detailed decision for howa new competency management system should be implemented and what functionalities thesystem should consist of.

43

Bibliography

[1] N. Anisha. “Competency Mapping of the Employees”. In: International Journal of Ad-vancements in Research Technology 1.5 (Oct. 2012), pp. 67–73.

[2] Daniele Apiletti, Elena Baralis, Tania Cerquitelli, Paolo Garza, Fabio Pulvirenti, andLuca Venturini. “Frequent Itemsets Mining for Big Data: A Comparative Analysis”. In:Big Data Research 9 (2017), pp. 67–83.

[3] M. Bedek, S. A. Petersen, and T. Heikura. “From Behavioral Indicators to Contextual-ized Competence Assessment”. In: 2011 IEEE 11th International Conference on AdvancedLearning Technologies. July 2011, pp. 277–281. DOI: 10.1109/ICALT.2011.87.

[4] Mahdi Bohlouli, Nikolaos Mittas, George Kakarontzas, Theodosios Theodosiou, Left-eris Angelis, and Madjid Fathi. “Competence assessment as an expert system for hu-man resource management: A mathematical approach”. In: Expert Systems with Appli-cations 70 (2017), pp. 83–102. ISSN: 0957-4174. DOI: https://doi.org/10.1016/j.eswa.2016.10.046. URL: http://www.sciencedirect.com/science/article/pii/S0957417416305875.

[5] Alan Bryman. Social Research Methods. 4th. Oxford University Press, 2012.

[6] Eimear Burke, Marcella Kelly, Evelyn Byrne, Toni Ui Chiardha, Miriam Mc Nicholas,and Adrienne Montgomery. “Preceptors’ experiences of using a competence assess-ment tool to assess undergraduate nursing students”. In: Nurse Education in Practice 17(2016), pp. 8–14. ISSN: 1471-5953. DOI: https://doi.org/10.1016/j.nepr.2016.01.004. URL: http://www.sciencedirect.com/science/article/pii/S1471595316000056.

[7] David C. McClelland. “Testing for Competence Rather Than Intelligence”. In: The Amer-ican psychologist 28 (Feb. 1973), pp. 1–14.

[8] M. Calenda and R. Tammaro. “The Assessment of Learning: From Competence to NewEvaluation”. In: Procedia - Social and Behavioral Sciences 174 (2015). International Con-ference on New Horizons in Education, INTE 2014, 25-27 June 2014, Paris, France,pp. 3885–3892. ISSN: 1877-0428. DOI: https://doi.org/10.1016/j.sbspro.2015.01.1129. URL: http://www.sciencedirect.com/science/article/pii/S187704281501188X.

44

https://doi.org/10.1109/ICALT.2011.87

https://doi.org/https://doi.org/10.1016/j.eswa.2016.10.046

https://doi.org/https://doi.org/10.1016/j.eswa.2016.10.046

http://www.sciencedirect.com/science/article/pii/S0957417416305875


https://doi.org/https://doi.org/10.1016/j.nepr.2016.01.004

https://doi.org/https://doi.org/10.1016/j.nepr.2016.01.004



https://doi.org/https://doi.org/10.1016/j.sbspro.2015.01.1129

https://doi.org/https://doi.org/10.1016/j.sbspro.2015.01.1129

http://www.sciencedirect.com/science/article/pii/S187704281501188X

http://www.sciencedirect.com/science/article/pii/S187704281501188X

Bibliography

[9] G. Calikli, M. Al-Eryani, E. Baldebo, J. Horkoff, and A. Ask. “Effects of AutomatedCompetency Evaluation on Software Engineers’ Emotions and Motivation: A CaseStudy”. In: 2018 IEEE/ACM 3rd International Workshop on Emotion Awareness in SoftwareEngineering (SEmotion). June 2018, pp. 44–50.

[10] S. M. Collins and M. Nicholson. “Assessment of the behavioural component of compe-tency to substantiate practitioner competency claims”. In: 6th IET International Confer-ence on System Safety 2011. Sept. 2011, pp. 1–6. DOI: 10.1049/cp.2011.0270.

[11] Lynn Crawford. “Project management competence for strategy realization”. In: (Jan.1998).

[12] J. Dorn, M. Pichlmair, K. Schimper, and H. Tellioglu. “Supporting competence manage-ment in software projects”. In: 2008 IEEE International Technology Management Conference(ICE). June 2008, pp. 1–8.

[13] Fotis Draganidis, Paraskevi Chamopoulou, and Gregoris Mentzas. “An ontology basedtool for competency management and learning paths”. In: 6th International Conferenceon Knowledge Management (I-KNOW 06) (Jan. 2006).

[14] James W. Drisko. “Competencies and Their Assessment”. In: Journal of Social Work Ed-ucation 50.3 (2014), pp. 414–426. DOI: 10.1080/10437797.2014.917927. eprint:https://doi.org/10.1080/10437797.2014.917927. URL: https://doi.org/10.1080/10437797.2014.917927.

[15] D. Covalt Dunning, Chip Heath, and Jerry Suls. “Flawed Self-Assessment: Implicationsfor Health, Education, and the Workplace.” In: Psychological science in the public interest: a journal of the American Psychological Society 5 3 (2004), pp. 69–106.

[16] Thomas F. Gilbert. Human competence: Engineering Worthy Performance. McGraw-Hill,1978.

[17] Usama Fayyad, Gregory Piatetsky-shapiro, and Padhraic Smyth. “From Data Miningto Knowledge Discovery in Databases”. In: AI Magazine 17 (1996), pp. 37–54.

[18] Andrea Fryrear. What’s a good survey response rate? https://www.surveygizmo.com/resources/blog/survey-response-rates/. July 2015.

[19] M. Gaeta, F. Orciuoli, G. Fenza, G. R. Mangione, and P. Ritrovato. “A Semantic Ap-proach for Improving Competence Assessment in Organizations”. In: 2012 IEEE 12thInternational Conference on Advanced Learning Technologies. July 2012, pp. 85–87. DOI:10.1109/ICALT.2012.168.

[20] Priscilla A. Glasow. Fundamentals of Survey Research Methodology. 2005.

[21] Romain Guillaume, Raymond Houé, and Bernard Grabot. “Robust competence as-sessment for job assignment”. In: European Journal of Operational Research 238.2 (2014),pp. 630–644. ISSN: 0377-2217. DOI: https://doi.org/10.1016/j.ejor.2014.04.022. URL: http://www.sciencedirect.com/science/article/pii/S0377221714003488.

[22] Jiawei Han and Jian Pei. “Mining Frequent Patterns by Pattern-growth: Methodologyand Implications”. In: SIGKDD Explor. Newsl. 2.2 (Dec. 2000), pp. 14–20.

[23] S. E. Hove and B. Anda. “Experiences from conducting semi-structured interviews inempirical software engineering research”. In: 11th IEEE International Software MetricsSymposium (METRICS’05). Sept. 2005, pp. 10–23.

[24] dr Kaur. “Competency mapping: a gap analysis”. In: The International Journal of HumanResource Management2249-6874 (Mar. 2012).

[25] B. Kitchenham and S Charters. Guidelines for performing Systematic Literature Reviews inSoftware Engineering. 2007.

45

https://doi.org/10.1049/cp.2011.0270

https://doi.org/10.1080/10437797.2014.917927

https://doi.org/10.1080/10437797.2014.917927

https://doi.org/10.1080/10437797.2014.917927

https://doi.org/10.1080/10437797.2014.917927

https://www.surveygizmo.com/resources/blog/survey-response-rates/

https://www.surveygizmo.com/resources/blog/survey-response-rates/

https://doi.org/10.1109/ICALT.2012.168

https://doi.org/https://doi.org/10.1016/j.ejor.2014.04.022

https://doi.org/https://doi.org/10.1016/j.ejor.2014.04.022



Bibliography

[26] Seolim Kwon, Robert R. Wadholm, and Laurie E. Carmody. “Assessing competencies:An evaluation of ASTD’s Certified Professional in Learning and Performance (CPLP)designation”. In: Evaluation and Program Planning 44 (2014), pp. 48–58. ISSN: 0149-7189.DOI: https://doi.org/10.1016/j.evalprogplan.2014.01.002. URL: http://www.sciencedirect.com/science/article/pii/S0149718914000111.

[27] Tobias Ley and Dietrich Albert. “Identifying Employee Competencies in DynamicWork Domains: Methodological Considerations and a Case Study”. In: J. UCS 9 (2003),pp. 1500–1518.

[28] James Lichtenberg, Sanford M. Portnoy, Muriel Bebeau, Irene Leigh, Paul D. Nelson,Nancy J. Rubin, I Leon Smith, and Nadine Kaslow. “Challenges to the Assessmentof Competence and Competencies”. In: Professional Psychology: Research and Practice 38(Oct. 2007), pp. 474–478. DOI: 10.1037/0735-7028.38.5.474.

[29] P. Liu and X. Li. “The Application of Expertise Knowledge Map in Human ResourceManagement”. In: 2011 International Conference on Management and Service Science. Aug.2011, pp. 1–4.

[30] Robyn Longhurst. “Semi-structured Interviews and Focus Groups”. In: Jan. 2003,pp. 117–132.

[31] H. Lu, C. Lo, and P. Lin. “Competence analysis of IT professionals involved in businessservices — Using a qualitative method”. In: 2011 24th IEEE-CS Conference on SoftwareEngineering Education and Training (CSEE T). May 2011, pp. 61–70. DOI: 10.1109/CSEET.2011.5876158.

[32] Robyn A. McNamara. “Evaluating Assessment with Competency Mapping”. In: Pro-ceedings of the Sixth Australasian Conference on Computing Education - Volume 30. ACE’04. Dunedin, New Zealand: Australian Computer Society, Inc., 2004, pp. 193–199. URL:http://dl.acm.org/citation.cfm?id=979968.979994.

[33] D. Pandove and S. Goel. “A comprehensive study on clustering approaches for big datamining”. In: 2015 2nd International Conference on Electronics and Communication Systems(ICECS). Feb. 2015, pp. 1333–1338.

[34] S.E. Rabionet. “How I learned to design and conduct semi-structured interviews: Anongoing and continuous journey”. In: Qualitative Report 16 (Mar. 2011), pp. 563–566.

[35] Per Runeson and Martin Höst. “Guidelines for conducting and reporting case studyresearch in software engineering”. In: Empirical Software Engineering 14.2 (Dec. 2008),p. 131. ISSN: 1573-7616. DOI: 10.1007/s10664-008-9102-8. URL: https://doi.org/10.1007/s10664-008-9102-8.

[36] Thomas A. Runkler. Data Analytics: Models and Algorithms for Intelligent Data Analysis.2nd. Springer Vieweg, 2016.

[37] Seema Sanghi. the handbook of competency mapping: understanding designing and imple-menting competency mapping in organizations. 3rd ed. SAGE, 2016.

[38] A. Savarimuthu, L. Arockiam, and A. Aloysius. “A study on the impact of emotionalquotient on program comprehension”. In: 2010 International Conference on Communica-tion and Computational Intelligence (INCOCCI). Dec. 2010, pp. 357–361.

[39] Mousumi Sengupta. Competency Mapping at Toyota Financial Services Corporation.

[40] Veikko Seppänen. “Evolution of competence in software contracting projects”. In: In-ternational Journal of Project Management 20 (Feb. 2002), pp. 155–164. DOI: 10.1016/S0263-7863(00)00043-0.

[41] Ganesh Shermon. competency based human resource management a strategic resource forcompetency mapping, assessment and development centres. Tata McGraw-Hill Education,2004.

46

https://doi.org/https://doi.org/10.1016/j.evalprogplan.2014.01.002



https://doi.org/10.1037/0735-7028.38.5.474

https://doi.org/10.1109/CSEET.2011.5876158

https://doi.org/10.1109/CSEET.2011.5876158

http://dl.acm.org/citation.cfm?id=979968.979994

https://doi.org/10.1007/s10664-008-9102-8

https://doi.org/10.1007/s10664-008-9102-8

https://doi.org/10.1007/s10664-008-9102-8

https://doi.org/10.1016/S0263-7863(00)00043-0

https://doi.org/10.1016/S0263-7863(00)00043-0

Bibliography

[42] Julianna Katalin Sipos. Frequent Item set Mining Methods Jiawei Han und Micheline Kam-ber. Data Mining – Concepts and Techniques. Chapter 5.2.

[43] V. Stepanenko and A. Kashevnik. “Competence management systems in organisations:A literature review”. In: 2017 20th Conference of Open Innovations Association (FRUCT).Apr. 2017, pp. 427–433. DOI: 10.23919/FRUCT.2017.8071344.

[44] Miyuki Takase, Masako Yamamoto, and Yoko Sato. “The factors related to self-otheragreement/disagreement in nursing competence assessment: Comparative and cor-relational study”. In: International Journal of Nursing Studies 80 (2018), pp. 147–154.ISSN: 0020-7489. DOI: https : / / doi . org / 10 . 1016 / j . ijnurstu . 2018 .01.011. URL: http://www.sciencedirect.com/science/article/pii/S0020748918300221.

[45] P.N. Tan, Michael Steinbach, and Vipin Kumar. “Cluster Analysis: Basic Concepts andAlgorithms”. In: Introduction to Data Mining (Jan. 2005), pp. 487–568.

[46] Á. Tóth and S. Tóvölgyi. “The introduction of gamification: A review paper about theapplied gamification in the smartphone applications”. In: 2016 7th IEEE InternationalConference on Cognitive Infocommunications (CogInfoCom). Oct. 2016, pp. 000213–000218.DOI: 10.1109/CogInfoCom.2016.7804551.

[47] Robert W. White. “Motivation Reconsidered: The Concept Of Competence”. In: Psycho-logical review 66 (Oct. 1959), pp. 297–333. DOI: 10.1037/h0040934.

[48] Gary Weiss and Brian Davison. “Data Mining”. In: (Jan. 2010).

[49] R. Yuvaraj. “Competency Mapping – A DRIVE FOR INDIAN INDUSTRIES”. In: Inter-national Journal of Scientific Engineering Research 2 (2011).

47

https://doi.org/10.23919/FRUCT.2017.8071344

https://doi.org/https://doi.org/10.1016/j.ijnurstu.2018.01.011

https://doi.org/https://doi.org/10.1016/j.ijnurstu.2018.01.011



https://doi.org/10.1109/CogInfoCom.2016.7804551

https://doi.org/10.1037/h0040934

A Appendix A – Questionnaire

The questionnaire below was distributed through email and created using Google forms. Aparenthesis indicates a radio button while square brackets indicates that the response werecheck boxes, where the respondent could pick several answers.

Introduction

In which city are you employed?( ) Linköping( ) Norrköping

What department do you belong to?( ) Technical documentation( ) Software development( ) Mechanics( ) Other

For how long have you worked at Etteplan?( ) 0 - 1 years( ) 1 - 3 years( ) 3 - 5 years( ) 5+ years

Competency

The concept competency can be divided into three categories: skill based, knowledge basedand behavioural based competencies. Skill based competencies discusses an individualsability to perform a task. For example someones ability to program and complete tasks.Knowledge based competencies is someones ability to understand concepts and process in-formation. Lastly, behavioural based competencies is an individuals characteristics and traits.

How important do you think it is to map skill based competencies?Not important - 1 2 3 4 5 - Very important

48

How important do you think it is to map knowledge based competencies?Not important - 1 2 3 4 5 - Very important

How important do you think it is to map behavioural based competencies?Not important - 1 2 3 4 5 - Very important

How do you define the concept "competence"?[ ] Practical knowledge[ ] Theoretical knowledge[ ] cognitive ability (e.g. the ability to record, process and reproduce information)[ ] Personal values (e.g. how you look at the world and the people around you)[ ] Self-Image (e.g. how an individual sees oneself)[ ] Traits (e.g. how an individual usually behave)[ ] Personal motives (e.g. which motives that drive a person into performing specific tasks)

If you have your own definition, or any additional ones, then please write them here(Free text response)

Competency Management System

Do you believe your personal development would benefit from using a competency man-agement system?( ) Yes( ) No

How?/Why not?(Free text response)

How do you think Etteplan would benefit from having a clearer overview of their em-ployees competencies?(Free text response)

Do you think you would act differently, or make other career choices if you used a sys-tem where you easily could map and follow your personal development?( ) Yes( ) No

Which of the following functionalities would you like to have in a competency manage-ment system?[ ] Clear visual functionality for mapping of competencies[ ] The possibility to update and manage your own competencies[ ] The possibility to see which competencies you have to develop to reach a certain point inyour career[ ] Follow-up and evaluation of completed projects[ ] Performance management and reporting[ ] Some kind of learning platform (courses and development of personal competencies)

If you have any thoughts, ideas, proposals etc. regarding the implementation of a newcompetency management system. Or if you possibly have additional viewpoints regardingthe previous questions, please leave them here(Free text response)

49

B Appendix B – List of PrimaryStudies

50

ID Title Author Publ. year Ref.A1 Supporting competence man-

agement in software projectsJürgen Dorn, Markus Pichlmair,Kathrin Schimper & Hilda Telli-oglu

2008 [12]

A2 Competence analysis of IT pro-fessionals involved in businessservices — Using a qualitativemethod

Hsin-Ke Lu, Chia-Hui Lo &Peng-Chun Lin

2011 [31]

A3 Effects of Automated Compe-tency Evaluation on SoftwareEngineers’ Emotions and Moti-vation: A Case Study

Gul Calikli, Mohammed Al-Eryani, Emil Baldebo, JenniferHorkoff & Alexander Ask

2018 [9]

A4 From Behavioral Indicators toContextualized Competence As-sessment

Michael Bedek, Sobah Abbas Pe-tersen & Tuija Heikura

2011 [3]

A5 Assessment of the behaviouralcomponent of competency tosubstantiate practitioner compe-tency claims

S.M. Collins & M. Nicholson 2011 [10]

A6 A Semantic Approach forImproving Competence Assess-ment in Organizations

Matteo Gaeta, Francesco Orci-uoli, Giuseppe Fenza, Giusep-pina Rita Mangione & PierluigiRitrovato

2012 [19]

A7 Competence assessment as anexpert system for human re-source management: A mathe-matical approach

Mahdi Bohlouli, nikolaos Mit-tas, George Kakarontzas, Theo-dosios Theodosiou, Lefteris An-gelis & Madjid Fathi

2017 [4]

A8 Robust competence assessmentfor job assignment

Romain Guillaume, RaymondHoué & Bernard Grabot

2014 [21]

A9 The Assessment of Learning:From Competence to New Eval-uation

M. Calenda & R. Tammaro 2015 [8]

A10 The factors related to self-other agreement/disagreementin nursing competence as-sessment: Comparative andcorrelational study

Miyuki Takase, Masako Ya-mamoto & Yoko Sato

2018 [44]

A11 Preceptors’ experiences of usinga competence assessment toolto assess undergraduate nursingstudents

Eimar Burke, Marcella Kelly,Evelyn Byrne, Toni Ui Chiardha,Miriam Mc Nicholas & Adri-enne Montgomery

2016 [6]

A12 Assessing competencies: Anevaluation of ASTD’s CertifiedProfessional in Learning andPerformance (CPLP) designa-tion

Seolim Kwon, Robert R. Wad-holm & Laurie E. Carmody

2014 [26]

A13 Evaluating assessment withcompetency mapping

Robyn A. McNamara 2004 [32]

A14 Competencies and Their Assess-ment

James W. Drisko 2014 [14]

A15 Challenges to the assessment ofcompetence and competencies

James W. Lichtenberg, SanfordM. Portnoy, Muriel J. Bebeau,Irene W. Leigh, Paul D. Nelson,Nancy J. Rubin, I. Leon Smith &Nadine J. Kaslow

2007 [28]

Table B.1: Detailed overview of primary studies 51

How to Assess and Map Employees Competencies1366342/FULLTEXT01.pdf · competencies the company potentially lacks. This thesis investigates how to assess, eval-uate and map employees

Documents