Measuring professional skills misalignment based on early ...

http://www.diva-portal.org

This is the published version of a paper published in European Journal of EngineeringEducation.

Citation for the original published paper (version of record):

Flening, E., Asplund, F., Grimheden, M. (2021)Measuring professional skills misalignment based on early-career engineers’perceptions of engineering expertiseEuropean Journal of Engineering Educationhttps://doi.org/10.1080/03043797.2021.1967883

Access to the published version may require subscription.

N.B. When citing this work, cite the original published paper.

Permanent link to this version:http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-301260

Measuring professional skills misalignment based on early-careerengineers’ perceptions of engineering expertiseElias Flening, Fredrik Asplund and Martin Edin Grimheden

KTH Royal Institute of Technology, Department of Machine Design, Division of Mechatronics, Stockholm, Sweden

ABSTRACTProfessional skills have long been perceived as lacking in junior engineers.Adopting a social realist theoretical framework of knowledge in practice, ahypothesis-based survey study of early career engineers’ perceptions ofengineering expertise was conducted. It investigated a professional skillsreadiness difference between initial career trajectories (hypothesis 1)through an analysis of engineering expertise perception, and whether thisdifference decreases over time as engineers mature (hypothesis 2). Bothhypotheses were supported by three statistical tests which establishedthe specific nature and size of this difference. Three themes wereidentified: Academic bias, Technical competence bias, and Rationality bias.Thematic analysis through the framework of these three themes indicateshow context and complexity (Semantic dimension) and Knowledge andKnower (Specialisation dimension) were understood in practice. The threethemes expressed challenges over these two dimensions in understandingTechnical knowledge, Collaboration, and the Legitimate basis for practice,leading to recommendations for education and practice.

ARTICLE HISTORYReceived 30 September 2019Accepted 9 August 2021

KEYWORDSProfessional skills;engineering education;engineering practice;Legitimation code theory;engineering roles

1. Introduction

The insistence that engineering practice immediately requires professional skills has become, if notuniversal, then at least uncontroversial (Itani and Srour 2016, 1; Winberg et al. 2018, 167). Thedifficulty for new engineers transitioning to engineering practice and industry’s concern abouttheir often lacking in professional skills are well documented (Brunhaver et al. 2018; Jesiek, Trellinger,and Nittala 2017; Trevelyan 2014, chapter 3; Trevelyan 2019). However, despite decades-long effortsof educational research and accrediting bodies such as ABET1 and ENAEE,2 calls for teaching theseskills largely remain as loud as they are vague in what it means to require effective problem-solving,teamwork, communication, and coordination. A recent exception is the extensive review anddetailed competency elaboration of Honor J. Passow and Passow (2017, Table 1), who also pointout an important reason for this vagueness, finding technical skills inseparable from effective collab-oration (491) and ‘non-technical’ skills inseparable from its technical context in practice (501). Themitigation of these two inseparability-views is central to curricular innovations that enable steppinginto contexts, such as CDIO and PBL (Edström and Kolmos 2014, 539–540). In contrast, Wolff (2018)has addressed the consequence of these two views by urging educators to step out of real-worldcontexts and help students understand the need for the ability to consciously shift between signifi-cantly different ways of thinking and meaning-making (192) in engineering problem-solving. Lack ofinstruction that respects these two views (Winberg et al. 2018, 177) and fails to inculcate this shifting-ability in students creates an education to practice misalignment (EPM) which seems to stand behind

© 2021 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis GroupThis is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided theoriginal work is properly cited, and is not altered, transformed, or built upon in any way.

CONTACT Elias Flening [email protected]

EUROPEAN JOURNAL OF ENGINEERING EDUCATIONhttps://doi.org/10.1080/03043797.2021.1967883

general exhortations of the need for professional skills. All engineers thus seem to need professionalskills immediately after graduation, but little is known of the relative perceived importance of theseskills during the early career in different engineering practice contexts (Brunhaver et al. 2018). Iden-tifying the nature of such differences in early practice would be helpful for understanding and pro-viding nuance to this EPM. It could also help to sensitise educators to the need to present a morecontextualised view of the role of professional skills to students, who (we think rightly) oftenexpress doubt toward universalised and isolated views of ‘non-technical’ skills (Downey 2015;Pascail 2011). Therefore, this study focuses on measuring practicing early engineers’ perceptionsof engineering expertise as a whole rather than on measuring students’ perceptions of separateskills for practice (c.f. Byrne, Weston, and Cave 2018; Shuman, Besterfield-Sacre, and McGourty2005). It aims to investigate differences in how early engineers in different practice-roles but fromthe same educational background understand engineering expertise.

There is a seeming consensus on seeing Professional or ‘non-technical’ skills as a generic compe-tency category important for all practicing engineers from the start (hereafter called the universality-view) and that such skills cannot be understood independently from technical skills (hereafter calledthe integrative-view) (Anderson et al. 2010; Bucciarelli 1994; Downey 1998; Trevelyan 2007, 2014;Vinck 2003). Studies are needed in this context (Brunhaver et al. 2018) and furthermore there is adearth of studies measuring skills-perceptions in such early practice while taking this integrative-view into methodological account. Hence the purpose of this study. This study proposes to putthe division between the ‘technical’ and ‘non-technical’ is on the analytical level rather than onthe level of data collection by using engineering expertise as a skill-integrated measurement con-struct. The study uses a hypothesis-testing approach to investigate differences in the relative impor-tance attributed to professional skills by early-career engineers of differing career trajectories.

Section 2 starts with a review of the thematic difference between engineering education andwork. This difference is viewed from the integrative-view of professional skills in engineering work,which is used to question the universality-view through the presented social realist theoretical frame-work. Based on this framework, Section 2 concludes with posing two hypotheses: the first (H1) beingthat early engineers value professional skills differently depending on discipline, and the second (H2)being that such difference lessens over time as the engineer matures. Section 3 develops the meth-odology for the statistical testing of the hypotheses using a survey. The results are presented inSection 4. The theoretical framework is used to analyse the results in Section 5. In the discussionof Section 6 the analysis is connected back to the purpose presented above, together with the dis-cussion of limitations and implications for education and practice. Section 7 shortly summarises andends the paper.

2. An integrative view of skill in early engineering practice

This section reviews the engineering education literature on differences between engineering workand education, professional skills and these skills’ misalignment between education and practice. Asocial realist theoretical framework is adopted, and two hypotheses are formulated.

2.1. The nature of misalignment between engineering work and education

Anderson et al. (2010, 154–155) argued that the nature of engineering work and design is often rep-resented by themes that concern the messiness of the work, the primacy of interdisciplinary collabor-ation, and practical problem solving (See also Bucciarelli 1994; Michael 1998; Vinck 2003; Walter 1990).Trevelyan (2008b, 2014, 2019) emphasised that themessiness of the work and the need for collabora-tive performance frame the nature of problem-solving in engineering work. Considering these over-arching themes, what specifically is it that an effective engineer does? Passow and Passow (2017)synthesised a portrait of generic engineering practice (Figure 5, 492) from their findings of system-atically reviewing the literature on generic competencies for engineering practice: This portrait

2 E. FLENING ET AL.

focuses on engineers as creating, implementing, and maintaining solutions under multidimensionalconstraints, many of which are ‘non-technical’. Practice is inherently collaborative because techno-logical endeavours are too complex for any single person to grasp. Effective engineers makedecisions based on data that they independently gather in their projects. They manage and collab-orate in these projects (often) without formal authority, taking initiative on what to do and whichfunctional, horizontal, and vertical boundaries to cross to achieve desired performance and functionin the target system.

However, students generally find themselves in a thematically opposite position to this portraitand the overarching themes. Most student work is predefined in scope and outcome, orientedtowardmeasurable individual performance, and focused on theoretical problem solving (c.f. the discus-sion of technical rationality by Cosgrove and O’Reilly (2018, 40–42) and the student norms of Leo-nardi, Jackson, and Diwan (2002)). This partly explains why engineering education andengineering work are two fundamentally different realms of social practice, with distinct commu-nities of practice in universities and firms (Korte, Brunhaver, and Sheppard 2015). These communitiesperceive different skills and competencies as important (Passow 2012, 98).

These thematic differences seem to generate significant misalignment between education andpractice, driving numerous attempts at engineering education reform. Problem/project-based learn-ing (PBL, both complementary but different approaches) and conceive-design-implement-operate(CDIO) (Edström and Kolmos 2014) are two major efforts at such reform. The notion of ‘T-Shaped’engineers (Rogers and Freuler 2015) is another associated curricular reaction to EPM. The ‘T’ is ametaphor describing a depth of expertise in a single field and a breadth in several others, strength-ening the ability to collaborate across disciplines with experts in many areas (See Figure 2 in Rogersand Freuler 2015). More specifically, design and capstone project courses are common features ofEPM-minimising efforts in engineering education, attempting to simulate engineering practice invarious ways.

However, despite the widespread use and long-established benefit of such curricular innovations,simulating practice inside of an engineering education context can be seen as problematic forseveral reasons: Engineering problems cannot be simulated to afford an understanding of ‘realworld’ contexts (Wolff 2018, 192). As found by Leonardi et al. (2002) in their interview study ofhow students act on their understanding of ‘good engineer(s/ing)’, being stuck in the same edu-cation-thematic context can engender counter-productive design practices in engineering students.While several attempts exist within PBL and CDIO to address problems of educational contexts beingdivorced from practice, such as by integrating and assessing student’s artistry, self-reflection, anddesign process (Cosgrove and O’Reilly 2018), engineering education must still be structured toallow learning to be predicted using curricula and learning goals, and repeatably assessedthrough the uniform examination of learning performances, which are time-bounded. Poor (non-failing) performance in a course,3 however defined and measured, does not affect4 the nextcourse. However, this is not the case in professional practice since poor performance in past workaffects the conditions of future work.

The paradox embodied here has several underlying assumptions that deserve to be problema-tised: It is assumed that an engineering degree guarantees ‘basic technical skill’, which Grimsonand Murphy (Ch. 9 in Hyldgaard Christensen 2015) call the Foundation Layer of engineering knowl-edge; It is assumed that such a degree does not afford a guarantee of ‘professional skill’; It is assumedthat all engineering practice has context-dependent skill/competency requirements; It is assumedthat ‘professional skills’ are generic, immediately-needed, and transferable over these contexts(representing a universality-view); It is assumed that this must imply that higher education insti-tutions cannot fully mitigate EPM, but should still try. Behind these assumptions stands a coreview shared by several studies in engineering education: That engineering education in both itsresearch and practice serves as ‘R&D departments’ and ‘production facilities’ of engineering compe-tency delivered to private companies in the form of graduate engineers. Such a view is by no meansheld by all engineering education researchers, or most universities themselves.5 Indeed, many

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 3

writings in this journal stand (implicitly or explicitly) ideologically opposed to such positions held byother prominent engineering education journals. Pascail (2011) described and reflected on this viewas being a consequence of engineering education’s adherence to a ‘skills approach’ by industry. InBrunhaver et al. (2018)’s critique, professional skills are seen as inherently general skills that are trans-ferable over engineering practice contexts and do not directly address learning contextual rulesets ofone’s firm, department, or group: Professional skills (if understood as general and transferable) mightthus help in learning such rulesets, but do not constitute them. In contrast, each firm will demandcompetency profiles that benefit their organisation’s specific needs, often based on the primarytypes of technology and engineering disciplines they deal with. Cappelli (2008) argued that overtime market pressures have motivated firms to offload the needed internal training of new hiresunto engineering education by demanding curricular focus on professional skills (i.e. a ‘skillsapproach’). These needs could easily be shortsighted, only reflecting current needs of the organis-ation and job market. Recent empirical research findings also support such opposition, such asthe recommendations of Wolff (2018, 192–193) that educators step out of ‘real world’ contexts,thereby affording a more conceptual view by explicating and teaching what Wolff calls code-shiftingbetween ways of thinking in engineering practice problem-solving: This is a stream of educationalinnovation that takes a direction different from that of CDIO and PBL, but in a manner that comp-lements rather than oppose them. The point is not that maintaining a fundamental ‘employability’view on engineering education is good or bad, but rather that this lens is a major underlying view-point to which one must relate when problematising skill-concepts in education, engineering orotherwise. For example, what might be opposed to is not the idea that higher education institutionsshould produce ‘viable’ graduates, but rather the way to go about it, a move which the code-shiftingargument in the Wolff-article is a good example. Indeed, Wolff’s introduction motivates theirresearch through the current state of the engineering job market, which is a common and arguablyreasonable move.

Furthermore, standardised basic technical curricular elements (the foundational layer) do notnecessarily form a paradox against context-specific engineering practice. These elements, such asmath and basic science, deliver desirable fundamental technical competencies that are general toall engineering students (also described as the competency ‘Apply Knowledge’ by Passow andPassow (2017, 493)). The paradox stems from the fact that such standardisation cannot be main-tained in the same way for professional skills since such must be integrated with the relevant engin-eering context to be effectively taught, learned, and practised. Writing from an Anglo-Saxonperspective based on a system of chartering engineers, Grimson and Murphy (Ch. 9 in HyldgaardChristensen, 2015, 171–172) argued that learning professional skills (in the top layer of their engin-eering knowledge model) is done in two parts: competence formed during the engineering degreeprogramme and competence formed over minimum eight years of initial professional engineeringpractice. In short, a universality-view of what skills are (generic) and when they are needed(always) can be maintained for the foundation layer, but not the top layer. The results of thisstudy came to reflect this view, although in a more nuanced fashion; seeing that such professionalskills might be expressed differently in early practice over varying disciplinary and institutional con-texts. However, over time perspectives around the importance and role of professional skills willalign over contexts and disciplines for the early career. Indeed, the eight-year time span of thepresent study covers the second part of Grimson and Murphy’s top layer.

In short, standardised curricular elements do deliver fundamental technical skills necessary for allengineers, but cannot fully deliver competence in so-called ‘non-technical’ skills that are dependenton the engineering practice context in which they are deployed: the universality-view holds for theformer, but not the latter. This paradox of standardised elements against context-dependent prac-tice is a central reason why engineering education cannot take on the responsibility of eliminatingEPM by itself. This paradox is a central driver for EPM. Therefore, we choose to investigate the natureof these ‘non-technical’ skills or competencies for early engineering practice. The next section dis-cusses and problematises the concept of professional skills further.

4 E. FLENING ET AL.

2.2. Professional skills as an integrated part of practice

There is no consensus on a strict definition of professional skills, even though a common way is torefer to the third criteria of ABET (Passow 2007; Shuman et al. 2005) or other accreditation schemes.Colwell (2010, 3) pointed out that if one were to ask educators in… engineering…what is meant bythe term ‘soft skills’, there would likely be some consensus on the list, but each educator asked wouldprobably have a different list. Referencing previous studies of engineering programme alumni, prac-titioners, and educators (Passow 2007; Shuman et al. 2005), Gilbuena and colleagues (2015) arguethat such limited consensus will include communication (written and oral), teamwork, project man-agement, and self-awareness.

However, looking at synonyms for these skills affords an understanding of how the concept is dis-cursively bounded: professional (in the sense of general), soft, social, transferrable, or non-technicalskills. Thus, the concept of a skill set of these terms is necessarily understood in relation to their oppo-sites. The discourse around engineering skills has often distinguished between technical or pro-fessional skills, even though this has long been argued to constitute a false dichotomy (Bucciarelli1994; Downey 1998; Trevelyan 2007, 2014; Vinck 2003). Shuman et al. (2005) argued that this dichot-omised view can be seen in the ABET criteria, stating that while the ABET professional skills can betaught, such teaching must be cognizant of teaching engineering in its appropriate context (51). Thisprinciple was observed in student behaviour by Gilbuena and colleagues (2015), who while investi-gating professional skills in capstone design projects found an interplay between professional skillsactivities and technical activities. While conducting an interview study on the preparedness of stu-dents for practice, Martin and colleagues (2005) also found that a clear link existed between technicalcompetency, and communication and teamwork (177); they concluded that The implications for cur-riculum development are that the non-technical skills can not be taught in isolation from the technicalcontext in which they will be used (179). Despite this, as Jesiek et al. (2017, 1) argued, engineeringeducation still keeps technical and non-technical learning separate, which they see as artificialand arguably counterproductive. Passow and Passow (2017) provide strong indirect empiricalsupport to this inseparability of the technical and social in their thorough review of engineering edu-cation research on needed competencies, stating that the most striking finding in this systematicreview is that technical competence is inseparably intertwined with effective collaboration (seefinding 2, 491).

This is where the study arrives at accepting the integrative-view of skill, where Professional Skillsdo not refer to a single well-defined skill set, but rather to a wide range of value-adding social per-formances (Trevelyan and Williams 2018). Establishing an operating definition for this study, theseperformances constitute an ability to enact technical knowledge, through social networks of pro-fessional actors, by continually recognising the present legitimate basis for practice. These threeparts build on each other: Seeing the legitimate basis enables collaboration in networks, which inturn enables the enactment of technical knowledge (see Figure 1).

This view, together with needed technical knowledge, constitutes Engineering Expertise. Thisrelates closely to Passow and Passow’s (2017) portrait of engineering practice from the previoussection. Discussing whether professional skills are important or not might thus miss the point ofengineering expertise: Different aspects or types of professional skills might vary in importancedepending on the work context of engineers (Winberg et al. 2018, 168), who for instance belong

Figure 1. Operating definition of professional skill.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 5

to different engineering disciplines and career-phases. The integrative-view and operating definitionmotivate a questioning of the assumption of the universality-view of professional skills. Especially forthe early career (Brunhaver et al. 2018).

Indeed, the phenomena of professional/soft/social/non-technical skills seem to be both inte-grated and integrating, in that it is an integrated skill set with a context-dependent compositionthat in turn integrates technical knowledge with other practice dimensions. The last part of the oper-ating definition (legitimate basis) is only indirectly present in the engineering practice portrait but iscentral to a social realist view of knowledge of-and-in practice. Therefore, theoretical support isneeded to frame how professional skills might be clearly understood as a part of engineering exper-tise, which this view can afford.

2.3. Theoretical frame and hypothesis

The operating definition necessitates the recognition of what legitimate practice is based on in eachsituation for the engineer. The idea of different bases for legitimate engineering practice is close toWolff’s (2018)writing on ‘code-shifting’between knowledge andpractice contexts.Wolff (2018) inves-tigated the engineering theory-practice divide (embodied in the EPM paradox) in industrial problem-solving of mechatronics engineers by employing a social realist framework called Legitimation CodeTheory (LCT) (Maton 2014). This is a multidimensional conceptual toolkit for researching knowledge-building. While problem-solving is not in itself the focus of the present work, LCT is particularly inter-esting for the integrative-view, more specifically Maton’s Specialisation Dimension and its visualisationin the specialisation plane (Maton 2014, 30): this dimension explores the basis of achievement underlyingpractices, dispositions and contexts, seeing knowledge-knower structures as constitutive of practices.The organising principles of these structures can be seen in relations along the two dimensions: epis-temic relations (ER) and social relations (SR), which are combined to constitute a number of specialis-ation codes: knowledge codes focus on what is known, knower codes focus on who knows, élite codesfocus on both, and relativist codes focus on neither (See Figure 2).

Specifically related to our aim of framing integrative understandings of professional skills,Winberg et al. (2018, Figure 1) used this plane to map out engineering knowledge as knowledge

Figure 2. The specialisation plane (Maton 2014, 30).

6 E. FLENING ET AL.

codes and Professional Skills as knower codes when reviewing the EER literature on curricularelements promoting employability. Their findings (Winberg et al. 2018, Figure 2) of ‘Hard skills insoft contexts’ parallel the integrative-view of this study: that engineering knowledge (knowledgecodes) and professional skills (knower codes) must be integrated when taught because these are inte-grated in practice (Èlite codes).

The Semantic dimension of LCT can serve to theoretically frame the universality-view. This dimen-sion represents how context-dependent and complex legitimate meanings are in any given practice(Maton 2020). Specifically, the organising principles of the semantic structures are semantic gravity(SG) and semantic density (SD), denoting degrees of context-dependency and complexity, respect-ively (Maton 2016, 15). Relative strengths along SG and SD combine to constitute a number ofsemantic codes (see 2016, 16): rhizomatic codes (SG−, SD+) see meanings as complex but indepen-dent of context; prosaic codes (SG+, SD−) are context-dependent but have a simple view of mean-ings; rarefied codes (SG−, SD−) reduce meanings to context-independent and simple forms; andworldly codes (SG+, SD+), where meanings are context-dependent and complex (See Figure 3).

Maton (2016, 17; 2020, 66) directly addressed the theory/practice-divide using semantic codes todescribe and question the false dichotomy of the universality-view, stating that the theory/practice-divide … represent[s] rhizomatic codes (SG−, SD+) [i.e. Technical Skills] and prosaic codes (SG+, SD−)[i.e. Professional skills], respectively, and exclude the possibility of rarefied codes (SG−, SD−) and worldlycodes (SG+, SD+) (2016, 17). Here we can see engineering expertise as embedded with worldly codesthat see expertise as a semantically … context-dependent stance that condense[s] manifold meanings(2016, 16).

Seeing how specialization codes and semantic codes can be used together to analyse the sameempirical data and offer complementary insights into the same phenonema (sic) (Maton 2016, 18),we apply the two LCT dimensions of Specialisation and Semantics to the understanding developedabove on the nature of Professional Skills. From this application, two hypotheses can now be formu-lated with the purpose to nuance the assumptions of the universality-view while accepting theassumptions of the integrative-view, necessitating understanding professional skills in relation toengineering expertise. As Wolff (2018, 193) found, we expect that early-career engineers willemploy different specialisation codes to understand the legitimate basis of their practice, andwhat they perceive as constitutive of expertise. However, because students are taught a

Figure 3. The semantic plane (Maton 2014, 131).

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 7

universality-view (SG–) of professional skills, despite their contextual-dependent (SG+) nature,various degrees of code-clash or code-match (Maton 2016, 13) on both the specialisation planeand the semantic plane should be expected. Such clashes are expected to arise between howearly engineers understand their practice and the legitimate basis for that practice, depending oninitial career direction. This can be embodied in perceived disciplinary role of the early engineer.We expect that the understanding and value placed on professional skills will differ over perceiveddisciplinary roles.

Hypothesis 1 – Early engineers acting in different engineering disciplines can ascribe a significantly differentvalue to professional skills.

Given that we know from previous research that professional skills are important for engineeringcareers as a whole, we would expect that the code-clash between legitimation codes used should drift(See Maton 2016, 13) toward increasing code-match over time, evening out discursive differences onthe issue of the perceived importance of the ‘non-technical’.

Hypothesis 2 – This difference in ascribed value will decrease over time in the initial career.

3. Methodology

This section starts with a description of the research process. It goes on to explains the choice ofstudy population and methods for data collection and analysis, ending with a discussion of validity.

3.1. Study design process

This study employs the statistical testing and analysis of a two-part survey on early-career engineersfrom one graduate-degree programme. See Figure 4 for a visual representation of how the theoryand methodology of the study relate to each other. The italicised arrow text boxes show how thestudy’s core concepts (square boxes) are connected. The rounded boxes show where in the textthese are explained. The Theory-part of Figure 4 concertises concepts from Early-career misalign-ment down to Engineering Expertise, which is operationalised in the Methodology-part throughthe questionnaire. The Methodology-part details how the instrument, empirical context, collection,and analysis method relate to each other.

The study has two core design requirements that deserve explicit description, the first concernsthe empirical context and sample, the second concerns how skills relate to expertise.

Design requirement #1: H1 and H2 motivated us to sample early-career engineers with identical and multidisci-plinary educational backgrounds. Identical educational backgrounds are important because any differencesfound might otherwise be attributed to differences in formal educational background, and Multidisciplinarybackgrounds are important because we want a sample of engineers acting in a broad range of professionalengineering contexts. The empirical context and sample are further discussed in Section 3.2 below.

Design requirement #2: the literature reviewmotivated us to think that, in both engineering education and prac-tice, technical and non-technical (i.e. professional) skills are empirically inseparable and holistically blended intothe construct of engineering expertise. Therefore, to enable an analysis of professional skills in an engineeringpractice context, we needed an instrument that measures expertise. Otherwise, any data received would bebased on a manufactured divide of these two inseparable concepts. This would have seriously weakened thequality of our study, further discussed in Section 3.5.

Since it is better to use an existing instrument if it measures the thing in question, rather thandevelop our own instrumentation, we first searched for existing questionnaires and inventoriesthat aimed to capture engineering expertise perception. Most available instruments were relatedto assessment using lists of competency-criteria such as ABET (c.f. Passow 2012; Passow andPassow 2017; Shuman et al. 2005), but in reading the work of James Trevelyan (‘The making of anexpert engineer’, 2014, 17–18) we found a 12-item questionnaire probing how engineering expertise

8 E. FLENING ET AL.

is understood. This questionnaire came with a grading scheme that awarded up to 10 points to eachquestion, for a total score of 120 points to assess how much the respondent’s expertise-perceptionagreed with the book’s research on the topic. The questionnaire and grading scheme, which can befound in the appendix (Table A4), is further discussed in Section 3.3.

Trevelyan’s writings on engineering work (Trevelyan 2008b, 2014, 2019) and skill (Trevelyan2008a, 2014; Trevelyan andWilliams 2018) form a significant part of our position on how engineeringexpertise depends on the intertwining of technical and professional skills. Furthermore, his question-naire and its grading scheme fit with the second design requirement. Therefore, we chose toimplement it (Study Survey Part II in Figure 4) in a survey administered to a sample in our empiricalcontext: junior mechatronics engineers. Study Survey Part I in Figure 4 probes career path and self-perceived roles of the respondent. In Part I, we address the first design requirement of situatingthe measurement in the early-career of engineers with similar educational backgrounds. Before

Figure 4. Study design overview.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 9

developing the questions for Part I, we screened our entire sample of early-career mechatronicsengineers manually through public information available on LinkedIn to assess what career pathquestions and role-alternatives seemed relevant to offer as answer options.

Subsequently, we assembled the items for the two parts into an electronic survey, pre-tested it forface validity, and sent it out to our sample (see Section 3.3). We divided the received data into twoindependent groups (see Section 3.4) and tested H1 by conducting an ANCOVA on the Part II-score,controlled for career length, between the groups. An independent U-test was conducted to under-stand what questions in Part II were driving the difference from the ANCOVA. We tested H2 using aPearson’s correlation test between the groups to see if the differences identified for H1 decreasedover time. This concludes the process description of the methodology.

3.2. Empirical context

To test the hypotheses, the study investigates a purposive sample of early-career engineers, consist-ing of alumni from a two-year Master’s programme in Mechatronics at [university name & location,redacted for the anonymous version of the manuscript]. Mechatronics engineering is a half-centuryold (Bradley 2010, 828, Ch. 2) multidisciplinary engineering field of industrial practice consistingof mechanical, electronics, control, and software engineering. A central concept in mechatronicsis the integration of its constituent engineering disciplines (Auslander 1996; Harashima, Tomizuka,and Fukuda 1996; Janschek 2012). However, few of our alumni practice as pure mechatronics engin-eers. Most transition into work as engineers within one of the constituent disciplines in a broad rangeof industrial contexts. Although students recruited to the Master’s programme have backgrounds invarious Bachelor’s programmes, Mechanical Engineering is the most common. Since the BSc andMSc programmes were integrated into five-year programmes before the Bologna Process stream-lined programme structures amongst its signatories (most of Europe), the Bachelor’s programmes(still) mainly prepare for the Master’s programme teaching basic math, science and the relevant pre-requisite applied technical subjects. This purposive sample is thus ideal for testing the two hypoth-eses, as these alumni have the same formal educational background but ended up practicing indifferent engineering communities.

The mechatronics, mechanics, electronics, and control engineering roles filled by our alumni haveremained relatively stable over time. However, the associated software-oriented roles have drasti-cally changed and proliferated during the last decade, in general due to digitalisation. In our empiri-cal context of mechatronics engineering design, this is due to the introduction of increasinglycomplex and software-intensive cyber-physical systems (Törngren et al. 2017). Indeed, in researchinghow mechatronics engineers engage in problem-solving, Wolff (2018, 192) found that software-oriented (marked as ‘logic’, or ‘computer control’ (Figure 1 in Wolff 2018, 184)) participants preferredpractice contexts where they could decide how to go about solving a static what. This indicated chal-lenges in code-shifting, especially in environments where they were under constraints as to how togo about solving a problem. This reflects the view of our mechatronics alumni and poses an inter-esting avenue for grouping: self-perceived software-oriented roles and those who identify otherwise.

3.3. Data collection method

All 346 graduates from the mechatronics programme between 2011 and 2018 were sampled. Whilethere are other somewhat similar university programmes in Sweden, the first design requirement ofthe research design necessitated a sample with the same formal educational background. As such,these 346 alumni constitute the total relevant population. Therefore, while statistical generalisationsto all early-career engineers might not be easy to make based on this purposive sampling approach,theoretical/logical generalisations can be made (Patton 2002, 237).

The two-part electronic survey was pre-tested to ensure face validity. It was sent out via e-mail,which detailed the structure and purpose of the study. After the stated deadline expired, reminders

10 E. FLENING ET AL.

were sent out. An acceptable response rate of 46% (n = 158, out of 346) was achieved. See Table A1:response rates in the appendix for a breakdown of the sample and response rates.

Part I of the survey measured career path, perceived work role, and competency profile througheight questions.6 These questions were selected based on a manual pre-screening of the sample toensure relevance and content validity. Common roles in the pre-screening, together with the coredisciplines of mechatronics engineering, constituted the available roles to choose from in Part I.

Part II uses the 12-item7 questionnaire developed by James Trevelyan (2014, 17–18). Theengineering expertise questionnaire in Part II assesses to what degree that the respondent’sunderstanding of engineering expertise aligns with the skill-integrated view of expertise thestudy uses. Trevelyan’s questionnaire asks the respondent to what degree, on a five-pointLikert scale, they agree with twelve statements about engineering practice. Each statement rep-resents one or several frequent misconceptions that paint engineering practice as more objective,solitary, and hands-on than is typically the case. The misconceptions in Trevelyan’s book onengineering expertise are supported8 by research such as the recent thorough review of compe-tencies for engineering work by Passow and Passow (2017). This is especially true in their findingsthat technical competence is inseparably integrated with social interactions (Finding 2), whereengineers spend the majority of their time communicating (Finding 3), and that what is actuallycoordinated in these communications are the necessary multiple competencies (carried bydifferent people) to enable engineering practice (i.e. problem-solving) to function (Finding 4).These findings by Passow & Passow match well with Trevelyan’s misconceptions of engineeringexpertise. All three see problem-solving as a fundamentally multidisciplinary exercise in collab-oration that coordinates heterogenous skill sets at different phases of product development. It isaround this view that the twelve statements in the Engineering Expertise questionnaire revolve. Foreach answer to the twelve statements, the grading scheme9 developed by Trevelyan assigns apoint value that reflects the correctness of the position relating to the above view. Each answer canscore up to 10 points, up to a max total of 120. The distribution of points over the five-point Likert-scale for each statement represents the approximately correct position to take on that statementaccording to the research that the book is based on. Several positions on the Likert-scale can affordfull points for some statements, while for others the research takes a more polarised view (such ason statement 9: You can only learn communication skills by practice; they cannot be taught, which isa seen as a mostly false statement).

3.4. Data analysis method

The data was split into two independent groups. Group 1 (n = 62) consisted of those respondentswho see their work-content as best described through the role of software developers, operationa-lised as the respondent ranking Software Developer (SW) as the first (n = 40) or second (n = 22)most important role ‘ … that best capture[s] what you actually do currently at work’ (PI-6 in TableA3). The reason for including those who ranked SW as the second most important role (see Table1 below) was primarily that none of these respondents ranked another core technical discipline(Mechanical/Electronic/Control) first. Conversely, the technical roles that were chosen above SW inimportance describe a category that is related to SW: Embedded Systems Developer (n = 10), Mecha-tronics Engineer (n = 4), Systems Architect (n = 4), Test Engineer (n = 2). None of these represent adifferent dominant primary technical field from the perspective of the empirical context of mecha-tronics engineering.

Group 2 (n = 97) consisted of those who rather see their work content best described with eitherthe other core technical fields or more directly described as managerial. This group was operationa-lised as the remainder of the respondents who were not part of Group 1 (See Table 2).

To test Hypothesis 1 an ANCOVA was conducted between the two groups on the EngineeringExpertise perception score from Part II, controlling for career length in line with the argument forHypothesis 2. To understand the nature of the difference in detail, a Mann–Whitney U-test was

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 11

performed to identify which individual questions in Part II drove the overall difference found by theANCOVA test.

A correlation test between test score and career length was conducted to test Hypothesis 2: If asignificant correlation could be observed for Group 1 but not Group 2, it would constitute supportingevidence for Hypothesis 2. This test is also needed as it shows the direction of any influence by careerlength. As discussed at the end of Section 2.3, such a correlation could then be analysed through thetheoretical frame and understood as going from code clash to code match over the early career (seeSection 5.4).

3.5. Validity concerns

The operating definition of professional skills in Section 2.2 argued for an integrated view of thetechnical and non-technical in engineering practice, leading to the use of the engineering expertiseconstruct, on which the survey instrument measured. Using this construct instead of a dichoto-mised and itemised view of skills strengthens the content validity of the study. However, thereremains the question of how to analytically distinguish between these two non-separable skill-dimensions when comparing the two groups on the engineering expertise construct. We arguethat this construct can be used to validate the two hypotheses, specifically because we focus onearly-career engineers. Due to the focus of engineering education on technical skills, the respon-dents are likely to be more mature regarding technical skill when transitioning into early-career

Table 1. Breakdown of roles in group 1.

Group 1: The software group

Role name Role not applicable Somewhat important Important Very important

Product manager 55 3 3 1Project manager/Team leader 50 8 3 1System architect 45 7 6 4Systems engineer 49 9 4 0Embedded systems developer 39 5 8 10Mechanical engineer 61 0 1 0Electronics engineer 60 2 0 0Mechatronics engineer 50 5 3 4Control engineer 58 3 1 0Test engineer 49 6 5 2Requirements engineer 57 5 0 0Industrial designer 62 0 0 0Division/department manager 61 1 0 0

Table 2. Breakdown of roles in group 2.

Group 2: The non-software group

Role name Role not applicable Somewhat important Important Very important

Product manager 78 5 5 9Project manager/Team leader 66 6 15 10System architect 69 10 13 5Systems engineer 69 4 9 15Embedded systems developer 74 5 5 13Mechanical engineer 88 3 4 2Electronics engineer 84 2 4 7Mechatronics engineer 74 10 8 5Control engineer 94 0 2 1Test engineer 70 13 6 8Requirements engineer 83 6 7 1Industrial designer 93 0 2 2Division/department manager 81 3 3 10

12 E. FLENING ET AL.

engineers. Thus, the likelihood that the two groups would differ widely on technical skill by chanceis low. Moreover, the fact that a candidate for a junior job position has graduated from an estab-lished engineering programme is an important factor for recruiting companies (Lang et al. 1999). Inshort, an engineering degree is accepted as a basic guarantee of technical skill, but not pro-fessional skill, for recent graduates. Indeed, this has been found to be the case in numeroussurveys on the readiness of engineering graduates that systematically identified communicationand teamwork as ‘competency gaps’ (c.f. Itani and Srour 2016; Meier, Williams, and Humphreys2000; Passow 2012; Passow and Passow 2017), the implication of which Sageev and Romanowski(2001, 690) boiled down to: Technical abilities are a given, communication and leadershipdifferentiate.

All respondents were chosen from the samemultidisciplinary engineering programme, mechatro-nics engineering, and as such share both basic technical (e.g. math/science) and advanced applied(e.g. electronics, embedded systems development) standardised curricular elements. Therefore, theobserved differences should not stem from differences in their engineering education background,which strengthens internal validity. This is intentional, since what we wanted to investigate iswhether differences in perceptions on professional skills can be found in the early career betweendifferent initial career paths. It is the existence, direction, and size of these differences themselveswhich are the focus, rather than the nature of the disciplines that the difference is foundbetween. The nature of practice disciplines is influenced by so much more than its members’ edu-cational backgrounds, which is what we control for in this study.

Another validity concern relates to the fact that the engineering expertise construct is based uponthe assumption that real engineering work, contrary to common misconceptions (Trevelyan 2014,48–54), contains little solitary design work and that the majority of worktime is spent in social inter-action, especially forms of technical collaboration (Passow and Passow 2017, 491; Trevelyan 2007,2014). Seeing how this is the nature of most engineering practice and a core part of the integratedview of engineering expertise taken in this study, there is a need to validate that the assumption ofsocial interaction constituting the majority of meaningful work and work time holds for our empiricalcontext. Otherwise, any difference found in the subsequent tests could be attributed to a potentialdifference in this assumption between groups, weakening internal validity. Therefore, we included aquestion that asked the respondent to estimate the amount of time spent in any kind of social inter-action to solve a task or problem. The result in Figure 5 below supports this assumption, in line with,for example, Finding 3 of Passow and Passow (2017), and therefore also taking the integration-view-point and using the engineering expertise construct. The distribution in Figure 5 is largely the sameacross the groups.

4. Results

The three tests are conducted to establish the existence of an overall difference in test score (theANCOVA test), the nature of that difference (the U-test), and finally the direction of the difference(the correlation test). Table A2 in the appendix gives the mean score and standard deviation foreach question in Part II for each group, and the figure below for the whole sample (Figure 6).

Figure 5. Time spent in interaction.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 13

4.1. Testing hypothesis 1

4.1.1. All questionsThe ANCOVA compared the two groups on the Engineering Expertise perception score while control-ling for career length. There was a linear relationship between test score and career length, asassessed by visual inspection of a scatterplot (see appendix “Scatterplot”). Standardised residualswere normally distributed, as assessed by Shapiro-Wilk’s test (p > .05). There was homoscedasticityand homogeneity of variances, as assessed by visual inspection of a scatterplot and Levene’s testof homogeneity of variance (p = .223). Since no cases in the data had standardised residualsoutside of ±3 standard deviations, no outliers were found. After adjustment for career length,there was a statistically significant difference in test scores between the two groups, F(156, 1) =5.4, p < .05, partial η2 = 0.033. Test scores were significantly smaller in Group 1 compared toGroup 2, with a mean difference of 6.0 points (95% CI, 0.89–11), p < .05 (Table 3).

Since the purpose of this first test is to test the existence of a difference in total score, and the sizeof the difference is not the most important aspect here, the significant difference identified by theANCOVA test is considered to support the first hypothesis. The next test drills down into the natureand extent of this difference between the groups.

4.1.2. Individual questionsTo determine which questions in part II contributed most to the observed test score difference, aMann–Whitney U test was conducted on each question. Distributions of the scores for the groups

Figure 6. Whole sample test score.

Table 3. Comparisons between the two groups.

Mean difference Std. Error Sig.

95% Confidence Interval for difference

Lower bound Upper bound

6.0* 2.6 0.021 0.89 11

*The mean difference is significant at the .05 level.

14 E. FLENING ET AL.

were similarly shaped, as assessed by visual inspection. Question P2-1, P2-5, and P2-6 proved to bestatistically significantly (p < .05) lower for the first group.

The size of the mean differences in Table 4 should be understood in relation to the mean size ofthe scores of the two groups (see Table A2 in appendix): For question P2-1, Group 1 scored 55%lower than Group 2; For question P2-5, Group 1 scored 17% lower than Group 2; For question P2-6, Group 1 scored 55% lower than Group 2. Note the large differences for P2-1 and 6, which arenot scored linearly and as such could be in danger of under-representing ‘close-to-right-answer’.The distributions of the answers (not scores of answers) over the two groups were visually inspectedto ensure that this was not the case.

It is at this level, where it is known which questions drive the general difference, that the relativesize of the results is relevant for the first hypothesis. Supported by the existence of a general differ-ence in total score, and the relative size of this difference on the level of significantly distinct ques-tions between groups, the first hypothesis is accepted.

4.2. Testing hypothesis 2

A Pearson’s correlation was conducted to test Hypothesis 2. The results are detailed in Table 5. Astatistically significant (p < .05) correlation of 0.3 was found between test score and career lengthfor Group 1, but not for Group 2.

Interpretations of effect size must be related to the relevant context of both the variables in ques-tion as well as the research. There is a paucity of literature on effect sizes in engineering education,but a common benchmark often used in many research fields (outside its home of behaviouralscience) is that of Cohen (1988) which presents a framework recommending Pearson’s r = .10, .30,and .50 to indicate small, medium, and large effects, respectively. However, recent critiques(Gignac and Szodorai 2016, 74) point out that Cohen’s labels were not developed from a systematic,quantitative analysis of data, such as in Hemphill (2003). Some even go so far as to argue that thisdecontextualised framing of effect size is ‘nonsensical’ (Funder and Ozer 2020, 157–158). Funder andOzer (2020) suggest, based on Gignac and Szodorai (2016)’s findings of 708 meta-reviewed corre-lations in social and personality psychology (of which this study is an application), considering aneffect size r = .30 to indicate an effect that is large and potentially powerful in both the short andthe long run in this context. Therefore, based on this guidance, we choose to interpret our effectsize in personal difference on the engineering expertise construct of r = .30 as medium to large.The reason for being conservative here is the distance in research contexts and the preliminarynature of our results: This is the first quantitative study of a holistic measure of this kind in engineer-ing education research.

Seeing the statistically significant and meaningful size of our results, hypothesis 2 is accepted.

5. Clashing views of the legitimate basis for expertise in practice

5.1. Analysis process

The previous section established the existence (H1: ANCOVA), nature (H1: U-test), direction (H1:ANCOVA & U-test), change (H2: Correlation), and size (H1: U-test, H2: Correlation) of how engineering

Table 4. Significant U-test result for three questions.

Question U Z Sig.Meandiff.

P2-1: An engineer who achieved higher grades at university tends to perform better inengineering work

2507 2507 0.048 1.6

P2-5: Being a successful engineer depends primarily on your technical expertise 2271 2271 0.007 1.1P2-6: Facts are more objective and unbiased when stated in terms of numbers rather thanwords

2478 2478 0.018 1.7

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 15

expertise is perceived between the two groups. Thereby both hypotheses were supported. We nowturn to the adopted theoretical frame of LCT, in terms of a number of legitimation codes, to interpretthe nature and change of the identified difference. Having coded the nature of the difference, that isthe three questions identified in Section 4.1.2, these codes will then be related back up to the overallconstruct of engineering expertise by relating them to the three parts of operating definition (Section2.2): enacting technical knowledge, collaborating in social networks, and seeing the legitimate basisof practice (see Figure 1). In doing so, we have closed a circle starting from the general, down to par-ticulars, and then relating those particulars back up again. The change of the difference will then becoded as a shift from code clash to code match to explain the hypothesised decrease of EPM over theearly career.

5.2. Three themes

The three significant statements found driving the overall difference in total test score representthree distinct themes10: The first theme is academic performance bias (P2-1), which states that uni-versity grades have little to no effect on career performance. The second theme is technical expertisebias (P2-5), which states that while technical expertise is what distinguishes an engineer, success inengineering primarily depends on effective collaboration. The third theme is rationality bias (P2-6),which states that numbers are meaningless without words to describe what they signify, a descriptionin words lies behind every number and assumptions about a physical process lie behind every measure-ment. The remaining Part 2 questions, which together constitute the overall difference, should not beunderstood as unimportant or identical in their perception over the two groups. Rather, that norelation could be found on these particular data with these particular methods. However, since itwas established that together they matter (H1: ANCOVA), it is prudent to point out that no differencewas found on the most prominent theme: collaboration. However, the second theme of technicalexpertise bias indirectly implicates a position toward collaboration, and as such is indirectly butweakly represented.

The three themes point to positions on both the specialisation plane and semantic plane. Theformer position denotes a code clash on how epistemic relations and social relations of engineeringexpertise are understood, and the latter represents a code clash on how the complexity and context-dependency of the technical and professional skills (which are constitutive of expertise) areunderstood.

Table 5. Correlations between career length and test score.

Groups Career length Test score

Whole sample Career length Pearson Correlation 1 .16*Sig. (2-tailed) .047N 159 158

Test Score Pearson Correlation .16* 1Sig. (2-tailed) .047N 158 158

Group 2 Career length Pearson Correlation 1 .042Sig. (2-tailed) .69N 96 96

Test Score Pearson Correlation .042 1Sig. (2-tailed) .69N 96 96

Group 1 Career length Pearson Correlation 1 .30*Sig. (2-tailed) .021N 62 62

Test Score Pearson Correlation .30* 1Sig. (2-tailed) .021N 62 62

*Correlation is significant at the .05 level (2-tailed).

16 E. FLENING ET AL.

In the first theme of Academic bias, Group 1’s semantic view of performance in practice has rhizo-matic codes. They (correctly) see the importance of having learned technical skills in university, dueto the technical complex nature of engineering problems (SD+). However, they fail to account for thecontext-dependent (SG+) nature of that importance for practice, that is, the requirements for enact-ing technical knowledge (Figure 1), which imply worldly codes. Therefore, Group 1 is at risk of code-clash, mistakenly seeing the legitimate basis for practice through knowledge codes (‘if you know theright thing you will succeed’) even when other codes form that basis. This could also pose a chal-lenge for required code-shifting behaviours in such cases.

For the second theme of technical expertise bias, Group 1 strongly positions the legitimate basisfor practice on knowledge codes (ER+/SR-), seeing technical expertise as constitutive of stronger epis-temic relations (ER+). However, since technical expertise is what distinguishes an engineer, indicatingknower codes that focus on the importance of social relations and identity for professional legitimacy(SR+). In short, to be an engineer at all technical expertise is needed. However, to be a successfulengineer (i.e. engineering expertise), collaboration is needed, indicating elité codes requiring bothidentity (SR+) and knowledge (ER+).

The last theme, rationality bias, is indicated weakly in the Specialisation dimension, but strongly inthe Semantic dimension: Group 1 clearly displays rarefied codes in seeing numbers as more objectiveand unbiased than words. Maintaining rarefied codes, implying a simplified (SD−) and context-inde-pendent (SG−) semantic stance toward facts, should not be interpreted as incompetence in therelated practice, but rather as a misunderstanding of the semantic structure of that practice. In engin-eering work, the semantic structure of facts about systems and processes is commonly context-dependent (SG+) and complex (SD+), indicating worldly codes. In other words, the engineer mightimplicitly understand both the complexities and dependencies inherent in a numerical measurementor data-point, without realising that such understanding cannot be taken for granted in others. Thisis usually not a problem in contexts where the legitimate basis for practice also rests on rarefiedcodes; for example, interacting with engineering colleagues that have the same implicit intimateknowledge about the current relevant fact(s). However, in contexts that require code-shifting behav-iour (Wolff 2018) to other codes, the engineer would find it challenging to recognise the legitimatebasis for practice, making collaborating to enact technical knowledgemore difficult. This parallels, onthe semantic plane, similar findings of Wolff (2018, 192) on the epistemic plane.

5.3. Code clashes in the operating definition

In terms of the operating definition (Figure 1), these three themes and their codes point to specifickinds of EPM problems for early practice in Group 1 compared to Group 2. The first theme indicatesmisalignment of the ability to dynamically recognise the ever-changing legitimate basis for practice.The second theme shows a gap for EPM on the role of collaboration. The third theme connects toproblematic views on both the nature of collaboration and what constitutes legitimate practice.Unsurprisingly, no code clash was directly indicated toward understandings of the enactment oftechnical knowledge (the last box in Figure 1) in any of the three themes: This matches both theassumptions of initial basic technical competency (the universality-view) and the rhizomatic codesidentified for the first theme as well as argued for by Maton (2016, 17). This matches the view devel-oped in Section 2 on the inseparability and contextuality of skill in engineering practice. The sourcesof possible EPM seem to be in the prerequisites of collaboration and legitimate basis for practice, andespecially evident in the possibility for code-shifting challenges. Table 6 summarises the code clashesthat represent the specific nature of this EPM.

It seems that Group 1 does not explicitly clash in understanding the importance of EPM on col-laboration. However, the themes of technical bias and rationality bias indirectly indicate EPM on col-laboration: The former express code clash in social relations in engineering practice and thelegitimising function of an engineering identity based on having technical knowledge; the latter

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 17

expresses a semantic code clash on the meanings of facts-in-practice, indicating possible rigidityregarding the code-shifting behaviour required for effective collaboration.

When it comes to misalignment in the ability to consciously perceive the legitimate basis for prac-tice, the themes of academic bias and rationality bias seem to play into the difference between thetwo groups. The first theme indicates connected code clashes in both the Semantic and the Special-isation dimensions. In semantically holding rhizomatic codes in their understanding of the role of aca-demic achievement, Group 1 seems to be at (relative) risk of a code clash in practice if maintainingknowledge codes in contexts where other codes form the legitimate basis for practice. Code-shiftingchallenges would probably ensue, which is also indicated by the code clash of the third theme. Hererarefied codes are deployed by Group 1 when understanding facts as representable in a simplified(SD-) and context-independent (SG-) manner in engineering practice, which goes against the under-standing developed in this study of engineering expertise and how Group 2 answered.

Thus, Code-shifting challenges seem to be core to the identified code clashes in the operatingdefinition.

5.4. Clash to match over the early career

One way to understand the code-shifting challenges is to relate them to the correlation testresults that supported Hypothesis 2. In the medium-strong effect size of the result, a shiftover time can be seen where both semantic perspectives on engineering skills and understand-ings of epistemic relations and social relations in engineering practice tend toward an integra-tive-view, resulting in increasing code match between the groups. This indicates a lesseningof EPM on professional skills, as the concept is defined here, for Group 1. In the Semanticdimension, Group 1 seems to shift from rarefied codes (non-understanding or non-consciousunderstanding) or Rhizomatic codes (understanding complexity aspects, but not the context-dependency of that complexity) to Worldly codes. In the Specialisation dimension, a shift fromknowledge codes (where knowing the right thing is seen as enough) toward Elité codes (wherehow you know it, being the right kind of knower, is equally important for expertise) is indicatedfor Group 1 relative to Group 2.

6. Discussion

The study starts from an integrative-view while questioning a universalised-view of engineering skillfor the early career, leading to an operating definition of professional skills which integrates technicalknowledge into engineering expertise. The purpose is to see if differing views of expertise could befound between initial career trajectories, indicating EPM on professional skills. Supported by the fra-mework of two dimensions of LCT that theorise knowledge-knowers relations and context-complex-ity aspects of meaning, a survey methodology tests and finds support for two hypotheses of an initial(H1) but decreasing (H2) difference.

Table 6. Semantic and specialisation code clashes of themes in the operating definition.

Technicalknowledge Collaboration Legitimate basis for practice

Academicbias

None found None found Knowledge codes VS all four possible(specialisation code clash)

Rhizomatic codes VS worldly codes (semanticcode clash)

Technical bias None found knowledge codes VS elité codes(specialisation code clash)

None found

Rationalitybias

None found rarefied codes VS worldly codes (semanticcode clash)

Rarefied codes VS worldly codes (semanticcode clash)

18 E. FLENING ET AL.

6.1. Elaboration on key findings

As expected, the primary difference/code-clash is not in the direct perception of technical knowledge,but in perceptions of collaboration and legitimate basis for practice. Surprisingly, collaboration wasonly indirectly indicated as clashing in two dimensions, technical and rationality bias. One way tointerpret this is that when directly asked whether it is important to collaborate in engineeringwork, no one would say ‘no’. This parallels the established view in the literature that the issue isnot whether or not ‘non-technical’ skills are important (Winberg et al. 2018, 167), but in what wayand when. This salient context-dependency can be seen when asking about aspects that stronglyrelate to collaboration, such as rigid semantic perspectives on the nature of facts or when explicitlyseeing the importance of knowledge but not the way it is important. In our results, the former can beseen in the rarefied codes VS worldly codes-clash for the rationality bias theme. Such simplified andcontext-independent views on the nature of facts in engineering work would challenge effective col-laboration, paralleling work on the importance of empathy in engineering by Walther, Miller, andSochacka (2017). The latter can be seen in the relatively weaker social relations of the code-clashin the technical bias theme. The operating definition uses elité codes to understand the role of thetechnical knowledge for the engineer as comprising strong epistemic and social relations. Theclash in this context imply that technical knowledge is only understood as having an internal andisolated value (ER+), missing the crucial identity-value (SR+) that technical knowledge constitutesfor an engineer. Thus, an indirect misapprehension of collaboration is indicated.

This makes for a more detailed portrait of skill perceptions in early practice which can offer a pre-liminary explanation of the non-universality of professional skills in the early career: Knowing thatcollaboration is important and that the legitimate basis for practice does shift might be the sameover early career trajectories, which was found to be the case in Passow and Passow (2017, 487),but knowing how might differ. It might be argued that this is just a matter of gaining professionalexperience, as posited by Grimson and Murphy (Ch. 9 in Hyldgaard Christensen 2015, 170) in pre-senting a view of acquiring professional competence (top layer).

However, while professional formation crucially takes place after transitioning to engineeringworkand cannot be pushed on or fully translated into the curriculum, students should be prepared in waysthat recognise the non-universal, situated, and integrative nature of professional skills. One way ofdoing this in the curriculum is through directed feedback on professional skills in the context ofdesign work (Gilbuena et al. 2015), and another would be the direction recommended by Wolff(2018, 192) to accept that non-universality of professional skills and contexts for problem-solvingand instead focus on developing code-shifting behaviour by making the codes explicit. Indeed,Wolff’s suggestion is supported by our findings that code-clashes in both collaboration and legitimatebasis for practice indicate code-shifting challenges in early practice. However, thequestion remains as tohow tomake the codes explicit. Wolff finishes bymaking a preliminary suggestion of issuing the sameproject brief to several student groups with varying affordances and constraints. This is an interestingpractical point of departure to enact this principle, but to step out of context and into amore conceptualperspective, educators needanunderstandingof the codes and impart them to students (192). It is herethat the operating definition can serve to motivate students as to why they should care about thecodes: without an explicit understanding of codes, it is hard to explicitly perceive the legitimatebasis for practice, in turn hampering collaboration andmaking the enactment of technical knowledgeitself difficult. The need for students to learn about codes and the requisite shifting behaviour can besituated in terms of ‘solving the problem’ and as a way of ‘using your engineering know-how’.

While in need of further investigation, one way of achieving this would be to focus on showingthe students (or letting them experience) the consequences of maintaining rarefied codes in thecontext of rationality bias. Not understanding the context-dependence of engineering data ortheir complexity severely affect the visibility of the legitimate basis of practice, which can be seenin the importance assigned to the concept of interiority as discussed by Cosgrove and O’Reilly(2018, 45–46), arguing that a reflexive stance to one’s own understanding can mitigate this risk.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 19

Another way to understand the results in terms of enabling this code-shifting behaviour would be toexplicate the consequences of Academic bias. This might seem counterintuitive at first to educatorsand students alike, but the importance here is not in stating that grades do not matter (they do),but in what way. The probability of finding real instances of clashes between knowledge codes andother specialisation codes during education is unlikely since most students and educators matchspecialisation codes in the context of academic bias. Therefore, it is important to impart a nuancedunderstanding of the role of grades to ensure that students know how to use them, as they willoften stand as legitimising social relations and consequently indicating elite codes or knower codes.A practical, but possibly sensitive, way of exemplifying this to students is to discuss deeply embeddedcounter-productive student group work practices of ‘excluding inferiority’ (See Work Practice 8 inFigure 1, Leonardi et al. 2002). While being a difficult subject, this could provide a semantic basisfor discussing perceptions of ‘incompetence’. A further example in the context of work placementof students can be seen in how the general development of professional skills are seen as unrelatedto grading and other assessments (Bennett, Richardson, and MacKinnon 2016; Edwards et al. 2015, 35–51). The reasons for this seem to often relate to not maintaining an integrative-perspective, either fromlack of industrial context knowledge on the part of educators or a view that students professional skillsdevelopment are their own responsibility (Amiet et al. 2020).

The implication of the shift found in Hypothesis 2, where social relations (understanding identity-legitimacy) and semantic gravity (understanding context-dependency) first clash but strengthen overtime, can also be used to motivate the importance of code-shifting to educators and students. Theimportance of foregrounding social relations and semantic gravity as a possible differentiating factorfor early practice can be motivated by Finding 6 in Passow and Passow’s review (2017, 498–499)which states that,whileproblem-solving is thecoreoverarching taskofengineers,what separates ‘ordin-ary’ from ‘outstanding’ engineers is effective coordination of other skills (499). This requires understand-ing and dynamically shifting along strengths of social relations as well as semantic gravity. Thus, at thetail end of a master’s programme, the engineering student is expected to be nominally capable inproblem-solving but needs to understand (even if phrased differently pedagogically) that this is thegeneral base layer for practice and that dimensions of Specialisation and Semanticsmust beunderstoodfor proper shifting in practice, especially over strengths of SR and SG. This is significant in the context of,for example, establishinga functionalmentoring relationshipduringworkplacementbetweena studentand a senior engineer, which is important for working efficiently (Davis, Vinson, and Stevens 2017). Suchrelationships are most common where responsibility is shared and hierarchy is weak (Davis et al. 2017).

6.2. Limitations

The preliminary nature of this work cautions against making broad claims about the nature of pro-fessional skills in general, and for any specific engineering discipline, including software-orientedones (such as in Group 1). The reason for this is the same as the departure of this study in the twoviews of integration and non-universality: The nature of each discipline and its knowledge base mustbe considered together with the practice context in which these are deployed, including such diverseconsiderations as industry-type, firm size, department type, kind of engineeringwork, level of ‘projectifi-cation’, and (inter)national contexts. This is less of a problem for the study itself since the intention is tofocus on the nature of difference itself, rather than the disciplinary fields.With that said, some commentscan tentatively bemade on the relation of the software-oriented roles of Group 1 to the results. Regard-ing the lower initial score, it is unlikely that early softwareengineers see engineeringexpertisedifferentlythan other engineers do because their professional practice ismorebound to goodgrades, blind faith innumbers, or that they see technical expertise as thebe-all-end-all of engineering (BjørnsonandDingsøyr2008): that it is not a question of misalignment of professional skills as such, but that software engineercommunities of practice do not require asmuch, or the same kind of, emphasis on professional skills forlegitimate peripheral participation (which would reflect the non-universality principle). In other words,the legitimate basis for practice might (initially) correctly be seen through knowledge codes.

20 E. FLENING ET AL.

Nevertheless, the longerone is amemberof sucha community, thegreater the increase in thebenefitsofthese skills (Lee, 1994), which is supported by Hypothesis 2.

6.3. Implications for engineering educators and industrial actors

Engineering educators should be mindful to motivate the need for professional skills to students, asthey can end up in different practice contexts. It might be discouraging if the emphasis on professionalskills seems not to be reflected in one’s first workplace. However, practice-oriented parts of curriculacan never fully represent actual practice, but can be valuable in making own attitudes visible to stu-dents and changing them (Paretti 2008). Gilbuena et al. (2015) exemplified how this enculturation intocommunities of practice can start already during education through feedback on professional skillssituated in senior design capstone courses. As such, effective attitude change strategies must bebuilt on an understanding of the more frequent initial career tracks of students in any programmebecause attitude is essential to learning professional skills (Byrne et al. 2018).

Industrial actors should not only consider that ‘students have not been taught what they need’when looking at the curricula of higher education institutions. Early-career engineers might beentirely forgiven for pushing back on suggestions that they lack in skill if neither their workcontext nor their social context challenges them on these skills. Newcomers cannot simply betold what constitutes expertise in their new practice context but must participate in the doingsthat constitute and necessitate such expertise.

Future research could dive into developing a deeper understanding of the semantic codes andspecialisation codes of the three themes. An interview study between the two groups couldachieve this. Another interesting avenue would be to further investigate the nature of this variabilityin early career misalignment: Does it differ for other fields? Are they predisposed to certain trajec-tories for initial misalignment, as is indicated for software developers in this work? Does it differin other possible measurements of misalignment than skills, such as empathy (Walther et al. 2017)?

7. Concluding comments

The transition from education to practice is still jarring for many students. This study shows that mis-alignment might exist along different themes depending on the engineering practice context thatnew engineers join, and as such makes a theoretical contribution in questioning an isolated and uni-versal view of non-technical skills for early practice. The practical contribution thus becomes an invi-tation to educators to strengthen attitude change strategies of professional skills, through anunderstanding of codes and code-shifting behaviours in and for students. Finding ways to showand motivate understanding engineering practice through legitimation codes might give earlyengineers an edge in their core activity of problem-solving, as was found byWolff (2018, 192) in prac-ticing engineers, if they can leverage this conceptual understanding into their practice.

Notes

1. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2020-2021/.

2. https://www.enaee.eu/eur-ace-system/standards-and-guidelines/#standards-and-guidelines-for-accreditation-of-engineering-programmes.

3. Here we mean “course” in the sense of a self-contained unit of learning, a structured number of which constitutea degree program, rather than in the UK sense where “course” means the degree program itself.

4. The student might have less stable required knowledge going into the next course, which obviously can worsenperformance, but the requirements put on the student does not change: Courses are (by design) independentenvironments of educational practice. However, development projects in engineering firms are not.

5. See the broad base of international signatories of the Magna Charta Universitatum, proclaiming academicfreedom and institutional autonomy principles for universities: http://www.magna-charta.org/magna-charta-universitatum.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 21

6. The pre-screening resulted in the inclusion of questions 1-4, and 8 into Part I. These were not used in the finalanalysis, but they have been included in the appendix for transparency.

7. We originally included four extra questions on top of the original twelve in Part II, but these are not used in thefinal analysis and are not a part of the scoring system which was designed by Trevelyan. They have beenincluded in the appendix for transparency.

8. Specifically misconceptions #1-5, #8, #10-12, and #15 in Trevelyan’s book.9. See the rightmost column-matrix of table 10 in the appendix for point-distributions for each statement. Trevel-

yan has made the grading scheme, together with a discussion of its point-distributions, available as an onlineappendix to his book The making of an expert engineer.

It is downloadable from the publisher under the file name App12-1-Practice Quiz 2 Self-assessment 140807 in acompressed folder found here: https://www.routledge.com/downloads/K24392/K24392_Appendices.zip.

10. The italicized explanations of each theme are directly cited from Trevelyan’s discussion motivating the gradingof each of these three statements (P2-1, 5, and 6 in table 10), again found under the file name App12-1-PracticeQuiz 2 Self-assessment 140807 at the download-link: https://www.routledge.com/downloads/K24392/K24392_Appendices.zip.

Disclosure statement

No potential conflict of interest was reported by the author(s).

ORCID

Fredrik Asplund http://orcid.org/0000-0001-5704-4504

Notes on contributors

Elias Flening is currently pursuing his PhD at the Division of Mechatronics at KTH Royal Institute of Technology. Heachieved a MSc in Industrial Engineering and Management from KTH in 2014, and has worked as a research assistantat the Stockholm School of Economics. Research interests include engineering design processes, engineering educationand project management in the mechatronics domain.

Fredrik Asplund received his PhD at the KTH Royal Institute of Technology in 2014, has been a Postdoctoral Researcherwithin the Mobility for Growth programme, and was employed as an Assistant Professor at KTH Royal Institute of Tech-nology in 2020. Throughout his academic positions he has taught courses in research methodology and supervised stu-dents at the master’s level. Research interests include system safety, empirical software engineering and engineeringeducation in the Cyber-Physical Systems domains. Dr Asplund is a Marie Curie Fellow and a VINNMER Fellow sincehis postdoctoral research at Rolls-Royce plc in Derby, UK.

Martin Edin Grimheden is associate professor and dean at the Machine Design department at KTH Royal Institute ofTechnology. Dr Grimheden has a combination of degrees in Engineering and Education. He has been heavily involvedin the transition into the BSc/MSc-system at KTH. His current research interest includes studies of learning in highereducation, of learning in Mechatronics, and of internationalization of higher education.

References

Amiet, D., J. Choate, J. Hoskin, and J. Dart. 2020. “Exploring Attitudes, Beliefs and Practices of Academic Staff TowardsUndergraduate Career Development in Non-vocational Courses.” Higher Education Research & Development, 40: 1–16.

Anderson, K. J. B., S. S. Courter, T. McGlamery, T. M. Nathans-Kelly, and C. G. Nicometo. 2010. “UnderstandingEngineering Work and Identity: A Cross-Case Analysis of Engineers Within Six Firms.” Engineering Studies 2 (3):153–174. doi:10.1080/19378629.2010.519772.

Auslander, D. M. 1996. "What is mechatronics?" IEEE/ASME Transactions on Mechatronics 1: 5–9.Bennett, D., S. Richardson, and P. MacKinnon. 2016. Enacting Strategies for Graduate Employability: How Universities Can

Best Support Students to Develop Generic Skills. Sydney: Australian Government Office for Learning and Teaching.Bjørnson, F. O., and T. Dingsøyr. 2008. “Knowledge Management in Software Engineering: A Systematic Review of

Studied Concepts, Findings and Research Methods Used.” Information and Software Technology 50 (11): 1055–1068. doi:10.1016/j.infsof.2008.03.006.

Bradley, D. 2010. “Mechatronics – More Questions Than Answers.” Mechatronics 20 (8): 827–841. doi:10.1016/j.mechatronics.2010.07.011.

22 E. FLENING ET AL.

Brunhaver, S. R., R. F. Korte, S. R. Barley, and S. D. Sheppard. 2018. “Bridging the Gaps Between Engineering Educationand Practice.” In U.S. Engineering in a Global Economy, edited by R. B. Freeman and H. Salzman, 129–163. University ofChicago Press.

Bucciarelli, L. L. 1994. Designing Engineers. Cambridge, MA: MIT Press.Byrne, Z. S., J. W. Weston, and K. Cave. 2018. “Development of a Scale for Measuring Students’ Attitudes Towards

Learning Professional (i.e., Soft) Skills.” Research in Science Education 50 (4): 1417–1433. doi:10.1007/s11165-018-9738-3.

Cappelli, P. 2008. Talent on Demand: Managing Talent in an Age of Uncertainty. Boston: Harvard Business Press.Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York: Taylor & Francis.Colwell, J. 2010. “Soft Skills for the New Economy: Their Place in Graduate Education in Engineering and Engineering

Technology.” Paper Presented at the ASEE Conference and Exposition, Louisville.Cosgrove, T., and J. O’Reilly. 2018. “Theory, Practice and Interiority: An Extended Epistemology for Engineering

Education.” European Journal of Engineering Education 45 (1): 38–54. doi:10.1080/03043797.2018.1544226.Davis, P., A. Vinson, and R. Stevens. 2017. “Informal Mentorship of New Engineers in the Workplace.” Paper presented at

2017 ASEE Annual Conference & Exposition, Columbus, Ohio. doi:10.18260/1-2–28527.Downey, G. L. 1998. The Machine in Me: An Anthropologist Sits Among Computer Engineers. New York: Routledge.Downey, G. L. 2015. “The Normative Contents of Engineering Formation: Engineering Studies.” In Cambridge Handbook

of Engineering Education Research, edited by A. Johri and B. M. Olds, 693–712. Cambridge University Press.Edström, K., and A. Kolmos. 2014. “PBL and CDIO: Complementary Models for Engineering Education Development.”

European Journal of Engineering Education 39 (10): 539–555.Edwards, D., K. Perkins, J. Pearce, and J. Hong. 2015. “Work Integrated Learning in STEM in Australian Universities.” Final

Report: Submitted to the Office of the Chief Scientist.Funder, D. C., and D. J. Ozer. 2020. “Evaluating Effect Size in Psychological Research: Sense and Nonsense.” Advances in

Methods and Practices in Psychological Science 3 (4): 509–509. doi:10.1177/2515245920979282.Gignac, G. E., and E. T. Szodorai. 2016. “Effect Size Guidelines for Individual Differences Researchers.” Personality and

Individual Differences 102: 74–78. doi:10.1016/j.paid.2016.06.069.Gilbuena, D. M., B. U. Sherrett, E. S. Gummer, A. B. Champagne, and M. D. Koretsky. 2015. “Feedback on Professional Skills

as Enculturation into Communities of Practice.” Journal of Engineering Education 104 (1): 7–34. doi:10.1002/jee.20061.Harashima, F., M. Tomizuka, and T. Fukuda. 1996. "Mechatronics - ’What Is It, Why, and How?’" An editorial. IEEE/ASME

Transactions on Mechatronics 1: 1–4.Hemphill, J. F. 2003. “Interpreting the Magnitudes of Correlation Coefficients.” American Psychologist 58 (1): 78–79.

doi:10.1037/0003-066x.58.1.78.Hyldgaard Christensen, S. 2015. Engineering Identities, Epistemologies and Values: Engineering Education and Practice in

Context: Vol. 2. Springer.Itani, M., and I. Srour. 2016. “Engineering Students’ Perceptions of Soft Skills, Industry Expectations, and Career

Aspirations.” Journal of Professional Issues in Engineering Education and Practice 142 (1): 1–12. doi:10.1061/(asce)ei.1943-5541.0000247.

Janschek, K. 2012. "Mechatronic Systems Design." Computer Applications in Engineering Education 4.Jesiek, B. K., N. Trellinger, and S. Nittala. 2017. “Closing the Practice Gap: Studying Boundary Spanning in Engineering

Practice to Inform Educational Practice.” Proceedings – Frontiers in Education Conference, FIE, 1–9, October. doi:10.1109/FIE.2017.8190503.

Korte, R., S. Brunhaver, and S. Sheppard. 2015. “(Mis)Interpretations of Organizational Socialization: The Expectationsand Experiences of Newcomers and Managers.” Human Resource Development Quarterly 26 (2): 185–208. doi:10.1002/hrdq.21206.

Lang, J. D., S. Cruse, F. D. McVey, and J. McMaster. 1999. “Industry Expectations of New Engineers: A Survey to AssistCurriculum Designers.” Journal of Engineering Education 88 (1): 43–51.

Lee, D. M. S. 1994. “Social Ties, Task-Related Communication and First job Performance of Young Engineers.” Journal ofEngineering and Technology Management 11 (3–4): 203–228. doi:10.1016/0923-4748(94)90010-8.

Leonardi, P. M., M. H. Jackson, and A. Diwan. 2002. “Rationalization and the Persistence of Counter-ProductiveTechnology Design Practices in Student Engineering.” Academy of Management Journal 52: 211–248.

Martin, R., B. Maytham, J. Case, and D. Fraser. 2005. “Engineering Graduates’ Perceptions of How Well They WerePrepared for Work in Industry.” European Journal of Engineering Education 30 (2): 167–180. doi:10.1080/03043790500087571.

Maton, K. 2014. Knowledge and Knowers: Towards a Realist Sociology of Education. London: Routledge.Maton, K. 2016. “Legitimation Code Theory: Building Knowledge About Knowledge-Building.” In Knowledge-building

Educational Studies in Legitimation Code Theory, 1–255. London: Routledge.Maton, K. 2020. “Semantic Waves: Context, Complexity and Academic Discourse.” In Accessing Academic Discourse

Systemic Functional Linguistics and Legitimation Code Theory, edited by J. R. Martin, K. Maton, and Y. J. Doran, 59–85. London: Routledge.

Meier, R. L., M. R. Williams, and M. A. Humphreys. 2000. “Refocusing Our Efforts: Assessing Non-technical CompetencyGaps.” Journal of Engineering Education 89 (3): 377–385. doi:10.1002/j.2168-9830.2000.tb00539.x.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 23

Michael, D. 1998. Thinking Like an Engineer.Paretti, M. C. 2008. “Teaching Communication in Capstone Design: The Role of the Instructor in Situated Learning.”

Journal of Engineering Education 97 (4): 491–503. doi:10.1002/j.2168-9830.2008.tb00995.x.Pascail, L. 2011. “The Emergence of the Skills Approach in Industry and its Consequences for the Training of Engineers.”

European Journal of Engineering Education 31 (1): 55–61. doi:10.1080/03043790500428965.Passow, H. J. 2007. “What Competencies Should Engineering Programs Emphasize? A Metaanalysis of Practitioners’

Opinions Informs Curricular Design.” Paper Presented at the 3rd International CDIO Conference. http://www.cdio.org/knowledge-library/documents/what-competencies-should-engineering-programs-emphasize-meta-analysis–0.

Passow, H. J. 2012. “Which ABET Competencies Do Engineering Graduates Find Most Important in Their Work?” Journalof Engineering Education 101 (1): 95–118. doi:10.1002/j.2168-9830.2012.tb00043.x.

Passow, H. J., and C. H. Passow. 2017. “What Competencies Should Undergraduate Engineering Programs Emphasize? ASystematic Review.” Journal of Engineering Education 106 (3): 475–526.

Patton, M. Q. 2002. Qualitative Research and Evaluation Methods. 3rd ed. Thousand Oaks: SAGE Publications.Rogers, P., and R. Freuler. 2015. “The ‘T-Shaped’ Engineer.” Paper Presented at the 122nd ASEE Annual Conference &

Exposition, Seattle, June 14–17.Sageev, P., and C. Romanowski. 2001. “A Message from Recent Engineering Graduates in the Workplace: Results of a

Survey on Technical Communication Skills.” Journal of Engineering Education 90: 685–693.Shuman, L. J., M. Besterfield-Sacre, and J. McGourty. 2005. “The ABET “Professional Skills” – Can They Be Taught? Can

They Be Assessed?” Journal of Engineering Education 94 (1): 41–55. doi:10.1002/j.2168-9830.2005.tb00828.x.Törngren, M., F. Asplund, S. Bensalem, J. McDermid, R. Passerone, H. Pfeifer,… B. Schätz. 2017. “Characterization,

Analysis, and Recommendations for Exploiting the Opportunities of Cyber-Physical Systems.” In Cyber-PhysicalSystems, edited by Houbing Song, Danda Rawat, Sabina Jeschke, and Christian Brecher, 3–14. Academic Press.

Trevelyan, J. 2007. “Technical Coordination in Engineering Practice.” Journal of Engineering Education 96 (3): 191–204.DOI 10.1002/j.2168-9830.2007.tb00929.x.

Trevelyan, J. 2008a. “Coordination in Mechatronic Engineering Work.” In Mechatronics and Machine Vision in Practice,edited by J. Billingsley, and R. Bradbeer, 51–61. Springer.

Trevelyan, J. 2008b. “The Intertwined Threads of Work.” Engineers Australia 80 (2): 38–39.Trevelyan, J. 2014. The Making of an Expert Engineer, 574. CRC Press.Trevelyan, J. 2019. “Transitioning to Engineering Practice.” European Journal of Engineering Education 44 (6): 821–837.

doi:10.1080/03043797.2019.1681631.Trevelyan, J., and B. Williams. 2018. “Value Creation in the Engineering Enterprise: An Educational Perspective.” European

Journal of Engineering Education 3797, 1–23. Taylor & Francis.Vinck, D. 2003. Everyday Engineering: An Ethnography of Design and Innovation. Cambridge, MA: MIT Press.Walter, V. 1990. What Engineers Know and How They Know It. The Johns Hopkins University Press.Walther, J., S. E. Miller, and N. W. Sochacka. 2017. “A Model of Empathy in Engineering as a Core Skill, Practice

Orientation, and Professional Way of Being.” Journal of Engineering Education 106 (1): 123–148. doi:10.1002/jee.20159.

Winberg, C., M. Bramhall, D. Greenfield, P. Johnson, P. Rowlett, O. Lewis,… K. Wolff. 2018. “Developing Employability inEngineering Education: A Systematic Review of the Literature.” European Journal of Engineering Education 45 (2): 165–180. doi:10.1080/03043797.2018.1534086.

Wolff, K. 2018. “Researching the Engineering Theory-Practice Divide in Industrial Problem Solving.” European Journal ofEngineering Education 45 (2): 181–195. doi:10.1080/03043797.2018.1516738.

Appendix

Two-part survey on engineering career and perceptions of expertise

Response rates for the whole sample and the two groups

Table A1. Response rates.

Cohort year Sample size Response frequency N respondents Group 1 Group 22011 26 42% 11 7 42012 33 45% 15 6 92013 50 62% 29 6 232014 36 36% 12 3 92015 44 48% 21 6 152016 33 33% 12 6 62017 61 44% 30 14 162018 63 44% 28 14 14Totals 346 46% 158 62 96

24 E. FLENING ET AL.

Table A2. Test score results for the two groups.

Groups Non-SW SW

Questions Mean SD Mean SDP2-1 4.5 5.0 2.9 4.6P2-2 3.6 4.0 2.9 3.9P2-3 2.1 3.4 1.2 2.6P2-4 4.1 4.9 4.7 5.0P2-5 7.7 2.8 6.6 2.9P2-6 3.1 4.7 1.5 3.6P2-7 9.7 1.7 9.8 1.3P2-8 7.9 1.9 7.9 1.8P2-9 3.3 4.0 2.6 3.7P2-10 5.4 5.0 5.5 5.0P2-11 8.0 4.0 7.6 4.3P2-12 3.4 4.8 3.7 4.9Total Score 63 16 57 15

Part 1: questions regarding career path and work-content

Table A3. Survey part I – career path and role.

ID Question text Answer optionsP1-1 Your first job after graduation was in the field of… Mechatronics; Mechanical; Software; Electronics; Control;

Other;P1-2 Do you regularly work on developing mechatronic systems? Yes, every day; Yes, every week; Somewhat, every month;

No, not anymore; No, I never did;P1-3 Did you do your Master thesis in Mechatronics? Yes; No, I did it in < free text>;P1-4 How many places have you worked at since graduation? 0; 1; 2; 3; 4; 5; 6; 7; 8+;P1-5 How much of your working time is spent interacting with

colleagues to solve a current task/problem?0%; 25%; 50%; 75%; 100%;

P1-6 Please choose up to three roles, ranked in importance, thatbest capture what you actually do currently at work.

Most Important; Important; Somewhat important; (SeeTable 1 for the 13 options)

P1-7 How competent are you in the fields below?a. Mechanical Engineeringb. Software Engineeringc. Electronics Engineeringd. Control Engineering

Not centrally important; secondary competency; Corecompetency;

P1-8 Where do you focus to create value for your employer?a. I’m leading one or several product developmentprojects/teams.b. I’m a core part of one or several technical developmentteams.c. I’m improving product development processes.d. I provide crucial support (any kind) to all ourdevelopment projects/teams.e. I’m brought in for my highly specialised technicalexpertise to facilitate complex design decisions.

This is not where I focus my efforts; Some of my deliveredvalue lies here; Most of my delivered value lies here;

Part 2: statements probing opinions about engineering expertise in practicePart 2 uses a five-point Likert scale with the options strongly disagree, disagree, neutral, agree, strongly agree for therespondent to take a position on each of the twelve statements. In the scoring table column, the grading schemesassigns a point-value for each answer on the Likert scale.

The following four questions were added to the original twelve from the questionnaire. In this article only the orig-inal twelve were used in the analysis. They are included here for completeness sake: Skill in building collaborativerelationships with other engineers to solve technical problems are more important than skill in solving the same problemsyourself; If you give a concise, rational argument based on quantified facts you will convince others; Understanding thereason for investment and budget decisions in a product development project is important for the engineer working inthat project; Expert engineers solve technical problems by independently developing their technical competence withouthelp so they can address the problem by themselves.

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 25

Table A4. Survey part II – Trevelyan’s (2014) engineering expertise questionnaire.

ID Statements for the respondent to take a position against Scoring table

P2-# [statement text]Stronglydisagree Disagree Neutral Agree

Stronglyagree

P2-1 An engineer who achieved higher grades at university tends to perform better in engineering work 0 0 10 0 0P2-2 As an engineer, it is critical that you accumulate sufficient technical knowledge by yourself to solve any problem that you are

likely to be confronted with10 10 10 2 0

P2-3 Engineering is a hands-on practical occupation 10 10 3 0 0P2-4 In engineering, many decisions are made on the basis of perceptions that can be inaccurate or incorrect 0 0 0 10 10P2-5 Being a successful engineer depends primarily on your technical expertise 10 10 7 3 0P2-6 Facts are more objective and unbiased when stated in terms of numbers than words 10 10 0 0 0P2-7 The ability to build collaborative relationships with more experienced engineers, suppliers, and site supervisors has more of an

effect on workplace performance in engineering than academic ability0 0 10 10 10

P2-8 most of what an engineer needs to know is learnt in the workplace 2 7 7 10 7P2-9 You can only learn communication skills by practice; they cannot be taught 10 7 0 0 0P2-10 In engineering, decisions are almost always based on technical facts, computation, analysis, results, and logic 0 10 10 0 0P2-11 Engineers often have to work with vague verbal statements of requirements from their clients 0 0 0 10 10P2-12 Novice engineers, on average, spend just as much time interacting with other people as senior engineers, who often have

management responsibilities0 0 0 10 10

26E.FLEN

INGET

AL.

Scatterplot

EUROPEAN JOURNAL OF ENGINEERING EDUCATION 27