MAKEITREAL PROJECTmakeitreal.info/wp-content/uploads/2017/10/Int...MAKEITREAL Project Erasmus+ KA2 2016-1-PL01-KA201-026492 4 Abstract In this interim report, we present findings from

MAKEITREAL PROJECT

Addressing underachievement in STEAM education through real product design and making practices

Erasmus+ KA2 2016-1-PL01-KA201-026492

Interim Evaluation Report

based on the 1st pilot in Poland

Lead Partner: Warsaw University of Technology (WUT)

Authors: Katarzyna Hołownicka (WUT)

Circulation: Public

Version: 01

Stage: Final

Date: August 2017

MAKEITREAL Project Erasmus+ KA2 2016-1-PL01-KA201-026492

2

Declaration

This report has been prepared in the context of the MakeITReal project. Where other published and

unpublished source materials have been used, these have been acknowledged.

Copyright

© Copyright 2016 - 2018 the MakeITReal Consortium

All rights reserved.

This document is licensed to the public under a Creative Commons Attribution-NonCommercial-

ShareAlike 4.0 International License.

Funding Disclaimer

This project has been funded with support from the European Commission. This communication

reflects the views only of the author, and the Commission cannot be held responsible for any use

which may be made of the information contained therein.

http://creativecommons.org/licenses/by-nc-sa/4.0/




3

Table of Contents

Abstract .......................................................................................................................... 4

Chapter 1: Introduction ................................................................................................ 5

1.1 Pilot participants profile ................................................................................................. 5

Chapter 2: Pre and post test ....................................................................................... 7

2.1 Students’ interests in different school subjects including STEAM ................................ 7

2.2 Relevance & School Experiences............................................................................... 10

2.3 STEM profiles ............................................................................................................. 13

2.3.1 Maths .............................................................................................................. 13

2.3.2 Science ........................................................................................................... 16

2.3.3 Technology/Engineering ................................................................................. 18

2.3.4 21st century skills ........................................................................................... 20

2.3.5 Arts ................................................................................................................. 23

2.4 Perceptions and future ................................................................................................ 24

Chapter 3: Feedback generated by the students ....................................................... 28

Chapter 4: Feedback generated by the participant teachers ..................................... 31

Chapter 5: Conclusion ............................................................................................... 34


4

Abstract

In this interim report, we present findings from case studies describing the 1st round of the learning

intervention based on feedback generated by the participant teachers and students from Poland. This

evaluation is aimed to inform the 2nd MakeITReal pilot phase.


5

Chapter 1: Introduction

This report concerns the evaluation of the project “Addressing underachievement in STEAM

education through real product design and making practices” co-funded by the Erasmus+

Programme of the European Union, KA2 project (2016-1-PL01-KA201-026492).

Interim report represents a first reflection and summary of the activities carried out during the first

10 months of the project lifecycle (from April to June 2017). The report presents the results of case-

study that was conducted with I Liceum Ogolnoksztalcace im. Marii Sklodowskiej – Curie of

Sokolow Podlaski (I LO).

MakeITReal is an Erasmus+ project that engages secondary school students (13-17 years old) that

underperform in STEAM (Science, Technology, Engineering, Arts and Maths) education and keep

distance from STEM related disciplines in creative product design and making practices. Special

focus is placed on teachers’ professional development and on building the pedagogical and technical

skills needed in order to facilitate the learning process, to support students’ in 3D modeling and 3D

printing and to make connections across STEAM disciplines.

The objectives for the project are to:

• Design and model an active, learner-centered teaching approach for engaging underachievers

into STEAM related projects through real product design and making practices

• To help underachievers in STEM related subjects to improve their performance and develop

21st century skills through their engagement in interdisciplinary projects related to real life

• Plan and enact activities and workshops that promote teacher professional learning and

pedagogic change

• Create OER that will support school community members (within and beyond the

partnership) to apply the MAKEITREAL learning intervention

• Open STEM education though the infusion of arts and the support of product design and

making practices moving beyond clichés according to which only STEM-talented students

can make it.

• Establish synergies among schools, academia and industry towards creative and meaningful

engagement in STEAM education.

The MakeiITReal learning intervention is carried out in two pilot stages. The pilots would cover one

school period of 3 months initially and 6 months subsequently. During the first pilot phase, three

interdisciplinary projects in STEAM education will be practiced, during the second pilot stage - seven

projects.

The tools for monitoring the learning interventions, documenting feedback, collecting feedback from

students and teachers are described in the pilot protocol (Intellectual Output No. 3).

1.1 Pilot participants profile

The project is addressed to students who demonstrate low performance in STEM education and keep

distance from STEM related discipline. A series of selection criteria were defined and agreed upon

by the whole consortium to be considered in the participants recruitment for two phases of the pilot

within the projects (Eligibility profile card). The guidelines are also include information on how to

select the students that are going to participate in the MakeITReal learning intervention. Accordance

with it, the priority to participation should be given to those that demonstrate low performance in

STEM related subjects and feel uncomfortable with STEM or think that STEM subjects are not

related to real life and do not allow them to be creative. Than a number of other criteria/conditions

that lead to underachievement and school failure should be also considered for the final selection.


6

In the first round of the pilot participated 53 students. There are very few withdraws due to

participants’ time constrains. A questionnaire survey was dispatched to all participated students.

Students were asked to provide basis information includes their sex and data on age.

Figure 1: Gender and ages of surveyed students

As shown in Figure 1Błąd! Nie można odnaleźć źródła odwołania., 24 (47%) of the respondents

were female. Thus, the sample comprised a reasonably equal proportion of males and females. The

students were 16-17 years old.

Evaluation data were also collected from 3 teachers who participated in the first round of the pilot

These teachers have been trained in 3D modeling and printing during the first teacher training

workshop (C1) that organised by the AUTH team in Thessaloniki (Greece). The teachers who

participated in the survey have been teaching STEM subjects for over 20 years (see Table 1).

Table 1: Teaching area and experience of surveyed teachers

Team Subjects taught by the participant teachers Year of teaching experience

Poland Teacher 1 Physics 32

Teacher 2 Mathematics 21

Teacher 3 Mathematics 30

14

17

14

8

0

2

4

6

8

10

12

14

16

18

Male Female

16 17


7

Chapter 2: Pre and post test

Students who participated in the 1st round of the pilot were asked to complete pre-test (before the

intervention) and post-test (after the intervention). Students received special codes that were used to

keep the assessment data anonymous and allowed us to compare results of two assessments taken at

different times. The survey aimed to explore students’ experiences, opinions, skills and views about

STEAM. The student survey comprised five sections: ‘General interests’, ‘STEAM interest’,

‘Relevance & School Experiences’, ‘STEAM profiles’, ‘Perceptions & Future’.

In summary, the MakeITReal learning intervention had a positive impact on students’ attitudes and

motivation towards STEM related disciplines. Students came to see these disciplines as more

pleasant. They are planning to work harder in their math and science courses. Students realized that

the things that they learn in Maths and Science matter in real and daily life. In particularly, students

appreciate STEM practical applicability in everyday life and they are more interested in creating new

objects. This positive impact of MakeITReal learning intervention on links STEM to real life has

been noticed despite the fact that students sometimes or rarely received such examples in schools.

The students were moderately positive about the link between Arts and STEM. In additions, they

were not sure about the impact of Arts on their motivation and interest in STEM subjects. Students

admitted that sometimes or more frequent were given the opportunity to express or replenish their

creativity, were allowed to use artistic methods to show their understanding of a concept/event/object

and were involved in hands-on and practical activities using also math and/or science in schools.

The students agreed with the statement “Knowing how to use math and science together can allow

me to make useful and creative things”. They stated that during the mathematic lessons the teachers

sometimes or rarely included also other subject areas or disciplines.

In addition, the students showed high levels of confidence in applying the skills and competencies

required in the 21st century. The students appear to be more confident as far as their problem solving,

decision making and presentation and communication skills are concerned.

The MakeITReal learning experience appears also to have the potential to inspire young students in

making STEAM a future career choice. The students agreed with the statement that their education

will create many future opportunities for them. In schools students sometimes talked about the types

of jobs and careers that use Science and Math but they would consider a job that involves using

science and/or maths.

2.1 Students’ interests in different school subjects

including STEAM

This section presents and discusses students’ responses to a set of questions that aim at exploring

students’ interests in STEAM subjects. The participant students were asked to select up to 3 subjects

that they are the most interested in and subjects that they are the least interested in. Below you will

find the students' preferences before and after the MakeITReal intervention (Figure 2 and Figure 3).


8

Figure 2: Favourite subjects in school

Note that respondents were able to choose up to 3 options.

Before the 1st pilot began, students were the most interested in Physics Sport, Foreign languages and

Geography. Later students pointed out Physics, Sports, Foreign languages and Algebra as the most

interesting subjects. It is noteworthy that after comparing the results of pre-post research one can

notice an increase in interest in Physics, Geometry and Algebra.

Figure 3: Unpopular subjects in school


2

2

4

11

7

23

9

13

14

23

14

7

6

1

3

5

16

10

29

10

13

17

24

14

7

8

0 5 10 15 20 25 30 35

Literature

History

Arts

Algebra

Geometry

Physics

Chemistry

ICT

Foreign languages

Sports

Geography

Music

Biology

Post-test Pre-test

30

29

9

8

6

13

7

4

2

3

5

8

17

23

27

7

5

3

11

5

8

1

4

5

6

23

0 5 10 15 20 25 30 35

Literature

History

Arts

Algebra

Geometry

Physics

Chemistry

ICT

Foreign languages

Sports

Geography

Music

Biology

Post-test Pre-test


9

In response to a question about the topic of least favourite subjects, the students most frequently

mentioned Literature, History and Biology before and in the end of the MakeITReal intervention.

In the study period, the decline of indications of Geometry, Algebra, Physics as the least interesting

subjects was observed. These findings are consistent with those of the previous question and indicate

the positive influence of the MakeITReal learning intervention on students’ interest in the area of

mathematics and physics. However, it is worth noting that there is no sign, so far, that this positive

tendency can necessarily inspire a positive long-lasting engagement in Math and Physics.

The students were also asked about their level of interest in Science, Maths, Arts and

Engineering/Technology. The figures from 4 to 7 show the level of students' interest in these subjects

before and after intervention using a 4-point Likert scale. Taking into account the answers

“Interested” and “Very interested” it can be seen increase students’ interest in STEAM disciplines.

The most increase in interest was noticed in case of maths and Engineering/Technology. Changes

were less apparent in the case of arts. Detailed statistics are given below.

Figure 4: The level of interest in Science

38 (out of 53) students were interested or very interested in science before the pilot. The number of

students who agree with this statement increases to 44 (11%) in the end of the workshop.

Figure 5: The level of interest in Math

Interest in maths decelerated 31 students before the MakeITReal interventions and 43 students (more

23%) in the end of the pilot.

1 1

14

8

2830

10

14

0

5

10

15

20

25

30

35

Pre-test Post-test

Not at all interested Not so interested Interested Very interested

5 5

17

5

1921

12

22

0

5

10

15

20

25

Pre-test Post-test



10

Figure 6: The level of interest in Arts

38 students stated that were interested or very interested in arts before the pilot. The number of

students who agree with this statement slightly increases to 43 (6%) in the end of the workshop.

Figure 7: The level of interest in Engineering and Technology

32 students were interested or very interested in Engineering and Technology before the pilot.

The number of students who agree with this statement increases to 41 (17%) in the end of the

workshop.

2.2 Relevance & School Experiences

The intention of this section is shading light into students’ school experiences with an emphasis on

STEM activities in the school, opportunities for creativity, expression of artistic skills and reflection

upon STEM connections to real life. The figures from 8 to 14 show how often students were exposed

to the various activities at school that might impact in increasing their interest in, and improving their

knowledge of STEM careers. In both the pre- and post-test, the students most frequently responded

that such activities were experienced sometimes at their class/school. Comparison the results Pre-

Post test showed that student noticed that they are more frequent involved in creative tasks that are

meaningful for them and teachers are somewhat more often inclined to incorporate other disciplines

during their lessons. Detailed statistics are given below.

3 2

1210

2830

10 11

0

5

10

15

20

25

30

35

Pre-test Post-test


8

2

13

9

20

27

1214

0

5

10

15

20

25

30

Pre-test Post-test



11

Figure 8: The distribution of answers for the question „You get examples of how the things you learn in Math & Science

matter in the real world

Before the pilot 30 students out of 53 stated that only sometimes received examples of how the things

they learn in Math & Science matter in the real world. In turn 4 students stated that this happened

frequently or always. Feedback provided by the students at the end of the pilot shows that this

opportunity has occurred slightly more frequent.

Figure 9: The distribution of answers for the question „When you study a mathematical topic, your teachers include also

other subject areas or disciplines (i.e. Arts, History, etc.)”

The vast majority of Polish students in both two phases of survey (45 students in pre-test, 38 students

in post-test) declared that when they studied a mathematical topic, their teachers rarely or never

included also other subject areas or disciplines.

244

1

13 14

3027

47

0

5

10

15

20

25

30

35

Pre-test Post-test

I do not know Never Rarely Sometimes Frequently or Always

2 2

23

18

2220

5

13

1

0

5

10

15

20

25

Pre-test Post-test



12

Figure 10: The distribution of answers for the question „Get involved in creative tasks that are meaningful to you”

According the pre-test, 27 students out of 53 sometimes were get involved in creative tasks that are

meaningful to them, while 14 students– frequently or always. The result of post-test survey shows

that this possibility was slightly more frequent.

Figure 11: The distribution of answers for the question „ You are given opportunities to express or replenish your

creativity”

About half of students (before and after the pilot) had opinion that they sometimes were given

opportunities to express or replenish their creativity. Over one quarters of respondents thought that

these opportunities were frequently or always offered.

Figure 12: The distribution of answers for the question „ You are allowed to use artistic methods to show your

understanding of a concept/event/object”

1 21

10 10

27

20

14

21

0

5

10

15

20

25

30

Pre-test Post-test


133

8 9

26 27

15 14

0

5

10

15

20

25

30

Pre-test Post-test


56

9

5

2221

12

16

5 5

0

5

10

15

20

25

Pre-test Post-test



13

Most of the students in pre and post-test stated that rarely (22 and 21 respectively) or sometimes

(12 and 16 respectively) they were allowed to use artistic methods to show your understanding of

a concept/event/object.

Figure 13: The distribution of answers for the question „Get involved in hands-on and practical activities using also

math and/or science”

Students' statements about whether they were involved in hands-on and practical activities using also

math and/or science were very divided. However, in this issue was the predominance of students who

answered "sometimes" (22 out of 53 in pre-test, 20 in post-test).

Figure 14: The distribution of answers for the question „Talk about the types of jobs and careers that use Science and

Math”

Every 7 students highlighted (before and after the pilot) that sometimes or more frequent they talked

about the types of jobs and careers that use Science and Math.

2.3 STEM profiles This section presents students’ reflection on their performance, perceptions and feelings in relation

to Maths, Science, Engineering/Technology, Arts and the 21st century skills. These five core thematic

areas are exploited to help us shade light into the students’ profile in STEAM. The students were

invited to state the extent to which they agree or disagree on each statement using a 5-point Likert

scale.

2.3.1 Maths

The figures from 15 to 20 show the students' views on set of statements related to the discipline of

Mathematics. The results of comparing pre- and post-test indicate that there is a growing trend in

1 1

910

15

12

2220

6

10

0

5

10

15

20

25

Pre-test Post-test


6 66

11

27

21

14 15

0

5

10

15

20

25

30

Pre-test Post-test

Never Rarely Sometimes Frequently or Always


14

students' positive perception of mathematics. They usually do well in Maths. The students consider

math as pleasant subject and appreciate its practical applicability in everyday life. Taking into

account the results of Pre-Post tests it can be seen that students are a bit more confident when they

are doing maths. Nevertheless, there is room for further improvements. Students has started seen

Maths as less hard for them. Detailed statistics are given below.

Figure 15: The distribution of answers for the question „I usually do well in Maths”.

Every 7 students agree or somewhat agree with the statement that they usually do well in Maths

(before and after the pilot).

Figure 16: The distribution of answers for the question „ I think learning maths will help me in my daily life”.

41 students out of 53 reported before the pilot that learning maths will help them in their daily life.

3 more students agreed with this statement in the end of the interventions.

1 25 6

107

3229

5

9

0

5

10

15

20

25

30

35

Pre-test Post-test

Disagree Somewhat disagree Neither Agree Nor Disagree Somewhat agree Agree

52

4 43 3

1614

25

30

0

5

10

15

20

25

30

35

Pre-test Post-test



15

Figure 17: The distribution of answers for the question „I enjoy doing Math”.

The number of students that enjoyed doing Maths slightly increased from 35 to 39 students (out of

53).

Figure 18: The distribution of answers for the question „I am sure of myself when I do Maths”.

Only 18 students in pre-survey agreed or somewhat agreed with the statement that they were sure of

themselves when they do Maths. The number of students with such opinion increased to 26 (15%) in

post-survey.

Figure 19: The distribution of answers for the question „Maths is hard for me”.

20 students disagreed with the statement that maths is hard for them before the pilot. In the end of

the workshop, 5 more shared similar opinion.

2 3

75

9

6

2523

10

16

0

5

10

15

20

25

30

Pre-test Post-test


2 2

14

11

19

1416

24

2 2

0

5

10

15

20

25

30

Pre-test Post-test


2 2

18

23

1719

14

5

24

0

5

10

15

20

25

Pre-test Post-test



16

Figure 20: The distribution of answers for the question „Maths is boring”.

7 out of 10 students disagreed or somewhat disagreed with the statement that Maths is boring (before

and after the pilot).

2.3.2 Science


science. The results of comparing pre- and post-test indicate that there is a growing trend in students'

positive perception of science. They enjoy doing science and appreciate its practical applicability in

everyday life. The students began to feel a little more confident when they were doing maths, but

they did not begin to perceive science as less difficult and less boring. Detailed statistics are given

below.

Figure 21: The distribution of answers for the question „I usually do well in Science”.

Approximately 34 students out of 53 students agree or somewhat agree with the statement that they

usually do well in Science (before and after the pilot).

2119

1718

1011

13

42

0

5

10

15

20

25

Pre-test Post-test


2 1

7 6

11 11

3032

3 3

0

5

10

15

20

25

30

35

Pre-test Post-test



17

Figure 22: The distribution of answers for the question „I think learning Science will help me in my daily life”.

On average, 34 students stated that learning science will help them in their daily life.

Figure 23: The distribution of answers for the question „I enjoy doing science”.

The number of the Polish students that enjoyed doing science slightly increased from 39 students out

to 42 students.

Figure 24: The distribution of answers for the question „I am sure of myself when I do science”

Only 23 students in pre-survey agreed or somewhat agreed with the statement that they were sure of

themselves when they do science. The number of students with such opinion slightly increased to 28

students in post-survey.

2

6 5

1113

26 26

8 9

0

5

10

15

20

25

30

Pre-test Post-test


2 223

10

6

19

2220 20

0

5

10

15

20

25

Pre-test Post-test


3 24

6

23

17

20

24

3 4

0

5

10

15

20

25

30

Pre-test Post-test



18

Figure 25: The distribution of answers for the question „Science is hard for me”.

Most of the students (34 students out of 53) before the pilot reported that science is not hard for them.

Nevertheless, after the pilot the number of students with the same view decreased (27 students).

Figure 26: The distribution of answers for the question „Science is boring”.

7 out of 10 students disagree or somewhat disagree with the statement that science is boring (before

and after the pilot).

2.3.3 Technology/Engineering


technology/engineering. The results of comparing pre- and post-test indicate that there is a growing

trend in students' positive perception of technology/engineering. They are interested in creating new

objects and are a little more aware of knowledge from math and science can be used to design

everyday objects. Detailed statistics are given below.

5 4

29

23

14

18

5 6

2

0

5

10

15

20

25

30

35

Pre-test Post-test


1416

25

20

9

14

3 31

0

5

10

15

20

25

30

Pre-test Post-test



19

Figure 27: The distribution of answers for the question „I like to imagine creating new products”

About 37 students out of 53 students stated that like to imagine creating new products (before and

after the pilot).

Figure 28: The distribution of answers for the question „I can use knowledge from Math and Science to design everyday

objects”

Almost half of the students (26 students out of 53) in pre-survey agreed or somewhat agreed with the

statement that they can use knowledge from Math and Science to design everyday objects. The

number of students with such opinion slightly increased to 30 students in post-survey.

Figure 29: The distribution of answers for the question „I usually do well in ICT class”

The number of students that usually do well in ICT class increased from 22 to 36 (26%).

1 1

6 5

10 10

25

22

11

15

0

5

10

15

20

25

30

Pre-test Post-test


2 1

810

17

12

20

24

6 6

0

5

10

15

20

25

30

Pre-test Post-test


6

3

9

5

16

9

16

27

6

9

0

5

10

15

20

25

30

Pre-test Post-test



20

Figure 30: The distribution of answers for the question „I enjoy exploring what makes machines work”

On average, 36 students out of 53 students stated that enjoy exploring what makes machines work


Figure 31: The distribution of answers for the question „I am good at building and fixing things”

Approximately 30 students agree or somewhat agree with the statement „I am good at building and

fixing things” (before and after the pilot).

2.3.4 21st century skills

The figures from 32 to 37 show the students' views on set of statements related to their confidence

in applying the 21st century skills. The results of comparing pre- and post-test indicate that the great

majority of students showed high levels of confidence in applying the skills and competencies

required in the 21st century. Detailed statistics are given below.

21

6 6

109

2021

1516

0

5

10

15

20

25

Pre-test Post-test


43

7

10

14

9

1820

1011

0

5

10

15

20

25

Pre-test Post-test



21

Figure 32: The distribution of answers for the question „I am confident that I can set goals and work towards achieving

them”

The vast majority of students (85%) agree or somewhat agree with the statement that they are

confident in setting goals and working towards achieving them (before and after the pilot).

Figure 33: The distribution of answers for the question „I am confident that I can work well in a team”

44 students out of 53 stated that they are confident that they can work well in a team (before and after

the pilot).

Figure 34: The distribution of answers for the question „I am confident I can include others’ perspectives when making

decisions”

1 2

85

3335

11 11

0

5

10

15

20

25

30

35

40

Pre-test Post-test


12 3

8

4

30

34

1311

0

5

10

15

20

25

30

35

40

Pre-test Post-test


134 4

31

26

16

20

0

5

10

15

20

25

30

35

Pre-test Post-test



22

47 students out of 53 stated that they are confident that they can include others’ perspectives when

making decisions (before and after the pilot).

Figure 35: The distribution of answers for the question „I am confident that I can find solutions when things do not go as

planned.”

43 students out of 53 stated that they are confident that they can find solutions when things do not

go as planned (before and after the pilot).

Figure 36: The distribution of answers for the question „I am confident that I can present my work in public and

communicate my ideas”

33 students out of 53 stated that they are confident that they can present their work in public and

communicate their ideas slightly (before and after the pilot).

2

118

3533

710

0

5

10

15

20

25

30

35

40

Pre-test Post-test


13

5

1013

8

30

26

46

0

5

10

15

20

25

30

35

Pre-test Post-test



23

Figure 37: The distribution of answers for the question „I am confident that I can take initiative to plan and execute tasks

that I have never performed before”

The number of students confident that they can take initiative to plan and execute tasks that they have

never performed before, increased from 27 to 32.

2.3.5 Arts

The figures from 38 to 40 show the students' views on set of statements related to Arts. The results

of comparing pre- and post-test indicate that the students are moderately positive- regarding the link

between Art and STEM. Detailed statistics are given below.

Figure 38: The distribution of answers for the question „Arts has nothing to do with Science, Technology, Engineering

and Maths”

About 9 out of 53 students agree or somewhat agree with the statement that Arts has nothing to do

with STEM subjects (before and after the pilot).

1

5

9

20

12

21

24

68

0

5

10

15

20

25

30

Pre-test Post-test


4

8

21

14

2021

6

9

21

0

5

10

15

20

25

Pre-test Post-test



24

Figure 39: The distribution of answers for the question „I feel motivated when through STEM subjects I am given

opportunities to participate in Arts activities”

The number of students that agreed or somewhat agreed with the statement „I feel motivated when

through STEM subjects I am given opportunities to participate in Arts activities” slightly increased

from 16 to 19.

Figure 40: The distribution of answers for the question „I am interested in exploring science concepts when combined

with Arts”

20 students out of 53 agree or somewhat agree with the statement „I am interested in exploring

science concepts when combined with Arts” (before and after the pilot).

2.4 Perceptions and future

The figures from 41 to 48 show the students' views on set of statements related to their perception of

STEM subjects and their plan for the future. The students were invited to state the extent to which

they agree or disagree on each statement using a 5-point Likert scale. The results of comparing pre-

and post-test indicate that the MakeITReal learning experience appears to have the potential to inspire

young students in making STEAM a future career choice. Detailed statistics are given below.

7

3

911

2120

1413

2

6

0

5

10

15

20

25

Pre-test Post-test


4 4

8 8

21 21

18

15

2

5

0

5

10

15

20

25

Pre-test Post-test



25

Figure 41: The distribution of answers for the question „I'm planning to work harder in my math and science courses”

45 students out of 53 reported that they are planning to work harder in their math and science courses


Figure 42: The distribution of answers for the question „Knowing how to use math and science together can allow me to

make useful and creative things”

46 students before the pilot and 39 students after the pilot agreed or somewhat agreed with the

statement that knowing how to use math and science together can allow them to make useful and

creative things.

Figure 43: The distribution of answers for the question „It is important for me to become knowledgeable in science and

science concepts”

3 2 34 4

24

2020

26

0

5

10

15

20

25

30

Pre-test Post-test


3 22 32

8

25

1921 20

0

5

10

15

20

25

30

Pre-test Post-test


3 2

57

2725

1719

0

5

10

15

20

25

30

Pre-test Post-test



26

44 students out of 53 reported that it is important for them to become knowledgeable in science and

science concepts (before and after the pilot).

Figure 44: The distribution of answers for the question „Knowledge of math and science can help me understand how the

world works”

About 44 students out of 53 agreed or somewhat agreed with the statement that knowledge of math

and science can help them understand how the world works (before and after the pilot).

Figure 45: The distribution of answers for the question „If I learn engineering, then I can improve things that people use

every day”

42 students out of 53 in pre-survey agreed or somewhat agreed with the statement „If I learn

engineering, then I can improve things that people use every day”. The number of students with such

opinion increased to 45 in post-survey.

1 11 2

6 7

25

16

19

27

0

5

10

15

20

25

30

Pre-test Post-test


1 2

57

3

222020

25

0

5

10

15

20

25

30

Pre-test Post-test



27

Figure 46: The distribution of answers for the question „I expect to use science and math when I get out of school”

On average 41 students out of 53 agreed or somewhat agreed with the statement that they expect to

use science and math when they get out of school (before and after the pilot).

Figure 47: The distribution of answers for the question „I would consider a job that involves using science and/or

maths”

The number of the Polish students that agreed or somewhat agreed with the statement that they would

consider a job that involves using science and/or maths, slightly increased from 36 to 39 students.

Figure 48: The distribution of answers for the question „My education will create many future opportunities for me”.

On average, 8344 students out of 53 agreed or somewhat agreed with the statement that their

education will create many future opportunities for them (before and after the pilot).

1 224

7 7

21

16

20

24

0

5

10

15

20

25

30

Pre-test Post-test


21

34

119

16

1920 20

0

5

10

15

20

25

Pre-test Post-test


42

5 6

16

21

27

24

0

5

10

15

20

25

30

Pre-test Post-test



28

Chapter 3: Feedback generated by the students

Students were encouraged to provide us with their feedback about their perception of the MakeITReal

workshop after they participated in it. They completed surveys (feedback form) consisted of eight

questions (both opened and closed). Unfortunately, not all the students who participated in the

workshops completed this part of the survey (32 out of 53). Because of the many open questions in

the survey, it was decided to conduct it online using the Google form, so that students have enough

time to complete. The questionnaire response rates appear to be lower in the case of an online

questionnaire than in the paper questionnaire. The above will be taken into account when planning

the evaluation of second round of the pilot.

The Figure 49 shows the description of the workshop in which the students participated. The students

were invited to choose adjectives that better describe MakeITReal intervention. From the figures it

is apparent that the workshop mostly was described as “creative” and “interesting”. Detailed statistics

on are given below.

Figure 49: Choosing adjectives that better describe the workshop experience


28 students out of 32 perceived workshop as “creative”, 25 students as “interesting” and 11 students

as “neutral”.

The students were most satisfied with the possibility of designing and printing new objects,

possibility of showing their creativity and final result of their work. They felt it was really valuable

to work with team on the projects. Students appreciated practical dimension of this work and

emphasized that it was a new experience for them. Some students appreciated skills and knowledge

acquired while working on projects that can be used in future work and in everyday life.

“I liked the most possibility to prove myself and print my first designed thing”

(The Polish student)

The student asked about the least favourite thing in the workshop, they mainly mentioned the long

print time of the objects, the small number of meetings during the project and problems with the

computers on which they designing the objects. Individuals also reported problems with FreeCad

errors, complained about not very clear instructions on how to use this program or small number of

meetings. It should be stressed that the large percentage of respondents did not indicate what they

would rate low in the workshop and many answered that they liked everything.

The participant students were asked what they learned at the MakeITReal workshop. The vast

majority of students believe that they gained knowledge about 3D modeling and printing, how works

10

1

2

11

7

28

1

3

25

Challenging

Difficult

Boring

Neutral

Fascinating

Creative

Easy

Fun

Interesting

0 5 10 15 20 25 30


29

3D printer and how works FreeCAD. Some of the students reported that they learned about the

benefits of 3D printer. Others thanks to the workshops learned how to work in a group.

“At the workshop, I learned 3D modeling and printing and FreeCAD

programming” (The Polish student)

On average, 8 out of 10 participants students would recommend the MakeITReal Workshop to their

colleagues. Only one person answered “Maybe No” when suggesting this workshop to her/him

friends and classmates. Detailed statistical data are shown in Figure 50.

Figure 50: The distribution of answers for the question “Would you propose this workshop to a friend/ classmate?”

In addition, about 7 out of 10 students are willing to participate in additional 3D projects. Only one

person had different opinion. Several students were not sure if they would like to take part in

additional activities. Detailed statistical data are shown in Figure 51.

Figure 51: The distribution of answers for the question „ Would you like to participate in additional projects like the

ones you were recently involved in?”

The students were asked about their suggestions for workshop improvement. Many students referred

that there is no need for improvements. The other responses centered around the following

suggestions:

More meetings

Organization of workshops not only on school grounds.

The students made the following suggestions that refer their ideas for objects that they would like to

3D model and print:

18

7

6

1

Yes

Maybe Yes

Not sure

Maybe No

11

11

7

1

Yes

Maybe Yes

Not sure

Maybe No


30

Missing small parts for machines

Dice

Sunglasses

Wheel rim

Pendant

Figures

Jewellery


31

Chapter 4: Feedback generated by the participant teachers

The survey with teacher was carried out at the end of the pilot. The participant teachers completed

online questionnaire which consisted mainly of closed questions and aimed to obtain information

about their experiences from the MakeITReal 3-months workshop.

At first, a WUT representative gave a presentation on the MakeITReal project at the school in two

classes. He put emphasis on creation of three dimensional objects, showed art crafts made with 3d

modeling/printing and explained the meaning of STEAM. As it turned out, 57 of the 61 students

expressed great interest in participating in the project. In the first pilot round, the teachers decided to

give the chance to participate in the workshop of all interested students. The school's management

agreed to carry out the workshops also during the lessons. Then for each interested student was filled

eligibility cards. The first part of thematic tables related to “Perceptions and attitudes towards STEM”

in eligibility cards were filled out by the students themselves and the other three tables were filled

by the teachers. In the first round of the pilot participated 53 students. Individuals resigned from the

MakeITReal workshop due to lack of time and little interest.

After the selection of the students, the teachers encourage them to work in teams on 3 projects namely

the “geometrical solids”, “the keychain” and “the penguin” project. 11 teams have been created.

According to the teachers, most of the teams completed the projects. The teachers spent on average

over 4 hours in each project. The duration of the 1st pilot phase (three months) was considered

adequate for carrying out these three projects The teachers were satisfied with the way this workshop

was carried out.

The teachers assessed students’ collaborations very well, but it was not the highest possible rating.

In their opinions the students enjoyed the workshop and were interested in.

The participant teachers were also asked about the main difficulties/challenges that students faced

during the implementation of the projects. The teacher mainly reported the challenges related to

technical difficulties with the selected software and with the 3D printer. These hardships had no

serious impact on the pilot implementation.

The participant teachers were also given a set of statements and they were invited to express the extent of their

agreement or disagreement on a five-point Likert scale. The statements are related to the impact that the workshop is

likely to have on students’ collaboration, problem solving and presentation skills, motivations towards STEAM,

performance in STEM, self-esteem and confidence. All statements were positive. The teachers appear more certain about

the development of skills (collaboration and presentation skills) and less certain about the adoption of a new attitude

(see Table 2 -

Table 9Błąd! Nie można odnaleźć źródła odwołania.). Furthermore all teachers agreed with the

statement that MakeITReal learning intervention can set a basis whereupon underachievement in

STEM can be tackled.

Table 2: The distribution of answers for the statement „ After this workshop it is likely that most of the students have

improved their collaboration skills”

Answer Count

Strongly agree 3

Agree 0

Neutral 0

Table 3: The distribution of answers for the statement „ After this workshop it is likely that most of the have improved

their presentation skills”

Answer Count


32

Strongly agree 3

Agree 0

Neutral 0


improved their problem solving skills”

Answer Count

Strongly agree 0

Agree 3

Neutral 0

Table 5: The distribution of answers for the statement „ After this workshop it is likely that most of the students are more

confident and ready to take new initiatives”

Answer Count

Strongly agree 0

Agree 3

Neutral 0


enhanced self-image, self-esteem”

Answer Count

Strongly agree 0

Agree 3

Neutral 0


been motivated towards STEM”

Answer Count

Strongly agree 0

Agree 3

Neutral 0

Table 8: The distribution of answers for the statement „ After this workshop it is likely that most of the students have a

more positive attitude for STEM subjects”

Answer Count

Strongly agree 0

Agree 3

Neutral 0


improved their performance in STEM”

Answer Count

Strongly agree 0

Agree 3

Neutral 0

The teachers agreed also that MakeITReal interventions can help linking STEM to creative, artistic

hands-on practises as well to real life and research/entrepreneurial practises.


33

Teachers were asked to express their views on educational resources provided to them, the support

from the project implementations offered by the team and necessary equipment to support students

during the 1st pilot period. The teachers reported that the educational resources were very useful (see

Table 10). The support provided by the project team was also rated as very useful (see Table 11).

The teachers agreed with the statement that they were well-equipped to support students during the

workshop (see Table 12), but they were not quite convinced in this matter. Probably affected by the

delay in purchasing a 3D printer due to administrative procedures. During this time, teachers could

count on support from the WUT worker, who shared his printer during the workshop.

Table 10: The distribution of answers for the statement „The educational resources that you were provided with were

…”

Answer Count

Very useful 3

Useful 0

Somewhat useful 0

Table 11: The distribution of answers for the statement „The support from the project implementation team was…”

Answer Count

Very useful 3

Useful 0

Somewhat useful 0

Table 12: The distribution of answers for the statement „I was well-equipped to support my students during the project

implementation stage…”

Answer Count

Strongly agree 0

Agree 3

Neutral 0

The participant teachers were also invited to express the extent to which they agree or disagree to

this statement: “The MakeITReal learning intervention offered me opportunities for professional

development and skill-building (both from a technical and a pedagogical perspective)” (see Table

13). Teachers agreed on this issue.

Table 13: The distribution of answers for the statement „ The MakeITReal learning intervention offered me opportunities

for professional development and skill-building (both from a technical and a pedagogical perspective)”

Answer Count

Strongly agree 0

Agree 3

Neutral 0

All the teachers are also confident that they can continue implementing the workshop in the school.


34

Chapter 5: Conclusion

In this interim report were presented findings from case studies in Poland describing the 1st round of

the learning intervention. Surveys were collected from teachers and students exploring their opinions

of the activities they had undertaken and the results of these were analysed to provide information

about their impact on students and teachers and to inform the 2nd MakeITReal pilot phase.

The survey evidence indicated that there was an increase in student interest in, and positive attitudes

toward STEM subjects, activities and careers following the MakeITReal intervention.

The results suggested that MakeITReal intervention may be effective in stimulating students interest

in, and positive attitudes toward STEM subjects, activities and careers. The whole learning

intervention was well received by the participant students and teachers. Students’ responses show

that the MakeITReal workshop set a basis whereupon students’ interest in STEAM disciplines can

be challenged.

Findings on the gain in STEAM perceptions after participating in the 1st pilot confirmed that students

came to see these disciplines as more appealing. Students reported an increase in their interest in

STEM, desire to learn STEM, their understanding of importance of STEM, and the connection

between STEM and real life. They are planning to work harder in their math and science courses.

The students realized that the things that they learn in Maths and Science matter in real and daily life.

In particularly, students appreciate STEM practical applicability in everyday life. This positive

impact of MakeITReal learning intervention on links STEM to real life has been noticed despite the

fact that students sometimes or rarely received such examples in schools. In addition, the participant

teachers agreed that that MakeITReal practices can help linking STEM to real life and

research/entrepreneurial practices. Real world problems are not isolated issues and often require

multi-disciplinary problem solving approaches drawing on findings from STEM disciplines.

Students stated that during the mathematic lessons the teachers sometimes or rarely included also

other subject areas or disciplines.

When it comes to the stimulation and motivation and the adoption of a more positive attitude for

STEM subjects, it appears that the participant teachers recognize that the MakeITReal learning

intervention facilitates this purpose. Almost all teachers agreed that MakeITReal intervention had

impact on students’ performance in STEM, but they were not quite convinced yet. Furthermore, most

teachers agreed with the statement that MakeITReal learning intervention can set a basis whereupon

underachievement in STEM can be tackled.

Based on the evaluation findings, the MakeITReal interventions appears to be successful in engaging

students in STEAM. Students asked for more sessions, which confirmed their great interest in the

workshops.

The main challenges, that were encountered, were related to occasional technical problems with the

software and 3D printer. However, none of the aforementioned challenges had a negative effect on

the deployment of the learning intervention.

The students were moderately positive about the link between Arts and STEM. In additions, they

were not sure about the impact of Arts on their motivation and interest in STEM subjects . However

the teachers agreed that that MakeITReal practices can help linking STEM to creative and artistic

hands-on practices. Students admitted that sometimes or more frequent were given the opportunity

to express or replenish their creativity, were allowed to use artistic methods to show their

understanding of a concept/event/object and were involved in hands-on and practical activities using

also math and/or science in schools. The students agreed with the statement “Knowing how to use

math and science together can allow me to make useful and creative things”.


35

The MakeITReal learning experience appears also to have the potential to inspire young students in

making STEAM a future career choice. The students agreed with the statement that their education

will create many future opportunities for them. In schools students sometimes or more frequent

talked about the types of jobs and careers that use Science and Math but they would consider a job

that involves using science and/or maths.

The students showed high levels of confidence in applying the skills and competencies required in

the 21st century. They appear to be more confident as far as their problem solving, decision making

and presentation and communication skills are concerned. Engaging students in product design and

making practices with the support of 3D modeling and 3D printing in a relatively short period of time

with limited resources helped them to improve their teamwork and creativity skills and gain

confidence in their ability to design, implement and analyse.

The teachers stress the fact that the MakeITReal workshop/pilot integrates indeed features that can

inspire most of the students in enhancing their skills (i.e. presentation skills, collaboration skills).

They appear more certain about the stimulation towards the development of new skills and less

certain about the actual adoption of a new attitude.

There was also an agreement that the MakeITReal learning intervention offered participant teachers

opportunities for professional development and skill-building (both from a technical and a

pedagogical perspective); towards this end, the support provided by the MakeITReal team was highly

acknowledged.

The current study found positive indications that engaging students in product design and making

practices following the Maker Movement trend promotes interest in STEAM. Nevertheless, it’s

worth to think how can we better forge the link between Arts and STEM?

MAKEITREAL PROJECTmakeitreal.info/wp-content/uploads/2017/10/Int...MAKEITREAL Project Erasmus+ KA2 2016-1-PL01-KA201-026492 4 Abstract In this interim report, we present findings from

Documents