Statistic report-bib satisfaction-internet version

Student Satisfaction with theInternational Business B.A. Programme

Vanessa Günther - K. Navi Phifer - Alina Sachapow--------------------------

3 February 2011Applied Statistics & Research Methods – Mr. Holger

Lütters – HTW Berlin

VS.

2

Outline

1. Press Release2. Management Summary

2.1 Preparation/Hypothesis2.2 Objectives2.3 Research Design2.4 Methodology2.5 Workpackages

3. Topic Relevance

3

Outline

4. Findings4.1 Students Expectations4.2 Best Subjects and Why4.3 Worst Subjects and Why4.4 Satisfaction and Education4.5 Satisfaction and Age4.6 Satisfaction and Job Experience4.7 Teachers´ Abilities4.8 Programme Criticism4.9 Programme Improvements4.10 Positive Remarks

4

Outline

5. Conclusion6. Challenges7. Criticism and Possible Improvement

7.1 Survey Design7.2 Question Formulation7.3 Process

5

1.Press Release

Teachers Held Largely Responsible for Student Satisfaction, concludes an independent study of Berlin’s College of Applied Science (HTW) students.

The push for international competitiveness is becoming stronger and more intense in Germany, forcing universities to rise up to the challenge in order to stay on top. The implementation of completely internationally oriented study programmes are met with a chorus of praise from both political and academic circles.

But just how effective are these programmes? By measuring student satisfaction, a study completed by HTW students explores the practical application of international courses at the HTW both in the classroom and in real life – and the results were quite unexpected: regardless of age or prior life experience, TEACHERS were mainly responsible for student satisfaction and the necessary skills they gained (or did not learn) for their future careers.

6

2. Management Summary

Overall, HTW Students were pleasantly satisfied with their BIB programme; much criticism for improvement was given, along with feedback that satisfaction with teachers played the largest role in satisfaction with the programme (the teachers‘ ability to teach, variety of material and methods of testing, practical application of subject, etc.)

Factors such as age and previously completed degrees of higher education, surprisingly played no conclusive role in determining student satisfaction with the HTW‘s BIB programme.

7

2.1 Preparation/Hypotheses

• Research Question: Are the students of the BIB programme satisfied?

• Hypotheses: • Students with job experience might have higher

expectations for the programme• Students who have studied at other universities

may have more specific (and possibly more demanding) expectations

• Age and therefore different life experiences could influence how satisfied people are with the programme

8

2.2 Objectives

Find out about:• Expectations of the students• Satisfaction with: • Fellow students• Course material• Subjects• Teachers • Administration

9

2.3 Research Design

• Conclusive Research Design:• Objective: Test Hypotheses• Characteristics: • Information necessary: Are BIB students satisfied? Why

and why not?• Interviewees: BIB students of all semesters currently

studying at HTW• Sample is representative -> about 20 students of every

semester covered

10

2.3 Research Design

• Research Process:• Develop a survey with all the important criteria to be

able to judge satisfaction• Make a Paper & Pencil and a online version to collect

primary data• Ask students in person with the Paper & Pencil version

to fill out the surveys• Forward the online version to the 7th semester

students• Evaluation and reporting of data

11

2.3 Research Design

• Findings:- Should be helpful to draw conclusions about the

programme and possible improvements of it- Reveal possible areas of improvement or clarification of

the research process itself (i.e. Does the survey accurately represent students’ opinions? Does the data clearly outline which areas of the program are working well or poorly?)

• Outcome:- Findings can hopefully be used to improve the programme

12

2.4 Methodology

• Data Collection:- Quantitative survey

• 18 questions- Questions about satisfaction or dissatisfaction- General information (age, gender etc)- All questions were not obligatory (some left blank)- 10 scale questions (poorly to well and sometimes vice versa)- 8 open end questions

• 2 versions: - Paper & Pencil (handed out in person )- Online via Questfox (sent through email & Facebook links →

impersonal)

13

2.5 Division of Workpackages

• Division of labour:Development of Questions

Creating Questfox

Preparation before

distribution

Distribution of Survey

Alina X X X

Navi X X X X

Vanessa X X X

14

2.5 Division of Workpackages

Typing in the data

PSPP & creating of statistical

tables, charts

Report & Evaluation of

results

Presentation

Alina X X X

Navi X X X X

Vanessa X X X

15

3. Topic Relevance

• Being students of this programme, we find this survey to be highly relevant!

• The idea of a satisfaction survey came to us because there was so much trouble last semester with an unqualified teacher (XXXX, in XXX Dept.). The third semester students and their foreign exchange classmates rallied together to petition the unfair results of the exam.

16

3. Topic Relevance

• Now this semester, we are still hearing some complaints about teachers or subjects and so we thought, „Are students truly unhappy? Or are they just ready to complain about any subject they don‘t like or any class which is difficult?“

• Does student satisfaction even have any merit? Do students know what they‘ll need for their future careers, or do they only know how to write exams and memorize formulas? This survey strives to remove the subjective „I hate math“ and put forth an objective reasoning for students‘ expectations and satisfaction with the programme.

17

4. Findings• Over 25 countries were represented

• Twelve people chose to withhold their nationality

• Roughly 63% of the BIB students (58 of 92 responses given) are of German nationality

• Approximately one-fifth of BIB students are from developing countries

18

4. FindingsGender of HTW Students Surveyed

• Number of Respondants:

• Female – 64• Male – 29• „Uncertain“ – 3• No Answer – 8

19

4. FindingsStudents Surveyed and Their Respective Semester

Semester0

5

10

15

20

25

30

35

No Answer12357

No.

of S

tude

nt R

espo

nses

20

4.1 Student Expectations

Top 5 Expectations of the programme:

1. English Language (44 votes)2. Internationality & Business topics (27 votes each)3. Good career opportunities (16 votes)4. Location Berlin (12 votes)5. Experience abroad (11 votes)

21


Our hypotheses: After thinking more thoroughly about their expectations and satisfaction, students would change their opinion (namely, that they would be less satisfied).

TRUE

22


Question #2: Overall, how well have your expectations for this programme been met?

Questions #11: Generally speaking, what is your level of satisfaction with your study programme?

23


2nd Time Asked

1st Time Asked

0 5 10 15 20 25 30 35 40

6 - Well54321 - Poorly

Student Satisfaction with the HTW BIB Programme

24

4.2 Best Subjects and Why

Best subjects:

– Macroeconomics with Professor XXX (26 votes)– Microeconomics with Professor XXX (23 votes)– Business Law 1 with Professor XXX (18 votes)– Introduction to Business with XXX (12 votes)– Financial Accounting with Professor XXX (8 votes)

25

4.2 Best Subjects and Why

• Reason for best subjectsSubject teacher Other reason

Economics Fair, organized, great language skills, theory and practice well combined, very good teacher, huge efforts to explain content, good material, always prepared, always able to answer, competent, motivated structured, transparent and fair assessment, friendly, helpful, understandable

interesting subject, good participation opportunities, Important for future, plenty of examples

Business Law 1 Competent, great knowledge, knows how to teach, cares about students, able to teach dry topics in an amazingly interesting manner

relevant for future career, challenging

Introduction to business

good teacher, interesting subject, good teaching style, cool, creative

interesting, most interactive, relevant for future career, not only theoretical

Financial Accounting

Good explanation, good structure, good teaching style, stays in timetable, good speed

26

4.3 Worst Subjects and Why

• Worst subjects:– Research Methods with Professor XXX (12 votes)– Marketing with XXX (11 votes)– Mathematics with XXX (9 votes)– Mathematics with Professor XXX and Corporate

Finance with XXX (each 7 votes)– Communication Skills with Professor XXX (6 votes)

27

4.3 Worst Subjects and Why

• Reasons for worst subjects Subject Teacher Other Reason

Research Methods Bad teacher, does not know what he is talking about, no guidance, unjust, incomprehensible grading

Nothing learnt, irrelevant, information partly only in German available

Marketing Confused, no knowledge about subject, could not answer questions, unorganized, unfair, learnt nothing, subjective evaluation of students, no interest in student’s progress, unclear expectations, uncomfortable atmosphere

28

4.3 Worst Subjects and WhySubject Teacher Other Reason

Mathematics Does not explain well, very impatient

Mathematics Professor not able to explain, not able to work with class, not able to catch attention

too demanding, very difficult, very fast

Corporate Finance Bad language skills, little knowledge about subject, can’t answer questions, doesn’t pay attention to presentations, speaks only Russian

Communication Skills Bad structure, bad feedback, poor language skills

very theoretical, absolutely unnecessary and boring

29

4.4 Satisfaction and Education

Our hypothesis: Students who have completed study programmes (either Bachelor‘s or Master‘s degrees) are less satisfied with the programme than students with no prior university experiences (since they with no experience have nothing to compare with).

30

4.4 Satisfaction and Education

Number of students (frequency) who have completed Bachelor‘s degree:

5

Number of students who have completed Master‘s degree: 2

Total number of students surveyed (N)*: 104

Relative frequency of students with degrees of higher education (Baachelor‘s & Master‘s):

0.067

The relative frequency is not large enough to provide an accurate, unbiased answer to our hypothesis.

*We used the total number of students rather than those who marked „BA, MA, school,“ because logically ALL students must have attended school and received their A-levels before being admitted to the HTW – despite the fact that only 63 students marked „school“.

31

4.5 Satisfaction and Age

Our hypothesis: Age and therefore different life experiences could influence how satisfied people are with the programme.

Conclusion: too difficult to measure

• Students ages 20 – 26 nicely dispersed / represented• Only one student age 30• No students older than 30

32

4.5 Satisfaction and Age

33

4.6 Satisfaction and Job Experience

Our hypothesis: Students with job experience might have higher expectations for the programme and thus a resulting lower level of satisfaction.

(slightly) TRUE

• 32 Students had part-time work experience• 34 Students had full-time work experience• Some marked both full and part-time• Some marked multiple options other than work

34

4.6 Satisfaction and Job Experience

1 - Poorly 2 3 4 5 6 - Well0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%No Work Experience

(Full & Part-time) Work Experience

Phifer, Sachapow, Günther 2011

Total responses without work experience: 126Total responses of full and part-time work: 66

Respondants ticking more than one option (i.e. „international experience“ and „school“) are counted multiple times

HTW BIB Student Satisfaction

35

4.7 Teachers´Abilities

Are students whose mother tongue is English more or less satisfied with their teachers‘ language abilities?

Surprisingly, German students held a more negative opinion of their teachers‘ English abilities. This could be due to the fact that most English-speaking students come from African countries, or that most of the native English speakers are in the first and second semester. Or perhaps English-speaking students are just nicer.

36


GermanEnglish

Other

0

5

10

15

20

25

2 34 56 - Bad enough to make you cry 1 - Fantastic

Number of Respondents(frequency)

Bilingualresonpentswere countedtwice

HTW BIB Student Satisfaction with Teachers‘ Language Abilities & Mother Tongue

37


38

4.8 Programme Criticism

Quality of teachers• There should be a better quality of teachers and • Higher entrance reuqirements for teachers • More professors would be nice, not only teachers• Better evaluation of teachers• Teachers should be obliged to make TOEFL too • Some teachers as in Introduction to Business are

too unorganized

39


Subjects• Some topics feel like a waste of time• Too much learning by heart for exams• There should be a broader variety of languages• Although 2/3 of the course selected HR, XXX failed to

manage that everyone who wanted gets a HR topic. There were too many Marketing topics and too few HR topics

• Learning target is often not transparent, seems like only filling out the time

• Some subjects as research Methods 1 are useless, one doesn’t learn anything

40


Administration

• There was never the chance to complain anywhere, because nobody wanted to help students, „they“ always just tell us how great they are

• Students should be better informed about AWE, everyone has different information on that

• Unorganized, uncoordinated, nobody takes responsibility, people in charge are not approachable

• In the administration nobody has really a clue of how the system is working and if one needs certain information, one has to talk to many people and does not know more in the end.

41


Administration• Mrs. XXX is rude • XXX is not able to argue on a high level with good arguments, he

cannot communicate

General Criticism• Some students hate the comparison of their semester to other

semesters• Student was not able to get credits from his previous study

program for this course because of the language, but they think that those classes were useless here and they did not learn anything

42

4.9 Programme Improvements

Things that should be improved

• It is not helpful to be forced to do presentations in EVERY class, sometimes it is better to get something explained by the teacher; in the end one has to double and triple check because the presentations are not sufficient, sometimes those presentations take too much time of the class, which could have been used better. If presentations still necessary, then it should rather be done like in economics where professor XXX has his actual lectures and lets students present on related topics so that everybody can understand what is done.

43


• In the 4th semester there should be more options, not only marketing and HR

• There are very good and very bad teachers, that is why it is very difficult to assess the program

• Adding and elective C „finance“ in the 5th and 7th semester would be really interesting

• Having some more online classes would be nice• Some more events to encourage the interaction between

Germans and international students would be nice• There is someone needed in mathematics who pushes the

students practically

44


• Would be nice to learn more about ethics and about criticism of capitalism

• More scholarships should be available for good students to be able to concentrate more on their studies

• More co-work of students from the first and higher semesters appreciated

• Would be nice to be able to select some courses which are relevant for specialization and skip some irrelevant courses as regional studies

• Selection of courses should be a little more flexible (although hard because of the English language)-> would like to take some more courses from other disciplines as law

45

4.10 Positive Remarks

• It is getting better each year• Ms. XXX is a real enrichment for the programme, she

does her job really well, although there are many incompetent people around and above her

• The staff in the international office is very competent and friendly

• Great program, much practice, not too theoretical• It is a nice study program due to its internationality• It is nice that the university tries to make everything

online• Some teachers are very good

46

5. Conclusion

• Overall satisfaction was good• Students mostly complained about poor teaching quality (inability to

explain, moving too quickly) and subjects which were „pointless“ and „useless“ → graphs still look quite positive because people had no choice but to choose the average quality

• [What to change, if we could go back in time] We should have had one hypothesis and based the survey around it (more in-depth)

• [What to change, if doing another survey in the future ] Students felt much more comfortable giving feedback and criticism to us in person (perhaps easier to say than to write, or it felt more „off the record“)... Maybe it is better to do personal interviews (qualitative instead of quantitative surveys), especially when students care about their programme and the response rate would be high

47

5. Conclusion• Evaluation of best and worst subjects is difficult

→ subjects from 5th and 7th semesters have less chances to be chosen due to the fact that less people attended these classes so far

(e.g. Business Ethics)→ sometimes we just didn't know which teacher taught when...(e.g. if a seventh semester student says that they liked marketing most, that is hard to find out which teacher they had AND it is also not really comparable to later marketing classes because of the different teacher)

• Hard to judge quality of the whole programme → too few students who went through all of it (7th semester or alumnis)→ results concerning the level of satisfaction would be completely different if

there would have been more higher semester students or alumnis been answering (the programme is very new and faced a lot of problems in the early stages; many people worked hard on the improvement of it already)

• Cannot form direct linkage between work experience and satisfaction with our data (needs to be more precise and detailed in certain areas)

48

6. Challenges

• Finding time for everyone in the group to meet (different schedules, living in different locations)

• Limited number of questions– Wanted to keep survey as short as possible (should fit on

one piece of paper front and back)• Accurate representation– Limited quantity of data because there is no 4th and 6th

semester at the moment and the alumnis are not represented

– 7th semester students had a different version of the survey (online); the different environment could have impacted their answers

49

6. Challenges• Timing of survey– Survey done before exam time (students are stressed or scared

to criticize teachers or won’t criticize teachers until their marks are received)

• Topic too broad– Satisfaction is hard to judge because there are many different

factors influencing it– More detail is required to be precise about satisfaction and

possible bias of the students… but number of questions and time were limited

• Learning PSPP & Questfox• Navi and the American way of thinking

50

7. Criticism

• Survey design (Screenshots of the online survey on the next couple of slides)• Question formulation• Process

51

The Online Survey

52

53

54

55

56

7.1 Survey Layout discussion

In case we would have done the comparison:

• Putting the logo of the HTW on the front page would make the survey look more professional

→ but when we originally wantetd to distribute the same surveys to HWR students, they could have felt that this survey has a competitive character and students could be afraid of answering honestly ( nobody would want to downgrade his programme)

• We then thought about including both logos on the front page → but that would still cause competition between the two.

• So our conclusion was, that the only way of receiving honest answers would be to have two different layouts: a HTW-survey with HTW logo and another one for the HWR with their logo on it.

57

7.1 Evaluation of Survey Design

• Paper & Pencil+ Must not be filled out sequentially+ Possibly higher response rate+ Verbal feedback

- Illegible handwriting- Not all questions answered

- It is necessary to type up all of the questions- Some students did not attend the classes

when we distributed the survey

58


• Online+ Link forwarded to 7th semester students → (otherwise they could not have been reached)+ Easier data compilation (Questfox)+ 0-drop-out rate from online survey

- Not in school environment- Easier to just “click away”- Possible fraud - Time consuming to create an online survey

additionally

59


Of both forms, overall...• High flexibility of data collection because two

different versions• Stimuli used (only personal): Distribution of

sweets, opportunity to improve the programme (chance that results will be published)

• Little control over the environment (only possible in choosing different courses where to ask the students)

60

7.1 Evaluation of Survey DesignSummarized

• Response rate (personal)- High (because they could not run away!)- High completion rate (100% except a few answers left blank)

• Response rate (online):- Higher than expected, 100% completion (no drop-outs)

• Anonymity: - No student ID or name

• Speed:- It took about 15 minutes to distribute, let it be filled out and

collecting it in every class• Bias towards interviewers:

- Little because we are also BIB students• Costs:

- Only printing costs

61

7.2 Question Formulation

• Teacher evaluation questions were too general because there are very good and very bad teachers, so the average will always be chosen

• One suggestion was that we should make the questions a bit longer and more complex: e.g.:„what percentage of your teachers would you rate as being: (1) highly competent (2) competent (3) average (4) difficult to understand (5) not helpful at all

62


• The mother tongue could play an important role in students‘ understanding of the lectures, but some felt that it singled them out and invalidated their anonymity

• We should have had listed the mother tongue options as „German, English, or other“ - to make sure that anonymity is given

• When we asked about the age, people also were concerned about their anonymity→We should have used age brackets instead to avoid

this from happening (age 20 and under, ages 21 – 24, ages 25 – 28, etc.)

63


• When we asked about the age, people also were concerned about their anonymity

→We should have used age brackets instead to avoid this from happening (age 20 and under, ages 21 – 24, ages 25 – 28, etc.)

• In some questions our data might not be accurate because software like questfox and pspp cannot detect spelling mistakes → (e.g. nationality,…) we could have offered again options to tick the box instead

• We should have been more precise e.g. when choosing best or worst subject we should have asked the students to include the name of the teachers of the class because it is hard to find out which teacher had which class

64


• When people had to tick what they did before studying at HTW in order to find out, if that might have influenced their expectations and therefore their level of satisfaction with the programme

→ we did not offer enough choices like Diploma or studies that were not finished → people had to cross other, eventhough they might be highly qualified in some other fields of study

• We should have offered a box like some colleges to also be able to recognize these kinds of qualifications

65

7.3 Process

• We did not get any answer from the HWR → we should have contacted them much earlier (as soon as we decided on the topic)

• Dependent on other lecturers in terms of reaching them and to get permission to run surveys during their classes → we should have started to prepare this earlier

• Better organisation/structure (complete Questfox before starting to type information into it)

66

Thank You

HTW-BerlinCollege of Applied Sciences, Berlin

Kasey Navita Phiferwww.kNavita.com

Statistic report-bib satisfaction-internet version

Education

students satisfaction

programme students

htw students

bib students

students expectations4

students trul

students opinions

students of thebib programme