TR/05/B/F/ PP 178-009 “INTERNAL EVALUATION FINAL REPORT ” Prof. Lorenzo Fiorito.

TR/05/B/F/ PP 178-009

“INTERNAL EVALUATION FINAL REPORT ”

Prof. Lorenzo Fiorito

Specific aims of the project• To develop innovative approaches in Technology Education,• to adapt the content and supply of technology education to

suit new qualification needs through cooperation between educational and training bodies,

• to create a set of standards for European Technology Education Qualifications to be transparent and recognised at all levels and in all countries concerned,

• to develop new high quality curricula and training materials, • to realise international cooperation between educational and

training establishments and enterprises on national and European basis

• to enhance employment standards of Technology Education graduates and make them competitive in the job market by flexible training system and modules

• opening up Technology Education systems to a wider part of the world.

Expected results

1. Design of new curricula for Technology Education

2. New training material for the for Technology Education

3. Model training tools

4. Contact database of the national bodies, vocational schools, professional institutions and educational institutions regarding Technology Education.

5. Country reports about the current status and training/new qualification needs of Technology Education in partner countries

6. Printed copies of the project activities and results (in a form of handbook, in English

Workplan

The project is sub-divided into 8 work packages

1 Project Management and Coordination

2 Design of web based data acquisition medium-Set up database - Continuous update

3 Background analysis of technology education programs – Identification of current training needs and new qualification needs

4 Development of methodology

5Development of training modules-Determination of national exceptions - Integration of work related training and ICT

6 Development of training material and tools

7 Feedback and Evaluation

8 Valorisation

Time schedule of the project work plan (revised)

WPI

M1                                              

II                                                

III   S1        C1

IV                S2

V                      M2

   S3

VI                              M3

 S4

VII                                    M4

VIII                                        C1

11 12 1 2 3 4 5 6 7 8 910

11

12

1 2 3 4 5 6 7 8 9 1011

20052006

2007  

Approach to evaluation

• The evaluation is intended to address the following questions:

• Did the outcomes meet the original objectives?• Analyse the differences between outcome and

objectives, identifying factors that have contributed to those differences.

• Evaluation of the strengths and improvements and make recommendations concerning good practice.

• To evaluate the productivity and tasks done in the project.

WP1 Project management and Coordination

• Effectiveness of communication system (extent to which partners were updated on project developments, efficiency of information flow, WP reports submitted in time, prompt response to problems/issues arising)

• Effectiveness of finance and administration systems (expenditure tracked efficiently, data collated and circulated regularly)

• Effectiveness of decision-making/planning process (evidence that workplan is flexible/adaptable to changing needs as the project develops, regular review of plans, equal participation of all partners in decision-making, documentation of decisions)

Performance indicators

WP1 Achieved results Project management and Coordination

• Establishment of project• Roles and work plan accepted• Communication channels established• Evaluation methods decided• Dissemination plan agreed• Basic concepts relevant to the development of

the common programme clarified

Organisational Structure

Czech Rep.

Germany

Italy

Bulgaria

Spain

Greece

Turkey

STEERING GROUP

PARTNERSHIPGROUP

WP1 Achieved results Project management and Coordination

Comments• Partnership agreement signed• Project leadership was agreed, with a responsible project

leader, a project management group• Sharing of duties through the splitting of the total project into

work packages (WP). Each WP was allocated a certain amount of work, shared between partners.

• Finance: Each partner was given responsibility for accounting, reporting and documenting their own expenses within the defined framework.

• Ownership and rights to the outcomes of the project, the IPR and other rights for joint products.

WP1 Achieved results Project management and Coordination

Kick off and Agreement: November 2005

WP1 Achieved results WP1: Communication and management

• On a weekly basis the Contractor informed the Partners about the latest developments and issues to be discussed within the project via e-mail. Activities and announcements have been located to the “partner announcement section of the project web-site and the partners are automatically informed via. e-mail about the location of the new announcement.

WP1 Achieved results WP1: Project management

Steering Committee Meetings

Meeting No

Type Venue Date Related Work Package

1 Steering Committee

Sofia, Bulgaria November 2005

WP1

2 Steering Committee

Naples, Italy October 2006

WP5

3 Steering Committee

Seville, Spain February 2007

WP6

4 Steering Committee

Prague Czech Republic

June 2007 WP7

WP1 Achieved results Project management

Conferences

Conference No

Type Venue Schedule Included Work

Package

1 Conference Greece June 2006 WP2

2 ValorisationConference

Ankara, Turkey

November 2007

WP8

WP2: Web site and contact database Performance Indicators

• Setting up project main web-site and sub-sites

• Effective data acquisition tools within the interactive web medium

• Instant and continuous monitoring of the results

• Effective internal and external monitoring of the results

WP2 achieved results:

Website; Completed: December 2005.

Country report database:added March 2006

• Comments: The webpage (www.modularte.gazi.edu.tr) created at the start of the project worked both as a communication tool with partners and stakeholders (for valorisation purposes), and a working platform of the project.

• In addition to the public area, a password-protected area on the web allowed the partners to access documents, reports, and the database.

• The web-site is built in a dynamic way and all databases are included within the site.

• Both public and private areas were constantly monitored

WP3: Web site and contact database Performance Indicators

• Positive feedback on usability • Varied sample of target group in partner

countries for contact database• Varied distributions of the questionnaires

in each partner country • Exhaustive reports about the status of

technology education in partner countries • Complete and on time final reports of the

partners on the overall WP tasks

WP3: Achieved resultsContact database

Completed: January/February 2006

Comments

• All partner countries created a contact database of the national bodies, vocational schools, professional institutions, educational institutions and enterprises related with technology education. Entries amount to a total of approx. 600.

WP3 Achieved results (2) Country Report Database Content:

Completed: June 2006 (Presented at the Conference in Larissa)

Comments The database is exhaustive as for:• Schools where technology education is taught• Units and applications regarding the technology

education or equivalent courses. • The National Educational system chart of the

partner countriesand relevant legislations • Current status of the Technology Education or

equivalent teacher training system in the partner countries

WP3 Achieved results Survey on Qualification needs of T. E.

Completed: June 2006 (presented at the conference in Larissa)

Comments:

• The 7 Questionnaires on the contact database of each

country aimed to analyse the situtation in :

Technology Education in Schools Technology Education

Teacher and Teacher Training System (teacher, teacher

training, programme) were well undesrtood and used

The rate of Questionnaire responses on overall contacts is

in the average of 50%

WP4 Development of methodology Performance Indicators

• Establishing a Common European work description of technology education

• Final report on methodology to be a basis for the development of training modules

• Clear tasks fro each partner to be accomplished according to key competences in the partner countries to be acceptable in European dimension.

WP4 Achieved resultsDevelopment of methodology

Concluded: October 2006 (Meeting in Naples)

• Establishment of agreed common requirements and description of five main modules

• Main Module 1 – Fundamentals • Main Module 2 – Systems and Processes• Main Module 3 – Information and

Communication• Main Module 4 – Sociotechnology• Main Module 5 – Didactics of technology Tasks assigned to each partner for developing

modules according to their competences

• Agreement on contents and structure of each module (both compulsory and elective)

• Translation of the all module contents into native languages including the national exceptions

WP5 Development of training modulesPerformance indicators

Main outcomes were:• Presentation of Main Modules• Overall Module System Partnership

Discussion, Changes and Revisions, Fixing of the Credits

• Structure of modules and Subject Oriented Areas for each lecture and on exemplary teaching materials

• Translations in native languages

WP5 Achieved resultsDevelopment of training modules

First part presented in Febr. 2007 (Meeting in Seville)Mostly Completed in June 2007 (Meeting in Prague)

some additional tasks completed in July

• Effective preview and usability of learning tools on website

• Effectiveness of the tools reviewed via identification of strengths and weaknesses via feedback from target group (WP7)

WP6 Development of training tools Performance indicators

• The learning tools were situated on the website: moreover, the interactive CD-ROMs containing the detailed training curricula and the training material of specified basic module were finalised.

• These were reviewed during evaluation workshops in each partner country (WP7): to date the response is positive

WP6 Achieved resultsDevelopment of training tools

Completed: July –September 2007

• Feedback from the target groups (evaluated on a national basis)

• Completed Internal evaluation report about the overall evaluation of the feedback

• External evaluation about the project activities and products Evaluation (from external experts of a European level)

WP7 Feedback and evaluationPerformance indicators

• External evaluation – Started in July. • Internal evaluation – Promoter deals with the

monitoring but the evaluation of the productivity and tasks done in the project was given as a task to the Italian partner.

• Feedbacks was obtained from target groups: teachers, academicians, students.

• The evaluation sheet was prepared by the Italian partner and sent to all partners, who organised workshops in October.

WP7 Achieved resultsFeedback and evaluation

Completed: November 2007

WP8 ValorisationPerformance Indicators

• Dissemination of materials and information to a range of relevant audiences and via a variety of appropriate methods/media at partner country and transnational level

• Organisation of events with participation of target groups and policy makers

• Effectiveness of the dissemination process in encouraging usage of learning materials and tools, also contributing to mainstreaming and sustainability (ex-post)

WP8 Achieved results

Valorisation activities- Started Nov. 2005 – ongoing

1. The main valorisation tool is the project web-site. All activities conducted within the project are either done by means of the project web-site or transferred to the web-site.

2. Press releases in local newspapers in partner countries: number of press releases within the partners countries: Turkey: 5 - Czech Republic: 3 - Spain: 1 - Greece: 2 Bulgaria: 2- Germany: 4 - Italy: 1

3. A summary of the project in each partner’s web-site

4. Local and national, workshops

5. Valorisation conferences, with the partecipation of policy makers, stakeholders and target groups

To end on:To end on:The opinion of the The opinion of the

partners partners MODULARTE MODULARTE

regarding the regarding the Overall ProcessOverall Process

A questionnaire was submitted to all partners in order to get their opinion concerning the different areas of the process

Partners’ Questionnaire

Not agree Partly agree

Agree Completely agree

The tasks assigned to your institution are not appropriate for the competence of your staff

Some of the tasks assigned to your institution are appropriate for the competence of your staff but something need to be changed

The tasks assigned to your institution are appropriate for the competence of your staff

The tasks assigned to your institution are appropriate for the competence of your staff you wouldn’t change anything

Appropriateness of the tasks assigned to each institution: did the participants collectively constitute a consortium of high quality? Were the participants well-suited and committed to the tasks assigned to them?

Poor Intermediate

Good Very good

the quality of the management is deficient

some aspects work but some others need to be changed

the quality of management is satisfactory

the quality of management is satisfactory and nothing need to be changed

Evaluation of the quality of the management: was the organisational structure well matched to the

complexity of the project and to the degree of integration required? Was the project management

demonstrably of high quality?

oThe rules for project activities implementation were clear, precise and acceptable for you.oThe tasks assigned for your institution corresponded to the project’s aims.oThe tasks assigned to your institution corresponded to the competence of your staff.oHow would you rate the quality and effectiveness of the project management?oThe results of the country report proved useful for the project’s aims

Which items we evaluated

oHow would you rate the effectiveness of the course evaluation in your country?oHow do you rate the effectiveness of the communication between partners during the life of the project?oHow would you rate the quality and effectiveness of the of the project web site. oHow would you rate the dissemination/valorisation events carried out so far by your organisation?oHow would you rate the meetings as for the results and the socialisation between the partners?

Which items we evaluated (2)

The highest scores were for: How would you rate the quality and effectiveness of the project management? (4)

The lowest scores were for :How do you rate the effectiveness of the communication between partners during the life of the project (3)•The tasks assigned for your institution corresponded to the project’s aims (3.2)

The overall opinion about the project process was positive.

•The partners judged the quality management very good•They were satisfied with the adequacy of tasks assigned to their institution and the tools for the ongoing of the project, such as the project website, rated effective, and the dissemination events, valued as fruitful. •The same opinion was valid for the meetings, helpful as for the results and the socialization and the rules for project activities implementation rated clear, precise and acceptable enough. •Some doubts were expressed about the effectiveness of the communication between partners during the life of the project and the correspondence of the tasks assigned to their institution for the project’s aims.

CLOSING REMARKS

1)Result orientation:The aims of the project were clear

2) Focus on beneficiaries:Always present. Too early to forecast impact on target groups

3) Partnership: Acknowledged by the project partners

4) People involvement: Generally high throughout the project

5) Leadership: Added value

CLOSING REMARKS

6) Strategy: Sound and well fitted

7)Processes: WPs aligned on the basis of a dynamic work model

8) People results: Good network to be sustained

9 )Impact: The project is liable to induce broader impact exceeding original aims of time, targets and partners through further investments to build on achievements and results

Thank you