Top Banner
EDUCATION EVALUATION CENTER MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 1 MECS EEC WORLD BANK RURAL EDUCATION AND DEVELOPMENT PROJECT MONGOLIAN NATIONAL ASSESSMENT OF PRIMARY EDUCATION MATHEMATICS AND READING Ulaanbaatar city 2008
138

Bizdata.mn Report Eec

Oct 14, 2014

Download

Documents

www.Bizdata.mn BUSINESS DATABASE OF MONGOLIA
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 1

MECS EEC WORLD BANK

RURAL EDUCATION AND DEVELOPMENT PROJECT

MONGOLIAN NATIONAL ASSESSMENT OF

PRIMARY EDUCATION MATHEMATICS AND READING

Ulaanbaatar city

2008

Page 2: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 2

EDUCATION EVALUATION CENTER OF MONGOLIA

MONGOLIAN NATIONAL ASSESSMENT OF PRIMARY EDUCATION MATHEMATICS AND

READING

Ulaanbaatar city

2008

Page 3: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 3

CONTENT

i. Preface ii. Brief conclusions of the National assessment of mathematics and reading

CHAPTER 1. INTRODUCTION OF NATIONAL ASSESSMENT OF MATHEMATICS AND READING

Ts. Unurbayan, Ts. Erdenetsetseg, M. Boldsaikhan, D. Erdene, S. Bat-Erdene, D. Altangerel, D. Ganbold, O. Munkhchandman

1.1 Introduction 1.2 Needs of the assessment 1.3 Administration of the national assessment 1.4 Target population of the national assessment 1.5 Research methodology 1.6 Items and questionnaires development methodology 1.7 Booklets design 1.8 Item scoring method

CHAPTER 2. FINDINGS AND RESULTS OF THE NATIONAL ASSESSMENT OF MATEMATICS AND

READING

D. Monkhor, Ts. Unurbayan, Kelvin Gregory, Ts. Erdenetsetseg, M. Boldsaikhan, M. Ganbat, D. Erdene

2.1 Summary of the findings and results of the national assessment 2.2 Findings and Results of Mathematics 2.3 Findings and Results of Reading 2.4 Results of study in mathematics and reading of Kazakh students 2.5 Items analysis in mathematics 2.6 Items analysis in reading 2.7 Skills levels in mathematics 2.8 Benchmarks in mathematics 2.9 Skills levels in reading 2.10 Benchmarks in reading 2.11 Performance of international items

CHAPTER 3. FACTORS ASSOCIATED WITH LEARNING ACHIEVEMENT

Ts. Erdenetsetseg, M. Boldsaikhan, J. Bileg-Orshikh, M. Gantuya

3.1 Factors associated with location and population 3.2 Factors associated with learning environment (external and internal) 3.3 Other factors having impact on mathematics and reading

CHAPTER 4. METHODOLOGY AND APPROACHES OF THE NATIONAL ASSESSMENT

M. Boldsaikhan, Ts. Oyunsuren, Kelvin Gregory

4.1 Sampling 4.2 Methodology of the survey 4.3 Creation of information databases, software programs applied 4.4 Training seminars and workshops

Page 4: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 4

4.5 Quality control/monitoring

CHAPTER 5. CONCLUSIONS AND RECOMMENDATIONS

Ts. Unurbayan, D. Monkhor, Ts. Oyunsaikhan, Ts. Erdenetsetseg, M. Boldsaikhan

5.1 Learning achievement of maths and reading 5.2 What are influencing learning achievement? 5.3 How learning achievement can be improved? 5.4 Recommendations

CHAPTER 6. ANNEXES

6.1 Research team of members 6.2 Glossary 6.3 References 6.4 Web 6.5 Photos

Page 5: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 5

ABBREVIATIONS

DPSE - Department of Primary and Secondary Education

MECS – Ministry of Education, Culture and Science

EEC – Education Evaluation Center

SSE – School of Secondary Education

NITP - National Information Technology Park

SOLO taxonomy - Structure of Observed Learning Outcomes1

PSE- Preschool education

READ - Rural Education and Development Project

UNICEF - The United Nations Children’s Fund

UNESCO - United Nations Education, Scientific and Cultural Organization

IEA - The International Association for the Evaluation of Educational Achievement

IRT- Item Response Theory

OECD - Organization for Economic Cooperation and Development

PIRLS - Progress in International Reading Literacy Study

PISA- Program for International Student Assessment

SPSS - Statistical Package for Social Sciences

TIMSS - Trends in International Mathematics and Science Study

1 See glossary of terms

Page 6: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 6

i. PREFACE

The World Bank has been providing continuous financial and various kinds of support and assistance towards enlightening and educating the children of Mongolia. From 2007 to 2009, a substantial part of the assistance provided within the scope of the “Rural Education and Development” project implemented by MECS, with funding provided by the World Bank, has been presented to the Education Evaluation Center (EEC). The EEC has conducted the national assessment with two stages during the period of 2004-2005. Based on the experience of organizing the national survey on mathematics and science lessons at the primary education level, the purpose of the study was to utilize the findings to improve the quality of study. In the study, 8,553 individuals were included in the questionnaire, and 5,920 students completed the tasks. For the study, 6,550 surveying booklets were printed; an information database was created to be disseminated to the public. This was carried out with over 250 teacher’s hard work and effort. By conducting the study, we were able to determine the influences of the internal and external factors of learning, the general population, residential situations, and conditions to study according to internationally accepted methodologies of mathematics and reading for primary education students nationwide.

The purpose of the study was to increase the technical capacity, enhance the capacity of human resources to adjust our country’s characteristics to a new method of classifying the example items by the levels of assessment standard, benchmarking, scoring, creating questions, sampling method, and software for assessment of practical works of learning achievement in the future.

The “National Assessment of Primary Education Mathematics and Reading” was aimed at identifying levels of mathematics and reading and reveal factors that influence these skills, and furthermore, to use study findings in developing policies. This report conveys how the study has been administered nationwide, its findings, and recommendations.

Finally, I would like to extend my deepest gratitude for the provision of invaluable financial and necessary technical assistance and support in conducting this study to the World Bank and MECS, especially to the following individuals: Dr. R. Bat-Erdene, READ National Director, Head of Education Department of MECS; Mr. Cristobal Ridao Kano and Mr. Charles Abelmann, Former READ Task Team Leaders; Dr. Kin Bing; Lead Education Specialist at World Bank, READ Task team leader; Ms. Marguerite Clark, Senior Specialist, WB; and Mr. Kelvin Gregory, International Consultant and Professor of Flinders University, Australia for the provision of advisory services during the course of the project. Additionally, I would like to express special thanks should go to the Heads and methodologists of Aimag Education and Culture Departments, secondary schools administrations, teachers and parents for their cooperation and provision of valuable assistance and support for the successful organization of the assessment at the national level. Finally yet importantly, we are thankful to the members of the EEC Project Team and staff of the Education Evaluation Center for organizing and administering the national assessment.

TS. OYUNSAIKHAN

DIRECTOR OF EDUCATION EVALUATION CENTER

Page 7: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 7

ii. BRIEF SUMMARY OF FINDINGS OF THE NATIONAL ASSESSMENT OF MATHEMATICS AND READING

The purpose of the study is to research Mongolian elementary education math and reading performance, using international standardized testing methods; to determine how location, population, curricular and extracurricular learning environments influence learning; and to use the results to improve the quality of education in Mongolia.

The study was organized in two stages. The first stage was a pilot study, completed in September, 2007. The main study was then conducted in April and May of 2008.

For the main study, 4,750 students from 166 schools were chosen from among 46,067 5th graders nationwide during the school year of 2007-2008. Also, the administrators of 165 schools, 166 lead teachers, and 1,944 parents and guardians took part in the study. One of the objectives of the study was to compare performance of Kazak and Mongolian language speaking students. Accordingly, 36 Kazak language schools in Bayan-Ulgii aimag and one middle school in Khovd aimag were included in the study.

The Structure of Observed Learning Outcomes (SOLO taxonomy) was used for test development. Each test-taker had their own testing materials. Each test section had its own code. The RASH method was used to check test validity. Test performances were calculated and entered into the MySchool database program created for the study. International testing questions comprised of 14.2-36.3 percent of the total test questions. A weighted method was used to choose team members for the study. Training seminars were conducted for team members, meetings were held for parents and guardians who took part in the study, interviews were conducted with teachers, examining the difficulties of the test, and their suggestions and recommendations were incorporating into the study.

Test scores of students who took part in the study are as follows: Math, 45.4%; Reading, 41.2%; with an overall average of 42.8%. The average score for urban students were 13.6% higher in math, and 10.1% higher in reading compared to rural students. Rural school students’ performance was 23.6% lower in math, and 18.7% in reading compared to urban students. This indicates that the quality of education can be correlated to location.

There were 3 basic versions of the study booklets with 9 variations. The math section consists of 28 questions of which 22 or 78.6% were multiple-choice, and the remaining 6 or 21.4% were open-ended questions. The reading section consisted of 35 questions of which 26 or 74.3% were multiple-choice, and the remaining 9 or 25.7% were open-ended questions. The average score for multiple-choice questions was 46.7%, and the average score for open-ended questions was 36.3%. For 12 ‘fill-in-the-blank’, ‘complete’, ‘compare’, ‘create’ and ‘draw’ open ended questions, the average score was 4.1 or 34.6%. 88.9% of students scored lower than 60% on the test. This indicates that elementary school students are not proficient in ‘fill-in-the-blank’, ‘complete’, ‘compare’, ‘create’ and ‘draw’ types of questions.

Test analysis was completed using Item Response Theory (IRT), which suggests that there were no unqualified, too difficult, or too easy questions. The average scores for Math and Reading were normally distributed.

Page 8: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 8

Graph 1. General Score Distribution

Scaled Score

Influences on math and reading, found after interviewing teachers, administrators, students, parents and guardians, and head teacher:

According to the results presents that the average score for math was 45.4% and the average score for reading was 48%. This indicates that students’ knowledge is below international standard requirements.

According to the results presents that the performance of open ended questions were lower than the performance of multiple choice questions.

Students performed poorly on question of analysis, comparison, analogy, and reasoning, as well as with questions of creativity and organization. This indicates that students’ creative thinking abilities need further development.

Students who scored high on the math test also scored high on the reading test. This indicates that math and reading influence the quality of education the most and are correlated with one another. Students who do well on both tests have a higher chance of performing well in other subjects.

Geometry and Algebra test performance results were lower than Statistics and Probability scores. Writing scores were lower than Listening Comprehension scores and Reading Comprehension scores.

Students studying in Kazak language have lower academic performance than students studying in Mongolian language schools.

Students scored below average on questions from international standardized tests. Classes with over 35 students scored lower than classes with an average of 35 students. So the

number of students in a class has affects academic performance. Students who share a desk with one other student scored higher than those sharing with 2 or more

students in both rural and urban areas. This illustrated that students should not be placed on a desk alone or with 3 or more students.

Students who live in a dormitory or with foster families scored lower than students living at home. Students who have a home library had greater academic performance.

Page 9: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 9

When students have difficulties understanding the lesson or subject matter they suffer from psychological pressure which leads them feel discouraged to learn and consider dropping out of school.

Giving creative homework based on student’s knowledge, comprehension, and needs, providing significant amount of time to completing the tasks and then evaluating the performance has a positive influence on academic performance.

5.8% of students had some type of disability, with vision accounting for the majority of the disabilities. Students with vision, hearing, and mobility impairment can study with the general population, but more research is needs to be conducted in terms of students with mental disabilities.

Students who entered school at the age of 6 demonstrated higher academic performances compared to those who entered at the age of 7 or 8.

Students who complete their homework independently demonstrated higher academic performance.

Students’ academic performance depends on teacher preparation. Teachers who prepare for class by Subject, Section and Unit, and Teachers who seek consultation

have greater influence on their students’ academic performance. Academic performance can depend on teacher’s treatment of and respect for their students.

Therefore, teachers need to see their students as individuals, and need training in forming relationships with students.

Teachers in the elementary education system that use tools they create, or those created by their students have a greater positive influence on students’ academic performance than those who use computer and LCD equipment.

Teacher’s level of education has a positive correlation with students’ performance. The number of years of teaching experience of teachers has no influence on students’ academic

performance. Teacher training and seminars do not directly influence teachers’ teaching methodologies. Rural schools indicated a higher level of negative psychological pressure and impinged on

students’ freedoms. The study presented that administrators evaluate their teachers more accurately today.

Page 10: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 10

Table No. 1. Proposals and Directions of Work to be done

Current Issues Responsible entity Direction of the work to be done Duration

Standards of primary education To undertake measures in connection with the content difficulty of primary education standards, unclear and not adequate terms and terminologies used and lack of environmental standards.

MECS, Institute of Education and other organizations, and tertiary institutions

- Develop and adopt standards of learning environment

- Carry out evaluation of standards and renewing of standards and its refining

- Develop National Program of Education

- Revise evaluation standards and procedures of primary education, and their enhancement

Short term

Textbooks Issues to be tackled include poor quality of textbooks for primary education, and the determination of whether optional textbooks meet the criteria.

MECS, Education and Culture Departments of Aimags and the capital city, teachers, authors of textbooks

- Ensure every student possesses textbooks

- Develop one high quality textbook written according to the standards

Short term

Teacher activities Enhancing teaching skills of primary education teachers, implement trainings for capacity building and development of teachers, establish students’ workload accurately, equip teachers with knowledge and skills and subsequently encourage them to provide students with homework that is creative and that will match the age and psychological development of their students.

MECS, Institute of Education and other organizations, and teacher training education institutions Education and Culture Departments of Aimags and the capital city, teachers and school administrators

- Establish teacher development centers in each school and effectively organize training seminars and workshops

- Effectively restructure the national teacher in-service training system

- Rationally establish teachers’ workload and improve their team working skills

- Manage teachers’ social issues - Establish students’ workload in

accordance with students age and psychological specifics

- Expand and enhance the computer use of teachers

Short term

Class size To minimize the amount of large classes that affect workload, health and quality of teaching, and learning for both teachers and students.

MECS, Institute of Education and other organizations, Education and Culture Departments of Aimags and the capital city, teachers and school administrators

- Maintain a class size not larger than 35 students per class (to establish a vacant position for an assistant teacher per grade)

- Improve the interior environment, health and sanitation conditions and yet the comfort of classrooms and facilities.

Short term

Training and education activities for teacher training To enhance the quality of graduates from teacher training institutions to meet the growing demand and requirements of quality criteria.

Government, MECS, teacher training and education institutions

- Select and enroll children who are genuinely motivated and interested in teaching

- Provide Government support to educational institutions involved in teacher training

- Capacity building of teaching personnel of teacher training institutions and improvement of training programs and curriculums, particularly, enhancing content of didactic programs on teaching, thinking and research

Short term

Supply and quality of a dormitory Improve the supply, quality and environment of dormitories

MECS, Education and Culture Departments of Aimags and the capital city, and school administrators

- Build and develop new dormitories - Improve environmental and

conditions

Long term

Page 11: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 11

Level of livelihood To pay attention to the existing link between the level of poverty and livelihood that directly affects learning achievement.

Parliament of Mongolia, Government of Mongolia, MECS and parents

- Policy regulation adopted at State and Government levels

- Increase government support to be provided to students who come from economically less capable families

Long term

Urban and rural location To eliminate the difference in learning achievement among urban and rural students.

Government, MECS, School administrators, parents

- Enhance environmental and conditions of rural schools

- Regulate migration - Undertake measures to improve the

quality of teaching personnel of rural schools

Short term

Co-operation To improve and expand the link and networking between schools, teachers and parents.

MECS and school administrators, teachers and parents

- Enhance related regulations and procedures

- Enable school activities to be more transparent for parents and enhance their involvement and responsibilities

Medium term

To link primary education with pre-school education To increase enrollment of pre-school education and enhance the quality of pre-school education activities.

MECS, Education and Culture Departments of Aimags and the capital city, administrators of school and kindergartens

- Increase enrollment of children to kindergartens

- Enhance the quality and accessibility of preschool education services

- Enhance the link between and integration of pre-school and secondary educations.

Medium term

National survey To improve poor implementation, policies and decisions made respect to education due to irregular organizing of such national scale surveys.

MECS and other donor countries and banks

- Undertake sample surveys by improving its design and methodologies

- Develop policies based on study findings and implement activities accordingly

- Establish practices for training teachers to gain knowledge and skills about test items designed in accordance with the international methodologies and apply them in their teaching

- Implement national similar surveys in all other subjects

- Enhance information database and software development

- Enhance the capacity building of personnel and its supply, and improve their specialization

Long term

Mathematics and reading To carry out analysis and assessment of insufficient quality of mathematics and reading and enhance learning achievement and its quality.

Education and Culture Departments of Aimags and the Capital city, teachers, administrators of schools

- Ensure that each school and teacher have developed and enriched test database that is piloted and reliable

- Train students in taking tests - Introduce consistently in teaching

advanced techniques, methodologies and approaches to develop creative thinking

- Pay special attention for students to acquire skills to solve and comprehend problems and creative writing skills

- Teach students understand and work with data, figures, diagrams and tables

Long term

Remarks: Duration in this table has been identified as short, medium and long terms. Short term implies to priority and urgent issues to be addressed, medium-term refers to actions to be undertaken in compulsory manner, and long-term refers to measures that have to be implemented on a regular basis that is to be undertaken in a compulsory manner.

Page 12: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 12

CHAPTER ONE. INTRODUCTION OF NATIONAL ASSESSMENT OF MATHEMATICS AND READING

1.1 INTRODUCTION

Over 40 percent of the total population of Mongolia consists of people under the age of 18. Today, more than 240,000 students are engaged in primary level education in secondary schools of the country.2

This study has been conducted to identify the level of mathematics and reading of students acquiring primary education, identify factors that influence them and reveal their causes. The study aimed to study the following issues and process the results obtained. These include:

Implementation, quality and accessibility of primary education mathematics and reading standards, (at what primary education level are the knowledge and skills of students?) What particular issues require attention in future?

Do learning achievements vary among school location, urban and rural schools, male and female students and between Mongolian and Kazakh students?

What are the primary factors that impact learning achievements? (what are the learning environment dependent factors, and what are the positive and negative impacts and how it can be improved?

How does the performance compare with that of international test items with results of other countries?

At what level are the issues of school environment, buildings and facilities; social issues of teachers; issues related with the teaching personnel, teaching methodology, in-service teacher training; livelihood conditions; and supply, availability and quality of books and textbooks? How can they be improved and through what means?

What are recommendations for teachers, students, school administrators and parents?

Mathematics and Literature have been selected as the core subjects of general education, and the subjects have become to be required to study in-depth out of all the other general subjects. A professional team was formed to implement the study for two years developing test items to assess students learning achievements. The team made assessments in the subjects’ course curriculum and evaluation methods and hence made close analysis on whether the set standards met the international evaluation standards. The testing items have been developed in compliance with requirements set by highly qualified team of teachers and researchers. They were piloted for quality improvement only at the end of the study, when the final results came out they made their vital contribution to the analysis with their expertise and commentaries. Thus, the study aimed to achieve an objective yet realistic conclusion and understanding.

The achievements in learning were determined on Mathematical and Reading Comprehension skills. To reveal influencing factors of learning achievements in, and to clarify reasons and identify approaches to

enhance learning conditions for students, questionnaires were taken from student representatives, selected school administrators, teachers3, aimag methodologists of primary education, and parents and guardians.

The survey book of students had 3 main parts for mathematics, reading comprehension and other survey parts and it contained a total of 63 tasks and questionnaire with set of 38 questions. Questionnaire taken from students contained questions about family and school environments, communication and interaction, supply of learning materials such as textbooks, sense of motivation and enthusiasm towards learning, etc. Questionnaires for

2 Number of students enrolled in primary education for the academic year of 2007-2008, MECS 3 Teachers in this report are referred teachers who have been teaching at primary level.

Page 13: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 13

school administrators, teacher representatives, and parents and guardians were comprised of 16-50 questions related to students’ health, family environment, learning motivation and interest, school environment, student development, supply and availability of textbooks, supply of teaching facilities, effectively communication among parents and teachers.

Newly developed software was used developing and processing information database of the study; gave the study an advantage of diminishing the probability of error to and extent of a certain percentage.

National study was carried out in two phases and the pilot study was organized from the end of September to early October 2007. The pilot study involved school administrators (school principal and heads of departments) and teachers with Masters Degrees from 39 schools were selected through sampling, and 1,170 6th grade students, and 758 5th grade students. The main study had been organized during April-May of 2008 and it has involved 4,750 students from 166 schools, management (school principal and managers of studies) of 165 schools, teachers with master’s degrees, and 1,944 parent and guardian representatives.

The study was conducted by taking consideration of Mongolian ethnic groups as well. Major Representative of national minorities in Mongolia is Kazakh. Most Kazakh people live and inhabit in Bayan-Ulgii aimag, and learning is held in Kazakh language in accordance with Mongolian educational standards. They survey was conducted for Kazakh students as well. Materials were translated to Kazakh language. All secondary schools in Bayan-Ulgii aimag, and 5th-11th grade students in Khovd aimag participated. Other national minorities communicate in Mongolian language hence there was no need to study them specifically.

Results of Mathematics and Reading Comprehension achievement was expressed through scaled scores used widely in international assessment surveys which applies an estimation that would mean arithmetic score of 500 with standard deviation to be equal to 100; based on that results of students involved in the study, their scores were converted into scaled scores with subsequent establishment of benchmark levels. If a student who has achieved a score equal4 with a mean score of the national study, it meant that the student was assessed with a mean of 500 scores. Also it means that 2/3 of students were assessed in a range from 400 to 600 scores. Drawing conclusions from scaled scores expressed through converting enables to determine outcomes more objectively. Or in other words it meant that outcomes will not differentiate depending upon levels of item difficulty.

1.2 NEEDS OF THE ASSESSMENT

Assessment and enhancement of the quality of education has been among the most important issues emphasized by policy makers within the education sectors of the global community. As a result of increased importance attached to studies that assess learning achievements in many countries, its methodology itself is getting more elaborate and comprehensively integrated. Similarly, a necessary requirement has evolved in Mongolia to conduct such studies on a regular basis with the aim to develop the educational system and enhance its overall benefits and outputs.

Results of our study could be compared and integrated with results of global/international studies to assess learning achievement of mathematics and life skills conducted by UNESCO methodology in 1999. One similar example is that the results of two studies have similarly indicated that learning achievement of students who have entered into school at the ages of 6-7 are higher than those who have entered at the age of 8. Therefore, enrollment of students into primary education at the age of 6 beginning the school year from 2008 in Mongolia could be understood as the right direction to undertake within the framework of educational reforms.

A study to assess learning achievement of students have been carried out 4 times in Mongolia and this study is the second survey carried out with the purpose of assessing learning achievement of students in Mathematics and Reading Comprehension. By conducting this kind of study on a regular basis it is believed that it will assist in determining the development and progress of primary education in Mongolia.

Reading comprehension and mathematics education enables to develop students cognitive skills to improve their value and understanding of the wealth of knowledge and intellectual values accumulated by human kind, and

4 National study to assess learning achievement, UB, 2006

Page 14: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 14

as well as their, processing and judgment skills. On the other hand, it has been decided to identify the levels of mathematics and reading and the factors that impact them in order to overcome difficulties and find ways to ensure greater satisfaction of social needs since these subjects are studied due to the importance of laying out the basics and build the foundation for many other applied natural and social sciences, and also an individual’s need to participate actively in multifaceted social activities have been taken into account along with the need to receive, transmit and process information; express one’s own views and thoughts; ability to tackle issues rationally and confront them; comprehend and recognize patterns through observation and testing; and develop a citizen who is able to treat any issues critically behave actively and consciously in making decisions by applying knowledge and language of literature and mathematics when involved in social relations.

Many of us tend to think that if a student has acquired good learning skills in primary classes, they then have the potential to obtain quality education at basic and complete secondary education levels. Have students been obtaining the proper and required level of primary education? Have our teachers been obtaining the necessary skills? What are the possibilities for decent work? Do teachers work honestly with genuine commitment? Does my child learn and improve when coming to class? Whether the tools my child needs is available at school? Whether schools are clean? Whether my child’s teacher scolds my child? Does my child feel cold at school? My child’s teacher seems to be a very good person, but the teacher claims that our child is not performing well. Am I able to buy books for my child? Are books expensive? Are these books really needed? Do other children bully my child? How would the methodology and skills of teachers compare with teachers of neighboring school? Is it correct to encourage this teacher? How are teachers of my aimag if compared to teachers from Khentii aimag? My child says that the school desk is too low, would the teacher get angry if I address that issue? What would happen if I do/don’t? At what level is my child though teaching of mathematics in our aimag has been good? My child got A in Mongolian language but received unsatisfactory results in exams. What does that mean? How are standards being implemented? What will happen if I address these issues? If I do background research myself it would take a lot of time, energy and skills, however, these are the questions we have to deal with. What would be the correct solution to these issues? All of these issues then have been studies in relation to mathematics and reading of students and attempts have been made to identify if not all but those priority issues to resolve urgently.

1.3 ADMINISTRATION OF THE NATIONAL ASSESSMENT

Professional teams have been appointed within the READ project and these teams have worked during every phase of the study. Study teams comprised of specialists from professional institutions, research organization and lecturers from institutions of higher education engaged in research and study, specialists from aimag and capital city Departments of Education and Culture, administrators and teachers from schools involved in the study have been divided into teams.

A team to perform and provide management tasks; a panel of specialists to develop themes, items and questionnaire; a team to administer study solely on its own; a team to check items with open ended responses, a team to create a study information database; a software provision team; a team to develop a report and assist teams; and international and domestic consultants have worked during the entire course of the study project implementation. The panel to develop test items along with the management and information technology teams have worked full time during the project implementation, whereas other teams have worked in accordance with associated work directives, demand and requirements. Team members to develop items for mathematics and reading have established a joint agreement where the responsibilities and tasks of each team have been clearly identified and kept teams fully engaged throughout the entire study provisioning. For instance, teams to study “Standards of Primary Education”, “Curriculum Framework”, design and calibration of items to assess learning achievement internationally, SOLO and Bloom taxonomies, creation of themes and items database, their pilot testing, development of an approach for scoring and coding with the aim to obtain objective and realistic study findings and results. Members of the questionnaire team was assigned the task to develop, pilot and further refine a questionnaire that would properly reveal factors that are impacting learning achievement, and contemporary trends

Page 15: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 15

in modern society and economy and conduct training on actual administering and organization of study surveys for selected teachers of institutions of higher education and secondary schools both in urban and rural areas, actual administering of studies to reveal factors that impact learning achievements, study evaluation approaches of open ended questions developed by project team members, ensure objective and accurate checking of mathematics and reading items by trained team of secondary school teachers; entering of checked, coded and scored data by secondary school teachers obtained from questionnaires for students, parents and teachers; mutual control of the other, creation of a database to reveal objectively the study results and outcomes, and ensure the provision of software to process recorded and entered information, etc. The report preparation and development team participated in the study project to write a report for policy makers, researchers, teachers, parents and the general public respectively based on facts and evidences and in accordance with the study findings.

Table No.2. Phases in Administration of the Study, Duration

Date/Duration Activities

Year 2006

August - November Appointment of teams;

Year 2007

January - July Creation of task item database, conducting of a study;

August - September Preparation of the pilot study (sample design, preparation of materials for study and printing, develop guidelines, appoint teams to administer study);

from September 15 till, November 5

Pilot study conducted;

November - December Creation of an information database for pilot study, editing data, conduct analysis, preparation of report;

December Provision of preparatory work for main study (sample design, creation of information database of participants);

Year 2008

January - March Prepare and finalize survey booklet;

April Conduct small trial tests and produce final version of study booklet;

February - March Conduct training seminars, training for team members to administer testing and further study;

Throughout the project Cooperation with international and national consultants;

April Printing of study materials, allocation and distribution;

April - May Conduct main study;

Page 16: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 16

From, June 1 till July 15 Coding of open ended items (developing an approach, appointing of teams, revision and corrections)

May - July Develop software;

July - September Creation of information database for main study, editing data;

October – December Preparation of report for main study, conduct analysis for submission, dissemination to the general public.

1.4 TARGET POPULATION OF THE NATIONAL ASSESSMENT

A National study has been carried out in two phases; a pilot study and a main study. 1,170 students from 39 schools were involved in the pilot study. Schools, classes and students who were involved in the pilot study were not included in main study. 4,750 students from 166 schools were involved in the main study which was selected in accordance with the sample design of the study.

5,107 students from 169 classes in 169 schools have been sampled for the study out of a total of 46,067 students studying in 5th grade of all Mongolian schools within the school year of 2007-2008. At that particular day, when testing was held all students who attended class had been involved except for 172 who were sick, 72 had excused absences and 74 had skipped class that day. Also 89 students have been transferred in and 121 students have been transferred out of schools which were involved in the study. 5,200 survey booklets have been printed with special dedication for every student with their names, national ID registration number, class and group.

The main study involved all aimags except for Gobisumber and Dundgoby. A total of 169 schools from 19 aimags have been sampled to be involved in the main study, but 2 special schools with not mainstream curriculums were not included in the assessment. Also Ulaanhad bag, Altai soum of BayanUlgii aimag elementary school couldn’t be included in the study due to bad weather conditions.

In addition 166 primary school teachers, 165 school administrators, methodologists from 21 aimag Education and Culture Departments and 1,944 parents and guardians representatives participated in the survey questionnaires of the assessment study.

Students targeted

39.4 percent or 1,871 urban, and 60.6 percent or 2,879 rural school students were involved in the study. Out of 15 students that were included in the study to represent the total student population within their age range

If an analysis of urban and rural students is carried out in terms of study coverage and representation then 63,1 percent of total students sampled (if considered without weighting) were rural and remaining 36,9 percent were urban students. School representation changes when the weight of urban and rural students is considered.

Table No. 3. Number of Urban and Rural Students

Urban or rural

Students involved in the study

Number of students by representation Percentage Number of students Percentage

Urban 16,836 36.9% 1,871 39.4% Rural 28,779 63.1% 2,879 60.6% Total 45,615 4,750

Page 17: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 17

Based on Table 3it can be stated that the sample design of the assessment study reflects national statistics. In terms of locations covered, 36.9 percent were urban, 26.5 percent were from aimag centers, and 34.3 percent from soum centers and 2.3 percent were from bag level schools (without considering their weight).

Table No. 4. Number of Students by Location

Location Students involved in the study

Number of students by representation Percentage Number of students Percentage

Capital city 16,836 36.9% 1,871 39.4% Aimag center 12,106 26.5% 1,285 27.1% Soum center 15,627 34.3% 1,330 28.0%

Bagh 1,046 2.3% 264 5.6% Total 45,615 4,750

50.1 percent of students involved in the assessment study were male and 49.9 percent were female. The average age was between 11 and 12. And 80.8 percent of students spoke Mongolian, 14.6 percent spoke Kazakh, 0.4 percent spoke Buriad, 0.5 percent in Durvod, 0.6 percent in Tuvian, 1.3 percent in English, 0.3 percent in Russian, 0.7 percent in Chinese and the remaining 0.4 percent in other languages. 67 percent of students had been attending kindergarten from age 1-7 and the average attendance kindergartens were 2.6 years.

Teachers

A total of 166 teachers who taught mathematics and Mongolian language to 5th grade students participated

in the study. 7.2 percent were male and 92.8 percent were female. When determining the number of years the primary school teachers who participated in the study dedicated

themselves to the service the longest serving teacher has been working for 42nd school year whereas the youngest teacher was in her first year of teaching career.

92.2 percent of teachers were primary school teachers and the remaining were teachers of Mongolian language and mathematics and in considering the level of educational qualification, 38,3 percent of teachers had special professional education, 41,5 percent had a bachelor’s degree in teaching; 3.7 percent had a Masters degree; and 1.2 percent had completed secondary education and 15.0 percent were certified professionals.

10.3 percent of teachers had no certificate or licenses to teach and the remaining percentage of teachers had teaching licenses/certificates.

On average teachers traveled 2.2 km to get to work and travel back home in 19 minutes. Most of them or 72.9 percent of them walked to work.

Most teachers or 73.5 percent owned their own apartments, 12 percent lived with their parents, siblings and relative’s home; 12.5 percent rented apartments; 39.7 percent lived in traditional gers. 22.3 percent lived in housing connected to the centralized heating and 29.8 percent of them lived in housing with no potable and hot waters.

78.2 percent of teachers live in apartments with 24 hour electricity supply, 72.1 percent live in housing which do not supply heat, 75.5 percent live in apartments without hot running water supply, and 63.7 percent have potable water supply on-delivery or from water supply kiosks; 56.9 percent had not computers; 90.5 percent had no internet access; and 34.6 percent of teachers owned their own vehicles. The total number of hours teachers worked on average were 19.6 hours per week, ranging from the lowest load of 4 hours per week to 31 hours per week.

When considering teachers’ experiences and period of teaching, the teachers involved in the study, 40.4 percent of mathematics teachers were teaching mathematics for 5th consecutive years, and 38 percent of Mongolian language teachers were teaching for 5 consecutive years in their classes. Class size of primary school class settings varied. In urban schools there were classes with a number of students up to 55. An average class size has been larger in urban schools compared to rural ones. Official regulations have set the maximum number of students per

Page 18: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 18

class not to exceed 35. The average class size was 40 students per class when selective analyses of schools with class size of more than 35 students per class were undertaken.

Schools

95.8 percent of schools involved in the study were public schools and 4.2 percent of them were privately owned schools. It can be considered that representation of both public and private schools is sufficient. And the pie-chart shown below represents schools involved in the study by their location.

Figure No. 2. Location of schools involved in the study

74.5 percent of schools involved in the study were regular mainstream schools whereas 3 percent of schools were schools with vocational training and 22.5 percent of them with advanced curriculum studies.

The largest school involved in the study had 5,746 students with 174 classes whereas the smallest school from a bag had 4 classes with 18 students. The school with the largest number of 5th grade classes had 13 groups and 396 students, whereas a school with the smallest number of 5the grade class had 1 group with 3 students. When the number of school staff was calculated the largest school had 381 staff and the smallest school had 3 employees.

School management

The survey of school administration involved principals and training managers from 165 schools of which 65.5 percent of surveyed persons were school principals or head masters and the remaining were school managers. The average age of school administrators was 46, the youngest was 24 years old, and the oldest was 64 years old. Out of all the longest serving manager has been working for 32 years and the shortest serving manager has been working in her first year. 44.8 percent of school administrators were male and the remaining were female teachers.

Most of the school managers graduated from the Mongolian National University, Mongolian National University of Education, Teacher Training College and Teacher Training School in Arkhangai aimag. 47.2 percent of school managers have graduated from one educational institution, whereas 15.2 percent have graduated from the Academy of Management. In terms of professional background 26.7 percent of school administrators were primary school teachers, 15.8 percent were mathematics teachers, 12.1 percent were Mongolian language teachers and the remaining teachers taught other subjects. 3.0 percent of school administrators was persons with special education, 7.9 percent were people with diploma level of education, 47.3 percent had completed their bachelors in Education, 40 percent had acquired their Masters in Education and 0,6 percent had PhD in education.

Page 19: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 19

Representation of parents and guardians

56.2% of parent and guardian representatives of students involved in the study were mothers, 16.3% were fathers, 4.9% were older brothers, 14.6% were older sisters, 4.4% were grandparents and 3.6% were neighbors, relatives and dormitory teachers.

The education level of parents and guardians were determined by the following. 14.7% of fathers had bachelor’s degree, 30% had special professional education, 30.7% had completed their secondary education, 25% had professional education, and 6% had primary education. 17.4% of the mothers had their bachelor’s degree, 31.4% had special professional education, 31.3% had completed secondary education, 22% had secondary education, 5.5% had primary education and 14.4 % of guardians had their bachelor’s degree, 25.4% had professional education, 30.9% had completed secondary education, 17.9% secondary education, 7% had primary education. 1% of fathers, 2% of mothers and 3.5% of guardians had no form of formal education.

Methodologists from aimag ECD in charge of primary education

21 methodologists in charge of primary education from aimags have been involved in the study and 85.7% were female and 14.3% were male. The age range of the methodologists spanned from 32 to 55 and the average age was 41. 77.8% of methodologists were primary school teachers, 5.5% were Mongolian language teachers and 16.7% were of other disciplines. Considering their level of education, 10% had teaching certificates, 35% completed their bachelors in Education and 55% had acquired master’s in Education. In terms of the level of qualification, 4.8% were teachers with the rankings of methodologists, 66.7% were teachers with lead teacher rankings and 28.6% had advisory level of qualification rankings.

Aimag ELD methodologists have been working within the education sector for 3-34 years and have worked as methodologists in charge of primary education for 1-21 years.

1.5 RESEARCH METHODOLOGY

The factors that impact learning achievement and their interrelationships have been identified in addition to determining learning achievements of mathematics and reading of students through the national assessment study of learning achievements in primary education.

Many methodologies have been applied during the course of the study and furthermore, these methods and approaches have now been made possible to be used in further studies and surveys. It has been considered that this study has far more advantages compared to other studies conducted in Mongolia in previous years, since relevant instruments and approaches along with international methodologies have been applied accordingly during each phase of the study. Specific methodologies and approaches used in this study are as follows:

1. In designing survey tasks, blueprints have been developed in accordance with the SOLO taxonomy. 2. Survey booklets have been developed in compliance with international approaches. 3. System of numbering and coding has been introduced for each survey items. 4. Booklet design has been used in the distribution of study booklets; 5. Item analysis has been carried out using classical statistics approaches of RASH (IRT) model tasks of

ConQuest programming; 6. Newly developed MySchool software has been used when specifically in creating information

database; 7. SPSS (Statistical Package for Social Sciences)-13 has been used in conducting analysis of study results

and processing data for information database; 8. Analysis has been made through converting students test results into scaled scores in addition to using

average performance percentages of students in mathematics and reading ;

Page 20: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 20

9. Weighing approach that takes into account representation characteristics has been used in developing sample design.

10. A team to carry out and administer the examination were selected and specially trained through a sequence of training workshops and seminars.

11. International test items have been included at 14.2-36.3% of the total items to carry out a comparative study.

12. Meeting with parents was organized in classes that participated in the study and discussions and dialogue were held among the parents and organizers.

13. Discussion and dialogue was held with representatives of teachers and methodologists in charge of primary education on priority issues facing primary education and their proposals and viewpoints have been reflected duly in the study report.

1.6 ITEMS AND QUESTIONNAIRES DEVELOPMENT METHODOLOGY

Education standard have been in effect in Mongolia since 1 January, 2005 and thus this year is the 4th year since the standards have been enacted for its implementation.

The aim of education standard lies in planning of a curriculum for delivery of a particular level of education to students, measurement, and assessment of the quality of activities undertaken for its implementation and monitoring, and serves as the main factor for the standard of development.

Mathematics and reading referred to in this study implies skills which have been prescribed by standards of primary education in mathematics, Mongolian language and literature.

Primary education standards in mathematics

Certain skills have been selected for assessment of competencies such as interactions using mathematical language, mathematical based reasoning, model-based estimations and problem solving within four domains of mathematical competencies that include number and numeracy, algebra, geometry, probability and statistics of primary education mathematics. About 70 % of skills prescribed by primary education standards of mathematics have been included for assessment through this study.

Primary education standards in Mongolian language

Standards of Mongolian language is a document that has established benchmarks in the domains of meaning of each language unit, structure, modification and culture in acquiring of basic skill levels of reading, listening, speaking and writing by students. Primary education standard have four competencies: knowledge, execution, existence and socialization and therefore, expresses knowledge, skills, attitude and development acquired by a human being through an integrated approach.

Learning activities of Reading (selection of a topic for reading, seeking out for that material, reading for oneself and others, reflecting what has been read, comparison with one’s own previous knowledge and viewpoints, provision of personal judgment, application of knowledge and skills from personal life), Writing (memorizing a selected text, writing by heart/from memory or expository writing, creative and initiative writings, essay writing, selection of a theme for writing passage, gathering relevant materials, drafting, proof-reading and refining of a draft, final writing, analysis and assessment of own text and passages written by others), Listening (listening to a text and working with it) have been included in the assessment study. By this study we have targeted to study specifically reading but not all skills prescribed within Mongolian language standard.

Page 21: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 21

Content of items designed and used for mathematics test has covered number and numeracy, algebra, geometry and probability section and question of reading has involved 3 skills as listening, reading and essay or creative writing skills and designed within the scope of language meaning, structure, transformation and culture sections.

Members of question development teams have performed and completed the following tasks and assignments:

Members of the national assessment teams have developed a detailed work plan, researched and analyzed concepts, content, evaluation, methodology and sample curriculum framework of primary education standards;

Team members have created a questions database which comprised of texts and questions to suit the age range and psychological specifics of students acquiring primary education and have carried out analysis of each questions and tasks;

Team members have studied and analyzed international methodologies of assessment of mathematics and reading, content of questions and tasks, and their calibrations and designs;

Team members have agreed and established a ratio of questions and tasks by studying the evaluation methodology of SOLO taxonomy;

Team members have identified which sections need more attention in primary education standards; Test questions and tasks were reviewed and improved by international and domestic consultants based on

active exchange of viewpoints and ideas; Pilot testing was conducted after test items were developed and reviewed; Results of the pilot test were analyzed and items which were too difficult or easy, or had poor discriminate

abilities were either eliminated or re-designed and improved; Evaluation and assessment approaches and procedures were developed after questions and tasks were

developed; Training was organized for methodologists, specialists and teachers of primary education after the

development of the evaluation and assessment procedures and instructions were completed and their views and proposals have been integrated;

After review and improvement of test items, they were given for review and assessment by members of consulting teams;

Final versions of the study booklets have been developed and completed; The National assessment was implemented; Questions and tasks with open ended responses were coded; Database of multiple-choice and open ended response questions and tasks were created.

Following strict requirements after designing and developing of test items the following were undertaken:

To establish an optimal number of test items and time required for completion in accordance with students age and psychological specifics;

To establish the maximum number of pages for each test booklet; To develop questions and tasks according to each phase and level of cognitive understanding; To apply variable forms and types of questions and task design and content; To ensure quality and appropriateness of questions and tasks, and their level of difficulty as well as their

discriminate abilities; To match questions and tasks with content standards of primary education standards; To apply items which are based on SOLO taxonomy approach; and To have answer keys for questions and tasks to be developed.

Page 22: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 22

Members of the project team have studied Primary Education Standards and SOLO and Bloom taxanomies prior to carrying out of this national assessment and have shared a consideration that SOLO taxanomy is an approach to reveal students’ skills more comprehensively and fully, and develop questions and tasks in blueprints based on the creation of their databases and pilot testing of those questions and tasks.

Questions and tasks that reveal each level of cognitive understanding have been developed according to the required compulsory knowledge to be acquired by students in each of those levels from the content standard of the national primary education standards when blueprints of test questions and tasks were developed. Please refer to the blueprint of mathematics and reading from a Table No. 5.

SOLO taxanomy including the following 5 levels:

Pre-structural level: Basic notions should be acquired at this level; Uni-structural level: Students have recognized and understood only one notion and are not able to establish

links and associations with other notions; Multi-structural level: Students can use number of notions that are interrelated and linked in between; Relational level: Students possess complete understanding of systems or structures within a notion and are

able to judge and evaluate interrelation and dependence; Extended abstract level: Students are able to add new notions or concepts on ones which have been

acquired and are able to use and apply under new conditions and circumstances.

Table No. 5. Blueprint of Mathematics

Section Standard content

Levels of Knowledge Acquisition Total Items

2L 3L 4L Number Percentage

NU

MB

ER

AN

D N

UM

ER

AC

Y

Writing decimals in words 4* 12 42.9% Using property of numbers divisible by 4 1* Solving word problems using time units 2* Recognizing relationship between time measurements 3* Ordering whole numbers 8* Finding a number related to a given proportion 5* Comprehending, reading a map and calculating the distance between 2 cities

6*

Finding the value of a numerical expression 7* Understanding and reading a bar graph 10* 14* Finding necessary information from the given bar graph and comparing them

11*

Finding the mean of numbers from the given bar graph 12* Number of questions and tasks from Number and Numeracy section 3 7 2

AL

GE

BR

A

Finding relationship between the area of floor and weight of painting 16*,17* 8 28.6% Finding a variable understanding the algorithm and checking

19*,20*

Finding a fraction using a diagram and expressing it with whole and fractional parts

21*

Finding the variable from the expression and the value of it 22* Finding the coordinate of a point on the numerical ray 18* Composing an expression with a variable 9*

Number of questions and tasks from algebra section 6 2

GE

OM

ET

RY

Finding relationship between a point and a segment and counting all possible segments

28* 5 17.8

Finding the Area of a rectangle 15* Labeling angles 26* Dividing a square into 4 equal triangles 13* Finding the height of solid rectangular using the volume and other measurements

27*

Number of questions and tasks from Geometry section 3 2

P R O B A

Counting possibilities 25* 3 10.7% Defining a pattern 23*

Page 23: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 23

Counting possibility to match a given condition 24* Number of questions and tasks from Probability section 1 2 Number of all questions and tasks 3 17 8 28 Weight % 10.7% 60.8% 28.5% 100% PS: *- Number of Questions and Tasks in booklets 1,4,7

Table No. 6. Blueprint of Reading

Section Standard content Levels of Knowledge Acquisition

Total Percentage 2L 3L 4L 5L

Mea

ning

of a

text

Finding the theme of a text; R5A2_02, R5A2_17 2

89.2%

Finding the main idea of a text;

R5A2_03, R5A2-07 R5A2_18

R5A3_018, R5A3_06

R5A4_05 R5A4_06

7

Finding the intended purpose of a text;

R5A1_14 1

Finding the main idea of a part of a text;

R5A1_01, R5A1_06 R5A1_19, R5A1_21

R5A2_12 R5A3_08 R5A4_02 7

Finding of an additional idea of a text;

R5A1_04, R5A1_07 R5A1_08, R5A1_09 R5A1_10, R5A1_11 R5A1_12, R5A1_13

R5A1_15 R5A1_16, R5A1_20 R5A1_22, R5A1_23

R5A2_05 R5A2_08, R5A2_09 R5A2_13, R5A2_14

R5A3_05 21

Finding support and key words and sentences;

R5A1_02 R5A2_06, R5A2_11 R5A3_02 4

Defining an object and its features;

R5A1_03, R5A1_18 R5A2_16 3

Finding meaningful relationships between language units;

R5A1_17 R5A2_01 R5A3_03, R5A3_07 R5A3_10,

R5A4_04 6

Finding the main idea of a paragraph;

R5A3_04 1

Expressing a main idea of a text through an idiom;

R5A4_01, R5A4_07

2

Distinguishing between similar and opposite meanings of paragraphs;

R5A2_04, R5A2_15 2

Distinguishing between similar and opposite meanings of sentences;

R5A2_10 R5A3_09 R5A4_03 3

Defining an intended purpose of sentence;

R5A1_05 1

Number of items from textual meaning section;

23 18 10 7 58

Tex

tual

st

ruct

ure Recognizing a paragraph; R5B1_01 1

6.2%

Finding parts of a text; distinguishing them;

R5B3_01, R5B3_02, R5B3_03

3

Number of items from textual structure section;

1 3 0 4

Tex

tual

tran

sfor

mat

ion

and

cultu

re

Creative writing based on observation of a drawing;

R5CD_01A, R5CD_02A R5CD_03A, R5CD_01B, R5CD_02B,

R5CD_03AB, R5CD_01C, R5CD_02C R5CD_03C,

9

4.6%

Number of items in textual transformation and culture section

0 0 0 9 9

Total Number of questions and tasks

24 18,0 13 16 71 100

Total Percentage 34% 26% 19% 23%

Page 24: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 24

Test items to assess learning achievement of students in mathematics and identify their levels were developed within the scope of primary education standard section of mathematics including number and numeracy, algebra, geometry and probability statistics. Questions and tasks of the algebra section and multi-structural level tasks have been included predominantly taking into consideration the following factors as age and specifics of psychological development of children of this age. On the other hand, content framework of the education standards was taken into account.

Test items in mathematics have been selected in accordance with 3 levels out of 5 levels of cognitive understanding approach known as SOLO Taxanomy which covered a uni-structural (2L), multi-structural (3L) and relational level (4L) questions and tasks since the National assessment was meant to evaluate mathematics skills at the primary education level. The percentage ratio of the three levels of questions and tasks were split as follows: 2L-2 level questions and tasks have taken up 7.1%, 3L-19 questions and tasks 67.8% and 3L-7 questions and tasks have taken up 28% of all questions and tasks hence indicating that the questions and tasks have been appropriately selected to match the age and psychological characteristics of primary school students.

Questions and tasks of uni-structural level in mathematics tests have included tasks to test skills in comparing numbers, interpreting and reading column diagrams, and comparing the given information from column diagrams. Multi-structural items included tasks to test related skills of students such as writing decimals in figures; applying features of multiple-digit numbers to be divided into four; finding the relationships between time measurement units; matching numbers with given percentages; understanding, interpreting and estimating line or floorplan drawings done under a concrete established scale; calculating the sum of mixed multiple digit numbers through compound operations; calculating arithmetic means of numbers from information given in column diagram; establishing relationships between measurement units of an area and weight; finding an unknown number from a given expression; expressing fractions and whole numbers and integers in numbers; finding the meaning of an expression by determining unknown operational elements; finding coordinates of a point on a number frame of reference; creative writing an expression with unknown elements; determining the area of a parallelepiped; naming angles; dividing geometrical shapes and determining the frequency patterns. Relational level items have a purpose to reveal skills related with the transfer of motional and time measurement units; interpreting and understanding column a diagram; determining and examining an unknown number in a given algorithmic data sequence; determining the relationship between line segment and a point, counting all potential line segments that could be established; finding the relationship between a right angle parallelepiped volume and measurements; counting the number of possibilities to estimate volume and counting possible terms or factors that will keep objectivity or truthfulness of particular term and condition.

Questions and tasks to reveal language skills in reading included questions selected from three major parts that comprised of listening, reading and creative writing from a visual provided.

Reading tasks of the test for primary education have been developed based on cognitive skill levels established according to the SOLO Taxonomy approach. SOLO Taxonomy has 5 cognitive levels. Test items in reading have been selected according to the 4 levels out of 5 levels of cognitive SOLO Taxanomy approach to meet the objectives and covered uni-structural (2L), multi-structural (3L), relational level (4L) and abstract (5L) items since the National assessment was meant to evaluate reading at the primary level of education. The percentage ratio of those selected 4 levels of questions and tasks were divided as follows: 2L-24 level questions and tasks took up 39.4%; 3L-18 questions and tasks 27.3%; 4L-10 questions and tasks 162% and 5L-12 questions and tasks have taken up 18.2% of the total questions and tasks. This indicates that questions and tasks were appropriately selected to match the age and psychological characteristics of primary school students.

Listening

Listening is considered as one of the important elements of reading and one of the variety of methods used in acquiring and obtaining knowledge and information. In other words, in addition to reading, students are required to acquire listening skills to comprehend the content of what has been listened to and process information received.

Page 25: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 25

Therefore, in order to reveal the level of reading a text called “Why do birds migrate” and the questions and tasks were designed containing the main ideas of each paragraph for students to provide their answers. These questions and tasks were multiple-choice questions with discriminate ability.

Reading

Reading questions and tasks were developed according to the requirement set in the standards of primary education for primary school student to read a minimum of 70 words per minute. Following this requirement 3 texts with around 210-260 words each were selected with the notion that students would spend 3 minutes reading a text with questions and tasks developed to reveal the meaning, structure, transformation and culture of language and answers provided in multiple-choice format. Types of textual style, genres and forms have been taken into consideration when selecting reading texts for the test. The purpose of this item was for students to demonstrate that they are able to comprehend by reading texts of any form of style and genre.

Essay/Creative writing

The theme and the number of words were given through questions and tasks to be performed by students in order to reveal their essay or creative writing skills. Students were given a drawing to match the theme provided to guide or hint since essay or creative writing tasks were questions and tasks of relatively high difficulty and abstract level that required them to think and write creatively. Creative writing of students were assessed during the pilot study through questions and tasks were asking them to reproduce what had been understood by them after listeninig to a recorded text. However, results of the pilot study have shown that this items was not a creative task since students had been writing pretty similar texts, therefore, the design of this task has been changed into a task which required students to write a creative essay under the given topic and hints derived from a visual. A visual was selected and provided containing themes and content which have been relevant and appropriate for creative writing by any student, both urban and rural.

QUESTIONNAIRE DESIGN Since the major aim of the national assessment has been to provide explanations about factors having

impact on learning achievements in association with learning achievements and to identify ways of improvement. Therefore, greater attention has to be paid to the design of the questionnaire. Various forms of questionnaires were used, including fill-in-the-blank and multiple-choice questions, however, questions were adjusted in their design accordingly depending on who these questions were directed to or asked from and made it as easy and plain to understand.

Questionnaires were designed and developed to survey teachers, students, school administrators, aimag methodologists in charge of primary education and parent and guardian representatives.

To develop questions that are realistic and objective as possible, questions that are asked from students, for instance, questions were developed to irritate them and made easy for them to answer and reveal their assessments realistically. Certain issues that are difficult and delicate such as whether a child is an orphan or not, employment of parents, level of livelihood standards, health conditions, disabilities that could possibly be delicately perceived by students have been decided to be directed towards their master teachers and 16 specific questions have been added to the questionnaire designed for master teachers.

In a questionnaire developed for school administrators, issues have been included related to school learning environments, teaching staff, supply of books and textbooks, quality, social issues and skills of teachers, state of students’ knowledge and morale development, financial and physical environment and whether there are effective working legal environment in place etc.

Page 26: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 26

Questionnaire to be taken from parents and guardians included questions related to school learning environments; knowledge, skills and methodology of teachers; information about students; and satisfaction and education of parents.

Participants of the national assessment have been surveyed on the following major group of issues related to family environment (education of parents, whether they have their own separate room and satisfactory livelihood supply, whether they possess their own set of books and desk; regarding power supply conditions, whether their children feel hungry and cold, whether their children went to kindergarten, employment of parents, how far their homes are from their school and how many of them are in their families, etc.), school learning environment (supply and availability of textbooks and books, classroom environment, whether school restrooms are outside or inside the building, the level of prestige of the school, how far the school is from students’ homes, whether training inventories are sufficient, whether school has a library, the number of students that share a desk, the size of the class is adequate, whether the school has their own facilities, the quality of buildings and facilities, when the school was built, the location of the school, whether the school has many classes and facilities, the source of school electricity supply, school prestige, how long students have been studying together with their classmates, whether students have sports facilities to perform their physical activities etc.), social environment (how much attention is provided from the central and local governments to education, whether standard are relevant and influential to reality), finances and physical infrastructure (level of living standards, possession of sportswear and textbooks, possession of home TV set, vacuum cleaner, personal computer, home availability of restroom and access to the internet, phone etc.), teaching personnel (whether teacher conducts good lessons, level of teacher education, age of teachers, in-service teacher training, work experience, whether teachers prepare electronically produced materials and use computers during their lessons, whether teachers make ethical mistakes, whether their teachers have been writing books or textbooks, use of teaching and training materials, how teacher skills are evaluated, whether teachers provide extra tutorial lessons to students lagging behind in their studies, whether teachers provide explanations on questions asked on unclear issues and respect their students, etc.), friends and friendship environment (communication and relationship between friends, whether children are engaged in playing various games, etc.), leisure time (how students spend their leisure time, and how much time students spend to attend social clubs and payment-based trainings, in helping their friends study and perform household chores, in cleaning their classrooms and facilities, in doing their homework, in taking care of their siblings, in studying at libraries, in playing games and watching TV, in working to earn money, etc.).

In carrying out analysis of how the above mentioned factors have been associated with learning achievements, the following issues have to be considered:

Ensure fulfillment with an aim or goal-oriented; Questions should avoid content duplication and ambiguity. It should be precise and clear to

understandable; Questionnaire should not have an imperative or superior language tone of order; Establish a rational timeline needed for completing the questionnaire and its overall completion and

size of the questionnaire; Questionnaire should begin with easy questions and lead to difficult questions; Logical order should be followed and applied; Carry out surveillance on whether questionnaire content is agreed by.

Page 27: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 27

1.7 BOOKLETS DESIGN

National assessment booklets of primary education have been published in 9 different versions and each version has consisted of three parts, including:

Mathematics section; Reading section; Student’s questionnaire section;

Study booklet has been printed in A4 format, 38 pages in Times New Roman font. Name and school as

well as which booklet to work on have been all indicated clearly on the cover of the assessment booklet, such as on its first page (full name, name of aimag, soum, class group, school ID5, control ID, students ID, control code, registration booklet and gender).

Prior to each section, students were given Attention Notes with instructions given on how to complete test items and reminders on time needed to complete each tasks which were to help students prevent possible mistakes. On the other hand, this could have a positive impact in respect to having more objective and realistic results from the national assessment.

Items have been designed and developed to identify levels of primary education mathematics, which comprised of 66 distinct questions and tasks with sound discriminate abilities, combined with one version bridging problem, and 28 questions and tasks to be completed in 60 minutes, whereas to identify reading had consisted of 71 questions and tasks with proper discriminate abilities with 35 questions and tasks to be completed in one version, and 39 questions were included in the questionnaire section to respond within 20-30 minutes to reveal factors that impact learning achievement of students.

Furthermore questions, task design and planning included the following allocations conducted in terms of tasks structure and educational standard domain representation as inclusion in each of test versions a combination of 12 from the number and numeracy section of primary education mathematics, 8 from the algebra section, and 5 geometry section. 3 questions and tasks from probability statistics were split into 3 uni-structural level items, 17 multi-structural level items and 8 relational level tasks according to cognitive SOLO taxonomy approach. In terms of reading test comprised of 58 questions and tasks on language meaning from primary education reading skills, 4 language structure section and 9 language modification/transformation and culture section which were split into 24 uni-structural level, 18 multi-structural items, 13 relational structure and 10 from extended abstract structure in accordance with cognitive SOLO taxonomy approach.

Moreover, Coded Notes were inserted for each task at the upper left corner of a page indicating subject, section and level of cognitive understanding of each task which were conducted with to use as a reference in making analysis of items easier. For instance:

Coded notes referenced as R5A2_01, M5A3_03 for each item shall be decoded as R-reading skill, M-mathematics, 5-class involved in the assessment, A-language meaning section, B-language structure section, CD-language transformation and Culture section/A-number and numeracy section of mathematics, B-algebra structural section, C-section of probability statistics, D-geometry section/, 2-uni-structural level of cognitive understanding, 01-item No.1 etc. Also coded notes on lower right corner of a page as B2_M15 /В2_R5/ is decoded and understood as B-study booklet, 2-Number of a study booklet, M-mathematics/R-reading/, 15-number of an item in that particular study booklet etc.

Study booklets of the national primary education assessment have been developed in 3 basic versions and were published in 9 different publications by changing the order of items in booklets. The following tables present how items of mathematics and reading were created for each study or assessment booklet.

5 Number of a school

Page 28: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 28

Table No. 7. Design of Study Booklets for Mathematics Items

Section Cluster Number of questions and tasks Number and Numeracy А А1 А2 8 Algebra B B1 B2 5 Probability C C1 C2 2 Geometry D D1 D2 3 Bridging task items SC 10

Note: clusters - А, А1, А2, В, В1, В2;

If each study booklet is considered in detail:

Table No. 8. Design of Mathematics Items in Assessment Booklets

Study booklets I II III IV V VI VII VIII IX

A B B A1 B1 B1 A2 B2 B2 SC C SC SC C1 SC SC C2 SC B D A B1 D1 A1 B2 D2 A2 C SC D C1 SC D1 C2 SC D2 D A C D1 A1 C1 D2 A2 C2

Table No. 9. Design of Reading Items in Study Booklet

Type of skills Type of texts Cluster Number of question and tasks

Items for listening SC1 SC1 SC1 5

Reading comprehension

Cognitive Science text A A1 A2 5 Text from modern fiction B B1 B2 5 Text from traditional folklore C C1 C2 5 International test items SC2 SC2 SC2 12

Creative essay writing based on a picture shown

Creative writing of a text on a given theme

SC3 SC3 SC3 3*

total 35

Table No. 10. Design of Reading Each of Study Booklet

Study booklets I II III IV V VI VII VIII IX

SC1 SC1 SC1 SC1 SC1 SC1 SC1 SC1 SC1

Page 29: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 29

A B C A1 B1 C1 A2 B2 C2 B C A B1 C1 A1 B2 C2 A2 C A B C1 A1 B1 C2 A2 B2 SC2 SC2 SC2 SC2 SC2 SC2 SC2 SC2 SC2 SC3 SC3 SC3 SC3 SC3 SC3 SC3 SC3 SC3 In addition, 10 minutes of break time has been recommended to be taken after mathematics and reading test

sections. At the end of tests a questionnaire was included for students to respond.

1.8 ITEM SCORING METHOD

The measures of literacy and mathematics in the Nationwide Research of the Success of the Elementary Education Systems were multiple-choice and open ended questions. Each multiple choice question had 4 options and only one correct answer. One point was given for each correct multiple-choice answer, and 0 points for each incorrect answer. 1-5 points for each open ended question, depending on the number of steps taken to get an answer. Two scores were reported. First was the simple raw score, or the number of points. The second score was a diagnostic score. This second score was reported using a 2 digit code known only by the researchers. The researchers used an instruction book in scoring the tests. For the open ended questions, a 2 digit code was used. The teacher wrote the code according to the instruction book, and a monitoring team also scored the tests. Any differences were resolved by the leader of the monitoring team.

The open ended question coding was done using the following instructions (to create the 2 digit code):

First digit. If the answer is correct the first digit is 1-5. For a wrong answer, first digit “7” is used. If there is no answer, use “9.” For answers that are correct, the first digit represents the score. For example, “21” means the score given is “2,” and “31” would mean it is “3.” Three represents a better performance than two.

Second digit. This digit shows diagnostic information. The second digit is 0-9. For example, “21” and “29;” “29” shows the performance is far from correct, but the “20” shows that of all the answers that received a “2” for the first digit, this one was the best.

We created coding instruction books for each test. One math test is shown as an example:

Table No. 11 Example math open ended question scoring

Code Answer Test Number: M5D4_06 Version Number B4-B28

Test: Connect the given dots to create as many lines as possible Correct Answer 20 Made correct lines, wrote the table number wrong 21 Made wrong lines, table number was right 29 none Incomplete 10 1 line was correct, and the answer was right 11 2 lines were correct, but answers were not written 12 2 answers were right, but there were no lines 13 Other incomplete answers

Page 30: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 30

Incorrect Answers 70 Connected the lines correctly but answered wrong 79 Wrong answer (erased, illegible, bubble incompletely filled, etc…) No answer 99 Missing

Page 31: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 31

CHAPTER 2. FINDINGS AND RESULTS OF THE NATIONAL ASSESSMENT OF MATEMATICS AND READING

2.1 SUMMARY OF THE FINDINGS AND RESULTS OF OF THE MATEMATICS AND READING

Percentages of the mean score of students’ performance in mathematics were 45.4% and 41.2% in reading comprehension by the National study of primary education assessment. Analysis of outcomes of reading assessment has been carried out in two ways, first including performance results of international test items, and second, without including the performance of international reading test items. The average percentage of performance scores was 48% in considering assessment results without including international test items for reading. Analysis of reading items and general outcomes have been presented in this report with the scoring which has included international test items, however, international test items have not been included in the provision of analysis and explanation of factors associated with learning achievement at the national level hence it was required to do so.

Table No. 12. Mathematics and Reading learning achievement of Students

Percentage Standard error Mathematics 45.4 0.2

Reading (with international items) 41.2 0.2

Reading (without international items) 48 0.2

State examinations in secondary education schools in Mongolia have been conducted since 1997 and until 2005 these examinations had been administered by independent organizations such as the State Professional Inspection Authority and later by the Education Evaluation Center, which were responsible for developing exam themes and items. Comparison of learning achievement of students in primary education mathematics and reading has yielded similar results with results of those previously undertaken examination results. The Education Evaluation Center started to administer state examinations for secondary education schools in the country since 2005 in accordance with the uniformly established content and selected theme, hence it provides a real possibility for viewing the results of this learning achievement assessment in primary education math and reading in a very objective and unbiased manner, and that it is not an independent assessment.

Results of our study have yielded very similar results with the findings of previous studies carried out in Mongolia to assess learning achievement.

In comparing learning achievements between urban and rural schools, student performance from the capital city has been 13.6% higher than that of rural students.

Table No. 13. Link between the learning achievement of urban and rural students

Reading Mathematics

Percentage Standard error Percentage Standard error

Urban 47.4 1.08 54.06 0.43

Page 32: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 32

Rural 37.5 0.89 40.4 0.34

Learning achievement of students from the capital city (urban)6 was higher by 14% from the learning

achievement of rural children in mathematics, and by 10 % in reading. The following Figure presents the performance of learning achievement of rural and urban children in

mathematics and reading separately.

Figure No. 3. Mathematics and Reading: Urban/Rural students

Mathematics Reading Urban Rural Urban Rural

Number of students Number of students Considering results of learning achievements, the learning achievement have varied and differed.

Table No. 14. Link between Learning achievements of students and location

In comparing the learning achievement of students living at bag with learning achievements of urban children, their performance results have been lower by 18.7% in reading and by 23.6% in mathematics which indicate that the quality of teaching at rural and urban schools have a huge gap. The following figure shows how learning achievements vary according to the location of students.

6 Students from the capital city are accounted as urban students.

Mathematics Reading

Percentage Standard error Percentage Standard error

Capital city 54 0.4 47.9 0.3 Aimag center 47.2 0.5 42.1 0.4 Soum center 35.8 0.4 35.1 0.4

Bag 30.4 1.3 29.2 1.5

Page 33: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 33

Figure No. 4. Dependence of Mathematics and Reading from Location

City Aimag Soum Bag City Aimag Soum Bagh center center center center

Number of students Number of students

Learning achievement differences associated with location has been analyzed and in later chapter analysis

details will be provided on factors thought to have impact on learning achievement because of place of living and studying.

The fact that 49.9% of participants involved in the study were girls and remaining 51.1% were boys have

provided wide opportunities for further comparative study.

Table No. 15. Link between Mathematics and reading and gender

Gender Mathematics Reading

Percentage Standard error Percentage Standard error

Boys 45,5 0,4 39,0 0,3

Girls 45,4 0,3 43,3 0,3

Performance of girls and boys in mathematics has been very similar whereas in reading girls have

performed better by 5 % in comparison to boys when learning achievement of students was compared by gender.

Page 34: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 34

Figure No. 5. Comparative Learning Achievement by Gender

Mathematics Reading Boys Girls Boys Girls

Number of students Number of students In considering learning achievement related with the age of school entrance, the learning achievement of

students who have entered at age 6 have been higher than students who have entered at the age of 7 and 8.

Figure No. 6. Link between Learning Achievement and age of school entrance

From this it can be concluded that the learning achievement would be higher if students enter schools at

earlier age.

Page 35: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 35

2.2 FINDINGS AND RESULTS OF MATHEMATICS

Mathematics part of the survey booklet had included 28 items. Twenty two items or 78.6% were multiple choice items, 6 questions and tasks or 21.4% were open ended questions. The survey booklet had 3 basic versions and by changing the sequence of the test items a design for allocating survey booklets has been produced and therefore, 9 versions of the survey booklet has been obtained. The true level of primary education mathematics in reality has been estimated by carrying out analysis of test item performances, for instance, by performing analysis and estimation of each general level, each domain and cognitive levels.

The performance percentage and frequency of dependency have been closely related with normal distribution when the mean performance score of students involved in the study has been compared with the total number of students. It has shown here that design and calibration of test items were congruent with the given requirements. The percentage of the mean performance scores of mathematics items has been 45.4 % which is lower than the average level of standard implementation.

Figure No. 7: Distribution of Performance Students (by number of students)

Score Results of the survey have shown that 34% of students’ have performed of 60% and above, and 75% of

them had a mean percentage of performance scores below 60%. An analysis has been carried out on how the performance would be rated when determining the type of

item and its responses. An average score of a selective number of items have been 11.2% or successful performance percentage has been at 51.2%. 0.5% of students have been able to perform the highest score which was 22.

The mean score of 12 open ended questions and tasks including filling, matching, calculating, creating and drawing tasks were 4.1% or 34.8% of the performance success, and 88.9% of students have performed with a successful performance lower than 60% . That has shown that the performance ability of open ended items of students involved in the primary education in which student had to perform such mental skills and procedures as performing mathematical calculation, drawing diagrams and providing visual prompt, has been poor if compared to the performance ability of selecting type of items. In performing of an IRT analysis of each item, there were no items which had not met the requirements imposed, or with poor discriminate ability or unfit level of item difficulty. However, it is needed for 5the grade students to acquire skills and abilities to perform items which would require such mental skills as problem solving and creative thinking.

Page 36: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 36

Table No. 16. Performance of Students in Each Mathematics Section

Section Percentage share of students with performance of less than 60%

Percentage share of students with performance of 60% and above

Percentage of mean performance

Number, numeracy (AD) 55 45 53.4

Algebra (BD) 77.6 22.4 33.5

Geometry (DD) 65.4 34.6 50.3

Probability, statistics (CD) 92.3 7.7 29.3

Students test performance has shown that the majority of students or 96.5% of them have performed

unsatisfactory to problems in probability statistics. The fact that the students’ ability to perform problems in geometry, and numbers and numeracy domain has been better than the ability to perform items from other two domains might be more likely to be related with the number of class hours, content of a textbook and methodology of teachers. Hence teachers of primary education are in need of mastering each domain of standard content and teaching students to acquire the required knowledge and skills and learning methods and skills to perform items and tasks.

Table No. 17. Performance on Each Levels of Mathematics Items

Level

Percentage share of students with

performance of less than 60%

Percentage share of students with performance

of 60% and above

Percentage of mean performance

Unstructured (2L) 45.2% 54.8% 67.6%

Multi-structural (3L) 61.2% 38.9% 52.1%

Relational (4L) 90.5 % 9.5% 28.6%

Results have shown that students have performed below average on items studied in previous grades and

have performed better in executing multi-structural items. That indicates that the abilities of student to explain, define, make lists, visualize and make decisions have been acquired relatively well. Performance of relational items has been executed very poorly. This has shown that the abilities and skills of student to perform items of relative difficulty that require the performance of mental procedures as analyzing, comparing, revealing of cause, determining relationship and patterns have been weak. Here it indicates the need for mathematics teachers to improve their students’ abilities and skills to perform items and tasks that would develop their mathematical thinking and reasoning, experiences of mathematical modeling and identification of patterns and teach their students the methods and approaches to solve these kinds of problems and items in addition to training students to acquire knowledge and skills outlined in mathematics standard.

78.4% of students who have successfully performed level 2 (2L) items of mathematics have also successfully performed on problems and items from number and numeracy domain. Whereas students who were able to resolve problems in probability statistics have performed items in 2 levels with the performance success rate of 40,3 %s. It has been evident that students have been able to perform unstructured items of number and numeracy domain. Students who have been able to perform successfully items in level 4 or the tasks of most difficult level have performed well all items in all domains successfully.

Page 37: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 37

The mean performance percentage of students who have demonstrated a success rate higher than 90% in mathematics was 93.6%. These students have demonstrated an average successful performance rate of 69.7% in reading. Students who study well or got higher assessment rates in mathematics have shown satisfactory assessments in reading as well. Mathematics education predominantly develops mental skills as summarizing, analyzing and comparing. It also proves the belief that students who have a solid basic knowledge in mathematics can successfully study in any other subjects. Therefore, greater yet increased attention should be rendered to the teaching of mathematics in primary education.

It is assumed that the general level of mathematics education have been lower than the standard implementation average that might be related with the fact that education standards, content framework and teaching curricula have just commenced their implementation shortly after their renewal. It also proves our assumption that learning achievement of mathematics is directly related with reading. A student who is able to read and comprehend well indeed has also been doing well in mathematics, and his/her learning achievement is better and competencies are acquired at higher levels. Therefore, it clearly indicates that students should be taught first of all skills and abilities to read and comprehend, as well as read fast and properly, and be able to explain and express what they understand. The aim of the analysis of results and outcomes of this assessment study has been the efforts to identify reality and ways of improving learning achievements further, but not assess someone.

2.3 FINDINGS AND RESULTS OF READING

Results of the assessment study have been analyzed to identify the general level of reading of students and their results in each domain and cognitive levels.

The reading section in each survey booklet included 23 items to test listening and reading skills and writing a story on a visual prompt provided and additionally, one set of 12 task items from the international PIRLS 2006 study of learning achievement assessment. The international section included 6 multiple-choice questions and 6 open ended questions and tasks related with reading of a text and test summarizing skills of students. Analysis of the results of reading items could be carried out in two distinct ways.

First, analyses of the results have shown that the mean percentage of performance scores, including international items, was 41.2%.

Figure No. 8. Distribution of Performance Students (inclusive of international test items)

Score

Second, if the percentage of the mean performance scores were estimated without including international

test items the mean percentage of performance scores was 48%.

Num

ber o

f stu

dent

s

Page 38: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 38

Figure No. 9. Distribution of Performance Students (not inclusive of international test items)

Score When the results of the reading of primary education which are shown in Figures 8 and 9 above are

compared, and reveal that students have not been able to perform well in the international reading items. If international items are considered alone, the mean percentage of their mean performance score has been 31.6% and it is 16% lower than the analysis of performance results estimated without including international items. That occurrence has indicated that in Mongolian language classes at the primary education level we lack performance of similar tasks and items with the same design and calibrations and content which would require the same level of abilities. It has also been assumed that the below average percentage of performance scores of international items might be related with the specifics of national teaching structure and methodology.

It has also been considered how the type of items performed by students is influencing the results of the study. 26 questions or 74.3% of the total of 35 items of the reading section of the test were multiple choice questions and 9 questions or 25.7% were open ended. The mean performance score of open ended questions were 46.7% and the mean performance score of open ended questions were 36.3%.

25% of students involved in the assessment study have performed 60% and above in the multiple choice section. In carrying out performance analysis of 9 open ended questions, 18.3% of all students have performed 60% and higher and these 9 questions included tasks to determine the main ideas and topics of a text, the main and underpinning concepts of the passage, defining the main points and ideas and do creative writing. It indicates that students in primary education have been performing poorly in obtaining skills such as analyzing texts, determining the main ideas and defining its structure, creative writing in the proper style, structure and format, summarizing texts.

Test included 5 questions to assess listening and results have shown that the mean percentage of the performance scores were 47.3%. Students have achieved 53.9% of the mean performance scores in performing 15 tasks in reading.

To assess students’ creative writing skills they were each given a drawing titled “How I learned to play chess” (see below), and topics «My Friend», «Summer Vacation» and a task of a creative writing within 35-40 words. The drawing has been plotted to trigger stimuli in students to engage in creative writing within a wide range of themes including one as the difference between urban and rural lifestyles, nomadic and settled life, industrial and environmental, developmental conditions of the country, property, ownership, etc.

Page 39: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 39

Picture No. 1. The picture is used in the survey

It should be noted that the mean percentage of students’ performance scores in creative writing skills were presented at 41.9% indicating that creative writing is a part of language use and application has been executed by students was not satisfactory. Inclusion of this type of item in national assessment for the first time has been both innovative and potentially applicable approach to be practiced in teaching at primary education levels. That indicates that teaching of meaning and structural domains have mostly prevailed over to modification and cultural domains which largely require language application but ones which have been mostly neglected in teaching of Mongolian language at primary education level. Students not have been able to acquire creative writing skills due to an insufficient amount of time allocated for performing creative writing tasks in the last years and as a result led students to do poorly in gaining such skills as expressing of what was understood through reading, writing proper story using grammatically correct and structurally sound language.

An assessment scheme to evaluate skills and abilities such as expressing the main idea from a selected theme and content framework; writing grammatically proper and structurally sound language have been used in order to define creative writing skills of students to write a story based on a visual.

The mean percentage of performance scores achieved by students in an integration of theme and contents, and creative writing skills used as the initial criteria of essay/story writing was at 39%; 40.2% for correct use of words and structure; and 45% for grammar respectively.

It is required and needed to develop skills for children to express themselves in writing and speaking at primary education level. However, teachers pay less attention to creative writing in their teaching and assess students by their expository writing in reality. In other words, students are taught mostly knowledge and theory and less attention is paid to skills related with the actual use of language as creative writing and sound writing skills or use of correct, relevant words and structure, grammatically correct language. That point has been emphasized repeatedly during interviews held with aimag methodologists in charge of primary education.

The content of each domain has been taken into account in designing survey blueprint. (AD-58 items 89.2%, BD-4 items 6.2%, CDD–3 items 4.2%)

Page 40: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 40

Table No. 18. Percentage of mean scores of performance of students in each language domain

Domain Percentage share of students with performance of less than 60%

Percentage share of students with performance of 60% and above

Percentage of mean performance

Content (AD) 80.4 19.5 41.4

Structure (BD) 65.2 34.7 36.7

Modification, culture (CDD) 75.5 24.4 41.9

Items to assess skills such as recognition of passages from structural domain, finding and distinguishing

text parts have been performed unsatisfactorily when the mean percentages of performance scores of students in each language domain were analyzed. Items of content domain have been included most importantly to assess learning achievement of primary level students in reading. This was not emphasized and attention was not paid within this assessment study. Students who have been able to perform items from content section well were also skilled to execute well items from other sections that were satisfactory. It means that if a text is well understood or comprehended by students then they are able to use their previous knowledge in new conditions. Therefore, particular attention has to be paid to the delivery skills to students for listening and reading of texts, expression of what has been comprehended and text analysis. In addition to that, students should be engaged in the performance of items to check their mastery of such aforementioned skills.

Primary education teachers of Mongolian language should study each domain of the standard content and be concerned with teaching and the delivery of information and knowledge systematically, and present information in a well-balanced manner to teach every domain and have their students acquire the critical and creative thinking skills.

Uni-structural test items included tasks such as finding the main ideas and subjects of a text, identifying additional or underpinning ideas of a text, finding ideas in each text sections, defining the main points and ideas, finding meaningful links between language units, defining the purpose of sentence utterance and recognition of a passage. Multi-structural test items included tasks to assess student’s skills as distinguishing between themes, main and additional ideas of a text, finding of main/core and key words, phrases, distinguishing passage of similar and opposite meanings of sentences. Relational test items included tasks to reveal skills as determining the sequence and order of a content, correct matching of relational conclusions, finding true and false conclusions and responding to questions of ‘who’, ‘why’, ‘when’, ‘where’, and ‘how’. Extended abstract test items included tasks to assess skills such as requiring students to write a conclusion to an extended abstract item, fill-in-the-blank, creative writing and expressing the main ideas of a text with an idiomatic phrase.

Table No. 19. Percentage of Mean Performance Scores of Each Level in Reading Items

Levels Percentage share of students with performance of less than 60%

Percentage share of students with performance of 60% and above

Percentage of mean performance

Uni-structural (2L) 86.4 13.6 36.7

Multi-structural (3L) 56.1 43.9 50.5

Relational (4L) 67.9 32 47.4

Extended abstract (5L) 79 20.9 42.3

Students have performed previously covered or recall level items at below average than the standard

implementation, whereas multi-structural items have been performed above average. From there it shows that students have been performing relatively well in showing skills such as comprehending hidden meanings by comparing several notions and recognition and identification links between them. Students have shown that they

Page 41: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 41

have weak skills in performing items of relative difficulty such as analyzing and comparing, revealing of a cause, finding relationship and patterns, providing conclusions and creative writing. It is recommended that teachers should pay greater attention in teaching reading to their students of items and tasks that would help their students develop their skills of reading, abstract thinking, observation and identify patterns and having their students to perform explanatory and creative writing tasks.

Students have performed reading items of uni-structural level unsatisfactorily (2L). Uni-structural level test items have been mainly tasks which included skills such as finding main ideas and subject of texts, and defining features, and students have not performed these type of international items successfully as well. The mean percentage of mean performance scores has been 43.5% when performance scores of international items were deducted. It indicates here that reading skill is somewhat incomplete and poorly developed. Students who have performed successfully the most difficult items of level 5 in reading have also successfully performed test items of other domains as well.

It is assumed that the general level of reading being lower than the standard implementation average might be related with the fact that education standards, content framework and teaching curricula have initiated its implementation shortly after its renewal. It also proves our assumption that learning achievements of reading acquired by students’ are directly related with the level of mathematics education. A student who is able to read and comprehend well has also been performing better in learning achievement, and has potentially higher possibilities to acquire information and knowledge. Therefore, it clearly indicates that students studying at the primary education level should be taught properly and master the following skills and abilities such as high level of listening and reading, and sound explanatory and expressive skills.

If performance of items of reading are analyzed, skills of 4L and 5L levels such as the ability to concentrate one’s attention on what is being read or listened to, distinguishing between what is important and what is not, comprehension of links between content parts, finding relational causes, and comparative analysis to provide conclusive remarks by comparing own viewpoints with text ideas have been acquired by students relatively poorly. In general, it is observed that teachers, parents and guardians do not pay sufficient amount of attention to the development of reading skills of their students and children. Adults have been emphasizing their children’s correct and fast reading techniques but not to reading which has been supported by assessment study findings.

2.4 RESULT OF STUDY IN MATHEMATICS AND READING OF KAZAKH STUDENTS

Kazakh represent the largest minority group in Mongolia. The majority of ethnically Kazakh Mongolians live in Bayan-Ulgii aimag, and they comprise of 4.4% of the total population. Kazakh Mongolians consist of 5% of the total population between the ages of 0-147. 10% of Khovd aimag(province) are Kazakh, most of which live in Khovd soum (county). One of the purposes of the study was to compare Kazakh language students to Mongolian language students, For this reason, 36 Kazakh language schools in Bayan-Ulgii aimag, and one middle school in Khovd aimag, Khovt soum were included in the study. This includes almost all Kazakh language schools, and comprises 15.3% of total test takers. Math and Civics test conducted by EEC from 2004-2006 was not conducted in Kazakh language, which may have accounted for their lower scores. Hence, for the current study, 1,000 test booklets were translated into Kazakh language.

Page 42: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 42

2.52.5 Picture 2. Kazak test participant’s location.

The average test score for Mongolian language students was 43.3, and the average for Kazakh students was 32.9. Male Mongolian language students’ average score was 42%, and male Kazakh language students’ average score was 32.7. Female Mongolian language students’ average score was 44.6%, and female Kazakh language students’ average score was 34.2. The average math score for Kazakh language students is 36.4% for males, and 36.1 for females. These numbers are 9.7-11.4% lower than Mongolian language students’. The average math score for the total Kazakh students was 36.2%, which is 9.5 % lower than the Mongolian language students. The average Reading score for Male Mongolian language students is 39.5, and 43.8 for females. Kazakh language students’7 average Reading score was 28.7 for males, and 32.9 for females, which is 10.8% lower than the Mongolian language students.

Figure No. 10. Comparison of Learning Achievement of Students who study in Mongolian and Kazakh languages in Mathematics and Reading Comprehension

Mathematics Reading Comprehension

In Mongolian language In Kazakh language In Mongolian language In Kazakh language

Number of students Number of students

The average Reading score was 43.2% for Mongolian language students, and 31.7% for the Kazakh language students.

Table 20. Below is the result of students’ academic success by location of school.

Location

Mathematics Reading

Mongolian Language Students

Kazak Language Students

Mongolian Language Students

Kazak Language Students

7 Analysis of the status of Kazak Children’s’ education, Great Britain Children’s foundation, 2005

Page 43: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 43

Unit 30.1 31 30.1 27.8

County 35.6 38.1 34.9 32.9

Province Capital 47.9 35 42.5 29.3

Capital 54. 0 - 47.4 -

Academic performance is higher for Mongolian Language students in more densely populated areas, but academic performance for Kazakh language students are higher in county capitals. The average score for the multiple choice math questions was 40.3% for Kazakh language students, which was 11% lower than Mongolian language students. The average score for Kazakh language students for the open math questions was 25%, which was 10% lower than Mongolian language students. The average score for the multiple choice reading questions was 36.1% for Kazakh language students, which is 10.8% lower than Mongolia language students. The average score for Kazakh language students for the open reading questions was 25.7%, which was 10.8% lower than Mongolian language students. The average score for individual questions was 28.6% which is 9.5% lower than Mongolian language students, and the average score for grouped questions was 34.2%, which is 13.8% lower than Mongolian language students. Average score for reasoning questions was 24.8% which is 16.6% lower than Mongolian language students. Average score for abstract questions was 37.6% which is 7.7% lower than Mongolian language students. The average score for the Language meaning questions was 22% for Kazakh language students, which is 10.8% lower than Mongolia language students. The average score for Kazakh language students for the Language Structure questions was 17.5%, which was 19.1% lower than Mongolian language students. The average score for Figurative Language and Idiom questions was 35.4% which is 5.7% lower than Mongolian language students. The Kazakh language students’ average composition score for the essay based on a picture was 34%, the style score was 33.3%, and the spelling score was 39%. This shows that we need to pay more attention to improving Kazakh language students’ language ability. The average score for the individual type math questions was 50% for Kazakh language students, which was 15% lower than Mongolian language students. The average score for Kazakh language students for the group type math questions was 41.4%, which was 10.9% lower than Mongolian language students. The average score for the math reasoning questions was 22.7% for Kazakh language students, which was 16.6% lower than Mongolian language students. For the individual type math questions, both Kazakh and Mongolian language students performed relatively well when compared to performance on other types of questions. The average score for addition Math questions was 42.3%, which is 10.7% lower than Mongolian language students. The average score for algebra math questions was 27.1% for Kazak language students, which was 9.1% lower than Mongolian language students. The average score for Geometry was 41.1, which is 8.9 % lower than Mongolian language students. This comparison shows us that Kazakh language students score lower on all types of test than Mongolian language students. This requires us to pay more attention to the quality of education for Kazakh language students, and take action to improve the situation. We introduced the level of familiarity with the test subject, and categorical analysis of education.

This study indicates that Kazakh language students have many barriers preventing high academic performance, including: school environment, dormitory conditions, undeveloped infrastructure, unavailability of books in their native language, teachers’ relatively low education level, no standardized evaluation of schools far from county and unit centers, not having a consistent advisor, inadequate training tools, high education costs, and a

Page 44: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 44

low standard of living. These barriers were discovered from feedback from school administrators, head teachers, teachers, parents and guardians, and students through the process of the study.

2.5 ITEM ANALYSIS OF MATHEMATICS

In addition determining learning achievement levels of students, and identifying ways of how the content

of primary education standards, curriculum framework of teaching content, training program and methodology of teachers have been affecting learning, item analysis has also been aimed at providing analysis of items own qualities to reveal learning achievement of students. Test items have been developed based on the cognitive SOLO taxonomy approach and item data has been analyzed by RASH model analysis of Conquest software. In relation to each content framework, structure and quality, the item analysis has been carried out for each performance item, for each study booklet, for each section, for each level and type along with analysis of their interrelations.

The mean percentage score for each booklet has been quite similar when the performance of mathematics items has been considered for each booklet. It could be concluded that the development of questions and tasks in each survey booklet has been carried out at a similar level.

Figure No. 11. Learning Achievement in Mathematics by Each Booklet

Survey booklets

Each item has been analyzed in terms of its level of difficulty (P) and discriminate (D). Examples of such analysis are given for survey booklet No.4.

Page 45: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 45

Figure No. 12. Item Difficulty Analysis of Booklet No.4

Out of all test items included in Survey Booklet Four, 4 items were difficult, 2 items were easy and 22 items were of average level when the level of item difficulty for survey booklet No.4 has been considered. The most difficult item No. М5В3_06 was an item from number and numerical section, with multi-structural design of multiple-choice questions. That task was aimed to test the skills related with locating coordinates of a point on a frame of reference. The easiest item No.М5S2_03 belonged to number and numerical section, with uni-structural design and open ended questions, which was aimed to test student’s comprehension, correlation/equation and comparison skills of data given in a bar diagram and to identify their differences.

Figure No. 13. Discrimination of Items in Booklet No.4

4 test items had poor discriminated abilities whereas other test items have been developed with good

discriminated ability when this criterion of items has been considered. When considering an item No. M5С4_06 with poor discriminated ability, it was an item that belonged to open ended questions, from probability section and of relational level item. This item has also been a difficult task. It was a task to find numbers which would meet concrete terms and conditions. Also item No. M5С4_05 which also had poor discriminated ability belonged to the probability section was a relational and multiple-choice question. Since it has been an item to test thinking skills of students and was of relative difficulty might have been performed unsuccessfully.

32.1% of the items of mathematics were bridge items. 22.2% of these were international questions and 66.7% were combined. The skills of formalization, to change the information as picture and diagram, finding the relation, to review, to count the possible opportunities were determined by bridge items.

Page 46: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 46

As a result the students got an average of 5.17 points instead of a maximum of 10 points. 6.2% of all students completed all bridge items correctly and 2.4% of all students completed all bridge items wrong. 44.9% of students completed the bridge items above 60%. The students completed daily regular problems above 80%.

To summarize, items that belonged to the probability section and to open ended questions were computed by students inadequately. It might be related to the fact that students have have not understood the terms of math problem given or they have not been performing these type of problems often. Futher, in teaching primary education mathematics students should be more engaged in performing math problems of open ended, relational and extended abstract levels to develop their mathematical reasoining and thinking.

2.6 ITEMS ANALYSIS OF READING

The performance of reading items have been analyzed in relation to each item, survey booklet, section, level and type along with the analysis of their interrelations.

The mean percentage of performance scores for each booklet has been quite similar when performance of

reading items for each survey booklet were compared. A conclusion can be made that test items in each study booklets have been developed at similar level.

Figure No. 14. Performance of Reading Items (in each booklet)

Survey booklets

Each item has been investigated by the level of its difficulty (P8) and discriminated ability (D9). Item difficulty level has been shown below examples of items taken from survey booklet No.6.

8 Difficulty 9 Discrimination

Page 47: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 47

Figure No. 15: Item Difficulty in Booklet No.6

In considering items in Booklet No.6, it has revealed that 6 items were of difficult levels whereas 7 items

were easy. In analyzing the 6 difficult level items of multiple choice, 3 items were from international test and 3 items to test listening. The poor performance achievement of students in doing items for listening is most likely related either the type of items were never performed or with the weak level of their listening. Also poor performance of international test items might be related either with the incomplete understanding of the terms and conditions of item task or the idea of a text. The majority of students have not acquired the reading that would require reflective thinking on what has been read, comparative correlation with own previous knowledge and viewpoints, provision of personal judgment, and recalling or retelling of a text to oneself and others and communication skills. The most difficult item No. R5A1_07 has been to test the skills of finding additional, underpinning idea of a text that belonged to the language content section, with a uni-structural level and of multiple choice questions. However, the easiest item No. R5В3_02 has been from content domain, with relational level, multiple choice questions designed to identify the order and sequence of text parts. These two items were international test items.

Figure No. 16: Discriminant Ability of Items in Booklet No.6

Items No. R5A1_07 and R5A1_14 were with poor discriminate ability, but were also the most difficult tasks. These two items were multiple choice international test items and students most likely have guessed their answers due to not being able to understand item terms and main idea of a text.

17% of the items of reading were a packet of items with The Clamp of clay, which from international program PIRLS and it was bridge item. Determination of the sequence and structure of first parts and finding of main, additional and aimed meaning of the texts t were determined by these open ended 6 and multiple-choice 6 items.

An analysis of how the performance of reading items were carried out by taking into account such basic learning activities as selection of reading themes, seeking necessary materials, reading for oneself or to others, reflective thinking over what has been read, comparative correlation with own previous knowledge and viewpoints,

Page 48: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 48

and provision of personal judgment, application in own life and practice, memorizing of a selected text, performing dictation, creative writing/paraphrasing, similar/imitative writing, essay writing, selection of a essay topic, seeking related materials, drafting, refining and polishing up writing, finalization of a writing by analysis of own and others writing and provide an assessment.

2.7 SKILL LEVELS OF MATHEMATICS

Levels of mathematics and reading have been established based on study findings of international learning

achievement studies establishing benchmarks. A comparative analysis of national studies carried out in countries as Vietnam10 and Serbia11 have been compared and analyzed in establishing benchmark levels of mathematics skill.

All mathematics and reading items have been analyzed by the ConQuest program to identify their difficulty

levels12. In devising levels of mathematics skills all items included in the assessment study have been listed in accordance with their level of difficulty and then were divided into 5 levels and mathematics skills associated with each item have been identified. These levels were meant to establish in detail the acquisition of knowledge and skills by students required in primary education. This enables to identify what students could do and what they could not do and also provides a possibility to establish levels of skills in mathematics. Further, it requires study and research on these issues in detail more comprehensively.

Table No. 21. Competency Levels in Mathematics

Competency Levels

Mathematics skills

Level 1 Writing decimals in numbers/digits; putting numbers in ascending order; applying feature of multiple-digit numbers to be divided into four; dividing simple shapes into equal parts; determining of patterns in positioning of shapes; to be able to calculate the area of a rectangle;

Level 2

Creating expressions with unknown element; reading and understanding information given by column/bar column diagram and finding the meaning of number expression; writing decimals in numbers; reading/interpreting and comprehending of data given in bar diagram; finding meaning of a number expression; writing decimals in figures, recognizing and naming angles; be able to calculate volume and interdependence between measurements of a right-angled parallelepiped;

Level 3

Writing of mixed numbers and calculating their sum; finding the distance between points using scale; calculating of unknown element/quantity in an equation; calculating quantity of an expression with unknown element; calculating a number and matching to a percentage; be able to estimate repetition and identify interdependency;

Level 4 Finding number of all possible segments that can be established when connecting between all given points; transferring units; determining a variable by using given conditions; be able to calculate problems that are interdependent;

Level 5

Counting number many multiple-digit numbers; writing down all possible three-unit numbers which would maintain truth of given terms; finding out coordinates of a point on frame of reference; to be able to solve a math/word problem with two and more steps using interdependence of time, area and weight units;

10 “Reading and mathematics Assessment study” Volume 2., The World Bank, August 2004 11 Grade Three national assessment of student achievement, Belgrade, Jan 2005 12 Level of Difficulty is performance results of students drawn by a program. Here it is defined in accordance with IRT theory.

Page 49: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 49

Figure No.17 below shows how all items of assessment study have been divided into 5 difficulty levels.

Figure No. 17. Classification of Mathematics Items by Competency Levels

The above classification shows that students have acquired skills in levels 1-3 better than those in levels 4-5. In

other words greater attention should be paid in teaching of mathematics to the development of those relevant skills in students needed to determine number of all possible segments that can be established when connecting between all given points, transferring units, determining a variable using given conditions, solving problems on motion, solving interdependent problems, counting number of multiple-digit numbers, writing down all possible three-unit numbers which would maintain truth of given terms, finding out coordinates of a point from a frame of reference; to be able to solve a math/word problem with two and more steps using interdependence of time, area and weight units.

Table No.13 shows how the level of difficulty of items used in assessment testing relates to skills which have been sought to be revealed by that item and to the performance percentage of that particular item.

Table No.22. Mathematics Skills Matching Mathematics Items and Performance Percentage

№ Number

Item Number

Difficulty Mathematical skills Performan

ce 1 15 M5S3_07 -1.853 Finding the area of a rectangle; 80.1 2 61 M5C3_07 -1.705 Identifying the sequence in ordering patterns of geometric

shapes; 77.9

3 42 M5C3_04 -1.699 Identifying the sequence in ordering patterns of geometric shapes;

77.8

4 23 M5C3_01 -1.614 Identifying the sequence in ordering patterns of geometric shapes;

76.5

5 10 M5S2_02 -1.577 Interpreting and understanding bar/column diagram; 76.1 6 13 M5S3_05 -1.427 Dividing a square into four equal triangles; 73.7 7 36 M5A2_16 -1.26 Putting numbers in ascending order; 70.9 8 32 M5A3_12 -1.245 Writing decimals in digits/numbers; 70.7 9 8 M5A2_08 -1.202 Arranging numbers in ascending order; 69.8 10 51 M5A3_20 -1.168 Writing decimals in digits/numbers; 68.9 11 29 M5A3_09 -1.132 Recognizing/identifying feature of multiple unit numbers to be 68.8

Page 50: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 50

divided into 4; 12 55 M5A2_24 -1.125 Putting numbers in ascending order; 68.3 13 64 M5D3_07 -1.102 Matching angles with names; 82 14 48 M5A3_17 -1.1 Recognizing/identifying feature of multiple unit numbers to be

divided into 4; 67.6

15 26 M5D3_01 -1.061 Matching angles with names; 81.2 16 45 M5D3_04 -0.985 Matching angles with names; 80 17 7 M5A3_07 -0.962 Finding the meaning of a number expression; 65.4 18 4 M5A3_04 -0.911 Writing decimals in digits/numbers 64.5 19 54 M5A3_23 -0.9 Finding the meaning of a number expression; 64.2 20 1 M5A3_01 -0.882 Recognizing/identification of a feature of multiple unit

numbers to be divided into 4; 63.9

21 35 M5A3_15 -0.853 Finding the meaning of a number expression; 63.5 22 14 M5S4_06 -0.819 Represent given information in a table with pie chart; 62.8 23 11 M5S2_03 -0.636 Understanding and interpretation of information provided by

bar chart, comparison of numbers and determining of differences;

59.2

24 50 M5A3_19 -0.495 Based on the estimation of repeating cycle of week days to identify weekday order of a concrete day;

56.2

25 9 M5S3_01 -0.376 Making up an expression with a variable element; 54 26 57 M5B4_12 -0.344 Finding out unknown number using algorithm; 53.4 27 65 M5D4_08 -0.303 To calculate volume and interdependence between

measurements of a right-angled parallelepiped; 53

28 31 M5A3_11 -0.287 Based on the estimation of repeating cycle of week days to identify weekday order of a concrete day;

52.2

29 27 M5D4_02 -0.285 To calculate volume and interdependence between measurements of a right-angled parallelepiped;

51.8

30 60 M5B3_15 -0.246 Calculating of unknown element/quantity in a equation; calculating quantity of an expression with unknown element;

51.6

31 19 M5B4_02 -0.027 Finding out unknown number using algorithm 46.7 32 40 M5B3_09 0.014 Writing down mixed numbers using given drawing and

calculating their sum; 45.9

33 59 M5B3_14 0.031 Writing down mixed numbers using given drawing and calculating their sum;

46

34 3 M5A3_03 0.079 Based on the estimation of repeating cycle of week days to identify weekday order of a concrete day;

44.7

35 52 M5D4_09*10

0.088 Determining the number of all possible segments that can be established when connecting between all given points;

69

36 38 M5A3_21 0.132 Finding a number to match with the percentage; 43.7 37 5 M5B4_07 0.133 Finding unknown number using algorithm 43.5 38 53 M5A3_05 0.214 Finding a number to match with the percentage; 42 39 34 M5A3_22 0.255 Estimating the distance between towns using scale; 41.3 40 46 M5A3_14 0.258 Estimating the distance between towns using scale; 41.1 41 33 M5D4_05 0.258 Calculating the volume and interdependence between

measurements of a right-angled parallelepiped; 41

42 41 M5A3_13 0.281 Finding a number to match with the percentage; 40.7 43 12 M5B3_10 0.297 Calculating of unknown element/quantity in a equation;

calculating quantity of an expression with unknown element; 40.2

44 22 M5S3_04 0.443 Finding the mean arithmetic indicator/number by using data 37.6

* Half of point used in evaluation when performances of items were estimated and these items were considered that has been performed to certain extent.

Page 51: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 51

given in bar column diagram; 45 6 M5B3_05 0.481 Calculating of unknown element/quantity in a equation;

calculating quantity of an expression with unknown element; 36.6

46 58 M5A3_06 0.531 Estimating the distance between towns using scale; 35.8 47 21 M5B4_13 0.539 Finding the smallest number out of unknown numbers by using

given terms and conditions; 36.1

48 20 M5B3_04 0.555 Writing down mixed numbers using given drawing and calculating their sum;

35.3

49 28 M5B4_03 0.556 Finding the smallest number out of unknown numbers by using given terms and conditions;

35.3

50 49 M5D4_03* 0.678 Determining the number of all possible segments that can be established when connecting between all given points;

73

51 18 M5A4_18 0.682 Solving a motion problem by transferring time measurement unit;

33.2

52 66 M5B3_01 0.761 Finding coordinates of a point on frame of reference; 31.6 53 17 M5S3_09 1.03 Finding the amount of money to be spend by using terms of

previous problem; 40

54 39 M5B4_08 1.069 Finding the smallest number out of unknown numbers by using given terms and conditions;

26.2

55 30 M5A4_10 1.099 Solving a motion problem by transferring time measurement unit;

25.9

56 47 M5D4_06* 1.189 Determining the number of all possible segments that can be established when connecting between all given points;

87

57 16 M5S3_08 1.337 Solving a math/word problem based on terms and conditions of previous problem by using interdependence of area and weight units;

22.4

58 2 M5A4_02 1.383 Solving a motion problem by transferring time measurement unit;

21.6

59 37 M5B3_06 1.531 Finding the coordinates of a point on frame of reference; 19.3 60 56 M5B3_11 1.546 Finding the coordinates of a point on frame of reference; 19.7 61 44 M5C4_06 1.717 Writing down all possible three-unit numbers which would

maintain truth of given terms; 17.7

62 25 M5C4_03 1.809 Writing down all possible three-unit numbers which would maintain truth of given terms;

23.3

63 43 M5C4_05 1.884 Counting and identifying correct number of three-unit numbers;

14.9

64 24 M5C4_02 1.916 Counting and identifying correct number of four-unit numbers; 14.8 65 63 M5C4_09 1.969 Writing down all possible three-unit numbers which would

maintain truth of given terms; 19.5

66 62 M5C4_08 2.043 Counting and identifying correct number of five-unit numbers; 14.1 It can be assumed that skills associated with level 1 and 2 have been acquired by students sufficiently since

the percentage of performance of non-difficult items (Item Difficulty is less than -0.8) was estimated at more than 60 %s as can be observed from the data given in table No. 13 shown above.

Distribution of item difficulties were mapped into the same scaling with distribution of students’ performance is shown in Figure No.17 below.

Page 52: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 52

Figure No. 18. Distribution of Difficulty Levels11 of Mathematics Items

Student number: 4750, items number: 66

Difficulty Students Number of items

THE STUDENTS ARE INVOLVED IN THE STUDY, THEIR AVERAGE PERCENTAGE OF THE

MATHEMATIC SKILL IS -0.14.

2.8 BENCHMARKS OF MATHEMATICS

Currently, there are no uniform levels, benchmarks established that are available in our country to assess mathematics and reading of primary level students as of yet. Based on the study findings, it attempts to establish a national benchmarks for primary education at the national level for the first time. Furthermore, in the future it is required to study this approach comprehensively and introduce an assessing achievement for every subject.

11 Order number associated with items where items are listed in order of descending item difficulty.

LEVEL 5

LEVEL 4

LEVEL 3

LEVEL 2

LEVEL 1

Average

Page 53: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 53

Benchmark is established to identify knowledge and skills that a certain percentage of students are able to both perform and acquire. Benchmarks in mathematics have been established using the mean percentage of students’ performance scores in accordance with international approaches.

Table No. 23. Benchmarks of Mathematics

Levels of achievement

benchmarks in mathematics

Percentage of

performers

Percentage of non-

performers Comment

Scaled score14

1 25 percentile 75% 25% 75 percent of students have

performed and 25 % of students have not performed

395

2 50 percentile 50% 50% 50 % of students have performed and 50 % of students have not performed

530

3 75 percentile 25% 75% 25 % of students have performed and

75 percent of students have not performed

650

4 90 percentile 10% 90% 10 percent of students have

performed and 90 percent of students have not performed

740

This benchmark is divided into 4 levels of 25, 50, 75 and 95 percentiles and mapped onto or matched with estimated scale scores. For instance, the 25th percentile indicates that 75 percent of students have achieved scores more than 395 and 25 percent of students have achieved less than the scaled score of 395; the 75th percentile indicates that 25 percent of students have achieved scores above 650 and 75 percent of students have achieved scores less than 650; and level of 90th percentile indicates that 10 percent of students have achieved scores above 740, and 90 percent of students have achieved less than 740. The percentage of students who have knowledge and skills that would match particular level is shown in Table below.

Table No. 24. Percentage of Students to Match Benchmarks Levels of Mathematics Achievement

Level of Benchmark

Percentage of student at that particular level

Percentage of students who have

achieved that particular level

Percentage of students who have not achieved

that particular level

Below than 25th percentile 15.3

1 25 percentile 48.2 84.7 15.3

14 This score is established based on results of this study.

Page 54: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 54

2 50 percentile 27.9 36.5 63.5

3 75 percentile 7.6 8.6 91.4

4 90 percentile 0.9 0.9 99

The table above shows that 15.3% of the total number of students have the knowledge and skills in mathematics to match below the 25th percentile of the benchmark; 48.2% would match the 25th percentile 27.9% match average level; 7.6% match the 75th percentile benchmark level; and only 0.9% match the 90th percentile of the mathematics benchmark. Here it is understood that the upper level benchmark contains the lower level benchmark.

Each benchmark level of knowledge and skills to be acquired in mathematics competency has been determined for each section below.

Page 55: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 55

25 PERCENTILE

Number and numeracy section – Reading and writing multiple-digit numbers; performing basic 4 operations; application of a feature of multiple-digit numbers to be divided into 4; computation of the interdependence between week days, months and year; writing decimal fractions in digits; reading and understanding bar diagrams.

Algebra section - expressing one member of a distance formula /distance travelled, speed, time elapsed/ by others;

Geometry section – finding or computing the area of a rectangle; dividing a square into four equal triangles;

Probability, statistics section – distributing simple shapes into groups and identifying patterns or core features;

Example item code: R5S2_02

Final school year examination of Bat is shown in bar columns. What score did Bat get in Mongolian language?

А. 80

B. 75

C. 70

D. 90

Students shall be able to understand basic notions of mathematics and perform simple operations at simple level.

Page 56: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 56

50 PERCENTILE

Number, numerical section – comparing multiple unit numbers and putting them in ascending or descending order; applying division features of multiple unit numbers; performing of combined operations a number expressions; drawing or obtaining information from a table.

Algebra section – creative writing of expression with unknown member; finding the height from the formula of defining volume of a rectangular parallelepiped;

Geometry section – naming, recognizing and matching an angle; finding or calculating the volumes of spatial simple shapes;

Probability section – Defining of a pie chart by tabular information provided; finding a given information from a bar diagram and performing of comparing.

Example item No. М5S4_06

Students shall be able to understand the interdependence of major notions acquired and determine and apply their patterns; and be able to perform arithmetic operations of multi-unit numbers with up to 9 units.

Page 57: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 57

75PERCENTILE

Number and numeracy section – solving a problem on motion; converting time measurement units; solving word problems with two and more step operations;

Algebra section – finding unknown element from given data, expressing unknown element with mixed number, and finding the meaning of the equation;

Geometry section – transforming geometric shapes following the given directions, determining the area of a rectangle and solving of two step problem;

Probability, statistics section – counting all possible options of a number given with a limited amount of number units.

Example item No. М5B3_01

Students shall convert and apply the given information from one to another, assess and judge similar and different features of the problem, solve compound problems, and find the size of simple geometrical shapes and their interdependence.

Page 58: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 58

Which one is the coordination for C point?

À. B. C. D.

90 PERCENTILE

Number, numeracy section – finding numbers to match given percentages and solve a two-step word problem;

Algebra section – to establish the interdependence between the unit of area and measurement of weight, finding unknown numbers in an expression represented through algorithm and checking it;

Geometry section – expressing the interdependence/correlation, similar and different aspects of geometric shapes; linking the coordinates with segments;

Probability, statistics section – counting number of true occurrences of concrete terms and number of segments that have linked points on a given horizontal surface.

Example item No. М5D4_06

Students shall be able to use their knowledge in unfamiliar situation and able to comprehend ways to consolidate and summarize; be able to apply knowledge acquired in reality; be able to identify ways to classify notions and definitions and be able to take into account several criteria in their estimations based on the given criteria; and find number to match given percentage and solve a two-step, word problem.

Page 59: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 59

Connect dots provided below. Fill in the table with the maximum number of segments you could count which would end up on these dots.

Table No.:

2.9 SKILL LEVELS OF READING

Reading items have been divided into 5 levels according to their difficulty based on students’ performance results which have been processed by the ConQuest software.

These levels have been established to comprehensively identify the skill levels acquired by students according to the primary education standards.

This has enabled to identify the reading of students, what they are able to do and not able to do. Furthermore, these issues have to be researched more comprehensively in the future.

In devising levels of reading, first, a list of item were produced hierarchically in accordance with the level of item difficulty that was then classified into reading levels in accordance with the knowledge and skills students have to acquire in accordance with primary education standards. In doing so it has been revealed that students have acquired the skills associated with levels 1-3 better than the skills in levels 4-5. In other words, greater attention must be paid for teachers have their students to acquire or develop skills such as correct matching of relational conclusions in finding the content order of a text, and provide answers to the questions like ‘who’, ‘what’, ‘when’, ‘where’, ‘why’, and ‘how’; writing concluding summaries, performing various tasks related to the completion of an idea in a sentence and creative writing and paraphrasing. However, skills could be classified in accordance with other possible variations and categories different from this classification into 5 levels.

Page 60: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 60

Table No. 25. Levels of Reading skill

Level Reading skill level

Level 1 To identify the subject, its features, location and place where action is taking place, timing, aim of the sentence, and finding key words;

Level 2 Finding relationships between the content of language units, and main and additional ideas or points of a text, determining similar and opposing meanings of paragraphs, writing in grammatically correct language and neatly;

Level 3 Finding the topic of a text, defining the interrelation of meanings between parts of a text, comparative analysis of features of a subject; expression of a main idea of a text through an idiom;

Level 4 Finding the sequence order of the content of a text, correct matching of relational conclusions; finding the correct conclusions; answering questions of ‘who’, ‘what’, ‘when’, ‘where’, ‘why’, and ‘how’;

Level 5 Writing a summary, complete , to finish a sentence by completing its idea/meaning, completion of an idea from , creative writing, completion of a sentence idea or point, creative writing and paraphrasing;

Table shown below provides how all test items have been listed in a hierarchical manner according to their

level of difficulties and then classified into 5 levels.

Figure No. 19. Classification of Reading Items into Skill Levels

Page 61: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 61

Table shows how each of the reading items have been listed in accordance with their difficulty indicators in ascending order or from easiest towards the most difficult. The first column indicates the number associated with an item.

Table No.26. Skills Associated with Items

№ Number Items Difficulty Skills Text name Performance 1. 42 R5A3_06 -1.719 Finding the main idea of a text A teenage boy 81.1 2. 49 R5A4_03 -1.558 Distinguishing similar and opposite

meanings of sentences A teenage boy 78.5

3. 51 R5A4_05 -1.431 Finding the main idea of a text lullaby 76.88 4. 34 R5A2_16 -1.387 Defining the features of an object sun 75.5 5. 26 R5A2_08 -1.191 Finding additional idea of a text a teenage boy 72.7 6. 17 R5A1_22 -1.002 Finding an additional meaning of a

text a smile 69.8

7. 14 R5A1_19 -0.962 Finding an finding of an idea of part of a text

lullaby 70.4

8. 18 R5A2_13 -0.89 Finding an additional idea of a text baadai 66.8 9. 31 R5A1_23 -0.877 Finding an additional idea of a text a smile 68.5 10. 16 R5A1_21 -0.783 Finding an idea of part of a text baadai 67.7 11. 36 R5A2_18 -0.721 Finding the main idea or point in a

text a smile 63.8

12. 33 R5A2_15 -0.696 Distinguishing similar and opposite meanings of paragraphs

sun 63.3

13. 53 R5A3_01 -0.672 Finding the main idea or point in a text

reason why birds migrate

62.4

14. 37 R5A4_07 -0.646 Expressing the main idea of a text through an idiom

a smile 63.4

15. 55 R5A2_07 -0.634 Finding of a main idea or point in a text

golden apple 60.5

16. 25 R5B3_01 -0.632 Finding parts of a text and distinguishing them

golden apple 62.6

17. 13 R5A1_18 -0.587 Defining the features of a thing or object

our world 61.5

18. 54 R5A1_09 -0.58 Finding an additional idea of a text little lump of clay 61.6 19. 7 R5B1_01 -0.566 Recognizing a paragraph tsookhondoi 60.4 20. 35 R5A2_17 -0.554 Finding the theme of a text Smile 59.9 21. 27 R5A2_09 -0.542 Finding an additional idea of a text Our world 60.4 22. 47 R5A4_01 -0.496 Expressing the main idea of text

with an idiom Golden apple 55.4

23. 2 R5A1_02 -0.352 Finding key words and sentences reason why birds migrate

55

24. 11 R5A2_10 -0.329 Distinguishing similar and opposite meanings of sentences

our world 55.1

25. 28 R5A1_16 -0.297 Finding an additional idea or concept from a text

a teenage boy 53.9

26. 41 R5A3_05 -0.208 Finding an additional idea of a text golden apple 51 27. 56 R5B3_03 -0.142 Finding parts of a text and

distinguishing them a teenage boy 53.3

Page 62: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 62

28. 40 R5A3_04 -0.063 Finding the main idea of a paragraph opposite poles 46.4 29. 46 R5A3_10 -0.041 Finding the content relationship

between language units sun 49.4

30. 29 R5A2_11 -0.037 Finding key words and sentences lullaby 48.7 31. 5 R5A2_12 -0.026 Finding an idea of part of a text baadai 47.7 32. 30 R5A1_05 0.011 defining purpose of a sentence golden apple 47.5 33. 44 R5A3_08 0.016 Finding an idea of part of a text baadai 48.3 34. 68 R5A2_14 0.036 Finding an additional idea of a text sun 47.3 35. 32 R5CD401C 0.037 Writing in a grammatically correct

language during paraphrasing or creative writing

my friend 27

36. 58 R5A1_08 0.059 Finding an additional idea of a text little lump of clay 57 37. 4 R5A1_04 0.073 Finding an additional idea of a text tsookhondoi 47.1 38. 71 R5A3_09 0.09 Distinguishing similar and opposite

meanings of sentences sun 45.4

39. 1 R5CD402C 0.102 Writing in a grammatically correct language during paraphrasing or creative writing

my friend 26

40. 20 R5A1_17 0.112 Finding the content relationship between language units

our world 46.9

41. 15 R5A1_01 0.114 Finding an idea of part of a text reason why birds migrate

45.1

42. 59 R5CD403C 0.118 Writing grammatically correct language while paraphrasing or writing creatively/an essay

my friend 25

43. 50 R5A2_02 0.13 Finding a topic or theme of a text tsookhondoi 44.1 44. 62 R5A1_20 0.148 Finding an additional idea of a text lullaby 47.8 45. 52 R5A1_10 0.154 Finding an additional idea of a text little lump of clay 47 46. 21 R5B3_02 0.155 Establishing a sequence order of

events in a text little lump of clay 52

47. 39 R5A4_04 0.161 Finding the content relationship between language units

our world 44.2

48. 19 R5A3_03 0.199 Finding the content relationship between language units

opposite poles 42.6

49. 23 R5A4_06 0.291 Finding the main idea or point in a text

baadai 42.2

50. 3 R5A2_03 0.325 Finding the main idea or point in a text

tsookhondoi 39.5

51. 64 R5A2_01 0.415 Finding the content relationship between language units

reason why birds migrate

68.7

52. 70 R5A2_05 0.472 Finding an additional idea of a text opposite poles 36.6 53. 63 R5A1_03 0.567 Defining the features of a thing or

object reason why birds

migrate 35.6

54. 67 R5CD401B 0.571 Writing neatly during paraphrasing or creative writing

learnt how to play chess

29

55. 66 R5CD403B 0.585 Writing neatly during paraphrasing or creative writing

learnt how to play chess

27

56. 43 R5CD402B 0.594 Writing neatly during paraphrasing or creative writing

learnt how to play chess

27

57. 12 R5CD401A 0.658 Creative writing based on visual summer vacation 27

Page 63: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 63

prompt 58. 45 R5CD402A 0.675 Creative writing based on visual

prompt summer vacation 26

59. 65 R5CD403A 0.68 Creative writing based on visual prompt

summer vacation 25

60. 69 R5A3_07 0.68 Finding the content relationship between language units

lullaby 31.9

61. 38 R5A3_02 0.72 Finding key words and sentences tsookhondoi 30.9 62. 22 R5A2_04 0.798 Distinguishing similar and opposite

meanings of paragraphs Opposite points 29.4

63. 8 R5A1_11 0.884 Finding an additional idea of a text little lump of clay 28.4 64. 57 R5A1_06 0.92 Finding an idea of part of a text little lump of clay 25 65. 10 R5A2_06 0.928 Finding a core and key words and

sentences Opposite poles 26.8

66. 24 R5A1_13 0.936 Finding additional idea of a text little lump of clay 43 67. 61 R5A1_15 0.962 Finding additional idea of a text little lump of clay 28.3 68. 60 R5A1_12 1.031 Finding additional idea of a text little lump of clay 41 69. 48 R5A4_02 1.291 Finding idea of part of a text little lump of clay 23.09 70. 9 R5A1_14 1.695 Finding intended purpose or point of

a text little lump of clay 17.3

71. 6 R5A1_07 2.228 Finding additional idea of a text little lump of clay 11.1

The distribution of item difficulty has been mapped onto the same scaling with the distribution of students’ performances as shown in Table No.19 given below.

Figure No. 20. Distribution of Levels of Difficulty of Reading Items, Number of Students and Items12

Students number: 4750, items number: 71

Difficulty Students Number of items

12 Item number in a list where items are listed in accordance with their level of difficulty

Page 64: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 64

THE STUDENTS ARE INVOLVED IN THE STUDY, THEIR AVERAGE PERCENTAGE OF THE READING SKILL IS -0,14.

2.10 BENCHMARKS OF READING

Benchmarks in Reading have been established based on the mean percentage of performance scores and in accordance with internationally applied approaches.

LEVEL 5

LEVEL 4

LEVEL 3

LEVEL 2

LEVEL 1

Average

Page 65: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 65

Table No. 27. Benchmarks in Reading

Benchmark levels of reading

comprehension achievement

Percentage performed

Percentage of non-performers Comment Scaled

score13

1 25 percentile 75% 25% 75 percent of students have performed

and 25 percent are non-performers 409

2 50 percentile 50% 50% 50 percent of students have performed and 50 percent are non-performers

547

3 75 percentile 25% 75% 25 percent of students have performed and 75 percent are non-performers

684

4 90 percentile 10% 90% 10 percent of students have performed and 90 percent are non-performers

758

This benchmark is divided into 4 levels of 25, 50, 75 and 90 percentiles and mapped onto or matched with estimated scale scores. For instance, the 25th percentile indicates that 75 percent of students have achieved scores more than 409 and 25 percent of students have achieved less than the scaled score of 409, the 75th percentile indicates that 25 percent of students have achieved scores above 684 and 75 percent of students have achieved scores less than 684, and level of 90th percentile indicates that 10 percent of students have achieved scores above than 758 and 90 percent of students have achieved below 758. The Percentage of students who have knowledge and skills that would match particular level is shown in Table No.28 below.

Table No. 28. Students involved in the Assessment Study Matched with Associated Performance Benchmarks

Level of benchmark

Percentage of student at that particular level

Percentage of students who have achieved that

particular level

Percentage of students who have not achieved

that particular level

25 percentile 23.5

50 percentile 44.8 76.5 23.5

75 percentile 29.8 31.7 68.3

90 percentile 1.9 1.9 98.1

Table No.28 above indicates that 23, 5 percent of total students have the knowledge and skills in reading

to match the 25th percentile level of benchmark; 44,8 percent of them match the 50th percentile 29,8 percent match

the 75th percentile benchmark; and 1,9 percent match the 90th percentile benchmark level. Here it should be

13This score is provided based on results of this assessment study.

Page 66: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 66

understood that the upper level benchmark contains the lower level benchmark in it. Knowledge and skills that is to

be acquired in reading have been identified for each benchmark level and section below.

Page 67: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 67

25 PERCENTILE

Content section – finding the main idea of a text and its parts and additional idea of a text, finding and distinguishing similar and opposite meanings of sentences;

Structural section – determining/finding and distinguishing structural parts of a text;

Modification section - summarazing and descriptive interpretation of a text;

Cultural section – content of a text, expressing own feelings derived from reading the text to others;

Artistic section of a fiction/passage - reading passages from a child fiction and traditional folklore dedicated for primary levels of education, understanding and comparing them, and identifying characters from passages;

Language section of a fiction/passage – fast and comprehended reading of a text, memorizing and solving riddles

Example item No.: R5B3_01

What is the correct sequence of events from the text?

А. Poor boy has cheated the king by lying. King has given to a poor boy a golden apple. King has given a golden apple to a boy.

King has sent a message. Nobody could tell a lie.

B. Nobody could tell a lie. A poor boy has told a lie and cheated a king.

King has sent a message. King has given a golden apple to a boy.

C. King has sent a message. Nobody could tell a lie. A poor boy has told a lie and cheated a king.

King gave a golden apple to a boy.

D. Nobody could tell a lie. A poor boy has told a lie and cheated a king. King gave a golden apple to a boy. King has sent a message.

Determining a subject of text and its features, place, location and timing of events, and purpose of a sentence; finding key units; finding main idea of a short passage or fairy tale; finding and distinguishing similar and opposite meanings of sentences;

Page 68: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 68

50 PERCENTILE

Content section – identifying the main idea of a text or passage; distinguishing the finding of the main idea or point in a text; distinguishing similar and opposite meanings of sentences; describe things and their features; finding additional idea of a text; finding a topic or theme of a text; finding key words and sentences in a text;

Structural section - recognizing paragraphs; finding and distinguishing structural parts of a text;

Modification section - summarizing and descriptive interpretation of a text, descriptive and summary interpretation of part of a text;

Cultural section – expressing a content and one’s own thoughts and impressions from a text read to others; defining the level of a text style; creative writing of a text

Artistic section of a fiction/passage - distinguishing and determining a genre of read passage or fiction; reading, comprehending and comparison of fiction and folklore dedicated for preschool children and determination of characters in a fiction; comparing similar and different features; determining the theme of a text read

Language section of a fiction or passage - scanning through a text fast; comprehensive and declamatory reading of a text

Example item: R5A1_15

What is the main idea or purpose of a text?

А. Human’s soul is changeable as clay.

B. Happiness is invaluable and precious thing which could not be priced.

C. Someone is the happiest if he or she does good things for others and has set own goals clearly.

D. The best method to create the most beautiful thing is to create with clay.

Writing cohesively and without mistakes; and finding the topic and main idea of a short passage; finding an idiomatic phrase that expresses the main idea of a text; distinguishing additional ideas, points and parts of a text; finding the structure and additional idea of a larger text; finding the main idea of a popular science text for cognitive purposes; finding key words and sentences; distinguishing similar and opposite meanings of paragraphs and sentences within a text; finding content relationship between linguistic units and identifying of their characteristics or features; finding the main, hidden or additional ideas of a fairy tale; finding an idiomatic expression that expresses the main idea of a text; determining of structure and parts of a text;

Page 69: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 69

75 PERCENTILE

Content section - finding additional ideas of a text, finding key words and sentences, finding the content relatiship between language units, defining the purpose of a sentence, finding an idea of a part of a text, finding the a main idea of a paragraph, finding the content relationship between language units, finding of an additional idea of a text, distinguishing between similar and opposite meanings of sentences, finding the main idea or point in a text, expressing the main idea of a text through an idiom;

Structural section - recognizing paragraphs, finding and distinguishing structural parts of a text; determining type of a text that was read;

Modification section – summarizing and descriptive interpretation of a text; summarizing and descriptive interpretation of part of a text; paraphrasing content of a text in oral and written form

Cultural section – expressing a content and own thoughts and impressions from a text read to others; defining a level of text style; creative writing; finding and distinguishing descriptive fictional writings in a text

Artistic section of a passage - distinguishing and determining a genre of a read passage or fiction; reading, comprehending and comparing fiction and folklore dedicated for preschool children and determination the characters in the fiction that was read; determining of a theme of a text that was read and retelling it to oneself and others, determining characters and their features from a fiction or passage that was read and conduct comparative assessment of their similar and different features

Language section of a passage - scanning through a text fast; comprehensive and declamatory reading of a text and listening

Example item No.: R5A4_01

Find an idiom that matches the main idea of a text:

А. Visitors return home and the rain stops.

B. If you lose your horse you can find it and get it back but if you said a word you cannot take it back

C. Brother teaches and a sibling listens and obeys.

D. Mountains are covered by snow a folk is burdened by age

Finding the theme of a text; determining the content relationship between textual parts; comparative analysis of subject features and characteristics; expressing the main idea of a text with an idiom; finding the theme, main idea and an idiom that expresses it from a short fiction or passage, and finding and distinguishing its additional hidden ideas and parts; finding a structure and additional ideas in a larger text or passage; finding of main idea of a popular science text for cognitive purposes; finding key words and sentences; distinguishing similar and opposite meanings of paragraphs and sentences within a text; finding content relationship between linguistic units and identification of their characteristics or features; determining of a thing or object and its characteristics; finding the main and hidden or additional ideas of a fairy tale and an idiomatic expression that expresses its main idea; determining structure and parts of a text;

Page 70: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 70

90 PERCENTILE

Content section - finding additional idea of a text, finding the main idea or point in a text; finding the content relatiship between language units; defining an item or object and its features; finding key words and sentences , distinguishing similar and opposite meanings of paragraphes , expressing the main idea of a text through an idiom, finding the main idea of a paragraph, defining the purpose of a sentence, finding of an intended purpose or point of a text;

Structural section - recognizing paragraphs, finding and distinguishing structural parts of a text; determining type of a text read; determination of a type of a text read or listened to;

Modification section - summarizing and descriptive interpretation of a text, of part of a text; paraphrasing of a part of a text in writing and speaking; descriptive paraphrasing of type/genre of a selected text in writing and speaking;

Cultural section – expressing a content and own thoughts and impressions from a text read to others; defining the level of text style; finding descriptive fictional writings from a text and explanation of its style and form; creative writing of a text distinguishing indent and paragraphs; and creative writing of a text with appropriate language to match and express the genre specifics of a fiction

Artistic section of a passage - distinguishing and determining a genre of read passage or fiction; reading, comprehending and comparing fiction and folklore dedicated for preschool children and determining characters in a fiction read; determining a theme of a text read and retelling it to oneself and others in an orderly sequence; retelling and writing one’s own impressions, assessments and conclusions about the fiction read and listening to impressions of others; determining characters and their features from a fiction or passage read and comparative assessment of their similar and different features; explanatory and persuasive speaking with examples

Language section of a passage - scanning through a text fast; comprehensive and declamatory reading of a text and listening attentively

Example item: R5A1_08

Finding the main theme of a text; determining the content relationship between parts of a text; comparing characteristic features of subjects; expressing the main idea of a text with an idiomatic phrase; finding the main and additional ideas of a larger text and ideas from textual parts;

Page 71: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 71

2.11. PERFORMANCE OF INTERNATIONAL ITEMS

Inclusion of international items of mathematics and reading (TIMSS, PIRLS) in the national assessment survey has enabled national results to be compared with the outcomes of international assessment surveys.

Performance of International Mathematics items: An attempt has been undertaken to compare the performance of students with the results of international

assessment test by including 4 international mathematics test items. 4 test items from TIMSS 1995, 2003 and 2007 have been included in the national assessment test. In fact, high expectations were placed on the performance results of 2 test items included from TIMSS 2007, however, since the findings from the assessment study have not yet been finalized the comparison list could not be released. Regardless an item analysis has been provided below. Item No. 1: TIMSS 1995 item No. (M5C3_01)14. This was an item from the geometry section and asked students to identify pattern in sequencing simple geometric shapes. Mongolian students have performed this item with the average percentage of performance of 0.77%. The performance of this item has been higher than the performance of students from countries as New Zealand, Israel, Norway, Portugal, Thailand, Iran, Cyprus and Greece.

Table No. 29. Performance of International Test Item L13

Countries Average Standard error

South Korea 0.93 -0.01 Japan 0.92 -0.01 Nederland 0.86 -0.02 Austria 0.85 -0.02 Singapore 0.85 -0.01 Latvia 0.83 -0.03 Hong Kong 0.83 -0.02 Check Republic 0.8 -0.02 Hungary 0.8 -0.02 USA 0.79 -0.02 Australia 0.79 -0.02 Canada 0.78 -0.03 Ireland 0.77 -0.02 Mongolia 0.77 -0.01 Slovenia 0.76 -0.03 Great Britain 0.75 -0.02 Scotland 0.75 -0.02 New Zealand 0.71 -0.03 Israel 0.7 -0.03 Iceland 0.69 -0.03 Norway 0.66 -0.03 Portugal 0.58 -0.03 Thailand 0.55 -0.03 Iran 0.55 -0.03 Cyprus 0.54 -0.03 Greece 0.54 -0.03

14 TIMSS 1995 item No. L13. ASMML04”ASMM/WHICH SET IN SAME PATTERN (C)

Page 72: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 72

Kuwait 0.45 -0.02 Item No.2: This was an item from TIMSS(M5S3_01)15 in 2003. It was a mathematical problem with a pre-structural level of difficulty, a multiple choice question. It has been an item to assess the ability of students to create an expression with unknown element after reading the first mathematical word problem. Mongolian students have performed this item with an average percentage of 0.54 and performance result were higher than the performance of students from countries such as Italy, Armenia, Norway, Philippines, Iran, Morocco, Tunis and Yemen students.

Table No. 30 Performance of International Test Item No. M012048

Countries Mean Standard error

Singapore 0.87 -0.01 Taiwan 0.82 -0.01 Hong Kong 0.76 -0.02 Indiana (USA) 0.75 -0.02 USA 0.74 -0.01 Netherlands 0.72 -0.03 Russian Federation 0.7 -0.02 Belgium 0.69 -0.02 Japan 0.68 -0.02 Cyprus 0.68 -0.02 Latvia 0.68 -0.02 Moldova 0.67 -0.02 England 0.67 -0.03 Hungary 0.65 -0.02 Lithuania 0.65 -0.02 Slovenia 0.65 -0.02 Scotland 0.63 -0.02 Canada (Ontario) 0.63 -0.02 Canada (Quebec) 0.63 -0.02 Australia 0.59 -0.02 New Zealand 0.55 -0.02 Mongolia 0.54 -0.01 Italy 0.53 -0.02 Armenia 0.48 -0.02 Norway 0.42 -0.02 Philippines 0.4 -0.02 Iran 0.38 -0.02 Morocco 0.32 -0.02 Tunis 0.25 -0.02 Yemen 0.21 -0.03

15 TIMSS 2003 item No. M012048.

Page 73: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 73

Performance of International Items in Reading

Reading assessment test has included a text called «Little lump of clay» with 12 set of items which was

selected from international reading testing program PIRLS-2006. Since performance results of these international test items by Mongolian students have been assessed below the average results of international test performance, it could be assumed that similar assessment items have not been performed by students in their acquisition of reading skills. In fact, PIRLS items have required a higher level of reading comprehension skills students. Analyses of international items are provided below.

Item No. 1: PIRLS 2006 test item No. (R5A1_15)16. This item belonged to the content section and was of medium level of difficulty and was to assess student’s skills to determine the main idea or point in a text. The average performance percentage of Mongolian students was 0.28 which ranks it higher in comparison to the performance results of students from countries such as Indonesia, Morocco, Kuwait and South Africa.

Table No. 31. Performance of International Test Item R011C13M

Countries Mean Standard error Russia 0.8 0.01 Italy 0.78 0.02 Singapore 0.69 0.02 Latvia 0.68 0.02 Israel 0.68 0.02 Slovakia 0.68 0.02 Lithuania 0.68 0.02 Canada (British) 0.67 0.02 Canada 0.66 0.03 Bulgaria 0.66 0.02 Canada (Alberta) 0.66 0.02 USA 0.64 0.01 Taiwan 0.64 0.02 Hungary 0.61 0.02 Great Britain 0.61 0.02 Hong Kong 0.61 0.02 Canada (Nova) 0.6 0.02 Scotland 0.6 0.02 Germany 0.59 0.02 Ireland 0.58 0.02 New Zealand 0.58 0.02 Romania 0.56 0.02 Belgium 0.55 0.02 Canada (Cubic) 0.54 0.03 Poland 0.52 0.02

16 PIRLS-2006 item No. R011C13М.

What is the main message of this story?

À. People are easy to knead and shape like clay.

B. There is a great deal of happiness in the world.

C. Everything is happy when it finds a purpose.

D. Pottery is the best way to do beautiful things in

the world.

Page 74: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 74

Netherlands 0.52 0.02 Austria 0.51 0.02 Macedonia 0.51 0.02 France 0.5 0.02 Georgia 0.48 0.03 Luxemburg 0.48 0.02 Belgium (French) 0.47 0.02 Denmark 0.46 0.02 Iran 0.44 0.02 Norway 0.44 0.02 Spain 0.43 0.02 Moldova 0.43 0.02 Slovenia 0.39 0.02 Trinidad and Tobago 0.37 0.02 Sweden 0.35 0.02 Qatar 0.31 0.01 Mongolia 0,28 0,01 Indonesia 0.26 0.02 Morocco 0.22 0.02 Kuwait 0.22 0.02 South Africa 0.21 0.01

Page 75: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 75

CHAPTER 3. FACTORS ASSOCIATED WITH LEARNING ACHIEVEMENT

In addition to assessing the learning achievement of primary education to reveal the most influencing factors, questionnaires obtained from teacher, school administrators, students, methodologists in charge of aimag primary education, parent and guardian representatives have been analyzed against learning achievement. In undertaking this analysis, research has been carried out whether location, demographic factors and learning environment (external and internal) were associated and influencing learning achievement. Also an analysis has been carried out whether there are specific factors having distinctive impact on mathematics and reading separately.

Learning environment17 has become a topic of extensive discussion, thus researchers, scholars and managers of educational institutions have been addressing this issue with increased attention during the recent years. Concept and research on learning environment has begun to be studied scientifically since the 1950’s and its main focus has initially been dealing with the physical environment of learning.

Therefore based on the recognition of the importance of learning environments, a comprehensive analysis has been undertaken in this report on learning environment distinguishing internal and external components. The ‘Environmental model of a university’ developed by American researchers as E. Turban, J. Merdit (1950) is necessarily referred to when scientifically based theory of learning environment is considered. That concept recognizes that “A system consists of three main components as input-progress-output and it exists within concrete distance from the environment and surrounded by definite boundaries. Hence external environment shall comprise of environments of nature, politics and legislation, socio-economics, socio-population and information-technology environments according to the definition of the concept provided by researchers.

In this report learning achievement of mathematics and reading of students have been analyzed in regard to demographic, and internal and external learning environment factors.

3.1 FACTORS ASSOCIATED WITH LOCATION AND POPULATION (Difference between urban and rural students, gender, Mongolian and Kazakh nationalities)

It has been observed that learning achievement are different among rural and urban students based on the

study results. This study has aimed to distinguish these differences among with attempting to propose possible measures that could be taken to decrease the gap.

The difference between learning achievements of rural and urban students in urban and rural settings have been 10 percent in favor of urban students (referred to in chapter No.1).

Analysis of learning achievement factors have been provided below associated with residency and demographics which has been undertaken in connection with questionnaires taken from students, school administrators, teachers, methodologists and parents.

The average age of 5th grade students involved in the study from rural and urban schools has been between 11 to 12.

Forty nine point four percent of urban students and 50,2 percent of rural students involved in the national assessment study were males. The mean percentage of performance scores of urban students have been higher in comparison to rural students by 10 percent for both girls and boys when learning achievement were compared to urban and rural and yet by gender differences.

20 Kh. Tsetsegjargal., Researchers perception about learning environment. “Educational reforms and learning environment” Thematic Scientific Conference. 2007;

Page 76: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 76

Figure No. 21. Link Learning Achievement of Urban and Rural Students by Gender

In analyzing of how learning achievement of students has been associated with their early kindergarten

attendance has shown that the average percentage of performance scores of urban students who attended kindergarten was 54.8%, and of rural students was 44.1%, whereas the average percentage of performance scores of students who had not attended in kindergarten was 51.8% for urban students and 39.3% for rural students. This comparison has shown that learning achievement of students who attended in kindergarten was higher that students who had not attended kindergarten both for urban and rural children. Learning achievement of children who attended in kindergarten in urban setting has been higher in comparison to rural setting indicating a link with differences of urban and rural settings in relation to learning and a child’s developmental environment, skills of teachers and care and involvement of their parents. Learning achievement of children who had not been attending kindergarten has been higher in urban setting which is largely associated with local specifics such as poor employment of parents and guardians in rural areas and poor environment for a child’s development, and a lower level of access to kindergarten in comparison to urban areas. Study finding have revealed that involvement in preschool education services has significant impact over the learning achievement of students.

It has been identified that most urban children live with their families and spend little time travelling to school via public transportation or vehicles whereas rural children live either with their families or dormitory or homes of their elder siblings and relatives, and spend more time travelling to school either by walking or using traditional animal (horse) or cart riding upon analysis of students’ living situations and the distance to their schools, and how much time it is spent when travelling to and from school, and the means of transportation are used. In considering relationship with learning achievement, performance of students who travel to school using traditional animal or cart riding has been the lowest (at 29.3%) and the highest (50%) among students who travel using public transport. It is most likely that children get tired travelling long distances from unable to draw their attention to homework and lack of free time. The average score of students who walk to school has been 42.3% whereas for children who are given a ride by a car was 46.9%. 77% or the most of the students travel to and from schools by walking, 3.8% by a vehicle, and 0.6% by traditional or horse or animal drawn cart, 14.8% by public transportation. In estimating the average distance and duration of travel students spend about 17.5 minutes for 700 meters from home to school and back.

In analyzing how learning achievement is related with the place of residency the learning achievement of children who stay at home with their families has been higher than of children who live either in school dormitory or homes of their relatives during years of primary education. 8% of 5th grade students who live in rural areas and 1% of urban students have been living in dormitories. It clearly indicates that there is strong requirement for the drastic improvement of living environment and conditions of students living in dormitories, and as well as training and child development activities organized in dormitories and increase accessibility and coverage of rural

Page 77: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 77

schoolchildren in dormitories. Also, it should be noted that reading of students who live with their families has been enhancing as they study advance further in their grades.

Figure No. 22, Where they live mostly during the school year?

In associating learning achievement of urban and rural school students with the availability of the following items and things found in their dwellings upon investigation, it has been revealed that the difference in learning achievement to range from 0,13-15,04 among students who possess or do not possess those items and things. Learning achievement of children who have a TV set in their dwelling has been the highest from those who do not have TV set by 15,0 percent. However, some studies21 indicate that learning achievement of children who watch TV for long hours has been poor. Also internet is not being used much by primary level students and the difference has been only 0,13 percent. Learning achievement of children who live in dwellings with a supply of electricity for 24 hours and availability of electrical appliances and more comfortable living conditions has been higher from other children by 0,13-15,04 percent or in average by 6,41 percent. From these comparative analyses it has been assumed that the availability of those appliances, training inventories and items that would enable students to save time and live in more comfortable environment which in return would influence positively learning achievement of students. Favorable atmosphere yet comfortable environment for living have been important factors to impact learning achievement of students positively.

Figure No. 23. Do you have the following items and things in your dwellings?

21 National Survey of Assessment of Learning Achievement 2006, Report of MEA and EEC

Page 78: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 78

Considering how learning achievement is associated with the availability of books at students’ homes, the learning achievement of urban students who have had books in number range of 26-100 was 59.6% and 51.1% for rural students, both of which have been relatively higher than learning achievement of students with no books. Learning achievement of urban students with no books has been higher of learning achievement of rural children by 13% which could be related with the greater possibility of urban children to access information from various other information sources (TV’s, radios, FM’s, internet and publication media). No significant difference has been revealed among rural and urban areas when a question was asked from 5th grade students whether they could access internet which might indicate that they have little or no knowledge of using or applying internet sources.

When learning achievement was investigated on how it could be associated with the number of students sharing one desk in class and different depending upon urban and rural settings it has shown that learning achievement of 2 students sharing a desk has been higher. However, the difference between urban and rural students have been 11.5% and it has shown that sitting alone or sharing a desk by three students is not been appropriate or acceptable.

Figure No. 24. Learning achievement of urban and rural students associated with the number of students sharing a classroom desk

In comparing the supply of textbooks, books and learning inventories between urban and rural settings, the learning achievement of students who newly bought mathematics textbook newly was 55.6% for urban students and 46.1% for rural students. The average performance scores of urban students who own Mongolian language textbooks were 53.6% whereas for rural students it has been 40.7%. Also average percentage of performance scores of urban students with new mathematics textbooks provided from the school were 53.7% whereas it was 40.9% for rural students. Also students who have old books or use a book alone or share between many other students only in school have outperformed in learning achievement assessment students who have no books at all. It is also possible that students with new textbooks might have more interest and motivation to study actively.

22.3% of urban fathers and 29.3% of urban mothers have a tertiary level of education, whereas 6,3 percent of rural fathers and 8,4 percent of rural mothers have a tertiary level of education achievement. Assessment study findings have shown that higher the level of education of fathers and mothers of students the learning achievement of their children would be higher as well.

Page 79: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 79

Figure No. 25. Impact of level of education of parents on learning achievement of students

It is associated with better attitude and care of parents with tertiary level of education achievement in ensuring creation of adequate learning environment and conditions for their children and provision of the necessary support in promotion of learning motivation and interest in their children, establish good links with schools and teachers and have better skills in helping them in their study. The average percentage of performance scores of students with parents who have either primary level of education or no education has been low at 29.7-31% which could be related with the low level of their knowledge and poor skills and attitude in assisting their children in their learning and education. No results have been obtained when investigation was made on how employment of parents is associated with the learning achievement of their children. In fact, 21.3% of fathers involved in the questionnaire were herders, 17% were unemployed, and 16% were privately engaged in various businesses. In case of mothers involved in the questionnaire 17.7% were herders, 16.2% were retired and 7.2% were unemployed. Study findings have shown that the number of mothers that were employed was higher in comparison to fathers but they retired earlier than their husbands.

Also when the level of livelihood was analyzed in relation to learning achievement of urban and rural students, the learning achievement of students from families with better living standards were at 51.4% and at 49.4% for families with normal living standards which were relatively higher than level of 39.8% of learning achievement of students from families with lower living standards.

Learning environment of urban schools has been better than those in rural schools. That has been affecting learning achievements substantially. For instance, learning achievement of students from 45.5% of schools where washrooms and corners with water outlet and sink were available has been higher by 7.07% from students who were studying in schools where not such facilities were available. However, in spite of the compulsory requirement and need for washroom facilities for students only 10% of urban schools and 37% of rural schools still do not have any of these facilities at all.

In investigating common sicknesses or health conditions of students involved in the study, 1.3% had ophthalmological or ocular problems and 1% otolaryngological problems. The nature of health problems could be in large related to the state of school heating. 39.3% of all schools involved receive centralized heating supply, 38.6% had their own boiler systems and 9% had traditional fire-based stove heating. The probability of rural children of getting sick has been far greater than children in urban schools since students tend to be affected by imbalanced temperature conditions in most rural schools where they get either cold or hot abnormally due to lack of connection to the school centralized heating supply system.

When responses of students to the question of whether they get cold on the way to school and back home have been analyzed in connection with their learning achievement has revealed or given the following circumstances or results.

Page 80: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 80

Figure No. 26. Learning Achievement (in association with impact of cold)

When also a question was asked whether students get cold in their dormitories or dwellings where they live during non-school hours, the learning achievement of students who do not live in cold dwellings were higher than those who get cold regularly and live in poorly heated dwellings.

In comparing learning achievement of rural and urban students by the teachers’ subject specialization learning achievement of students of urban teachers have been higher than students of their rural colleagues. When considering learning achievement by subject specialization of teachers, the learning achievement of students of primary teachers has been 53.1% for urban students, and 41.9% for rural students whereas the average percentage of performance scores of students whose class teachers have acquired degrees or certificates in mathematics were 55.5% in urban areas and 45.2% in rural areas, whereas learning achievement of students whose class teachers have specialized in Mongolian language has been 56.5% in urban areas and 56.3% in rural areas. These results have shown that learning achievement of students in mathematics and reading who have been taught by subject specialized teachers has been higher both for rural and urban students.

An analysis of whether the level of education of teachers has been associated or has an impact on learning achievement of students have shown that the learning achievement of students whose teachers have had completed secondary education received special professional certificates in education in comparison to teachers who have their bachelors and master in education was lower. As the education level of teachers was higher the learning achievements of students were higher accordingly. That implies that the skills and level of teachers’ education affected the learning achievements of their students, hence their achievements were relatively higher.

Page 81: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 81

Figure No. 27. Comparison of learning achievement by level of teacher education

The learning achievement of rural and urban students were not related to the number of years of teaching service (for instance, average scores of students whose teacher have had 1 year of experience has been 63%, 62% by teachers who have had 10 years of teaching experience and 53% for students whose teachers had been teaching for 42 years etc.).

Learning achievement of students has not been associated so much with who was the manager of school as the results of the assessment have shown.

Most rural parents have not evaluated their children as talented or well performing students whereas urban parents have evaluated their children very highly. Evaluation skill of urban parents has been higher in comparison to rural parents.

Negative incidents tend to happen more in rural schools when the issue of how often incidents of praising, scolding and violation of student’s freedom by teachers of urban and rural students tend to happen an addition to surveying of how often students were praised and bullied by their classmates.

Figure No. 28. Difference between urban and rural students in some aspects

It is thought that it has been negatively impacting the students’ motivation and willingness to learn and

hence it affected their learning achievement as well. Since these kind of issues tend to happen more in rural areas compared to urban it also indicates the need to pay attention to the ethics of rural teachers accordingly.

Page 82: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 82

In surveying the motivation of students whether they are motivated to go to schools through parents stated that 2.2% of urban students and 3.2% of rural students were not motivated to go to school. In considering how students have been spending their free time upon analysis of how much time they spend on doing their home work, helping household chores, looking after their siblings, spending time with their schoolmates, playing various games and watching TV, parents’ results have shown that rural students have been spending more time for these activities mentioned above.

Figure No. 29. Learning achievement of urban and rural students in association with their spending of leisure time

Low level of learning achievement of rural students could likely be related with the poor management of their free time with more time spent doing household chores and other household duties.

Learning achievement of students who are used to getting someone’s assistance in doing their homework has been lower compared to those who do their homework independently and without assistance. It might be related with the fact that those who help them have been not able to provide the kind of assistance that children needed to think and learn themselves.

3.2 FACTORS ASSOCIATED WITH LEARNING ENVIRONMENT (EXTERNAL AND INTERNAL)

School environment plays an important role in student’s learning achievement. When considering learning environment it has been classified into external and internal environments where law and legislation, finance and economics, family and social, and a physical environment has been included in the analysis of external environment. Analysis of internal environment and how it was having impact over learning achievement has included learning inventories, teacher methodologies; approaches and skills, in-service teacher training, psychological environment of teaching (psychological environment of collectives and communication), relationship and interaction between collectives and friends, leisure time and development of student morale.

Page 83: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 83

EXTERNALLY DEPENDENT FACTORS (Laws and legislation, finance, economy, family, society and physical environment)

Most schools involved in the assessment study or 95.8% of them were studying in public schools. Considering the learning achievement in these schools, the average scores was 46.5% (499.6) for public schools and 49.9% (519.2) for private schools. The results have shown that learning achievement of students in privately owned schools was a bit higher in comparison to students in public schools. It is assumed that small class size in private schools and higher living standards of students have impacted positive learning achievement of students studying there.

An analysis of how types of different schools have been impacting learning achievement of their students has shown that the average scores and scaled score was 43.1% (485.9) for students in schools with conventional, mainstream teaching program and curricula, and 52.3% (536.9) for students from schools with training programs of professional orientation and 56.6% (543.8) for students from schools with advanced, in-depth level of study programs and curricula in select subjects which indicates that the quality of teaching in schools with advanced, in-depth teaching of curricula in selected subjects have been higher in comparison to schools with regular, mainstream training program and curricula. It is also most likely to be associated with reality that schools with advance and in-depth study curricula on select subjects have been following this practice of selecting their teachers and students based on higher criteria and requirements.

Learning achievement of students varied depending upon location factors.

Figure No. 30. Learning achievement associated with location

In general, schools within the capital city, aimag and soum centers and bags have problems related with shortage of teaching personnel and poor skill of teachers. Considering the enrollment of teachers in in-service training seminar and workshop results have shown quite similar outcomes with results shown in Figure No. 30 above. In other words, teachers from bag schools have hardly been involved in in-service training programs. In addition to training seminar and workshop practices such as monitoring and evaluation exercises being undertaken by public educaiton administration organizations, advisory and consultancy services, and representatives assigned from MECS and other institutions with the task to evaluate and assess teaching and learning at schools which in general, have been providing information, evaluation and inspection for schools less and less due to the location as it is distanced from the capital city.

Page 84: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 84

Analysis of how school heating and electricity supply have been associated with the learning achievements of students has shown that mean performance scores of students studying in schools with centralized heating was 52.7% (535) and 47.1% (505) for schools with local heating and 40.9% (467) for students in schools with their own boiler heating system and 38.6% (454) for schools with conventional heating. Thus indicates that learning achievement depends upon heating conditions, warmth and comfort level of the learning environment. In other words students learn better if they do not feel cold. It is obvious that conditions for smooth organization of teaching and training will be disrupted if rooms and facilities are not well heated and cold since warm interior conditions impact effective provision of smooth training activities and affects health and hygiene.

Figure No. 31. Learning achievement associated with the heating of school (Scaled scores)

When a question was asked from the school administration about the quality of school building facillities, 33-40% responded that building and facilities were suitable for utilization and maintaining building facilities require rehabiliation and repair. 25.2% of school administrators also responded that building foundations, roofing and ceilings, piping system and electrical cable wiring were apt for utilization. Survey has shown that most schools have 2 buildings for schooling purposes. Survey results have shown no direct link between learning achievement and the quality of school building faciliites. Also when the capacity of school building facilities were analyzed, the average capacity of first school building facilities was able to hold 600 students. 57% of all schools involved were centrally heated, 77.6% had 24 hour supply of electricity. 50.3% of schools were connected into potable water supply gridline. 50.3% of schools were connected to the sewage system and 60 percent of schools involved were in a condition that requires complete rehabilitation. When it was asked whether complete rehabillitation was undertaken within the past three yearss 60% of schools were not rehabilitated completely at all. 35 schools or 21.2% of schools involved had no exercise facilities and 80 schools or 48 percent had no culture facilities. The average performance scores of students in schools with culture hall was 50.3% (522) and 42.6% (477) for students from schools where they had no such facility. It can be concluded from the results that culture and performance of activities have an impact over students motivation to learning. In clarification of classroom facilities apt for conducting lessons for entire classes, schools had on average 15 regular size classrooms and a school with the most number of classroom available had 45 classroom facilities whereas the average number of study rooms for smaller group lessons was only 3 which indicates that avaibility of smaller rooms for group work has been insufficient given its ratio of the per number of students in a school.

16.4% of schools involved had no computers available for teachers in primary classes to use for preparing and use during their lessons and the average number of computers per school was 3-4 computers. When a question was asked on how often they used computers 84% responded that they use it occasionally or do not use it at all. In analysis of financial resources for schools in 2007, on average 81.6% of the school budget were resources allocated from the central and state budgets. The remaining budget of a school was comprised of 4.2% from training fees,

Page 85: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 85

0.4% from school development fund, 1.1% from the support economic activities, 1.3% from operational income of other activities, 0.5% from donations and assistance provided from local economic entities, 0.5% from donations and assistance provided by parents, 0.7% by donations and assistance provided by Non-Governmental Organizations, 0.2% from donations and assistance provided from individuals, 1.7% from donations and assistance provided from foreign organizations and individuals and 1.2% from other income generated. Textbooks are the most needed learning tool for students. If textbooks are of good quality then opportunities for students to learn individually are increased adequately and yet more opportunities are created for students to study better. For the past few years optional textbooks have increasingly become available for each subjects and when school administrators and teacher opinions have been surveyed 89.6% supported to abolish this practice of optional textbooks and stick to one core textbook and only 6% has supported the practice of having optional textbooks and 2.4% answered that they don’t know. 92% of all teachers surveyed did not supported the practice of having optional textbooks. 66.1% of school administrators and teachers have evaluated the quality of teaching of Mongolian traditional script as satisfactory, and 44.8% of them have evaluated skills of teachers as good; 47.9% as satisfactory. By the Decree No.236, 2007 of the Minister for Education, Culture and Science of Mongolia it has been provisioned to introduce teaching of traditional Mongolian script beginning 3rd grade and teach it in an integrated lesson with subjects of Mongolian language, reading and traditional Mongolian script. And in order to identify how effective that decision has been a question was asked from school managers from what grade it would be more relevant to introduce subject of a mongolian traditional script to which most of them have answered that it would be more effective to introduce teaching of traditional script from 5th to 8th grade. (18.8% prefer this subject from 5th grade; 20.6% from grade 6, and 33.3% from 7th grade) However, 30.5% of teachers believe that it would be more effective to introduce traditional script to be introduced from 6th grade, and 42% from 7th grade of junior secondary level. When an analysis has been made in respect to how learning achievement has been associated with the living environment of teachers it has shown that learning achievement of students who were taught by teachers living in housing that is connected into centralized heating system was 51.5% (528) which has been higher than the performance of students taught by teachers who have been living in other kinds of housing. It is evident that teachers living in more comfortable living conditions have been having positive impact over their work, interaction with their students, and provide assistance and influences teachers development, their methodologies and skills. Availability of electricity, potable and hot waters, car and electrical appliances in teacher homes do not have direct impact over learning achievement of students. Aslo issues such as the number of hours taught per week and school shifts teachers work in were not likely reasons to be having impact over learning achievement of students.

62% of teachers have replied that they have been paying increased attention to their school-based professional development and 80% of teachers have considered in-service trainings and seminars organized locally to be effective. 89.3% of teachers have responded that it would be very effective if teachers will have in-service training organized for teachers over a period of one month period in the capital city every 5 years. 62.3% of teachers currently working are not fully satisfied with their career and if an opportunity with higher salary, bonuses and other incentives and rewards is offered to them 62.3% answered that they will accept the offer. The average performance scores of students taught by unqualified teachers were 48.4% (511). However, 59.4% of teachers like their job and 73.5% of them would not be interested in making career, and 80% of teachers are interested in developing their professional knowledge and skills. It has shown that though teachers do like their work they would be interested in such social issues as having increased salaries and additional bonuses.

An analysis of teacher responses to survey questions about their schools have shown that 60% consider their school buildings and facilities need to undergo major rehabilitation and 40% have replied that the school building and facilities they work in are in acceptable condition. 68.2% of teachers from schools with dormitories have answered that their school buildings with should be rehabilitated and 66% responded that the number of students to be enrolled per class or class size should be reduced. 69.1% of teachers have emphasized the need for the

Page 86: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 86

establishment of teacher development centers whereas 70.5% of teachers responded that they need a room or facility where they can prepare for lessons and teaching. 82.1% expressed that supply of training inventories, equipment and computers should be increased and 88.1% have considered that their utilization should be enhanced. 94% of teachers surveyed have answered that items and inventories required for teachers to develop and produce teaching aids, kits and materials should be allocated for teachers.

29.4% of teachers have evaluated the quality of services provided by school canteen and 23.3% have evaluated the quality of services provided by their school library as being satisfactory. Therefore, the quality of internal school services, namely, services provided by school canteen and library should be taken into account at all levels of educational management and needs to be improved drastically. In analyzing how learning achievement of students has been associated with the type of dwelling children live in, the performance of students who have been living in flats with a steady supply of potable hot and cold water have shown better results in comparison to students living in dormitories with no supply of tap hot and cold waters which indicates that learning achievement depends directly from the type of dwelling children live in and living conditions and their comforts levels.

Table No. 32. Link between Learning Achievement and Type of Dwelling

Type of Dwelling Scaled score

1 In ger /traditional felt covered dwelling/ 473 2 In a flat connected to the centralized gridline supply system and tap potable

and hot water available; 574

3 In a flat connected to the centralized gridline system but has no tap potable and hot water available;

523

4 In public housing which has no connection to the centralized supply system and has water supplied on delivery;

495

5 In a dormitory which is connected to the centralized supply system and has tap potable and hot waters available;

467

6 In dormitory which is connected to the centralized supply system but has no tap potable and hot waters available;

441

7 In a private housing connected into centralized supply system and has tap potable and hot waters available;

514

8 In a private housing connected into centralized supply system but has no potable and hot water available;

528

9 In private housing which has water on delivery but has hot and potable water available from the tap;

510

10 In private housing which has water on delivery but has potable water available from the tap;

473

11 In private housing which has water on delivery but has no hot and potable water available from the tap;

511

Learning achievement also directly depend on the level of living standards. It is assumed that this is most likely to be dependent upon and related with better learning environment, conditions established and opportunities created in households with higher living standards for their students.

Page 87: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 87

Figure No. 32. Learning achievement associated directly with the level of livelihood

Children with physical disabilities study with normal students together in the same school and classes in our country. Though it promotes their social integration18, during the study it has been observed that a study should be undertaken whether care and attention they have been receiving were effective, adequate and well targeted. 5,8 percent of all students involved have had some kind of disability. Most commonly occurring disability, in terms of share has been related with eye sighte.

Figure No. 33. Percentage share of types of disabilities found in students

Analysis of learning achievement of physically disabled children has shown that children with eye, hearing and support organ disabilities could manage studying together with normal kids. But care and attention towards children with mental disorders should be enhanced and improved.

Table No. 33. How learning achievement is associated with children with physical disabilities

Kind of Disability Average scores Scaled scores

Mentally disabled 23.2 365 Hearing disability 41.8 472 Verbal organs 30.7 408 Ophthalmology 44.2 486 Support organs 39.7 461 Others 29.8 403

18 Effective since 2006 following Decree No. ... by Minister for ECS.

Page 88: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 88

Analysis of health problems encountered by students has shown that 4.9% have some kind of chronic disease. And most common health problems and sicknesses include otolaryngological diseases.

Figure No. 34. Percentage Share of Health Problems

Table given below shows how learning achievements have been associated with certain types of health problems encountered by students and analysis indicates that sicknesses and type of health problems were the factors that impact learning achievements of students.

Table No. 34. Comparison of students sicknesses with learning achievement

Types of health problems Average scores Scaled score

Stomach 58.9 571 Kidney 48.9 514 Heart 46.3 498 Ophthalmology 45.1 491 Otolaryngology 35.4 436 Psychiatric problems 26 382 Lung 22 359 Others 44.8 490

When issues related with either school drop-out rates or repeating a class were looked at it has revealed that 1.7% of all students who were involved in assessment have had school drop-out problems. Students who had school drop-out or repeating a year have shown 33.6% (403) of learning achievement in the assessment. Students do have to undergo repeated study because of failure to succeed in their study and it is affected by many factors including illness, migration and etc.

Analyses of whether students were orphaned or have not yielded that 0.09% of all children involved were either full orphans or half orphans. In terms of learning achievement full orphans has shown an average scores at 41.5%, students with no father have performed at 44.1% and students with no mother have shown 41.4% respectively. Results indicate that learning achievement of children who live with their mothers have been slightly higher than learning achievement of children who are full orphans or live with their fathers.

Page 89: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 89

When learning achievement of students were associated with the number of students sharing one desk, the performance of students who share a desk with one other student was higher by 8.12% in comparison to the performance of students who share a desk with two or more students.

Figure No. 35. Number of students sharing a desk in association with learning achievement

Also when an analysis has been carried out whether chairs and desks occupied by students were comfortable and convenient, learning achievement of students who responded that their chair and desks are comfortable and convenient was higher by 10.98% from those children who responded that the chairs and desks being either too tall or too small. This analysis shows that students need more appropriate and convenient desks and chairs that would comply with their age and specifics of their physical development.

When analysis has been made whether learning achievement was associated with the location and type of restrooms students have at their schools it has yielded that learning achievement of students who have their school restroom facilities indoor was higher by 12.3% than students in schools where restrooms were located outside and were higher by 19.79% compared to schools that have no restroom facilities.

Physical training is a subject for primary education students is undoubtedly one of the most appealing and needed subjects. Though most schools have proper exercise facilities it’s no secret that many schools still provide physical trainings in halls and facilities that are not designed for that particular purpose. Learning achievements of students from schools with exercise facilities appropriate for provision of physical training was higher by 5.67-8.41% compared to the performance of students from schools where they had no appropriate exercise facilities or had none at all.

Most of students of primary education live in families with 2-4 members in it. When an analysis has been conducted whether learning achievement is associated with the number of members in a family, employment status of family members and the number of people residing or sharing their homes or dwelling it has shown no significant link or correlation that impact learning achievement. For instance, the average scores of students living in families of 12 members was 32% and 27.1% for families of 14 members and has changed to 13.8% for students living in families of 13 members. As for the employment status of family members the average scores of students from families where two members have been employed was 45% and 37.1% for students living in families where 7 family members have been employed.

69.4% of students involved in the national assessment have been living in their homes. When the number of family members sharing the same home has been considered whether it affects the learning achievement of students

Page 90: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 90

it has shown no significant correlation with learning achievement as in the case of employment status of family members mentioned earlier.

However, when learning achievement has been considered in correlation with the students’ evaluation of their levels of family living standards surveyed the results were that learning achievement of students from families with middle income was good or better. Learning achievement of students who have assessed their level of family living standards as good and average have performed better in comparison to students who have answered as poor and very bad.

Figure No. 36. Average scores (By level of Living standards)

When learning achievement has been considered in association with hunger among students experienced in their homes and at their schools it has given the following results. The average performance scores was 31.42% for those who are always hungry, which has indicated lower performance than of students who have answered that they feel hungry sometimes or feel hungry outside of or not during lessons and those who feel hungry occasionally by 13.4% which indicates that hunger is the factor that affects negatively to the learning achievement of students.

FACTORS DEPENDENT ON THE INTERNAL ENVIRONMENT

(Physical infrastructure for training, training inventories, teacher methodologies and skills, in-service teacher training, psychology of teaching, environment for creative actions, relationship and communication between classmates and within schools, leisure time and promotion of personality development)

When students were surveyed on how long they have been studying together with their classmates 8.8% responded that they have been studying for the first year, 10% for the second year, 4.8% for the third year, 34.4% for the fourth year and 41.8% for the fifth year with their classmates at the same school. In fact, it would not be so well justified to consider that the number of study years by students in the same class does not have significant impact over learning achievement of students hence learning achievement of students who have studied for 2-4 years in the same class has been higher than those students who have studied in the same class for 5 years.

Page 91: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 91

Figure No. 37. Comparison of Learning Achievement with Number of Years Studied in Same Class

74.8-94.8% of students had possession of necessary clothing items and learning inventories when the availability of school uniforms, complete set of textbooks for grade 5, warm clothing for winter season and sport wear for physical training was considered.

Possessions of the following items have not had significant impact over their learning achievement. However, the following things or factors could be related with the state of living standards of families.

Figure No. 38. Learning Achievement Associated with Material in Possession of Students

35.2% of students own of computers and the average their performance scoring was 52.3% (533) and 64.7% had no computers and the average performance scores was 45.8% (495). 18.5% of all students had access to the internet and there has been no significant change observed in learning achievements of students with respect to ones who have and those who do not have access to the internet. In other words, the availability of computers and access to the internet are not significant factors that currently impact learning achievement. However, when availability of books at home, and places of living residence of students has been looked at whether they impact learning achievement, it has revealed that learning achievement of students who have had books were higher comparing to those who had no books. Availability of a library with books for reference and use

Page 92: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 92

has had quite a significant impact over their learning achievement. Motivation and interest of students to read have been growing stronger in accordance with the number of books available in their home libraries.

Figure No. 39. Learning Achievement Associated with the Number of Books at Homes

Students have been spending most of their leisure time during school days on such activities as doing their home work, reading at their school library, helping out with household chores, taking care of their siblings, meeting their friends, playing various games, watching TV. However, students have been spending most of their time watching TV and helping with household chores, when a question was asked on time that was spent for earning income to support their family livelihood was surveyed.

Figure No. 40. Learning Achievement Associated with the Number of Hours Spent on Activities during Leisure Time

When habits of spending of their free time was looked at in association with learning achievement the performance of students who have been spending most of their free time on doing their home work was the highest. 14.9% of students have been spending a certain amount of time for work to earn some money and that has been affecting negatively in regard to their learning achievement. Meeting friends, playing various games, taking care of siblings, attending libraries have not been influencing much learning achievement. Helping with household chores influences student’s upbringing and character development. Most primary education students have not been using library books but the performance of students who go to libraries and read books have been better and influencing their learning achievement positively. Learning achievement or the average performance scores of students who go

Page 93: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 93

to libraries 2-3 times per week was 50.5% and for students who have not been reading books in library the performance has been 47.3%, hence it shows that there is a need for enriching libraries with books and publication written for children and increase the use of library books by children. Learning achievement of students who have been spending 1-2 hours a day meeting their friends and doing their homework has been higher indicating that the practice of giving amount of homework for primary level students which would take about 1-2 hours for them to complete has been an appropriate amount for maintaining and supporting learning motivation of students. Also meeting friends for 1-2 hours a day was an appropriate amount of time for students to socialize, however, in other circumstances it tends to not be useful for meeting and wastes their time. Learning achievement of students who attend extracurricular activities and other activities, and trainings was higher by 5.6% in comparison to students who do not. An analysis of textbook supply in connection with learning achievement shown that purchasing new textbooks has positive impact on learning achievements. When parents and guardians have been surveyed with questions about issues related to teachers praising their children, scolding, violation of their freedoms, being praised by their classmates, or being bullied or beaten by others at their schools, getting involved in fights and conflicts, using swear words, damaging school assets and properties, and missing class, they have mostly answered that regular praising or scolding of their children have not had any significant impact over the learning achievement of their children in this school year. However, learning achievement of students who have been bullied or beaten by others or have gotten involved in fights and conflicts, who use swear words and damage school property and assets and miss classes have been lower by 5-10 percent in comparison to the performance of other students. Being involved in such issues either regularly or sometimes has been causing for the lack of motivation and interest of students to learn and would likely lead or result to students not having interest in attending school. Parents and guardians of primary school students have evaluated the quality of primary education as being unsatisfactory. The average learning achievements of students whose parents or 51.4% of them have assessed the quality of primary education as good has been at 49.6% (517). Also 80% of parents have assessed teachers with master’s degrees as highly experienced and good teachers of the school that their children attend. In considering in-service teacher training of school administrators staff it has been revealed that 68.4% of them have attended 1-5 days training on education management issues, 32.1 percent have attended training on pedagogy and educational psychology, 31.5% on teaching methodology and approaches, 16.3% on computer use and application, and 15.1% have attended one month training on improving English language and computing courses. And they have attended the aforementioned in-service training courses on average of 1-5 times. 15.2% of school administrators staff involved in the surveying have graduated Management programs at the Academy of Management. However, involvement experience of school administrative staff in in-service training has not been influencing learning achievement of students much. 57.6% of school administrators have answered as good to the question of whether they have been paying attention to school based in-service training of teachers, 39.5% as satisfactory, and 2.6% as they have been paying poor attention to the organization of school based in-service training activities. Subsequently, when it was asked how the results were 87% responded as having positive results and 12.8% as not effective. Learning achievements of students from those schools that have been paying a sufficient amount of attention to school based in-service training of teachers was higher by 6.9% in comparison to schools that have been paying poor attention to school based in-service training of their teachers. Teachers have been involved in short term in-service teacher training seminars and workshops on methodologies of basic mathematics and Mongolian language, pedagogy and psychology, computing and English language programs. But the qualities of trainings have been inconsistent or variable when they were asked on how learning achievement of students has been related with their involvement in-service teacher training. Based on the aforementioned facts a conclusion can be made that in-service teacher training is seen by most teachers as attending in-service teacher training workshops and seminars. Teachers have been involved in short and long term in-service trainings organized in the form of seminars and workshops in the capital city, aimag center and at schools which are being organized in accordance with plans approved and require payments. Results and outputs

Page 94: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 94

of these seminars have been inconsistent and variable depending upon who organizes them. But learning achievement of students have not been dependent on where training seminars and workshops are being organized, moreover, short term, high quality, useful, demand driven training seminars could be organized at the schools. As a priority for training areas teachers need to have in-service training programs organized on teaching methodologies, pedagogy and psychology, student character development and methodology of evaluation and assessment. When school administrators were surveyed on whether their school teachers and non-teaching staff have been involved in ethical issues they have answered that issues and problems related with as being involved in theft or robbery, drinking alcohol in the workplace, putting up fights, use of swear words, violating students rights and freedom, damaging school properties, missing work, and teachers coming late for classes or finishing their lessons early have been occurring regularly. 30.3% of teachers and non-teaching staff are involved in ethical issues mentioned above. 49.3% of teachers have been missing work, 78.7% have had problems of coming late to class or finishing their lessons early which has negative affects learning achievement of students. Teachers missing their lessons have more negative impact over learning achievement of students than coming late to class.

Figure No. 41. Learning Achievement Associated with Teachers missing their lessons and Teachers completing their lessons early

However, 1.5% of all teachers and non-teaching staff have been involved in theft or robbery, 30.5% of them have drunk alcohol at the workplace, 3.4% have put up with fights, 36.8% of them have used swear words offending students, and 12% have had problems related with violating children’s rights and freedom which are all indeed serious incidents in breach of teachers’ ethical norms and regulations and could negatively affect learning, upbringing and morale of students. In addition to teachers and non-teaching staff, school administrators were also surveyed on how often similar incidents have been occuring among students. Survey results have shown that 4.9% of students have been involved in theft and robbery, 33.2% have been involved in bullying each other, 22.4% have been putting up fights and conflicts, 56.3% used swear words and 14% have been involved in issues related to damaging school property in this school year. Learning achievement of primary school students have had the most negative affects when students were missing classes. The average performanc scores of students who have not been missing class were 49% and for students who have been missing class the achievement was 36.5%. Involvement of teachers of primary education, students and non-teaching staff in ethical issues mentioned above has clearly indicated that increased attention should be paid to the issue that only warm-hearted people could educate and produce the same people. Teachers have accepted that the aforementioned ethical issues exist to a certain extent when researchers have

Page 95: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 95

clarified further about the objectivity of those ethical issues from teachers and have emphasized to take into account many social and family based factors which were having influence over those problems. 11% of teachers surveyed have been spending 1-2 hours in preparation for their lessons, 37% have spent 3-4 hours, 23% have spent 5-6 hours and 28% spend more than 7 hours on weekdays when they have been surveyed on how much time they spend in fullfilling certain type of activities, inlcuding preparatory work for lessons. 11.8% of teachers have been teaching payment based tutorial lessons but when learning achievement of teachers who have been teaching fee based tutorials it was 54.4% of the average percentage of their performance scores. This fact allows us to make a conclusion that only a few number of teachers are involved in teaching of payment based tutorial lessons and learning achievement of their students have been higher in comparison to students of other teachers. Also 95% of all teachers surveyed have spent on average more than 1 hour during each working day a week doing their household chores but learning achievement has not been dependent much from the number of hours teachers have spent doing their household chores. 12.4% of teachers have been doing other jobs to complement their income. Over 60% of teachers have been engaged in team work and cooperation when teachers were surveyed on how frequently and how many hours they have been working together on a daily basis and per week on tasks on teaching of particular concept or theme, preparation of teaching materials, peer observation of teacher lessons and exchange of experiences etc. Learning achievements of students whose teachers have been involved in cooperation with one another was higher in comparison to learning achievement of those teachers who have not been engaged in cooperation hence proving how cooperation among teachers has been more fruitful and effective. However, 30.3% of teachers have responded that they allow other teachers to attend and observe their classes, but it did not have direct positive impact over learning achievement of their students.

Figure No. 42. Learning Achievement Associated with Cooperation and Team work of Teachers on Concrete Issues

That has shown that exchange of experiences among teachers themselves has significant impact over improvement of learning achievement of their students.

3.3 OTHER FACTORS HAVING IMPACT ON MATHEMATICS AND READING

Results of primary education assessment carried out on assessment of mathematics and reading of students have clearly shown that this has been an effective and yet an important study.

Factors which have been having impact over learning achievement of students were mentioned in detail in the previous chapter, but in this chapter we have focused on other factors which have been affecting learning achievement but were not mentioned in previous chapters.

Students themselves have answered that 31.8% have succeeded in mathematics and acquired with A, and 44.3% got B, 16.8% attained C level, and 5.8% received D and 1.3% with F respectively when they were surveyed what was their performance in mathematics. Moreover, when learning achievements were compared those students

Page 96: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 96

who have assessed themselves as having good performance results in mathematics have shown similarly good results in testing of their reading as well.

Figure No. 43. Self assessment of Students of their Mathematics Performance Compared with learning achievement in Mathematics and Reading

Following the results have been obtained when self assessments of students in Mongolian language subject

have been compared with their performance results.

Figure No. 44. Self Assessment of Students of their Performance in Mongolian Language Compared with Their Learning Achievement in Mathematics and Reading

These results have indicated that teachers of primary education tend to assess and evaluate their students by comparing students among themselves and not against the standard provision of knowledge and skills.

71.5% of students have possessed textbooks purchased in mathematics and reading when they were surveyed about the supply of textbooks in mathematics and reading skill development classes. However, 7% of students surveyed did not acquire textbook in mathematics and 9% did not acquire textbooks in reading .

Page 97: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 97

Figure No. 45. Availability of Textbooks in Mathematics and Reading among Students

When an issue of how learning achievements were associated with the supply of textbook has been taken into consideration it has clearly shown that the average performance of students with textbooks in mathematics received 483 scores whereas students with no textbooks in mathematics have shown learning achievement of 460 scores and similarly the performance of students with textbooks in reading was 475 and a score of 464 for ones who had no textbooks in reading. Learning achievement of students with new textbooks in mathematics and reading purchased and possessed by them was on average 520 scores when an issue of whether learning achievement is associated with conditions of the availability of textbooks among students such as whether they have purchased and possess new books, whether textbooks are used by one student or shared among many, whether it is used regularly or has to be returned on a certain date, etc. So results have indicated that learning achievement of students has been dependent upon the possession of own textbooks by students and having new textbooks has positive impacts on learning achievement of students in their respective subjects.

99.2% of students have assessed their teachers as good in mathematics and 86.3% have assessed their Mongolian language teachers as good when they were asked whether they would agree with good teaching of their subjects by Mongolian and mathematics teachers. 96.9% of students have expressed that knowledge gained through mathematics lesson will be of use further in their study whereas 87% have considered that knowledge obtained through Mongolian language subject will be of use in their further studies. Learning achievement of students who have assessed teaching of their mathematics teachers as not good was 511 scores in mathematics, and 505 in reading, and learning achievement of students who have assessed their teachers as good in Mongolian language was 514 in mathematics and 509 in reading. Learning achievement of those students who have believed that knowledge gained in mathematics will be used in the future has achieved 514 scores in mathematics and 509 in reading. On the other hand, learning achievement of those students who have believed that knowledge gained in Mongolian language will be used in their further study have achieved 516 scores in mathematics and 512 in reading. The aforementioned facts have indicated that most students believe that the quality of teaching by their teachers has been good and knowledge acquired through mathematics and reading will be useful further in their studies. Learning achievement of students in mathematics who have believed that mathematical knowledge would be of importance in their further studies was among the highest performances in mathematics.

Also students have answered that 70.2% of math teachers treat them with respect, 27.7% of teachers occasionally treat their students with respect, 1.9% have no respect for them at all and students also responded that 67.5% of Mongolian language teachers always treat them with respect, 29% respect them occasionally, and 2.6% do not treat them respectfully at all when students were surveyed about their respect towards their teachers and vice

Page 98: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 98

versa about teachers respecting them. Learning achievements of students were strongly associated with the respect of their teachers. Therefore, teachers should consistently treat their students as a party or subject who deserves their full respect and acquire communicative skills for better interaction with their students so that it would both be pleasant and satisfying for students but most importantly, it creates more possibilities for higher learning achievement of their students.

66.1% of students have responded that mathematics has been easy for them to understand whereas 67.1% consider that Mongolian language has been an easy subject. Learning achievement of students who have considered that mathematics and Mongolian language were easy subjects for them have shown higher performance results in their tests in comparison to the performance results of students who have considered those subjects and not easy from time to time and moreover, these students who have considered these subjects as easy were students who have been doing good in their studies and who have better basic knowledge and good learning skills which have made easy for them to study.

Figure No. 46. Learning Achievement of Students Associated with Students Perception of Mathematics and Mongolian Language as Easy Subjects

Poor understanding of a subject or lack of understanding of the subject matter by a student at all from the very beginning of the subject teaching have been inflicting a lot of pressure over a child’s psychology. Because of these circumstances it is most likely that they cause shortcomings to occur in their education as falling behind the rest of the students in class, and becoming introverted and more susceptible to being bullied by others or to copy and imitate others. Hence, children have replied that the following issues have been confronted by them a lot when some of the issues within this national evaluation have been considered as whether they feel ashamed from their classmates and friends because of not understanding mathematics, whether they have been bullied by their peers or get scolded by their family members.

Table No. 35. Learning Achievement Associated with Students Feeling Ashamed from Others, Bullied by Others and Scolded by Adults Because of Lack of Understanding of Subjects

Mathematics Reading

Always Sometimes Never Always Sometimes Never

I feel ashamed from my classmates and friends because of not understanding in classes;

456 496 542 446 496 534

Page 99: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 99

I get bullied by my peers because of my poor study and performance in class;

442 463 534 433 455 532

I get scolded by my family because of lack of understanding of lessons that are taught;

448 493 543 436 494 533

I feel ashamed from my classmates and friends because of not understanding of classes;

450 490 536 440 484 534

I get bullied by my peers because of poor study and performance in classes;

437 457 535 424 447 531

I get scolded by my family members because of lack of understanding of lessons;

442 494 538 431 489 534

National assessment have shown that all people who have been working in the educational system should

pay attention towards the elimination of ill practices from happening such as students being scolded because of poor performance or not being able to understand the subject matter, and being bullied by peers or feel ashamed by others. Since learning achievement of students who regularly feel ashamed of not being able to understand or been scolded by their family or bullied by their peers because of poor study have shown very unsatisfactory test performance which could be related with psychological pressure that children have to be faced with because of issues caused by poor achievement. Poor understanding could become de-motivated to attend classes and learn during normal or regular school days. Also, this could be related to poor level of family living standards, or communication and misunderstanding existing between students and teachers, or poor internal atmosphere built among classmates. All aforementioned problems have been having strong negative impact over learning achievement of students.

Students who have been scolded by teachers because of poor understanding have performed unsatisfactorily in Mongolian language and mathematics related to the aforementioned problems and issues.

Figure No. 47. Learning Achievement Associated with Students Who have been Scolded by Teachers because of Poor Understanding

Page 100: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 100

The most number of additional questions are being asked by students in mathematics as it has been revealed from an analysis of how well teachers have been explaining to students about items which were not understood during their lessons.

Figure No. 48. Learning Achievement Associated with the Teachers Explanation of Items that Were Not Understood by Students during their Lessons

Based on the above facts a conclusion could be made that students who have been lagging behind their

peers and studying poorly have been genuinely interested in asking additional questions from their teachers to understand their subject content. Learning achievement of students have been affected positively if teachers have been providing students with additional explanations on items and content which have not been picked up by them during lessons with genuine commitment and desire.

An analysis has been made on the responses students have provided to questions surveyed on issues related with homework in mathematics and Mongolian language including issues whether they have been able to do their homework on their own, whether home work has been difficult, who they get assistance from, whether home tasks are checked by teachers, and whether calculator is allowed to be used in mathematics.

78.2% of teachers replied that home work is given almost after all lessons, and 8.4% responded that for homework is given for half of all lessons and 13.4% have said that their students are given homework on some parts of lesson content when teachers have been surveyed with questions of how often they are given homework and for how much time teachers expect for their students to complete their homework. Also teachers have the following time estimations needed for completion of homework given to their students: 45.8% target 5-30 minutes, 42.4% estimate for homework to be completed within 31-60 minutes, 7.1% between 61-90 minutes, 3.4% for 91-120 minutes, 0.7% for less than 15 minutes, and 0.4% for more than 120 minutes. The most optimal homework have been homework that would require less than an hour when timing has been associated with learning achievement.

Page 101: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 101

Figure No. 49. Learning Achievement Associated with Students Home Work Items

In fact, 50% of students have replied that it has not been difficult to do their home work in Mongolian

language on their own and 44% have considered that doing their homework on their own in mathematics have not been difficult. 33% of students always receive support and assistance from their friends and family members in doing their homework, 41% get occasional assistance, and 26% have not been receiving assistance at all. 12% of students have said that they use calculator always during their math classes, 6.6% use it occasionally and 81.5% do not use it at all when students were surveyed whether they are allowed to use calculator during math classes. 10.4% of students have always copied their homework from others, 12.4% copy sometimes and 77.2% have never copied homework from others. Also 66% of students have replied that their homework is always checked by teachers, 27.3% have said teachers sometimes check homework, and 6.7% have said that their teachers never check their homework. Learning achievement of both groups of students who have responded that doing homework has been difficult to do on their own and who have said it has always been difficult to do their homework were both lower than of learning achievement of students who have said that doing homework in both mathematics and Mongolian language has not been difficult for them. Learning achievement of students whose teachers have been checking their homework always was high, whereas the learning achievements of students whose teachers have been checking their homework sometimes or never have decreased. These facts indicate that learning achievement could most likely be enhanced and positively affected if teachers give their students homework in the amount to be

Page 102: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 102

completed within few hours or 1-2 hours and are engaged in regular checking. Figure No.50 shown below indicates that learning achievement is affected negatively if primary school students are always allowed to use calculator during their math lessons. In addition, parent representatives have been surveyed with the question of who provides their children with assistance and support in doing homework and the results have been compared with their learning achievements.

Figure No. 50. Learning Achievement Associated with the Assistance Students Get in Doing Their Homework

From these data it can be concluded that learning achievement of students is affected positively if they do

their homework on their own without getting assistance from anyone. Students have been engaged in copying of their homework from others when they cannot do them on their own and teachers are engaged in their checking.

30-35% of teachers always use teaching materials and equipment in presenting their lessons, 33-43% use it sometimes and 21.5-36% do not use them at all when the use of teaching materials and training equipment by teachers have been surveyed from students.

Figure No. 51. Learning Achievement Associated with the Use of Teaching Materials and Training Equipment by Teachers

Uses of teaching materials have had more positive impact over learning achievement of students in

comparison to the use of teaching equipment and technologies when these factors have been associated with learning achievement. That might be related with improper, not effective and rational use of teaching equipment

Page 103: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 103

and technologies by teachers. Use of teaching materials and teaching equipment sometimes has been associated with better performance of students in both mathematics and Mongolian language which might indicate that the constant use of teaching materials and training equipment or no use of them at all could likely be not optimal. The above facts indicate that occasional and rational use of teaching materials and training equipment during math and Mongolian language lessons of primary education might have positive impact on learning achievements. Teachers have answered that they have been using teaching materials, handouts mostly when they were surveyed about what other training inventories they use themselves and whether these have been having impact over learning achievement in addition to the use of teaching materials and kits, training equipment and technologies. However, computers, LCD projectors have almost never been used by them whereas models and objects, things that have been created by students, and technical equipment are being used occasionally as many teachers have replied. It is assumed that learning achievement shall be enhanced if those teaching equipment and technologies will be used, however, the use of objects and models created by students have been more effective if these teaching materials and training equipment are distinguished in accordance with their importance.

Figure No. 52. Learning Achievement Associated with Use of Teaching Materials by Teachers

Development of students’ self-study skills has been more effective as analysis and results of national

assessment have revealed. It has been shown that it is not important to always use and apply technical equipment and other inventories or give lots of homework, but rather select methods and approaches which have been suitable for the age and specifics of students’ psychology to develop their skills were most important in enhancing their learning achievement.

Teachers have used many methods and approaches in evaluating and assessing their students’ performance, of which 36.7% of teachers used regularly and 62.2% occasionally, items and tasks that they have developed in mathematics and similarly, 60.4% of teachers have used regularly and 38.5% occasionally items and tasks that teachers have developed on their own during Mongolian language lessons. 43.2% of teachers have been using verbal responses during their math classes always whereas during Mongolian language classes it has been at 50.2%. Also 42.2% of teachers have been assessing students by their homework, and 45.2% of teachers assess their students regularly by creative tasks completed by them during math classes whereas 40.7% of Mongolian language teachers have assessed their students regularly according to their completion of creative tasks. In addition, 69% of teachers have replied that they use creative tasks to assess their students not always but occasionally.

Page 104: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 104

Table No. 36. Learning Achievement Compared by Teacher Evaluation of Students

Subject

Form

Learning achievement in mathematics

Learning achievement in reading comprehensions

Always Sometimes Never Always Sometimes Never

Mat

hem

atic

s

With own developed tests 516 502 442 509 497 452

By conventional task developed by a teacher

504 511 432 501 503 443

By oral responses provided 503 511 527 504 503 503

By home work 499 519 484 500 510 478

By a work performed individually 508 504 347 504 500 379

By creative task 502 510 505 497 503 497

Mon

golia

n la

ngua

ge

With own developed tests 505 509 442 500 506 452

By a conventional task developed by teacher

508 509 517 506 500 419

By given oral answer 510 509 475 507 502 454

By home work 498 521 476 500 510 467

By a work completed independently

509 508 440 500 505 427

Creative task 498 510 485 499 503 482

The aforementioned facts have shown that teachers have been using a variety of methods and approaches in

assessing and evaluating their students. Moreover, these methods and approaches have had important impacts over the improvement of learning achievement of students. Assessment of students by performance and fulfillment of tasks and items developed by teachers themselves, and by self-study or creative work completed by students on a regular basis have shown that these approaches have been more successful in comparison to use other forms of assessment as verbal response based assessment, performance of students home tasks or other creative assessment methods. Independent testing and other examinations have been used in our country but in reality most primary education teachers have been using widely conventional assessment methods and approaches. However, 66.2% of teachers have responded that they assess their students by tests developed by them whereas students have replied that 66.7% of their teachers use tests occasionally and 30% of teachers have used tests regularly. Hence, it indicates that efforts should be undertaken to upgrade skills and experience of students in doing tests from their current existing levels and practices.

11.5% of teachers were assessed as very good, 67% as good, 17.5% as satisfactory and 2.5% as poor when professional teacher skills have been requested to be assessed by school managers for those math and Mongolian language teachers who have been teaching in classes involved in the national assessment.

Page 105: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 105

Figure No. 53 . Teacher Professional Qualification Associated with Learning Achievement of Students

It could be judged that school administrators have assessed their teachers realistically and objectively since

learning achievement of students who have been taught by those teachers who have been assessed as very good and good teachers by their managers were higher than the learning achievement of teachers who have been assessed as satisfactory and poor. This fact also shows that students who have been taught by good and professionally qualified teachers have more opportunities to achieve better learning results and performance in their learning.

Furthermore, teachers have been requested to assess students who were involved in the national assessment in respect to their levels of speaking, reading, listening and writing skills using the following three assessment criteria as satisfactory, not satisfactory and medium level. Students involved in the national assessment were assessed with the following results: 56% were assessed as satisfactory and 43.5% as in medium level in their speaking, and 68.6% as satisfactory and 29.8% as in medium level in their reading, and 67% as satisfactory and 32.8% as in medium level in their listening, and 50% were assessed as satisfactory and 49.3% as in medium level in their writing skills by their teachers. And if these assessment results are compared with outputs observed from the national assessment than teachers have assessed their students quite higher. It was most likely that teachers have assessed those students involved in the assessment by comparing them with other students of their schools. In fact, students who had been assessed as not satisfactory in their speaking and listening skills have performed better in national assessment tests in comparison to those students who had been assessed as satisfactory in the same skills by their teachers. It also could indicate that their assessment methodology has been not realistic and objective. On the other hand, it is more likely for primary education teachers to assess their students a bit higher since assessment had been done by teachers themselves.

Page 106: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 106

CHAPTER 4: METHODOLOGY AND APPROACHES OF THE NATIONAL ASSESSMENT

4.1 SAMPLING

Two core objectives of the READ Language and Mathematics Assessment were a) to report on achievement

for urban and rural students, and b) Mongolian-language and Kazakh-language students. Another requirement was to follow, as closely as possible, TIMSS 2007 sampling procedures since the EEC had experience with such procedures. The Mongolian TIMSS 2007 sample design used private-public schools as the stratification variable. To address the reporting requirements, two additional strata of school location and school language instruction were added. Schools were sampled with probability proportional to size from each stratum and then intact classes were sampled from selected schools.

Sample designs such as that used in the READ Language and Mathematics Assessment typically allocate the number of schools to be selected from each stratum using one of two methods. First, the number of schools to be sampled from each stratum may be allocated according to the student population in that stratum. For example, if the percent of students in the Mongolian-language schools was 95 percent, then the 95 percent of the sampled schools should come from Mongolian-language schools. Second, the number of schools to be selected from each stratum is based upon reporting requirements. Having an equal number of schools in each stratum will result in approximately equal sampling precision for each stratum, regardless of the stratum size. The READ Language and Mathematics Assessment sample design represents a compromise between these two sampling approaches while favoring the latter approach. Thus, Kazakh language schools are over-sampled as are rural schools. Schools were sampled with probability proportional to size within each stratum.19

The READ Language and Mathematics Assessment used a two-stage stratified cluster sample design. Following TIMSS, the sampling precision for the national estimate of language and mathematics achievement was set at the same precision as would result from a simple random sample of 400 students. Assuming that students within schools are somewhat similar to each other but statistically different from students in other schools, the target school sample was to be at least 150 schools and approximately 4500 students. The need to over-sample small, rural schools and the desire to sample intact classes increased this sample size slightly. All Kazakh schools were located in rural areas and were sampled. This represents an over-sampling of Kazakh schools and students and thus sample weights much be used in any analyses involving Kazakhs.

Sample Weights23

The sample weight calculations follow the three steps in the Mongolian Language and Mathematics Assessment sample design. First school sample weights are calculated, followed by class weights and finally student weights.

(a) School sample weights The first stage weight represented the inverse of the first stage selection probability assigned to a sampled

school. The sample design required that school selection probabilities be proportional to the school size (PPS) school size being enrolment in grade 5. The first stage weight for the ith school was thus defined as:

isc

i

MBWn m

23 This sampling strategy was described in my visit workshop visit to the EEC and is detailed in the report on that first visit.

Page 107: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 107

where n was the number of sampled schools in the stratum mi was the measure of size for the ith school and M is the number of students in the stratum.

First stage weights were calculated for all sampled schools and replacement schools that participated. A school-level participation adjustment was required to compensate for a Kazakh school that was sampled but did not participate and was not replaced. Two special schools that were sampled were removed from the calculation of this adjustment. The school-level participation adjustment was calculated separately for each explicit stratum. The adjustment was calculated as follows:

1

1

s r nrsc

s r

n n nAn n

where sn was the number of originally sampled schools that participated, and 1rn the number of first replacement

schools, that participated, and nrn the number of schools that did not participate. The final first stage weight for the ith school, corrected for nonparticipating schools, thus became:

i isc sc scF A BW

(b) Class sample weights

The second stage weight represented the inverse of the second stage selection probability assigned to a sampled classroom. The sampled intact classrooms using equal probability sampling. For the ith school, let Ci be the total number of classrooms and ci the number of sampled classrooms. Using equal probability sampling, the final second stage weight assigned to all sampled classrooms in the ith school was

1

ii

cl i

CFWc

(c) Student sample weights

The third stage weight represented the inverse of the third stage selection probability attached to a sampled student. Since intact classrooms were sampled, then the basic third stage weight for the j th classroom in the i th school was simply

,1 1.0i j

stBW The student non-participation adjustment was calculated separately for each participating classroom as follows:

, ,

,,

i j i ji j rs nrst i j

rs

s sAs

The third and final stage weight for the jth classroom in the ith school thus became

, , ,1 1

i j i j i jst st stFW A BW

The numbers used to compute the student adjustment were derived from the actual test administration form and thus take into account student transfer out of the schools (these students are excluded), students who transferred into the class or school (these students are included) and also any students who were missing or absent from the test.

Page 108: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 108

(d) Overall weight The overall sampling weight was simply the product of the final first stage weight, the final second stage weight, and the final third stage weight. Since intact classrooms were tested the overall sampling weight was

, , , , ,1 1

i j i j i i j i j i jsc sc cl st stW A BW FW A BW

It is important to note that sampling weights vary by school and classroom, but that students within the same classroom have the same sampling weights.

The overall weight gives an estimate of the total population size and is sometimes called the Total Weight (totwgt). This weight can be algebraically adjusted back to the sample size. The adjusted weight is called the house weight (houwgt) and gives the weighted contribution of each assessed student to the sample.

The sample called for students in 169 schools to be tested. Two Ulaan Baatar special schools (identification numbers 495 and 523) were excluded after sampling. The number of schools sampled in this stratum was adjusted down to reflect this change. Two Russian language schools (identification numbers 540 and 537) were replaced by Mongolian language schools (identification numbers 600 and 499). The replacement schools were the next schools in the ordered-according to size list of schools within the stratum and thus are the first replacement schools. Since these Russian-language schools were in an urban, private stratum they have effectively been retained in this stratum.20 One Kazakh school (identification number 35) did not participate, reportedly due to adverse weather conditions. The adjustment for school weights was used for this school.

Target and Achieved Sample Proportions

The sample was designed to represent reporting-segments of the grade 5 population. As a quality control check, and also to facilitate communication to stakeholders, this section compares the numbers and proportions of the sample, unweighted and weighted, to target numbers and proportions. The target numbers and proportions are obtained from the census data collected by the Ministry of Education at the start of the school year.

The weighted samples are very close to the desired, target numbers. For example, the number of students in Mongolian-language schools is 43,585 and the weighted sample is 43,434, while the number of students in Kazakh-language schools is 2484 and the weighted number is 2,182 (Table ). Note that the proportion of Kazakh students in the sample (15.4%) is effectively adjusted to approximately the correct proportion by the use of sample weights. Similarly the over sampling of bag students (5.4% of the sample, see Table ) is corrected through the use of sample weights.

Table No. 37: Numbers and proportion of Mongolian-language and Kazakh-language school students in weighted and unweighted samples.

Unweighted House Weight Total Weight Target Mongolian Language 4,020 (84.6) 4,523 (95.2) 43,434 (95.2) 43,585 (94.6) Kazakh Language 730 (15.4) 227 (4.8) 2,182 (4.8) 2,484 (5.4) Total 4,750 4,750 45,616 46,067

20 The presence of Russian-language schools was made know sometime after the second sample was drawn. Russian-language schools and English-language schools should have been excluded from the study since the testing languages were Mongolian and Kazakh.

Page 109: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 109

Table No. 38: Numbers and proportion of students by school location in weighted and unweighted samples

Unweighted House Weight Total Weight Target City Students 1,871 (39.4) 1,753 (36.9) 16,836 (36.9) 16,343 (34.5) Rural Students 2,879 (60.6) 2,997 (63.1) 28,779 (63.1) 29,728 (64.5) Total 4,750 4,750 45,615 46,067

Table No. 39: Numbers and proportion of students by school location in weighted and unweighted samples (Capital City, Aimag City, Soum, and Bag)

Unweighted House Weight Total Weight Target Capital City 1,871 (39.4) 1,753 (36.9) 16,836 (36.9) 16,408 (35.6) Aimag City 1,285 (27.1) 1,261 (26.5) 12,106 (26.5) 11,938 (25.9) Soum 1,330 (28) 1,627 (34.3) 15,627 (34.3) 16,763 (36.4) Bagh 264 (5.6) 109 (2.3) 1,046 (2.3) 1,023 (2.2) Total 4,750 4,750 45,615 46,067

Schools from 2 aimags were not involved in the sampling after the second sampling. These aimags students were with few. The sampling was designed for involving a national scale not only aimags level. Each involved students in the study they were representing a definite number of students. In according to student’s number of each schools, their representing were differently. For example: 23 students were participated in the study from Khairkhan soum of Arkhangai aimag and each students are representing 13, 5 students in nationwide.

Participation Rates

Two Russian language schools were sampled but since the assessments were to be conducted in Mongolian and Kazakh these schools were appropriately replaced by the next schools in the sampling frame. Thus there was only one validly sampled school that did not participate. The unweighted school participation rate is 99,4% (166 of 167).

The unweighted student participation rates, for each location, are shown in table. The student participation rates in all geographical locations were high. The lower participation rate for bagh schools may be partially explained by the higher sensitivity of the small sample to fluctuations to a few low participation rates.

Table 40: Unweighted participation rates for school locations

Location Number of sampled schools

Number of participating students

Unweighted student participation rate

Capital City 59 1,871 94.54 Aimag City 41 1,285 96.03 Soum 50 1,330 92.68 Bagh 16 264 83.28

Weighted School Participation Rates

Two weighted school-level participation rates were computed.

The denominator remains unchanged in both equations and is the weighted estimate of the total enrollment in the target population. The numerator, however, changes from one equation to the next. Only students from

Page 110: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 110

originally sampled schools were included in the first equation and students from first replacement schools were added in the second equation. They were as follows:

, ,

,1 2

, ,

,

si i j i j

sc cl sti jsc s

wtd s r ri i j i j

sc cl sti j

BW FW FWR

FW FW FW

1, ,

,11 2

, ,

,

s ri i j i j

sc cl sti jsc r

wtd s r ri i j i j

sc cl sti j

BW FW FWR

FW FW FW

sc cwtdR = weighted school participation rate for originally-sampled schools only,

1sc rwtdR = weighted school participation rate, including sampled and first replacement schools.

where both the numerator and denominator were summations over all responding students and the appropriate classroom-level and student-level sampling weights were used. Ω and Δ take the value one since no sub sampling was involved . Note that the basic school-level weight appears in the numerator, whereas the final school-level weight appears in the denominator.

The denominator remains unchanged in the two equations and is the weighted estimate of the total enrolment in the target population. The numerator, however, changes from one equation to the next. Only students from originally sampled schools and from classrooms with at least 50 percent of their students participating in the study were included in the first equation. Students from first replacement schools were added in the second equation.

The weighted school participation rates were 98% and 100% (after rounding). The weighted school participation rates are excellent.

The weighted student participation rate was computed as follows:

1, ,

,1

, ,

,

s ri i j i j

sc cl sti jst

wtd s ri i j i j

sc cl sti j

BW BW BWR

BW BW FW

where both the numerator and denominator were summations over all responding students from participating schools. Note that the basic student level weight appears in the numerator, whereas the final student-level weight appears in the denominator. Furthermore, the denominator in this formula was the same quantity that appears in the numerator of the weighted classroom-level participation rate for all participating schools, sampled and replacement. The weighted student participation rate is 95%. This is excellent.

Weighted Overall Participation Rates

Two weighted overall participation rates were computed. They were as follows: ov swtdR weighted overall participation rate for originally sampled schools only

1ov rwtdR weighted overall participation rate, including sampled and first replacement schools.

Page 111: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 111

Each weighted overall participation rate was defined as the product of the appropriate weighted school participation rate, weighted classroom participation rate and the weighted student participation rate. They were computed as follows:

ov s sc s cl stwtd wtd wtd wtdR R R R

1 1ov r sc r cl stwtd wtd wtd wtdR R R R

The weighted overall participation rate for originally sampled school is 93%. The weighted overall participation rate when the replacement schools are included is 95%. Overall, school level and student level participation rates meet international sampling requirements.

4.2 METHODOLOGY OF THE SURVEY

Twenty five groups went on a 3-15 day mission to organize nationwide research of elementary schools. One group of researchers planned to go to the provinces. The Education Evaluation Center (EEC), teachers, and Research Organization members worked together to develop the program.

The research team first attended a seminar on the purpose, direction, and methods of the research. They also received instruction booklets and other materials, and rehearsed conducting the research in Darkhan City “Od” Complex School.

After this training, the research began across 19 provinces (Dundgobi and Gobisumber provinces did not participate in the research), in 166 schools, with 4750 students, 165 school administrations and elementary school teachers taking part. The instructions given to the research team (teachers trained in the research methods, and the head of the province’s Elementary Education System) were:

Get information about how to organize the research, Attend training seminar Receive research materials Organize and conduct the research at the chosen schools Collect research data Report information about participants, including their attitude towards the research, and any complaints,

and well as the research method. Document the research process, and the debriefing with parents and students, using photographs and video

recording

Throughout the research process, team members worked with school administrators, students, teachers, and parents, by introducing them to the methods, purpose, and instructions of the research. In the chosen classes, the head of the province’s Elementary Education System, a representative of the school, and the class teacher went into the class, and greeted the students and introduced the research to create a healthy environment for the research. After that, they gave the survey to the students, and 5 minutes of instruction on how to complete the survey. After a set time, judged to be long enough for the students to give satisfactory answers, the researchers gathered up the surveys, regardless of the student’s progress, so that the students would not become disruptive.

During the survey students who asked for help with specific questions or answers in the survey were told, “Sorry, I can’t tell you that, you have to pick the answer you think is right.” Researchers did, however, give instructions on how to complete the survey. Students were not allowed to leave the room during the survey, except in special conditions (ie. The student is sick). If a student leaves the room, the researcher records it, collects the

Page 112: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 112

survey before the student leaves, and returns it to him or her when the student comes back. If a student was late for a research session, they were included in the next session.

After the data was collected, the researchers thanked the students and teachers. Then they met with the school administrators and parents to answer any questions they had and take complaints about the research. Questions and complaints were included in a written report.

Finally, the research data and reports were given to the EEC.

4.3 CREATION OF INFORMATION DATABASES, SOFTWARE PROGRAMMS APPLIED

Researchers created a program to enter student’s information, to create a database, and record monitoring.

Tools Used to Create the Program

Information Database was created using Borland Interbase 6.5 software Borland Delphi 6.0 software was used to create a program to track scores and register reports.

Structure

The software consists of 2 parts:

1. Information database – EDUCATIONCENTRE.GDB 2. Programs

a. StudentScoreEntry – This is used to compile students scores b. StudentInfo – Used to enter students information, comparison of registered scores, and

create reports.

Information database and the program will work as a client-server. The program will have an Ini file that will instruct you which server to use. The Ini file will appear as follows:

[Database] Server Address=(Server IP Address)

The information database will be located on the servers in

D:\My Documents\EDUCATIONCENTRE.GDB

“StudentInfo” is only accessible by an account with supervisor privileges.

“StudentScoreEntry” is only accessible by authorized StudentScoreEntry team members. Each member will have his or her own entry ID.

Page 113: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 113

Figure No. 54. Structure of Information Database

Tables in Information Database are stores under the following 2 categories

1. Main Registration a. REG_CLASS – Registration of school and classes b. REG_STUDENT – Student registration c. REG_SCORE – Open registration of students’ score d. REG_SCORE_ALL – Complete registration of the students’ score e. STUDENTS_RELATED – Students who were involved in the study f. STUDENTS_NOT_RELATED – Absent (sick, excused, unexcused) students]

2. Supplementary Information a. LU_USER_ACCOUNT – Program entry rights information b. LU_SCORE_TYPE – Open scoring code c. LU_TASK—Questionnaire items d. LU_TASK_ITEM—Multiple choice answers

Program Functioning

The program will have 3 steps

Page 114: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 114

1. “StudentInfo” will be used to register information of schools and students. 2. “StudentScoreEntry” will be used to register students’ score using the following instructions

a. Open ended responses /Open ended/ i. Students in the 20th percentile

ii. All students /100th percentile/ b. Multiple Choice answers /Hundred percent/

i. 100 percent ii. 20 percent

1. Teachers 2. EEC team

3. Use “StudentInfo” to do comparisons using the following instructions: a. Open ended question 100 percent with 20 percent b. Multiple choice 100 percent with EEC and Teachers registered 20% score

The greater the percent of the comparison, the more correct the student’s scoring registration is.

Explanations about “StudentScoreEntry” software.

When you start the program, there will be a log in window. Each user will be given an individual log in ID.

After logging in, there will be a main menu window on the right side of the computer screen

Score registration (open) instructions

This window is for entering the student’s 100 or 20 percent scoring information for open ended questions. One user cannot enter both the 100 and 20 percent information. If one user enters the 100 percent information, another user must enter the 20 percent information.

Page 115: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 115

The “Registered” and “question type” field will show the information of registered members. The type of information you will enter into the program will be there. For this window, “Test1” user will enter the Math score.

In the “School number” field, you must enter the student’s school number. You must then choose score registration type from the “Students” field. Next, click the “Choose” button. Now the results will be displayed in the window. Now you can enter the test scores for the students you have selected.

Registration of students score/Total

You will enter all of the student’s information in this window.

First enter the school number, then choose the question type, and click the “choose” button. Students’ information will then be displayed on the left side of the window. If scoring information has been entered, it will be displayed on the right side of the window. If no scoring information has been entered, you can enter it now, and click the “enter score” button.

This window shows complete scoring information.

If 100% of a student’s information is entered, his or her name will be highlighted in blue.

Student’s score registration (20%)

In this window, you will enter the multiple choice information for 20th percentile of the teachers, and EEC. One member of the team cannot enter the information for both groups. The steps for entering the 20th percentile score are the same as the 100th percentile.

Page 116: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 116

Settings Window

Window to look for and correct meaning of questions and answers

“StudentInfo” Program Explanation

When you start the program, the main window will appear on the left side of your screen.

Student’s information registration window

Page 117: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 117

To view this window, click “register” on the main window

Instructions:

1. Fill in the school and class information at the top of the window 2. Click “Save” button to save. 3. Import students information from a 5 column excels spreadsheet. 4. Click on the “Import” button. The spreadsheet should look like the following example. If the spreadsheet is

formatted incorrectly, there will be an error.

Amarmend Bat-Erdene 1996/3 AK96032815 2 Batdelger Batmagnai 1997/3 AK97030717 2 Bayanmunkh Byambasuren 1996/7 AK96072001 1 Bayanmunkh Byambadorj 1996/7 AK96072001 2

a. The first column is the Surname b. 2nd column is the given name c. 3rd column is year and month the student was born d. 4th registration number e. 5th gender

5. After importing the information, the student ID, Monitor number, and survey number will be displayed. 6. You will be able to change, fix, or add information later. 7. If you hit the “Print” button, a print preview will be displayed, and will print when you click the printer

icon.

Page 118: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 118

Student Information List

To get to this window, you click the “List” button on the main window.

Instructions:

1. If you click the “Choose” button without entering any information, you will see the school and class information. After you enter information into the input fields and click “Choose” again, the information you chose will be displayed.

2. If you choose any options from the drop down boxes and click “more,” that information will be displayed.

Open ended question code checking window

To get to this window, click the “Check Score” button on the main window. This window will show the results of the comparison of the 20% and 100% students for open ended questions.

Page 119: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 119

There are 2 methods of comparison

1. Use all the digits of the score – use complete score for the comparison 2. Use only the first digit of the score – Compare only raw scores.

Check “show differences,” and click the “Choose” button. A table will be displayed with the difference highlighted in yellow.

Information about who entered the data and the accuracy of the scoring will be displayed at the bottom of the window (The higher the percentage, the more accurate the scoring).

Multiple choice code checking window

To reach this window, click “Repair Check Score” button on the main window. Teachers or EEC teams 20% and 100% scores will be compared here, and displayed the same way as in the “Open ended question code checking window.”

Page 120: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 120

If you click the “Unregistered Schools List” button, the following window will appear:

Repair Window

This window will display the names of the students who were absent during the testing.

4.4 TRAINING SEMINARS AND WORKSHOPS

In October, 2006, the main researchers created several teams of research assistants, including 9 math teams, 9 literacy teams, 4 testing teams, 4 IT teams, 4 leader teams, and 8 helper teams. During the same month, a seminar was organized for the teams. In the seminar, the teams learned about the purpose of the research, national and international education research methods, the methods of scoring, and shared their research experience, and discussed potential problems that can occur during the research process.

In March, 2007, the members of the testing team were chosen. In April, the pilot tests were conducted. April, 2008, a study organization seminar was held, and preparations for the main project were completed.

In February, April, and October of 2008, seminars were held to prepare specialists for the evaluation of elementary education systems. The seminar was for City and Province School education managers and the heads of

Page 121: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 121

the Elementary Education system, and taught them how to organize the study, new scoring methods, how to do diagnostic scoring, and to conduct the same seminar in their own Provinces.

In May, 2008 the teachers who would score the open ended questions were chosen, and seminars were organized for those teachers. The open ended instruction book was created during this seminar, to avoid the problems of non-standardized scoring of the open ended questions.

4.5 QUALITY CONTROL/MONITORING

The first step of the quality control process was the creation of a quality control team, which worked on each step of the test creation. They also conducted research to choose the most appropriate questions for the surveys to be included in a question database, and examined the pilot research that had already been conducted, retesting. They analyzed and improved the test quality. They took part in the main survey process, and made instructions for scoring the tests, creating the database, and making reports. They reexamined teachers scoring for accuracy and fairness.

Researchers believe that general requirements were followed for the creation of the test creation process. The test included, multiple choice, fill in the blank, comparison, and open ended type questions for mathematics, Mongolian language, and literature following the Bloom and Solo taxonomy, and the TIMMS, PIRLS, and PISA tests. After the test was created, it was checked for quality and time for completion. If any test items did not meet quality standards, they were removed. The tests for students, parents, foster parents, Home Room advisors, teachers, and school administrators were discussed with national and international advisors. City and Province teachers who took part in the test, school administrators, and the heads of the Province Elementary Education systems were consulted. Each of these steps was conducted by members of the quality control team.

After the data collection was completed, researchers prepared the open ended question scoring teachers, and held a seminar and created the open ended question scoring instruction book to avoid the problems of non-standardized scoring. Each teacher checked 20% of another teacher’s scoring, and if any differences occurred, the team leader examined the entire test. The quality control team also worked on score coding, entered data into the database, and each member checked 20% of another member’s data entry.

Researchers created a program that will contain information of the students that took part in the study. The program works in the following way:

1. StudentScoreEntry – This part is used to record students score 2. StudentInfo – This part is used to record student information, and to compare score information.

This system will help to avoid any problems that may occur with the database.

Some students chosen for the survey, and who’s data had been entered into the system were absent during the survey. They were removed from the database, and replaced by another student, thus keeping all of the data accurate. After creating the database, researchers checked for mistakes as a final step before data analysis.

The RASH ConQuest program was used for test quality. A recommendation report was made for teachers and parents, following scientific guidelines, while also being easily understood by a layman.

Page 122: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 122

CHAPTER 5: CONCLUSIONS AND RECOMMENDATIONS

5.1 LEARNING ACHIEVEMENT OF MATHEMATICS AND READING

The Nationwide Math and Literacy Education Level research team make the following conclusions about the education level.

Foreign countries are using standard requirements (benchmarks). 4 levels of benchmarks, and ability level measurements, were created based on the results of this research. The unique aspect of this research is that it focused on the problems that will need to be addressed in the future.

What is the level of success of math and literature education?

The average math ability test score was 45.4% and literature was 48%. This shows that the student’s ability is substandard.

Students were less successful with open ended questions than multiple choice questions. Students who scored highly on the math test, also scored highly on the literature test. This shows that math

and literature skills are connected. These abilities are positively correlated to general success in other subjects.

Math test results show that geometry and addition levels are higher than probability and statistics. Literature test results show that writing levels are below reading and listening levels.

Kazak language schools’ average test score was lower than that of Mongolian language schools. Mongolian test scores are below the international average. Urban students scored 13% higher in math, and 10% higher in literature than rural students, and 24%

higher in math and 20% higher in literature than students in county subunits (very small, undeveloped towns). This shows urban students have higher math and literature levels than urban students.

Researchers found the following influences on student’s math and literature ability based on test results, and input from administrators, teachers, and parents.

5.2 WHAT FACTORS INFLUENCE LEARNING ACHIEVEMENT?

As the result of study the factors, which influenced the skill of mathematics and reading, including questionnaires from representatives from the school administrators people, methodologists from Aimag in charge of primary education, parents and guardians and learning achievement were studied and following factors were identified.

A. Influence of School Environment

Classes with a higher than average number of students (35), performed lower than classes with fewer students. Students who sit in pairs scored higher than students who sat alone or in groups greater than 2. This result was consistent in both urban and rural areas Students with a better classroom environment, measured by heating, furnishing, and cleanliness, scored higher than other students. Chairs and desks that were the wrong size for students had a negative influence on test performance. Students in specialized schools scored higher than those in regular schools.

Page 123: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 123

Students in schools with better classrooms, gymnasiums, and music classes scored higher than students in other schools. Industrial training and other training programs take a small amount of the income for public schools.

B. Influence of Extracurricular Activities and Environment

Students who receive training before school (ie. Preschool) are more successful. Students with a longer commute scored lower than students who lived closer to the school. Students who live in dormitories or with foster families scored lower than average. Students with better home environments (television, electronics, etc…) were more successful. And students in stable families scored higher then families in dysfunctional families. Students with a home library are more successful. The higher the parents’ education level, the more successful the students are. Students from families with substandard income scored lower than average. Students’ level of success was not significantly influenced by number of family members, number of employed family members, or number of people living in their household. Urban parents pay more attention to their children than rural parents. Students’ level of hunger at home and in school is correlated with level of success. This result implies that lunch programs should be continued and improved. All levels of respondents supported this program. One influence on rural students lack of success is the amount of time spent doing chores at home.

C. Individual differences in students

Gender did not influence test results. Students attending extracurricular classes scored higher on the tests. This finding implies that below average students should attend extracurricular classes. Basic knowledge and desire to learn has a direct influence on student’s success. Students who do not understand the lessons during class become depressed and discouraged. This leads to a lower desire to learn, and a higher dropout rate. Homework: giving creative work based on student’s ability and giving feedback on their performance can have a positive influence on students’ academic success. 5.8% of the students have some kind of disability, mostly vision problems. Students with vision, listening and bodily disabilities can be placed with regular students, but the success of mentally disabled students placed with regular students needs to be researched by professionals. 4.3% of the students have some kind of disease; Mostly ear, nose, and throat diseases. 1.7% of the students have been absent from school for extended periods before. The reasons can be, low desire to learn, poor psychological environment, sickness, or moving. Students who have been absent from school for extended periods ad less successful than average. Students who began school at age 6 scored higher than those who began school at age 7 or 8. Students who do homework individually have higher levels of success.

D. Teacher Influence

Page 124: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 124

Test results showed that math and literature classes should be taught by professional teachers. Teachers’ morality, responsibility, and discipline, and students’ upbringing and background will influence academic success. Academic success is also dependant on the level of the teachers’ class preparation. Teachers who are members of teaching teams that help each other prepare for classes, take part in each other’s classes, and discuss their work have more academically successful students. Teachers’ respect of the students positively influences academic success. Teachers who received relational training, and think of students as one side of the relationship, have more academically successful students. At an Elementary Education level, it is better to use teaching materials developed by the teacher and students, than to use computer s or LCD projectors. Teachers who take part in continuing education have more successful students. Students of teachers with long term, and those with short term teaching experience show no significant difference in test scores. Teachers’ participation in seminars shows no significant result on students’ scores. Oppression and negative psychological pressure from teachers is more common in rural areas.

E. Miscellaneous

Teachers want to have teacher development centers at their school, and to take part in seminars and training approximately once every 5 years. Most teachers want a salary system that rewards good performance. The study shows that teachers need training in the use of technology in education, teaching methods, psychology, student development, and assessment. Teachers rate their students higher than test scores indicate. Parents and foster parents are satisfied with teachers’ performance. The study shows that school administrators more accurately report teachers’ performance than in previous years. Elementary school teachers need more computer training. The survey shows that school administrators and teachers both want a national curriculum (same books used everywhere). Teachers report that greater supplies of computers and technology will improve academic performance. The positive impact administrators can have on students’ academic success comes from the attention they pay to teacher training. Teaching methods and effectiveness in urban areas is better than in rural areas. The study has shown that a favorable environment for study has been created more successfully in urban schools than in rural schools. Private school students score higher than public school students because there are fewer students in the class, teachers with a higher level of education, and a better learning environment.

5.3 HOW LEARNING ACHIEVEMENT CAN BE IMPROVED?

Based on findings and results of the national assessment a following proposal for some necessary yet concrete measures to be undertaken for implementation is being proposed with the overall purpose of enhancing learning achievement and outcomes.

Page 125: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 125

Table No. 41. Proposals or Directions of Work to be done

Whom Issues to tackle Direction of the work to be done Policy makers

Standard, textbooks of primary education: To undertake measures in connection with the content difficulty of primary education standards, unclear and not adequate terms and terminologies used; and lack of environmental standards;

To develop and adopt for standards and redevelop the standards

To develop and adopt standards of learning environment;

Revision of evaluation standards and procedures of primary education, and their enhancement;

To develop National Program of Education;

To have textbooks for every children

Teacher preparation process;

Issues to be tackled include poor quality of textbooks of primary education, and finalization of whether optional textbooks are rational;

To ensure for every student to possess textbooks and develop one high quality textbooks written in accordance with the standards

Teacher preparation activities Improving graduator’s quality of pedagogical universities. Teacher activities Enhancing teaching skills of primary education teachers, implement capacity building and development of teachers, establishment of student’s workload comprehensively, equip teachers with knowledge and skills and subsequently encourage them to give to students as their home work creative tasks that will match age and psychological development of their students;

Enroll and choose students willing to teach

To support for the pedagogical universities from government by policy.

Improving the teaching personnel of teacher training institutions and improvement of training programs and curriculums, particularly, enhancing content of didactic programs on teaching, thinking and research;

Teacher development centers established in each school and training seminars and workshops are organized effectively;

Comprehensive reforming of the national teacher in-service training system;

To establish rationally teacher workload and improve their team working skills;

Integrated tackling of social issues of a teacher;

Establishment of students workload in accordance with students age and psychological specifics;

To increase computer using of teachers To study in international item writing

methods To have tested and piloted test fund for

every teacher and schools To introduce the new technology and

effective methods in the study To work skill required items by students

permanently

Schools and teachers

Class size To abolish large classes which impacts negatively workload, health and quality of teaching and learning for teachers and students;

To maintain class size not larger than 35 students per class (to establish a vacancy of an assistant teacher per grade 1)

Improve interior environment, health and sanitation conditions and yet comfortable of classrooms and facilities;

Urban and rural location: To diminish difference in learning achievement between

To enhance environment and conditions of rural schools;

Page 126: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 126

urban and rural students; To regulate migration and movement of population;

To undertake measures with the aim to improve quality of teaching personnel of rural schools;

Co-operation: To enhance link and networking between schools, teachers and parents;

To enhance comprehensively related regulations and procedures;

To make school activities more transparent for parents and enhance their involvement and accountabilities;

To link primary education with pre-school education; To change low coverage of pre-school education and enhance insufficient quality of pre-school education activities;

To increase number of children to be covered by kindergartens;

To enhance quality and accessibility of pre-school education services;

To enhance link and integration of pres-school and secondary educations;

To enhance school library fund To pay attention to the books for kids a few and no enhancement for library fund

To enhance the school library for the students

To improve the textbook’s quality and distribution

To provide by textbooks and instruments To pay attention to their homework and for reason of backwardness connect with no textbook

To provide by textbooks and

instruments To provide by uniform

Students and parents

To involve in training along the interest To pay attention to involving in training, spending their free time a properly To have books at their home To pay attention to the learning achievement of students have books are higher than no books

To involve in training center and clubs for their children

To enhance by books and texts suitable

to their interests

Mathematics and reading comprehension: To carry out analysis and assessment of insufficient quality of mathematics and reading comprehension and enhance learning achievement and its quality;

To train students to undertake tests; To pay special attention for students to

acquire skills to solve and comprehend problems; comprehend and write texts;

To teach students to understand and work with data in figures, diagrams and tables;

Remarks: Duration period or timeframe in this table has been identified as short, mid and long terms. Short term implies to priority and urgent issues to be addressed, mid-term refers to actions to be undertaken in compulsory manner, and long-term refers to a measures that have to be implemented on regular and yet to be undertaken in compulsory manner.

5.4 RECOMMENDATIONS

The following recommendations, based on research findings, are provided to school administration, parents, and researchers, to enhance the quality of elementary education.

To Teachers:

Care for Your Students from the Bottom of Your Heart: You need to learn about the individual background of your students and develop a flexible relationship with them, especially with the less successful students. You can talk to the students privately, developing trust, talk to their parents, and give them creative work at their own developmental level. This will improve the student’s desire to learn.

Continual Development: Teachers should improve their foreign language and technological abilities, and use them as part of the teaching process, because this is a century of knowledge and information.

Page 127: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 127

Every Teacher is a Researcher: Analyzing your own teaching quality, student’s educational development, and students’ level of academic success, is the basis for effective work. That is why teachers should learn and become skilled with research methods.

Every Teacher Should Use Information Technology (IT) in Their Training and Research: Without improving your knowledge of IT, there is no way you can use it to enhance the quality of the education system.

Create an Opportunity to Work With Every Individual Student: You can improve the individual relationship with students through assigning teamwork, individual tutoring, and using parents as assistants.

Have a Passion for Your Work: The root of success is to have a passion for what you do.

Work Together With Parents: It is important to work with students’ parents to encourage the students’ independent learning and success.

Every Teacher Should Be Part of A Team: Creating a project, enhancing teaching methods, learning from other teachers, and solving problems is most effectively managed as a team.

Make Quality Preparations For Class: Be aware that how well you are prepared for class has a direct impact on your students’ level of success.

Use Assessment Effectively: The most effective measure of your performance is the results of your students’ tests. That is why it is important that teachers be skilled at creating tests, interpreting results, and also to teach your students test taking skills.

Every Teacher Should Maintain a Test Question Bank: Having a standardized collection of test questions, organized by school, department, and class, and continually updating these questions is very important.

For School Administrators:

Enhance Teacher Ethics and Accountability: Teachers’ ethics and level of accountability have a direct impact on the quality of teaching in a school. This is why administrators should pay attention to modern leadership skills, quality education service, teachers’ ethics, and work accountability.

Create A Favorable Learning Environment: Having a favorable environment is an effective way of improving students’ academic performance. This is why you should focus your resources on creating a positive learning environment.

Consistent and Regular Teacher Development: A well developed teacher is the main tool of improving the education system. This is why it is important to have a teacher development center at each school, and to consistently send teachers to training seminars.

Pay Attention to Teachers’ Social Issues: remember a salary system that rewards positive performance results in a higher retention rate of qualified, effective teachers. An industry program at the school can be used to supplement your budget, giving you extra funds to reward the teachers with money and benefits.

Cooperate With Parents and Stakeholders: Working with, and listening to, parents and stakeholders can greatly improve the quality of education at your school.

Encourage Students’ Socialization, Interests, and Talents: By organizing sport and art competitions, professional training, and extra activities at school, you encourage students’ socialization and talents.

Page 128: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 128

Reach Out To Each of Your Students: Having a real relationship with your students can help you to have better success as an administrator, and to have a successful school. Encourage individual learning in students will increase the effectiveness of your school.

For Parents:

Have Consistent Contact Between School and Parents: The level of your child’s knowledge, ability, and development depends not only on your family, but also on the school, classmates, and teachers. So it is important to be in contact with the schools.

Supply Educational Materials To your Child: Study results showed that students who have their own books, and have a home library, have a higher level of academic success. It is also important to provide your child with other educational materials, and a uniform.

Create A Warm Environment In Your Home: Having a warm, calm environment in the home, has a very strong positive influence on your child’s success.

Let Your Child Take Part In Extracurricular Activities, Depending On His or Her Interests: Students that attend extracurricular activities have higher than average levels of success.

Have A Home Library: One way to improve your child’s academic performance is to have a home library with useful, quality books. And don’t forget to keep it up to date with new books that your kids like to read.

Influence of Friends: How, and with whom, your child spends his or her free time can have a strong impact on his or her academic performance.

Ensure That Children Understand The Importance of Learning: During the research process, many teachers expressed a desire for parents to emphasize the importance of learning, to teach them to know how important information is.

Talk To Your Children Openly And Frankly On A Regular Basis: One weak side of Mongolian parenting style is being unable to speak openly with your children. If you speak openly and honestly with your children when they are young, they will be open and honest with you when they are in high school. Speaking openly is an important ability. Research results showed that most students do not have open speaking skills. Parents speaking openly with their children are important for their development and academic success.

Preschool Education: The study showed that students who went to preschool before entering school had higher levels of academic success.

Page 129: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 129

Page 130: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 130

CHAPTER 6: ANNEXES

6.1 RESEARCH TEAM OF MEMBERS

Project team leader: Ts. Oyunsaikhan /Director of EEC/ International consultants:

Kelvin D. Gregory /Lecturer of Flinders University, Australia, Doctor/ Cedrick Croft / New Zealand, consultant/

National consultants: Ts. Unorbayan /Director of SMS of MNUE, prof./

A. Monkhor /Head of Department of Physics and Technology of the MUE, professor/ Director of the NITP (National Information Technology Park)

Project Organization, Research, Preparation of National Assessment Specialist Seminar Organization, and Report Team

Ts. Oyunsaikhan /Director of EEC/ Ts. Erdenetsetseg /Senior Specialist, EEC/ М. Boldsaikhan / Specialist, EEC/ D. Binderya / Specialist, EEC/ J. Bileg-Orshikh / Specialist, EEC/ G. Bilegsaikhan /Specialist, EEC/ М. Gantuya / Specialist, EEC/ J. Аltantsetseg / Specialist, EEC/ Ts. Rentsendorj / Specialist, EEC/ J. Ganzorig / Specialist, EEC/ B. Nyamsuren / Specialist, EEC/ I. Enkhnasan / Specialist, EEC/ B. Batzaya / Specialist, EEC/ E. Tuvshinzaya / Specialist, EEC/ L. Ganbat / Specialist, EEC/ D. Оyunsuren /Programmer/

Test Creation Team

D. Erdene /Khan-Uul district, school №32, manager of study/ D. Аltangerel /BZ district, high-school Shine-Mongol, teacher/ S. Bat-Erdene /SB district, school №105, manager of study/ О. Munkh-Chandmani /BZ district, school №111, teacher/ М. Boldsaikhan /Specialist, EEC/ O. Мyagmarsuren /MNUE,teacher/ D. Ganbold /BZ district, school №84, teacher/ O. Tsendsuren /ECAC,methodologist/ D. Аltantsetseg /MECS, specialist / S. Мunkhjargal /MNUE, teacher/ S. Batmunkh /SIA Chingeltei district, inspector/ D. Khishgee /NUM, School of Economics, lecture/

B. Myagmarsuren / NUM, School of Economics, lecture/

А.Batjargal

Page 131: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 131

Preparation of National Assessment Specialist Seminar Attendants, and Heads of the Province Education System

N. Enkhgerel /Arkhangai aimag/ B. Urantsetseg /Bulgan aimag/ B. Saulesh /Bayan-Ulgii aimag/ N. Dolgorsuren /Bayankhongor aimag/ N. Аriuntugs /Dornogovi aimag/ N. Bat-Erdene /Khovd aimag/ N. Sumyagarav /Sukhbaatar aimag/ N. Tserenjav /Govi-Altai aimag/ N. Buyannemekh /Umnugovi aimag/ N. Тumenjargal /Khentii aimag/ Б. Davaamaa /Uvurkhangai aimag/ Ц. Serjmee /Khovsgol aimag/ N. Lkhamragchaa /Zavkhan aimag/ N. Мyagmarsuren /Selenge aimag/ N. Pagva /Uvs aimag/ N. Battsengel /Central aimag/ N. Bumtuya /Dornod aimag/ N. Dashnyam /Оrkhon aimag/ N.Аmgalan /Darkhan-Uul aimag/ N. Enkh-Аmgalan /Govi-Sumber aimag/

Preparation of National Assessment Specialist Seminar Attendants/Researchers (Training Managers)

N. Buyankhuu /Secondary school № I, Arkhangai aimag/ I. Buzmaa /Secondary school, Khotont soum, Arkhangai aimag/ Ch. Erdenechimeg /Secondary school, Bulgan soum, Bulgan aimag/ P. Îyunchimeg /Secondary school, Khutag-Undur soum, Bulgan aimag/ J. Êulaikhan /Primary school, Ulgii soum, Bayan-Ulgii aimag/ H. Riza /High school № 3, Bayannuur soum, Bayan-Ulgii aimag/ D. Naranbileg /Nomuundalai complex school, Bayankhongor aimag/ D. Uranchimeg /Secondary school, Buutsagaan soum, Bayankhongor aimag/ Ts. Îyunchimeg /Secondary school №I, Sainshand, Dornogovi aimag/ S. Àmartsengel /Primary school, Zamin-Uud/ Ya. Nasanbat /Secondary school Progress, Jargalant soum, Khovd aimag/ M. Îtgonbayar /Secondary school, Bulgan soum, Khovd aimag/ Ì. Tuul / Òemuulel complex school, Sukhbaatar aimag/ B.Purevsuren /Secondary school №I, Munkhhaan soum, Sukhbaatar aimag/ Ch.Tsagaantsooj /Secondary school №I, Govi-Altai aimag/ S.Hurtsgerel /Secondary school , Beger soum, Govi-Altai aimag/ Î.Àltantsetseg /Complex school Erdmiin Dalai, Saintsagaan soum, Dundgovi aimag/ T.Ishgen /Secondary school №2, Saintsagaan soum, Dundgovi aimag/ Ts.Îyunsuren /Secondary school №I, Gurvantes soum, Umnugovi aimag/ L.Unur /Secondary school, Bayandalai, Umnugovi aimag/ J.Sukhbaatar /Secondary school №I, Herlen soum, Hentii aimag/ Ch.Hishigtogtokh /Secondary school №I, Umnudelger soum, Hentii aimag/ G.Dulmaa /Secondary school, Sant soum, Ovorhangai aimag/

Page 132: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 132

Sh.Erdenechimeg /Secondary school №I, Uvurhangai aimag/ B.Gumed /Complex school, Delgerhangai soum, Huvsgul aimag/ J.Àltangerel /Secondary school №3, Murun, Huvsgul aimag/ G.Tseendorj /Secondary school, Erdenekhairkhan soum, Zavkhan aimag/ N.Munkhtuya /Complex school Chandmani-Erdene, Erdenekhairkhan soum, Zavkhan aimag/ Sh.Chimedtsogzol /Secondary school №2, Mandal, Selenge aimag/ B.Narangerel /Secondary school №4, Sukhbaatar, Selenge aimag/ G.Bayarmaa /Secondary school, Selenge aimag/ S.Jonomoo /Secondary school №1, Ulaangom, Uvs aimag/ Ch.Tseveen /Complex school Shine-Ireedui, Uvs aimag/ B.Batchimeg /Secondary school, Bayanchandmani soum, Central aimag/ N.Îyunkhuu /Complex school Humuun, Central aimag/ D.Dorjnamjim /Complex school Shine hugjil, Dornod aimag/ Ì.Haraa /Secondary school №12, Dornod aimag/ B.Bolormaa /Complex school Erdenet, Orkhon aimag/ B.Bum-Ayush /Complex school Bayan-Undur, Orkhon aimag/ P.Baigalmaa /Complex school Oyuni-Ireedui, Darkhan-Uul aimag/ D.Byambaa /Secondary school №14, Khongor soum, Darkhan-Uul aimag/ L.Narantuya /Secondary school №1, Govi-sumber aimag/ D.Tserenchimed /Complex school Bolovsrol, Baganuur district/ D.Gankhuyag /Complex school Bolovsrol, Baganuur district/ L.Tsolmon /High school №3, Shiveegovi soum, Govi-sumber aimag/ B.Narantsetseg /Secondary school №1, Selenge aimag/ D.Erdene /Secondary school №32, Khan-Uul district/ S.Bat-Erdene /Secondary school №105/ Ê.Dolgorsuren /Secondary school №39, Chingeltei district/ Ts.Tsolmonbaatar /Secondary school №23, Chingeltei district/ J.Ìunkhzul /Secondary school №114, Khan-Uul district/ B.Chantsaldulam /Secondary school №111, Bayan-Zurkh district/ Ts.Tsastsetseg /Secondary school №50, Chingeltei district/ B.Nyamsuren /Complex school Shavi, Bayan-Zurkh district/ Ch.Îyuntulkhuur /Complex school Buyant-Ukhaa, Khan-Uul district/ D.Baasansuren /Secondary school №38/ D.Tuvshinbayar /Complex school Erdmiin-Undraa/ S.Тuul /Secondary school №9/ Ts.Adilkhishig /Secondary school №35/ G.Оyun-Erdene /Secondary school №74/ Ts.Оdsuren /Secondary school №18/ Z.Dolgorsuren /Secondary school №16/ H.Batkhuu /Secondary school №2/ D.Baigalmaa /Secondary school №42/ G.Narantuya /Secondary school №4, Sukhbaatar district/ D.Аltangerel /High school Shine-Mongol, teacher/ D.Ganbold /Secondary school №84, Bayan-Zurkh district, teacher/ Î.Ìunkhchandmani /Secondary school №111, Bayan-Zurkh district, primary teacher/ H.Ìunkhjargal /Teachers Colledge/ U.Ìyagmarsuren /Teachers Colledge/ B.Îyunchimeg /Secondary school №1, Govisumber aimag, primary teacher/

Page 133: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 133

D.Ìunkhzul /Complex school Golomt, Nalaikh district, primary teacher/ Yo.Bayarmaa /Complex school Golomt, Nalaikh district, primary teacher/

Preparation of National Assessment Specialist Seminar Attendants from the Capital, Training Managers

B.Enkhtuya /Secondary school №88/ R.Аmgalan /Secondary school №53/ B.Sodnombaljir /Secondary school №44/ B.Narantuya /Complex school Erdmiin Urguu/ О.Tsetsgee /Secondary school №47/ G.Chandmaa /Secondary school №54/ J.Hishigt /Secondary school №58/ B.Gerelsaikhan /Secondary school №24/ G.Tsolmon /Secondary school №61/ B.Dolgormaa /Secondary school №105/ B.Аltantsetseg /Complex school Ireedui/ Ts.Enkhtuya /Secondary school №75/ Т.Мyagmartseren /Complex school Tsetsee gun/

Took Part in the Study

B.Аsemgul /Secondary school №1, Sukhbaatar district, teacher/ B.Zulmira /Secondary school №20, Bayangol district, teacher/ О.Badraa /Huvsgul aimag, teacher/ D.Gereltsetseg /Govisumber aimag, teacher/ Ts.Buyangerel /Secondary school №39, teacher/ Ts.Dondogmaa /Govisumber aimag, teacher/ Sh.Munkhtsetseg /Hantaishir Institute, lecture/ D.Baigalmaa /Secondary school №106, teacher/ Ch.Enkhtuya /Secondary school №1, teacher/ J.Baigali /Secondary school №17, teacher/ Ts.Оdsuren /Secondary school №18, teacher/

Page 134: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 134

6.2 GLOSSARY

RASH model - an analysis model which examines distribution of item difficulty levels with the distribution of performance of students by referring them onto same scaled scoring;

Blueprint – detailed information about each item;

SOLO taxonomy – theory by Biggs, Collos (1982), the taxonomy’s main purpose is how to define a quality and results developing of students ability and performance;

StudentScoreEntry – information about student scores;

StudentInfo- data registering and entering tool of student information;

ConQuest –researching program for items. (Wu, Adams & Wilson, 1998)

1L, 2L, 3L, 4L, 5L – levels of SOLO Taxonomy;

AD, BD, CD, DD, CDD – abbreviations to indicate domains in mathematics and reading;

Claster –items version of part;

Page 135: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 135

6.3 REFERENCE

1. Ministry of Education, Science and Culture. 2004. Recommendation of Standard of Primary and Secondary Eduvation. Ulaanbaatar

2. Ministry of Education, Science and Culture. 2004. Recommendation of Standard of Primary Education. Ulaanbaatar

3. Ministry of Education, Science and Culture. 2004. Standard of Primary Education Of Mathematics. Ulaanbaatar

4. Ministry of Education, Science and Culture. 2004. Standard of Primary Education Of Mongolian Language. Ulaanbaatar

5. TIMSS-1999, TIMSS-2003 Technical Report. 2000, 2004 6. Boston College. 2006. PIRLS 2006 International Report. USA 7. Ministry of Education. 2000. Monitoring Of Learning Achievement of 4th and 8th Grade. Ulaanbaatar 8. Ministry of Education, Science and Culture. 2005. Assessment Of Primary Education Learning

Achievement-2005.. Ulaanbaatar 9. Ministry of Education, Science and Culture. 2007. Assessment Of Secondary Education Learning

Achievement-2007.. Ulaanbaatar 10. Education Evaluation Center.2006. National Assessment Of Students Achievement. Ulaanbaatar 11. Save The Children of Great Britian. 2005. The Education Of Kazakh Children - Brief Report of Survey,

Analysis of Children Educational Condition

Page 136: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 136

6.4 WEB

1. www.mecs.pmis.gov.mn

2. www.iea.nl

3. www.eec.mn

4. http://www.savethechildren.mn

5. http://www.mongoleducation.mn

6. http://www.mongolianlaws.mn

7. http://www.forum.mn

8. http://www.unen.mn

9. www. mea.org.mn

10. www.timss.edu

11. www.worldbank.org

Page 137: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 137

6.5 PHOTOS

The minister of Education culture and science O.Enhktuvshin, chairman of authority of professional education R.Bat-Erdene and the director of EEC Ts. Oyunsaikhan have signed the contract of the project. (The

representatives of the ministry of education, culture and science, World Bank and EEC. Ulaanbaatar. 23-nov-2006)

In range of the project a four stroke seminar that provides the specialists of the national assessment was organized. During the elementary seminar chairman the authority of supervisory, research and assessment of Ministry of

Education, Culture and Science M.Baasanjav, international advisor prof. Kelvin Gregory, The director of EEC Ts. Oyunsaikhan, Specialist of authority of primary and middle education D.Altantsetseg and the Translators

(Complex Nuht 20-22-Feb2008)

A Four stroke seminar that provides the specialists of the national assessment was organized. (The participants of elementary seminar. Feb.2008)

Item writing team members are shown with international and national consultants.

Page 138: Bizdata.mn Report Eec

EDUCATION EVALUATION CENTER

MONGOLIAN NATIONAL ASSESSMENT OF MATHEMATICS AND READING 138

The methodologist of the primary education in the authority of Education and Culture of Aimags, who were the leaders of the survey team in their aimags, national advisor of the project, Specialists of EEC, team members of the

project. ( Darkhan- Uul aimag , Center Zulzagyn Gol, 9-10- Feb- 2008)

As representative 4750 students of 166 schools have been taken part in the survey. (During the study, 22-april-2008

The survey is conducted in the range of READ project and involved many schools from countryside than center schools. Some students are shown with teachers and director. (Mogoi bag, Huvsgul aimag )

After the survey has taken, Kin Bing specialist at WB, Pratik Tandon economist at WB, P.Bat-erdene national director of the READ project, D.Khishigbuyan coordinator of the READ project are making brief meeting.

About 100 teachers were involved to the elementary seminar that provides the specialists of the national assessment. During teamwork (Complex Nuht 22-Feb-2008)

About 100 teachers were involved to the elementary seminar that provides the specialists of the national assessment. During teamwork (3rd seminar. 1-2-oct-2008)