DECISION-MAKING IN EDUCATION BASED ON MULTI-CRITERIA RANKING OF ALTERNATIVES THE EFFECT OF SOME UNCERTAINTY TYPES ON COMPETITIVE SCORES OF HEI APPLICANTS Bakhrushin V.E., Ignakhina M.A. Classic Private University, Zaporizhia [email protected]
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DECISION-MAKING IN EDUCATION BASED ON MULTI-CRITERIA RANKING OF
ALTERNATIVES
THE EFFECT OF SOME UNCERTAINTY TYPES ON COMPETITIVE SCORES OF HEI
APPLICANTS
Bakhrushin V.E., Ignakhina M.A.Classic Private University, Zaporizhia
Methods of multi-criteria ranking
Methods of multi-criteria ranking of alternatives are widely used in decision making in various areas. They are based on the traditional approaches of multi-criteria decision-making. In particular there are the methods of criteria convolution, the main criterion, maximin and maximax assessment of alternatives and others.
But in the case of multi-criteria ranking it is need to choose not one, but several alternatives that are the best from a certain point of view.
Some examples of linear convolution
Assessments based on the linear convolution of criteria are the most popular. They are built as weighted sums of partial indicators.
Well-known examples of such estimates are the university rankings, competitive scores of Ukrainian HEI applicants and so on.
It may be assumed that in the nearest future similar assessment methods will be used for the formation of state order for specialists training at HEI, allocation of the scholarships to students, grantmaking for scientific research etc.
Some problems
One of the main problems of multi-criteria ranking is to ensure validity, reliability and accuracy of methods used.
These characteristics depend greatly on the choice of particular criteria, accuracy of measurements, weighting coefficients, random fluctuations and other factors.
To reduce the risk of errors some general principles (for example, the Berlin Principles on Ranking of HEI) and standardized methods are used.
Some examples of multi-criteria ranking
Some examples of multi-criteria ranking
National Universities Rankings (US News &World Report)
National Universities Rankings (US News &World Report)
National Universities Rankings (US News &World Report)
Forbes RankingIndex % of total
Student Satisfaction Student evaluations from RateMyProfessorStudents transfer
25
Post-Graduate Success Salary of alumni; Forbes listsNobel and Pulitzer winners, Guggenheim and MacArthur Fellows, National Academy of Sciences members; winners of an Academy, Emmy, Tony or Grammy
32,5
Student Debts Student' loan debt 25
Graduation Rate Students actually finished their degrees in four years
7,5
Academic Success Prestigious scholarships and fellowshipsGoing on studying for Ph.D.
10
Positions of some HEI in ratings QS ARWU THE Forbs
Amherst College − − − 10United States Military Academy − − − 9Harvard University 2 1 2 7Yale University 4 9 8 6California Institute of Technology 6 5 1 21University of California, Berkeley 14 3 6 37University of California, Los Angeles 17 10 9 44Columbia University 9 7 10 20Massachusetts Institute of Technology 1 4 4 5Pomona College − − − 8Princeton University 7 6 5 4Stanford University 3 2 3 2Swarthmore College − − − 3Williams College − − − 1University of Chicago 5 8 7 24
Competitive score
Competitive scores of the Ukrainian HEI applicants are calculated on the basis of external independent evaluation (EIE) scores. These scores are actually the quantiles of the normalized distribution of raw EIE test scores. So they essentially depend on random variations in the complexity of test items and the tested applicants’ level of training. In this regard, it is necessary to analyze the possibility of using the test results of different years at entrance campaign.
Algorithm
initial data (raw test scores) –generated or real;determination of success thresholds;excluding the results, that are lower than the
threshold, from the obtained vectors;calculation of corresponding values of
empirical distribution function as the ratio of its rank to the vector length;
calculating of outcome scores as the values of the inverse standard normal distribution function;
transforming of results into 100 – 200 scale.
Competitive score
Uncertainty factors
unknown complexity of the test, which changes from year to year;
unknown level of training of graduates, which also varies from year to year;
unknown not normal distribution of test results;
unknown relation between the total score and objective assessment in traditional scale;
…
Mathematics / Physics
Year2007 2008 2009 2010 2010 2011 2012 2012 2013 2013 2014
Mean 28,9 20,1 25,7 36,2 34,7 30,7 37,9 40,1 33,6 29,5 28,6
σ 20,9 14,4 20,5 21,2 21,9 19,3 20,7 20,3 18,6 19,4 20,5
Kα 0,92 0,80 0,90 0,90 0,92 0,89 0,88 0,89 0,88 0,89 0,91
Year2007 2008 2009 2010 2011 2012 2013 2014
Mean 23,5 35,9 26,5 32,8 29,8 32,0 31,2 27,5σ 16,1 17,2 14,6 18,7 13,4 16,0 15,6 16,4Kα 0,82 0,81 0,80 0,86 0,80 0,83 0,81 0,86
Mathematics (2011 – 2014)
English (2011 – 2014)
Model: 1 or 2 thresholds
General results
Obtained results show that considered uncertainty factors may lead to variations of outcome EIE scores in the range of 2 – 10 points. Ceteris paribus their effect is more significant in the case of asymmetric and/or non-uniform distribution of raw scores. The largest differences are observed at the edges of the distribution – for the worst and best applicants.
When evaluating possible differences between the scores of 2015 and previous years, we can expect an additional variation due to the systematic but not controlled changes in samples of applicants. They are associated with a change in the entrance conditions in 2015.
Related Documents
