Alternative System

Alternative System for Admission into Engineering ProgrammesReport of the CommitteePresented to the Council of Indian Institutes of TechnologyA proposal for consideration

AcknowledgementProf Acharya and his Committee membersDr BK Gairola; Sri V Joshi; Sri H Bhartia; Sri M TuliDirectors of all IITs2063 people who participated in opinion pollDirector and Experts from ISI and their studentsColleagues from MHRD, DST and NICChairpersons of school boards andChairman and members of the IIT CouncilSome Media representatives and all those who helped in the exercise

Decision of the Council in its 41st meetingProf Acharya committee had been commissioned to study the present system of conducting examinations for admission into engineering progammes of the country and suggest alternativesThe committee presented its interim report in the 41st meetingThere was unanimity that the present system required a change as proposedThe committee was enlarged to address the issue of the need to recognize diversity of learning

Major findings of Acharya CommitteeScreening based on normalized Board scores at Standard X and/or Standard XII and Multiple Choice examination replacing the two stage JEE from 2006. Entry barrier to be raised to 60% in the +2 examinations. Factors, other than the Standard XII marks and AIR based on PCM testing, such as raw intelligence, logical reasoning, aptitude, comprehension and general knowledge need to be considered. Need to factor in school performance more significantly into the selection process.

Major findings of Acharya CommitteeDecision based on one time test needs to be re-examined. Opportunities to improve must be built in.Students must be relieved of the pressure of multiple JEEs. Influence of coaching for JEE needs to be minimized.Urban-rural and gender bias has to be eliminated or at least minimized.The objective type of examination lends itself to undue influence of coaching. The conventional pen and paper examination with well designed long and problem solving oriented questions should be revived by keeping numbers in any JEE within reasonable limits.

Work of the present committeeStudy of Acharya committee workMade Non formal survey among hundreds of school students, parents, employers, faculty and media personMet and decided on General approachesHeld with Chairmen and nominees of all school boardsCollected data on scores of school board examinationsWrote down a philosophy document and placed in public domainCarried out survey of public opinion pollEnrolled ISI for carrying out exploring statistical methods for normalization of school board scores Met with Directors and Senior Faculties of IITsPrepared a draft report for comments and advice of the Council

Consultation with states and School boardsFirst stageGained access to data on scores on school board examinations through formal mechanismsEnrolled participation in development of NTSSecond StageAssistance in harmonization and electronic access to dataBuilding trust and development of a process with safeguards and integrity

Public participation through opinion poll On-line opinion survey among the people of IndiaFor multi parametric grading system as against single test models of JEEScreening out as against selection strategiesWith Responder profile, opinion polls, suggestions for alternative national test systems; risk mitigation strategiesSurvey time slot open for three weeks

Responder profile to the poll 2063 participated59% students; 8% teachers, 5.5% parents, 23.4% non-teaching professionals, less than 1% coaching~80% of respondents had taken entrance examinations in their lives1220 students had participated80% students were from engineering stream and 95% of them had taken entrance examination~160 teachers had participated~90% of them are engaged in tertiary education

Analysis of the current system of admission into engineering960 of 2063 commented on the multiplicity of entrance examinations with different sociological implications715 agreed with the view expressed947 of 2063 commented on need for reform85% voted for reform in admission system

Inputs for reforms fo admission systemsReduction in the number of examinations to oneTesting knowledge intensity, alignment to the 12th class syllabusReduction of dependency on coaching and pressure on studentsEmphasis on aptitude.. (More than 90%)Transparency in processesRemoval of negative markingOnline processesMultiple centers, better scheduling

Responses to reform directionsFactor-in performance in school boards66% in favor and 34% not in favor45 of those disfavor fear that board examinations do not assessing capability and 30% fear non-uniformity620 responded to question on Indian equivalent of SAT type73% voted in favor of aptitude type test646 responded to types of tests70% prefer a mix of aptitude and advanced type tests629 responded questions on test featuresDominant support is for high-filter type and SAT type tests

On summaryMore than 85% supported the concept of a single entrance test for admission into engineering programmes and voted for reforms70 % voted in favor of one test with provisions for testing both aptitude and advanced knowledge66% of people favor factoring in school board scoresOf 34% People who disfavor fear primarily the problem of non-uniformity. This could be addressed.

Some Important suggestions for the committee from opinion pollNational Test in place of multiple competitive examination is generally welcomedConcerns expressed about the process integrity and fairness of testing methodologyNormalization methodologies across school boardsMultiple chances for candidates for improving scoresA single screening examination with a mix of aptitude (like SAT type) and advanced (like JEE type)

Study for normalization of scores of school board examinationsCommittee gained access to some relevant past data with a view to examineStability of scores of the same school board over timePotentials for normalization of scores across various boardsEnrolled ISI into carrying out statistical studies for normalization of board scores

Work carried out Indian Statistical InstituteFor exploring normalization methodologies for school board scores

Pilot testingSelected four boards for pilot testingCentral Board, TN Board, WB Board, ISEEvaluation years 3-4 years for each boardVariations in Density of populationCBSE (5-6lakhs), TN(5.6-7.3 lakhs), WB (3.0 -4.6 lakhs, ICSE 25000-56000Evaluation of stability of scores over time for the same boardPotentials for mapping the profiles of several boards onto one selected board through monotone transformations

Models testedSelect a percentile score (P) for all boards and determine the scores (X1) for P across boardsMappingModel 1: Y1= {Xn X1}/{Xm-X1}Model 2: Y2 = Xn/X1where Xn , X1 , Xm are scores obtained by any candidate, marks corresponding to percentile P, and maximum scores obtained by any candidate in each board. Y1 will range from 0 to 1.0; while Y2 will be ratios in the range from 1.0 to Xm/X1

Observed Relationships of ScoresFor the four boards over timeModel 1: Y1= {Xn X1}/{Xm-X1}Where aggregate score percentages are used

Standardized (Normalized) score

Normalized percentile rank

Some general conclusions emanating from the ISI studyPercentile scores are relatively stable over the periods studied for each board when aggregate percentage scores are analyzedIt is possible to normalize percentile ranks across boardsCorrelations of normalized percentile ranks against percentile ranks of various boards map on to common linear relationship

Transformations onto one theoretical boardISI Experts are certain that this transformation will be the same for all boards for all years

It appears that for normalization of school board scoresA statistical method is feasible after all. ISI may be encouraged to develop the methodology further and extend it to all boards and reconstruct past scenario for present IIT and NIT students over the last four years for revalidation of the method

A pilot test among a select group of students: A suggestionA group of statistically significant number of some volunteers from current student population may be enrolled into a pilot test for evaluating the utility of different models and suggestions. This pilot test has to be designed with care and confidentiality based on informed consent of all involved. The merit or otherwise of the approach will be discussed within the committee before decisions are taken

Summary of work done so farOpinion poll reveals support for reforms and favor single examination with provisions for both aptitude and advanced while making provisions for factoring-in scholastic performance in school board examinationsISI study presents a methodology for scientific normalization of scores across school boardsPercentile scores are stable over each board and it is possible to carry out monotone transform board scores and accomplish normalization across boardsWeighting options for school and entrance tests performance are considered

Two approaches consideredApproach 1weighing consistency of performance in school board examinations and employ them for testing ability to write solutions andOne objective screening test with two sections; one for testing the aptitude and the other advanced knowledge in domain areas.Approach 2weighing consistency of performance in school board examinations and employ them for testing ability to write solutions andone objective aptitude test based on multiple choices and computer based correction systems

Considerations of six different optionsOption 1: Deployment of Scores as criteria based on class XII performance onlyEqual weighting of school board scores A1and A2Equal weighting of aptitude scores A4 and advanced scores A5Normalized score= {A1 + A 2+A4 +A5 }/4Option 2: Deployment of Scores as criteria based on class XII performance onlyEqual Weighting of board score A3 which is {A1+ A2}/2Equal Weighting of Aptitude scores A4 and A5Normalized score ={A3 +A 4+A5}/3

Considerations of six different optionsOption 3: Deployment of Scores as criteria based on consistency of performance at class X and Class XII levels as well as in National Level Aptitude and Advanced TestsEqual weighting for aggregate as well as subject performance at class X and Class XII levels where ) 0.1X (normalized score at class X in aggregate + normalized score at class X in subjects of choice + normalized score at class XII + normalized score at class XII in subjects of choice)One third weighting of aptitude score 0.3 A4One third weighting of advanced score A5Normalized score =0.1{ Normalized aggregate class X + normalized class X subject score + Normalized class XII aggregate + Normalized class XII subject score} + 0.3 A3and 0.3 A5Option 4: Deployment of School Board Performance as screening but not as determinant for National ranksSpecify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank50% weighting of National Level Aptitude score A4 for candidates passing the cut off of percentile rank50% weighting of National Level Advanced Score A5 for candidates passing the Normalized score = 0.5 A4 +0.5A5

Considerations of six different optionsOption 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system according freedom for the individual institutions to select mixing proportions within a pre-specified guidelineOption 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system whereNormalized score=0.5 A2+0.5A4

Recommended preferences of the committeeOption 2: Deployment of Scores as criteria based on class XII performanceOption 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system0.5 A2+0.5A4Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system with freedom for the individual institutions to select mixing proportions within a pre-specified guidelineOption 4: Deployment of School Board Performance as screening but not as determinant for National ranks; Specify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank; and rank by 0.5 A4 +0.5A5

A SuggestionA committee of experts from engineering institutions could be assigned the task of interacting with ISI Group for internalization of methodology for normalization of board scoresIITs could be assigned the task of setting up a question paper for National Screening Test based on objective examination models and conduct the examinations for the year 2012-13For Aptitude examination like SAT, we may take the help of NTS or any other global agency

Thank you For the patience and waiting We recommend normalization of school board scores for factoring-in based on ISI inputsSingle National level test to cover aptitude and advanced or aptitude alone andJudicious mix of school and national test performance