7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
1/51
Moderation of the FSA
Best Practice
Linda Meyer
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
2/51
INTEGRATED FSA
The FSA for the qualification willfocus on the extent to which alearner can demonstrate appliedcompetence .Applied competence, in terms ofthe NQF is evidenced throughlearners ability to integrate
concepts, ideas and actions inauthentic, real-life contexts and isexpressed as practical,foundational and reflexivecompetence
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
3/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
4/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
5/51
ASSESSMENT METHODS
In setting the FSA, SETQAA would likeCertification/ Outsourced Partnersand/or assessors to take note of thedifferent assessment methods andencourage the use of a wide variety ofmethods.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
6/51
SETTING OF FINAL INTEGRATEDSUMMATIVE ASSESSMENT (FSA)
Function and role of theCertification Partner/OutsourcedPartners
The Certification/OutsourcedPartner is responsible for setting ofthe FSA at NQF Level 4, with amemorandum of model answersand a grid showing its linkageswith the assessment criteria of the
qualification.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
7/51
FSA
All FSA tools and memoranda, evenin draft form, are the property of theServices SETA ETQA and may not berevealed to, or discussed with anyunauthorized person. The highlyconfidential nature of these
assessment instruments must besafeguarded at all times.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
8/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
9/51
A second assessment instrumentshould be set as standby assessmentinstrument. SETQAA have the right topropose the use of either assessmentinstruments for the FSA, if necessary.
FSA
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
10/51
The Certification/OutsourcedPartner must ensure that theassessment instruments are not
identical.The Certification/OutsourcedPartner must also ensure thatneither of these assessmentinstruments are the same as anyprevious assessment instruments.
FSA
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
11/51
MODERATION CRITERIA
The moderator must ensure that theassessment instrument is aligned withthe purpose and rationale of thequalification and that it complies withthe applicable SAQA level descriptor.Draft assessment instruments must fallwithin the ambit and in accordancewith the guideline document.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
12/51
Functions and role of themoderator
After the Certification/OutsourcedPartner has set the FSA instrument,with a memorandum of model
answers, an internal moderator,appointed by theCertification/Outsourced Partner,must moderate all draft assessmentinstruments and memoranda for theassessment instruments under his/her
jurisdiction.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
13/51
Functions and role of themoderator
During the moderation, theassessment instrument andmemoranda are to be objectivelyassessed by the moderator with
regard to the following:
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
14/51
Functions and role of themoderator cont.
The moderator must verify thatpapers set for the FSA`s are not
identical.The moderator must verify that theUnit Standards are appropriate to the
registered qualification.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
15/51
Functions and role of themoderator
The moderator must ensure that thefinal summative integrated
assessment instrument is alignedwith the purpose and rationale of thequalification and that it complies withthe applicable SAQA level descriptor.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
16/51
Functions and role of themoderator
The moderator must ensure that the finalintegrated assessment instruments mustfall within the ambit and in accordancewith this guideline document.The extent to which the prescribed UnitStandards are covered as well as theassurance that nothing is asked that isoutside the scope of the UnitStandard/Qualification must be verified.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
17/51
Functions and role of themoderator
The spread and number of marksallocated according to the weighting
and credits as per the registeredlearnership/qualification must beassessed.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
18/51
Functions and role of themoderator
The effectiveness of the type ofassessment methods used, written
questions, essays, paragraphs, casestudies, must be assessed.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
19/51
Functions and role of themoderator
The questions, terminology,numbering, spelling, punctuation,sentence construction, units, andsymbols must be clear,unambiguous and grammatically
correct so that candidates will knowexactly what is expected of them.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
20/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
21/51
Functions and role of themoderator
It should be borne in mind that ittakes considerable time to readthrough certain assessmentinstruments.The memorandum must ensure thatmodel answers give enough detailfor the assessor of the FSA to conductan accurate and fair assessment.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
22/51
Criteria for moderation of the FSAinstrument
Standard of the paper (validity,reliability and fairness) Questions of various types e.g. MultipleChoice Questions, Case studies,paragraphs, data response, essay, etcQuestions from which candidates are tochoose are they of equal difficultylevelCorrect distribution in terms of cognitivelevels (Blooms Taxonomy)
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
23/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
24/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
25/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
26/51
Criteria for moderation of the FSAinstrument
Overall: how does the standard of the assessment instrument comparein relation to other qualifications/sassessment instruments andprevious assessment instruments.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
27/51
Criteria for moderation of the FSAinstrument
Intellectually challenging andallowing for creative responses fromcandidates
Suitability of examples andillustrations
Relationship between mark allocation, level of difficulty andtime allocation
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
28/51
Criteria for moderation of the FSAinstrument
FSA do not count more thancontinuous assessment of providers the objective for the FSA is to check the standard across providers andacross qualifications -STANDARDISATION
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
29/51
Technical Criteria
Cover page with all relevant detailssuch as time, unit standards andinstructions to candidates
Clarity of instructions to candidates Lay out: learner friendly Correct numbering
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
30/51
Technical Criteria
Mark/weighting allocation clearlyindicated
Quality of illustrations, graphs, tablesetc must be print ready
Complete memorandum with modelanswers with mark/weight allocationand provision for alternatives
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
31/51
Registered Unit Standards andQualifications
Relevance to registeredQualification, i.e. unit standards,specific outcomes, assessmentcriteria
Levels of questions Coverage of unit standards Weighting and spread of contents
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
32/51
History of tool
Full history of FSA (when it will beconducted, who will be assessed,when and where) with all drafts,internal moderators comments etc.must accompany the assessmentinstrument each time it is submitted
to the external moderator (SETQAA)
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
33/51
Cognitive skills
What conceptual constructs of the unitstandards does the assessmentinstrument deal with?
e.g. reasoning ability ability to communicate ability to translate from verbal to symbolic ability to compare and contrast ability to see causal relationship ability to express an argument clearly
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
34/51
Cognitive skills
Are these constructs representative of thebest and latest developments in thetraining of this knowledge field?Are the questions challenging and allowingfor creative responses from candidates?Suggested application of cognitive levelsfor an NQF Level 4 qualification is: 10% knowledge
20% comprehension 40% application 30% analysis, synthesis, evaluation
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
35/51
Language and bias
Correct terminology Appropriate language register for the
level of the learner Avoidance of gender, race, cultural,
provincial bias Clear and unambiguous specification of
instructions within questions e.g. list,describe
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
36/51
Time
Relationship between difficulty level of questions and time
Relationship between time and mark
allocation and answer requirements
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
37/51
Internal Moderation
Is there evidence that the paperhas been internally moderated?Quality, standard and relevance of input from internal moderator.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
38/51
Competency of Assessor
Knowledge of assessor setting the FSAwith regard to qualification.
Knowledge, experience, expertise in
assessment
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
39/51
Marking Memorandum
Correctness of memorandumCorrespondence with questions.Alternative answers provided
Does it facilitate assessment?
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
40/51
Alignment Grid
Did they use the grid showing thealignment of the instrument with thepurpose, rational, assessmentcriteria and NQF level descriptor of the qualification as issued by theSSETA used?
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
41/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
42/51
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
43/51
Only after written approval from theexternal moderator, SETQAA willapprove and sign off the finalassessment instrument and notify theCertification/Outsourced Partner toproceed with the distribution of the
assessment instrument and conductthe assessment.
CONTINUOUS ASSESSMENT VS FINAL SUMMATIVE ASSESSMENT
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
44/51
STAKEHOLDERS INVOLVED INQA OF ASSESSMENT
PROCESSES Provider internal assessors
Provider internal moderators Certification/Outsourced Partnerassessors Certification/Outsourced Partnermoderators SETQAA external moderators SETQAA external verifiers
TRAINING PROVIDER ASSESSMENT Assessment should happen on a CONTINUOUSbasis during learnership/training formative aswell as summative Evidence gathered in PoE Internally moderated and reports submitted toSETQAA at regular specified intervals (Seemoderation policy) Externally moderated/verified by SETQAA PoE submitted for external moderation by CP atend of learning
FSA (PRACTICALOBSERVATION)
Should be set and conductedCONTINUOUSLY and/or AT END oflearnership/ qualification Set and assessed by registeredassessors, moderated by registeredmoderators appointed by CP Externally verified by SETQAA
FSA (WRITTEN COMPONENT ) Conducted AT END oflearnership/Qualification Set and moderated by registeredassessors and moderators appointedby CP Assessed and moderated by registeredassessors and moderators appointedby CP Externally moderated/verified bySETQAA appointed moderators
PRACTICAL COMPONENT THEORETICALCOMPONENT
CONTINUOUS ASSESSMENT VS FINAL SUMMATIVE ASSESSMENT
SETQAA DOC
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
45/51
FSA Moderation process
APPLICATION FOR MODERATION OF FINALINTEGRATED SUMMATIVE ASSESSMENTINSTRUMENT RECEIVED BY SSETQAEXTERNAL MODERATOR APPOINTED BYSSETQANO COMMUNICATION PERMITTED BEWEENDEVELOPER AND MODERATOR
MODERATION OF FINAL INTEGRATEDSUMMATIVE ASSESSMENT INSTRUMENT / RPLTOOL
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
46/51
FSA Moderation process
PRELIMINARY REPORT PREPARED BYMODERATOR AFTER THE FSA/RPL TOOLHAS BEEN MODERATED - REMEDIATIONIDENTIFIED
TOOL SENT BACK TO DEVELOPER FORREMEDIATION TO BE ADDRESSEDTHE DEVELOPER MAY REMEDIATE /REQUEST ADDITIONAL INFORMATION OR APPEAL THE MODERATION DECISIONTHROUGH THE SSETA APPEALSPROCEDURE
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
47/51
FSA Moderation process
REMEDIATION COMPLETEDSENT TO EXTERAL MODERATOR TOCONFIRM REMEDIATION & FINALREPORT SUBMITTEDELECTRONIC VERSIONS SENTFSA/RPL TOOL REGISTERED ONDATABASE
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
48/51
The Certification/OutsourcedPartner assures the rectification isdone and sends the final drafttogether with the Moderators reportto SETQAA for external moderation.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
49/51
The Certification/OutsourcedPartner assures the rectification isdone and sends the final drafttogether with the Moderators reportto SETQAA for external moderation.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
50/51
Only after written approval fromthe external moderator, SETQAAwill approve and sign off the finalassessment instrument and notifythe Certification Partner to proceedwith the distribution of the
assessment instrument andconduct the assessment.
7/31/2019 Linda Meyer - Moderation of the FSA - Best Practice
51/51
Thank you