Who barged into my testing? Miriam Sester Retorta.

Who barged into my Who barged into my testing?testing?

Who barged into my Who barged into my testing?testing?

Miriam Sester RetortaMiriam Sester Retorta

a

The Scream (Edvard The Scream (Edvard Munch)Munch)

TT

• Think about the mistakes that your students make, or the mistakes you make yourself when using a language that is not your mother tongue. How do you feel about them? Please, write three sentences using the word “Mistake” in them!!!!

Think Globally, Act Locally

No teaching is effective if it is not affective

1. What is evaluation?

2. What are the reasons for anxiety and the derogatory image of tests?

3. Common questions teachers ask:

* Should we evaluate the student with numerical grades, alphabet grades, descriptive reports?

* How can we develop effective evaluation instruments?* How can we establish fair criteria to evaluate someone?* When should we flunk a student?* What criteria should we use to evaluate written work?

Common beliefs and uses:

It is an action of judgment of results achieved, analysis of performance, measurement of capacity, evaluate ‘the whole of the student’.

Selective and discriminatory function of grades and concepts: serious damage due to failure.

Bureaucratic, punitive functions which hinder life projects of many people.

• What it should be:• Evaluation is essential to education. It is

inherent and indissoluble when it is related to questioning, inquiring, reflecting about our students’ and our own actions.

• Problem: educators generally view teaching and evaluating as two distinct moments and not interrelated !!!!

• ‘Mistakes are seen as failure and doubts as lack of knowledge’.

• Jussara Hoffmann, (2001) In an educational dimension students’ mistakes and doubts are considered a highly significant episode which lead to educational actions. In this dimension evaluate is to create the opportunity to reflect about the actions taken in the classroom in a permanent teacher watch, which will instigate students to new questions from previously answered questions”.

• Coherent evaluative practice:• 1) It demands from teachers deep and detailed studies

of theories of knowledge.• 2) It demands a vision, at the same time, broad and

detailed of his/her discipline. • 3) It demands an eternal search of comprehension of

students’ difficulties and the search of new opportunities to develop knowledge.

• 4) We need to know like our students do and find out why they think that way.

Why evaluate

We cannot reduce an instrument of evaluation to results only: to select or classify. We must use it to improve our educational actions. Evaluate is an investigative procedure so we can go way further in the accompanying the process of knowledge construction. Evaluate is an instrument of investigation of BOTH students and teacher. (sometimes even school and curriculum)

DON’Ts: The only notes a teacher writes on the test are: super, great, fair, poor, study harder, pay attention, incomplete answer, etc – Generic recommendations !!!

Exemple: doctor-patient and tests.

It is not enough to diagnose the disease. You must offer adequate treatment !!!!!

Doctor, engineer and teacher – success or failure

Assignments: group work (trabalho em grupo ou equipe), individual paper, compositions– CRITIRION OF CORRECTION.

ERROR ANALYSIS - to diagnose what is the real problem. Must be done WITH the student !!!

•If making mistakes is part of learning, and correction is part of teaching, how do the two of them go together????

Mistakes•The most important sort of mistakes is a mistake that leads to a misunderstanding.

• (examples from: Mistakes and Correction. Julian Edge, Longman)

Mistake 1• Imagine a student, Cheng, who

wants to buy a jacket in Britain. She goes into a shop and says:

• Please will you to show me coats?

• (What is the most important mistake here?)

Mistakes 2• Peter: How long are you here for?• Amira: Two years• Peter: Wha...! You are already here

since two years?• Amira: No, no. I am come yesterday.• Peter: Oh, yester....• Amira: No, no, last week. I mean I came

last week.• (What is the most important mistake here?)

Mistake 3• Nesrin: Good Morning.• Shopkeeper: What can I do for you?• Nesrin: Give me two apples.• (Shopkeeper weighs two apples, puts them in a bag and gives them to Nesrin)

• Shopkeeper:Anything else?• Nesrin: No. How much this?• Shopkeeper: For pence.

• People don’t usually mind if foreigners make grammatical mistakes....

• But people are not very forgiving of social mistakes.

• It is more important to correct mistakes which affect the meaning of several sentences than to correct small grammatical points inside one sentence!!!

• A n Italian who went to New York.

Exemplo 1: Juliana 2ª série ensino fundamental – prova de matemática.

* Leonora tem 15 balas. Leonel tem 8 balas. Quantas balas Leonora tem a mais que Leonel?

Juliana: Leonora tem 7 balas a mais que Leonel.

8+7=15

Princípio da reversibilidade: capacidade de reverter, relações, realizar operações inversas.

Exemplo 2: relato de uma estagiária

“Ao visitar uma sala de crianças maiores e analisar a decoração, verifiquei a existência de trabalhos das crianças: eram flores de papel plantadas em vasos com areia. Inicialmente fiquei feliz ao notar a presença de algum trabalho delas. Porém, logo chamou-me a atenção o fato de os vasos serem idênticos e, mais do que isso, as flores serem todas iguais. “Encucada” questionei as crianças acerca do trabalho. Desse questionamento retirei que quem havia feito os vasos fora a professora e que também ela é que havia colocado areia dentro, também a professora desenhou as pétalas das flores, também a professora colocou o cabinho (todas da mesma altura) e também foi ela que colou as pétalas.

Quando perguntei:

-Tu colaste estas pétalas tão bonitas?

Criança:

-Não, porque eu tinha colado todas assim juntos, então a professora arrumou.

A essa altura cabe a pergunta:

Mas o que enfim as crianças fizeram?

Qual a concepção de avaliação que parece evidenciar a professora observada pela estagiária?

(Quantas vezes não se responde ao aluno sem dar tempo sequer dele pensar? A professora quis garantir o resultado esperado.

)

Exemplo 3: Marcelo é aluno da 3ª série do ensino fundamental. Prova de matemática

-Numere as linhas do caderno de matemática de 1 a 10. Escrevam apenas os resultados das seguintes multiplicações: (e ditou)

-3x7; 4x7; 8x7; 9x7; 2x7; 6x7; 5x7; 1x7; 7x7 e 10x7.

-Marcelo respondeu no caderno:

-1. 7

-2. 14 Após o exercício, os alunos trocaram de cadernos e corrigiram os

-3. 21 exercícios uns dos outros. O colega de Marcelo atribuiu-lhe 2 pontos.

-4. 28 A professora revisou a correção e confirmou o resultado.

-5. 35 Como você, professor(a), procederia no lugar da professora de Marcelo??

-6. 42 ©

-7. 49 Professora : ele memorizou a tabuada na seqüência

-8. 56 Pesquisador descobriu: O menino relacionou aos itens que a professora

-9. 63 mandou enumerar.

-10. 70 ©

Exemplo 4: Uma criança da 4ª série. Estuda o tema “características dos mamíferos” para prova bimestral.

(no livro)* A temperatura dos mamíferos é constante.

* suas garras e dentes são adaptados ao meio ambiente.

A professora acompanha a leitura da menina e pergunta:

- O que é ‘temperatura’?

- O que é constante?

-O que é meio ambiente dos animais?

-O que quer dizer adaptados?

Exemplo 5: A professora perguntou: - O que é desmatar?

A aluna: - tornar vivo de novo!!!!

Depois de analisar o problema a professora chega a conclusão: vocabulário do dia-a-dia: desarrumar, despentear.

Perguntas:

O meu aluno compreende? O meu aluno Não compreende? Por quê?

Exemplos adaptados do livro:

Avaliação: Mitos e Desafios – Jussara Hoffmann

Slips, errors and attempts

• Slip: If the teacher thinks that a student could self-correct a mistake. Eg1

• Error: If a student cannot self- correct a mistake in his or her own English, but the teacher thinks that the class is familiar with the correct form. EG2

• Attempt: When the teacher knows that the students have not yet learned the language necessary to express what they want to say.Eg3

• How would you grade/evaluate your students??????????

Be careful! * I wish I went my grandmother’s

house last summer.• This could be a slip for one student, an

error for another, and an attempt for a third.

• Or for the same student: it might be an attempt today, an error next month and a slip next year....

Learning Steps• A child is learning English as its mother tongue.

• Nina catched a balloon. • Nina got two foots.

• A student (learning English as a FL) in English 2:• I goed to school yesterday.

• (In time, this child/ student will learn that these two very good and useful rules also have exceptions to them. These examples are clear signals that the learning steps are being made.)

Learning Steps• But many of the things we call mistakes

and see as problems are in fact signals that our students are successfully learning the language. They are taking the necessary learning steps.

• Now substitute the word “Mistake” in your sentences to “Learing Steps” and see what you’ve got!!!!

Teacher’s Role• Our job as teachers is NOT just to

point out differences between our students’ language and standard English. That is too negative a role. Our job is also to encourage the growth of the language by appreciating the learning steps. (And show this process to the students!!!)

When should we correct students???

•Risks and chances •Fluency and accuracy

History of testing The Pre-scientific Era (before the 50’s) –

(Spolsky) or Garden of Eden (Morrow)

The Psychometric-Structuralist Era (from 50’s to end of 60’s) –(Spolsky) or The Vale of Tears (Morrow)

The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou The Promised Land (Morrow)

The Pre-scientific Era (Spolsky) or Garden of Eden (Morrow)

• It was not based on scientific knowledge, and for this reason, the idea of testing a language as a distinct activity did not exist. According to Spolsky, developing a test was art.

• Theoretical basis did not existed and the methods were made up from impressionist observations of their creators. They used the same activities from the classroom to evaluate progress: translation, dictation and composition.

• There was no clear difference between testing and teaching.• Garden of Eden – where there were no worries and there

were no distinctions of good and evil. There were no questioning.

• It lasted from the beginning of language teaching to Second World War.

Grammar-translation Test1. Dê a correta forma do presente do indicativo do verbo “to be”.1. He _______ a good student.2. They ________ old friends3. It _______ on the other side of town.4. We _______ cousins.5. Mr.Smith _______ out of town.

2. Traduza para o português as orações abaixo:1. They are good students. _____________________________2. His wife is sick. ________________________.3. We are busy today. __________________________4. She is his cousin. __________________________

3. Agora, faça a versão para o inglês.1. Ele é um bom amigo meu. _________________________________.2. Ele está enfermo. _____________________________________.3. Não há aulas no Domingo. ______________________________.4. O João está ausente na aula hoje. ____________________________

Reading test (30’s and 60’s)A Famous ComposerOne day, a young man went to visit a famous composer and asked him for

advice on how to compose a symphony.“How old are you?” the composer asked.“Eighteen.”“I think that you are too young to compose a symphony,” then the composer

told him. “Try something easier.”“But you composed symphonies when you were eight,” protested the young

man.“Yes,” agreed the famous composer, who was Mozart, “but I did not ask

anyone how to compose them.”(adapted from “English I” p15 – Amadeu Marques, 1978, Ed Ática) Refer to the reading and write or complete the answers to the following questions:1. Who was the famous composer?2. Why did the young man visit the famous composer?3. What did Mozart ask the young man?4. How old was the young man?5. Did Mozart advise the young man to compose another symphony?6. What did Mozart advise the young man?7. Was the young man too old to compose a symphony?8. Did the young man protest?9. What did Mozart do when he was eight?10. Did Mozart ask anyone how to compose a symphony?

Marcuschi, L.A. Exercícios de compreensão ou copiação nos manuais

de ensino de língua? In: Em Aberto, Brasília, ano 16, n.69, jan/mar, 1996

Tipologia de questões abertas de interpretação de textos:1. “Falta de Horizonte” – apenas há repetição ou cópias do que

está dito no texto.

2. “Horizonte Mínimo” – paráfrases são utilizadas, ou seja, uma espécie de repetição com outras palavras.

3. “Horizonte Máximo” – inferências: atividades de geração de sentidos pela reunião de várias informações do próprio texto, ou pela introdução de informações e conhecimentos pessoais outros não contidos no texto.

4. “Horizonte Problemático” – São perguntas que vão muito além das informações do próprio texto. São leituras de caráter idiossincrático, bem pessoal, onde o investimento de conhecimentos pessoais é muito grande e chega a ser preocupante.

5. “Horizonte Indevido” – leitura errada.

Nery, M. Rosa Questões sobre questões de leitura. Campinas, SP. Alínea Editora, 2003.

The Psychometric-Structuralist Era (1920’s-60’s) Spolsky

The Vale of Tears (Morrow)Baker (1989:29)• The 1920s and 30s saw a great vogue for psychological testing. Large numbers

of tests investigating every aspect of the psyche from intelligence to job aptitudes were produced and millennial predictions were sometimes made about the social benefits that large-scale testing of this kind would bring. Few of these tests actually delivered the

miraculous solutions which had been promised but they survive today in the form of

intelligence tests and, in a less serious form, as magazines quizzes of the kind “Are you a good husband?”

1. Behaviorist psychology offered us concepts such as : the multiple-choice method – descrete-pont tests (testes de itens isolados); terms such as reliability, validity, practicality, psychometric

The Psychometric-Structuralist Era (1920’s-60’s) Spolsky

The Vale of Tears (Morrow)2. According to the academic tradition which

helped define the limits of language testing –Structuralist linguistics. They analyzed the fragmented linguistic system in bits of pieces and then described possible ways the bits and pieces could be united again to form bigger pieces of speech. Hierarchy – phonemes, words, phrases, etc. “Language was a huge puzzle”.

The Psychometric-Structuralist Era (1920’s-60’s)

• Vestibular UFPR (1977) Descrete-point tests = Questões de itens isolados

48. Assinale a alternativa correta.Mr. & Mrs.Brown invited us to ____ house, and there we met a

cousin of ______.a) their, hisb) Our, herc) Their, theirsd) His, hise) Her, hers49. The girls ______ were here this morning are probably Sally´s

friends. a) Whichb) whom c) who d) whose e)

who´s

The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou

The Promised Land (Morrow)

Psycholinguistics – Frank Smith (1978a/b); Kenneth Goodman (1967, 1976a/b)

Hymes (1967, 1972), Canale&Swain(1980), Canale(1982)

The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou The Promised Land

(Morrow) 70’s/80’s?

• From a psycholinguistic perspective, language was seen less as a taxonomic structure well defined and more like a creative and dynamic functional system

• The contribution of sociolinguistic was centered on the conception of communicative competence which not only covered the knowledge of rules to form grammatical sentences, but also on rules of adequate usage of sentences in different contexts.

• Therefore the psycholinguistic and sociolinguistic perspective helped develop basis for valid testing.

The Psycholinguistic-Sociolinguistic Era

1. Dictation. Prepare to take dictation.

________________________________________________________________________________________________________________________________________________________________________________________________________________________

2. Fill int the gaps in the passage below with na appropriate word.The history of money is very interesting. Today money is used everywhere to

buy goods and services. In the past _________were among the earliest objects used as money. These were originally used as _________ but gradually the people in the Pacific began to use them in exchange for food and other items.

In other countries many different objects were used as money. Rice was used in the ___________, whereas in Africa they used ________ which is still in use as money in certain parts of that continent today.

A person´s weath was often demonstrated by the number of _________ he owned and it was even possible in some countries to buy a ___________ with these animals

Coins came in many shapes and sizes. In Russia they were made from ________ but in _______ they were made from leather. The _________ of the coins, however, was the main factor which led to the invention of money made from __________.

3. Compositon: Write about what you did on your last vacation!!!!

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Where are we heading to?

Communicative paradigm of testing

Objectives: 1. Try to simulate tasks that learner will perform in the

future. 2. Integrate abilities3. Authentic and contextualized content4. Qualitative and holistic corrections 5. Direct vs indirect6. Performance vs knowledge

Exemple: Celpe-Bras Certificado de Língua Portuguesa para Estrangeiros – Ministério da Educação.

Types of testingProficiency testsAchievement DiagnosticPlacementEntranceMastery (esp)

Concepts• Validity

- construct- face- content

• reliability• Practicality

• Always in tension

reliability

validity

practicality

Validity, reliability and practicality

Validity - Content – does your test consist of a respresentative

sample of the abilities and sub-abilities worked in class? - Construct – when a test evaluates an ability (not only the

content) that it is intended to evaluate.- Face - Does your test look like a reliable test?Reliability- It answers the following questions: - Will the same evaluator grade the same tests, the same

way in different times?- Will different evaluators grade the same at the same time?- Did the test allow the testtakers to grade the same at

different times?Practicality- A test must be feasible within a limit of time and budget. It

must have high cost-benefit.

What are students’ views of language

tests?• St1 – We spent ten lessons conjugating the past tense but on

the test there were only two conjugations.• St2 – I never learn anything from tests because the teacher

never corrects the mistakes I make, so I end up at the same place where I was before I took the test, except now I also have a bad grade.

• St3 – The teacher bases the grade on one test, so if I do not feel well on the same day that the test was given, I flunk the course.

• St4 – I don’t believe that any test is a good measure of my performance, let alone on a day when I do not feel well.

• St5 – I don’t see the connection between the test and my knowledge, otherwise, how can I explain the fact that I get good grades on English tests, but last week, when I met a n American, I couldn’t say anything in English? How come we never speak on tests?

What are students’ views of language

tests?St6 – Our teacher uses the test as a punishment, whenever we

don’t behave she tells us we will have a test the following day. Sometimes she even asks us to take out a sheet of paper and write the test on the spot.

St7- A test really makes me study, I would have never opened a book if not the test. I think the pressure is good for me.

St8 – I hate to flunk, tests always show me that I am a failure.St9 – It seems that whenever the teacher is unprepared she

asks us to write a test.St10 – I don’t mind the test, what I do mind is that it takes the

teacher such a long time to correct the tests. By the time I get the test back, I forgot what the test was all about.

What are students’ views of language

tests?• St11 – I think that it is really strange that

whenever I study hard, I don’t get a good grade but when I don’t study at all, I happen to succeed. Boes it say something about me or about the test?

• St12 – What I hate the most is when the teacher does not tell us in advance what the test will cover. It seems that I am always studying the wrong things.

• St13 – Why do we need tests? The teacher knows how well we are doing anyway.

What’s wrong with my test?

• Using tests as a disciplinary tool – as a punishment.

• There is a gap between what is taught and what is tested.

• There is a delay in returning test papers so the feedback, when given, becomes irrelevant.

• There is a n absence of stisfactory corrections and explanations of erros – too little information that could be used by students for correction/improvement.

• There is a feeling of failure that results from obtaining poor results on test.

What’s wrong with my test?

• Teachers themselves do not feel confident in their tests.

• There is little knowledge on the part of the teachers as to the purpose and uses of language tests and their role in language learning.

• Using tests exclusively for grading.• Using tests without propor planning.• No coorporation among teachers about tests;

no analysis of tests after they have been administered.

What’s wrong with my test?

• Not discarding bad tests or bad items• Not preparing students for tests or for testing

procedure.• Not giving feedback about tests• Lack of testing knowledge and training• Using one testing method to tap given

knowledge.• Using tests which were intended for different

populations and not checking if they are appropriate for a given population.

The Multiple Intelligences by Howard Gardner - Frames of Mind: The Theory of Multiple Intelligences

* Gardner suggested there were at least 7 different kinds of intelligences, i.e., 7 different ways of perceiving and “knowing” the world . This also suggested that people solve problems which imerge in every day life using 7 different learing styles.

a

Logical / Number smart - You may be number

smart. You will be good at mathematics and other number activities; you are also good at

solving problems. This is sometimes called being Logical

smart.

a

Linguistic/ Work smart You may be word smart.

You will enjoy reading, writing and talking about things. This is sometimes

called being Linguistic smart.

a

Kinaesthetic / Body smart –

You may be body smart. You will enjoy sports and are good at swimming,

athletics, gymnastics and other sports. This is

sometimes called being Kinaesthetic smart.

a

Musical – Music Smart You may be music

smart. You will enjoy music and can recognise sounds, and timbre, or

the quality of a tone. This is sometimes called being Musical smart.

a

Visual/Spatial-Picture smart

You may be picture smart. You will be good at art and also good at

other activities where you look at pictures like map reading, finding

your way out of mazes and graphs. This is sometimes called being

Visual/Spatial smart.

a

Interpersonal /People smart You may be people smart. You

will like to mix with other people and you will belong to lots of clubs. You like team

games and are good at sharing. This is sometimes called being Interpersonal

smart. 

a

Intrapersonal / Myself smart You may be myself smart. You will know about yourself and

your strengths and weaknesses. You will probably keep a diary. This is sometimes

called being Intrapersonal smart.

Did anyone barge into your testing ?????