Shall we discuss how to effectively challenge the tyranny of ETS standardized exams? May I strongly recommend reading this earnest critique by Clarissa, a frustrated English teacher who has taught TOEIC and TOEFL and prominent ESL blogger for many years?
TOEFL, TOEIC, and jobs « Readable Blog
As somebody who has often criticized - even ranted against - the overuse and misuse of standardized exams, consider me quite sympathetic of your analysis. It's also worth noting that no outside study has even found a correlation between TOEFL scores and actual academic performance in either university or graduate school. Although I'm less familiar with the TOEIC, it seems to share many of the perceived advantages for large institutions (test consistency, convenience, cost) and the same disadvantages (artificial context, limited scoring, weak correlation with alleged goals, excessive test preparation).
Having said all that, it seems a tad unfair to attack the TOEIC as " still not perfect or very realistic." Not perfect? Isn't that asking for far too much? No standardized test - and very few human endeavors - are "perfect." Noting the TOEIC is not "very realistic" in the next phrase only adds some irony to the critique. May I suggest it "not realistic" to expect any exam to be "perfect"?
Further, I'm not sure that we can wish away either the TOEIC or TOEFL or just advise our students to disregard the results. As an experienced TOEIC and TOEFL teacher, you have seen - first hand - the power to open doors or defer dreams that these strange standardized exams continue to hold. Scores matter. ETS holds real power. Power must be recognized.
Let us, instead, speak truth to power as your fine article does - and prepare students to develop their authentic English skills. For instance, I would recommend students both study for the TOEIC with both a traditional test prep book and the multiskill Target Score (Cambridge University) that correlates authentic newspaper articles to TOEIC vocabulary. In other words, we need to both help our students raise their scores and authentic English communication skills in the short run. TOEFL and TOEIC scores matter. We must be realistic in our counseling to students.
Of course, we also need to find ways to reduce the influence of standardized exams in both international companies and educational institutions, but that is a very daunting, difficult challenge. I would prefer, for instance, that universities supplement the TOEFL with their own language tests - especially for speaking skills. Videotaping a 10-15 minute interview, for instance, can fairly accurately assess student speaking skills and conversational abilities.
I deeply appreciate the author's profound misgivings about both the TOEFL and TOEIC. Far too few English teachers seem willing to note that the ETS emperor has far too few clothes on!
Ask more. Know more. Share more.Create Compelling Conversations.Visit www.CompellingConversations.com
Showing posts with label oral tests. Show all posts
Showing posts with label oral tests. Show all posts
Sunday, November 14, 2010
Wednesday, December 31, 2008
New Year: Old Dilemmas for Assessing Listening and Speaking Skills for English Language Learners
As we welcome the new year, we can anticipate some familiar pleasures and puzzles.
For English language teachers, establishing reasonable standards for oral communication remains a complicated, difficult challenge. Students must be able to clearly communicate their ideas with adequate pronunciation, but we must allow for a rich variety of accents too. Some ESL teachers also find themselves placed in the role of both gatekeeper - upholding standards of admission to university courses - and coach. Some tension between these roles occurs when evaluating a student's oral skills for mandatory courses.
English teachers, especially ESL and ESOL working with adult English language learners, can build on the experience of international organizations working with refugees and immigrants.
Check out this practical chart with ten categories for listening comprehension and speaking skills developed by the United States Office of Refugee Resettlement and revised by the Spring Institute for Intercultural Learning. Where did I find it? As so often, I went to the Center for Applied Linguistics website.
What are reasonable standards for listening comprehension and speaking skills for adult English language learners? That’s a difficult question, and context matters. This chart, however, seems quite sensible.
SPL stands for Student Performance Level here. The descriptors for listening comprehension and oral communication are intended to establish a standard that government agencies, educational institutions, and non-profits can all use to share evaluations. Despite the bureaucratic title, the chart itself contains excellent descriptions that English teachers and testers can use for adult English language learners.
After having been in several long faculty discussions over standards for oral skills, I appreciate the explicit standards combined with some flexibility.
This chart, developed for adult refugees, resembles other charts, yet includes more details and an explicit acknowledgment of economics. I like that awareness even if this factor can sometimes be misused to justify low standards in adult education programs. (After all, illiterate peasants move to the United States, Canada, England, and Australia in hopes of improving their economic and social status.)
Our job as educators is to provide our students with the language skills to live fuller, more satisfying lives – in English – wherever they choose to live and work.
Student Performance Level (SPL) Descriptors for Listening Comprehension and Oral Communication
SPL General Language Ability
Listening Comprehension Oral
Communication
0
No ability whatsoever
No ability whatsoever
No ability whatsoever
1
Functions minimally, if at all, in English. Can handle only very routine entry-level jobs that do not require oral communication, and in which all tasks can be easily demonstrated. A native speaker used to dealing with limited English speakers can rarely communicate with a person at this level except through gestures.
Understands only a few isolated words, and extremely simple learned phrases.
Vocabulary limited to a few isolated words. No control of grammar.
2
Functions in a very limited way in situations related to immediate needs. Can handle only routine entry-level jobs that do not require oral communication, and in which all tasks can be easily demonstrated. A native English speaker used to dealing with limited English speakers will have great difficulty communicating with a person at this level.
Understands a limited number of very simple learned phrases, spoken slowly with frequent repetitions.
Expresses a limited number of immediate survival needs using very simple learned phrases.
3 Functions with some difficulty in situations related to immediate needs. Can han¬dle routine entry-level jobs that involve only the most basic oral communication, and in which all tasks can be demonstrated. A native English speaker used to dealing with limited English speakers will have great difficulty communicating with a person at this level.
Understands simple learned phrases, spoken slowly with frequent repetitions.
Expresses immediate survival needs using simple learned phrases.
4 Can satisfy basic survival needs and a few very routine social demands. Can handle entry-level jobs that involve some simple oral communication, but in which tasks can be easily demonstrated. A native English speaker used to dealing with limited English speakers will have difficulty communicating with a person at this level.
Understands simple learned phrases easily, and some simple new phrases containing familiar vocabulary, spoken slowly with frequent repetitions.
Expresses basic survival needs including asking and responding to related questions, using both learned and a limited number of new phrases. Participates in basic conversations in a few very routine social situations. Speaks with hesitation and frequent pauses. Some control of basic grammar.
5 Can satisfy basic survival needs and some limited social demands. Can handle jobs and job training that involve following simple oral instructions but in which most tasks can also be demonstrated. A native English speaker used to dealing with limited English speakers will have some difficulty communicating with a person at this level.
Understands learned phrases easily and short new phrases containing familiar vocabulary spoken slowly with repetition. Has limited ability to understand on the telephone.
Functions independently in most face-to-face basic survival situations but needs some help. Asks and responds to direct questions on familiar and some unfamiliar subjects. Still relies on learned phrases but also uses new phrases (i.e., speaks with some creativity) but with hesitation and pauses. Communicates on the phone to express a limited number of survival needs, but with some difficulty. Participates in basic conversations in a limited number of social situations. Can occasionally clarify general meaning.
6
Can satisfy most survival needs and limited social demands. Can handle jobs and job training that involve following simple oral and written instructions and diagrams. A native English speaker not used to dealing with limited English speak¬ers will be able to communicate with a person at this level on familiar topics, but with difficulty and some effort.
Understands conversations containing some unfamiliar vocabulary on many every¬day subjects, with a need for repetition, rewording or slower speech. Has some ability to understand without face-to-face contact (e.g. on the telephone, TV).
Functions independently in most survival situations, but needs some help. Relies less on learned phrases; speaks with creativity, but with hesitation. Communicates on the phone on familiar subjects but with some difficulty. Participates with some confidence in social situations when addressed directly. Can sometimes clarify general meaning by rewording. Control of basic grammar evident, but inconsistent; may attempt to use more difficult grammar but with almost no control.
7
Can satisfy survival needs and routine work and social demands. Can handle work that involves following oral and simple written instructions in familiar and some unfamiliar situations. A native English speaker not used to dealing with limited English speakers can generally communicate with a person at this level on familiar topics.
Understands conversations on most everyday subjects at normal speed when addressed directly; may need repetition, rewording, or slower speech. Understands routine work-related conversations. Increasing ability to understand without face-to-face contact (telephone, TV, radio). Has difficulty following conversation between native speakers.
Functions independently in survival and many social and work situations, but may need help occasion¬ally. Communicates on the phone on familiar subjects. Expands on basic ideas in conversation, but still speaks with hesitation while searching for appropriate vocabulary and grammar. Clarifies general meaning easily, and can sometimes convey exact meaning. Controls basic grammar, but not more difficult grammar.
8 Can participate effectively in social and familiar work situations. A native English speaker not used to dealing with limited English speakers can communicate with a person at this level on almost all topics.
Understands general conversation and conversation on technical subjects in own field. Understands without face-to-face contact (telephone, TV, radio); may have difficulty following rapid or colloquial speech. Understands most conversations between native speakers; may miss details if speech is very rapid or colloquial or if subject is unfamiliar. Participates effectively in practical and social conversation and in technical discussions in own field. Speaks fluently in both familiar and unfamiliar situations; can handle problem situations. Conveys and explains exact meaning of complex ideas. Good control of grammar.
9 Can participate fluently and accurately in practical, social, and work situations. A native English speaker not used to dealing with limited English speakers can communicate easily with a person at this level.
Understands almost all speech in any context. Occasionally confused by highly colloquial or regional speech.
Approximates a native speaker’s fluency and ability to convey own ideas precisely, even in unfamiliar situations. Speaks without effort. Excellent control of grammar with no apparent patterns of weakness.
10
Ability equal to that of a native speaker of the same socioeconomic level.
Equal to that of a native speaker of the same socioeconomic level. Equal to that of a native speaker of the same socioeconomic level.
Original chart from http://www.cal.org/caela/esl_resources/slspls.html
These standards, of course, remain more relevant for adult educators and workplace programs than more academic programs. English teachers should, however, create classroom activities where students can engage in extended conversations in English on a wide variety of topics. The higher levels of this chart (SPL 9-10) articulate an excellent standard for all English language learners, including academic English and Business English students.
Ask more. Know more. Share more.
Create Compelling Conversations.
Visit www.CompellingConversations.com
For English language teachers, establishing reasonable standards for oral communication remains a complicated, difficult challenge. Students must be able to clearly communicate their ideas with adequate pronunciation, but we must allow for a rich variety of accents too. Some ESL teachers also find themselves placed in the role of both gatekeeper - upholding standards of admission to university courses - and coach. Some tension between these roles occurs when evaluating a student's oral skills for mandatory courses.
English teachers, especially ESL and ESOL working with adult English language learners, can build on the experience of international organizations working with refugees and immigrants.
Check out this practical chart with ten categories for listening comprehension and speaking skills developed by the United States Office of Refugee Resettlement and revised by the Spring Institute for Intercultural Learning. Where did I find it? As so often, I went to the Center for Applied Linguistics website.
What are reasonable standards for listening comprehension and speaking skills for adult English language learners? That’s a difficult question, and context matters. This chart, however, seems quite sensible.
SPL stands for Student Performance Level here. The descriptors for listening comprehension and oral communication are intended to establish a standard that government agencies, educational institutions, and non-profits can all use to share evaluations. Despite the bureaucratic title, the chart itself contains excellent descriptions that English teachers and testers can use for adult English language learners.
After having been in several long faculty discussions over standards for oral skills, I appreciate the explicit standards combined with some flexibility.
This chart, developed for adult refugees, resembles other charts, yet includes more details and an explicit acknowledgment of economics. I like that awareness even if this factor can sometimes be misused to justify low standards in adult education programs. (After all, illiterate peasants move to the United States, Canada, England, and Australia in hopes of improving their economic and social status.)
Our job as educators is to provide our students with the language skills to live fuller, more satisfying lives – in English – wherever they choose to live and work.
Student Performance Level (SPL) Descriptors for Listening Comprehension and Oral Communication
SPL General Language Ability
Listening Comprehension Oral
Communication
0
No ability whatsoever
No ability whatsoever
No ability whatsoever
1
Functions minimally, if at all, in English. Can handle only very routine entry-level jobs that do not require oral communication, and in which all tasks can be easily demonstrated. A native speaker used to dealing with limited English speakers can rarely communicate with a person at this level except through gestures.
Understands only a few isolated words, and extremely simple learned phrases.
Vocabulary limited to a few isolated words. No control of grammar.
2
Functions in a very limited way in situations related to immediate needs. Can handle only routine entry-level jobs that do not require oral communication, and in which all tasks can be easily demonstrated. A native English speaker used to dealing with limited English speakers will have great difficulty communicating with a person at this level.
Understands a limited number of very simple learned phrases, spoken slowly with frequent repetitions.
Expresses a limited number of immediate survival needs using very simple learned phrases.
3 Functions with some difficulty in situations related to immediate needs. Can han¬dle routine entry-level jobs that involve only the most basic oral communication, and in which all tasks can be demonstrated. A native English speaker used to dealing with limited English speakers will have great difficulty communicating with a person at this level.
Understands simple learned phrases, spoken slowly with frequent repetitions.
Expresses immediate survival needs using simple learned phrases.
4 Can satisfy basic survival needs and a few very routine social demands. Can handle entry-level jobs that involve some simple oral communication, but in which tasks can be easily demonstrated. A native English speaker used to dealing with limited English speakers will have difficulty communicating with a person at this level.
Understands simple learned phrases easily, and some simple new phrases containing familiar vocabulary, spoken slowly with frequent repetitions.
Expresses basic survival needs including asking and responding to related questions, using both learned and a limited number of new phrases. Participates in basic conversations in a few very routine social situations. Speaks with hesitation and frequent pauses. Some control of basic grammar.
5 Can satisfy basic survival needs and some limited social demands. Can handle jobs and job training that involve following simple oral instructions but in which most tasks can also be demonstrated. A native English speaker used to dealing with limited English speakers will have some difficulty communicating with a person at this level.
Understands learned phrases easily and short new phrases containing familiar vocabulary spoken slowly with repetition. Has limited ability to understand on the telephone.
Functions independently in most face-to-face basic survival situations but needs some help. Asks and responds to direct questions on familiar and some unfamiliar subjects. Still relies on learned phrases but also uses new phrases (i.e., speaks with some creativity) but with hesitation and pauses. Communicates on the phone to express a limited number of survival needs, but with some difficulty. Participates in basic conversations in a limited number of social situations. Can occasionally clarify general meaning.
6
Can satisfy most survival needs and limited social demands. Can handle jobs and job training that involve following simple oral and written instructions and diagrams. A native English speaker not used to dealing with limited English speak¬ers will be able to communicate with a person at this level on familiar topics, but with difficulty and some effort.
Understands conversations containing some unfamiliar vocabulary on many every¬day subjects, with a need for repetition, rewording or slower speech. Has some ability to understand without face-to-face contact (e.g. on the telephone, TV).
Functions independently in most survival situations, but needs some help. Relies less on learned phrases; speaks with creativity, but with hesitation. Communicates on the phone on familiar subjects but with some difficulty. Participates with some confidence in social situations when addressed directly. Can sometimes clarify general meaning by rewording. Control of basic grammar evident, but inconsistent; may attempt to use more difficult grammar but with almost no control.
7
Can satisfy survival needs and routine work and social demands. Can handle work that involves following oral and simple written instructions in familiar and some unfamiliar situations. A native English speaker not used to dealing with limited English speakers can generally communicate with a person at this level on familiar topics.
Understands conversations on most everyday subjects at normal speed when addressed directly; may need repetition, rewording, or slower speech. Understands routine work-related conversations. Increasing ability to understand without face-to-face contact (telephone, TV, radio). Has difficulty following conversation between native speakers.
Functions independently in survival and many social and work situations, but may need help occasion¬ally. Communicates on the phone on familiar subjects. Expands on basic ideas in conversation, but still speaks with hesitation while searching for appropriate vocabulary and grammar. Clarifies general meaning easily, and can sometimes convey exact meaning. Controls basic grammar, but not more difficult grammar.
8 Can participate effectively in social and familiar work situations. A native English speaker not used to dealing with limited English speakers can communicate with a person at this level on almost all topics.
Understands general conversation and conversation on technical subjects in own field. Understands without face-to-face contact (telephone, TV, radio); may have difficulty following rapid or colloquial speech. Understands most conversations between native speakers; may miss details if speech is very rapid or colloquial or if subject is unfamiliar. Participates effectively in practical and social conversation and in technical discussions in own field. Speaks fluently in both familiar and unfamiliar situations; can handle problem situations. Conveys and explains exact meaning of complex ideas. Good control of grammar.
9 Can participate fluently and accurately in practical, social, and work situations. A native English speaker not used to dealing with limited English speakers can communicate easily with a person at this level.
Understands almost all speech in any context. Occasionally confused by highly colloquial or regional speech.
Approximates a native speaker’s fluency and ability to convey own ideas precisely, even in unfamiliar situations. Speaks without effort. Excellent control of grammar with no apparent patterns of weakness.
10
Ability equal to that of a native speaker of the same socioeconomic level.
Equal to that of a native speaker of the same socioeconomic level. Equal to that of a native speaker of the same socioeconomic level.
Original chart from http://www.cal.org/caela/esl_resources/slspls.html
These standards, of course, remain more relevant for adult educators and workplace programs than more academic programs. English teachers should, however, create classroom activities where students can engage in extended conversations in English on a wide variety of topics. The higher levels of this chart (SPL 9-10) articulate an excellent standard for all English language learners, including academic English and Business English students.
Ask more. Know more. Share more.
Create Compelling Conversations.
Visit www.CompellingConversations.com
Subscribe to:
Posts (Atom)
Posts
