ESL Snake Oil
I have recently found (can’t remember where exactly) Doug Noon’s Borderland blog. Fascinating and thought provoking, a true story teller.
Yesterday Doug sounded off on a test that I’ve never heard of, but I think his post is relevant for many of us today:
DIBELS and the Seductive Lure of Snake Oil
“…if it’s taught because it’s tested, what typically gets taught is not the skills or content, but test-taking itself.” (Borderland, 2006.)
I see the crossover into ESL. The TOEIC, TOEFL etc. How often do our prep classes turn into TEST TAKING classes. Where we focus on how to hack the test and shortcut to success.
Students can score high on the test, but has anything really useful happened to their English skills? I’ve said it before: I know people who have scored high on the TOEIC (Test of English for International Communication) but cannot hold their own in a conversation.
If you were a university admissions administrator or placement counselor in an L2 country, what would you use to gauge students’ L2 language proficiency?
Comment by Charles — January 11, 2006 @ 10:36 am
Hi Aaron,
Thank you for your comment. As a person who studied Spanish in high school and at the university for a while, I think you offer a great example of the limitations of test data.
Administrators and policy makers are drawn to these assessment tools because teacher observations are regarded as “anecdotal” and “unscientific,” as opposed to the supposedly more accurate information that is generated from standardized measures. If the professional opinion of teachers held more weight, the crude numerical data generated by standardized tests would be irrelevant. An uncritical belief in the objective truths that scientific observation seems to offer is a product of the technical age we’re living in.
Comment by Doug Noon — January 11, 2006 @ 11:05 am
Thanks for your comments Charles and Doug,
Charles:
I’m not advocating that we abandon the TOEIC or TOEFL. They have their uses. What I do passionately hope for, is that focus shifts from hacking the test, to preparing for life. What good would a high TOEFL score do you if you really couldn’t use English in real life? Sure you could get into college or university, but could you survive there? Two very different issues I think.
TOEFL and TOEIC are useful weeders, but we must be careful of something Doug mentions in his Snake Oil post: What we teach very often reflects what we believe. I teach English, but I also teach what I believe about learning English. If I place all my eggs in the TOEIC basket, there is a big possibility that I, my students, the job market, etc, will link high scores to language proficiency. A link that is very often quite false.
So no…I don’t advocate the destruction of these tests. They may very well have a useful place. But what happens in prep classrooms? Do we teach to pass a test? Will that really help students handle reality - or simply pass the test?
Doug:
I can’t remember where I read this, but it was on a district website - “Data driven classrooms.” Scared me. I totally agree with you - numerical data is crude and cold. We’re talking about people here aren’t we? Wouldn’t we get safer and tailored information from teachers vs. standardized measures?
Comment by Aaron Nelson — January 11, 2006 @ 2:34 pm
I’m not sure that administrators simply regard teacher’s comments as anecdotal. After all, they like to have letters of recommendation. But different teachers do have different standards, and some communicate better. Standardized tests give administrators a way to compare students across teachers and schools (which often have different grading systems, especially for international students).
As far as teaching, some prep for these tests is fine, as part of the test is just being familiar with it. But for the most part, extensive use of the language will be of much more value–even for passing standardized tests.
Comment by Charles — January 14, 2006 @ 7:05 am
Personally I don’t hold much stock in TOEIC/TOEFL, but there is a rough correlation between ability and score. However a much more accurate approach would be to have a double blind interview series. Two teachers, having recieved rater training, would interview each learner without know what the other teacher scored them. This however is not very practical in Large programs due to time constraints and administrative costs.
Comment by EFL Geek — January 20, 2006 @ 1:13 pm