Testing…testing…one two three

March 6th, 2006
by Cleve

So I’m feeling guilty for browsing through BoingBoing in the middle of the work day and, lo and behold, they provide a wonderful rationalization: an interesting article on some research that purports to show that repeated testing improves learning (well, memory).

Reading through the article, it makes intuitive sense: frequent feedback on what is retained improves retention. It also fits my personal learning style. And it fits with what I do as a language teacher: “tests” that are so frequent that they cease to be thought of as tests. This gives everyone great feedback and improves performance as well as metacognitive practice.

But…of course testing and measurement are pariahs in the edutech community now. I don’t work in the institutionalized public sector, so I can’t say how bad things are, testing-wise, in the school systems, but I assume it’s horrible and that explains why everyone is so against testing and measurement.

But I can’t help feeling that anti-test bias is confusing goals and means. It’s the way testing is carried out that is the problem, not testing itself. You have to give feedback on performance to learners - how can you do that without some way of assessing what their performance is? You can have learners help decide what the assessment schema is, given their goals, but even with dreaded standardized testing, it’s useful feedback.

The much-maligned TOEIC is a good example: I know it doesn’t measure production, and the accents are (were?) all from unusually bland US speakers. Yet with TOEIC results there is a correlation with overall language ability. I did linguistic auditing for a big multinational for many years and as corporate policy the TOEIC was used as an initial placement tool. And it actually worked very well - we coupled it with an oral interview and the TOEIC score was usually pretty much right on with what we arrived at after the 30′ interview. So in my experience there is correlation. I’d be interested in any studies on this if anyone knows of any.

Of course you can’t use the TOEIC exclusively, especially as a progress test. That would be an invalid use of the TOEIC tool. You need to use it as part of the overall assessment suite, together with other tools (self-assessment, teacher assessment, performance reviews, portfolios, etc.) But to say “the TOEIC sucks” is like complaining about a screwdriver because it won’t pound in a nail very well (or build a house).

From what I read, granted, institutionalized education is abusing testing tools as well (tests as hammers?). But that doesn’t mean that teachers don’t need to think about how to measure if what we are doing works or not. We are all enthused about new ways to teach and learn - why? Because these new ways help people learn about the new world and how to interact with it, better than previous ways? I assume that as teachers we are enthusiastic about new techniques because they are in some way “better”, but that very comparison is an implicit measurement. We are all measuring all the time, albeit often subconciously and subjectively. So instead of this “hidden” measurement, let’s just acknowledge that we measure and just try to find the best ways to do it.

Two requests: first, I admit that I just “don’t get it” when it comes to the anti-measurement thing. If anyone would like to help me out with some links, I’d appreciate it. Second, amazingly, I found myself disagreeing with every sentence in a Stephen Downes post this morning (I actually went in and checked - I have a problem with every sentence, except the sentence “Sheesh.” which was OK). What am I missing? Downes is an idol - what gives? What are the “terms of success” that Downes would establish? Again, any linkage anyone could send would be great. Sorry for being clueless.

3 Comments