Ohhhh . . . Princeton Review. Now I get it.
June 25th, 2007 by Mike Barrett
While I was researching a recent post on the race and gender gaps in the SAT, I came across this 2003 article in the Pittsburgh Post-Gazette:
http://www.post-gazette.com/pg/03239/215443.stm
It discusses some of the points I’ll be making in my post. But I was very surprised by something buried toward the end of the article.
After trotting out the usual pro-testing comments from the College Board and the usual anti-testing comments from FairTest, the article mentions Alexandra Freer, the author of The Girls’ Guide To the SAT.
Freer starts out making some good points. She notes that the SAT requires a different skill-set from regular high school tests, for example. But then she says stuff like
On the SAT, you need to take a smart guess after eliminating the obvious wrong answers and going on to the next problem—or else you’ll run out of time
and
the SAT, which is nearly all multiple choice, rewards those who take risks and guess well.
Nothing could be further from the truth. The SAT doesn’t reward guessing; it punishes it. And since every SAT question can be answered in under 30 seconds (which is well under the amount of time allotted to each question), running out of time on the SAT is usually only a problem for students who have been coached incorrectly. Uncoached students and properly coached students almost never report running out of time.
So I was more than a little puzzled to read Freer’s comments. And then I moved on to the next sentence in the article:
Freer . . . has taught SAT prep courses for the past 15 years for the Princeton Review . . .
Suddenly, it all made sense.
Tell Others About This Article
This entry was posted on Monday, June 25th, 2007 at 12:29 pm and is filed under SAT. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
June 27th, 2007 at 11:54 am
lol that sounds about right
June 29th, 2007 at 2:49 am
What is meant by “the SAT does not reward guessing, it punishes it”?
June 29th, 2007 at 11:52 am
Good question. The answer is kind of detailed.
The SAT subtracts a quarter of a raw point for every wrong answer (every right answers gets you one whole raw point). This wrong-answer penalty is meant to counteract any lucky guessing you might do: in theory, if you guess randomly on a selection of questions with 5 possible answer choices ( (a), (b), (c), (d), and (e) ), mathematical probability says you should be right 1/5 of the time and wrong 4/5 of the time. If you gain one raw point for each correct answer and lose 1/4 point for each incorrect answer, then your overall guessing shouldn’t affect your score. For every right answer that gets you one point, you’ll have four wrong answers that lose you 1/4 point each.
Many popular SAT companies, like Kaplan and Princeton Review, teach you a “guessing strategy” that supposedly exploits this penalty. This guessing strategy says that if you can eliminate one or two answer choices from a question, you should go ahead and guess from the remaining answer choices. If you can do this consistently, then you should come out ahead, according to the theory, because you’ll be guessing from fewer than five answer choices–even though you’ll only lose 1/4 of a point for every wrong answer, as though you were guessing from five choices. In this way, you can up the odds. At least in theory :)
The problem with this approach is that it relies on probability and randomness in order to work. But nobody chooses an answer choice on the SAT at random–you choose an SAT answer based on your ideas about the question. That means the entire notion of relying on probability is flawed, because probability relies on random selection.
Because SAT questions are written to trick people who don’t know how to analyze them, it’s very hard to use the guessing strategy. This is why high-scoring SAT-takers never guess.
Is this something people want to hear more about? I’d be happy to write a longer article on the subject. Leave me a comment.
June 30th, 2007 at 2:17 am
I am very interested in this question and have tried some mathematical analyses more sophisticated than the standard expected value calculation. Would be happy to discuss and especially interested to hear more of your ideas.
June 30th, 2007 at 11:02 am
hi guess guess–
great! i’ll post a longer article on the subject later today. when you get a chance, could you post some of your calculations, either as a response to this thread or to that one? i’d be interested to see what you’ve come up with.
thanks!
mike
June 30th, 2007 at 4:26 pm
My approach was to look not at the expected raw score but the expectation of F(score) where F is some function that quantifies the benefit to the applicant of getting a particular score. After all, raw score has no importance in itself, the applicant is more interested in the effect of the score on admission or scholarship chances, impressing friends and employers, or other such things.
The expected value calculation works fine in this setup and concludes that guessing is advantageous provided that F is convex and disadvantageous if F is concave.
The trouble is that in reality we have a discrete function that is more complicated than either side of that dichotomy and the analysis starts to become fuzzy. Your objection about the guessing being nonrandom can be overcome in various ways, but even in the simpler case that one doesn’t account for it, the model does not lead to clear conclusions.
July 1st, 2007 at 1:35 am
hmm. that’s gonna require me to re-think how to lay out what i planned on writing. i’ve taken a very different approach to the problem, one that’s non-statistical. i think any effective analysis of guessing on the SAT has to take into account the situation that an SAT-taker finds himself in when choosing to guess on a particular SAT question (as opposed to some other type of multiple-choice question). there are considerations that can’t be quantified (or, at least, they can’t be quantified by *me* :) ). i’ll sleep on figuring out the best way to put that forth and post that article tomorrow.
thanks,
mike
July 2nd, 2007 at 1:18 pm
[…] recent post on an article I found about the Princeton Review included a passing remark that the SAT punishes guessing, rather than rewarding […]