Test-Prep Manifesto

A while ago, I got an email from a person who grades essays for the SAT. One of the first things in the email was this:

First of all, you didn’t hear any of this from me. Pearson [the company that grades the SAT essays] has a thing about talking to the media, and I think most of this is fairly general, but PLEASE don’t quote me.

So I won’t reveal the person’s name or gender, or even the date the email was written. But there are things in this email the world should see. It was pretty long, so I’m only including the two most important parts:

. . . I agree with your points - I tell [people] to make up stories if needed, make up dates, names, whatever. As readers we can’t deduct for wrong info, and personally if someone told me the Treaty of Ghent was signed in 1385 or 1762 or whenever it actually was signed, I wouldn’t have a clue if it was right or not, but as a reader I do notice that detail . . .

First, you can see from this excerpt that the SAT Essay doesn’t penalize you for making up your examples. Of course, in real life, the examples you use in your writing have to be true. The College Board is knowingly giving students a “writing test” that doesn’t test the skills used in good writing.

Now, maybe that’s a big deal, and maybe it’s not. But we’re just getting started.

Take a look at this last quote:

It’s impossible to grade by looking at the guidelines [provided by the College Board]. . .

The grader says it’s impossible to use the provided scoring guidelines to grade an essay. Now that’s definitely a big deal–the grader is saying, point-blank, that the scoring guidelines given to students and teachers to prepare for the SAT essay aren’t actually used by graders. (Of course, if you’ve read my test prep white paper you already know this–it’s explained on page 10. Still, it’s good to hear it from an actual grader.)

Now take a look at the first excerpt again. That’s the worst part of the whole email–the grader says that Pearson Education, the company that grades the SAT essays, doesn’t want its graders talking. (Pearson Education also administers the GMAT and a ton of K-12 tests, and calls itself “the world’s largest integrated education company.”) Why would that be? There’s no way to know for sure.

So this email tells us three things that should make you pause:

1. The SAT Essay deliberately chooses not to test real writing skills.
2. The graders don’t use the grading criteria provided to students.
3. The company that grades the essays doesn’t want you to know how they do it.

Frankly, all of that ticks me off a little bit. And this situation isn’t limited to the SAT Essay. Every major standardized test has its problems, and most of them are hidden from you by the testing companies. Here are a few more brief examples:

• The GRE CAT shows you different types of questions depending on your performance, so students who score high aren’t tested on the same things as students who score low.
• Knowing how to guess on the LSAT can usually raise your score 5 percentile points or more.
• The writing assessment on the GMAT is partially scored by a computer.

But that’s only the beginning. Colleges and universities use standardized tests to rank your application, so you’d think they’d have no problem getting ranked themselves. And you’d be dead wrong.

U.S. News and World Report publishes annual rankings of all kinds of programs, from undergraduate specialties to professional schools. Even though most programs employ a similar approach to rank their applicants, they seem to hate when somebody does it to them. For example, the heads of almost every accredited law school signed an open letter to law school applicants entitled “Deans Speak Out” claiming that

“the arbitrary weighting of numerical factors makes ranking systems an unreliable guide to the differences among law schools that should be important to you.”

In other words, it’s fine for schools to use questionable numeric indicators to rank you, but it’s not fine for somebody else to come along and rank the schools.

There’s more. Most testing companies don’t just stop at making money off your testing fees–they actually sell their own test prep advice as well, to try to get you coming and going. And the worst part of this whole arrangement is that the testing advice they sell isn’t very good. (You should ALWAYS use the testing companies’ materials for test questions, but in general it’s not a very good idea to trust their test-taking advice.) You can learn a little more about this in our test-prep white paper.

And the big national test-prep companies aren’t much better. They charge you ridiculous rates for bad advice that takes a long time to learn. (Again, see the test-prep white paper for examples.)

So let’s see where that leaves you:

• Testing companies do a bad job and seem to know it. And they don’t want you to find out.
• Schools know that numerical ranking standards aren’t very good tools, but still judge you by them.
• You can’t get very good advice from the testing companies.
• You get even worse advice from most test-prep companies, and it’s expensive.

If you’re one of those people who’s really good at standardized tests, like I am, then this actually puts you in a nice position: You can sail through the tests that give most other students a hard time and make your application stand out pretty easily.

But if you’re like most test-takers, you’re in a very frightening situation. You have to do well on a flawed test because schools are going to weigh your numerical performance even though they know it’s not a good indication of what you have to offer. You can’t turn to the testing companies themselves for help, and you can’t turn to large companies for good advice either.

To put it simply, the state of standardized testing in America is out of control.

Don’t get me wrong–colleges and universities need ways to rank their applicants and evaluate their fitness for a program. I’m not saying that everyone should be allowed into any program they want. I’m just saying that the tools we use to rank students today are, in a word, awful.

I’m not the only person who thinks this, as you probably know, but I think I’m the only one who’s come up with the solution I’ve found.

There are web sites like fairtest.org that do a good job uncovering all the problems with the standardized testing industry. There are Presidents and Deans of universities who complain about standardized tests in the media. Neither of these approaches has met with much success.

My approach, unlike theirs, is market-based: I want to make it so easy for you to improve your test scores that standardized test scores eventually become meaningless.

After that, colleges will either pressure testing companies to make better tests (which would be a slight improvement), or they’ll finally move on to things like authenticated personal statements, interviews, and verified resumes to make their admissions decisions (which would be ideal).

So affordable, effective test prep for everybody is the short-term goal, and a shift to a more personalized admissions process is the long-term goal.

Towards that end, here are four promises from me to you:

1. I’ll only give you advice that I’ve personally used to beat tests.
2. My students will have access to personal, direct help from me.
3. Grammatix will offer a no-questions-asked refund policy within 48 hours of receipt of your materials.
4. Grammatix won’t charge an arm and a leg. In order to reach our goal, everyone has to be able to afford our help.

Let the revolution begin.

>>Now you know where we stand.