Why Teach Students to be Testwise (Testwiseness Part Two)

Some instructors may be horrified at the suggestion that we teach students test-taking strategies, let alone testwise strategies. Allow me to suggest why teaching students some testwise strategies is a good thing.

1. It levels the playing field

   Some students already know slogans like "when in doubt, choose c" even if they don't know the rationale behind it. If some students are testwise and others are not, it is not a level playing field. The assessment becomes inaccurate because the test is now measuring testwiseness rather than the skills and knowledge intended, since students who are testwise can significantly increase their scores (on poorly constructed tests). Instructors need to ensure their assessment is measuring learning outcomes not cultural capital. Teaching everyone the same basic testwise skills levels the playing field by ensuring everyone has the same knowledge.

2. It reduces test anxiety

   Students often do poorly in testing situations because of test anxiety. Students with test anxiety often feel they have no control over the many variables that make the outcome of the test uncertain: which subtopics will questions be drawn from, how hard each question will be, how long it will take them to remember and answer or solve for a question, the reading level of the question, the ambiguity of the question stem or of the answers offered, and so on. Having some strategies available for topics about which they know or recall little significantly reduces overall anxiety because it gives the student a sense of control. "Well, at least I know what to do if I have no clue what the answer is!" As part of a lesson on how to study for and take multiple-choice tests (two previous posts on this blog), giving students some testwise strategies goes a long way towards lowering (though never eliminating) test anxiety.

3. It forces teachers to write better multiple-choice tests

   Testwise strategies only work on poorly constructed tests, so if students know testwise strategies, teachers are forced to develop the test-construction skills, and to take the time necessary, to write the better tests necessary to defeat testwiseness.

Students must always be cautioned, however, that the only way to do well on tests is to learn the course material, and that they should not rely on testwise strategies. It is natural for some students (especially in middle school) to embrace testwise strategies as a substitute for studying, so it is important for instructors to note that these strategies will not work on one's own tests, because one's own tests are properly constructed.

Sadly, I have also encountered some instructors who take teaching test-taking strategies to the point of emphasizing these strategies over actual curriculum (i.e., teaching to the test). Such teachers are thinking like a middle school adolescent and should be relieved of their teaching certification.

Testwiseness (Part One)

The best way to do well on a test is to know the answers. However, when faced with a question you cannot answer, these tricks will sometimes work on poorly written multiple-choice tests. Only use them if you do not know the answer and have to guess.

1. When you have to complete a sentence, see if one of the answers fits better grammatically than the others.

   23. A dog is an
           A) animal
           B) machine
           C) mineral
           D) vegetable

   In this question, the “an” gives you a clue to the right answer, because you know the correct answer has to start with a vowel. “An animal” works, but it would have to be “a machine” or “a mineral” or “a vegetable” to be a correct sentence. Since the question says “a dog is an” rather than “a dog is a”, the answer has to be “animal” — because “animal” is the only one that fits with the “an” in the question.

2. See if one of the answers is stated in more "textbook language" than the others.
Instructors sometimes write questions by quoting a statement right out of the textbook, using the first half as the "question" and the second half as the correct "answer". Then they have to invent 3 wrong answers in their own words. Sometimes it is possible to identify which answers are in the textbook's style, and which in the instructor's.
3. See if one of the answers repeats key words from the question.

   43. An example of a nuclear reaction is
           A) hydration
           B) combustion
           C) sublimination
           D) nuclear fission

   Even if you do not know what the question means, the answer is probably (D) because it repeats the word “nuclear”, which is a key word in the question.

4. See if one of the answers is much longer than the others.

   The person making the test wants the right answer to be inarguably correct. To make it inarguably correct, they sometimes add more details to the correct answer, making it much longer than the others.

   45. The freezing point of water is
           A) 100 degrees centigrade
           B) 0 degrees Fahrenheit
           C) 0 degrees Kelvin
           D) 0 degrees centigrade for pure water at sea level

   (D) is the correct answer.

5. See if one of the answers sticks out as the “odd one out”.

   Since there is only one correct answer to a question, that answer has to be different from the other answers. Consequently, if you see two or three answers that all mean the same thing, they must be the wrong answers.

           A) Boy
           B) Girl
           C) Son
           D) Lad

   Since “Boy”, “Son”, and “Lad” all mean roughly the same thing, chances are the answer is “Girl”, even if we do not know what the question is.

   This trick is dangerous, however. Sometimes the “odd one out” can fool you. Maybe the answer was “Son” because it is the only answer about family relationships. But this trick might help if you are guessing blind anyway.

6. If you have to guess, and spot a typing error in one of the answers, choose one of the other answers.

   Sometimes the instructor making a multiple-choice test will proofread it by reading the question, looking at the right answer, and going on to the next question. They may forget to proof the wrong answers. So if there is an error, it is more likely to be a wrong answer. (This trick used to work better before spell-checking software became common, but not all instructors avail themselves of spell checking, and you can still check for homonyms.)

7. If you have to guess, eliminate answers with unconditional words like "always" or "never".

   In the real world, it is usually possible to find one or two exceptions to every rule, so answers with unconditional statements in them like "never" or "always" are usually the wrong answer (except in math or science where absolutes are possible). Similarly, for the same reason, weasel words like "approximately", "often", "usually", sometimes indicate a correct answer.

8. In questions where "all of the above" is used, see if you can eliminate one of the answers; or if you can identify two of the answers as correct.

   If two answers are correct, "all of the above" must be correct. If any one of the other answers is clearly wrong, "all of the above" must be wrong. By eliminating "all of the above" along with the wrong answer, you are now down to choosing out of two answers, and so have a 50/50 chance of getting the question right just by guessing.

9. In questions where "none of the above" is used, choose "none of the above".

   On tests where "none of the above" is only used occasionally, it is often the right answer. Even when it is not correct, it is often possible to argue for it by demonstrating some inadequacy in each of the other answers, thus making "none of the above" correct by default.

10. If you have absolutely no idea what the answer is, choose ‘(C)’.

    Instructors often try to “hide” the correct answer in the middle of the wrong answers. So they unconsciously choose (C) for the correct answer more often than any other letter. [A few instructors habitually choose B instead, If they are using a computer program or test bank, however, this may not work, since computers are usually programmed to randomize the placement of the correct answer.

Techniques for Taking Multiple-Choice Tests

1. Read the question and try to answer it BEFORE looking at the answers.

   Many students find it helpful to actually cover the answers with a bookmark so they cannot see the answers until after they have thought of an answer for themselves.

   1. The capital of Alberta is

   Read the question. You know the answer is “Edmonton”. Start by looking for “Edmonton” among the answers. By thinking of the answer first, you are less likely to be fooled by a wrong answer.

   1. The capital of Alberta is
         A) Ottawa
         B) Calgary
         C) Edmonton
         D) Lethbridge

   Sometimes, the answer you expected will not be there:

   1. The capital of Alberta is
         A) a
         B) b
         C) e
         D) l
   

   Look at the alternatives, and choose the one that answers the question, even it if is not quite what you expected. ("A" is the capital letter in Alberta.)

2. Make sure you look at ALL the answers before choosing:

   1. Edmonton can be described as Alberta’s
         A) capital city
         B) gateway to the north
         C) largest theatrical center
         D) all of the above
   

   Even though (A) and (B) are true, (D) is the correct answer. It is important to read all the answers, and not just take the first correct answer you see.

3. Plan ahead.

   If a test includes both multiple-choice and written response questions, divide your time based on the number of marks allocated to each section.

   If a test includes both multiple-choice and essay questions, read the essay question first, then answer the multiple-choice questions, then return to the essay. This allows your subconscious to work on the essay question while you are doing the multiple-choice questions, and will make it easier to start on the essay. The multiple-choice questions themselves might also provide clues, information, or ideas that you can use in answering the essay question.

4. Do not spend too much time on any one question.

   Some students end up rushing through questions at the end of the test because they spent too much time trying to answer one question early on. Remember that each multiple-choice question is worth the same number of marks, and so you should divide your time equally between all the multiple-choice questions.

   Sometimes a question will seem to have no right answer:

   2. The capital of Alberta is
         A) Red Deer
         B) Calgary
         C) Taber
         D) Lethbridge

   There may not be a right answer; the test writer may have made a mistake. It is important not to waste too much time trying to answer an impossible question. Choose an answer at random, but circle the question number so you can come back to it later if you have extra time. Go on to the next question.

   If a question is too hard, or you just do not know the answer, choose an answer at random and come back when you have completed all the questions you do know. Use whatever time is left over at the end of the test to tackle these time consuming questions.

5. Do not keep changing your answer.

   Sometimes a question will seem to have two right answers:

   1. Which of the following is a capital city?
         A) Ottawa
         B) Calgary
         C) Edmonton
         D) Lethbridge

   Choose the answer that seems best to you (Ottawa?) and move on to the next question. Do not keep changing your mind. Research shows that your first choice was probably the right one. Most people who change their answers will change from a correct one to a wrong answer. Only change your answer if you are absolutely sure you made a mistake. (For example, if another question on the test suddenly reminds you of the right answer.)

6. If the question asks you something you do not know, see if one of the other questions can provide a clue to the right answer.

   It is often possible to remind yourself of an answer by remembering the rest of the lesson in which that concept was taught. Since tests often closely follow the order in which the content was taught, it is sometimes possible to remind yourself of the answer by looking at the neighbouring questions.

   Sometimes questions will provide direct hints about other questions. For example, question 10 might be "In what year did the Japanese bomb Pearl Harbor?" and question 15 might be, "What advance warning did the Americans have of the Dec 1941 bombing of Pearl Harbor?" Question 15 then provides the likely answer to question 10.

7. If the question asks you something you do not know, see if you can get rid of any of the wrong answers before you guess:

   3. The capital of Alabama is
         A) Montgomery
         B) Birmingham
         C) Edmonton
         D) Ottawa

   You may not know anything about Alabama, but you do know that Edmonton and Ottawa are Canadian cities, and so are almost certainly wrong answers for this question. By crossing out Edmonton and Ottawa, you only have to guess between (A) or (B). That means you have a 50/50 chance of getting it right, just by guessing.

   [The answer is (A). Birmingham is the largest city in Alabama, but Montgomery is the capital.]

   By carefully eliminating answers you know are wrong, you can increase your chances of guessing correctly.

   Imagine that you only know the answers to half the questions on a test. Normally, your score would be 50%. But if you can eliminate one wrong answer for each question you are unsure about, your score would be 66%. If you can eliminate two wrong answers for each question you are unsure about, you could raise your score to 75%.

8. After you have finished the test, go back to those questions you circled as being too hard or having no right answer.

   See if you can answer them now. Take as much time as you have. Never leave a test early, unless you are sure you have answered every question correctly.

   If you still cannot answer the question, then guess. You have a 25% chance of getting it right anyway (on a four-choice multiple-choice question); more if you can eliminate one or more of the wrong answers. Never leave a blank on a multiple-choice test.

Studying for Tests

1. Begin studying for a test by reviewing the course outline.
• Check to see if the instructor has identified course goals. These will tell you what the instructor thinks the course is about. It is obviously helpful to know what the instructor believes are the important skills and concepts to be learned in the course, since these are likely to be the focus of the examination. (Where these do not closely match, one has good grounds for a grade appeal.)
• Check the course outline for the list of required and optional readings for the portion of the course covered in the current test, and ensure you are familiar with these materials.
• Check the course outline for web pages/Blackboard/WebCT/Moodle, podcasts, videos or other on-line materials for which you are responsible. Do not assume these will have been mentioned in class.
• Check to see if the instructor has identified how the examination will be marked. Some instructors include scoring rubrics for written response questions within the course outline.
• A few instructors may include essay examination questions in their course outlines; if they do, these should guide your note taking and course reading throughout the semester.
2. Check your own notes to see if the instructor has identified particular concepts or skills as likely candidates for the test.
   • Always highlight in your notes any time an instructor says, “This will be on your test”. Always record any discussion of the examination or review of sample questions by the instructor.
   • Always ask your instructor what will be on the test; they will often tell you.
   • Ask to borrow notes from your peers for any classes you missed in case those were the classes in which the instructor discussed the upcoming examination, or reviewed key concepts and skills.
3. Review the textbook and other required readings for the course.
   • Read the introduction and conclusion of each chapter to remind yourself what they are about. Quickly skim the introductory and concluding paragraphs of each section of the chapter; skim topic sentences of each paragraph of sections that you do not recall immediately. (This assumes you have actually read the material during the course.)
   • Use the table of contents and the index to identify key pages in the text or reading and concentrate on those sections most relevant to the course goals/topics, rather than re-reading the entire text word-for-word.
4. Predict what will be on the test by asking yourself what you would put on a test to examine the knowledge and skills required in this portion of the course.
   • Some students find it useful to study with one or more peers so that they may quiz each other to identify strengths and weaknesses to more finely tune what they need to study further. For discussion courses, it is often useful to debate topics identified in the course outlines with peers prior to an examination to help identify relevant arguments and to strengthen one’s own position.
   • If the course includes another assignment that addresses some of the skills/knowledge taught in this portion of the course, these aspects may be downplayed on the examination; the examination is likely to emphasize those skills and concepts not already covered in the other assignment.

Great Examples of Bad Test Questions

[Reprint of a post from another blog that maybe fits here better...]

As an evaluation instructor, alumni often send me examples of atrocious examinations they encounter in their subsequent courses. (Most of our alumni go on to a second undergraduate degree or graduate work.) Graduates of my exam-making course generally do very well on other people's exams, because once one knows how to write good examination questions oneself, it is easy to spot the errors in poorly written questions, and so be able to work out the answers. (See http://www.edu.uleth.ca/courses/ed3604/take/mc/how.html#Tricks for examples of what I mean.) On the other hand, my graduates also become hypercritical of sloppy exam-writing, and are often offended by the poor evaluation technique of otherwise excellent instructors. And thus my collection of really bad examinations continues to grow as alumni mail me the poor examples they encounter in other programs.

This week a former student sent me a wonderfully awful test from which I have drawn the following examples. (I won't, for obvious reasons, identify the campus that that student is now on, but suffice to say, this is from an experienced instructor at a legitimate North American university of some little repute, and not in any way an exceptional or unusual case.)

Mummification is first mentioned in 2nd Dynasty texts.
A) True
B) False
C) Maybe

Okay, ignore for the moment the embarrassment of a university instructor using true and false questions, how can one have a "maybe" category in a true/false items?! The whole point of true/false is that they address absolutes. The maybe category is invalid because a case can always be made for 'maybe' -- in this case, that there may well be other texts that have yet to be discovered. Since some questions on this test are true/false and others true/false/maybes, I would suspect "maybe" as the correct alternative anyway, since the instructor probably used it in those cases where there is some existing debate in the field (say an ambiguous reference in some earlier text that may or may not refer to mumification) but it doesn't really matter &emdash; given a "maybe" in a true false question, I can always justify it as the correct answer. It will always win any formal grade appeal.

Old Kingdom Kings did no trade with Asiatics.
a) Not True
b) Not False

The classic double negative question! I have been looking for one of these for years! All the test construction textbooks warn against the use of a double negative (negative in both stem and alternatives) but I have never actually seen a real example of one of these before. Evaluation nstructors have always had to make up our own examples, and students always say, "Oh, nobody would really do that, would they?" and now at last I have a real example.

"Despite his reputation as a tomb robber, Belzoni was nevertheless a fine archaeologist".
A) True
B) False

The archetypal "opinion" question, the ultimate taboo in true/false item writing. Again, I have been looking for an example like this for quite a while. True/false questions can only be used for testing absolutes, not opinions, since one can always make the case for the other side (however tenuously) and we do not score people on their opinions in a democracy. In a formal grade appeal, the student will always win.

The rest of the test is of similarly disappointing quality. Every campus has some kind of Teaching Development Center (or at least a Teaching Development Committee, if the campus is too small to afford dedicated staff) that sponsors 'how to' workshops on instruction and assessment techniques, but of course those that need the workshops are never the ones who attend.

The sad thing is that the former student who sent me this test had been absolutely raving to me about what a wonderful professor this was and what a great course and how much the student was enjoying the class, prior to the test. Afterwards, the student was so disappointed with being robbed of the opportunity to demonstrate the deep learning achieved in that class, that their enthusiasm was considerably eroded. The student still considers that professor an all time favorite, but then this is a student motivated by a thirst for knowledge rather than grades, and so perhaps more willing then most to forgive such tragic flaws.

Tricky Directions in Test Questions

This is another meme doing the rounds on the web:

I think we can take it as read that "Draw a dinosaur wearing spectacles and holding a pen" does not relate directly to course objectives. I am a somewhat open to the idea of demonstrating to students the need to (a) read what the question is asking carefully and (b) the importance of showing one's work in such examinations, but um...hoping this was for an ungraded assignment (in class or homework) not a question on an actual exam. As a practice exercise, one of these a semester is fine—a little levity goes a long way, and there is a point to be made. [Doing this more than once would be bad: once is about student learning; twice is about the instructor projecting their self-image as a cool teacher. I think it safe to assume the current example is a one-off by a superior instructor, not role modelling a routine to be adopted by other, less grounded instructors.] So, no harm done if used sparingly and only in a non-graded context.

As an actual exam question, humour is a no-no. Because the examination is supposed to be targetting legitimate curricular objectives: unless your science curriculum includes "demonstrates a sense of humour" or "fine motor skills in the production of art" questions such as this example are out of line. Further, although the instructor may believe the injection of humour is a way to reduce stress, the fact is that at least some students will be panicked by it. Knowing in their hearts that an examination is no place for levity, their reaction to encountering it is likely to be disbelief and the assumption that they have misunderstood the question. In this instance, they will waste precious seconds obsessing about the meaning of the dinosaur. How does the height of the dinosaur change the question? Is the question now 32 meter's height, plus the height of the dinosaur? What is the significance of the pen? the Spectacles? Such superfluous information can drive a student crazy. And that's for normal kids. Throw in say, OCD or second language issues, and one is screwing up the accuracy of the assessment, not only for that question, but all the other objectives being examined that had that fraction of time less to think about.

So, occasional funny questions on practice exercises, maybe. Understand that some students will share your sense of humour and respond well, but others will not. But that can be addressed in class if the class atmosphere is a positive and safe one. On an actual for grades test or assignment, never.

Silly Questions

Teachers like to say, "There is no such thing as a silly question." This is a facile and pointless thing to say to students, because they know perfectly well that there are, and that everyone will laugh at them if they say something stupid out loud. Or, more accurately, that it sometimes feels like everyone is laughing at them. Denying that one can accidentally ask a stupid or wrong question just makes matters worse: not only are students faced with the risk of exposure, they can expect no support from someone who refuses to even acknowledge problem that such a problem exists. (Like my telling my daughter she can't live in residence at X university because their Dean of Students insists no student has ever been raped in residence. Since that is not remotely credible, I am not sending her to a school which operates on the basis of denial and cover up. I want her to go to a school that is addressing the problem face on, and which is going to offer its residents every conceivable support, not ask her to shut up about it lest it ruin the Dean's perfect reputation. Saying "there are not stupid questions" is like confabulating 'no reported rapes" with "no rapes": kids know what you're really saying is, "You have no grounds to complain about how I handle your questions because I will never acknowledge that I was being condescending...")

Students learn they can ask silly questions by the way the teacher models appropriate responses. If one resists the urge to say, "Really?" in condescending voice, then students will never know it was a stupid question and the issue will never arise. If other students are inclined to giggle, cut that short by saying, "I'm glad you asked me that because that is such a common point of confusion I have to remember to go over that at least a couple of times each term." And so on.

If one feels the need to announce that this particular class is intended as a safe environment, than something along the lines of "There are stupid questions, but that's okay, you are likely not the only one wondering about that topic/issue, so please have the courage to be the one to put that into words." Reframming from 'stupid' to 'courage' goes a long way to creating that positive classroom environment.

Further, even the stupidest question may not be stupid at all—it just came out wrong. Making fun of that slip of the tongue is NEVER a good idea. Even if you think the kid is 100% okay with such banter, chances are s/he isn't. One of my student teachers shared with the class how in grade 7 he was trying to ask about how long the war of 1812 lasted, but it came out "When was the War of 1812?" The teacher made a joke of it, and my student laughed right along with everyone else because, although thoroughly embarrassed, one has to be seen to be able to laugh at oneself, right? But the teacher didn't let go of it and would ask, "When was the treaty of 1615 Zac?" and so on every opportunity. Worse, other teachers in the school took it up, until it became a daily humiliation for the student. But he never said anything or allowed any of his teachers to suspect how he was responding to their jibes because he wanted to appear 'cool'. But he was not cool with it. It made that term hell for the student, and his grades nosedived. He stopped wanting to go to school. And his voice still quavered recalling that term a decade later when recounting it to my class.

It always amazes me how much rhetoric there is about taking a stand against bullying in the schools, when in fact it is the teachers who are often the worst bullies. Just saying.

Which brings us to the issue, what do we do with the student who asks deliberately stupid questions? You know the kid I mean, the wisenheimer in the back row who has figured out how to disrupt class by asking time-wasting obvious questions.

Step one: stop thinking of that kid as a wisenheimer. Again, you can't assume that the kid in the back isn't seriously asking, even if he is projecting a strong image of "I'm only asking this to be disruptive, because of course everyone knows that answer already, right?" Not so much. To be cool, kids have to pretend in front of their peers they know everything, and posing as a wisenheimer is a very effective strategy for them to be allowed to ask questions to which they really, really need the answer.

Step two: real or not, treat every question with the same dignity and give as serious a reply as you would to an honour or transfer student who really needs to know. If the student really wanted to know, they will be grateful; if they were being a jerk, well, treating their answer as serious defeats the purpose of derailing the class. "I'm glad you asked that Roldand! Because that's the key to it all!" (Or "that's more complicated than one might think!" or "I can't emphasize / go over this enough times" or etc) Suddenly they have been cast in the role of legitimate helpful student. There may be a brief period of confusion where they try to figure out whether you are really that gullible or whether you are playing them but either way, extinguishes the (apparently) negative attitudes and the next thing you know, they are fully participating in a class where someone is suddenly taking their input seriously--whether they started out that way or not.

Finally, there is the not-as-silly-as-you-assumed question. If a student asks a stupid question, respectfully poke around a bit by asking why they are asking to ensure you understood the actual question. It may not be a silly question at all, but merely that you did not understand the context or significance of why they were asking. Think of Giant Pandas next time you get a stupid question in class....