Thursday, July 30, 2015

Meme: Item Analysis

But sometimes the complaint is justified and the solution is item analysis before the exam is given back to identify questions that were too hard, not taught, off topic, or badly/confusingly written, and then eliminating those questions (except for those who got them correct) to produce a fair assessment...use anchor question to ensure standards across years.

Saturday, July 25, 2015

Meme: Setting Writing Assignments

Another example of a student far more creative than the assignment structure. Also an assignment that makes assumptions about student lives that may not be valid and therefore discriminate against those students. But mostly, if we give students lame stimuli, we should not be surprised when we get mediocre work. Good on this kid for rising about both issues.

Saturday, July 18, 2015

Meme: Item Writing

"Tell me what happened in your brain" got exactly the answer it deserved. You want clear answers, ask clear questions. (But really, what was the kid supposed to answer? What process is involved here that is so complex it requires an explanation?) What more diagnostic information was being sought? if the kid answers 14, then they know how to do this one. Stop wasting student's time and yours.

Sunday, July 12, 2015

Exam Cheats

Thanks to Barbara Bloemhof for drawing this brief video to our attention. And, as she notes, "remember that one country's technology is completely importable to any other country by anyone (of any nationality... is my point...) We will never win a game of "My cheating detection technology is better than your technology to cheat" - so the alternative is to win hearts and minds over to choosing integrity, through sound argument and assessments."

(or here

As a former test development specialist, I can attest that the provincial exams faced all sorts of technological cheating. It took us a bit to notice that many kids travelling overseas to visit families for the holidays would bring back, say, calculators that were three or four years more advanced than those available in Canada. For example, we didn't know there were such things as graphing calculators until after one was confiscated from a student during provincials. I'm not convinced that was even cheating since no one had said you couldn't use a model x100 calculator or whatever. The initial response was to simply ban all calculators, but of course that's stupid. We can't pretend that technology isn't out there and in use on a daily basis so it is obviously stupid to make students work out by hand problems that are never worked out by hand in real world. Better we design better tests (better problems) that actually assess whether students know what they are doing and then give them ALL graphing calculators. To take just that one example.

My favorite example, though, was the kid they caught with an early (this was 20 years ago!) precursor to Google Glass. Kid is idly tapping his fingers on the desk while thinking, and examiner is on his way over to ask him to stop that because it is distracting to others, then realizes that there is a pattern to the taps-- kid is projecting virtual keyboard onto the desk, and typing questions and getting answers via glasses. Very alert examiner to catch that way back then, though (as the video mentions in passing) such "cheater spectacles" fairly routinely available these days.

The solution, of course, is to set assessments for which cheating is not helpful. Although one colleague did have one student who downloaded entries from the internet for her 'personal diary' assignment, that is a lot rarer. (And more obvious--really, you grew up on a fishing boat? I did not know that. Neither did your parents, apparently.) As long as assessment is focused on regurgitating facts, then access to the internet/cheatsheets/accomplice is helpful; but once one moves to more authentic assessments, not so much. Defeat cheaters by making it an open book exam; or drop exams and ask them to complete assessments that are personally meaningful, and that generate meaningful feedback, not just a grade. Have multiple assessments, not just one high stakes test that makes cheating worthwhile, even 'necessary'. If assessments serve as a means to an end (i.e., to help people learn) rather than as the end in itself (e.g., teaching and learning to the test, just to get a grade) then the problem of cheating just goes away.

On a related note:

(March 21, 2015 BBC) Photo shows parents/friends climbing walls outside Bihar school Inda to hand cheatsheets to students writing finals inside. "3 to 4 people helping a single student...a total of six to seven million people helping students cheat," he said. "Is it the responsibility of the government alone to manage such a huge number of conduct a 100% free and fair examination?" Well, um, yeah.

Saturday, July 11, 2015

Meme: Punctuality

Although punctuality is important, if someone is habitually late, it may mean they are caught in structural problems not of their making--day care dropoff opens ten minutes before class, the bus runs late, classes are scheduled with too little time between for distances between classrooms. (Lowering grades for late or absent is inaccurate assessment and inappropriate unless an official course objective.)

Monday, July 6, 2015

Taking Student Questions

An important and often overlooked component of good assessment practice is monitoring and responding to the class' reactions while one teaches. Nothing is more frustrating to students than having a question that an instructor will not pause to answer, and nothing is more useful to instructors than knowing if the students are 'getting it' while one is still teaching the concept, rather than weeks later on an exam (when it is too late to easily correct misunderstandings). Successful instructors are the ones who watch for signs they may be losing the class' attention or comprehension, and immediately adapting the lesson to get it back. Asking students questions is a good way to get a sense of their level of understanding, but one must always be prepared to take questions from students as soon as they come up. If a student misses some point early on, their confusion may quickly spiral out of control, whereas a simple and timely clarification can resolve such difficulties.

Not that I have always followed my own advice.

Possibly my worst teaching moment was when a student in the front row started searching through her purse, eventually pulling out a pen and yellow sticky note. I found her lack of attention distracting but managed to carry on lecturing...but then she scribbled a note on the sticky pad and pushed it towards me. I said something like, "Just hold your questions for a moment, until I get through the next few slides" but she persisted in pushing the note at me, even lifting it off the desk and waving it a bit. Highly annoyed at her impatience, I snatched the note and read out, "Your fly is open."

Moral: It really pays to take student questions as soon as possible.

Saturday, July 4, 2015

Meme: Bias

Good example of how assessment can overlook cultural differences...check your biases at the door, make sure your grading is evaluating student knowledge of official curricular objectives, not your own assumptions.