Exam security

This is funny, but also not. At a certain university (NOT the one I taught at) the IT department invested a quarter of a million dollars on a new security system, touted as unbreakable. The instructor in an advanced graduate course in the computer science department made that week's assignment to break the security system, thinking it would be humbling for the students to encounter at least one assignment that was not doable. My friend, a computer genuis, broke in to the supposedly secure system in about an hour, and posted his "how to get around security" assignment on the [open] class website, as per the assignment directions. By the end of the evening, every computer geek on campus (and beyond) had a copy, and the new security system was thus rendered useless. My friend was summoned before the Dean, but simply brought a copy of the assignment, complete with directions to load the completed solution on the open class site, and was pretty much off the hook (though he had to listen to a long lecture on 'judgement' and 'taking initiative against security breaches rather than creating them" and so on). What happened to the prof, I was never told. The morals here are: (1) don't be a wisenhimmer and assign criminal acts to your students, even if you think it impossible or that they will get that it's a joke assignment, (2) don't assume your students aren't way smarter than you are and can't do something just because you can't.

Found on Twitter

Found on Dr. Alex Couros (@courosa March 10, 2018) Twitter feed with the comment "Well played".

I deeply appreciate any instructor who grades the answer as presented rather than only the expected replies. Full marks here for both the student and the professor!

Bringing Notes to the Examination

A meme downloaded from "S**t Academics Say" Oct17, 2017:

There are several points to be made here.

First, good on the examiner for recognizing that was on them, not the student. I detest when an instructor makes a mistake and then tells a student, "well, that's not what I meant!" Unless mind reading is criteria in your psychic examination, it is inappropriate to hold students accountable for the instructor's intentions. If they got the literal answer to one's poorly worded question or followed one's poorly worded instructions literally, they get the mark or behaviour.

Second, you have to love this student! That is out of the box thinking! Exactly what every discipline needs more of.

Third, open book examinations are actually harder. Okay, if it's closed book for everyone else and open book for you, that does give one a slight advantage, but not as much as most people assume. There is relatively little knowledge that we need to memorize, as opposed to understanding and having available to us on our computers or google or etc. In most disciplines, one ends up memorize the important facts because that's easier than looking them up every time. If the fact is one you have to look up, it is by definition one that did not come up enough to be worth memorizing. Exams should be testing understanding, not just rote memorization. If the exam is testing higher orders of cognition (analysis, synthesis, evaluation and so on) then making information available to students makes for a better test of skills than the usual rote memorization that many exams seem to be geared towards.

Plagiarism: Two Unsuccessful Examples

In a recent discussion of plagiarism with some editors, I was reminded of an example from my own teaching experience that I now relate here for your amusement.

As a new prof I had the singular experience of an undergraduate handing in a paper to the course I was teaching on the sociology of education that, as it turned out, I had written when I had taken the same course a decade or so before. Unbeknownst to me, the paper must have been photocopied by a classmate of mine years before and entered into the frat files. The original title page must have long since been discarded by the frat boys as who knows how many times the paper had been recycled, so my student merrily retrieved it from the files, typed up a new cover page with his name, and handed it in to it's author. It took me a minute to recognize after all those years, but there was something about it that was strangely familiar. And as it occurred to me that it was in fact, my paper, I recognized that the typing had the cracked line through the 'e' that I had had on my typewriter that year. I actually went through the effort of digging through the garage to find the original paper, and yup, it was identical. So that was an 'F' for the student, and an anecdote for me.

The other example that I recall from roughly the same period was a colleague of mine telling of the case of a seminary student in his PhD defense. The originally scheduled external examiner had called in sick, so the supervisor had located a replacement. "You're in luck," he had told the student, "Dr X from the University of Y in Germany is in Calgary for a conference and has agreed to fly up to sit in as your external. It's a great honor because he is apparently quite an expert on your topic." But of course, turns out he really was an expert because when he opened the student's dissertation, he discovered at once it was his dissertation translated from the German. Since one does not get a PhD in theology for translating someone else's doctoral thesis, that was another FAIL. But I have to say, great example of the direct intervention of God against someone unworthy of the doctor of theology, because how else to explain the coincidence? The poor bastard must have thought himself completely safe from discover (this being decades before the internet).

Moral of the story: don't plagiarize or god will get you. .

Tricky Directions in Test Questions

This is another meme doing the rounds on the web:

I think we can take it as read that "Draw a dinosaur wearing spectacles and holding a pen" does not relate directly to course objectives. I am a somewhat open to the idea of demonstrating to students the need to (a) read what the question is asking carefully and (b) the importance of showing one's work in such examinations, but um...hoping this was for an ungraded assignment (in class or homework) not a question on an actual exam. As a practice exercise, one of these a semester is fine—a little levity goes a long way, and there is a point to be made. [Doing this more than once would be bad: once is about student learning; twice is about the instructor projecting their self-image as a cool teacher. I think it safe to assume the current example is a one-off by a superior instructor, not role modelling a routine to be adopted by other, less grounded instructors.] So, no harm done if used sparingly and only in a non-graded context.

As an actual exam question, humour is a no-no. Because the examination is supposed to be targetting legitimate curricular objectives: unless your science curriculum includes "demonstrates a sense of humour" or "fine motor skills in the production of art" questions such as this example are out of line. Further, although the instructor may believe the injection of humour is a way to reduce stress, the fact is that at least some students will be panicked by it. Knowing in their hearts that an examination is no place for levity, their reaction to encountering it is likely to be disbelief and the assumption that they have misunderstood the question. In this instance, they will waste precious seconds obsessing about the meaning of the dinosaur. How does the height of the dinosaur change the question? Is the question now 32 meter's height, plus the height of the dinosaur? What is the significance of the pen? the Spectacles? Such superfluous information can drive a student crazy. And that's for normal kids. Throw in say, OCD or second language issues, and one is screwing up the accuracy of the assessment, not only for that question, but all the other objectives being examined that had that fraction of time less to think about.

So, occasional funny questions on practice exercises, maybe. Understand that some students will share your sense of humour and respond well, but others will not. But that can be addressed in class if the class atmosphere is a positive and safe one. On an actual for grades test or assignment, never.

Wrong answers....

My daughter was laughing her way through Richard Benson's "F in Exams: The Very Best Totally Wrong Test Answers", which is a pretty funny read, and asking me what I thought. Well, mostly I thought the answers were amusing, and the questions that illicited them poorly written. If you ask students, "Where was George Washington born" and they answer, "in a bed", you cannot mark it wrong. It correctly answers the question asked. If you want the name of the city, you have to ask "In what city was Washington born?" And so on. But half the answers in Benson's book are in fact wrong answers, but funny nevertheless, either because they are cheeky, or because they simply reveal an understandable misunderstanding. As it happens, I came across just such an example as I was marking my own exams at that moment. A student who meant to refer to the educational limitations of "rote memorization" wrote instead "rogue memorization" -- an unfortunate slip of the pen, or perhaps a genuine mishearing of what the student thought his/her profs have been saying. But I kind of like the phrase! Yeah, you really don't want your students dwelling on rogue memorization. Given the randomness of some of the answers to the questions posed on the exam, I greatly fear some of my class have in fact experienced rogue memorization when studying for the exam....