Exam security

This is funny, but also not. At a certain university (NOT the one I taught at) the IT department invested a quarter of a million dollars on a new security system, touted as unbreakable. The instructor in an advanced graduate course in the computer science department made that week's assignment to break the security system, thinking it would be humbling for the students to encounter at least one assignment that was not doable. My friend, a computer genuis, broke in to the supposedly secure system in about an hour, and posted his "how to get around security" assignment on the [open] class website, as per the assignment directions. By the end of the evening, every computer geek on campus (and beyond) had a copy, and the new security system was thus rendered useless. My friend was summoned before the Dean, but simply brought a copy of the assignment, complete with directions to load the completed solution on the open class site, and was pretty much off the hook (though he had to listen to a long lecture on 'judgement' and 'taking initiative against security breaches rather than creating them" and so on). What happened to the prof, I was never told. The morals here are: (1) don't be a wisenhimmer and assign criminal acts to your students, even if you think it impossible or that they will get that it's a joke assignment, (2) don't assume your students aren't way smarter than you are and can't do something just because you can't.

More States Opting for Robo Grading of Student Essays.

This article from NPR on the use of software to grade student papers refers to a trend that is so appalling, I almost don't know where to begin. I was for a decade the person responsible for one of the standardized test programs in Alberta, and we were very proud of our involvement of classroom teachers at every stage of the development and marking process so that the exams would reflect actual classroom practice and the results would be normed accurately across Alberta. We trained teachers to use rubrics to mark accurately and consistently so that it wouldn't matter which marker the student got or what time of day the marking happened, or other chance factors. We did not train them to be robots, but designed rubrics that allowed them to award excellence, even when the student's answer did not easily fit within the rubric. If more than a few answers didn't fit the rubric, we changed the rubric. Routinely.

In social studies, we eliminated bias by making it explicit: when a marker found himself grinding his teeth over a student's stated opinions, instead of trying to grade it himself, he would hold the paper up above his head and shout "right-wing nut-job" or "left-wing snowflake" (as the case applied) and a suitably left or right oriented marker would happily swap it for its opposite, so that every paper was marked by a sympathetic marker. When a paper managed to offend everyone, it was taken off the marking floor and sent to a special committee of veteran markers who would grade it as a team. We went out of our way to not be robots.

This is NOT the case with many standardized tests, particularly in the US where it is private publishers rather than the ministry of education designing the exams. There are so many things wrong with most standardized testing programs, I will limit myself in this post to the observation that the purpose of most standardized testing programs is not to educate, nor to reward diligent learning, but to reproduce the class structure. Most tests reward the culture of the white upper-middle-class, and screw everybody else. They are there to explain inequality by saying, "Well you had your chance, but you only got a 58% on your test, so you deserve to spend the rest of your life in the underclass, unlike Frank here, who got 98%"(because we asked Frank questions in a way that makes the most sense to most white males, about stuff that matters to the professional/management class that Frank grew up in and had picked up on from his professional parents by the time he was 8, and because Frank had money for tutors, books, computers, if he happened to turn out a bit 'slow').

So...replacing teacher-markers with robots is perfect. Relying on software APPEARS to increase objectivity by removing the last vestiges of human intervention (what will be identified as 'bias' as these programs are being implemented) and keeps conscientious teachers from giving an "A" to a paper for its ideas when the student has written "ain't never" which is grammatically incorrect--i.e., has written her answer in the perfectly clear dialect of a to-be-suppressed population. A good teacher knows quality writing when they see it; a software program has the algorithms to suppress the underclass.

Not why I became an educator.

Detecting Plagiarism Tip 2

A recent post by a colleague on an editing site made the point that

if text is copied and pasted into Word from e.g. Wikipedia, it can include non-breaking spaces (small raised circles instead of the usual dot seen with show/hide is turned on). These are practically impossible to type by mistake! Similarly, when text is copied and pasted from PDFs, odd line breaks/returns are imported—another give-away.

Thought I'd pass that along.

Exam Cheats

Thanks to Barbara Bloemhof for drawing this brief video to our attention. And, as she notes, "remember that one country's technology is completely importable to any other country by anyone (of any nationality... is my point...) We will never win a game of "My cheating detection technology is better than your technology to cheat" - so the alternative is to win hearts and minds over to choosing integrity, through sound argument and assessments."

https://www.youtube.com/watch?v=hLD-07tba1A&feature=youtu.be

(or here

As a former test development specialist, I can attest that the provincial exams faced all sorts of technological cheating. It took us a bit to notice that many kids travelling overseas to visit families for the holidays would bring back, say, calculators that were three or four years more advanced than those available in Canada. For example, we didn't know there were such things as graphing calculators until after one was confiscated from a student during provincials. I'm not convinced that was even cheating since no one had said you couldn't use a model x100 calculator or whatever. The initial response was to simply ban all calculators, but of course that's stupid. We can't pretend that technology isn't out there and in use on a daily basis so it is obviously stupid to make students work out by hand problems that are never worked out by hand in real world. Better we design better tests (better problems) that actually assess whether students know what they are doing and then give them ALL graphing calculators. To take just that one example.

My favorite example, though, was the kid they caught with an early (this was 20 years ago!) precursor to Google Glass. Kid is idly tapping his fingers on the desk while thinking, and examiner is on his way over to ask him to stop that because it is distracting to others, then realizes that there is a pattern to the taps-- kid is projecting virtual keyboard onto the desk, and typing questions and getting answers via glasses. Very alert examiner to catch that way back then, though (as the video mentions in passing) such "cheater spectacles" fairly routinely available these days.

The solution, of course, is to set assessments for which cheating is not helpful. Although one colleague did have one student who downloaded entries from the internet for her 'personal diary' assignment, that is a lot rarer. (And more obvious--really, you grew up on a fishing boat? I did not know that. Neither did your parents, apparently.) As long as assessment is focused on regurgitating facts, then access to the internet/cheatsheets/accomplice is helpful; but once one moves to more authentic assessments, not so much. Defeat cheaters by making it an open book exam; or drop exams and ask them to complete assessments that are personally meaningful, and that generate meaningful feedback, not just a grade. Have multiple assessments, not just one high stakes test that makes cheating worthwhile, even 'necessary'. If assessments serve as a means to an end (i.e., to help people learn) rather than as the end in itself (e.g., teaching and learning to the test, just to get a grade) then the problem of cheating just goes away.

---

On a related note:

(March 21, 2015 BBC) Photo shows parents/friends climbing walls outside Bihar school Inda to hand cheatsheets to students writing finals inside. "3 to 4 people helping a single student...a total of six to seven million people helping students cheat," he said. "Is it the responsibility of the government alone to manage such a huge number of people...to conduct a 100% free and fair examination?" Well, um, yeah.