Saturday, June 30, 2018

More States Opting for Robo Grading of Student Essays.

This article from NPR on the use of software to grade student papers refers to a trend that is so appalling, I almost don't know where to begin. I was for a decade the person responsible for one of the standardized test programs in Alberta, and we were very proud of our involvement of classroom teachers at every stage of the development and marking process so that the exams would reflect actual classroom practice and the results would be normed accurately across Alberta. We trained teachers to use rubrics to mark accurately and consistently so that it wouldn't matter which marker the student got or what time of day the marking happened, or other chance factors. We did not train them to be robots, but designed rubrics that allowed them to award excellence, even when the student's answer did not easily fit within the rubric. If more than a few answers didn't fit the rubric, we changed the rubric. Routinely.

In social studies, we eliminated bias by making it explicit: when a marker found himself grinding his teeth over a student's stated opinions, instead of trying to grade it himself, he would hold the paper up above his head and shout "right-wing nut-job" or "left-wing snowflake" (as the case applied) and a suitably left or right oriented marker would happily swap it for its opposite, so that every paper was marked by a sympathetic marker. When a paper managed to offend everyone, it was taken off the marking floor and sent to a special committee of veteran markers who would grade it as a team. We went out of our way to not be robots.

This is NOT the case with many standardized tests, particularly in the US where it is private publishers rather than the ministry of education designing the exams. There are so many things wrong with most standardized testing programs, I will limit myself in this post to the observation that the purpose of most standardized testing programs is not to educate, nor to reward diligent learning, but to reproduce the class structure. Most tests reward the culture of the white upper-middle-class, and screw everybody else. They are there to explain inequality by saying, "Well you had your chance, but you only got a 58% on your test, so you deserve to spend the rest of your life in the underclass, unlike Frank here, who got 98%"(because we asked Frank questions in a way that makes the most sense to most white males, about stuff that matters to the professional/management class that Frank grew up in and had picked up on from his professional parents by the time he was 8, and because Frank had money for tutors, books, computers, if he happened to turn out a bit 'slow').

So...replacing teacher-markers with robots is perfect. Relying on software APPEARS to increase objectivity by removing the last vestiges of human intervention (what will be identified as 'bias' as these programs are being implemented) and keeps conscientious teachers from giving an "A" to a paper for its ideas when the student has written "ain't never" which is grammatically incorrect--i.e., has written her answer in the perfectly clear dialect of a to-be-suppressed population. A good teacher knows quality writing when they see it; a software program has the algorithms to suppress the underclass.

Not why I became an educator.

Monday, May 7, 2018

Homework Assignments

A clear explanation of why homework is stupid and why standardized testing has problems.

I have always opposed homework. Teachers should teach, parents should be family. Asking kids to take work home in K-12 system is wrong-headed because:

Homework Significantly Undermines Equality of Educational Opportunity

Homework greatly increases the disparity between family backgrounds, undermining equal educational opportunity

  • Some parents work at two jobs to support their families, or work evenings or otherwise are unavailable to help their kids with homework. It is clearly unfair to pit these kids against kids whose parents can do their homework for with them.
  • Some families speak white-middle-class English; some families speak lower-class English; some black dialect; others speak little or no English at. It is unfair to send home assignments in the expectation that all parents can help their children equally.
  • Some families need the children to do chores to help with the farm or family business; or disabled siblings, or elder care; or other responsibilites; while other families place no such demands on their children.
  • Some parents value education and school work; others—having been traumatized by their own school experiences with racial, class, religious, or other biases, and bullying teachers—think, for example of residential schools, to take just one obvious example—do not. It is not fair to demand that kids from these different backgrounds be pitted against each other on homework assignments.
  • Some homes have resources—computers, books, a table/space to work at, separate bedrooms for each kid—while others do not.
  • Some homes have the income to hire tutors when kids struggle; others do not.
  • Homes vary (across all classes, cultures, and backgrounds) in stability. Why are we asking kids to complete work at home when their parents are in the middle of a screaming divorce, or they have bullying siblings, or there are other types of disruption?
Schools are supposed to be safe places for students to learn. Our responsibility is to teach them in schools where we can control the learning environment and make sure it represents a level playing field. Any time we send work home, we privilege some students, undermine others.

Homework Undermines Teacher Accountability

As a parent, it frequently drove me crazy to find that I was expected to teach my child reading, writing, and arithmetic, while my child's teacher devoted hours of classroom time to non-academic activities. For example, I routinely experienced that whenever the family sat down to enjoy a family movie night on the weekend, that my daughters had already seen the film at school. "Thursday afternoons are movie-day," they would tell me. Why do my kids have two hours of homework each evening, but teachers have time to pre-empt family activities? What's wrong with this picture?
I get that there should be some 'fun' times at school, to team-build, to reward diligence, to enhance learning. But what I am increasingly observing is these activities displacing learning. I increasingly see a generation of teachers who are more concerned with occupying the children than teaching them.

To cite just one blatant example from my child's education: my wife was present in our daughter's classroom as a volunteer when, in the last week of school, the teacher announced, "Oh, by the way, I forgot to teach printing this year. I'm going to hand out the printing exercise books now for you to take home. Have your parents teach you printing over the summer because you'll need that for next grade." An extreme example to be sure, but underlying every homework assignment is the delegation of teacher-work to parents, who may or may not be in a position to take on the responsibility.

Homework Encourages Racism, Class Bias, and Victim-Blaming

I hear teachers—and not just my inexperienced student teachers, but senior classroom teachers—dismiss this or that student's lack of progress by blaming the parents. "What can you expect? Ruby comes from ________(fill in the blank with "broken home" or "native home" or "trailer park home" or whatever other unacceptable bias teachers allow themselves. There is no clearer example of blaming the victim than the lowering of expectations for students based on class, race/ethnicity, parental marital status or whatever. What can I expect from Ruby? The same as every other student in the classroom because it is the teacher's job to get every single student to mastery of the subject content. There are no valid excuses for a student failing, except perhaps a doctor's diagnosis of massive brain injury, in which case they are probably not in your class anyway. Homework assignments mask teacher bias by shifting responsibility, along with a high proportion of the teacher's workload, to the family. Instead of taking responsibility for each child's learning, they accept responsibility only for those students whose parents have the social capital to do the teacher's job for them.

Bah, humbug.

Good teachers get the work done in their classrooms; they do not assign homework.

Tuesday, April 17, 2018

Grading True/False Questions

A student just told me that they had gotten a true-false wrong, but knew the answer. They had marked it "True" and then written in the margin, "It's true, but you made a typo: it should be x/y" (or whatever the math had been— I didn't understand the question well enough to know what she was talking about). So, as instructor what do you do with that?

Mark it correct of course. The correction demonstrates the knowledge we were testing for, and even if they missed the entire point of how True-False is supposed to work, we're ultimately trying to determine their knowledge of course objectives, not their understanding of test-format. So that's just funny, but deserves a mark.

Indeed, I would argue that all True-False should work that way. Good T/F design* includes students explaining why the F options are false. That not only catches the sort of problem above, but ensures that students choosing F aren't just guessing, and that they know the right answer. There may also be more than one way to make a false statement true...that's all okay, as long as it demonstrates the students knowledge (or lack thereof).

T/F Ice freezes at 0 degrees C

That's true; but 'false' and adding "at sea level" is also correct and tells you even more about the student's knowledge. Both answers should get the mark.


*Okay, that's an oxymoron: almost all T/F are useless and to be avoided, but there are a few special circumstances—memorization of definitions, names, or dates, as if any of that is ever useful—where they can be used successfully.

Saturday, March 10, 2018

Found on Twitter

Found on Dr. Alex Couros (@courosa March 10, 2018) Twitter feed with the comment "Well played".

I deeply appreciate any instructor who grades the answer as presented rather than only the expected replies. Full marks here for both the student and the professor!

Friday, March 9, 2018

Teaching Excellence

Only slightly off topic of student assessment is the question of how one evaluates good teaching. Just as with student assessment, defining one's objectives and how to measure progress toward those goals is in fact a highly political activity. Why this learning objective and not that one; why is this particular standard considered inadequate, satisfactory, or excellent; why this group's (students? peers? parents? administrators? stakeholders? taxpayers?) perceptions of quality rather than some others? Do we measure excellent teaching against student expectation, student learning, student engagement, student enjoyment, student self-fulfillment; or by employer needs and expectations, graduate employment figures, graduate life chances; by political socialization or active citizenship; critical thinking or ideological conformity; or societal arts and culture, inventiveness, entrepreneurialism, the reproduction or elimination of poverty and injustice... You get the idea.

I'm very pleased to have "Excellence for what? Policy Development and the Discourse of the Purpose of Higher Education," appear as a chapter in the just-released Routledge collection, Global Perspectives on Teaching Excellence. The collection is basically a reaction to recent legislation in the UK that attempted to measure and mandate teaching excellence in higher education. My wife and I wrote a critique using my discourse analysis model of the purpose of higher education applied to the new legislation to suggest that the government's definition of 'excellence' might be somewhat problematic from the perspective of students and learning.

Monday, January 22, 2018

Dyslexia and assessment.

Here is a good video on the experience of dyslexia from the student perspective.

I'm often shocked by how poorly many teachers adapt assessments to deal with disabilities, such as dyslexia. I hosted a panel on "Dyslexia, dysgraphia and the writing experience" at a writer's conference in August, and the room was filled with parents openly weeping over how their dyslexic child had been treated in the school system. The parents were immensely relieved to see a panel of writers and editors talking about how they had managed to not just overcome/deal with their disability, but actually become successful readers, writers and editors in spite of what happened in their schooling.

I keep running into teachers who don't know what dyslexia is or how to deal with it, even though research suggests it affects about 15% of the population. They're always surprised when I tell them that the reason this child can't spell is that they are dyslexic or dysgraphic.

"I just thought they were being lazy,not making the effort to learn how to spell the word."

Well, "lazy" is better than "stupid", I suppose, which is how the peer group labels poor readers/spellers.

"Why are you making this child read out loud in front of their peers? What is the purpose of this public humiliation?" I will ask them.

"Well, you've got to read out loud. It's in the curriculum."

"Okay, but have them do that at recess or after class in private. Assessments are supposed to be confidential."

"Really? But this course objective says, 'to an audience'".

"Okay, but then give them the reading ahead of time so they can decipher it at home, learn it as a recitation if necessary, so they are able to be successful. Do not make them do a cold reading where so you set them up for a public failure."

"I don't know. That seems to be giving them an unfair advantage, somehow."

And so on, endlessly. No variation from the routine, the easy; no sense that accommodations are necessary to level the playing field. Often hints that the teacher secretly believes the child really is stupid, because how hard is it to read this very elementary book? Most troublesome are the teachers that, having assigned the child to the lowest reading group (which is of no help because they are still dyslexic and now surrounded by peers who act out with various forms of antisocial behavior because they are all being subjected to the daily humiliation of being forced to attempt reading aloud; and the reading material is so far below grade level as to kill anyone's motivation to read) keep them in that group for social studies ("well, that's mostly reading too, right?") and math and art ("Well, they might as well stay in the same group all day, right? It's just easier for everyone") even though the research is conclusive that streaming isn't useful for any student, and in the long run, not even a benefit to the teacher since it leads to more behavior and control issues.

And then there was the teacher with the fill-in-the-blank worksheet. Read the chapter, fill in the (apparently random--not even key words) blanks. When I questioned why she was making this dyslexic child read and fill in the blanks when the student (a) couldn't read and (b) couldn't write, she conceded that that might be a problem.

"I'll give the child the answer sheet. That way they won't have to read the text or spell the words."

But then, why make them do it at all? What was the possible purpose, the learning outcome, of laborously transferring random words from one sheet to another, one letter at a time? How is that an authentic assessment? How exactly did that teach this student the content or the spelling, when they couldn't actually read the sheet?

"But what do you want me to do? How can I teach the course if they don't fill in the worksheet?"

If a student teacher had said that to me, I would have failed them on the spot. Worksheets are your only teaching strategy? You're done now! But it wasn't a student I was dealing with, so okay, how to explain that they had confused means and goals. The worksheet is supposed to help students achieve a learning outcome--but somewhere along the line, completing the sheet had become the goal for this teacher, a meaningless daily productivity that the teacher could point to to say, "my students are working and learning". But of course they weren't learning at all.

Well,not the content, anyway. They were learning that they hated that teacher's subject, that school is inherently boring, that work is tedious. If the purpose of schooling is to condition graduates to the tedium of the adult workplace, conditioning them to tolerate a 9-5 routine of pointless, alienating labour, then yeah, this teacher was doing just fine. But I don't think that's why we chose to become teachers.

And don't get me started on some teachers' insistence on teaching cursive, when no adult under thirty ever uses cursive writing; and dysgraphic kids literally can't do it. It is exactly like demanding the kid in a wheelchair compete in the 100 meter race or fail phys. ed., but somehow, nothing deters these adults from pointlessly torturing and humiliating children with these learning disabilities.

Schools as we know them were built around reading and writing. I think that's still important, but I'm confused why audiobooks and instructional video and the rest are not also considered valid if they help nonreaders achieve learning objectives. Why pride of place to just that one medium of print? Why steamroller over the 15% of kids who can't read or write by insisting that that is the only way to complete assignments? It's assessment abuse.

Find other ways to find out what the student knows and can do. That is the point of any assessment. If a student cannot do the default assignment, find some other way to assess their knowledge and skills. The assessment is the means to the goal, not the goal. It's that simple.