Text laundering
This morning I received this link on plagiarism software via the Columbia Journalism School mailing list – which is significantly more interesting than most faculty mailing lists, I might add.
In the article, the author, Neuroskeptic, describes the smallish bit of work one has to go through to “launder” text in order for the standard plagiarism detector software to deem it original. The example Neuroskeptic gives us is, ironically, from a research ethics program in Britain called PIE which Neuroskeptic is accusing of plagiarizing text:
PIE Original: You are invited to join the Publication Integrity and Ethics (herein referred to as PIE) as one of its founding members. PIE, a not-for profit organisation, offers free membership to all interested individuals. Please join us and become part of this exciting new movement in the world of publishing ethics; it is the professional home for authors, reviewers, editorial board members and editors-in-chief.
Neuroskeptic: You are invited to join Publication Integrity and Ethics (herein referred to as PIE) and become one of its founding members. PIE, a not-for profit organisation, offers interested individuals free membership. Please join this exciting new movement in the publishing ethics world; PIE is the professional home for reviewers, editorial board members, authors, and editors-in-chief.
This second, laundered piece of text got through software called Grammarly, and the first one didn’t.
Neuroskeptic made his or her point, and PIE has been adequately shamed into naming their sources. But I think the larger point is critical.
Namely, this is the problem with having standard software for things like plagiarism. If everyone uses the same tools, then anyone can launder their text sufficiently to jump through all the standard hoops and then be satisfied that they won’t get caught. You just keep running your text through the software, adding “the’s” and changing the beginning of sentences, until it comes out with a green light.
The rules aren’t that you can’t plagiarize, but instead that you can’t plagiarize without adequate laundering.
This reminds me of my previous post on essay correction software. As soon as you have a standard for that, or even a standard approach, you can automate writing essays that will get a good grade, by iteratively running a grading algorithm (or a series of grading algorithms) on your piece and adding big words or whatever until you get an “A” on all versions. You might need to input the topic and the length of the essay, but that’s it.
And if you think that someone smart enough to code this up deserves an A just for the effort, keep in mind that you can buy such software as well. So really it’s about who has money.
Far from believing in MOOC’s destroying the college campus and have everything online, in my cynical moments I’m starting to believe we’re going to have to talk to and test people face to face to be sure they aren’t using algorithms to cheat on tests.
Of course once our brains are directly wired into the web it won’t make a difference.
mathbabe 
Interesting. Ross Anderson wrote about this sort of attack as a general defeat of ‘public-key’ analogues of watermarking images/sound/movies, in his book “Security Engineering”. If you have a testing tool that a third party can run without needing to know your secret key, then an attacker can just make random changes until the test passes. He didn’t make the connection to plagiarism detection but it’s neat how the same idea works.
LikeLike
Reminds me of the old adage (I think it’s one of Murphy’s Laws?), that it’s impossible to make anything foolproof, because fools are so damn ingenious!
LikeLike
Turnitin is better known. However, faculty use it to detect student plagiarism, and students generally don’t get a second chance, so this doesn’t apply.
http://en.wikipedia.org/wiki/Turnitin
“…I’m starting to believe we’re going to have to talk to and test people face to face to be sure they aren’t using algorithms to cheat on tests.”
Yes. I have personal experience with how easy online cheating is, but schools and legislators keep pushing online courses as if cheating were not a problem.
It is worth remembering that disruptive technologies (C. Christensen) are defined as inferior to their predecessor-technology on at least one product-dimension, but reach broader markets. Online courses qualify under this definition as a disruptive technology.
LikeLike
There’s also this roaming out there too and health insurers are using software to analyze your voice. This is not new but getting to be more mainstream. So if you are talking, it’s not “this voice is recorded for quality purposes” it’s more like we are unleashing millions of algorithms to analyze your voice. Originally when the software came out it said it could tell by your voice over the phone if you had prostate cancer.
I kind of wondered how many false positives it had with diagnosing women:) Of course there are error factors, but I look at it this way, if you get scored, it’s one more piece of data that can be sold that adds to the data selling epidemic we have in the US.
http://ducknetweb.blogspot.com/2014/05/this-call-may-be-recorded-for-quality.html
So even if we talk, there’s algos chasing us everywhere.
LikeLike
I think the problem in academic situations isn’t that a particular piece of software isn’t 100% effective for an evaluation task. Humans doing these tasks are going to miss things as well.
We are notoriously bad at being consistent. It is how we deal with the particular errors that a particular method is prone to have that determines whether it is a good idea or not. Automated methods tend to be cheaper and the institutions who are starting to use them are turning a blind eye to their problems. Depending on the circumstances, combining multiple assessment systems (automated, trained evaluators, peer evaluators, random humans) could very well be preferable to any single system. Unfortunately, I don’t think this is likely to happen anytime soon.
LikeLike
This seems to raise some tensions. For example, normally we would want anything as important as grading software to be transparent, so that we can look out for built in biases, etc (as you’ve mentioned in the context of other tools). But if it is transparent then it is easy to game (if it is not transparent then it is just unreasonably expensive to game) in the way you are describing. You could try to get around this by building in private keys and randomizing parts of the algorithm based on those, but then you are admitting that your grading is in some important way randomized (because if there is high correlation in outcomes from different keys then you can just optimize with your own key and still expect a high grade with the prof’s key).
It seems like the only solution is to build good software, where the only way to game it is to achieve the outcome desired by the instructor, but then you’ve probably built not a grading software but a full teacher.
LikeLike
The fundamental problem of essay-grading software is not that it can be gamed in this manner, but that it is measuring the wrong things in the first place – current software uses simplistic measures of style that are wrongly taken as being proxies for the ability to understand and reason about a topic.
This problem is not limited to software: human graders of the SAT essay, for example, are so constrained in how they perform the task that they are effectively running the same algorithms as the software.
LikeLike
Talking to people in person? Testing students face-to-face?
Mathbabe, I think you’ve independently redevloped the “viva voce” …
LikeLike
“Viva voce”? I just ran across that phrase in the context of Roberts Rules of Order where it
refers to votes taken by voice. From context, I’m guessing that you mean something more
like evaluating oral presentations (or perhaps a thesis defense).
LikeLike
Thesis defense and oral exams. No questions are out-of-bounds! I’ve not heard of the term being used for … many years now.
LikeLike
I am constantly amazed and horrified, as I was all throughout my education, at what people will do to avoid getting an education. Essentially, this is likely to be less of a problem than we think because the students in question will lack the skill to edit a piece of writing effectively in the first place.
I think an excellent idea would be, for the tech literate professor, to insist on the kids completing a short written activity during the first class — during it, while the professor sits at the front and watches. Count the kids in the class, make sure you get that many papers. Ask them to write three paragraphs about their favorite topic, themselves, anything at all. But it has to be at least 3 paragraphs of 3 sentences each (or make it any length necessary to get a sufficient sample), and must have their name and the date at the top.
You could run that through a program and make sure that any subsequent pieces you get have hallmarks of being written by the same person. Of course, that wouldn’t work for an online course. But I wonder if it wouldn’t, in this day and age, be easy enough to pull someone’s existing work from other online courses and see if there is a voice there or if the pieces in question are disparate enough to clearly be from different authors.
Of course, you can buy anything. Absolutely everything in this culture is for sale, especially degrees. But apparently, eventually, those people are going to have to hire someone to write for them because they will be illiterate to an extent that becomes impractical.
LikeLike
With regard to plagiarism detection specifically, this is just the start of an arms race. As you point out, the next step will be the creation of paraphrasing software. The response of the plagiarism detectors will be to apply all the paraphrasing software they can get their hands on to the samples they are comparing, looking for any match. The response of the paraphrasing software writers might be to seek paraphrases that match multiple independent sources with equal probability…
By then, it will be impossible to have an original thought.
LikeLike