Tonight I’m taking my adorable husband with me to accept an award from the Special Libraries Association of New York called the PrivCo Spotlight Award. Here’s the description of the award and their reasoning in choosing me, from their website:
This award celebrates website founders and bloggers, curators of distinctive collections, solo librarians, mentors and teachers, conference organizers, and librarians typically working outside the traditional scope of SLA-NY award consideration. As a data scientist and author of the “MathBabe” blog, we feel Cathy O’Neil strongly embodies the spirit of SLA NY. Her book Weapons of Math Destruction was published in 2016 and has been nominated for the 2016 National Book Award for Nonfiction.
I’m a huge fan of librarians – they are, in my opinion, the original ethical data scientists – so I’m both honored and psyched for the award and for the chance to meet the kind people at SLA-NY as well as the other award winners.
There’s an article that’s been forwarded to me by a bunch of people (I think first by Becky Jaffe) by Anya Kamanetz entitled How One University Used Big Data To Boost Graduation Rates.
The article centers on an algorithm being used by Georgia State University to identify students in danger of dropping out of school. Once identified, the school pairs those wobbly students with advisers to try to help them succeed. From the article:
A GPS alert doesn’t put a student on academic probation or trigger any automatic consequence. Instead, it’s the catalyst for a conversation.
The system prompted 51,000 in-person meetings between students and advisers in the past 12 months. That’s three or four times more than was happening before, when meetings were largely up to the students.
The real work was in those face-to-face encounters, as students made plans with their advisers to get extra tutoring help, take a summer class or maybe switch majors.
I wrote a recent book about powerful, secret, destructive algorithms that I called WMD’s, short for Weapons of Math Destruction. And naturally, a bunch of people have written to me asking if I thought the algorithm from this article would qualify as a WMD.
In a word, no.
Here’s the thing. One of the hallmark characteristics of a WMD is that it punishes the poor, the unlucky, the sick, or the marginalized. This algorithm does the opposite – it offers them help.
Now, I’m not saying it’s perfect. There could easily be flaws in this model, and some people are not being offered help who really need it. That can be seen as a kind of injustice, if others are receiving that help. But that’s the worst case scenario, and it’s not exactly tragic, and it’s a mistake that might well be caught if the algorithm is trained over time and modified to new data.
According to the article, the new algorithmic advising system has resulted in quite a few pieces of really good news:
- Graduation rates are up 6 percentage points since 2013.
- Graduates are getting that degree an average half a semester sooner than before, saving an estimated $12 million in tuition.
- Low-income, first-generation and minority students have closed the graduation rate gap.
- And those same students are succeeding at higher rates in tough STEM majors.
But to be clear, the real “secret sauce” in this system is the extraordinary amount of advising that’s been given to the students. The algorithm just directed that work.
A final word. This algorithm, which identifies struggling students and helps them, is an example I often use in explaining that an algorithm is not inherently good or evil.
In other words, this same algorithm could be used for evil, to punish the badly off, and a similar one nearly was in the case of Mount St. Mary’s College in Virginia. I wrote about that case as well, in a post entitled The Mount St. Mary’s Story is just so terrible.
This is a guest post written by Matt Freeman, an epidemiologist and nurse practitioner. His fields are adolescent and men’s health. He holds a doctorate in nursing from Duke University, a masters in nursing from The Ohio State University, a masters in epidemiology and public health from The Yale School of Medicine, and a Bachelor of Arts from Brandeis University. His blog is located at www.medfly.org.
It was mid-morning on a Saturday. I had only hand luggage, and had checked in online the day before. I arrived at the small airport exactly one hour before departure. I was a bit annoyed that the flight was delayed, but otherwise not expecting too much trouble. It was a 90-minute flight on a 70-seat regional jet.
By my best estimate, there were 80 passengers waiting to enter the security checkpoint. Most seemed to be leisure travelers: families with little kids, older adults. There was an abundance of sunburn and golf shirts.
The queue inched along. As I looked around, anxiety was escalating. There was a lot of chatter about missing flights; several people were in tears knowing that they would certainly have their travel plans fall into disarray.
One TSA employee with two stripes on his lapels walked his way through the increasingly antsy crowd.
“What is the province of your destination?” He asked the woman next to me.
“Yes, which province? British Columbia? Ontario?”
Confused, the woman replied, “I’m going to Houston. I don’t know what province that’s in.”
The TSA agent scoffed. He moved on to the next passenger. “The same question for you, ma’am. What is the province of your destination?”
The woman didn’t speak, handing over her driver’s license and boarding card, assuming that was what he wanted. He stared back with disdain.
There are no flights from this airport to Canada.
When it was my turn, I volunteered, “I’m going to Texas, not Canada.”
“What are the whereabouts of your luggage?” He asked.
“Their whereabouts? My bag is right here next to me.”
“Yes, what are its whereabouts?”
“It’s right here.”
“And that’s its whereabouts?”
This was seeming like a grammatical question.
“And about its contents? Are you aware of them?”
“Yes,” I replied, quizzically.
He moved on.
I missed my flight. The woman next to me met the same fate. She cried. I cringed. We pleaded with the airline agent for clemency. The plane pushed back from the gate with many passengers waiting to be asked about the whereabouts of their belongings or their province of destination.
The agent asking the strange questions and delaying the flights was a part of the SPOT program.
The SPOT Program
In 2006, the US Transportation Security Administration (TSA) introduced “SPOT: Screening Passengers by Observational Techniques.” The concept was to identify nonverbal indicators that a passenger was engaged in foul play. Some years after the program started, the US Government Accounting Office (GAO) declared that, “no scientific evidence exists to support the detection of or inference of future behavior.”
SPOT is expensive too. The GAO reported that the program has cost more than $900 million since its inauguration. That is just the cost of training staff and operating the program, not the costs incurred by delayed or detained passengers.
The “Science” Behind Behavioral Techniques
The SPOT program was developed by multiple sources, but there is one most prominent psychologist in the field: Paul Ekman PhD.
Ekman published Emotion in the Human Face, which demonstrated that six basic human emotions: anger, sadness, fear, happiness, surprise, and disgust, are universally expressed on the human face. Ekman had travelled to New Guinea to show that facial expressions did not vary across geography or culture.
Ekman’s theory was undisputed for 20 years until Lisa Feldman Barrett PhD showed that Ekman’s research required observers to select from the list of six emotions. When observers were asked to analyze emotions without a list, there was some reliability in the recognition of happiness and fear. The others emotions could not be distinguished.
When confronted with skepticism from scientists, Ekman declined to release the details of his research for peer review.
Charles Honts, Ph.D., attempted to replicate Ekman’s findings at the University of Utah. No dice. Ekman’s “secret” findings could not be replicated. Maria Hartwig PhD, a psychologist at City University of New York’s John Jay College of Criminal Justice, described Ekman’s work as, “a leap of gargantuan dimensions not supported by scientific evidence.”
When asked directly, a TSA analyst pointed to the work of David Givens, Ph.D., an anthropologist and author. Givens has published popular works on body language, but Givens explained that the TSA did not specify which elements of his own theories were adopted by the TSA, and the TSA never asked him.
The TSA’s Response
When asked for statistics, TSA analyst Carl Maccario cited one anecdote of a passenger who was “rocking back and forth strangely,” and was later found to have been carrying fuel bottles that contained flammable materials. The TSA described these items as, “the makings of a pipe bomb,” but there was no evidence that the passenger was doing anything other than carrying a dangerous substance in his hand luggage. There was nothing to suggest that he planned to hurt anyone.
A single anecdote is not research, and this was a weak story at best.
When the GAO investigated further, they analyzed the data of 232,000 passengers who were identified by “behavioral detection” as cause for concern. Of the 232,000, there were 1,710 arrests. These arrests were mostly due to outstanding arrest warrants, and there is no evidence that any were ever linked to terrorist activity.
What Criteria Are Used in the SPOT Program?
In 2015, The Intercept published the TSA’s worksheet for behavioral detection officers. Here it is:
As much as the TSA’s behavioral detection mathematical model is hilarious, it is also frightening. The model seeks to identify whistling and shaving.
If I score myself before a typical flight, I earn eight points, which assigned me to the highest risk category. If one followed the paperwork, I should have been referred for extensive screening and law enforcement was to be notified.
Considering that the criteria include yawning, whistling, a subjectively fast “eye blink rate,” “strong body odor” and head turning, just about everyone reaches the SPOT threshold.
The Risk of Scoring
Looking past the absence of evidence, there are further problems with the SPOT worksheet. “Scored” decisions can detract common sense. It does not matter if a hijacker or terrorist fails to whistle or blinks at a normal rate if he or she blows up the airplane.
The Israeli Method
As an Israeli national, I became accustomed to the envied security techniques employed at Israel’s four commercial airports.
The agents employed by the Israeli Airports Authority (IAA) do indeed “profile” passengers, but their efforts are often quicker, easier, and arguably more sensitive.
The questions are usually reasonable and fast. “Where have your bags been since you packed them?” “Did anyone give you anything to take with you?” “Are you carrying anything that could be used as a weapon?”
The IAA is cautious about race and religion. The worst attack on Israeli air transportation took place in 1972 at Ben Gurion Airport. Twenty-six people were killed. The assailants were Japanese, posing as tourists. Since that attack, the IAA has attempted to include ethnicity and religion only as components of its screening process.
Although many have published horror stories, the overwhelming majority of passengers do not encounter anything extraordinary at Israeli airports. The agents are usually young, bubbly, right out of their army service, and eager to show off any language skills they may have acquired.
Is There a Better Answer?
Israel does not publish statistics, and I could not tell you if their system is any better. The difference is one of attitude: most of the IAA staff are kind, calm, and not interested in hassling anyone. They do not care how fast you are blinking or if you shave.
Given the amount of air travel to, from, and within the United States, I doubt that questioning passengers would ever work. The TSA lacks the organization, multilingual skills, and service mentality of the Israel Airports Authority.
The TSA already has one answer, but they chose not to use it in my case. I am a member of the Department of Homeland Security’s “Global Entry” program. This means that I was subject to a background check, interview, and fingerprinting. The Department of Homeland Security vetted my credentials and deemed that I did not present any extraordinary risks, and could therefore use its “PreCheck” lane. But this airport had decided to close its PreCheck lane that day. And their SPOT agent had no knowledge that I had already been vetted through databases and fingerprints… arguably a more reliable system than having him determine if I blinked too rapidly.
Until 2015, the PreCheck program also meant that one need not pass through a full-body scanning machine, in part because the machines are famously slow and inaccurate. They are particularly problematic for those with disabilities and other medical conditions. But the TSA decided that it would switch to random use of full body scanners even for those passengers who had already been vetted. Lines grew longer; no weapons have been discovered.
- The SPOT program has been proven to be ineffective. There is no rational reason to keep it in place.
- There must not be quotas or incentives for detailed searches and questioning in the absence of probable cause.
- Passengers consenting to a search should have the right to know what the search entails, particularly if it involves odd interrogation techniques that can lead to missing one’s flight.
- The TSA should respect previous court rulings that the search process begins when a passenger consents to being searched. Asking questions outside of the TSA’s custodial area of the airport is questionable for legal reasons.
- Reduce lines. The attacks in Rome and Vienna were more than four decades ago, but that has not dissuaded the TSA. Get the queue moving quickly, thereby reducing the opportunity for an attack. The more recent attack in Brussels still did not change TSA policy.
- Stratified screening, such as the PreCheck program, makes sense. But it TSA staff elect to ignore the program, then it is no longer useful.
Benton H, Carter M, Heath D, and Neff J. The Warning. The Seattle Times. 23 July 2002.
Borland J. Maybe surveillance is bad, after all. Wired. 8 August 2007.
Dicker K. Yes, the TSA is probably profiling you and it’s scientifically bogus. Business Insider. 6 May 2015.
Herring A. The new face of emotion. Northeastern Magazine. Spring 2014.
Kerr O. Do travelers have a right to leave airport security areas without the TSA’s Permission. The Washington Post. 6 April 2014.
Martin H. Conversations are more effective for screening passengers, study finds. The Los Angeles Times. 16 November 2014.
The men who stare at airline passengers. The Economist. 6 June 2010.
Segura L. Feeling nervous? 3,000 Behavioral Detection Officers will be watching you at the airport this thanksgiving. Alternet. 23 November 2009
Smith T. Next in line for the TSA? A thorough ‘chat down.’ National Public Radio. 16 August 2011.
Wallis R. Lockerbie: The Story and the Lessons. London: Praeger. 2000.
Weinberger S. Intent to deceive: Can the science of deception detection help catch terrorists? Nature. 465:27. May 2010.
US House of Representatives. Behavioral Science and Security: Evaluating the TSA’s SPOT Program. Hearing Before the Subcommittee on Investigation and Oversight. Committee on Science, Space, and Technology. Serial 112-11. 6 April 2011.
Yesterday I came across this Reuters article by
The title employs a common marketing rule. Frame bad news as good news. Instead of saying, Big data shifts costs to pessimistic tweeters, mention only those who will benefit.
So, what’s going on? In the usual big data fashion, it’s not entirely clear. But the idea is your future health will be measured by your tweets and your premium will go up if it’s bad news. From the article:
In a study cited by the Swiss group last month, researchers found Twitter data alone a more reliable predictor of heart disease than all standard health and socioeconomic measures combined.
Geographic regions represented by particularly high use of negative-emotion and expletive words corresponded to higher occurrences of fatal heart disease in those communities.
To be clear, no insurance company is currently using Twitter data against anyone (or for anyone), at least not openly. The idea outlined in the article is that people could set up accounts to share their personal data with companies like insurance companies, as a way of showing off their healthiness. They’d be using a company like digi.me to do this. Monetize your data and so on. Of course, that would be the case at the beginning, to train the algorithm. Later on who knows.
While we’re on the topic of Twitter, I don’t know if I’ve had time to blog about University of Maryland Computer Science Professor Jennifer Golbeck. I met Professor Golbeck in D.C. last month when she interviewed me at Busboys and Poets. During that discussion she mentioned her paper, Predicting Personality from Social Media Text, in which she inferred personality traits from Twitter data. Here’s the abstract:
This paper replicates text-based Big Five personality score predictions generated by the Receptiviti API—a tool built on and tied to the popular psycholinguistic analysis tool Linguistic Inquiry and Word Count (LIWC). We use four social media datasets with posts and personality scores for nearly 9,000 users to determine the accuracy of the Receptiviti predictions. We found Mean Absolute Error rates in the 15–30% range, which is a higher error rate than other personality prediction algorithms in the literature. Preliminary analysis suggests relative scores between groups of subjects may be maintained, which may be sufficient for many applications.
Here’s how the topic came up. I was mentioning Kyle Behm, a young man I wrote about in my book who was denied a job based on a “big data” personality test. The case is problematic. It could represent a violation of the Americans with Disability Act, and a lawsuit filed in court is pending.
What Professor Golbeck demonstrates with her research is that, in the future, the employers won’t even need to notify applicants that their personalities are being scored at all, it could happen without their knowledge, through their social media posts and other culled information.
I’ll end with this quote from Christian Mumenthaler, CEO of Swiss Re, one of the insurance companies dabbling in Twitter data:
I personally would be cautious what I publish on the internet.
I’ll also stop by beforehand at WORT for a conversation with Patty Peltekos on her show called A Public Affair, as well as afterwards at the local NPR station, WPR, for a show called To The Best of Our Knowledge. These might be recorded, I don’t know when they’re airing.
What a city! Very welcoming and fun. I should visit more often.
I’ve become comfortable with my gadfly role in technology. I know that Facebook would characterize their new “personalized learning” initiative, Summit Basecamp, as innovative if not downright charitable (hat tip Leonie Haimson). But again, gadly.
What gets to me is how the students involved – about 20,000 students in more than 100 charter and traditional public schools – are really no more than an experimental and unpaid workforce, spending classroom hours training the Summit algorithm and getting no guarantee in return of real learning.
Their parents, moreover, are being pressured to sign away all sorts of privacy rights for those kids. And, get this, Basecamp “require disputes to be resolved through arbitration, essentially barring a student’s family from suing if they think data has been misused.” Here’s the quote from the article that got me seriously annoyed, from the Summit CEO Diane Tavenner herself:
“We’re offering this for free to people,” she said. “If we don’t protect the organization, anyone could sue us for anything — which seems crazy to me.”
To recap. Facebook gets these kids to train their algorithm for free, whilst removing them from their classroom time, offering no evidence that they will learn anything, making sure that they’ll be able to use the childrens’ data for everything short of targeted ads, and also ensuring the parents can’t even hire a lawyer to complain. That sounds like a truly terrible deal.
Here’s the thing. The kids involved are often poor, often minority. They are the most surveilled generation and the most surveilled subpopulation out there, ever. We have to start doing better for them than unpaid work for Facebook.
This is a guest post by an IT Director for a Fortune 500 company who has worked with many businesses and government agencies.
It was my mistake. My daughter’s old cell phone had died. My wife offered to get a new phone from Verizon and give that to me and then give my daughter my old phone. Since I work with Microsoft it made sense for me to get the latest Nokia Lumia model. It’s a great looking phone, with a fantastic camera, and a much bigger screen than my old model. I told my wife not to wipe all the data off my old phone but to just get the phone numbers switched, and we could then delete all my contacts from my old phone. While you can remove an email account on the phone, you can’t change the account that is associated with Windows Phone’s cloud. So my daughter manually deleted all my phone contacts and added her own to my old phone – but before that I had synced up my new phone to the cloud and got all my contacts downloaded to it. Within 24 hours, the Microsoft Azure cloud had re-synced both phones, so now all the deletes my daughter did propagated to my new phone.
I lost all my contacts.
I panicked, went back to the Verizon store, and they told me that we had to flash my old phone to factory settings. But they didn’t have a way for me to get my contacts back. And they had no way for me to contact Microsoft directly to get them back either. The Windows Phone website lists no contact phone number for customer support – Microsoft relies on the phone carriers to provide this, apparently believing that being a phone manufacturer doesn’t require you to have a call center that can resolve consumer issues. I see this as a policy flaw.
I had the painstaking process of figuring out how to get my phone contacts back, maybe one at a time.
But the whole cloud syncing made me think about how we’ve now come to trust that we can have everything on our phones and not think about adequately backing it up. In 2012, the Wired reporter Mat Honan reported about how a hacker systematically deleted all his personal information including baby photos on his Apple devices he had saved to the cloud. The big three phone manufacturers now (Apple, Google and Microsoft) have a lot of personal information in their clouds about all of us cell phone users. Each company, on its own, can each create a Kevin Bacon style “six degrees of separation” contacts map that would make the NSA proud. While I lost over 100 or more phone contacts, each one of those people would likely also have a similar or more contacts plugged into their phones, and so on. If the big three (AGM, not to be confused with Annual General Meetings) colluded together, they could even create a real time locator map showing where all our contacts are right now all round the world. Think of the possibilities for tracking: cheating spouses, late lunches at work, what time you quit drinking at the local, what sporting events you go to, which clients your competitors are meeting with etc. Microsoft’s acquisition of LinkedIn makes this sharing of information even more powerful. Now they’ll have our phone numbers and email contacts and some professional correspondence too.
I don’t trust Google. Their motto of “don’t be evil”, almost begs the question why do they have to remind themselves of that? Some years ago they were reported as scanning emails written to and from Gmail accounts. Spying on what your customers think of as private correspondence comes to my mind as evil. And just last week Yahoo admits to doing the same thing on behalf of the government, scanning for a very specific search phrase. I hope the NSA got their suspect with that request, and it wasn’t just a trial balloon to see how far they could go with pressuring the big data providers and aggregators. Yes, I can see the guys in suits and dark glasses approaching Marissa Mayer, “Trust us, this will save lives. We believe there’s the risk of an imminent terrorist attack”. I hope they arrest someone and bring charges, even if to justify Marissa’s position.
So why do I bring all that up? I believe we need consumer personal data protection rights. Almost like credit reporting. The big three (AGM) personal data aggregators and Facebook and LinkedIn collect a lot of personal data about each of us. We should have the right to know what they keep about us, and to possibly correct that record, like we do with the credit bureaus. We should be able to get a free digital copy of our personal data at least annually. The personal data aggregators should also have to report who they share that information with, and in what form. Do they pass along our phone contact information, or email accounts to 3 rd party providers or license that to other companies to help them do their business? The Europeans are ahead of America in protecting privacy rights on the internet, with the right to be forgotten, and the right to correct data. We should not be left behind in making our lives safer from invasion of our privacy and loss of personal security.
We need to know. The personal data aggregators need to be held to higher standards.