Guest Post: how to be a data scientist at a non-profit
This is a guest post by John Santerre, a 5th year Ph.D. student at the University of Chicago in the Computer Science Department. Previously a photojournalist, John has worked with nonprofits and NGO’s off and on for the last ten years. This summer he served as a Research Programmer at The Eric and Wendy Schmidt Data Science for Social Good Fellowship. His master’s work, under Prof. Lek-heng Lim, involved the use of Hodge Decomposition for rank disambiguation while his Ph.D work is at Argonne National Laboratory and involves scalable Machine Learning techniques for use on Cancer and Anti-Microbial Resistance (AMR).
The recent mathbabe post What can a non-academic mathematician do that makes the world a better place struck a chord with me. Over the last ten years I’ve worked as a photojournalist on and off with nonprofits photographing everything from Sigourney Weaver and Anna Wintour to documenting drug dealers in Puerto Rico, rebel fighters in Burma/Myanmar, and the UN Peacekeeping effort in Haiti. I was so involved with nonprofit work, I founded my own, just to provide photography services to other nonprofits. Most recently I spent the summer as research programer for the Eric and Wendy Schmidt Data Science for Social Good Fellowship. I thought my experience might offer a small amount insight, at least for those who are truly new to working with nonprofits or “social good” organizations.
The most rewarding and challenging aspect of working at a nonprofit is the responsibility you bear to educate the organization about the limitations and potential you present. This can’t be overstated. The organization you choose to work with will have any number of teachers, secretaries, drivers, programmers, and support staff all with clearly delineated jobs descriptions. As the outsider who, as in T.S. Eliot’s poem, has “Come … to tell you all, I shall tell you all,” you will almost certainly be alone in your role. In fact, that is the explicit reason we seek out such opportunities: working at the “tip of the spear” presents an opportunity for our skills to be uniquely impactful. Offering insights that the organization wouldn’t have access to, or perhaps cannot afford, can be vastly fulfilling.
However, this brings with it an inherent Faustian bargain. Just as the violinist Joshua Bell was ignored in the DC Metro but adored in the philharmonic [1], so too will many of your finely tuned skills fall on comparatively (computationally?) deaf ears. In my experience, you have/get to “check” your craft at the door. In my role as a photographer this often meant my most useful skill was taking comparatively simple photographs [2]. Now it often means my technical contribution to nonprofits is less than my potential. In fact, the clients are more than happy to explicitly express that. Often they are looking for a “sanity check” and understand that I am overqualified for their problem. In fact they often seek out and will only work with someone who they perceive is overqualified. I personally don’t mind this. In the right organization you can build on your own personal skill sets. In fact, I’ve never been challenged in the ways I had expected. Even so, there are a host of fairly common challenges and insights that are orthogonal to my craft that have kept appearing over the last ten years which I’ll share.
1. You are alone.
You will likely be a solo consultant who has no one to brainstorm with, no one to advocate for an agenda with, and no one to share the burden of the tumultuous experience of making sense of a new work environment. To top all of that off, you are often working on a necessarily compressed schedule. Ryan Kappadal, a statistics professor at the Air Force Institute of Technology (10+ years of experience in data science) told me this summer that the Air Force integrates its data science teams into other units in pairs. It was instantly obvious to me how much more impressive it is for an organization to watch and interact with two professionals debating approaches and building a strategy out loud rather than listening to a pitch from a single perspective.
2. You will likely have different metrics of success.
Currently I work on classifying Anti-Microbial Resistance (AMR) at Argonne National Lab, a topic so important that the UN, POTUS, and WHO have all identified it as a top threat facing global humanity. When I explained my work to a ‘data scientist’ at a start-up I noted a 95% accuracy of one classifier. They responded slightly dismissively “Is 95% high enough?”. Sure it’s “just” the k-means classification accuracy on the MNIST data set, but it’s also high enough to identify consistently (despite low sample size), the gene regions that confer resistance to a particular antimicrobial.This provides evidence of the likely mechanism – i.e., cell wall transport – that has mutated thereby implying possible counter strategies for the biologists.
Perhaps more humorously, I traveled to Cambodia to work with a nonprofit at “Smoke Mountain”, the continually burning garbage dump in Phnom Penh. The most useful service I provided for them was photographing their Christmas card. I climbed atop a nearby building and photographed the children spelling out “Thank You!” in human gymnastic positions. Not exactly what I was expecting after traveling 1/2 way around the world!
I cannot stress this topic enough, so I will harp on one more little point. In my mind, the inculcation required to become a specialist in our craft can blind us to the impact our client requires. In NGO work the objective is to build a shared skill set between you and the organization, rather than develop new insights into the problem. Photographers are constantly looking for new ways to restructure the frame. Similarly, machine Learning people are constantly trying to find new ways to approach the problems. But working with organizations requires a different metric. I have to judge my contribution not by the number of trailing digits of prediction accuracy, but by the impact I have on how biologist’s approach the problem of AMR. I love the craft of both ML and photography, so stepping out of the role where the craft is the most important thing is always hard, but when appropriate it can be vastly rewarding.
3. Different organizations have surprisingly similar needs.
This summer at the DSSG, my role, along with another programer was to build a “best practices” pipeline across the DSSG fellows and individual teams [3]. Freed from providing results to a client we could write maintainable clean code, while simultaneously “looking over their shoulders” for similarities between workflows. While each group was different, there was surprisingly consistency across groups, especially in terms of client interaction. That is another way of saying a sampling of 3-5 such organizations will give you a good sense of what this work is actually like.
4. Social good doesn’t require a 501c3 status.
It can be more rewarding and impactful to provide sophisticated technical services to a for-profit start-up preventing relapse in drug addiction (i.e. TriggrHealth.com) than providing rudimentary analysis consulting with a nonprofit following bird migrations [4]. It can be more fulfilling working for a growing for-profit but non-partisan organization like BallotReady.org targeting voter engagement, than it is to work for an issue-based partisan nonprofit. Or maybe it’s not for you. We are fortunate enough to work in a time where both types of organizations require our services.
—
In summary, with such tremendous need for the intersection of statistics and computer science, I find I am overwhelmed with options, but only if I am flexible in what role I expect to provide. Having “a voice” as a photographer or a focused specialty as an academic are hallmarks of advanced practitioners for good reason. These are the contributions that move the field forward. Conversely, serving as an advocate is a generalist position. Recognizing that helps me to find unique ways to ensure both my professional progress and that the organization’s needs are met.
1. While people walked past him IRL, he did manage to get 160k youtube views of his being ignored however!
2. A.K. Kimoto traveled through northern Afghanistan as photographer for UNICEF taking simple portraits that were very much needed. Later he used the connections he gained to return and photograph this work, a far more subtle and evocative collection of imagery that was very close to his heart.
3. It’s a (not quite alpha stage) python grid search library across models and parameters. We named Diogenes and gave it the tongue in cheek slogan “Searching for an honest classifier”.
4. Although, I used to watch the fall migrations mountain-side, and .csv’s of the data might make for an interesting weekend!
Yarn Confessions
Readers, you might have missed me for the past few days. I know I’ve missed you (and so has Aunt Pythia).
Well, full disclosure on what’s been happening is in order: I’ve been organizing my yarn collection.
Yes, it’s true, I have a deeply alarming amount of yarn, which has hitherto been gathered in smallish bags tucked all over the house, in every nook, cranny, and corner.
Well, with the help of my good friend Elena, who is starting a side business to help people organize their homes (ask me for an amazing reference!), I have officially tamed the yarn beast.
Just to give you a sense of the sprawl, here are just the “odds and ends” yarns in the blue or purple spectrum:

Pretty much every ball here represents a project I have long finished. I always buy a bit too much yarn and then keep the extra. Yes, that’s my foot and boob shelf.
Of course, I have odds and ends in other colors too:
Of course, not all my yarn is in the “odds and ends” category. I have whole bags of unopened yarn I got on sale during one of my winter excursions to Webs, the biggest and best yarn store in the world (you can tell it’s a big deal because it owns the “yarn.com” url). Here’s an example:
Anyhoo, the entire collection is here, feel free to take a look. I am beyond shame and embarrassment at this point, because at some point, when all my yarn was splayed across my entire living room and dining room, I realized how amazingly beautiful it all is and how much I’ve gotten out of my hobby over the years.
Also, I think I might actually have more yarn than the average yarn shop, so there’s that option as well, if I’m ever really broke. Plus, now that my yarn is so nicely organized and tidy, I’d even be able to show it to people.
So there you have it, yarn confessions. Life is too short to be ashamed of your passion.
Obsessed with VW
So I’m kind of obsessed with the VW story. Specifically, I want to know what happened back in 2009 when they started cheating. What was that conversation like? And how many people were privy to the deception? And how did they think it was going to go undetected?
In case you haven’t read all available articles on this like I have, the VOX article is really informative. Here are the key facts:
- Diesel cars are better at gas mileage, worse at polluting out nitrogen oxide (NOx). We care more about NOx in the US than they do in Europe, which is why there’s so much more diesel in Europe.
- But recently we’ve started caring about gas mileage, so there’s been a spot for diesel cars that can pass the NOx emissions test, which most diesel cars cannot (at least for a given price).
- VW blew everyone away with a diesel car that seemed to have good gas mileage and good emissions test results.
- They were discovered by people hired by an independent group, the International Council on Clean Transportation, who hired people to stick a probe up the tailpipe of some VW cars and drive them from Seattle to San Diego, where it was discovered that the NOx levels were up to 35 times higher than was allowed. And that group wanted to know how VW did it so they could copy them.
- Basically they were discovered because their results were “too good to be true.”
So, back to my question. How did the decision get made, that they’d just cheat? Didn’t they know they’d eventually get discovered? It’s kind of like when teachers and principals change the results on students’ tests so they can get a bonus: short term thinking, and kind of obvious if you track erasure marks.
Or… another way of looking at this was that they really didn’t think they’d get caught. The evidence they’d need for that theory is that they cheated all the time like this and had never gotten caught, or they knew others did.
Another possibility: it was a small group of engineers who did this, looking for a large bonus. This kind of thing happens all the time in finance, where you cook the books in a short term way to get a pay-out. Could this be true? Certainly many of the raw ingredients were already available – surely the software already existed for the engineers to test performance and emissions under all types of conditions, so putting it together with a simple “if” statement wouldn’t be too hard. But that begs the question of how they’d explain it to their boss.
In any case, there’s an internal VW story – or perhaps industry-wide story – here that I’d love to hear.
Interrogating algorithms
What with the recent discovery that VW has been using software to cheat on emissions tests, there has been a sudden and widespread conversation taking place on how we can interrogate algorithms.
In an New York Times op-ed from yesterday, Zeynep Tufekci weighed in on both the VW scandal and another recent software problem of public interest, namely voting machines. She concludes that “…the public can’t always know if the device is working properly — but we can check its operation by creating auditable and hard-to-tamper-with logs of how the software is running that regulators can inspect.” She also notes that slot machines in casinos have regular such inspections, so it’s not impossible.
Another New York Times article profiles Columbia Law professor Eben Moglen, quoted as saying that “proprietary software is an unsafe building material,” because “you can’t inspect it.” That was in 2010. Ironically, the article explained, the reason automobile manufacturers gave for not allowing inspection is that individuals would set up their cars to cheat on admissions tests. Of course, that doesn’t explain why you wouldn’t open up the algorithms at least to regulators.
The inspection of algorithms is a concept that’s probably new to a lot of people, first because algorithms are marketed as “objective” and “fair,” second because they are almost by construction too complicated for an average person to understand.
But, as we’ve seen in this example, those are simply not good enough reasons not to do it anyway. There’s a trade-off when we take advantage of automation and algorithms: we get efficiency and scale, on the one hand, and on the other we lose control. In fact, we don’t really know what’s happening and when.
The very least we could do is ask them.
In The Crimson and Le Monde
Do you guys remember the event I went to last week? It was the kick-off event for a new group at Harvard called Gender Inclusivity in Mathematics (GIIS), and it went well. It was written up in The Crimson, together with an action shot of me and Moon giving out tough love advice:

This took place in the same room in which I wrote my quals, which might explain why I’m looking confused and horrified.
One of the organizers, Cherie Hu, wrote a blog yesterday inspired by last week’s event which ponders identity and mathematics.
Also! I was please to be featured in a story that ran in Le Monde last week with my friend and mathematical colleague Leila Schneps, who recently wrote a book with her daughter about bad math in courts systems. It’s in French, obviously, but if I remember my schooling correctly I am quoted in the article talking about destructive algorithms like the Value-Added Model for teachers and even political micro-targeting. No picture there though.
Pre-settlement funding companies in the lawsuit economy
This is a guest post by Ronald Sinai, the founder and CEO of Nova Legal Funding, a national lawsuit funding company based in Los Angeles. Prior to entering the legal finance field, Ron was a student at the University of California, Berkeley where he earned a Bachelor’s degree in 2014.
Personal injury law is a big business in the United States. With every traffic accident or slip and fall, a ‘lawsuit economy’ emerges in expectation for the looming monetary compensation for the plaintiff. Attorneys, medical treatment centers and litigation service providers make up the bulk of this network. This post will tackle the latest, fastest growing and most disruptive industry to enter the lawsuit economy: pre-settlement funding companies.
Pre-settlement funding is a financial lifeline for plaintiffs involved in personal injury litigation. It’s a cash advance on the future proceeds of a settlement for people who can’t wait years for their cases to finalize. Plaintiffs often use the advance to pay for living necessities, medical bills and other immediate financial obligations. Repayment to the funder is wholly contingent on the case being settled out of court or won in trial. The plaintiff repays nothing if the case is lost, making it a risky non-recourse investment for the funding company.
As a principal in a funding company, I felt obliged to contact Cathy after hearing her speak on Slate.com’s Money Podcast and reading her negative blog post about my industry. While it may surprise you, my intent in reaching out was not to correct her. In fact, her worries are completely valid and for good reason. Little-to-no oversight by regulators has allowed bad players in the legal finance industry to employ business models that place profits over ethics. Over time, such practices justifiably resulted in a bad name for the industry, with some comparing this otherwise justice-equalizing tool with payday loans.
It’s important that we don’t throw the baby out with the bathwater. Simply because a lack of regulation led to a swarm of bad actors doesn’t mean pre-settlement funding should go away. When done correctly, embracing the fundamental benefits of lawsuit funding can help our justice system become more equitable and accessible to everyone.
On the value of plaintiff funding
Funding does more than ensure fair and complete compensation for the injured—it ensures that our justice system is blind to an individual’s economic standing.
Pre-settlement funding empowers small plaintiffs against big insurance companies. Just as attorneys litigate and treatment centers heal, the funder adds value to the case by granting the plaintiff financial stability. Solid financial footing helps them reject early lowball settlement offers from insurance companies, who always seek to take advantage of a person’s vulnerable economic position.
No matter how obvious the negligence or big the damages, insurance companies always delay compensation. It’s the oldest and most effective trick in the book: gain leverage by inducing desperation. The adjuster capitalizes on this desperation by offering the plaintiff a lowball offer in exchange for a quick and early payout. Funding the plaintiff takes the leverage away from the adjuster, which results in a big win for the small guy.
On the problem with plaintiff funding
No matter how great the premise of consumer legal finance might be, a lack of oversight will continue to allow bad players to charge unreasonable rates and make the service abusive rather than valuable.
As an operator in the space, I know what deals are being made and under what terms. I’ve seen countless of funding agreements from dozens of companies since the inception of my business. Plaintiffs who previously received funding from other sources come to us in hopes of refinancing their expensive paper. I don’t have hard data, but the average rate seems to be 3-4% compounded monthly or 35-40% every six month period. That’s not including a possible broker fee (10-20% of funding) and an application fee of $250-$400.
Even worse, most companies charge rates that are uncorrelated with the risk profile of each individual case. In other words, a litigious person with a questionable slip and fall at Wal Mart will get the same rate as a victim who was rear-ended while stopped at a red light.
The “fund-everything-at-high-rates” business model
Companies that charge high rates have the luxury of relying less on proper underwriting than they do on volume. This business model is attractive for many reasons. First, higher rates makes it easier to swallow loser cases, which reduces underwriting requirements and drives an increase in deal flow. Relaxed underwriting also means hiring less attorneys, which leads to a massive reduction in overhead.
Secondly, a lack of proper underwriting makes the process hassle-free for the plaintiff’s busy attorney, who wants nothing more than to get the funding process over with. Believe it or not, attorneys whose clients bug them enough for cash will take a quicker funding process over lower rates every time. Same goes for the plaintiff: they like what is fast, not what is affordable. They only realize the mistake when it comes time to repay the funder.
The fix? Enforce rate caps, force careful underwriting.
Not all funders behave recklessly. At my firm, for example, we offer a fixed payoff schedule for 10-15% each consecutive 6-month period. At these rates, our underwriting has to be lock-tight in order to turn a profit. A call with the attorney needs to be arranged, all liens must be inspected, and a variety of documents must be submitted by the law firm before a deal is made.
This forces us to only fund meritorious claims where plaintiffs suffer serious damages, and negligence is clear. By this logic, rate caps will do more than stop unreasonable rates. They will also make it impossible for frivolous lawsuits to get funding and hurt our economy.
Aunt Pythia’s advice
Readers, Aunt Pythia is quite pleased with herself this morning. She has come up with an amazing solution to the problem of teenage dissipation and slovenliness.
Now, don’t get Aunt Pythia wrong: she’s got some amazing teenagers. They even do their own laundry, and take turns doing the dishes (when prompted!). But one thing they haven’t been able to do, no matter the level of coaxing, is to put away their clean clothes in their dresser. What invariably happens is they put their clean clothes in a bag, which gets turned over onto the floor in the following morning’s search for a clean sock.
Bottomline: their floors are always entirely covered with clothes.
Solution: get rid of their dressers altogether and replace them with a large “clean laundry” bin. These are the bins I bought which have just been delivered:
Strangely enough, their father doesn’t seem as excited as Aunt Pythia about the “clean laundry bin”. Something about the aesthetics, or the size. His tune will change when there’s no laundry on the floor, though, I assure you. I promise to update you on this miraculous cure to all things slipshod and/or lackadaisical.
OK, on with the advice! And after you enjoy said advice, please:
ask Aunt Pythia any question at all at the bottom of the page!
By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.
——
Dear Aunt Pythia,
I’m stuck in an interpersonal pickle and I need some insight from someone totally removed from the situation. Most of the time I have a pretty strong moral intuition but this has me at a loss.
I’ve known this woman, “Beth,” since high school. She has always been a difficult person to be a friend to, and I think I’m reaching my limit. (We’re both in our early 30s now, to give you an idea of the timeline.) Beth is difficult because she is a self-centered person, which is exacerbated by mental illness. Beth has been on medication for OCD since high school and for bipolar disorder since college.
While she is currently seeing a psychiatrist, she definitely never visited a mental health professional in high school and probably didn’t in college, either. According to her, she “diagnosed herself” with OCD, depression, and bipolar, then talked to her GP (a close friend of the family), who agreed with her assessment and wrote out the prescriptions. I don’t know how prevalent this kind of “self diagnosis” is, but I think this part of her background is relevant, so I’m including it.
For what it’s worth, I don’t doubt for a moment that she suffers from mental illness. I just worry that she is getting the wrong treatment, since she doesn’t seem any “better” after ten years of this particular cocktail of medication. (But I haven’t said any of this to her, and wouldn’t dare, because IMO that would be presumptuous and maybe she’s coming off worse online than IRL. That’s the job of a mental health professional.)
At the moment I am one of two people she talks to who aren’t her family (husband, in-laws, mother), her psychiatrist, or her current lover. (She has been having an affair for almost a year; this is not an open/”monogamousish” marriage.) I feel morally obligated to remain in her life to at least some degree, since I imagine she is probably very lonely, especially since she is in the middle of an argument with her only other friend. This “only two friends” situation is also something she’s told me; I’m not making any suppositions here. Otherwise, I would have cut ties a while ago.
I don’t like the person I become when I talk to her and I don’t think I have the right skillset or knowledge to help her. The only thing that happens as a result of our conversations is that she gives me minute-by-minute updates on her moods/activities, trash talks her husband, relates the sexcapades she’s having with her lover, and asks me for advice that she doesn’t follow. Occasionally she shares random news link with a few throwaway comments on them, and once in a while she asks me what I’m doing, but after a few lines of conversation everything is back to her.
Most people I think I could say, “I want to support you, but I’ve got a lot of stuff going on my self and it’s taking all of my cope just to deal with that. I’ll let you know when I’m feeling better.” or “You know, you tell me a lot about what’s going on with you, but you don’t seem to be displaying any interest in my life. I know that you care, of course, but it would be nice if you could show me that you do.” and, while it would sting, they would be able to handle it. But she is fragile enough that I think even that would crush her, considering that she is angry at her only other friend for essentially saying just that.
The silver lining in all of this is that I am hundreds miles of way and will remain there for the rest of my life, so I only have to interact with Beth online. At the moment I am basically checked out. I’ve limited myself to blase responses like “that sounds annoying” or “that’s good” to most things and outright ignoring what I think is the most harmful/unhealthy stuff she says, or the things that sound like a bid for attention or validation. Is this the best I can do? Should I tell her I need some alone time (or full-on ghost her) and reduce her social outlets by half? Am I overestimating my own importance? Am I underestimating her resilience? Am I making myself a martyr?
Thank you for your input.
Confused Friend
Dear Confused,
A few things. First, sympathy: your friend sounds really hard to deal with, and it’s kind of you to stick with her.
Second, I agree that she sounds like she has real problems, and I’m no professional so I wouldn’t hazard a guess what her problem is, but I’d suggest you spend some time looking at personality disorder profiles. I say that because it has helped me enormously in the past; when you encounter someone with a personality disorder, you feel bewildered and confused – and sometimes even partially responsible to help – but then, reading about the disorders, and the support groups for people who are married to people with them, you realize that you are not alone in your confusion, and that you are not capable of curing them.
Finally, advice. You are at risk of getting so fed up with your friend that you leave her entirely. Instead of letting your last ounce of true goodwill drip out of you slowly, I suggest you tell her about the difficulties you’re having, and asking for her help to remain friends, while you still can do it. Too often, people only express frustration at the point of no return, so the underlying message is, “you cannot convince me to be your friend anymore, it’s too late.” I would love to see your message be something more like, “you need to be a friend to me as well or else you’ll lose me.” It’s a much kinder message.
So, if you can do it, tell her truthfully what’s frustrating you, and be sure to tell her that you still want to be friends, and see what happens. In other words, don’t be a martyr, and don’t underestimate her resilience. If she cannot hear you, and gets upset and refuses to talk, then wait a few months or a year or two and get back in touch, because people often need time to recover, and their disorders often oscillate in terms of severity. Above all, keep careful track of what you’re thinking and doing versus what she accuses you of thinking and doing, because you’ll need to stay calm and reasonable, and that might be hard, but it’s what a good friend does.
Good luck,
Aunt Pythia
——
Dear Aunt Pythia,
I have been close friends with a guy from undergrad for six years. We met my freshman year and became best (platonic) friends that year. He was dating a girl from his hometown, but they were never very close. He felt obligated to stay with her for intense family reasons, but the emotional bond between them was minimal. They fought often and had very little in common. There was no sexual relationship.
They broke up during our sophomore year and he approached me about starting a relationship. I was in a bad place and was not ready to be in a relationship. They got back together about six months later.
We remained very close throughout college – ran together, studied together, went backpacking together. We both told each other everything. I thought that we were really just friends, and that the people who thought we were dating or should date were reading into things (professors, friends, etc. frequently assumed we were).
After graduation, we remained very very close and he remained dating his girlfriend, still under strict family pressure. I realized after we both graduated that I was in love with him. I was/am very physically attracted to him and emotionally bonded with him. I didn’t say anything to him. We both started doctoral programs in New England (in the sameish field) and are both two years in. We don’t see each other much (about every 2 months), but talk on the phone once a week, write, and text often.
They broke up about four months ago and I’m at a loss of what to do. They definitely won’t be getting back together, but at this point, I’ve lived in stagnation for so long that I’m afraid to tell him. I don’t want to lose my best friend, and the long wait has left me more scared than ever. I don’t even know if I want to tell him. What do I do? Help me, Aunt Pythia! I dreamed of this for so long, but now I don’t know what to do.
Perplexed and Frozen
Dear Perplexed,
OK, so two comments. First, nobody writes to Aunt Pythia so that she can say, “don’t go for it, it’s a trap!”. That doesn’t happen. So obviously what you’re looking for here is the green light. They don’t call me Aunt “Go For It” Pythia for nothin’.
Second, I’ma give you the green light here. Not necessarily because I think it will work out – although it well might! – but mostly because I need you to move the fuck on. Holy crap, lady, you gotta get your love life moving here, and it’s been according to my calculations 6 years of this platonic friend crap at least. You didn’t mention how many love affairs you’ve been having on the side in the meantime, so I’m going to imagine at least a few, but jeez. How can you be so patient?!
As for my advice, it’s the oldest and simplest plan in the book. Invite your friend to stay with you for the weekend, get everyone out of the area with strict instructions never to return, and drink a ton of booze. Easy peasy lemon squeezy, with an emphasis on the lemon squeezy. And please do it quick, my patience is completely worn out. And then please write back and tell me what happened.
Aunt Pythia
——
Dear Aunt Pythia,
I’m a liberal arts/sciences undergrad focused on the obstacles to just climate and economic policies. I’m also interested in economics/finance, the political process, and social justice, among other things.
I want to get work experience related to my interests before I graduate, so I’m planning not to take classes in Spring 2016 so that I can do an internship (or several), but I’m not sure about how to find the right opportunities. I’ve reached out through some social connections to folks who might be interesting, but I should do more.
Do you have any tips for finding internships? Or even better, do you know of any great people who could use a smart research assistant this coming Spring? I do good research.
Thank you, Idealistic Human
Dear Idealistic,
Great idea, and I’m sure my commenters will weigh in with ideas. Personally I’d find underfunded organizations that do good stuff and I’d simply ask them if they need help. The ones that advertise for internships are way too overstaffed and organized.
Aunt Pythia
——
Dear Aunt Pythia,
So, as a not young in body person (half a century, woohoo!) am a little surprised to find myself:
- with a job after almost 12 months out of work,
- excited like I was starting fresh, and
- worried about the future – aka ai/robots getting the work.
The job I am about to start shouldn’t last more than 5 years. The goal is to set up a reporting system for a variety of KPIs drawing on data from a variety of external organizations.
On the one hand, if I don’t manage to automate most of this, I would see it as a failure. On the other hand, what work will be left for others when I succeed?
I will be fine. After 5 more years of earning, I should be mortgage free and healthy savings. Should I feel a bit bad that I am helping software eat the world?
Frumpy Old Graduate Excitedly Yearning
Dear FOGEY,
A wise man (Suresh Naidu) once said to me, “protect the people, not the jobs.” I think he’s right. We are going to have to deal with the robot/ automation revolution sooner or later, and so instead of pushing to avoid automation, a futile gesture to save unnecessary and outdated jobs, we should be thinking about pushing for free college and training for the jobs of the future with all the money we’re saving as a result of this nifty automation revolution.
So, in short, no, don’t feel guilty. But be sure to do your part in figuring out what the future should look like for young people once you retire. Be an advocate for a fair and equitable future!
Aunt Pythia
——
Readers? Aunt Pythia loves you so much. She wants to hear from you – she needs to hear from you – and then tell you what for in a most indulgent way. Will you help her do that?
Please, pleeeeease ask her a question. She will take it seriously and answer it if she can.
Click here for a form for later or just do it now:
Jury duty
For the past two days I spent my time bored out of my mind at jury duty. And it’s not even that unpleasant or uncomfortable, and it even has pretty good wi-fi, but for some reason, seated as we are in a big room with 90 other people or so makes you kind of nuts. It’s like you’re on a two day plane ride to nowhere.
For reading I had with me A Confederacy of Dunces, which I’m reading for the second time, and which is great background for the story I’m about to tell you.
On the first morning of jury duty, you get to see your cast of characters, and it’s kind of amusing. In my case, we had an extremely overworked clerk named Bill, who was doing the job of three people, telling us how to fill out our forms in precise and extremely detailed patient language, repeating everything 5 times for clarity and emphasis. And I would have started to wonder at Bill’s constant repetition, except that in spite of it, there were a few people who would manage to get confused and go up to him – invariably in the middle of a task – and ask him questions.
One woman in particular seemed to do this a lot, and she was loud as well, and almost seemed hostile. It seemed like she was objecting to the form itself, and wanted to find a way to trip up Bill or something, as a way to get back at having to be at jury duty. To the credit of Bill, he was always extremely polite to her. That guy is a saint. But it didn’t prevent her from looking around at the crowd of people, as if she wanted confirmation that her plight was unreasonable.
So yesterday rolled around, the second day of interminable waiting, and it was much worse than the first day. Because, after all, we all knew how boring it would be, and we were all hoping we wouldn’t be called to do our actual civic duty. Being prepared to do it was surely enough. The guy next to me kept mumbling, “I gotta get outta here” under his breath, while shaking his leg furiously.
At around 11am, something happened that kind of broke through the tense fog of boredom. Namely, about half of all the cell phones started to beep loudly. It was an Amber Alert (since resolved). We all pressed “OK” and the beeping din subsided.
Except not for long. I guess people who are on different networks get their Amber Alerts at different times. So for the next 10 minutes or so, random cell phone beeps would happen and be resolved. For all but one phone, everyone’s noise eventually went silent for good.
That last phone, however, was left unattended, which meant that every 3 minutes or so, it beeped loudly for 15 seconds. The fourth or fifth time this happened, the loud lady from the previous day started loudly complaining, “THAT NOISE IS ANNOOOOOYING ME! CAN SOMEONE TURN DOWN THAT NOISE?! IT’S SO ANNOOOOYING!”
Some combination of how pent up everyone’s frustration was, and this loud woman, and probably also the book I was reading, made me start laughing uncontrollably at this point, which was slightly contagious but didn’t stop the loud woman from complaining.
A bunch of people started to explain to her about Amber Alerts, but she just kept telling everyone how annoyed she was (loudly). Finally, one of the clerks at the front, who had (very reluctantly!) decided to show up to work today, told her there was nothing she could do and could the woman settle down.
Well, that made her quiet for about 15 minutes, but it didn’t stop the Amber Alerts from sounding every 3 minutes. And every time they started again it was difficult not to laugh. After the fifth time, some guy who had been in the bathroom for the first kerfuffle made the mistake of saying to the group, “I think someone needs to look at their phone and deal with it,” which was the cue to the loud woman to start wailing again about the noise, and it made a bunch of us start laughing again (I admit I was the worst). This time the loud woman added some sarcastic comments about how SOME people seemed to think her suffering was FUNNY, which made me simultaneously laugh harder and consider suicide. The lady at the front asked us all to settle down again and we did.
At this point it had been going on for almost an hour, everyone was hungry, and it was nearing lunch time. Finally, just as we were being dismissed for lunch, someone sitting next to the loud lady proclaimed, and I literally have no idea why it took them so long, “Hey lady, that’s your phone!,” which she denied, but then the woman up front said, “Lady, that better not be your phone! Take out your phone and check it right now!”
Readers, it was her phone.
Data science job blurbs
I’ve been getting lots of people writing to me recently with data science jobs that I think look like nice jobs, with nice people, but I am personally not going to apply for. Instead of forgetting about them, I’ve decided to advertise them myself.
—
The first position is from Harmony Institute, a research center trying to understanding the social impact of media. They are looking for a Technical Data Analyst. Here’s their blurb:
HI is looking for a media-savvy researcher with an understanding of social science/humanities research and a passion for empowering causes for social good. The ideal candidate is interested in studying complex phenomena — such as information diffusion, media framing, and emotional responses to stories — from multiple, often indirect perspectives. You know how to hack code and wrangle data, but you’re just as concerned with big questions as big data.
In this role, you’ll be working closely with a team of social scientists to study various aspects of the social impact of media. Most projects have a critical data component, and you’ll be the go-to person for a broad range of data tasks: identifying indicators and datasets of interest based on research questions; collecting data from available APIs, databases, and even web scraping; verifying, documenting, and sharing the processed data; and either leading or facilitating the data analysis and visualization of results. You’ll also collaborate with and get support from our product focused data scientists and software engineers. Fortunately, HI has built up a large code base (in Python) to help you with your work, so you’ll rarely be starting from scratch. You can look forward to improving your programming and data skills through diverse, interesting research projects; being exposed to many different perspectives among media makers, funders, and researchers; and advance the field’s understanding of media’s impact on society.
The full description is here, and they’re also looking for a data engineer.
—
The second position is from Paperless Post, which you’re doubtless familiar with if you’ve ever been invited to a kid’s birthday party. They are looking for a data scientist, and here’s their blurb:
The data team at Paperless Post helps people throughout the company use data to make better business decisions. We’re a team of five people (three women, two men), and each person supports various business and product teams. We spend around three fifths of our time working on projects for teams, and two fifths working on data team projects.
We do all types of data work: data engineering, ETL, analysis, building production data systems, building visualizations, building dashboards, creating KPIs, and more. As the main data contact for a team, you get a lot of autonomy and flexibility when helping them best use data.
We’re looking to grow our team, and are looking for both junior and senior data people. Paperless Post is a great company to work for as a data person. We have a product that people enjoy and pay for, and we don’t sell our user data. We care about professional development, and are supportive of interests outside work. Within the company, the data team is viewed as a valued partner to the teams we work with.\
For questions, contact solomon@paperlesspost.com.
—
If you have a data science position that is not in finance, or ad tech, and is with super nice people, I plan to do it at least one more time, so email me at the gmail address listed on my About page.
The fight for 15
Whenever I hear an argument about the possibility of raising the minimum wage to $15 per hour, it sounds like this. Person A, who is for it, makes the case that it’s too difficult to live on minimum wage earnings, and it doesn’t make sense for someone working full time to struggle so much to feed their kids. Person B, who’s against it, says that 15 is too high, that too many employers will be unwilling to pay for unskilled workers at that rate, and they will replace such people with machines instead of doing so. Essentially, they argue the bad will outweigh the good.
Full disclosure: I am often Person A. I once figured out that if you take someone’s hourly wage in dollars, and you multiply by 2, then you get their yearly wages in thousands of dollars. That means an income of $100K per year is $50 per hour. That means an income of the current New York minimum wage, $8.75 per hour, is a measly $17.5K per year, which would be absolutely crazy to try to live on, according to my reckoning. In other words, I think about what I could theoretically live on, if I had a minimum wage job, and I have extreme sympathy for people who try to.
Let’s get back to Person B’s argument. It’s weird because it sounds like Person B is arguing for the sake of the poor, but they’re ignoring the vital question of what is a living wage. Let me give you an analogy.
You have a sick population, and they all need 3 pills per day to stay well. The pills are expensive, though, and so the people in charge of pill distribution give most people 2 pills per day. They argue that, if they gave out 3 pills per day to everyone, some people would have no pills. For the sake of those theoretical people, then, they give out only 2, and everyone remains sick.
In other words, for the sake of holding on to crappy jobs that pay below living wages, and where the employees need food stamps to survive, we don’t raise the bar so they can actually sustain someone in a basic way. It’s almost like we’re desperate to hold on to them because otherwise our unemployment rate would be higher.
I say, figure out what a living wage is, and raise the minimum wage to that level. I actually don’t know what the magic number should be, exactly. Is 15 big enough? Maybe it is, in some places, but maybe in others it’s actually smaller. It doesn’t have to be the same throughout the country. But for as long as we live in a country where the model is that a job is supposed to support you, we should make sure it actually does.
Aunt Pythia’s advice
Have you guys read the recent NY Times book review of three new sex and romance books? The books are called ‘Date-Onomics,’ ‘The Sex Myth,’ and ‘Modern Romance,’ and I have very strong opinions about them – surprise, surprise! – based only on the review.
Date-Onomics is premised on finding love by crunching the numbers and by assuming that all women are looking to snag a “good man” no matter what. Simplistic, but then again there are certainly numbers to consider, and the fact that more women than men attend college is definitely at odds with the way men don’t like to marry women who earn more than they do. And yes, I framed that to be an intentionally controversial way of looking at it.
Next, in the Sex Myth, they investigate the switch from everyone being a prude a short while ago to everyone supposedly – but not actually – being a kinkmeister now, and how we’d be better off not identifying ourselves so much with our sex lives. Also simplistic, since sex is a central aspect to our human identity. It’s not as if in the past we didn’t really care about each other’s sex lives; it’s just that sex lives were way more stifled. Name a moment in human history that we didn’t obsess and gossip about who was having sex with whom. I bet you can’t. Instead, I want us to have more than just sex as identities. It’s obviously terrible to only rely on your sex appeal, especially as you age and are suddenly unfuckable.
The third book reviewed is Modern Romance, and it seems to argue that we sometimes get carried away with the numbers and the seemingly endless options we have on the dating scene and forget to appreciate the humanity in each other. Also simplistic, because if you are the only person stopping to smell the roses, you will get trampled from behind. It’s a collective action problem, and a cultural problem.
OK, on with the advice! And after you enjoy said advice, please:
ask Aunt Pythia any question at all at the bottom of the page!
By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.
——
Dear Aunt Pythia,
I was reading Flash Boys (I know I’m a bit late to the game!) and was wondering if you might comment a bit on its accuracy as well as how you feel about IEX? I’m very tempted to believe Lewis/Katsuyama, but after watching some videos of them together with people vehemently denying all claims in the book I have become a bit hesitant. I was hoping your insider knowledge would be helpful! Thanks for your time.
Not Smart Enough To Know
Dear NSETK,
Here’s the thing, it doesn’t really matter. For a few reasons, among them:
- The harm that was done by that whole scam, which is totally believable, is pretty small in terms of the trillions sloshing around in the market.
- There are plenty of other scams going on just like it. That’s what finance people do.
- It doesn’t affect the public nearly as much as the big stuff did like the financial crisis.
Putting all that together, who cares. I mean, Lewis is a great writer, and he tells a great story, but this time I think he just kind of randomly chose the good guys and bad guys and convinced everyone something terrible was going on because it sells books, when in fact it’s business as usual in the world of high frequency trading. If we could get rid of HFT altogether, that would be great, but that’s not what seems to be happening.
My two cents.
Aunt Pythia
——
Dear Aunt Pythia,
I want to apply for an assistant professor job at a particular university. This school has the overall department split into sub-departments and two of the sub-departments have openings for next year. My research could apply to either of the sub-departments, but the same person is listed as the search coordinator for both positions, so there is no way that it will not be noticed if I apply for both jobs. Is it “bad form” to apply for two jobs in the same department? Or do I have to pick just one?
Under Decision Paralysis
Dear UDP,
Hey, great news! You are qualified for not one but two jobs at the same place! Use it as an advantage. Apply for both, and in each cover letter mention that you’re applying for both, and that what this means is that your research will unite the two sub-departments and create synergies that everyone will really enjoy. Moreover, you’re sure you’d be incredibly happy taking either job. It doesn’t matter if the same people read your folders or not: assume not, but be the first person to frame the way to think of this as good news.
Aunt Pythia
——
Dear Aunt Pythia,
How can we get America to not focus on the ‘hero’? I believe that this libertarian view has made the ‘winner take all’ acceptable. Those of us still employed are noticing that since only the ‘best’ are hired, there is no 2nd tier support staff, and we have to be do several jobs (thanks, computers).
Waiting For The Implosion
Dear WFTI,
Aunt Pythia hears you loud and clear. Did you hear about the new Harvard Business School report, where the alums they surveyed agreed that we should work on combatting inequality, the biggest problem of our day? Well, it might not surprised you to hear that they also said that the way to combat these problems are tax reforms and streamlined regulation, in other words stuff that will actually exacerbate it.
The answer is, I’m not sure. The way humans work is we care about individual stories. That’s why stuff got going about Syrian refugees after the picture of the little boy washed up on the beach. I guess what we need to do is make sure the individual stories we hear about are examples of larger issues important to a lot of people, rather than just aspirational hero-worshipping schlock.
Aunt Pythia
——
Dear Aunt Pythia,
I am a professional woman in my mid 30s, no kids, although I don’t mind if others choose have them, preferably responsibly.
Over the past few years, I have become friends with several members of a large family where both parents are immigrants. The mother, despite circumstances, has encouraged her children and herself to become educated and start businesses. They live in a remote small town, but come to the city often for social and business events.
My question concerns their 20 year old son “Alex”. Like several of his siblings, he was home schooled, though he has yet to finish an official program or pass the GED. I’ve offered to tutor him, but he hasn’t accepted. Alex can be very motivated about some things and has lots of ideas, but he seems to dream more than do, and has not looked for a job outside of helping his parents with their ventures.
So, recently, rather than “nagging” Alex about getting a GED or job, I’ve switched tactics to asking him what he wants to do and how he plans to get there. He’s pretty receptive to ideas but rarely takes action. Last week, when I asked him what he wanted in life, he said “20 kids!” I thought he was joking, but he seems to think he can go back to his father’s country, where he will not only be entitled to a bride, but also to her sheep, goats, and house. So, now what? How can I encourage Alex to work towards a dream that helps him become independent before bringing somebody or many somebodies into the mix?
20 Questions
Dear 20 Questions,
Talk about a cultural difference! It seems like these kids haven’t entirely left their home country. Home schooling is obviously part of that, but also the fact that the he is working in the family business doesn’t help.
Even so, he’s made friends with you, and you’re concerned. I think you should continue to be his friend, and help him think through what his future might be like. Would he want to bring his wife and 20 kids to the states? How would he support them? Stuff like that, which he might not have thought about. I would guess you could help him plan, and you may have some influence on his plans by doing so, but I don’t think you can change his plans entirely. But it sounds like you’re already doing this, so I would say, keep it up!
Also, keep in mind he’s only 20, and lots of things in his life will change before he actually has 20 kids, if he ever does.
Aunt Pythia
——
Readers? Aunt Pythia loves you so much. She wants to hear from you – she needs to hear from you – and then tell you what for in a most indulgent way. Will you help her do that?
Please, pleeeeease ask her a question. She will take it seriously and answer it if she can.
Click here for a form for later or just do it now:
I’ve found the soundtrack to my angsty teenaged life
I grew up in the Breakfast Club era, which is to say a time when every teenager had a soundtrack to their lives, depending on where they fit in the social strata of their particular high school.
We would make mixed tapes, and listen to them on constant loop on our walkmans, until they were scratchy and worn, and we would take odd jobs to pay for the monumental AA battery use. A sure sign of long-lasting and meaningful friendship would be if one teenager made a mixed tape and gave it to another teenager. That would be a deep sign, both of kinship and, of course, of musical identity.
Personally, I was a misfit. That doesn’t mean that I didn’t belong in any clique, it means, ironically, that I fit in with “the misfits,” which were their own group, proud of not getting along with any other groups, except at times we’d forge alliances with the druggies. For example, when I first got to Lexington High School – my freshman year was 1986 – there was a smoking section outside the principal’s office where the misfits and the druggies could all smoke whatever we wanted, for some reason there were really no rules, and it was a happy time. By the time I left, though, the smokers were forced very slightly off campus, which is to say about a block away on Park Drive, and the temporary misfit/druggie alliance was forever broken. We misfits retreated to the J-House lounge.
Anyhoo, I’m getting away from myself, because I meant to talk about soundtracks, but I instead got carried away with nostalgic memories of hanging out – for a brief time – with the cool, fucked up kids.
My soundtrack was simple: Led Zeppelin, Pink Floyd, and Jethro Tull. I wore an army jacket and was deeply misunderstood, and sometimes in the summer I’d tie dye shirts and refuse to participate in things. This is me refusing to participate in the junior prom:
Here’s why this all came up today. If I could go back in time and live high school all over again, which I tend to do without permission anyway, I’d listen nonstop to Neutral Milk Hotel’s album, In the Aeroplane Over The Sea, even though it was release in 1998 and I graduated in 1990. Hey, it’s a fantasy, and they don’t always make sense.
I dare you to tell me you don’t agree when you hear the title track (or else you already know it, in which case you already agree):
Make sure you listen to the bridge, which is the best part of any song, ever.
Bloggy young nerd women
Today I want to share with you all two recent blog posts from nerdy young women. And who doesn’t love nerdy young women!?
The first is by Michelle G, an M.I.T. student, class of 2018, who is majoring in “14,” which is M.I.T. code for economics. She wrote a blogpost called Picture yourself as a stereotypical male, about the question of intelligence, gender, and stereotype threat. I thought I knew all about that stuff but I learned quite a bit from her post. p.s. Larry Summers, I hope you read this.
The second is by Meena Boppana, a Harvard CS major/math minor who has guest blogged here on mathbabe before. This post is called The Making of A Girl Mathlete, and it describes her experience winning math competitions, often as the only girl. Even though I personally think math contests kind of suck, I appreciate how much Meena got out of them.
Speaking of the Harvard math department, I’ll be there next Monday, on a panel with Moon Duchin talking about gender inclusivity in mathematics. It’s the first of a series. Here’s the Facebook page of the event.
Holy crap, you guys rock
Yesterday I wrote a post complaining that I didn’t know how to find an awesome job. The thing is, most advertised data science jobs either make rich people richer (finance) or make poor people poorer (ad tech) [1]
Well, I asked for advice, and you guys seriously delivered. I am so very lucky to have such amazing commenters and friends, and as a small token of my gratitude I want to compile some of the advice I got.
- A lot of you encouraged me to try to first figure out what I want to do and then convince a company doing that to give me a job. Great idea! Someone even sent me articles with useful advice on how to do that, here and here.
- Someone suggest I look for independent contract work by searching this list. Great idea.
- I got a few people writing to me to encourage me to consider teaming up with the Data Science for Social Good crowd. Maybe I should start a New York chapter?
- Someone had a friend who made a huge list her favorite toys and then wrote to the companies that made them telling them she’d be great as an employee, and it totally worked. “Toys” can be taken to be a general term, of course.
- Someone encouraged me to consider the environment and the team I’d be working with rather than the job I’d be doing. Trouble is I tried that, it didn’t work. But it might work for someone else.
- A bunch of people mentioned working for non-profits as data people. Non-profits have their challenges to work with but they seem to need the help. Hopefully I’ll have a guest post soon on this issue.
- Many people wrote to me with ideas for specific jobs I should apply for. I will, thanks!
- Also, a few people just wrote saying I’d be fine and they had hope for me. Those were really nice emails.
- Finally, quite a few other data scientists wrote saying they, too, want to make the world a better place and are frustrated by the lack of obvious chances to do this. Obviously I’m not alone.
Anyway, thanks to everyone for their advice and encouragement. I’ll keep you updated.
1. In fact I consider it my go-to example of how “the market” fails, if you think the market is supposed to offer profitable opportunities to do stuff that is worthwhile and/or important for society. It’s kind of the opposite, but maybe – hopefully – I’m defining the market too narrowly, i.e. by searching for data science jobs on LinkedIn.
What can a non-academic mathematician do that makes the world a better place?
You may have noticed the long-standing tagline on my About page:
What can a non-academic mathematician do that makes the world a better place?
I’ve been thinking about this question a lot lately. I’m looking for a job nowadays since my book project is wrapping up and I have no source of income. So far my attempts at sorting through the LinkedIn “data science” jobs are leading to a huge list of finance and online advertising jobs (which I don’t apply to), and a few others which are somewhat more interesting thrown in the mix. But they typically have more than 100 people each applying to them.
Which is to say, I’ve applied to a bunch of jobs I only kind of want and haven’t heard back from almost any of them. It’s kind of depressing. Actually it’s super depressing.
So I’ve come up with another way of thinking about searching. Why don’t I start with organizations that I think are doing cool things and offer them my data science expertise? In a consulting role primarily, but also longer-term if they are interested.
I know there are places where you can sign up to be an “expert witness,” so I’m looking for something similar: an expert consultant. There must already be ways to do this, but I don’t know how. Obviously one way would be to try to get a job at IBM, but then I’d be back to working for clients in finance.
Advice would be deeply appreciated. You can comment here or send me email at the address on my About page.
Big data, disparate impact, and the neoliberal mindset
When you’re writing a book for the general public’s consumption, you have to keep things pretty simple. You can’t spend a lot of time theorizing about why some stuff is going on, you have to focus on what’s happening, and how bad it is, and who’s getting screwed. Anything beyond that and you’ll be called a conspiracy theorist by some level of your editing team.
But the good thing about writing a blog is that you can actually say anything you like. That’s one reason I cling so strongly to mathbabe; I need to be able to write stuff that’s mildly conspiracy-theoretical. After all, just because you’re paranoid doesn’t mean nobody’s out to get you, right?
Anyhoo, I’m going to throw out a theory about big data, disparate impact, and the neoliberal mindset. First I need to set it up a bit.
Did you hear about this recent story whereby Facebook just got a patent to measure someone’s creditworthiness by looking at who their friends are and what their credit scores are? They idea is, you are more likely to be able to pay back your loans if the people you’re friends with pay back their loans.
On the one hand, it sounds possibly true: richer people tend to have richer friends, and so if there’s not very much information about someone, but that person is nevertheless inferred to be “friends with rich people,” then they might be a better bet for paying back loans.
On the other hand, it also sounds like an unfair way to distribute loans: most of us are friends with a bunch of people from high school, and if I happened to go to a high school filled with poor kids, then loans for me would be ruled out by this method.
This leads to the concept of disparate impact, which was beautifully explained in this recent article called When Big Data Becomes Bad Data (hat tip Marc Sobel). The idea is, when your process (or algorithm) favors one group of people over another, intentionally or not, it might be considered unfair and thus illegal. There’s lots of precedent for this in the courts, and recently the Supreme Court upheld it as a legitimate argument in Fair Housing Act cases.
It’s still not clear whether a “disparate impact” argument can be used in the case of algorithms, though. And there are plenty of people who work in the field of big data who dismiss this possibility altogether, and who even claim that things like the Facebook idea above are entirely legitimate. I had an argument on my Slate Money podcast last Friday about this very question.
Here’s my theory as to why it’s so hard for people to understand. They have been taken over in these matters by a neoliberal thought process, whereby every person is told to behave rationally, as an individual, and to seek maximum profit. It’s like an invisible hand on a miniature scale, acting everywhere and at all times.
Since this ideology has us acting as individuals, and ignoring group dynamics, the disparate impact argument is difficult if not impossible to understand. Why would anyone want to loan money to a poor person? That wouldn’t make economic sense. Or, more relevantly, why would anyone not distinguish between a poor person and a rich person before making a loan? That’s the absolute heart of how the big data movement operates. Changing that would be like throwing away money.
Since every interaction boils down to game theory and strategies for winning, “fairness” doesn’t come into the equation (note, the more equations the better!) of an individual’s striving for more opportunity and more money. Fairness isn’t even definable unless you give context, and context is exactly what this mindset ignores.
Here’s how I talk to someone when this subject comes up. I right away distinguish between the goal of the loaner – namely, accuracy and profit – and the goal of the public at large, namely that we have a reasonable financial system that doesn’t exacerbate the current inequalities or send people into debt spirals. This second goal has a lot to do with fairness and definitely pertains broadly to groups of people. Then, after setting that up, we can go ahead and discuss the newest big data idea, as long as we remember to look at it through both lenses.
Project Occupy
I’m very happy to announce that the wonderful high school students that we worked with over the summer at Occupy Summer School have decided to self-organize into an after-school program at their school, the UAI, starting this fall (if you don’t know what I’m talking about, go check out our webpage or the press we got from the New Yorker written by Alex Carp).
A few of us from the Alt Banking group had a meeting with 5 of them earlier this week. Together they represent the organizing committee of their new activism group, which they named “Project Occupy.” Their plan is to hold weekly afternoon meetings, which will be topic-based, and for a given week they want material for their chosen topic in the form of a short video or an article. One of the topics they already chose for one of the first meetings is “cultural appropriation.” They then named a bunch of examples of cultural appropriation of black culture. I’m planning to send them this for reading material.
They are, as always, very sharp and observant. Teenagers are the best.
I am super proud of them and I can’t wait to see what they do. I told them how much they’ve taught us about keeping protests fun and generous, and giving away gifts of food or balloons (actually, both) to make their point well-received.
They want to eventually plan events around the topics they’ve learned about and gotten passionate about, and possibly also create podcasts or YouTube video series. Oh, and they want food at all their meetings. Gotta keep things fun.
The Continuing War on Teachers
We all know about the achievement gap, whereby poor kids don’t do as well on standardized tests as rich kids. It’s big and it’s growing. Logically speaking, we might try to solve it – to close the achievement gap – by lowering inequality. But that’s a hard thing to do, politically. It would require things like higher taxes and better minimum wages and stronger safety nets.
So instead, politicians everywhere have decided to simply assign blame to the school teachers who the last people to be seen with poor kids before they take the standardized tests.
If you watched the Republican debate, you might have seen Chris Christie emphatically suggest that the teachers’ union should be punched in the face. He’s willing to repeat that:
Speaking of getting rid of teacher unions, that’s exactly what they did in New Orleans after Hurricane Katrina, 10 years ago. And the results are mixed:
In Washington D.C., they hired School Chancellor Michelle Rhee to fix their ailing schools, so she came in and fired a bunch of teachers with bad scores, and then gave bonuses to a bunch of teachers and schools with good ones, and then ignored the ensuing cheating scandal. After she left scores largely went back to normal. The achievement gap is wider than it was before Rhee:
None of this particularly surprises me because, again, it’s likely not a teacher-specific problem, but rather an economic problem, and residents of New Orleans are still poor.
It’s a correlation versus causation problem, actually. We know poor kids do badly on tests compared to rich kids, and we look for something we can control that would change that story. Originally we focused on more tests, thinking that shining a light on the problem would automatically solve it. That didn’t work, so we turned our focus to teachers, again mostly as an easily available knob to turn. It’s taking a few years for the data to come out that this new method also isn’t addressing the problem.
There’s a new idea afloat on how to close the achievement gap, by way of Florida (h/t Jordan Ellenberg). Namely, to give bonuses to teachers that themselves got good SAT scores. A few details:
- The bonus is $10,000
- The cut-off is 80th percentile of SAT scores
- Teachers are eligible even if they took the test 30 years ago
- The teachers also need to be rated “highly effective” to earn their bonus
- Except if they are first year teachers, in which case they don’t have a rating yet, so just their high SAT scores will do
It’s probably unnecessary to mention exactly how ridiculous this is, but I do want to point that teachers have no motivation whatsoever under this new scheme. They either qualify or they don’t. Not sure what it’s supposed to achieve in terms of carrots or sticks. It’s the equivalent of giving tall men extra money to buy suits.
I’m waiting for the meta-analysis that shows the achievement gap doesn’t respond to firing bad teachers, or charter schools, or even more testing, or any other easy target.
So what should we be doing? I have two suggestions, and neither of them is politically easy.
First, if we really wanted to see progress, we should stop persecuting teachers and immediately normalize the funding of school systems so poor kids have equal or more resources than rich kids. That would have some effects – at least poor districts could afford the latest books, for example.
Second, even that kind of progress would take us only so far. As long as a deep inequality of opportunities exists, and mobility is low, we can expect a lack of investment in poor people. We need to address these issues on the societal level.
Litigation finance: a terrible idea
I watch my share of bad commercials on TV. I don’t have cable but I have an antenna so I can receive some free TV stations, including one that is clearly meant for senior citizens called CoziTV. Cozi regularly has marathons of Murder She Wrote, Magnum P.I., Hart to Hart, and Fantasy Island, all shows that I somehow can’t stop watching, partly I think because I get so much confirmation from them about how awful my childhood was.
Anyhoo, back to the commercials. I couldn’t help notice a proliferation of ads for help with a medical condition called “transvaginal mesh injury.” Basically the ads were asking whether the viewer had such a problem, and whether they’d like to perhaps talk to a lawyer at this free phone number. Pretty much every other ad was about this condition, so it seemed like a pretty big deal, at least for the intended audience of old ladies.
Well, I didn’t pay much attention to it, until I came across this fascinating Reuters special report on corrupt medical lending practices entitled New breed of investor profits by financing surgeries for desperate women patients and written by Alison Frankel and Jessica Dye.
The article outlines the following scheme: financiers find women who need this surgery, based on a defective medical part, but don’t have the money for it and whose insurance companies won’t immediately cover it. They offer them the financing now in return for part of the eventual settlement with the company that was responsible for the faulty mesh. But then they make a deal with a surgeon to overcharge for the surgery, and they inflate the costs as well, and at the end of the whole thing they take a large part of the settlement which the woman was entitled to, and sometimes the woman even ends up owing them money.
It’s horrible, but it’s really just one example of a large industry of what is known as litigation finance, which just means the world where people with lots of money decide to bet on outcomes of court cases.
A recent Bloomberg article discussed the growing industry of litigation finance, and describes how congresspeople are starting to pay attention to all the hedge funds getting in on the act.
Of course, the financiers defend this practice by pointing out that, with money from people like them, a given side in a legal proceeding has more resources to make their case. They would also point out that they only put money behind cases that have at least some chance of winning.
However, there are two big problems with it as a concept. First, it means that there will be more money available to lawsuits in general. We’ve already seen what happened with college tuition when something that’s already too expensive gets access to loans: it gets even more expensive. It’s an arms race. According to Bloomberg, the typical client for these litigation finance firms are big companies which use corporate law firms.
Second, the justice system is already super unfair, and it’s not like hedge funds are running into this game to improve the system. Rather, they are there to exploit the system. That’s what hedge funds do. So if they have detected a bias in the system, they are going to treat it like an arbitrage situation and throw money at it for all they’re worth. The side effects of that will have nothing to do with justice.
The real reforms we need to see with the justice system is a way for money to be less of a factor, not more.
The inevitability of sexual assault
Whose fault is it when a woman gets sexually assaulted? For most people I interact with, the answer seems obvious: the assailant is at fault. Otherwise we’re blaming the victim.
In spite of that commonsense logic, though, there seems to be a sustained argument on the other side of the debate, and not only from right-wing talk radio. For example, over on the Guardian there’s quite a discussion about how The Pretenders star Chrissie Hynde blames herself for previous sexual assault, with the following excerpt pretty much summing up her position:
She said: “Technically speaking, however you want to look at it, this was all my doing and I take full responsibility. You can’t f*** about with people, especially people who wear ‘I Heart Rape’ and ‘On Your Knees’ badges … those motorcycle gangs, that’s what they do.
“You can’t paint yourself into a corner and then say whose brush is this? You have to take responsibility. I mean, I was naive.”
In Bangladesh, we similarly see arguments for why people marry their daughters off young which rely on the inevitability of sexual violence:
“I photographed the wedding of Akhi’s 13-year-old sister last year, and when I asked her mother why she was marrying her daughter off, she described not feeling comfortable to let her walk to the corner store because she would be harassed by men and boys,” Joyce said. “She also said no boy wants to marry a girl older than 18. If a girl is still single past that age people will ask too many questions. She knew it was wrong to marry very early, but they weren’t from a wealthy family, and she told her daughter’s husband to wear condoms for a few years, so it will be OK. Marriage is seen as a cover of respect and protection for women. By not going to school, it reduces the risk of being sexually active outside the house or be harassed while commuting.”
Don’t get me wrong, I think sexual violence is outrageous and wrong. The last thing I’d want to do is to suggest acquiescence in the face of it. I don’t want to blame victims or make things worse for them in any way. But we hear arguments like the above from reasonable, thoughtful people, who have plenty to lose by being wrong. They’re coming from personal experience which we should listen to. We should, in addition, examine why the “logical” argument doesn’t seem to work with them.
I have a theory. It’s about what our cultural expectations about men are. I’ll divide it into cases.
- Men can more or less control violent urges and are not inherently sexual predators. In this worldview, women of all ages should be free to wear whatever we want and go wherever we want and expect only consensual sexual interactions. This is an ideal world, which one might call civilized. Mind you I’m not even talking about being bombarded with idealized visions of sexualized femininity on billboards (but that would be nice too). I think I more or less live in this world and that means I’m lucky.
- Men are divided into two groups: sexual predators and “others.” Just as it would be silly to ask a lion not to kill an antelope when it’s hungry, and similarly it would be ridiculous to think that (some) men wouldn’t rape a woman if they had the chance. That’s along the lines of the worldview of Chrissie Hynde. If you really think that there just simply are men out there who are uncontrolled and uncontrollable sexual predators, then of course it’s up to the individual to steer clear of them. And it doesn’t make her wrong and us right, it just means we have different expectations of what men are like.
- No man can be trusted, they’re all trying to take advantage of women at all times, and harassment is to be expected. This does seem to be more or less the expectation in certain situations, like in the above quote from a mother in Bangladesh as well as all the communities torn apart by war and insecurity and desperate poverty. If someone faces this kind of reality, they have to work within it, even if it means marrying off their 13-year-old daughter before she’s ready or willing. It’s a bad choice versus a worse choice.










