Archive
Systematized racism in online advertising, part 1
There is no regulation of how internet ad models are built. That means that quants can use any information they want, usually historical, to decide what to expect in the future. That includes associating arrests with african-american sounding names.
In a recent Reuters article, this practice was highlighted:
Instantcheckmate.com, which labels itself the “Internet’s leading authority on background checks,” placed both ads. A statistical analysis of the company’s advertising has found it has disproportionately used ad copy including the word “arrested” for black-identifying names, even when a person has no arrest record.
Luckily, Professor Sweeney, a Harvard University professor of government with a doctorate in computer science, is on the case:
According to preliminary findings of Professor Sweeney’s research, searches of names assigned primarily to black babies, such as Tyrone, Darnell, Ebony and Latisha, generated “arrest” in the instantcheckmate.com ad copy between 75 percent and 96 percent of the time. Names assigned at birth primarily to whites, such as Geoffrey, Brett, Kristen and Anne, led to more neutral copy, with the word “arrest” appearing between zero and 9 percent of the time.
Of course when I say there’s no regulation, that’s an exaggeration. There is some, and if you claim to be giving a credit report, then regulations really do exist. But as for the above, here’s what regulators have to say:
“It’s disturbing,” Julie Brill, an FTC commissioner, said of Instant Checkmate’s advertising. “I don’t know if it’s illegal … It’s something that we’d need to study to see if any enforcement action is needed.”
Let’s be clear: this is just the beginning.
Whither the fake clicks?
My friend Ori explained to me last week about where all those fake clicks are coming from and going to and why. He blogged about it on m6d’s website.
As usual, it’s all about incentives. They’re gaming the online advertising model for profit.
To understand the scam, the first thing to know is that advertisers bid for placement on websites, and they bid higher if they think high quality people will see their ad and if they think a given website is well-connected in the larger web.
Say you want that advertising money. You set up a website for the express purpose of selling ads to the people who come to your website.
First, if your website gets little or no traffic, nobody is willing to bid up that advertising. No problem, just invent robots that act like people clicking.
Next, this still wouldn’t work if your website seems unconnected to the larger web. So what these guy have done is to create hundreds if not thousands of websites, which just constantly shovel fake people around their own network from place to place. That creates the impression, using certain calculations of centrality, that these are very well connected websites.
Finally, you might ask how the bad guys convince advertisers that these robots are “high quality” clicks. Here they rely on the fact that advertisers use different definition of quality.
Whereas you might only want to count people as high quality if they actually buy your product, it’s often hard to know if that has happened (especially if it’s a store-bought item) so proxies are used instead. Often it’s as simple as whether the online user visits the website of the product, which of course can be done by robots instead.
So there it is, an entire phantom web set up just to game the advertisers’ bid system and collect ad money that should by all rights not exist.
Two comments:
- First, I’m not sure this is illegal. Which is not to say it’s unavoidable, because it’s not so hard to track if you’re fighting against it. The beginning of a very large war.
- Second, even if there weren’t this free-for-the-gaming advertiser money out there, there’d still be another group of people incentivized to create fake clicks. Namely, the advertisers give out bonuses to their campaign-level people based on click-through rates. So these campaign managers have a natural incentive to artificially inflate the click-through rates (which would work against their companies’ best interest to be sure). I’m not saying that those people are the architects of the fake clicks, just that they have incentive to be. In particular, they have no incentive to fix this problem.
Money market regulation: a letter to Geithner and Schapiro from #OWS Occupy the SEC and Alternative Banking
#OWS working groups Occupy the SEC and Alternative Banking have released an open letter to Timothy Geithner, Secretary of the U.S. Treasury, and Mary Schapiro, Chairman of the SEC, calling on them to put into place reasonable regulation of money market funds (MMF’s).
Here’s the letter, I’m super proud of it. If you don’t have enough context, I give a more background below.
What are MMFs?
Money market funds make up the overall money market, which is a way for banks and businesses to finance themselves with short-term debt. It sounds really boring, but as it turns out it’s a vital issue for the functioning of the financial system.
Really simply put, money market funds invest in things like short-term corporate debt (like 30-day GM debt) or bank debt (Goldman or Chase short-term debt) and stuff like that. Their investments also include deposits and U.S. bonds.
People like you and me can put our money into money market funds via our normal big banks like Bank of America. In face I was told by my BofA banker to do this around 2007. He said it’s like a savings account, only better. If you do invest in a MMF, you’re told how much over a dollar your investments are worth. The implicit assumption then is that you never actually lose money.
What happened in the crisis?
MMF’s were involved in some of the early warning signs of the financial crisis. In August and September 2007, there was a run on subprime-related asset backed commercial paper.
In 2008, some of the funds which had invested in short-term Lehman Brother’s debt had huge problems when Lehman went down, and they “broke the buck”. This caused wide-spread panic and a bunch of money market funds had people pulling money from them.
In order to avoid a run on the MMF’s, the U.S. stepped in and guaranteed that nobody would actually lose money. It was a perfect example of something we had to do at the time, because we would literally not have had a functioning financial system given how central the money markets were at the time, in financing the shadow banking system, but something we should have figured out how to improve on by now.
This is a huge issue and needs to be dealt with before the next crisis.
What happened in 2010?
In 2010, regulators put into place rules that tightened restrictions within a fund. Things like how much cash they had to have on hand (liquidity requirements) and how long the average duration of their investments could be. This helped address the problem of what happens within a given fund when investors take their money out of that fund.
What they didn’t do in 2010 was to control systemic issues, and in particular how to make the MMF’s robust to large-scale panic.
What about Schapiro’s two MMF proposals?
More recently, Mary Schapiro, Chairman of the SEC, made two proposals to address the systemic issues. In the first proposal, instead of having the NAV’s set at one dollar, everything is allowed to float, just like every other kind of mutual fund. The industry didn’t like it, claiming it would make MMF’s less attractive.
In the second proposal, Schapiro suggesting that MMF managers keep a buffer of capital and that a new, weird lagged way for people to remove their money from their MMF’s, namely if you want to withdraw your funds you’ll only get 97%, and later (after 30 days) you’ll get 3% if the market doesn’t take a loss. If it does take a loss, will get only part of that last 3%.
The goal of this was to distribute losses more evenly, and to give people pause in times of crisis from withdrawing too quickly and causing a bank-like run.
Unfortunately, both of Schapiro’s proposals didn’t get passed by the 5 SEC Commissioners in August 2012 – it needed a majority vote, but they only got 2.
What happened when Geithner and Blackrock entered the picture?
The third, most recent proposal, comes out of the FSOC, a new meta-regulator, whose chair is Timothy Geithner. The guys proposed to the SEC in a letter dated September 27th that they should do something about money market regulation. Specifically, the FSOC letter suggests that either the SEC should go with one of Schapiro’s two ideas or a new third one.
The third one is again proposing a weird way for people to take their money out of a MMF, but this time it definitely benefits people who are “first movers”, in other words people who see a problem first and get the hell out. It depends on a parameter, called a trigger, which right now is set at 25 basis points (so 25 cents if you have $100 invested).
Specifically, if the value of the fund falls below 99.75, any withdrawal from that point on will be subject to a “withdrawal fee,” defined to be the distance between the fund’s level and 100. So if the fund is at 99.75, you have to pay a 25 cent fee and you only get out 99.50, whereas if the fund is at 99.76, you actually get out 100. So in other words, there’s an almost 50 cents difference at this critical value.
Is this third proposal really any better than either of Schapiro’s first two?
The industry and Timmy: bff’s?
Here’s something weird: on the same day the FSOC letter was published, BlackRock, which is a firm that does an enormous amount of money market managing and so stands to win or lose big on money market regulation, published an article in which they trashed Schapiro’s proposals and embellished this third one.
In other words, it looks like Geithner has been talking directly to Blackrock about how the money market regulation should be written.
In fact Geithner has seemingly invited industry insiders to talk to him at the Treasury. And now we have his proposal, which benefits insiders and also seems to have all of the unattractiveness that the other proposals had in terms of risks for normal people, i.e. non-insiders. That’s weird.
Update: in this Bloomberg article from yesterday (hat tip Matt Stoller), it looks like Geithner may be getting a fancy schmancy job at BlackRock after the election. Oh!
What’s wrong with simple?
Personally, and I say this as myself and not representing anyone else, I don’t see what’s wrong with Schapiro’s first proposal to keep the NAV floating. If there’s risk, investors should know about it, period, end of story. I don’t want the taxpayers on the hook for this kind of crap.
Bad news wish list
You know that feeling you get when, a few years after you went to a wedding of your friends, you find out they’re getting a divorce?
It’s not a nice feeling. It’s work for you, and nasty work at that: you have to go back over your memories of those two in the past years, where you’d been projecting happiness and contentment all this time, and replace it with argument and bitterness. Not to mention the sorrow and sympathy you naturally bestow on your friends.
If it happens enough times, which it has to me, then going to weddings at all is kind of a funereal affair. I no longer project happy thoughts towards the newly married couple. If anything I worry for them and cross my fingers, hoping for the best. You may even say I’ve lost my faith in the institution.
Considering this, I can kind of understand why some religions don’t allow divorce. If you don’t allow it, then the bad news will never come out, and you won’t have to retroactively fit your internal model of other people’s lives to reality. You can go on blithely assuming everyone’s doing great. While we’re at it, no kids are getting neglected or abused because we don’t talk about that kind of thing.
By way of unreasonable analogy, I’d like to discuss the lack of conversation we’ve seen by the presidential campaigns on both sides about the state of the financial system. I’m starting to think it’s part of the religion of politicians that they never talk about this stuff, because they treat it as an embarrassing failure along the lines of a catholic divorce.
Or maybe I don’t have to be so philosophical about it- is it religion, or is it just money?
I had trouble following much about the two national conventions, because it made me so incensed that nothing was really being discussed, and that it was all so full of shit. But one thing I managed to glean from the coverage of the “events” being sponsored by the various lobbyist groups at the two conventions is that, whereas most lobbyists sponsor events at one of the conventions, like the NRA sponsors something at the Republican convention and the unions sponsor stuff at the Democratic convention, the financial lobbyists sponsor huge swanky events at both.
I interpret this to mean that they are paying to not be discussed as a platform issue. They seem to have paid enough, because I don’t hear anything from the Romney camp about shit Obama has or hasn’t done, or shit Geithner has or hasn’t done.
In fact, there’s a “Stories I’d like to See” column in Reuters column entitled “Tales of a TARP built to benefit bankers, and waiting for CEOs to pay the price”, and written by Stephen Brill, which discusses this exact issue in the context of Neil Barofsky’s book Bailout, which I blogged about here. From the column:
A presidential campaign that wanted to call out the Obama administration for being too friendly to Wall Street and the banks at the expense of Main Street would be using Bailout as the cheat sheet that keeps on giving. But with the Romney campaign’s attack coming from the opposite direction – that the president and his team have killed the economy by shackling Wall Street – and with Romney on record in favor of allowing the mortgage crisis to “bottom out” with no government intervention, the former Massachusetts governor and his team have no use for Bailout.
The second half of the article is really good, asking very commonsensical question about the recent settlement BofA got from the SEC for blatantly lying to shareholders around the time they acquired Merrill Lynch. Specifically the author notes that the (current) shareholders are left paying the (2008) shareholders, which is dumb, but the asshole Ken Lewis, who actually lied doesn’t seem to be getting into any trouble at all. From the column:
And, as long as we’re talking about harm done to shareholders, why wouldn’t we now see a new, post-settlement shareholders’ suit not against the company but targeted only at Lewis and some of his former colleagues who got Bank of America into this jam in the first place and just caused it to pay out $2.4 billion? (The plaintiffs here could be any current shareholders, because they are the ones who are writing the $2.4 billion check.) Again, did the company indemnify Lewis and other executives against shareholder suits, meaning that if a shareholder now sues Lewis over this $2.4 billion settlement, the shareholder is once again only suing himself?
Can someone please sort this out?
I really like this idea, that we have a list of topics for people to sort out, even though it’s going to be bad news. What other topics should we ask for on our bad news wish list?
Why are the Chicago public school teachers on strike?
The issues of pay and testing
My friend and fellow HCSSiM 2012 staff member P.J. Karafiol explains some important issues in a Chicago Sun Times column entitled “Hard facts behind union, board dispute.”
P.J. is a Chicago public school math teacher, he has two kids in the CPS system, and he’s a graduate from that system. So I think he is qualified to speak on the issues.
He first explains that CPS teachers are paid less than those in the suburbs. This means, among other things, that it’s hard to keep good teachers. Next, he explains that, although it is difficult to argue against merit pay, the value-added models that Rahm Emanuel wants to account for half of teachers evaluation, is deeply flawed.
He then points out that, even if you trust the models, the number of teachers the model purports to identify as bad is so high that taking action on that result by firing them all would cause a huge problem – there’s a certain natural rate of finding and hiring good replacement teachers in the best of times, and these are not the best of times.
He concludes with this:
Teachers in Chicago are paid well initially, but face rising financial incentives to move to the suburbs as they gain experience and proficiency. No currently-existing “value added” evaluation system yields consistent, fair, educationally sound results. And firing bad teachers won’t magically create better ones to take their jobs.
To make progress on these issues, we have to figure out a way to make teaching in the city economically viable over the long-term; to evaluate teachers in a way that is consistent and reasonable, and that makes good sense educationally; and to help struggling teachers improve their practice. Because at base, we all want the same thing: classes full of students eager to be learning from their excellent, passionate teachers.
Test anxiety
Ultimately this crappy model, and the power that it yields, creates a culture of text anxiety for teachers and principals as well as for students. As Eric Zorn (grandson of mathematician Max Zorn) writes in the Chicago Tribune (h/t P.J. Karafiol):
The question: But why are so many presumptively good teachers also afraid? Why has the role of testing in teacher evaluations been a major sticking point in the public schools strike in Chicago?
The short answer: Because student test scores provide unreliable and erratic measurements of teacher quality. Because studies show that from subject to subject and from year to year, the same teacher can look alternately like a golden apple and a rotting fig.
Zorn quotes extensively from Math for America President John Ewing’s article in Notices of the American Mathematical Society:
Analyses of (value-added model) results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. (Value-added model) estimates have proven to be unstable across statistical models, years and classes that teachers teach.
One study found that across five large urban districts, among teachers who were ranked in the top 20 percent of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40 percent.
Another found that teachers’ effectiveness ratings in one year could only predict from 4 percent to 16 percent of the variation in such ratings in the following year.
The politics behind the test
I agree that the value-added model (VAM) is deeply flawed; I’ve blogged about it multiple times, for example here.
The way I see it, VAM is a prime example of the way that mathematics is used as a weapon against normal people – in this case, teachers, principals, and schools. If you don’t see my logic, ask yourself this:
Why would a overly-complex, unproved and very crappy model be so protected by politicians?
There’s really one reason, namely it serves a political function, not a mathematical one. And that political function is to maintain control over the union via a magical box that nobody completely understands (including the politicians, but it serves their purposes in spite of this) and therefore nobody can argue against.
This might seem ridiculous when you have examples like this one from the Washington Post (h/t Chris Wiggins), in which a devoted and beloved math teacher named Ashley received a ludicrously low VAM score.
I really like the article: it was written by Sean C. Feeney, Ashley’s principal at The Wheatley School in New York State and president of the Nassau County High School Principals’ Association. Feeney really tries to understand how the model works and how it uses data.
Feeney uncovers the crucial facts that, on the one hand nobody understands how VAM works at all, and that, on the other, the real reason it’s being used is for the political games being played behind the scenes (emphasis mine):
Officials at our State Education Department have certainly spent countless hours putting together guides explaining the scores. These documents describe what they call an objective teacher evaluation process that is based on student test scores, takes into account students’ prior performance, and arrives at a score that is able to measure teacher effectiveness. Along the way, the guides are careful to walk the reader through their explanations of Student Growth Percentiles (SGPs) and a teacher’s Mean Growth Percentile (MGP), impressing the reader with discussions and charts of confidence ranges and the need to be transparent about the data. It all seems so thoughtful and convincing! After all, how could such numbers fail to paint an accurate picture of a teacher’s effectiveness?
(One of the more audacious claims of this document is that the development of this evaluative model is the result of the collaborative efforts of the Regents Task Force on Teacher and Principal Effectiveness. Those of us who know people who served on this committee are well aware that the recommendations of the committee were either rejected or ignored by State Education officials.)
Feeney wasn’t supposed to do this. He wasn’t supposed to assume he was smart enough to understand the math behind the model. He wasn’t supposed to realize that these so-called “guides to explain the scores” actually represent the smoke being blown into the eyes of educators for the purposes of dismembering what’s left of the power of teachers’ unions in this country.
If he were better behaved, he would have bowed to the authority of the inscrutable, i.e. mathematics, and assume that his prize math teacher must have had flaws he, as her principal, just hadn’t seen before.
Weapons of Math Destruction
Politicans have created a WMD (Weapon of Math Destruction) in VAM; it’s the equivalent of owning an uzi factory when you’re fighting a war against people with pointy sticks.
It’s not the only WMD out there, but it’s a pretty powerful one, and it’s doing outrageous damage to our educational system.
If you don’t know what I mean by WMD, let me help out: one way to spot a WMD is to look at the name versus the underlying model and take note of discrepancies. VAM is a great example of this:
- The name “Value-Added Model” makes us think we might learn how much a teacher brings to the class above and beyond, say, rote memorization.
- In fact, if you look carefully you will see that the model is measuring exactly that: teaching to the test, but with errorbars so enormous that the noise almost completely obliterates any “teaching to the test” signal.
Nobody wants crappy teachers in the system, but vilifying well-meaning and hard-working professionals and subjecting them to random but high-stakes testing is not the solution, it’s pure old-fashioned scapegoating.
The political goal of the national VAM movement is clear: take control of education and make sure teachers know their place as the servants of the system, with no job security and no respect.
Fair versus equal
In this multimedia presentation, Alan Honick explores the concept of fairness with archaeologist Brian Hayden. It’s entitled “The Evolution of Fairness”, and it’s published by Pacific Standard Magazine.
It’s a series of small writings and short videos which studies evidence of the emergence of inequality in the archaeological record of fishing at a place called Keatley Creek in British Columbia. While it isn’t the most convenient thing to go through, it’s worth the effort. Here are the highlights for me:
When the main concern of the people living at Keatley Creek was subsistence, their society was egalitarian – they shared everything and it wasn’t okay to hoard. Specifically, anyone found trying to game the system was ejected from society, which typically meant death.
As fishing technology improved, the average person could provide for themselves in normal times quite easily, and private ownership became acceptable and common. Those who game the system were no longer ejected, partly because the definitions were different.
At this point, Hayden suggests, people began to do things in small groups that seemed perfectly fair (“I’ll give you 20 fish loaves if you let me marry your daughter” or “Come to my feast tonight and invite me to your feast next week”) and moreover seemed like a private arrangement, until it became sufficiently widespread so that two things happened:
- The guys who didn’t have or couldn’t borrow 20 fish loaves couldn’t get married, or similarly the guys who couldn’t afford to serve a feast never entered into the feast-sharing ritual, and
- The truly rich guys would sometimes have a feast for everyone, which meant the poorer would “get something for nothing” and everyone would gain. Another way of saying this is that the poorer people would allow themselves to be coopted into the unequal system by the price of this free food. Those people who didn’t give feasts or cooperate with the free feasts were outcasts.
An interesting thing happened when Hayden goes to villages in the Mayan Highlands in Mexico and Guatemala which has similar size and social structure as the one on Keatley Creek (see the video on this page). He interviewed people about how the “rich” behaved in times of starvation. Did they take on a managerial role? Did they share and help out in bad times? This is referred to as “communitarian”.
Turns out, no, they exploited the people in the village in the hopes of having better status by the time things got better. They sold maize at exorbitant prices, took outrageous amounts of land for maize, etc. The driving force was individual self-interest.
The overall narrative describes the shifting definition of fairness as things became less and less equal, and how eventually the elite, who essentially got to define fairness, didn’t need to listen to the objections of the poor at all, because they had no power.
Sound familiar?
The author Alan Honick concludes by looking at our society and asks whether campaign finance laws, and Citizens United, is that different in effect from what we saw happening on Keatley Creek. He also points out that, because we humans are so individually obsessed with increasing our status, we can’t seem to get together to address really important issues such as global warming.
NSA mathematicians
When I was a promising young mathematician in college, I met someone from the NSA who tried to recruit me to work for the spooks in the summer. Actually, “met someone” is misleading- he located me after I had won a prize.
I didn’t know what to think, so I accepted his invitation to visit the institute, which was in La Jolla, in Southern California (I went to UC Berkeley so it wasn’t a big trip).
When I got to the building, since I didn’t have clearance, everybody had to stop working the whole time I was there. It wasn’t enough to clean their whiteboards, one of them explained, they had to wash them down with that whiteboard spray stuff, because if you look at a just-erased whiteboard in a certain way you can decipher what had been written on it.
I met a bunch of people, maybe 6 or 7. They all told me how nice it was to work there, how the weather was beautiful, how the math problems were interesting. It was strangely consistent, but who knows, perhaps also true.
One thing I’d already learned before coming is that there are many layers of work that happen before the math people in La Jolla are given problems to do. First, the actual problem is chosen, then the “math” of the problem is extracted from the problem, and third it’s cleansed so that nobody can tell what the original application is.
Knowing this (and I was never contradicted when I explained that process), I asked each of them the same question: how do you feel about the fact that you don’t know what problem you’re actually solving?
Out of the 6 or 7 people I met, everyone but one person responded along the lines, “I believe everything the United States Government does is good.” The last guy said, “yeah, that bothers me. I am honestly seriously considering leaving.”
Needless to say, I didn’t take the job. I wasn’t yet a major league skeptic, but I was skeptical enough to realize I could not survive in such an environment, with colleagues that oblivious. They also mentioned that I’d have to stop dating my Czech boyfriend and that I’d need to submit information about all my roommates for the past 10 years, which was uber creepy.
Nowadays I hear estimates that 600 mathematicians work at the NSA, and of course many more stream through during the summer when school’s not in session, both at La Jolla and Princeton. Somehow they don’t mind not knowing how their work actually gets used. I’m not sure how that’s possible but it clearly is.
This mindset came back to me, and not in a good way, when I read this opinion piece and watched this video in the New York Times a couple of days ago.
William Binney, a mathematician, was working on Soviet Union spying software that got converted to domestic spying after 9/11. In other words, they used his foreign spying algorithm on a new data source, namely American citizen’s raw data. He objected to that, so strongly that he’s come out against it publicly.
The big surprise is how come they let him know what they were actually up to. My guess is he was high enough up the chain that they thought he’d be okay with it – he’d been there 32 years, and I guess he was considered an insider.
In any case, watch the video: this is a courageous man. The FBI came into his house with guns drawn to intimidate him against his whistleblowing activities and yet he hasn’t been cowed. Indeed, after getting dressed (he was coming out of the shower when they exploded into his house), he explained to them the crimes of George Bush and Dick Cheney on his back porch.
As he explains, “the purpose is to monitor what people are doing”. He explains how people’s social media data and other kinds of data are linked over domains and over time to build profiles of Americans over time: “you have 10 years of their life that you can lay out in a timeline, that involves anybody in the country”.
Describing the dangers of this program, Binney was extremely articulate:
- “The danger here is that we could fall into a totalitarian state like East Germany”
- “We can’t have secret interpretations of laws and run them in secret and not tell anybody. We can’t make up kill lists and not tell anybody what the criterion is for being on the kill list”
- “Just because we call ourselves a democracy doesn’t mean we will stay that way.”
There you have it. The good news is that that guy is no longer helping the NSA do their thing.
But the bad news is, plenty of mathematicians still are. And if you want to find a community more trusting and loyal than mathematicians, I think you’d have to go to a kindergarten somewhere. Not to mention the fact that, as I described above, the problems are intentionally cleaned to look innocuous.
Another example, possibly the most important one of all, of mathematics being manipulated to potentially evil ends. We will have trouble proving actual evil consequences, of course, since there’s no transparency. The only update we will get is via the next whistleblower who can handle guns pointed at him as he leaves his shower.
Someone didn’t get the memo about regulatory capture
So there’s this guy named Benjamin Lawsky, and he’s the New York State Superintendent of Financial Services. Last week he blew open a case against a British bank named Standard Chartered for money laundering and doing business with Iran.
The other regulators don’t like his style one bit, even though he managed to force Standard Chartered to pay $340 million for their misdeeds, as well as look like bad guys. I’ll get back to why the other regulators are pissed but first a bit more on the settlement.
What’s not cool about a fine is that nobody goes to jail and they continue business as usual, hopefully without the money laundering (their stock has mostly recovered as well).
What is cool about the $340 million fine is that it took almost no time compared to other settlements with banks (a nine month investigation before the blowup last week) and that it’s actually pretty big – bigger, for example, then the proposed settlement SEC is making with Citigroup which judge Rackoff blocked for shorting their clients in 2008 and not admitting wrongdoing.
In this case of Standard Chartered, they may not be admitting wrongdoing but we’ve all already read the evidence, as well as the smoking gun email:
The business chugged along even after the banking unit’s chief executive in the Americas warned in a 2006 memo that the company and its management might be vulnerable to “catastrophic reputational damage” and “serious criminal liability.”
According to the regulatory order, a bank official in London replied: “You f- Americans. Who are you to tell us, the rest of the world, that we’re not going to deal with Iranians.”
[Aside: do you think, being a polite Brit, that this guy actually wrote “f-” in his email?]
Back to the other regulators. They are so used to working for the banks, it is inconceivable to them to publicize damning evidence before giving the heads up to the bank in question looking for a quiet settlement. That’s the way they do things. And then they never get much money, and nobody ever goes to jail. Oh, and it takes forever.
They argue that this is because they don’t have enough resources to go the distance with lawyers, but it’s also because their approach is so weak.
So naturally they’ve been pretty upset that Lawsky has balls when they don’t, especially since he doesn’t have nearly the resources that the SEC has.
My favorite ridiculous argument against Lawsky and his approach came from this article I read yesterday on Reuters. It stipulates that Lawsky is creating an environment where there’s a possibility of regulatory arbitrage. From the article:
But a central lesson of the financial crisis was the need for regulators to better cooperate and share information. Working at cross purposes creates opportunities for what’s known as “regulatory arbitrage,” whereby banks circumvent regulations by exploiting rivalries among their various overseers.
Um, what? That whole mindset is clearly off.
The goal would be the regulators get to decide who’s the bad guy, not the banks. And don’t tell me loopholes in the regulatory structure are introduced by having a regulator willing to do his job without sucking everybody’s dick first. Please.
And if I’m a regulator, and if it would work better to share my information with Lawsky to do my job as a regulator, you better believe I’m willing to share it with him if I can get credit alongside him for exposing illegal activities. That is, if I really want to expose illegal activities.
High frequency trading: does it hurt the little guy?
I’ve already written about high frequency trading here, and I came out in favor of a transaction tax to slow that shit down a little bit. After all, the argument that liquidity is good so more liquidity is better only holds to a point – we don’t need infinite liquidity. It makes sense to actually have a small barrier to trade – you actually have to think it’s a good idea one way or another, otherwise you have no incentive not to do something dumb.
And as we’ve seen recently with Knight Capital, dumb things definitely are likely to happen.
It’s been interesting to see the media reaction. On the one hand, the Room for Debate over at the New York Times has a bunch of people discussing high frequency trading (HFT), and the most pro-HFT guy essentially says that the SEC should keep up technology-wise with these guys, and everything will be ok. That’s called living in a fantasy world.
More interesting to me was Felix Salmon’s post yesterday, where he rightly complained that, all too often, journalists dumb down and simplify reporting on these things, and then he proceeds to dumb down and simplify reporting on this thing.
Specifically, he complains that no “little guys” were hurt in Knight’s crash, even though the press is always looking for the little guy that gets hurt. [Side note: he also complains about the LIBOR manipulation not hurting municipalities, which is false, it did hurt them. He needs to understand that better before he dismisses it.]
But, if I’m not dreaming, Fidelity was one of the large customers of Knight that’s pulled out, and if I’m not unconscious, Fidelity manages quite a few of my many 401K accounts, as well as a huge proportion of the 401K accounts in this country. So it’s quite possible that my retirement money was part of that massive screw-up which is now owned by Goldman Sachs, not that I’ve been notified by Fidelity of any harm (but that’s another post).
As for small investors vs. little guys, there’s a difference. If you have enough money that you’re investing it through brokers, I personally don’t count you as small, even if you appear small to Goldman Sachs. So I’m not interested in whether the small investor was all that harmed by Knight’s meltdown, but I’m pretty sure the small investor was scared away by it.
But looking at the larger picture, I’d definitely say this is an indication of the outrageous complexity of the financial system, which most definitely is hurting the little guy, i.e. the taxpayer. This complexity is why we have the government guarantee in place, the Too-Big-and-Too-Complex-To-Fail banks and markets, and the little guy on the hook when things melt down. Moreover, there’s a direct line from that whole mess to the destruction of unions and pension programs, even if people don’t want to draw it.
So if you want to be myopic you can say that this was one firm, making one major blunder, and it’s self-contained and that firm is failing just like it should. But if you take a step back you see they were doing this as part of a larger culture of competition for speed and technology that they are so focused on, they threw risk to the wind in order to achieve a tiny edge over NYSE.
That laser focus on having a tiny edge really is the underlying story, and will continue to be, at the expense of risk, at the expense of our retirement funds trading for us, without regard to unnecessary complexity or, yes, the little guy, until our politicians and regulators grow some balls and put an end to it.
I love whistleblowers
There’s something people don’t like about whistleblowers. I really don’t get it, but I know it’s true (I’m looking at you, Obama).
In particular, I hear all the time that you’re giving up on your career if you’re a whistleblower, that nobody would ever want to hire you again. But if I’m running a company, which I presumably want have run well, without corruption, and be successful, then I’m totally fine with whistleblowers! They will tell me truth and expose fraud. To say out loud that I don’t want to hire someone like that is basically admitting I’m okay with fraud, no?
I’m really missing something, and if you have an explanation I’d love to hear it.
In the meantime, though, I’ll say this: the web is great for anonymous whistleblowing (if anyone pays attention and follows up). Science Fraud is a great one that tells about scientific publishing fraud in the life sciences – see the “About” page of Science Fraud for more color. See also Retraction Watch for a broader look.
But then there’s another issue, which is that some people won’t seriously consider whistleblowers unless they identify themselves! What up? Facts are facts – if someone has given good evidence that can be checked independently, why should also submit themselves to being blacklisted for their efforts?
Here’s a good response to this crappy line of reasoning against anonymous whistleblowing by the Retraction Watch guys Ivan Oransky and Adam Marcus.
Columbia Data Science Institute: it’s gonna happen
So Bloomberg finally got around to announcing the Columbia Data Science Institute is really going to happen. The details as we know them now:
- It’ll be at the main campus, not Manhattanville.
- It’ll hire 75 faculty over the next decade (specifically, 30 new faculty by launch in August 2016 and 75 by 2030, so actually more than a decade but who’s counting?).
- It will contain a New Media Center, a Smart Cities Center, a Health Analytics Center, a Cybersecurity Center, and a Financial Analytics Center.
- The city is pitching in $15 million whereas Columbia is ponying up $80 million.
- Columbia Computer Science professor Kathy McKeown will be the Director and Civil Engineering professor Patricia Culligan will be the Institute’s Deputy Director.
Does mathematics have a place in higher education?
A recent New York Times Opinion piece (hat tip Wei Ho), Is Algebra Necessary?, argues for the abolishment of algebra as a requirement for college. It was written by Andrew Hacker, an emeritus professor of political science at Queens College, City University of New York. His concluding argument:
I’ve observed a host of high school and college classes, from Michigan to Mississippi, and have been impressed by conscientious teaching and dutiful students. I’ll grant that with an outpouring of resources, we could reclaim many dropouts and help them get through quadratic equations. But that would misuse teaching talent and student effort. It would be far better to reduce, not expand, the mathematics we ask young people to imbibe. (That said, I do not advocate vocational tracks for students considered, almost always unfairly, as less studious.)
Yes, young people should learn to read and write and do long division, whether they want to or not. But there is no reason to force them to grasp vectorial angles and discontinuous functions. Think of math as a huge boulder we make everyone pull, without assessing what all this pain achieves. So why require it, without alternatives or exceptions? Thus far I haven’t found a compelling answer.
For an interesting contrast, there’s a recent Bloomberg View Piece, How Recession Will Change University Financing, by Gary Shilling (not to be confused with Robert Shiller). From Shilling’s piece:
Most thought that a bachelor’s degree was the ticket to a well-paid job, and that the heavy student loans were worth it and manageable. And many thought that majors such as social science, education, criminal justice or humanities would still get them jobs. They didn’t realize that the jobs that could be obtained with such credentials were the nice-to-have but nonessential positions of the boom years that would disappear when times got tough and businesses slashed costs.
Some of those recent graduates probably didn’t want to do, or were intellectually incapable of doing, the hard work required to major in science and engineering. After all, afternoon labs cut into athletic pursuits and social time. Yet that’s where the jobs are now. Many U.S.-based companies are moving their research-and-development operations offshore because of the lack of scientists and engineers in this country, either native or foreign-born.
For 34- to 49-year-olds, student debt has leaped 40 percent in the past three years, more than for any other age group. Many of those debtors were unemployed and succumbed to for-profit school ads that promised high-paying jobs for graduates. But those jobs seldom materialized, while the student debt remained.
Moreover, many college graduates are ill-prepared for almost any job. A study by the Pew Charitable Trusts examined the abilities of U.S. college graduates in three areas: analyzing news stories, understanding documents and possessing the math proficiency to handle tasks such as balancing a checkbook or tipping in a restaurant.
The first article is written by a professor, so it might not be surprising that, as he sees more and more students coming through, he feels their pain and wants their experience to not be excruciating. The easiest way to do that is to remove the stumbling block requirement of math. He also seems to think of higher education as something everyone is entitled to, which I infer based on how he dismisses vocational training.
The second article is written by a financial analyst, an economist, so we might not be surprised that he strictly sees college as a purely commoditized investment in future income, and wants it to be a good one. The easiest way to do that is to have way fewer students go through college to begin with, since having dumb or bad students get into debt but not learn anything and then not get a job afterwards doesn’t actually make sense.
And where the first author acts like math is only needed for a tiny minority of college students, the second author basically dismisses non-math oriented subjects as frivolous and leading to a life of joblessness and debt. These are vastly different viewpoints. I’m thinking of inviting them both to dinner to discuss.
By the way, I think that last line, where Hacker wonders what the pain of math-as-huge-boulder achieves, is more or less answered by Shilling. The goal of having math requirements is to have students be mathematically literate, which is to say know how to do everyday things like balancing checkbooks and reading credit card interest rate agreements. The fact that we aren’t achieving this goal is important, but the goal is pretty clear. In other words, I think my dinner party would be fruitful as well as entertaining.
If there’s one thing these two agree on, it’s that students are having an awful lot of trouble doing basic math. This makes me wonder a few things.
First, why is algebra such a stumbling block? Is it that the students are really that bad, or is the approach to teaching it bad? I suspect what’s really going on is that the students taking it have mostly not been adequately taught the pre-requisites. That means we need more remedial college math.
I honestly feel like this is the perfect place for online learning. Instead of charging students enormous fees while they get taught high-school courses they should already know, and instead of removing basic literacy requirements altogether, ask them to complete some free online math courses at home or in their public library, to get them ready for college. The great thing about computers is that they can figure out the level of the user, and they never get impatient.
Next, should algebra be replaced by a Reckoning 101 course? Where, instead of manipulating formulas, we teach students to figure out tips and analyze news stories and understand basic statistical statements? I’m sure this has been tried, and I’m sure it’s easy to do badly or to water down entirely. Please tell me what you know. Specifically, are students better at snarky polling questions if they’ve taken these classes than if they’ve taken algebra?
Finally, I’d say this (and I’m stealing this from my friend Kiri, a principal of a high school for girls in math and science): nobody ever brags about not knowing how to read, but people brag all the time about not knowing how to do math. There’s nothing to be proud of in that, and it’s happening to a large degree because of our culture, not intelligence.
So no, let’s not remove mathematical literacy as a requirement for college graduates, but let’s think about what we can do to make the path reasonable and relevant while staying rigorous. And yes, there are probably too many students going to college because it’s now a cultural assumption rather than a thought-out decision, and this lands young people in debt up to their eyeballs and jobless, which sucks (here’s something that may help: forcing for-profit institutions to be honest in advertising future jobs promises and high interest debt).
Something just occurred to me. Namely, it’s especially ironic that the most mathematically illiterate and vulnerable students are being asked to sign loan contracts that they, almost by construction, don’t understand. How do we address this? Food for thought and for another post.
Income distributions and misleading poll questions (#OWS)
Disingenuous, pseudo-quantitative arguments piss me off.
In this recent Bloomberg View article entitled “Making the rich poorer doesn’t enrich the middle class,” Caroline Baum argues that middle class people would rather get more money than take away money from rich people. From the article:
Polling by the Pew Research Center shows that people aren’t interested in taking money from the wealthy. They just want a chance to get rich themselves.
But that’s a misleading question. It seems like a zero sum game when you put it that way, equivalent to something like, “Would you rather gain $100 or have a rich person somewhere lose $100?”.
But if you pose the question differently, and more in line with actual numbers, not to mention contextualized to reality in other ways, then you’d probably get the opposite.
Let’s take a look at wealth distribution from 2007, which I got here:
Let’s just say we’re being extreme and we take away all the wealth of the top 1% and give it to everybody equally (say we even give back some of it to those top 1%). That would mean that 34.6% get flattened out to 100 pots instead of one, which means that each of those percentiles gets about 0.35% more than they used to have. The middle 20% would grow from 4% of the overall wealth to (4 + 20*0.35)% = 11%. That’s still a lot less than 20%, but the wealth of the middle 20% is still nearly tripled by just this one percent re-distributing.
Said another way, it’s not tit-for-tat at all.
If we asked someone in the middle class which they want more, a 1% increase in their wealth or a top 1%’er to lose 1% of their wealth, then that might be very different. Consider the political influence that 1% represents, at the very least. Consider the fact that 1% of that person in the middle 20% is 173 times smaller than for the top 1%.
It’s still not fair, though, because the middle class is so squeezed on necessities like food, housing, education, medical expenses, and child care, that they can’t afford even a 1% loss. What if you took those out?
If you go even further and ask someone in the middle class which they want more, a 1% increase in their discretionary income or a top 1%’er to lose 1% of their discretionary income, then that might be very different still. I haven’t been able to find a similar graphic to work with to see the discretionary income distribution, but rest assured it’s even more unbalanced.
Caroline Baum, would you care to cover those questions on your next poll to the middle class?
Why is LIBOR such a big deal? (#OWS)
The manipulation of LIBOR interest rates by the big, mostly-European banks (but not entirely, see a full list here) was an open secret inside finance in 2008. As in so open that I didn’t think of it as a secret at all.
The fact that that manipulation is now consistently creating huge headlines is interesting to me – it brings up a few issues.
- People seem surprised this out-and-out manipulation was happening. That says to me that they clearly still don’t understand what the culture of finance is really like. The fact that Bob Diamond of Barclays claims to have felt “physically ill” when he saw the emails of the traders manipulating LIBOR is either an out-and-out lie or they guy is simple-minded, as in stupid. And word on the street is he’s not stupid.
- People still buy the line that most of the problems from the credit crisis arose from legal but wrong-headed efforts to make money, plus corrupt ratings on mortgage-backed securities. This is incredible to me. Let’s get it clear: the culture of finance is to take advantage of every opportunity to juice your bottom line, even if it’s wrong, even if it’s fraudulent, even if it affects the terms of loans on millions of houses and towns in other countries, and even if only your trading desk is benefiting.
- The LIBOR manipulation in 2008 was about more than that, namely trying not to look as bad as other banks, to avoid being the next Lehman. It was done in the name of not looking weak and requiring a government bailout. Bob Diamond still doesn’t think they did anything wrong by lying there. It was almost like they were doing something noble.
- Speaking of towns in other countries, read this article about how LIBOR manipulation has screwed U.S. cities to the ground. I’ve got a lot more to say about municipal debt and how that sleazy system works but it’s waiting for another post.
- Finally, why did it take so long for the media to pick up on LIBOR manipulation? It tempts me to make a list of the illegal stuff that we all knew about back then and send it around just to make sure.
Why are pharmaceutical companies allowed to do their own trials?
A recent New York Times article clearly addressed the problem with big pharma being in charge of its own trials. In this case it was Pfizer doing a trial for Celebrex, but I previously wrote about Merck doing corrupt trials for Vioxx (see How Big Pharma Cooks Data: The Case of Vioxx and Heart Disease). In the article, it has the following damning evidence that this practice is ludicrous:
- Research Director Dr. Samuel Zwillich, in an email after a medical conference discussing Celebrex, stated: “They swallowed our story, hook, line and sinker.”
- Executives considered attacking the trial’s design before they even knew the results. “Worse case: we have to attack the trial design if we do not see the results we want,” a memo read. It went on: “If other endpoints do not deliver, we will also need to strategize on how we provide the data.” This simply can’t happen. There should be an outside third-party firm in charge of trial design, and there needs to be sign-off on the design in advance so no monkey business like this takes place.
- Executives disregarded the advice of an employee and an outside consultant who had argued the companies should disclose the fact that they were using incomplete data – they were using only half. This kind of statistical dishonesty is the easiest way to get numbers you want.
- In another email, associate medical director Dr. Emilio Arbe from Pharmacia (which was later bought by Pfizer) disparaged the way the study was being presented as “data massage,” for “no other reason than it happens to look better.” Mind you, this statement was made in September 2000, so in other words the side effects of Celebrex have been known for over a decade.
- Medical Director Dr. Mona Wahba described it as “cherry-picking” the data. In May 2001.
Why is this happening? It’s all about money:
It is one of the company’s best-selling drugs, racking up more than $2.5 billion in sales, and was prescribed to 2.4 million patients in the United States last year alone.
How much are you in doubt that the people in charge are being pressured not to be honest? Dr. Samuel Zwillich claims the hook, line and sinker statement was probably about something else. The cherry-picking Dr. Mona Wahba now can’t remember what she meant.
This is bullshit, people. Statistics is getting a bad name, and people are suffering and dying from bad medicine, not to mention paying way too much for fancy meds that don’t actually help them more than aspirin.
What we need here is some basic integrity. And it’s not just a few bad eggs either – stay tuned for a post on Prof. David Madigan’s recent research on the robustness of medical trials and research in general.
Who should be on the Fed Bank Boards? #OpenFed
Please consider this a civic duty: nominate someone for one of the Fed Bank Boards.
Why? Because:
- these guys regulate the banks,
- they also decide on and implement monetary policy (things like interest rates),
- in times of trouble they also help bail out banks (think Tim Geithner, Lehman, and Bear Stearns)
- the current process is an old boy’s network.
If you haven’t been living under a rock, you might have heard that Jamie Dimon, the CEO of JP Morgan Chase, sits on the NY Fed Board. There have been a number of calls for his resignation or removal. But as Jonathan Reiss points out in this excellent Huffington Post piece, even better than removing Jamie Dimon and leaving it at that would be to call for all of the Fed Boards to be populated with people who represent the interests of 99% rather than their own narrow business interests. From his article:
…rather than complaining about individual cases, we should fix the process that appointed Dimon and will appoint his successor and 35 other directors to 3-year terms starting January 2013. There are systemic problems with how the directors are selected. The Government Accountability Office studied the bank boards and found they were neither diverse nor representative of the public despite a mandate requiring it. If we work now, this process can be greatly improved.
Did you hear that? The rules already stipulate diverse boards! From a Time article which picked up Reiss’s:
The fact is, the 1913 law creating the central bank was structured to avoid these conflicts. The Federal Reserve System is made up of 12 regional banks, each with nine board members — three of each of three “classes,” A, B, and C. Class A directors are to be from the banking industry and represent large, medium-size, and small banks. Both Class B and Class C directors are supposed to represent non-banking interests — labor, consumers, agriculture, and the like. But bankers select the Class B directors, and the governors of the Federal Reserve select the Class C members, in theory to help ensure their complete independence from the banking industry.
How well does the theory work? Take a look at the list of people on the NY Fed Board, in class C (ignoring classes A and B for now):
|
Lee C. Bollinger (bio) Chair, 2012 |
 |
|
|
| Kathryn S. Wylde (bio) Deputy Chair, 2013 President and Chief Executive Officer Partnership for New York City |
|
|
|
| Emily K. Rafferty (bio), 2014 President The Metropolitan Museum of Art |
A bit of background on Kathryn Wylde can be found here, where she was quoted defending Wall Street and trying to shame someone else into doing the same; the article calls for her resignation from the NY Fed. All three of them: Lee Bollinger, Kathy Wylde, and Emily Rafferty, are professional fundraisers. Which means they grovel at the feet of rich people (read: bankers) for a living. This is not the definition of representing “non-banking interests — labor, consumers, agriculture, and the like” I would come up with. In fact if I came up with a definition, there’d be a “no ass-kissing” stipulation.
Is this a problem just for the NY Fed? And why is it happening? According to Reiss:
Dodd-Frank commissioned a study of the bank boards of directors by the GAO. They found in 2010 of the 108 directors, only 5 represented consumers. Agriculture and food processing was better represented. Curiously, the GAO says that several reserve banks said it was “challenging” to find qualified consumer representatives who are interested in these positions. They attributed this to low pay (relative to corporate boards), restrictions on political activity and the requirement that they divest themselves of bank stock holdings. But I find it hard to believe that is the problem.
This is where you come in.
Reiss wants you to nominate qualified people for the local Fed Boards. He’ll compile the list and send them on to the reserve banks, since they seem to be having trouble finding qualified consumer advocates (for whatever reason they are only friends with rich bankers and their fans).
Some good news, the turnover is pretty high: the terms are three years, staggered, which means all 12 Fed Banks make 3 new appointments every year, and by custom nobody serves more than 2 terms. That means that within 6 years we could have a fairly representative board in each Fed if we do this right.
Tweet your nomination to the hashtag #OpenFed.
Jamie Dimon gets a happy ending massage at Banking Committee hearing
Do you guys remember how last week the #OWS Alternative Banking group, along with Occupy the SEC, wrote up some questions for Jamie Dimon, the CEO of JP Morgan Chase?
Yves Smith also posted those Alt Banking questions on Naked Capitalism, along with some commentary about the senators who were expected to be asking questions of Dimon. Among other things, she mentioned their connections to Wall Street money:
…this is likely to be at most a ritual roughing up. First, the hearing is only two hours, and that includes the usual pontificating at the start of the session. By contrast, Goldman executives were raked over the coals for 10 hours over their dubious collateralized debt obligations. The comparatively easy treatment is no doubt related to the fact that JP Morgan is a major contributor to the five most senior committee members. Per American Banker:
JPMorgan is Banking Committee Chairman Tim Johnson’s second-largest contributor over the last two-plus decades, according to the Center for Responsive Politics, which analyzes campaign giving from companies’ employees and their political action committees since 1989. The same is true for the committee’s top Republican, Sen. Richard Shelby, and its second-ranking Democrat, Sen. Jack Reed.
The committee’s number-two Republican, Sen. Mike Crapo, and its third-ranking Democrat, Sen. Charles Schumer, are not far behind their colleagues, with JPMorgan ranking third and fourth, respectively, among their contributors.
Second is that the format of these hearings, with each Senator getting only five minutes each per witness, makes it difficult for a questioner to pin an evasive or clever witness. It won’t be hard for Dimon to either run out the clock or bamboozle his interrogators. But he might, as he did in his hastily-called press conference announcing the losses, make more admissions to the effect that he and senior management weren’t on top of what the group was doing. That would support the notion that JP Morgan’s risk controls were inadequate, which would mean that Dimon’s Sarbanes Oxley certification for 2011, and potentially earlier years, was false.
Who wants to know what actually happened? To find out, go read Matt Taibbi’s Rolling Stones excellent article on the hearing, which is very much in line with what Yves predicted. One of my favorite lines from the article:
This is a guy who just committed a massive blunder with federally-insured money, a guy who is here answering questions because his company, at his direction, clearly and intentionally violated the spirit of the Volcker rule, and these clowns on the Banking Committee are asking Dimon for advice on how to write the rule! It was incredible. Can you imagine senators asking the captain of the Exxon Valdez what his ideas are for new shipping safety regulations – and taking him seriously when he says he doesn’t think they’re a good idea?
Stop, Question, and Frisk policy getting stopped, questioned, and frisked
I’m happy to see that Federal District Court Judge Shira A. Scheindlin has granted class-action status to a lawsuit filed in January 2008 by the Center for Constitutional Rights which challenged the New York Police Department’s stop-and-frisk tactics.
The practice has been growing considerably in the last few years by way of a quota system for officers: an estimated 300,000 people have been stopped and frisked in New York City so far this year.
From the New York Times article on the class-action lawsuit:
In granting class-action status to the case, which was filed in January 2008 by the Center for Constitutional Rights on behalf of four plaintiffs, the judge wrote that she was giving voice to the voiceless.
“The vast majority of New Yorkers who are unlawfully stopped will never bring suit to vindicate their rights,” Judge Scheindlin wrote.
The judge said the evidence presented in the case showed that the department had a “policy of establishing performance standards and demanding increased levels of stops and frisks” that has led to an exponential growth in the number of stops.
But the judge used her strongest language in condemning the city’s position that a court-ordered injunction banning the stop-and-frisk practice would represent “judicial intrusion” and could not “guarantee that suspicionless stops would never occur or would only occur in a certain percentage of encounters.”
Judge Scheindlin said the city’s attitude was “cavalier,” and added that “suspicionless stops should never occur.”
I feel pretty awesome about this progress, since I was the data wrangler on the Data Without Borders datadive weekend and worked with the NYCLU to examine Stop, Question, and Frisk data. Some of that analysis, I’m guessing, has helped give ammunition to people trying to stop the policy – here is the wiki we made that weekend, and here’s another post I wrote a few weeks later.
For example, if you look at this editorial from the New York Times from a few days ago, you see a similar kind of analysis:
Over time, the program has grown to alarming proportions. There were fewer than 100,000 stops in 2002, but the police department carried out nearly 700,000 in 2011 and appears to be on track to exceed that number this year. About 85 percent of those stops involved blacks and Hispanics, who make up only about half the city’s population. Judge Scheindlin said the evidence showed that the unlawful stops resulted from “the department’s policy of establishing performance standards and demanding increased levels of stops and frisks.”
She noted that police officers had conducted tens of thousands of clearly unlawful stops in every precinct of the city, and that in nearly 36 percent of stops in 2009, officers had failed to list an acceptable “suspected crime.” The police are required to have a reasonable suspicion to make a stop. Only 5.37 percent of all stops between 2004 and 2009, the period of data considered by the court, resulted in arrests, an indication that a vast majority of people stopped did nothing wrong. Judge Scheindlin rebuked the city for a “deeply troubling apathy toward New Yorkers’ most fundamental constitutional rights.” The message of this devastating ruling is clear: The city must reform its abusive stop-and-frisk policy.
Woohoo! This is a great example of data analysis where it’s actually used to protect people instead of exploit them, which is pretty rare. It’s also a cool example of how open source data has been used to probe shady practices- but note that there was a separate lawsuit to force the NYPD to open source this Stop, Question, and Frisk data. They did not do it willingly, and they still don’t have the first few years of it publicly available.
Here’s another thing we could do with such data. My friend Catalina and I were talking yesterday about one of the consequences of the Stop, Question, and Frisk data as follows. From a Time Magazine article on Trayvon Martin:
in the U.S., African Americans and whites take drugs at about the same rate, but black youth are twice as likely to be arrested for it and more than five times more likely to be prosecuted as an adult for drug crimes. In New York City, 87% of residents arrested under the police department’s “stop and frisk” policy are black or Hispanic.
I’d love to see a study that breaks this down in a kind of dual way. If you’re a NYC teenager walking down the street in your own neighborhood with a joint in your pocket, what are your chances of getting put in jail a) if you’re white, b) if you’re black, c) if you’re hispanic, or d) if you’re asian?
I think those numbers would really bring home the kind of policy that we’re dealing with here. Let’s see some grad student theses coming out of this data set.
Stop with the man-diets already, coffee is good for you.
Every now and then my husband goes on a “man-diet”, which is a term I’ve coined meaning a restriction that has absolutely no reasonable goal except the very last event, namely breaking the diet and thus having an awesome moment of relapse.
His favorite man-diet is the coffee man-diet. He’ll suddenly decide that his two espressos in the morning and two more during the day is too much and he “needs to cut down”. He’ll make it one coffee in the morning and one in the afternoon, and he’ll stare wistfully at my second (or third) morning espresso, complain vaguely of headaches, and generally be a space cadet (by which I mean more than usual).
This will go on for about 5 days or so, until one morning where he’ll wake up in an addict’s rage and rebelliously suck down two coffees whilst raving about the magical properties of caffeine.
This happens about once a year. Every time it happens I remind him that coffee is actually good for you and there’s no reason to try to break an addiction that’s doing you good- we don’t try to go off oxygen, do we?
Well here’s more evidence of that. And in case you’re wondering, yes I do ignore all negative evidence of any one of my theories, but in this case I’m pretty sure the evidence I collect on my side is quite a bit stronger than the stuff I ignore. From the Bloomberg article:
The study found that men who drank 2 to 3 cups a day had a 14 percent lower risk of dying from heart disease, 17 percent lower risk of dying from respiratory disease, 16 percent decreased chance of dying from stroke and a 25 percent lower risk of dying from diabetes than those who drank no coffee.
…
Women who consumed 2 to 3 cups of coffee a day had a 15 percent lower chance of dying from heart disease, 21 percent lower risk of dying from respiratory disease, 7 percent decreased chance of dying from stroke and a 23 percent lower risk of dying from diabetes.
mathbabe 