Hip Hop’s Cambrian Explosion: Part 2
Since Cee Lo got the last word on the previous post, the first word of today’s guest post goes to Senegalese rapper Sister Fa. The word is Sarabah, the inspiration for point 4 in this prolix paean to Hip Hop.
4. Hip Hop is international. Although it originated in the African American community as a critique of white supremacy and an expression of community pride, it has gained international popularity over the past three decades, going viral as a musical meme, and inspiring artists from across the globe to adapt it and hybridize it with local traditions. To make a biological analogy, Hip Hop has undergone a global adaptive radiation to rival that of the Cambrian explosion.
I highly recommend the documentary film The Furious Force of Rhymes, which includes interviews with rappers in Senegal, Israel/Palestine, the United States, and Germany, chronicling its global appeal and rapid evolution. The film is a tribute to young artists who are drawn to Hip Hop as a vehicle for exploring social and political inequalities. Artists like Sister Fa, who raps in Wolof, Manding, Jola, and French. Her song Life Am reads like a public service announcement about the importance of practicing safe sex to contain the AIDS epidemic, but it’s the catchiest PSA I’ve ever heard, which is why it’s so effective. I find myself singing along to the rhythmic “utiliser le preservatif” only to realize that’s about the least sexy thing it’s possible to say in French. Voici une video, Milyamba. Sister Fa is leveraging her fame as a rapper to campaign throughout West Africa against female genital mutilation. Subjected to the practice herself, she tells her story in the documentary film about her life and activism entitled Sarabah.
Sister Fa’s use of multiple languages in the course of one song is typical of the way in which international artists have embellished the form and upleveled the wordplay. I’m particularly fond of songs that blend English with other languages, like Ozomatli’s Nada’s Por Free, sung in Spanglish (“So me levanto off the suelo straight chillando with my pena.” Ha!). I’m kvelling over this video of Ozomatli performing in Spanglish at the Tu B’Shvat Nature Fest.
Code-switching and language-switching are natural stylistic choices for a medium concerned with raising questions about racial identity. It was not so long ago that Apartheid imposed arbitrary racial categories on South Africans and enforced them with singular cruelty. (Check out this excellent documentary on Apartheid featuring Archbishop Desmond Tutu.) In the context of a country attempting to decolonize its national psyche and forge more pluralistic and inclusive social structures, rapper Emile YX uses Xhosa, Zulu, English and Afrikaans to explore lingering questions of racial identity.
In Who Am I? Emile YX mixes it up like the chromosomes in his name:
I’m every brother and sister
Working in a factory
I’m Easter weekend
camping at Kommetjie
I’m Sunday braaivleis
Dik geeet en gaan slaap
I’m that stone thrower at Caspers
During Apartheid innie Kaap
I’m that dummies player, kennetjie en Akoos
I’m that ANC supporter saying the DA se ma se,
I’m that voes gamtaal talker, corner broker
Gooi neer jou tol want hier gaan jy stoke ja,
I’m that mass marcher and tyre burner,
Minimal wage sub-economic earner
I’m that doctor, lawyer and politician in the ghetto
Wait a minute … Most of them have moved out though.
So I’m that one who stays to be a good role-model,
My election promises are not just oral.
Even when I can’t understand all the content of a song, I enjoy listening for Hip Hop’s recognizable elements, as well as the unique aspects that mark its cultural adaptation.
Here are a few gems:
In Spanish: Humanidad by Ana Tijoux
In Tagalog: Bebot by the Black Eyed Peas
In Japanese: Togesashi by Inden
In Arabic: Al Kuffiyeh 3Arabeyyeh by Shadia Mansour
With Hip Hop’s increasingly international reach, it is ever more pluralistic in both form and content. What is it exactly that makes the genre so appealing to a diverse global audience?
5. Hip Hop is democratic. A key stylistic convention of the genre involves using multiple first-person narrators who rotate in order to collectively construct the story as a mosaic. In the Black Eyed Peas’ Beautiful People, each of three narrators takes turns recounting his experience of coming into his own, overcoming the obstacles presented by anti-semitism, poverty, and the INS. Pinoy poet apl.de.ap tells his story:
Back in the days when I was in Philippines,
I had a dream to be heard and to be seen.
Came overseas at the age of fourteen,
Stood behind the front, if you know what I mean.
The song explores questions of worth – self-worth, material worth, innate worth — themes which are woven into each person’s narrative so that the song tells a collective story while still allowing each individual to express his unique perspective. Macy Gray sings the melody, an adaptation of the chorus from the Beatles’ Baby You’re a Rich Man With her wonderfully scratchy voice (What I wouldn’t give to bring Janis Joplin back from the dead to hear her and Macy Gray growl a duet!) she asks, “How does it feel to be one of the Beautiful People?” Collectively, the Black Eyed Peas convey a fairly democratic sentiment: You refugee, you immigrant, you down-and-out listener (and yes you, reader): You are the beautiful people.
Another favorite example of a song that uses rotating narrators to tell a single story is the Fugee’s Rumble in the Jungle. Seven narrators take turns paying homage to the historic boxing match between Muhammad Ali and George Foreman that took place in Zaire in197. At the height of the Black Power movement, Muhammad Ali’s journey back to Africa took on a significance and symbolism that became a source of shared pride bolstered further by the fact that Ali was the World Heavyweight Champion and a radical to boot. Ali was an early Conscientious Objector to the Vietnam War, stating succinctly, “I ain’t got no quarrel with them Viet Cong; No Viet Cong ever called me ‘nigger.’” The song opens with a quote by Ali, so in a sense he becomes an eighth narrator:
“I’m gonna fight for prestige, not for me, but to uplift my little brothers who are sleeping on concrete floors today in America, black people who are living on welfare, black people who can’t eat, black people who don’t know no knowledge of themselves, black people who don’t have no future. I want to win my title and walk down the allies, set on the garbage can with the wineheads. I wanna walk down the street with the dope addicts, talk to the prostitutes. So, I can help a lot of people.”
I recommend the documentary When We Were Kings, a biopic about Ali, which situates Rumble in the Jungle in its historical context, including the glaring irony of the event: the fact that it was funded by dictator Mobutu Sese Seko, who had only recently assumed power via coup after the C.I.A. orchestrated the overthrow of the democratically-elected Patrice Lumumba. For more on the story of this egregious episode in American foreign policy, two good resources are the film Lumumba and the book The Poisonwood Bible by Barbara Kingsolver, a fictional account of the period surrounding Lumumba’s assassination. Interestingly, her novel also uses the device of rotating first-person narrators to structure the story. When telling a story about democracy – or the deliberate undermining thereof – the technique of weaving multiple narrators’s perspectives is apt in that it models pluralism.
Women’s voices are too often missing in Hip Hop, so I am particularly smitten with Cell Block Tango/He Had It Comin’ by Macy Gray, L’il Kim, and Queen Latifah. This song uses the multiple-narrator format to address a serious topic in a campy tone: domestic violence. The song’s over-the-top Broadway style (it was written for the musical Chicago) signals to the listener that we are in the realm of fantasy as each artist imagines violent revenge against her lover/abuser. Queen Latifah sings the first verse with characteristic sassitude:
I mean imagine, why was he hittin’ his woman?
Why was she takin’ that?
Now picture her fightin’ back, picture the ass kickin’
Think of his ass flippin’ down the stairs
And me at the top smilin’, he shoulda stopped wildin’.
Now could you picture me tryin’ to finish him off?
See why I pictured me on this side of the law?
High heels leave holes, you’d a thought I was gunnin’!
Now the cops comin’, I ain’t runnin’!
Hip Hop is very much a macho, male-dominant form, misogyny being its blind spot. So Queen Latifah’s is a minority voice. But that’s just it. Even if yours is a minority opinion, if you can voice it with enough panache, and what you’re saying smacks of truth, then you can have your day in court in Hip Hop. The democratic nature of the form has a lot to do with the method by which rappers rise in fame and credibility: the rap battle.
6. Hip Hop is competitive. Emerging rappers can be recognized in public because they carry a notebook in their pocket which they frequently take out to jot down rap lyrics and song hooks. They are avid consumers of rap music, noting other rapper’s cadences and rhyme schemes, and honing their skills in cyphers. A cypher is a group of rappers who get together and take turns improvising raps either acapella or over a beat. If you have ever seen this live, you know how impressive this ability to extemporize is. Here’s a home-made video of a stellar cypher I saw live as part of the annual Life is Living Festival. (Check out the 9-year-old who rocks the mic at 3:20 and 13:28!) Cyphers provide creative community and serve as training grounds for the high-stakes battles that put rappers to the test. Many of you are familiar with the battle scene in Eminem’s 8-Mile, a film that is notorious in my mind largely for its terribly unsexy sex scene. (And they didn’t even “user le preservatif!”) Here’s a link to the final battle scene in 8-Mile, in which Eminem wins by confessing his flaws to the crowd first, stealing his opponent’s best ammunition.
In a rap battle, two artists at a time compete. They are each given a time limit, taking turns addressing each other back and forth, all the while wooing the crowd with their artistry and skills. When the beat drops – one they have not heard in advance – they have to deliver a performance that: a) belittles the competitor; b) self-aggrandizes; c) references a comment or event that just occurred, demonstrating that the lyrics are improvised in the moment; and d) impresses the crowd. This last one is critical, as it is the audience who decides who the winner is in an instant and rambunctious plebiscite. Picture an Occupy General Assembly, but with different hand gestures. Hip Hop’s virtuosos are vetted in the democratic forum of the rap battle.
Apropos of democracy and internationalism, I’ll give the last word of today’s post, part 2 of 3, to the Occupy Wall Street movement. Enjoy this funky rallying cry from Occupy London.
Cheers!
Interlude: Newt Haiku
Photo: The newt on the left hovers above an egg sac moments before approaching the beast with two tails on the right.
On one of the first unmistakable days of Spring, I led a group of five six-year-olds on a walk through the UC Botanical Garden, where I am a docent. I say “walk,” but children at that age do anything but: they skip, bound, trip, jostle, spin,and vibrate in a sort of Brownian motion, but rarely walk. On this first morning of emboldening sunlight and tentative short sleeves, my group was particularly kinetic, their effervescence reaching a feverish pitch when we arrived at the Japanese pool and found the newts in a similar frenzy. Pairs of newts gripped each other in slippery contortions, splashy displays that incited the childrens’ curiosity. Lone newts trailed after mating pairs, latching on to a tail and rolling into a tan-and-yellow tangle of three- four- and five-newt bundles. One of the children asked, “What is that brown knot?” The adult chaperones tittered. Since Spring has inspired haiku for centuries, I offer this reply, in haiku form.
What is that brown knot?
The newts are dancing, children,
celebrating Spring.
What is that brown knot?
The newts are wrestling, children,
That’s how life begins.
What is that brown knot?
A miracle of nature,
one more mystery.
What is that brown knot?
The newts are in love, children,
euphemistically.
What is that brown knot?
A good old-fashioned tussle,
a frenzy, a fray.
What is that brown knot?
Survival of the fittest,
Shuffling DNA.
What is that brown knot?
An amphibian orgy,
It’s rude, kids, to stare.
What is that brown knot?
Sexual reproduction.
Ask that docent there.
Hip Hop’s Cambrian Explosion: Part 1
On her January 28th post entitled Does Hip Hop Still Exist? Mathbabe wrote:
“My oldest friend sent me some mixed CDs for Christmas. I listened to them at work one recent morning, and although I like a few songs, many of them were downright jarring. I mean, so syncopated! So raw and violent! What the hell is this?! It was hip-hop, I think, although that was a word from some far-away time and place. Does hip-hop still exist?”
Fortunately for me, I am that oldest friend, mixer of said CD, and guest blogger this week, here to answer Mathbabe’s question with the first of a three-part post entitled Hip Hop’s Cambrian Explosion.
______________
I discovered Hip Hop around the same time I discovered Mathbabe. In 1987, Hip Hop was a toddler living in Brooklyn while Cathy and I were teenagers living in suburban Massachusetts. As I walked home from school one afternoon, I popped Boogie Down Production’s debut cassette into my walkman and snapped to attention as KRS One delivered a high-energy critique of public schooling’s systematic omission of Black history from the curriculum. As I listened I found myself considering for the first time the ways in which I had been raised on a steady academic diet of European and American histories and literatures, with no mention of those of Africa, Latin America, or Asia. These were entire continents and peoples whose histories were tacitly deemed peripheral to the central drama of whiteness. I listened closely as KRS One, aptly known as “The Teacher,” educated me about the people studiously ignored in my history textbooks. Here is a delightfully dated video of that first song, You Must Learn: http://www.youtube.com/watch?v=RDd7UbJmdmw.
I am now nearly 40 and recently had the opportunity to meet KRS One at a concert in Berkeley, where I was able to thank him in person for supplementing my education. He is as dynamic as I remember, still using the mic as a vehicle to teach critical thinking, still building community by inviting up-and-coming rappers onto the stage to improvise with him, still innovating by rapping over electric violins spilling amplified Mozart over the surging audience. In this photo I took from stageside he reaches out to connect with the crowd:
And here I am, looking up at him.
Photo by Hugo Garcia, aka Steelo
As you can see in the photo, I plainly admire him, as I do any iconoclast who has the audacity and clarity to say so when the Emperor has no clothes. So as an avid fan of Hip Hop, I’d like to appeal its case for those of you who are new to the genre or are considering giving it a second listen. Why should you bother listening to Hip Hop? And what exactly is Hip Hop anyway? I offer this primer as a paean.
1. Hip Hop is political. Hip Hop gained national attention in 1989 when Public Enemy’s Fight the Power piqued the paranoia of white America. The now-classic ghetto anthem opens with Martin Luther King’s lilting oratory, not the more tepid, politically-milktoast MLK Jr. of the official public holiday, but the radical MLK Jr, who exhorts Americans not only to refuse to serve in the U.S. Army, but to switch allegiance to fight alongside the Viet Cong.
Chuck D and Flavor Flav at Yoshi’s in San Francisco. “Most of my heroes don’t appear on no stamps.” – Chuck D. Photo by Cherie Chavez
True to its origins, Hip Hop remains today the artistic genre of choice and the voicebox for people pushed to the margins of power by historical and social forces. And it’s not afraid to name those forces. Paramount among the themes tackled in Hip Hop is that of white supremacy, a topic — a phrase even — that tends to make white people uncomfortable. When rapper Brother Ali released Uncle Sam Goddamn, an overview of American racism — past and present — cell phone company Verizon responded by revoking its sponsorship of his tour. Corporations typically don’t profit by talking about racism, unless it’s in that “Rainbow Nation” manner of Benetton, which carefully eschews analysis of power relations. The video for Uncle Sam Goddamn includes some powerful historical footage.
Another of Hip Hop’s recurring themes is poverty. As Somalian-born rapper (and personal favorite) K’Naan explains:
…I remember when I was 7
When rap came mysteriously and made me feel 11
It understood me, and made my ghetto heaven
I understood it as the new poor people’s weapon.
Smart 7-year-old. The excerpt is from The African Way, a funky fusion of American-style rap vocals and East African drum rhythms. As K’Naan recounts in several of his autobiographical songs, he learned English by listening to rap music (it’s no coincidence he sounds so much like Eminem) in order to have a forum for speaking about the violence he experienced as a child growing up in his native Mogadishu. His beautiful Blues for the Horn is both lament and homage to the Horn of Africa. He narrates the story of Somalia himself so that no one can “make a mockery of our struggle like Hollywood plans to.” And despite the seriousness of his purpose, he carries on another of Hip Hop’s traditions: its sense of humor. Describing Mogadishu, he quips:
If you bring the world hoods to a seminar
We’re from the only place worse than Kandahar —
And that’s kinda hard!
In the song Somalia, he reminisces about his childhood:
We used to take barbed wire
Mold it around discarded bike tires,
Roll em down the hill in foot blazin’ —
Now that was our version of mountain bike racing!
Daaaaaammn!
Do you see why it’s amazing
When someone comes out of such a dire situation
And learns the English language,
Just to share his observations?
Probably get a Grammy without a grammar education.
Hip Hop has an unapologetic working class hero sensibility, like John Lennon’s Working Class Hero, but edgier. How’s this for edgier? 5 Million Ways to Kill a C.E.O.
The racism and classism that inhere in our justice system are the targets of Lauryn Hill’s epic rant in The Mystery of Iniquity. She seems to enter a poetic trance as she excoriates the American judicial system in a style that calls to mind the dogged dirge of Allen Ginsberg’s Howl. This is but a tiny excerpt of Hill’s stream-of-consciousness dressing down:
Do we expect the system made for the elect
To possibly judge correct?
Properly serve and protect?
Materially corrupt
Spiritually amuck…
Mafia with diplomas keeping us in a coma trying to own a piece
of the American Corona.
The revolving door:
Insanity every floor
Skyscraping, paper chasing,
What are we working for?
Empty traditions
Reaching social positions
Teaching ambition to support the family superstition?
With a bass voice like Barry White, rapper Lyrics Born questions our funding priorities in Stop Complaining:
I pay my taxes when I’m asked to.
I’m not enthusiastic about it, but shit, I make it happen.
Yeah, it’s last minute, but goddammit they cash it.
(“This is fiscal harassment, they keep touchin my assets!”)
Now I imagine I might be feeling different about it
If it was given outright, witness it helping somebody
But it just so happens in life, the school district’s too crowded
It ain’t no teachers in sight, that’s why the kids are so rowdy.
I just imagine some asshole with glasses on up at the Capitol
One of a thousand pawns packed in an office cramped up like animals,
Pictures of his sister, his mixture Lapso Apso-poodle
His 2.6 kids, and the missus thumbtacked to his cubicle
So damn detached from the average man’s planet, he cain’t fathom
That we could ever be anything other than stats, fat and taxable
He’s gettin his usual ritual 2 o’clock Cup of Noodles on
While he’s fuckin you on your W2, his John Denver music on.
The ongoing disparities in K-12 schooling and access to higher education are a common theme in Hip Hop. Shad K, who pursued his career in Hip Hop while simultaneously earning a Masters degree in Business, writes in Exile:
We’re taught not to question the status quo cuz the masses never get heard
unless you’re established
with expert professors in dress-shirts
and glasses that lecture to classes
from lecterns
where next term the best third will pass and
earn cash working as
desk clerks for the best firms in Manhattan.
Shad was born in Kenya, the son of Rwandan refugee Bernadette Kabango, whose autobiographical poetry he incorporates in the chilling song I’ll Never Understand. Shad’s mother reads in her own voice, telling the story of her family’s murder in the 1994 genocide, addressing those who committed violence against her, and raising questions about the possibility of forgiveness. Shad’s rap vocals interlace with his mother’s voice, interjecting questions about whose genocides matter and whose don’t. The video of I’ll Never Understand includes footage that defies commentary from the Rwandan massacres. How could one begin to talk about such atrocities but through art? As writer Victor Hugo observed, “Music expresses that which cannot be said and on which it is impossible to be silent.”
The musical conversation between mother and son brings me to Hip Hop’s next defining characteristic.
2. Hip Hop is intergenerational. One of the stylistic and structural conventions of the genre is sampling. The contemporary artist layers his/her vocal track over a repeated excerpt of a melodic track — the sample — by an older artist, alive or dead. The tradition of sampling older artists from a generation prior (e.g., Nina Simone, Ray Charles, etc.) began perhaps for practical reasons, as access to older songs was not limited by royalties and copyright. Regardless of the motivation, sampling has the effect of creating intergenerational dialogue, a musical conversation across time.
Hip Hop has its roots in the oral traditions of West Africa, where people still live in active relationship with their ancestors and respect for elders is a core cultural value. Hip Hop carries on this tradition of talking with the dead and honoring those who have paved the way. Erick Sermon’s Just Like Music is an ode to music’s healing power (“I wish music could adopt me!”), sampling musical legend Marvin Gaye. Sermon cleverly interweaves his contemporary vocals over Marvin Gaye’s melodies so that at one point they appear to be in direct conversation:
Sermon: Is that true, Marvin?
Gaye: Yeeeeeeaaah!
It’s no surprise that songs from the Civil Rights movement provide a rich pool of sampling material. Movin’ Forward by Collective Efforts samples Civil Rights song Ain’t Gonna Let Nobody Turn Me Around, and references Eyes on the Prize, the comprehensive and inspiring documentary on the history of the Civil Rights movement.
Fort Minor’s Kenji tells the story of the United States internment of Japanese Americans during World War II and honors elder survivors by incorporating original audio interviews of former internees.
I’ve talked a lot about politics here, but if Hip Hop were all politics, it would be two-dimensional, flat like a Soviet-era agitprop poster (you know the ones of the workers with the disproportionately huge fists). It is the next characteristic which gives Hip Hop its complexity, dynamism, and multi-dimensionality.
3. Hip Hop is poetry. On steroids. A bit like aural caffeine. While I first got hooked on rap for its incisive outsider critiques, I equally enjoy the verbal acrobatics and linguistic playfulness of the form. I’m a word nerd, a sesquipedalian, easily wooed by an orator who can wield an adjective, so the highly verbal genre holds a natural appeal for me.
Others have told me that they find the language of Hip Hop itself a barrier to listening, the lyrics so rapid-fire and abstruse as to be unintelligible to the uninitiated. Perhaps so, but many of us found Shakespeare difficult to parse at the beginning but ultimately worth the effort. As in any specialized field, rap has created a unique language, its own grid of intelligibility, with webs of cross-references and insider lingo that can be opaque to newcomers to the genre. Just as you would read Shakespeare with a dictionary at your side as a reference to make meaning of the text, rap music lyrics must be studied with the right reference materials at hand. The rate of word evolution in rap music is rapid, which can make it difficult to keep up with the neologisms. Fortunately, there’s Urban Dictionary, with user-entered definitions being added continuously.
Approaching Hip Hop with the same spirit of literary criticism used to analyze Shakespeare or T.S. Eliot reveals that its poets employ all of the literary devices standard in the craft: alliteration, assonance, consonance, onomatopoeia, irony, and variation. They amplify the metrical effects of diction and syntax with the percussion and syncopation provided by the music itself. It is the interplay of the two (or more often, multiple) meters – the rhythm inherent in language, and the nonvocal rhythms of layered drums, piano, trumpet, or kora – that so stimulates the linguistically-inclined mind.
What rap adds to the traditional toolbox available to written poetry is a tool only available to the spoken word artist: something called flow. Flow is a bit difficult to describe, but you know it when you hear it. Flow is that state a rapper gets into when the syllables are tumbling off the tongue in a waterfall of words, the cadences rising and falling, surprising and mesmerizing. Flow is that trance artists crave, that moment when the rational mind steps aside, time telescopes, and the artist becomes hypnotized by the presence of the muse. Flow is when the music comes through the musician rather than from the musician. Cee-Lo has it. I couldn’t agree more with his self-review in One for the Road: “Oh, his way with words! I want seconds and thirds!”
I’ll let Cee Lo have the last word for now, and will continue tomorrow with Part 2 of Hip Hop’s Cambrian Explosion.
Why Larry Summers lost the presidency of Harvard
Some people still think Larry Summers got fired from being the president of Harvard because of the ridiculous comments he made about women in math (see my post about this here) or because of the comments he made about Cornel West. Actually, the truth is something worse, and for which he should actually be in jail. It’s also something that makes Harvard look bad, so maybe that’s why it’s less known.
The subtitle of this post is: Why Larry Summers shouldn’t be made head of the World Bank.
I was inspired to write this by being disgusted at continued rumors that he could get yet another prestigious job. It’s like this guy can’t fail spectacularly enough! Let’s give him another chance!
Let’s set the record straight: Summers was directly involved with defrauding the U.S. Government (see below) and Russia. He admitted to not understand conflict of interest issues (see below). It is particularly appalling, knowing these things, that he would be considered for the World Bank head, which presumably requires nuanced understanding of such issues.
I’m using this article, entitled “How Harvard Lost Russia,” and written in 2006 in Institutional Insider (II), as a reference. More on that article and how it led to getting Summers fired below. And by the way, I’m not claiming this story is completely unkown: see this wikipedia article for a quick overview, for example, in addition to the II article. I just think it needs reviving at this crucial moment, before Summers gets more toys to play with.
Shleifer
So why did Summers lose his job at Harvard? It was because of his protecting a buddy, a fellow economist at Harvard named Andrei Shleifer.
Andrei Shleifer managed to get put in charge of helping Russia privatize stuff in the mid 1990’s. His mission was to make things more useful and transparent to the infant capitalist system. Through his wife and friends, Shleifer instead orchestrated a boondoggle on Russia. He invested money through his wife and helped his friend Jonathan Hay and his lover and friends invest theirs, and set up the very first mutual fund as well as thwarting the efforts of other people to set up their own funds. All of these things were strictly against the conflict of interest policy they were working under.
Shleifer got in trouble, and the U.S Government sued and won against Harvard and Shleifer. From the article:
The judge determined that Shleifer and Hay were subject to the conflict-of-interest rules and had tried to circumvent them; that Shleifer engaged in apparent self-dealing; that Hay attempted to “launder” $400,000 through his father and girlfriend; that Hay knew the claims he caused to be submitted to AID were false; and that Shleifer and Hay conspired to defraud the U.S. government by submitting false claims.
On August 3, 2005, the parties announced a settlement under which Harvard was required to pay $26.5 million to the U.S. government, Shleifer $2 million and Hay between $1 million and $2 million, depending on his earnings over the next decade. Shleifer was barred from participating in any AID project for two years and Hay for five years. Shleifer and Zimmerman were required by terms of the settlement to take out a $2 million mortgage on their Newton house. None of the defendants acknowledged any liability under the settlement. (Forum Financial also settled its lawsuit against Harvard, Shleifer and Hay under undisclosed terms.
Summers and Shleifer
Summers was good friends with this criminal, and used his position to protect him. From the article:
Shleifer remained close to his friend and mentor Summers; they talked to and saw each other frequently and continued vacationing together in the summer on the Cape. Then it became known in early 2001 that Summers was on the short list of candidates to succeed Neil Rudenstine as the president of Harvard University. Shleifer and Zimmerman began campaigning for Summers to get the Harvard post, giving meet-and-greet parties for him at their home. Summers stayed with them when he visited Harvard.
In March 2001, Summers was named president of Harvard. Shleifer, who had been courted by New York University’s Stern School of Business, decided to stay put.
Having his close friend as his boss would turn out to be quite helpful to Shleifer. Summers asserted in his deposition that he recused himself from any involvement in the university’s handling of the Shleifer matter, but the new president stayed involved anyway. Early in his presidency he told the dean of the faculty of arts and sciences, Jeremy Knowles, to keep Shleifer at Harvard.
“I expressed to Dean Knowles,” Summers testified in a deposition in 2002, “. . . that I was concerned to make sure that Professor Shleifer remained at Harvard because I felt that he made a great contribution to the economics department . . . and expressed the hope that Dean Knowles would be attentive to that. . . . I think he recognized and shared the concern.”
“Conflict of interest issues should be left to the lawyers” says Summers
This is the testimony that says to me, in no uncertain terms, that Summers cannot be put in charge of something politically sensitive:
Summers said conflict-of-interest “issues,” in his Washington experience, were “left to the lawyers.” He said he was sensitive to “ethics rules,” but testified that “in Washington I wasn’t ever smart enough to predict them . . . things that seemed very ethical to me were thought of as problematic and things that seemed quite problematic to me were thought of as perfectly fine. . . .”
More intervention on behalf of Shleifer
Maybe you’d think that getting sued by the US Government and losing $40 million might lose your job as a Harvard Professor. But you’d be wrong:
Knowles tells Institutional Investor that he does not remember Summers’ approaching him about Shleifer. “I don’t recall this particular conversation, but the president and I shared the goal of recruiting and retaining the best faculty, so it would have been perfectly natural for us to mention to each other the names of people that we certainly wouldn’t want to lose.” However, not long after Summers says he intervened on the professor’s behalf, Knowles promoted Shleifer from professor of economics to a named chair, the Whipple V.N. Jones professorship.
Shleifer’s legal position changed on June 28, 2004, when Judge Woodlock ruled that he and Hay had conspired to defraud the U.S. government and had violated conflict-of-interest regulations. Still, there was no indication that the Summers administration had initiated disciplinary proceedings. To the contrary, efforts were seemingly made to divert attention from the growing scandal. The message from the top at Harvard was, “No problem — Andrei Shleifer is a star,” says one senior Harvard figure.
The Summers-Shleifer friendship flourished. They spoke on the phone more than once a day, on average. Two months after the court ruling against Shleifer, he hosted Summers at a break-the-fast dinner on Yom Kippur.
One instance was a meeting early in the academic year that began in September 2004, less than two months after the federal court formally adjudicated Shleifer’s liability for conspiring to defraud the U.S. government. A faculty member asked Kirby why Harvard should defend a professor who had been found liable for conspiring to commit fraud. The second confrontation came early in the current academic year when another professor asked Kirby why Harvard should pay a settlement of $26.5 million and legal fees estimated at between $10 million and $15 million for legal violations by a single professor and his employee, about which it was unaware. On both occasions Kirby is said to have turned red in the face and angrily cut off discussion.
On at least one other occasion, Summers himself told members of the faculty of arts and sciences that the millions of dollars that Harvard paid in damages did not come from the budget of the faculty of arts and sciences, but didn’t say where the money came from. Those listening inferred he meant that the matter shouldn’t be of concern to the faculty and that they shouldn’t raise it, a curious notion, given that Shleifer was one of their own.
A spokesman for Summers said he was “unable to schedule” an interview with Summers for II in December, when this article was being prepared. As the lawsuit was against the university, not just the faculty of arts and sciences, the settlement came from “university funds available for these purposes,” the spokesman added.
…
Shleifer has never acknowledged doing anything wrong. Summers has said nothing. And so far as is known, there has been no internal investigation or sanction. “An observer trying to make sense of the University’s position on Shleifer, Ogletree and Tribe is driven to an unhappy conclusion. Defiance seems to be a better way to escape institutional opprobrium than confession and apology. . . . And most of all being a close personal friend of the president probably does one no harm.”
The article gets Summers fired
An anonymous person got a bunch of copies of the II article and stuck one in every Harvard faculty’s mailbox the morning of the no-confidence vote that got Summers ousted.
And just in case you’re wondering, here’s the website of Sheifer, still on faculty of Harvard.
VAM versus what?
A few astute readers pointed out to me that in the past few days I both slammed the Value-added teacher’s model (VAM) and complained about people who reject something without providing an alternative. Good point, and today I’d like to start that discussion.
What should we be doing instead of VAM?
First of all, I do think that not rating teachers at all is better than the current system. So my “compare the the status quo” argument goes through in this instance. Namely, VAM is actively discouraging teachers whereas leaving them alone entirely would neither discourage or encourage anyone. So better than this.
At the same time, I am a realist, and I think there should be, ultimately, a system of evaluating teachers, just as there is a system for evaluating me at work. The difference between my workplace, of 45 people, and the NYC public schools is scale. It makes sense to have a very large and consistent evaluation system in the NYC public schools, whereas my job can have an ad hoc inconsistent system without it being a problem.
There’s another problem which is nearly impossible to tease from this discussion. Namely, the fact that what’s going on in NYC is a disingenuous political game between Bloomberg and the teacher’s union. Just to emphasize how important that fight is, let’s keep in mind that as of now, although the union is much weaker than it historically has been, it still has the tenure system. So any model, VAM or not, of evaluation is somewhat irrelevant for “removing bad teachers” given that they have tenure and tenure still means something.
Probably the best way to decouple the “Bloomberg vs. union/tenure” issue (a massive one here in NYC) from the “VAM versus other” question is to think nationally rather than citywide.
The truth is, the VAM is being tried out all over the country (although I don’t have hard numbers on this) and the momentum is for it to be used more and more. I predict within 10 years it will be done systematically everywhere in the country.
And, sadly, that’s kind of my prediction whether or not the underlying model is any good or not! The truth is, there is a large contingent of technocrats who want control over the evaluation system and believe in the models, whether or not they are producing pure noise or not. In other words, they believe in “data driven decisioning” as a holy grail even though there’s scant evidence that this will work in schools. And they also don’t want to back down now, even though the model sucks, because they feel like they’ll be losing momentum on the overall data-driven approach.
One thing I know for sure is that we should continue to be aware of how badly the current models are, and I want to set up an open source version of the models (see this post to get an idea how it could work) to exhibit that. In other words, even if we don’t turn off the models altogether, can’t we at least minimize their importance while their quality is bad? The first step is to plainly exhibit how bad they are.
It’s hard for me to decide what to do next, though. I’m essentially a modeler who is hugely skeptical of models. In fact, I don’t think using purely quantitative models to evaluate teachers is the right thing to do, period. Yet I feel like if it’s definitely going to happen, better for people like me to be in the middle of it, pointing out how bad the proposed (or in use) models are actually performing, and improving them.
One thing I know I’d do if I were to be put in charge of creating a better model: I’d train on data where the teacher is actually rated as a good teacher or not. In other words, I wouldn’t proxy “good teacher” by “if your students scored better than expected on tests”. A good model would be trained on data where there would be an expert teacher scorer, who would go into 500 classrooms and carefully evaluate the actual teachers, based on things like whether the teacher asked questions, or got the kids engaged, or talked too much or too little, or imposed too much busy work, etc. Then the model would be trying to mimic this expert.
Of course there are lots of really complicated issues to sort out- and they are *totally unavoidable*. This is why I’m so skeptical of models, by the way: people think you can simplify stuff when you actually can’t. There’s nothing simple about teaching and whether someone’s a good teacher. It’s just plain complex. A simple model will be losing too much information.
Here’s one. Different people think good teaching is different. A possible solution: maybe we could have 5 different “expert models” based on different people’s definitions of good teaching, and every teacher could be evaluated based on every model. Still need to find those 5 experts that teachers trust.
Here’s another. The kind of teacher-specific attributes collected for this test would be different from the VAM- things that happen inside a classroom (like percentage of time teacher talks vs. student, the tone of the discussion, the number and percentage of kids involved in the discussion, etc,) and are harder to capture accurately. These are technological hurdles that are hard.
I think one of the most important questions is whether we can come up with an evaluation system that would be sufficiently reasonable and transparent that the teachers themselves would get on board.
I’d to hear more ideas.
Sausage Wall
I’m getting ready to go to Amsterdam this Sunday (get ready for an exciting guest blogger while I’m gone!).
I’ve been to Amsterdam a bunch of times since hooking up with my big-nosed Dutch husband, and I enjoy our visits to his family very much.
But it’s not what you’re thinking. I’m allergic to that stuff (I have very funny, inappropriate stories about that which you’ll have to ask me in person), plus I’m traveling with our 3 sons, so it’s all about bikes, canals, and food. I’m also allergic to art, so the museum scene is kind of irrelevant too. I know that’s blasphemous, but there you go, I just don’t get paintings.
So, about the food. I’ve got street food tastes, and much to the chagrin of my in-law family, I consider true Dutch delicacies to be the stuff you find in carts along the side of the main road between the train station and the place with all the pigeons. Mostly loompjes loompjas (skinny little spring rolls), ollieballen (donuts without holes, literally translated as “balls of oil”), and poffertjes (tiny pancakes). Mmmm… poffertjes.
Anyhoo, what I really wanted to discuss today is the sausage wall, which I dearly dearly love. It’s near the Central Train Station, and I never know exactly where it is but I always find it like a fucking homing device. I’m the pigeon, the sausage wall is my coop.
What is a sausage wall, you ask? It’s a tiny little hole in the wall fast-food restaurant where you put coins into slots, like a vending machine, and you get to open these tiny little doors, inside of which are these delicious sausage sandwiches and other strange things. So, weird little fried things, mostly in buns but sometimes not, of all descriptions, except you never know exactly what anything is made of.
Is it delicious, you may ask? Oh yes, it is. It is, for reals, but my guess is that the crucial ingredient that makes everything so good is that you have about 40,000 high people very nearby getting the munchies, and the result of this is unbelievable turnover.
I have never been to the sausage wall when there are fewer than 15 other people vying for the best sausage windows. On the supply side, there’s an army of Dutch people on the other side of the wall feverishly preparing fresh fried sausages (if that even makes sense). Thank God for those people, and who is the genius I can thank for coming up with this brilliant idea in the first place?
I can’t wait to get to Amsterdam, folks, the sausage wall is calling me and I can hear its cry.
Quants for the rest of us
A few days ago I wrote about the idea of having a quant group working for consumers rather than against them. Today I wanted to spell out a few things I think that group could do.
The way I see it, there’s this whole revolution of data and technology and modeling going on right now, but only people with enough dough to pay for the quants are actually actively benefiting from the revolution. So people in finance, obviously, but also internet advertising companies.
The problem with this, besides the lopsidedness of it all, is that the actual models being used are for the most part predatory rather than helpful to the average person.
In other words, most models are answering question of the form:
- how can we get you to spend money you don’t necessarily want to spend? or
- how good a credit risk are you? or
- how likely are you to come back and spend more money? or
- how can we anticipate the market responding to end-of-month accounting shenanigans? I just threw this one in to give people a sense of how finance models work.
It all makes sense: these are businesses that are essentially bloodthirsty, making their money off your clicks and purchases. They are not going away.
Then there are some models that are already out there trying to make the user experience more enjoyable. They answer questions of the form:
- If you like that, what else may you like? (Pandora, Netflix, Google)
- If you bought that last week, maybe you’d like to buy it again this week?
- If you bought that, maybe you’d like to buy this now?
The problem, as you can see, is that these second, helpfulish kinds of things quickly devolve into the first, predatory kinds of things. In other words, you’re being bombarded by ads and suggestions for spending more money than you actually want to spend.
What about stuff that you actually don’t want to do but that is probably not directly profitable to anyone? I’d love to see technology used to tackle some of these chores:
- Figuring out what the best deal is on loans (credit card, student loans or mortgages), without becoming a lawyer. Here I’m not just saying they should all be clear about their terms- that’s a job for the CFPB. I mean there should be a website that asks me a few questions about what I need a loan for and points me to the best deal available.
- Finding the best price for something. I discussed this briefly here.
- Warnings about weird conditions and agreements, like mandatory arbitration.
- Help finding a good doctor or a good plumber etc.
- Help knowing when to go to the DMV or other public services building so the lines are bearable, or even better a way to do stuff on your computer or phone and avoid the trip altogether.
- Getting your kids’ medical records to their schools and camps safely and efficiently.
Some of these things already exist (like the doctor thing) but aren’t well-known or well-publicized, so wouldn’t it be cool if this quant group also provided an app that would aggregate all these services into one place?
And for the things like #4, where you want to be warned before you buy, it’s too much work to go back to a webpage to check everything before buying. Instead, the app could follow you around the web when you’re shopping and overlay a “bullshit warning” icon on the potential purchase in situations where people have complained about unreasonable terms and conditions.
Let’s use technology in our favor. Instead of the companies collecting information about us, let’s collect information about them. Come to think of it, the CFPB should start this quant group today.
Versus what?
I’m going to specialize in short, curmudgeony blog posts this week.
Today’s topic: you always need something to compare a new thing with. It’s this versus what?
If it’s a model, compare it to noise. That is, go ahead and test a model by scrambling the “y”s and see how well your model predicts randomness. It’s a really good and inexpensive way of seeing whether your model is better than noise, so go ahead and do it. There’s even a name for this but I forget what it is (update from reader: permutation testing).
If it’s a plan for a system or the world, compare it to the status quo. I’m so sick of people discarding good plans because they’re not perfect. If they’re better than what’s currently going on, then let’s go with that. Which brings me to my last example.
If it’s someone’s proposal (person A), compare it to other proposals (person B). I don’t think it’s fair for people (person C) to nix an idea unless they come up with a better one. If person C is consistently doing that, it’s a good bet that they have something to protect in the status quo situation, which brings us to the previous example.
The Value Added Teacher Model Sucks
Today I want you to read this post (hat tip Jordan Ellenberg) written by Gary Rubinstein, which is the post I would have written if I’d had time and had known that they released the actual Value-added Model scores to the public in machine readable format here.
If you’re a total lazy-ass and can’t get yourself to click on that link, here’s a sound bite takeaway: a scatter plot of scores for the same teacher, in the same year, teaching the same subject to kids in different grades. So, for example, a teacher might teach math to 6th graders and to 7th graders and get two different scores; how different are those scores? Here’s how different:
Yeah, so basically random. In fact a correlation of 24%. This is an embarrassment, people, and we cannot let this be how we decide whether a teacher gets tenure or how shamed a person gets in a newspaper article.
Just imagine if you got publicly humiliated by a model with that kind of noise which was purportedly evaluating your work, which you had no view into and thus you couldn’t argue against.
I’d love to get a meeting with Bloomberg and show him this scatter plot. I might also ask him why, if his administration is indeed so excited about “transparency,” do they release the scores but not the model itself, and why they refuse to release police reports at all.
Why experts?
I recently read a Bloomberg article (via Naked Capitalism) about wine critics and how they have better powers of taste than we normal people do.
One of the things I love about this article is how it’s both completely dead obvious and at the same time totally outrageous when you think about it.
Obvious because when we see wine experts going on and on about what they can discern in their tiny sip, we know they either have magical powers or they’re lying, and since some of them can do this shit blindfolded we will assume they aren’t lying.
Outrageous because if you think about it, that means we follow the advice of people whose taste is provably different from ours. In other words, the word of experts is fundamentally irrelevant to us, and yet we care about it anyway.
My question is, why do we care?
Just to go over a couple of ground rules. First, yes, let’s assume that the wine experts really do have powers of discernment that are incredible and unusual, even though we have to trust an expert on that, which may seem contradictory. The truth is, this isn’t the first study that’s shown that, and I for one have hung out with these guys and they really do taste that minerally soil in the wine. I’m not even jealous.
Second, I’m not saying you care about wine experts’ opinions, but lets face it, lots of people do. And you could say it’s because of the performance that the experts give when they smell the wine and describe it, and I’ll agree that some of them can be poets, and that’s nice to see. But the truth is they also rate the wines with a number, and these numbers are printed in books, and lots of people carry these books around to wine shops and devote themselves to only buying wines with sufficiently high numbers, even though those people probably don’t themselves have the mouth smarts to tell the difference.
Now that we’ve framed the question, we can go ahead and make guesses as to why. Here are mine:
- People are hoping that they themselves are also supertasters. This kind of seems like the most obvious one, but I can’t help think that true supertasters would not need other people’s opinions at all.
- People think experts’ opinion of “good”, even if not completely the same as theirs, will be highly correlated, and so is better than nothing.
- People want to be seen drinking wine that supertasters would drink, as a sort of cachet thing.
I think #3 above is pretty much the definition of snob, and I think it exists but is not the major reason people do things (but I could be wrong). I’m guessing it’s more typically #2, but even so it doesn’t explain the really expensive high-end wine market’s huge appeal, unless there are way more supertasters than I thought out there.
I think this question, of why people listen to expert advice even when it’s mostly irrelevant, is an important one, because it happens so much in our culture, and clearly not just about wine.
I for one am attracted to the idea of going one step further and ignoring expert advice. I see a natural progression: first, people are ignorant, second, they learn what experts think, and third, they ignore experts and go with their gut.
But even sexier is the idea of never listening to experts at all: skip step two. Am I the only person who thinks that’s sexy? I mean, I guess it mostly means you’re wasting your time, but it also comes out in the end with less herd mentality.
I think this desire I have of skipping the expert advice is very tied into why I despise the echo chamber of the web and how we are profiled online and how our environment is constantly updated and tailored to our profile. It’s in some sense an expert opinion on what we’d like, given our behavior, and I hate the finiteness of that concept, possibly in part because I’ve designed models like that and I know how dumb they are.
Anyway, I’d love to hear your thoughts, and if I’ve missed any reasons why people like to hear partially relevant expert opinion.
Do not track vs. don’t track
There’s been some buzz about a new “do not track” button that will be installed in coming versions of browsers like google chrome. The idea is to allow people their privacy online, if they want it.
The only problem is, it doesn’t give people privacy. It only blocks some cookies (called third-party cookies) but allows others to stick.
Don’t get me wrong- without third-party cookies, the job I do and every other data scientist working in the internet space will get harder. But please don’t think you’re not being tracked simply by clicking on that.
And as I understand it, it isn’t even clear that third-party cookies won’t be added: I think it’s just an honor system thing, so third-party cookie pasters will be politely asked not to add their cookies.
But don’t believe me, visualize your own cookies as you travel the web. The guy (Atul Varma) who wrote this also open-sourced the code, which is cool. See also the interesting conversation in comments on his blog Toolness.
Let me suggest another option, which we can call “don’t track”. It’s when nothing about what you do is saved. There’s a good explanation of it here, and I suggest you take a look if you aren’t an expert on tracking. They make a great argument for this: if you’re googling “Hepatitis C treatments” you probably don’t want that information saved, packaged, and sold to all of your future employers.
They also have a search engine called “DuckDuckGo” which seems to work well and doesn’t track at all, doesn’t send info to other people, and doesn’t save searches.
I’m glad to see pushback on these privacy issues. As of now we have countless data science teams working feverishly in small companies to act as predators against consumers, profiling them, forecasting them, and manipulating their behavior. I’m composing a post about what a data science team working for consumers would have on their priority list. Suggestions welcome.
Charity auctions and hate crimes
I read an absolutely incredible story last night on Bloomberg.
This Morgan Stanley executive William Jennings (co-head of North American fixed-income capital markets) is being charged with a hate crime. Let me piece it together a bit.
On December 22nd Jennings hosted a charity auction at Morgan Stanley until 6pm, then went to Ink48, a hotel in midtown on the west side. After partying on the rooftops for some time, and drinking, his car service didn’t show up fast enough for him so he hailed a cab to take him to Connecticut, where he lives with his wife and three kids in a $3.4 million house.
When he got to Connecticut, he got into a fight with the cab driver and ended up refusing to pay, stabbing the guy in his hand with a knife (which required 60 stitches) while using ethnic slurs. Then he went away to Florida for two weeks on the DL. My favorite line from the article:
Jennings fell asleep during the trip, the driver said. Once at the destination, though, Jennings said “he did not feel like paying” because he was already home.
Up for debate and the trial: did he really refuse to pay or was he just arguing his fare? Was it really 60 stitches or is that an exaggeration? Did he really use ethnic slurs? I’m throwing in these questions because I want to be correct and because the overall point of my post won’t depend on these details anyway.
Not up for debate: he stabbed the cabbie, it was definitely an argument over money, and he was worried enough to go to Florida for two weeks.
Okay, now that I’ve summed this up I’m gonna connect it to charity auctions. Yes I am.
I’ve been to charity auctions myself. I want to devote an entire post to describing what such an event consists of; for now take it from me that they are orgies of self-congratulatory arrogance. And ironically, they are not at all charitable in the sense of being generous and tolerant.
They are in fact celebrations of self-centeredness, displays of careless overabundance. Yes, I’ll pay $120k to go to Australia for a week to golf, and I’ll do it for the poor children, and by the way also because I can afford to throw away such money and especially by the way because everyone in this room now knows that.
So I think it’s extra deliciously ironic that this guy went from that atmosphere to arguing with an Egyptian cabbie over a $200 fare (or maybe $300, if we want to be generous to Jennings and believe his “extortionist cabbie” sob story).
But my point is that, although the cab ride was a different atmosphere from the charity auction, his was not a different attitude at all: both parts of his evening centered on assumptions of entitlement and selfishness and the idea that he is somehow outside the regular rules and cannot be held accountable like normal people. From the article:
He then went on vacation to Florida, police said. Jennings told officers he subsequently called his lawyer after a friend told him police were looking for a suspect in the stabbing incident, according to the report.
“Jennings said he didn’t know what to do — he just wanted the whole thing to go away,” Darien Police Detective Chester Perkowski said in a court document filed with the report.
The part about the car service not showing up is absolutely key: these guys use car services a lot, and when you do that, you get used to not paying for such trivial little things as rides, or for that matter food or drinks. All such things are handed to you for free when you are this important (read: rich). Paying, writing a check or what have you, is reserved for ostentatious displays of wealth. I know hedge fund guys that don’t even carry money in their wallet because they never use cash. Actually I don’t know them personally but I know that this is true because they brag about it in the elevators.
I’m not trying to generalize this story – most Morgan Stanley execs haven’t been charged with knifing down working class cabbies. But it’s impossible for me not to see the consistency in the two events.
This is water
I just started reading Infinite Jest and it’s blowing my mind.
I’m a nerd so I had never heard of David Foster Wallace before reading his book, but now I’ve officially joined his cult. If I’m too late to this party I will start my own, one-woman cult.
As far as I’m concerned he’s the Elliott Smith of literature.
If you haven’t already, please read this, Wallace’s 2005 commencement speech to Kenyon College. I read it at work yesterday and bawled into my keyboard for about 20 minutes.
Open Models (part 2)
In my first post about open models, I argued that something needs to be done but I didn’t really say what.
This morning I want to outline how I see an open model platform working, although I won’t be able to resist mentioning a few more reasons we urgently need this kind of thing to happen.
The idea is for the platform to have easy interfaces both for modelers and for users. I’ll tackle these one at a time.
Modeler
Say I’m a modeler. I just wrote a paper on something that used a model, and I want to open source my model so that people can see how it works. I go to this open source platform and I click on “new model”. It asks for source code, as well as which version of which open source language (and exactly which packages) it’s written in. I feed it the code.
It then asks for the data and I either upload the data or I give it a url which tells the platform the location of the data. I also need to explain to the platform exactly how to transform the data, if at all, to prepare it for feeding to the model. This may require code as well.
Next, I specify the extent to which the data needs to stay anonymous (hopefully not at all, but sometimes in the case of medical data or something, I need to place security around the data). These anonymity limits will translate into the kinds of visualizations and results that can be requested by users but not the overall model’s aggregated results.
Finally, I specify which parameters in my model were obvious “choices” (like tuning parameters, or prior strengths, or thresholds I chose for cleaning data). This is helpful but not necessary, since other people will be able to come along later and add things. Specifically, they might try out new things like how many signals to use, which ones to use, and how to normalize various signals.
That’s it, I’m done, and just to be sure I “play” the model and make sure that the results jive with my published paper. There’s a suite of visualization tools and metrics of success built into the model platform for me to choose from which emphasize the good news for my model. I’ve created an instance of my model which is available for anyone to take a look at. This alone would be major progress, and the technology already exists for some languages.
User
Now say I’m a user. First of all, I want to be able to retrain the model and confirm the results, or see a record that this has already been done.
Next, I want to be able to see how the model predicts a given set of input data (that I supply). Specifically, if I’m a teacher and this is the open-sourced value added teacher model, I’d like to see how my score would have varied if I’d had 3 fewer students or they had had free school lunches or if I’d been teaching in a different district. If there were a bunch of different models, I could see what scores my data would have produced in different cities or different years in my city. This is a good start for a robustness test for such models.
If I’m also a modeler, I’d like to be able to play with the model itself. For example, I’d like to tweak the choices that have been made by the original modeler and retrain the model, seeing how different the results are. I’d like to be able to provide new data, or a new url for data, along with instructions for using the data, to see how this model would fare on new training data. Or I’d like to think of this new data as updating the model.
This way I get to confirm the results of the model, but also see how robust the model is under various conditions. If the overall result holds only when you exclude certain outliers and have a specific prior strength, that’s not good news.
I can also change the model more fundamentally. I can make a copy of the model, and add another predictor from the data or from new data, and retrain the model and see how this new model performs. I can change the way the data is normalized. I can visualize the results in an entirely different way. Or whatever.
Depending on the anonymity constraints of the original data, there are things I may not be able to ask as a user. However, most aggregated results should be allowed. Specifically, the final model with its coefficients.
Records
As a user, when I play with a model, there is an anonymous record kept of what I’ve done, which I can choose to put my name on. On the one hand this is useful for users because if I’m a teacher, I can fiddle with my data and see how my score changes under various conditions, and if it changes radically, I have a way of referencing this when I write my op-ed in the New York Times. If I’m a scientist trying to make a specific point about some published result, there’s a way for me to reference my work.
On the other hand this is useful for the original modelers, because if someone comes along and improves my model, then I have a way of seeing how they did it. This is a way to crowdsource modeling.
Note that this is possible even if the data itself is anonymous, because everyone in sight could just be playing with the model itself and only have metadata information.
More on why we need this
First, I really think we need a better credit rating system, and so do some guys in Europe. From the New York Times article (emphasis mine):
Last November, the European Commission proposed laws to regulate the ratings agencies, outlining measures to increase transparency, to reduce the bloc’s dependence on ratings and to tackle conflicts of interest in the sector.
But it’s not just finance that needs this. The entirety of science publishing is in need of more transparent models. From the nature article’s abstract:
Scientific communication relies on evidence that cannot be entirely included in publications, but the rise of computational science has added a new layer of inaccessibility. Although it is now accepted that data should be made available on request, the current regulations regarding the availability of software are inconsistent. We argue that, with some exceptions, anything less than the release of source programs is intolerable for results that depend on computation. The vagaries of hardware, software and natural language will always ensure that exact reproducibility remains uncertain, but withholding code increases the chances that efforts to reproduce results will fail.
Finally, the field of education is going through a revolution, and it’s not all good. Teachers are being humiliated and shamed by weak models, which very few people actually understand. Here’s what the teacher’s union has just put out to prove this point:
Economists don’t understand the financial system
Cross posted from Naked Capitalism.
A bit more than a week ago I went to a panel discussion at the Met about the global financial crisis. The panel consisted of Paul Krugman, Edmund Phelps, Jeffrey Sachs, and George Soros. They were each given 15 minutes to talk about what they thought about the Eurocrisis, especially Greece, the U.S., and whatever else they felt like.
It was well worth the $25 admission fee, but maybe not for the reason I would have thought when I went. I ended up deciding something I’ve suspected before. Namely, economists don’t understand the financial system, and moreover they don’t get that they don’t get it. Let me explain my reasoning.
The panelists all are pretty left-leaning guys, and each of them basically talked about how the U.S. government should stimulate the economy in one way or another. Krugman kept saying that hey, this isn’t too hard, we’ve seen financial crises before, and this is no different: we should immediately pass a massive stimulus package, that’s the one and only thing that we should be discussing. Sachs was very consistently saying we should do something else: namely, start planning long-term for the future. He focused on the percent of tax dollars going into infrastructure and basic education and research. Phelps also wanted stimulus, but he consistently referred to his own economic models in how exactly it should work. I didn’t completely follow his train of thought.
Soros was the most interesting of the four, in my opinion. He started by saying that we should all acknowledge that, as nice as it would be to think we can model the economy and feel control over the situation, this is a pipe dream and we should get used to not really knowing what will happen when we do one thing versus another. He suggested that we should instead work together to develop a theory, or perhaps even an philosophy, that assumes uncertainty itself. He ended by saying that, even with the three colleagues on the panel with him, who are essentially all united in thinking we need to be proactive, his ideas are essentially being ignored.
The rest of the evening essentially consisted of everyone ignoring Soros and arguing about how Keynesian they all were and how exactly different kinds of stimulus would work and which way they should use 2% of GDP to jumpstart the world’s economy. So basically exactly what Soros said would happen.
It got me more and more riled up. Here are these expert economists, two of whom have Nobel Prizes and the third who runs the Earth Institute at Columbia and is considered a huge swinging dick in his own right, and they don’t seem to acknowledge how much power they actually have over the situation (specifically, not much). For that matter, they clearly don’t know the nitty gritty of the financial system. To listen to them, all you need to do is spread a thick paste of money on the system and it would revive whole cloth. Soros is the exception, probably for the reason that he actually traded and made money inside the system.
At the end I asked a question, since they allowed a few questions, and as you know I’m not shy. I asked how we are going to make the system simple enough to actually make it possible to regulate it. Krugman basically said that Dodd-Frank is going to do it. My conclusion from that is that Krugman must really have only an outline in his head of how this stuff works- the devil, as we know, is really in the detail, and I’m too acquainted with the Volcker Rule’s list of exemptions to have a lot of hope on this score. To be fair, Phelps mentioned Amar Bhide’s book A Call for Judgment, which I’m reading and seems pretty good and at least addresses this exact issue head-on.
Overall, the evening brought me back to the credit crisis, and working at D.E. Shaw, when Larry Summers was consistently quoted at the firm as saying that the “magical liquidity fairy” needed to come and “spread some magical liquidity dust” in the markets to make everything better. No, I’m not kidding.
What I felt then and what I still feel is that these super influential economists are so high on their clean, simple economic models of the world (about the only variables of which are GDP, stimulus, and tax rates) that they focus on the model to the exclusion of the secondary issues. Sometimes you get important results this way: simplifying models can be really useful. But sometimes it’s really truly misleading to do so, and I believe this is one of those cases.
I’m left thinking that they (the economists) are so entranced with their simplified world view that still don’t understand what actually fucked up the world in 2007 and 2008, namely the CDO market’s implosion. Message to Krugman: this is not exactly like other financial crises, because it’s partly caused by complexity, and nobody seems to have the balls to fix it. The problem is that the financial system has been allowed to get so complicated and so rigged in favor of the people with information, that normal people, including homeowners, credit card users, politicians, and regulators have been left in the dark, and many of the little guys are still stuck in ludicrous contracts left over from the outrageous securitizations that took place in the CDO market.
What is especially enraging is how these same economists are still the experts that people turn to to help figure out how to get out of this mess, when they don’t actually understand the mess itself. Why else would a large audience be willing to pay $25 a piece to hear them talk about this? Why else would Obama be considering Larry Summers to lead the World Bank?
As an aside: please, Mr. President, do not let Summers lead the world bank. He does not understand the system well enough to lead it. And he is too arrogant to admit what he doesn’t know. I can introduce you to a bunch of people that may be less imposing but are more informed, more ethical, and wiser. Give me a call any time and we can chat and form a short list of candidates.
By the way, I’m not saying we shouldn’t have a major stimulus, or that we shouldn’t do longer term planning and invest more in infrastructure. I think we should do both. But I also think those efforts will be futile unless we enforce a basic system that is simple enough to be regulated. Otherwise we will be reliving this entire ordeal in another 15 years.
Vikram Pandit: let’s talk
Here’s the coverage from Business Insider.
Here’s the letter (also posted on Naked Capitalism):
Dear Mr. Pandit,
Last October, in an interview with Fortune Magazine, you extended an invitation to Occupy Wall Street for a face-to-face meeting. The Alternative Banking Group, an official working group of Occupy Wall Street, hereby accepts.
As CEO of Citigroup, you recently announced “a new Citi.” You said that you are now “working hard to create a culture of responsible finance.” Our mission as the Alternative Banking Group is exactly the same. We look forward to a fruitful dialogue.
Since this conversation is of importance to the general public, we will have a small camera crew with us to document it. The video will be shared on the websiteoccupy.com, an emerging media platform for the Occupy movement.
Please respond to this email at your earliest convenience to schedule a time and place.
Sincerely,
Cathy O’Neil
Facilitator
The Alternative Banking Group
Occupy Wall Street
Please comment with questions we can ask Vikram if he accepts our offer.
Math-Startup Collaborative at Columbia tomorrow
The Columbia Chapter of SIAM (Society for Industrial and Applied Mathematics) invites YOU to:
Spring 2012 Math-Startup Collaborative Wednesday, February 29 | 6:00 – 8:00 PM | Davis Auditorium, Schapiro Center
Meet Columbia alumni from NYC startups, including Bitly, Foursquare, and Codecademy, and learn about the role of math and engineering in their companies. Students interested in startup careers and internships, or those simply curious to see how math is applied in the wild, are especially welcome. The event, which consists of a series of presentations from our startup members, will be followed by a reception (with free food!). This is sure to be the largest applied math event of the semester!
Startup presenters include:
- Bitly –> http://blog.bitly.com/
- Foursquare –> https://foursquare.com/about/
- Codecademy –> http://blog.codecademy.com/
- BuzzFeed –> http://www.buzzfeed.com/about
- Sailthru –> http://blog.sailthru.com/
Space is limited! Contact Ilana Lefkovitz (itl2103@columbia.edu) for more information.
Please RSVP at bit.ly/Math–Startup-RSVP-2012
Co-sponsored by the Application Development Initiative (ADI). Reception sponsored by AOL Ventures.
Math teaching needs overhaul
My friend Tara sent me a message:
The President’s Council of Advisors on Science and Technology submitted a report on the challenge to producing more college graduates with STEM degrees. In particular, they point out mathematics as a bottleneck, and recommend (on p. 29) that “teaching and curricula [be] developed and taught by faculty from mathematics-intensive disciplines other than mathematics, including physics, engineering, and computer science.” Of course, there are physicists, engineers, and computer scientists on the Council, whereas there is no mathematician.
On some level, they do have a point. They seem to say that we (as a nation) are not doing a good job of teaching K12 mathematics. I strongly disagree with their conclusion that we should therefore take the college-level teaching of mathematics away from the experts in mathematics.
Hmm. I don’t know. I’ve been sounding a warning for a while now that math departments are way too complacent about the way they teach undergrads. I try to make people think of a math department as, to some extent, a brand, and that we should be trying to attract good majors and we should be trying to get more people psyched about math. To that end I am constantly trying to get people to care about the calculus curriculum, which is always at risk of being taken over by the physics, engineering, and economics departments, and I’ve consistently introduced “introduction to higher math” courses which explicitly teach proof techniques.
But there’s a major problem, at least in the very top research departments. Namely, the professors actually think math should be a hard and elite major, and that gives them an excuse to not care about the quality of the undergrad classes. That’s not how they say it, of course, but my experience is that’s how it works.
The other reason I think it makes sense to be a bit concerned about the brand is that if we mathematicians don’t start doing it, then someone else will start doing it for us. This President’s Council of Advisors report is exactly saying that. On the one hand it could be the kick in the ass that math departments need, but on the other hand considering how much reporting they are asking for, it could mean a tremendous amount of paperwork as well as a loss of independence of the math community.
I say mathematicians respond to this by admitting there’s a problem and coming up with a good plan that they organize and control. Otherwise I do think something else will and should be done.
Interestingly, there also seems to be a call in this report for more good math tutors. It reminds me of a commenter from yesterday who wants to start something called “Tutor for America”, which I think is an excellent idea.
Teaching scores released
Anyone who reads this blog regularly knows how detestable I think it is that the teacher value-added model scores are being released but the underlying model is not.
We are being shown scores of teachers and we are even told the scores have a wide margin of error: someone who gets a 30 out of 100 could next year get a 70 out of 100 and nobody would be surprised (see this article).
Just to be clear, the underlying test doesn’t actually use a definition of a good teacher beyond what the score is. In other words, this model isn’t being trained by looking at examples of what is a “good teacher”. Instead, it derived from another model which predicts students’ test scores taking into account various factors. At the very most you can say the teacher model measures the ability teachers have to get their kids to score better or worse than expected on some standardized tests. Call it a “teaching to the test model”. Nothing about learning outside the test. Nothing about inspiring their students or being a role model or teaching how to think or preparing for college.
A “wide margin of error” on this value-added model then means they have trouble actually deciding if you are good at teaching to the test or not. It’s an incredibly noisy number and is affected by things like whether this year’s standardized tests were similar to last year’s.
Moreover, for an individual teacher with an actual score, being told there’s a wide margin of error is not helpful at all. On the other hand, if the model were open source (and hopefully the individual scores not public), then a given teacher could actually see their margin of error directly: it could even be spun as a way of seeing how to “improve”. Otherwise said, we’d actually be giving teachers tools to work with such a model, rather than simply making them targets.
update: Here’s an important comment from a friend of mine who works directly with New York City math teachers:
Thanks for commenting on this. I work with lots of public school math teachers around New York City, and have a sense of which of them are incredible teachers who inspire their students to learn, and which are effective at teaching to the test and managing their behavior.
Curiosity drove me to it, but I checked out their ratings. The results are disappointing and discouraging. The ones who are sending off intellectually engaged children to high schools were generally rated average or below, while the ones who are great classroom managers and prepare their lessons with priority to the tests were mostly rated as effective or above.
Besides the huge margin of uncertainty in this model, it’s clear that it misses many dimensions of great teaching. Worse, this model, now published, is an incentive for teachers to develop their style even more towards the tests.
If you don’t believe me or Japheth, listen to Bill Gates, who is against publicly shaming teachers (but loves the models). From his New York Times op-ed from last week:
Many districts and states are trying to move toward better personnel systems for evaluation and improvement. Unfortunately, some education advocates in New York, Los Angeles and other cities are claiming that a good personnel system can be based on ranking teachers according to their “value-added rating” — a measurement of their impact on students’ test scores — and publicizing the names and rankings online and in the media. But shaming poorly performing teachers doesn’t fix the problem because it doesn’t give them specific feedback.
If nothing else, the Bloomberg administration should also look into statistics regarding whether it’s become a more attractive or less attractive profession since he started publicly shaming teachers. Has introducing the models and publicly displaying the results had the intended effect of keeping good teachers and getting rid of bad ones, Mayor Bloomberg?
I am the most boring person in the world
A few nights ago I went to a CFPB Town Hall Meeting after work. The discussion in the kitchen that morning went something like this:
me: “I’m going to be late tonight, guys, because I’m going to a CFPB Town Hall meeting… I’m really excited about it!”
my husband: “What the hell is CFPB?”
me: “Oh, it stands for the Consumer Financial Protection Bureau. You know, the thing that Elizabeth Warren started but then didn’t get to be in charge of? I need to go see if this guy Cordray is going to be pushy enough to lead an effective government agency. Today the issues at the meeting are things like checking accounts and debit cards and overdraft policies. I totally need to go, can you guys eat leftovers?”
my husband: “You are the most boring person in the world”
my three sons, simultaneously: “Yeah mom, he’s right. You are the most boring person in the world.”
Whatever. I guess they’re right, but I went anyway. After lots of incredibly congratulatory introductions, including a 5 minutes speech from New York Attorney General Eric Schneiderman, there were a bunch of questions from the audience.
There were lots of community groups represented, as well as individuals. Two themes emerged through the questions that seem like particularly egregious consumer issues affecting poor people.
First was the issue of pre-paid debit cards and the corresponding fees. This guy stood up at the microphone and described his friend who get a debit card for child support, court-ordered. But this debit card extracts enormous fees every time she takes money out, including things like $5 just to check the balance. The guy was saying, you know my friend needs that money for her children, and it’s not fair that so much of it goes to fees- it’s abusive. I was totally crying. I mean, I’m an easy cry, but still. That wasn’t the only story about such debit cards where there was no choice in the matter but the fees were extortionist.
Second the issue of Walmart issuing its pay to people in debit card form came up time after time as well. So it seems that Walmart is not only a retailer, but also a financial institution of the crappiest kind now. It issues debit cards as payment to people who don’t accept direct deposit or don’t have checking accounts, and again it seems that the money on the cards is somehow deeply tied to a fee structure. I need to look into this more (as does the CFPB) but I’m wondering off the top of my head whether people can just demand to be paid in cash instead. It’s like these people are being paid really badly, with very few benefits, and even when they get paid they’re being nickeled and dimed every step of the way. It’s like it’s not really their money even then.
So in other words, debit cards are the new check cashers, but maybe worse since their fee structure doesn’t seem to be as transparent.
Of course, I took the opportunity to ask a question too, since I am not shy. And I was told not to ask a question but rather to tell a story, but whatever, I decided to phrase is as “making three suggestions.” After introducing myself as coming from the Alternative Banking group, I mentioned the following:
- The CFPB should use its powers to bring together mortgage investors and homeowners to the same table, in order to align their interests and bypass the banks as servicers, since the banks are only endlessly delaying the process in order to extract fees.
- I mentioned that our group is working on a “find a credit union app” but that the CFPB should really be doing that with us, to help underbanked people find alternatives to crappy banking solutions (like debit cards).
- I mentioned that we had submitted a public comment letter demanding that the credit score models be open sourced, since there was no legitimate reason for such models, which directly affect consumers in their daily lives, to be kept proprietary.
Akshat was there too, from Occupy the SEC, and he asked about the Volcker Rule.
Condray took notes. I mean, what’s he going to say.
Well actually sometimes he did say stuff, like to Akshat, and for the most part it was something along the lines of, “that is not in our jurisdiction”, although there was one exception when he talked about how Walmart, being a retailer, is not in his jurisdiction but since it’s acting as a banking institution it actually is.
Overall I’m a bit disappointed. Although I did certainly like the fact that he held a town hall meeting at all, I am worried that he’s just too nice, and that he’s going to try to please everyone and be kind of wishy-washy. I would have loved to see him manage to sustain disgust at the abuses he was hearing about, but instead he sounded more concerned than angry, and I would put my money on angry any day. I want the CFPB to be led by a son-of-a-bitch that pisses people off and constantly tried to enlarge his jurisdiction rather than keeping well inside the lines. Time will tell.
mathbabe 