Home > data science, guest post > Sentiment should not be the new horizon in journalism

Sentiment should not be the new horizon in journalism

January 25, 2013

This is a guest post by Anchard Scott, and is cross-posted at aluation.

Nate Silver’s high-profile success in predicting the 2012 election has triggered a wave of articles on the victory of data analysts over pundits. Cathy has already taken on the troubling aspects of Silver’s celebrity, so I’d like to focus instead on the larger movement toward big data as a replacement for traditional punditry. It’s an intriguing idea, especially given the sad state of political punditry. But rather than making things better, it’s entirely possible that the methods these articles propose could make things even worse.

There’s no question that we need better media, especially when it comes to politics. If we take the media’s role to be making sure that voters are informed, then they’re clearly doing a poor job of it. And one of the biggest problems is that political coverage has largely abandoned any pretense of getting to the truth in favor of “he said/she said” and endless discussion of the horse race, with the pundits being the worst offenders. Instead of “Will this be good for citizens?” we get “Will this be good for the Democrats/Republicans in the next poll?”

This is where the big data proposals enter the picture, and where I think they go wrong. Rather than addressing the accuracy or usefulness of the information being provided to us as voters, or working to shift the dialogue away from projections of how a given policy will play in Iowa, the proposals for big data revolve around replacing pundits’ subjective claims about shifting perceptions with more objective analysis of shifting perceptions.

For example, this piece from the Awl convincingly describes the potential for the rapid analysis of thousands or even millions of articles as a basis for more effective media criticism, and as a replacement for punditry by “anecdata.” A more recent post from the Nieman Journalism Lab at least acknowledges some methodological weaknesses even as it makes a very strong case for large-scale sentiment analysis as a way of “getting beyond pundits claiming to speak for others.” By aggregating and analyzing the flow of opinion across social media, the piece argues, journalism can deliver a more finely tuned representation of public opinion.

It’s true that perceptions in a democracy matter a lot. But it’s also true that getting a more accurate read on perceptions is not going to move us toward more informative coverage, let alone toward better politics. Worse still, these proposals ignore the fact that public perception is heavily affected by media coverage, which implies that pulling public perception more explicitly into the coverage itself will just introduce reflexivity rather than clarification.

In other words, we could end up with a conversation about the conversation about the conversation about politics. Is that really what we need?

As I see it, there are two precedents here, neither of which is encouraging. Financial markets have been treated as a source of perfect information for a very long time. The most famous justification for this was Hayek’s claim that the price system inherent in markets acts as “a system of telecommunications” that condenses the most relevant information from millions of agents into a single indicator. Even if we accept this as being true when Hayek wrote his essay in 1945 (which we shouldn’t), it’s certainly not true now. That’s in part because financial markets have attracted more and more speculators who base their decisions on their expectations of what others will do rather than introducing new information. So rather than informational efficiency, we get informational cascades, herding and periodic crashes.

The other example is consumer markets, which have the most experience with sentiment analysis for obvious reasons. In fact, this analysis is only the latest service offered by an enormous industry of advertising, PR and the like that exists solely to engineer and harness these waves of sentiment and perception. Their success proves that perception doesn’t exist in some objective void, but is closely shaped by the process of thinking about and consuming the very products it’s attached to. Or to be wonky about it, preferences can be more endogenous than exogenous in a consumer society.

Which is ultimately my point. If we want to treat the information provided by the media – the primary source of information for our democracy – as a more and more finely tuned consumer good whose value is determined by how popular it is, then this sort of analysis is emphatically the way to go. But we should not be surprised by the consequences if we do.

Categories: data science, guest post
  1. chaletfor2
    January 25, 2013 at 10:24 am

    Very astute analysis

    • January 26, 2013 at 12:18 pm

      Thank you chaletfor2 – I appreciate the feedback. This is something I think about a lot, but getting the ideas down in writing was trickier than I expected it to be.

  2. None
    January 25, 2013 at 4:04 pm

    “The most famous justification for this was Hayek’s claim that the price system inherent in markets acts as “a system of telecommunications” that condenses the most relevant information from millions of agents into a single indicator.”

    I don’t think that means what you think it means. Even if you think the price may not reflect the reality of what something “should be worth”, at every single point it is still precisely what Hayak claimed it was: the synthesis of everybody’s interpretation and actions concerning various degrees of known information into a single indicator, it’s price.

    “So rather than informational efficiency, we get informational cascades, herding and periodic crashes.”

    A share price crash is generally utterly irrelevant to a company if it is only led by speculative selling. The company still makes its money and pays its dividends. If anything, it merely provides opportunity for non-speculative factual based investors to buy at bargain prices at the speculators expense.

    You can not get away from the fact that share prices are driven in the short term by sentiment. If sentiment changes over a matter of days due to news that some countries may default on their debts, you are going to get massive swings in share prices. This is simply rational. Most of the big swings at the moment are being caused by changes in sentiment concerning various countries massive debt levels, not speculation on individual companies per se. If there was not such massive governmental debt there would much smaller chance of “(inter)national recessions” – so the underlying fundamental problem in share price swings at the moment is governmental financial perturbation (again).

    • January 26, 2013 at 12:15 pm

      It sounds like you’re a Graham & Dodd kind of investor. That’s great – the sorts of dislocations I describe are crucial for someone like you to make money.

      The bigger problem I didn’t have the space to get to is that your interpretation of price is not the same as the way the rest of the world treats the price. I wasn’t able to find any polling data for this, but I have a hunch that for many Americans the stock market is seen as synonymous with the economy (or at least was until recently). So if the market is going up, then it must be true that “things are getting better.” But as you know, that’s not true at all – the problem is that the way index movements are reported every day, and the way economic news is framed, implies that this is the case, so this is what people believe. So looking at stock prices as a source of information about the economy – as opposed to as a source of information for an investor like you – leads to bad policy, misinformation and a general clouding of issues rather than a deeper understanding.

      There is also an enormous global policy structure built around the idea that these prices are efficient carriers of information about production and demand. This is the core of the Washington Consensus in agricultural policy, for example, which led in the 1980s and 1990s to demands that less-developed countries dismantle their local commodity boards, fertilizer and seed subsidies and other means of buffering local producers and consumers. The claim was that global prices were a much better guide precisely because they have more information content. But look what happened in 2007 – I have yet to see a compelling argument that fundamentals justified a doubling of food prices in global futures markets in less than a year. The results were terrible. Sure, many long-short investors stepped in and made a lot of money shorting commodities, but the millions of people who fell into poverty sure didn’t.

      And as to your final point, I agree with most of what you say there (my only quibble is the point about debt – I’m in the came that sees QE/LOTR and other central bank interventions as the driving factor in asset prices at the moment, rather than debt). Sentiment does indeed drive prices in the short term when there are big macro factors at work because there is no other way to evaluate what is happening. But here again, is this what we want to base an economy on? It strikes me as incredibly unstable.

      This is getting to be extremely long so I’ll stop here (and you may have just inspired a longer post, so I thank you for that), but my bigger point is something along the lines of this – we should let markets be markets. The problem is not the markets themselves, it’s the assumptions we make about them and the range of political solutions we rely on them to provide based on those assumptions that are the problem.

      At any rate, thank you for a thought-provoking comment.

  3. None
    January 27, 2013 at 5:54 am

    In the medium and long term prices ARE absolutely a very efficient carrier of information about production and demand. According to wikipedia the “Washington Consesus” was for “crisis wracked developing countries”. In situations like that you invariably have local government corrupt organisations handing out subsidies for production of X to their favoured (usually for a cut) partners, while levying huge taxes on imports of X. The result is poverty and corruption. The international price X in these situations is absolutely a much better indicator of true value. If someone wanted to, and could, produce and sell something cheaper locally with no subsidies that would be absolutely welcome and encouraged.

    As for the “doubling” in food prices in 2007-2008 well there is a wikipedia article on that enumerating a range of causes. The idea that it was largely speculation is absurd. For me it’s clear the rising price of oil, a global financial crisis, and droughts combined with the (inefficient and governmentally induced) removal of 100 million tons of corn annually to produce a small quantity of oil were probably the main drivers. According to the wikipedia article global food stocks had also reached quite a low point. If you want to claim it was speculators driving the price up, you have to asked who they sold their positions to to make a profit ? It’s not like it’s a bubble where everyone was climbing aboard to buy food in the expectation it would go up in value. Btw, I note reading the article that most of the “food riots” were actually “food and fuel” riots.

    “But here again, is this what we want to base an economy on? It strikes me as incredibly unstable.”

    It blows my mind that educated people can say this kind of thing. The idea that prices should be set by a subset of people for everyone else because they know what the correct price should be is just insane. Yes, when prices float they are sometimes a little bit unstable but in the long run it becomes, correctly, a weighing machine rather than a voting machine and that’s the guarantee of ultimate value. When prices rise, production increases. We want more food, the solution is simply higher prices because increased food production must come at the cost of production of something else.

  4. None
    January 27, 2013 at 11:17 am

    Btw, something not mentioned in the wikipedia article on the world food price crisis which only just occurred to me was the spike in the Baltic Dry Index around exactly the same time. Shipping prices exploded. This must have had significant effects on food transportation costs. Of course as soon as these prices went up so much, a huge number of ships were ordered, and the glut of ships coming online a year in the future brought in the speculators who could forsee the price dropping back due to the increase in fleet size and shorted the BDI. The result was a complete crash in the BDI to a much more reasonable level, with food prices declining again shortly afterwards.

  5. Daublin
    January 28, 2013 at 6:44 pm

    I agree with your title, but all your support sounds like you are demagoguing the talking points for a certain well-known political party. Maybe you should read fewer sentiment articles yourself!

    Regarding your swipes at market mechanisms, note that speculators often *dampen* price swings. A simple example is that grocery stories speculate in the weeks leading up to Halloween that there will be a lot of candy sells. If it weren’t for such speculation, then candy prices would be sky high and stores would be running out. Thankfully, groceries stores speculate like crazy, and prices are much more stable.

    Regarding your swipe about socialized medicine, note that any such system must have some way to say no. In the U.S., the IPAB has been set specifically to find ways to reduce spending on things that are relatively marginal. So I think I’d disagree with your linked article on which way the media has been misleading. It has been far more gracious than deserving to politicians who say that Medicare does not and will not ration care. It’s eggreciously false, and the people saying it are people who know better.

  6. thebrasstack
    February 3, 2013 at 1:20 pm

    Yes, yes, yes!

    This is a giant problem. A bubble, as far as I understand, is just a a blow-up solution to a differential equation. Divergent paths are just as possible as convergent paths. And sentiment analysis is really going to take us in bizarre positive feedback loops.

    But here’s the thing: incentives are local. Advertisers and media companies benefit from sentiment analysis. Politicians benefit (if they’re smart enough to see where the wind is blowing.) A lot of people have an interest in decoupling perception from reality and making it feed on itself endlessly. Every time you get a little better at optimizing, you get a little more money.

    Realistically, you’re not going to outlaw statistics. I don’t think the most ruthless totalitarian regime could manage that. So, like it or not, we’re moving towards a click-ocracy, unless we can figure something else out. I don’t know what “something else” would be, in any but the vaguest terms, but it’s something like “not using a greedy algorithm for everything.” The difference between what a chess-playing robot does (make a game tree all the way out to the victory condition and rule out branches that lead to a loss) and what a Roomba or a paramecium does (gradient descent). Now, how do you make government, or the economy, look more like Deep Blue than a Roomba? I don’t have a clue, but it’s worth thinking about.

  7. foakleys
    April 9, 2013 at 11:33 am

    I can tell you that what those administrations had to deal with are water off a duck’s back compared to what happened 3 years ago. My friends and I were talking about economic Armageddon 3 years ago. However, we’re missing the correct case study in history. The last time we had something like this was indeed the 1930s–not the early 80s. foakleys</a%3

  1. No trackbacks yet.
Comments are closed.
Follow

Get every new post delivered to your Inbox.

Join 1,300 other followers

%d bloggers like this: