The creepy mindset of online credit scoring
Usually I like to think through abstract ideas – thought experiments, if you will – and not get too personal. I take exceptions for certain macroeconomists who are already public figures but most of the time that’s it.
Here’s a new category of people I’ll call out by name: CEO’s who defend creepy models using the phrase “People will trade their private information for economic value.”
That’s a quote of Douglas Merrill, CEO of Zest Finance, taken from this video taken at a recent data conference in Berkeley (hat tip Rachel Schutt). It was a panel discussion, the putative topic of which was something like “Attacking the structure of everything”, whatever that’s supposed to mean (I’m guessing it has something to do with being proud of “disrupting shit”).
Do you know the feeling you get when you’re with someone who’s smart, articulate, who probably buys organic eggs from a nice farmer’s market, but who doesn’t expose an ounce of sympathy for people who aren’t successful entrepreneurs? When you’re with someone who has benefitted so entirely and so consistently from the system that they have an almost religious belief that the system is perfect and they’ve succeeded through merit alone?
It’s something in between the feeling that, maybe you’re just naive because you’ve led such a blessed life, or maybe you’re actually incapable of human empathy, I don’t know which because it’s never been tested.
That’s the creepy feeling I get when I hear Douglas Merrill speak, but it actually started earlier, when I got the following email almost exactly one year ago via LinkedIn:
Hi Catherine,
Your profile looked interesting to me.
I’m seeking stellar, creative thinkers like you, for our team in Hollywood, CA. If you would consider relocating for the right opportunity, please read on.
You will use your math wizardry to develop radically new methods for data access, manipulation, and modeling. The outcome of your work will result in game-changing software and tools that will disrupt the credit industry and better serve millions of Americans.
You would be working alongside people like Douglas Merrill – the former CIO of Google – along with a handful of other ex-Googlers and Capital One folks. More info can be found on our LinkedIn company profile or at www.ZestFinance.com.
At ZestFinance we’re bringing social responsibility to the consumer loan industry.
Do you have a few moments to talk about this? If you are not interested, but know someone else who might be a fit, please send them my way!
I hope to hear from you soon. Thank you for your time.
Regards,
Adam
Wow, let’s “better serve millions of Americans” through manipulation of their private data, and then let’s call it being socially responsible! And let’s work with Capital One which is known to be practically a charity.
What?
Message to ZestFinance: “getting rich with predatory lending” doesn’t mean “being socially responsible” unless you have a really weird definition of that term.
Going back to the video, I have a few more tasty quotes from Merrill:
- First when he’s describing how he uses personal individual information scraped from the web: “All data is credit data.”
- Second, when he’s comparing ZestFinance to FICO credit scoring: “Context is developed by knowing thousands of things about you. I know you as a person, not just you via five or six variables.”
I’d like to remind people that, in spite of the creepiness here, and the fact that his business plan is a death spiral of modeling, everything this guy is talking about is totally legal. And as I said in this post, I’d like to see some pushback to guys like Merrill as well as to the NSA.
mathbabe 
What no comments yet?
I find it interesting that only the individual gets to be analyzed. What used to be called the ‘little guy’ now more like the suckers or sheep.
When will the ‘scores’ for the companies be published so people who have the desire can make an informed decision of where not to shop or where not to work?
LikeLike
I know someone who worked directly for him at this venture, and while he didn’t sound like someone I’d personally want to know, I _did_ give him social responsibility cred for looking to provide a less-evil alternative to current payday lenders.
LikeLike
Good stuff and keep it coming as it relates to the data selling epidemic. So much of the credit information is becoming more flawed as they don’t care about accuracy, they just want to make a buck and when errors are made and found, who gets the job, you do on your own dime and it can be costly. There’s no accountability and I hope what you said is true that at some point the models are looked at and more disappointingly so they are legal. We have so many digital duds in government today that can’t see this.
With the mention of FICO, they have their behavioral analytics using your credit score and other information that claim to be able to predict if you will be a patient that will take your prescriptions and they score you on other additional data. The Query Masters at some places will just combine any kind of data to make a buck. They call it their Medication Adherence program and sell it to insurers and more. You get a score between zero and 500, so you can see how this data if used out of context could eventually affect what kind of prescriptions you might be able to get.
http://ducknetweb.blogspot.com/2011/06/fico-analytics-press-release-marketing.html
Well maybe my campaign to have all data sellers including banks buy a license, the only way I can see to even begin regulating some of this and excise tax those making billions doing it. FICO must have wanted to sell the analytics in a bad way as a little later down the road they bought a healthcare revenue cycling software system..go figure. It appears like Zest wants to give creepy FICO a run on getting some of this so called business. So how long before Zest rolls out a medication compliance set of algos too:) We come back to context and ethics and too many out there flipping bad and evil algos for profits.
http://ducknetweb.blogspot.com/2012/11/fico-buys-cr-software-and-gains-access.html
LikeLike
Awesome, MB. Not sure, however, that we can ever regulate creeps away from commerce any more than we can from, say, seedy dive bars. Creeps are themselves rampant in our regulatory agencies (and, God knows, the politicians who create them) so the remedy for these creeps (I prefer the phrase ass clowns) is a bitch. But you keep us thinking, and me personally about matters such as Big Data that I’d not otherwise have know about. For that you rock — thank you!
LikeLike
Thankfully, not quite. The model they are talking about possibly constitutes a credit reporting service pursuant to the definitions of the Fair Credit Reporting Act (FCRA), 15 U.S.C. 1681 et seq. It’s one thing to collect this data on people: the moment that you start using it to make credit decisions, some of them adverse, you run into a pretty formidable wall of regulation. Of course, these guys are probably sure that the law doesn’t apply to them – and would probably just work to get this (and any other consumer friendly legislation) expunged. But at the moment, there is a reason that these businesses are fairly expensive to run. And people are starting to get fairly saavy about how this works from a consumer standpoint. I see lots of frivolous lawsuits, but now and then one sticks. The violation has to be willful before you get actual damages. But I’m guessing some of this guy’s powerpoint presentations would be very useful towards proving willful violations. Just a guess.
I love the law. Even now, when they’ve done so much to gut it.
LikeLike
Privacy is a two-edged sword. In a representative democracy, for good or ill, it is the companion of free choice and free enterprise. It basically means keeping Powers Out There out of the individual’s business, personal or commercial. Thus, the same arguments that may be marshalled against invasions of individual privacy can be marshalled against rules or governance telling commercial entities what and how they behave. The only common point is the Governance. Hence, in the States, when “privacy” is raised, the Governance is the target, in the limit.
Alas, note the same target was raised and highlighted with the imposition of health and food safety regulations.
Note also that in several countries in the EU, while they deeply distrust the Googles of the world, they consider their government on their side. So Norway, a reasonable bastion of liberal values, trusts its government enough to allow all Internet traffic to first flow through their servers on its way to its public, because they want their government to help protect them from the ills and evils out there.
For anyone who knows anything about the actual Internet, in contrast with some imagined fairyland, there’s much out there who seeks to simply gain at anyone’s expense, and it is there each day.
In reasonable, liberal societies with dependence upon such structures, I propose a question: What organization or nexus will protect them against these? And what would you imagine it look like? Do you think they’ll go away if the Governance goes away?
Which is worse? A government? Or a bunch of corporations which have their quarterly earnings as their Prime Directive?
LikeLike
The corruption of formal modelling in banking and finance is the result of the competitive culture that we live in.
The huge challenge consists in de-programming the indoctrination that starts in people’s homes, in kindergarten, and in primary school. Children encounter the concepts of money and economics through the lens of competition, and the concepts are presented as a given. Critical thinking on this topic is not encouraged. Instead, primary school children are taught at length how to write “persuasive” texts. Whilst the word peruasive is defined as good at persuading someone to do or believe something through reasoning or the use of temptation, reasoning takes the back seat relative to the use of temptation.
The indoctrination continues throughout higher education, including university curiculums in economics and finance that don’t explain the construction of money, and a popular culture that celebrates the posession of “wealth”, which has morphed into a synonym for “financial products”. To really cement the role of money as a store of status and value, the plain term of “financial product” is replaced by “financial asset” for marketing purposes.
As a result, in the everyday language we use, the default interpretation of the terms value, asset, and wealth is heavily biased towards the semantics of financial products. The effect is that even those talking about alternative approaches to household management of scare resources (economics) can end up being interpreted as talking about new forms of financial products by a large proportion of the audience.
The indoctrination is complete when it comes to those working in the financial sector. Yes, they do experience and see the symptoms of a system that is no longer working, but they are only ever looking for solutions within the box of the established dogma of competition. Most economists have also fully fallen into the trap of indoctrination, to the extent that leading economists can get by with very limited exposure to behavioural economics and without any exposure to biological and neurological evidence. Some confidently claim that “decision making for personal financial gain is human nature”, a statement I’ve heard first hand just a few months ago.
The concept of competition has permeated everyday language in very subtle ways, leading nearly everyone who is not familiar with recent scientific insights into human behaviour down the garden path. If you then consider the average level of appreciation for and understanding of complex systems found amongst politicians, the picture is starting to look fairly bleak.
A new paradigm for household management must start with the language we use and the language we consciously avoid. I have gotten into the habit of defining basics terms such as value, asset, and wealth before using them in a conversation, to minimise the risk of being misunderstood.
In terms of impact, the worst abuses of personal information are not perpetrated by governments but by the small number of corporations that are competing for our data. Until recently is was largely futile to talk about the issue given the lack of awareness and concern in the wider public. Have outlined the war over data in the following post http://jornbettin.com/2013/07/09/governance-of-big-data-cloud-formations-cyclone-alert/.
Over the last five years I have noticed that investors in technology ventures have become increasingly desperate in their attempts to buy their way, and to transfer their power, from the monetary economy into the data economy. Money is flowing into increasingly ridiculous ventures that have no tangible value to offer to society, which in turn says a lot about how much or how little value is left in the interest bearing and debt-based money we use.
LikeLike
@Jorn Bettin, I almost agree, but not quite. After all, early probability was moved by questions like how to divvy up the remaining pots of prematurely ended games of chance. In their great book, THE MATHEMATICAL EXPERIENCE, which I recommend everyone with a technical inclination to read, Davis and Hersh have a subsection titled “Underneath the Fig Leaf” dealing with “Mathematics in the Marketplace” and “Mathematics and War” among other topics, which trace the history to the 1400s, in the case of commerce, and the French Revolution, in the case of war.
It is true that may in business — and in war-related activities — see little difference between the Tolkienesque wizard and a group of physicists or mathematicians who can coax success out of banality and perhaps even out of failure. That they fail to understand the side conditions and complexities is partly the responsibility of those who serve them. On the other hand, in my interaction with some of these people, I find they simply do not care.
I find it’s worth reading various deconstructions and interpretations of the 2007 fisaco, including nearly anything on it but Professor Andrew Lo of MIT, per http://web.mit.edu/alo/www/Papers/august07.pdf and http://oversight-archive.waxman.house.gov/documents/20081113101922.pdf
LikeLike
This sort of thing is spreading like kudzu and will soon be the norm. Here’s another company in Europe doing the same: http://www.kreditech.com The VC community seems very excited about these kind of businesses, so expect more of them in the next few years.
LikeLike