mathbabe

I asked my friend Nikolai last week what I should learn if I want to be a really awesome data scientist (since I’m an alpha female I’m sure I phrased it more like, how can I be even more awesome than I already am?).

Being a engineer, Nik gave me the most obvious advice possible: become an engineer.

So this past weekend I’ve looked in to learning Scala, which is the language he and I agreed on as the most useful for large-scale machine learning, both because it’s designed to be scalable and because the guys at Twitter are open sourcing tons of new stuff all the time.

That begs the question, though, to what extent can I become an engineer by reading books about languages in my spare time? According to Nik, real coding is an experience you can’t do alone. It’s more like joining a band. So I need to read my books, and I need to practice on my computer, but I won’t really qualify as an engineer until I’ve coded within a group of engineers working on a product.

Similarly, a person can get really good at an instrument by themselves, they can learn to play the electric guitar, they can perfect the solos of Jimi Hendrix, but when it comes down to it they have to do it in conjunction with other people. This typically means adding lots of process they wouldn’t normally have to think about or care about, like making sure the key is agreed upon, as well as the tempo, as well as deciding who gets to solo when (i.e. sharing the show-offy parts). Not to mention the song list. Oh, and then there’s tuning up at the beginning, choosing a band name, and getting gigs.

It’s a similar thing for coders, and I’ve seen it working with development teams. When they bring in someone new, they have to merge their existing culture with the ideas and habits of the new person. This means explaining how unit tests and code reviews are done, how work gets divided and played, how people work together, and of course how success gets rewarded. Moreover, like musicians, coders tend to have reputations and egos corresponding to their skills and talents which, like a good band, a development team wants to nurture, without letting itself become a pure vehicle to it.

Which means that when you hire a superstar coder (and yes the word seems to be superstar- great coders can do the job of multiple mediocre coders), you tend to listen more carefully to their ideas on how to do things, and that includes how to change the system entirely and try something new, or switch languages etc. I imagine that bands who get to work with Eric Clapton would be the same way.

I’ve been in bands, usually playing banjo, as well as in chamber music groups back when I played piano, so this analogy works great for me. And it makes me more interested in the engineering thing rather than less: my experience was that, although my individual contribution was slightly less in a band setting, the product of the group was something I was always very proud of, and was impossible to accomplish alone.

Now I don’t want you to think I’ve done no coding at all. As a quant, I learned python, which I used extensively at D.E. Shaw and ever since, and some Matlab, as well as SQL and Pig more recently. But the stuff I’ve done is essentially prototyping models. That is, I work alone, playing with data through a script, until I’m happy with the overall model. Since I’m alone I don’t have to follow any process at all, and trust me you can tell by looking at my scripts. Actually I wrote some unit tests for the first time in python, and it was fun. Kind of like solving a Sudoku puzzle.

The part I don’t think works about the coding/ band analogy is that I don’t think coders have quite as good a time as bands. Where are the back-stage groupies? Where are the first two parts of sex, drugs, and rock ‘n’ roll? I think coding groups have their work cut out for them if they really want to play on that analogy.