mathbabe

This is a guest post by mathematrucker.

There are a lot of pros and cons to being an over-the-road (OTR) truck driver, namely, one who spends weeks or even months at a time on the road. The pros can outweigh the cons for those like me who enjoy long highway trips. But this may be about to change.

A huge issue right now is surveillance. Inward-facing cameras that keep a constant watch on the driver may soon become the norm. Swift Transportation (the largest carrier in the U.S.) began installing them in all its company-owned trucks a few months ago.

Most OTR drivers are allowed to drive up to eleven hours per work shift and seventy hours every eight days. Their actual driving hours frequently reach these limits. That’s a lot of time to be in front of a running camera, never knowing for sure who might be watching you. If these cameras become widespread they are sure to cause many drivers (including me) to look for different work.

ELDs and Hours of Service Rules

One surveillance tool that is already well established is the electronic logging device (ELD). The Federal Motor Carrier Safety Administration (FMCSA, the regulatory body that covers interstate trucking) recently sent a rule to the White House for final review mandating that all trucks use them.

Federal hours-of-service regulations require drivers to maintain an up-to-date log book. Unlike paper logs, ELDs effectively prevent drivers from driving way past the legal limits. But they can also severely hinder drivers from driving short distances when they need to.

Most OTR drivers are paid by the mile—the more miles they drive, the more money they make. This provides a strong incentive to use all eleven driving hours per work shift. With paper logs, if a driver needs to exceed the limit by a few minutes to get to a safe place to sleep (versus stopping after say ten hours, possibly sacrificing some pay), they can. With ELDs this same scenario might force the driver into choosing between (1) sacrificing pay, (2) sacrificing overnight safety by stopping wherever, or (3) recording a logging violation to get to the safe place.

But many ELDs offer a fourth option as well: gaming the device (without tampering with it). Carriers can and usually do program some flexibility into ELDs. For example, trucks might be allowed to travel up to one mile below 10 MPH without the current duty status going to line 3 (driving), with resets occurring every time it does go to line 3. (Anyone catching a whiff of loophole here may want to hold their nose before reading the next sentence…) Many ELDs update and display the current duty status every second, but only record it at the top of every minute.

If the truck is not on a freeway, the driver can easily game such a device by alternately accelerating to around 45 MPH and decelerating to below 10 MPH once every minute, perhaps signaling a phantom turn to help avoid notice. To keep from decelerating too late, a smartphone can be used to watch the time to the second. Though no logging violation gets recorded, this technique does leave a trail of evidence on the device that might be noticed in an audit, but audits are infrequent. The alternative—recording a logging violation—will be detected immediately.

The FMCSA’s proposed mandate should require that ELDs record duty status by the second. It probably doesn’t.

The hours-of-service rules themselves are far from perfect. For one thing they do too little to prevent employers from depriving drivers of sleep. Ones who sleep nights are routinely directed to drive all night. Such orders went into my personal “go ahead, fire me if you need to” (for refusing) category many years ago, but this shouldn’t have been necessary.

The hours-of-service rules never said anything about time of day until a new rule was introduced in 2013 requiring two 1 AM to 5 AM periods in every thirty-four-hour rest break (such breaks reset hours driven to zero). Strong industry resistance caused this rule to be suspended in December 2014.

Of course, the problem of fatigue at the wheel will finally be solved when automation replaces truck drivers. Some studies predict it will happen soon. Anyone under the age of thirty (forty?) should take this into consideration if they are thinking about becoming a trucker.

Per Diem Pay

Due to a corporate tax strategy that has gained wide acceptance in recent years, income figures reported nowadays for OTR truckers are probably considerably lower than actual—perhaps by as much as twenty percent.

For many years the tax code has supplied OTR truck drivers with a surprisingly generous deduction called the standard meal allowance. For the past several years it has been $47.20 (80% of $59) per day on the road. Multiplied by 300 days (a typical number), this equals $14,160. The driver’s log book suffices to document how many days were spent on the road.

This sweet tax deduction sours into something called per diem pay when employers decide to get in the middle and “reimburse” drivers instead. They get the money to pay for this by reducing wages. Some companies report this nontaxable pay in Box 14 “Other” on the W‑2 but many do not report it at all; they are not required to.

Per diem pay is bad for drivers and good for companies. Companies mainly benefit from the reduction in payroll taxes. Per diem can also subtly reduce driver vacation pay per week: many companies pay 1/52 of the previous year’s earned income. Drivers also get socked with an “administrative fee” when per diem pay is used. I’ve seen this fee as high as 3% of gross income. The other downsides to per diem are too numerous to go into here. More info can be found in this article.

The reason I left my longtime, relatively well-paying OTR job five years ago wasn’t so much because my employer switched over to using per diem pay, it was more because of a deceptive “more pay in your pocket” ad campaign it foisted on drivers while per diem was still optional. Most drivers were not fooled by it. After spouting the same nonsense over and over for nearly two years, the company finally made per diem mandatory in late 2009.

Rather than resign immediately, I decided to stay on until the following June when my annual vacation pay would accrue. Without knowing it at the time, this decision would also bring me the million-mile safe driving award (during calendar years in which I had no preventable accidents, my paid miles at this company added up to more than a million by 2010). The purpose of these awards is mainly to promote the company image, but they do also look good on the resume, so I gladly accepted.

Closing Remarks

Returning to the theme of pros and cons, I close with a few pros:

• It doesn’t take an inordinate amount of training to get behind the wheel of a truck. Many say more training should be required—and they are probably right—but as of now it doesn’t take that much. Viewed strictly within the context of job-seeking my years of formal math education have served little to no useful purpose to date. By contrast, in 1994 after just a three-week training course plus three more weeks of paid, on-the-road training, I was earning a modest living doing something that didn’t even seem like work (it still doesn’t).
• One of the standard pros people cite is you don’t have a boss breathing down your neck. You do have one telling you where to go and when, but yes, the physical distance helps. Unfortunately inward-facing cameras threaten to obliterate the no-boss-breathing advantage.
• It’s real easy to get out of jury duty if you’re an OTR driver and you don’t want to go.

I’ve been enjoying thinking about ways for Uber drivers to game the surge pricing algorithm at Uber. I don’t know how it works, exactly, but I’m going to imagine that it’s along these lines:

• there are well-defined neighborhoods in a city. This seems to be corroborated by the way the Uber app works for both drivers and riders.
• in a given neighborhood, there are two groups: people asking for a ride who haven’t yet been picked up, and drivers looking to give a ride.
• If the number of riders is 5 more than the number of drivers, then it becomes a “surge zone” for some amount of time, say 30 minutes.

Of course, I made up the numbers 5 and 30, but I’m guessing it’s more or less of this form, and those particular values don’t matter for the rest of the discussion anyway.

So here the thing, Uber wants to keep their riders happy, but to do that they actually tend to want to avoid creating surge situations, since surge situations usually imply riders wait longer and pay more. On the other hand, Uber drivers prefer surges, since they get paid more, and sometimes much more.

That means Uber drivers have a great incentive to game the system and create artificial surges. One way they can do this is by waiting outside an area that might become surge, wait for it to become surge, and then go into that area and swoop up a rider.

But it would make a lot more sense for drivers to work together to do this. Imagine what would happen if all the drivers agreed to sit together in some central location, wait for surge pricing somewhere, and then assign people in order to go get those riders. Pretty much all the rides would become surge. Again, that wouldn’t make the riders happy, but it would benefit the drivers.

All they’d need to coordinate this is something like a walkie talkie system. Or an app. And oh, wait, such a thing already exists, and it’s called Blinkr (hat tip Alex Rosenblat). Instead of congregating in the same place, though, they had an even simpler idea, namely to turn off their Uber app, thus decreasing the local supply of drivers, then wait for surge pricing.

It’s something like an Uber strike, and it requires coordination, but I don’t think it’s illegal, right? I mean, Uber can’t fire them for doing this, since they aren’t employees, right?

I’m super proud to say that Alex Carp, a journalist who was present for more than half of Occupy Summer School last month, did a fantastic job of writing up the OSS experiment for the New Yorker’s Talk Of The Town column.

Here is Alex’s New Yorker Piece, called “Protest U,” please enjoy!

We also got coverage from the German Public Radio, which includes a picture:

I’m the blue-haired person on the left.

Anyone who speaks German can go ahead and tell me if it’s a nice piece.

What, what? My friend Becky Jaffe sent me this video of Tig Notaro giving advice:

I thought it was moving and intimate, just like an advice column should be. What do you think, should Aunt Pythia go video?

Pros for Aunt Pythia going video:

1. the connection with the audience,
2. doesn’t have to wear anything below the waist,
3. can show nutella crepes in real life, no more boring pics.

Cons for Aunt Pythia going video:

1. maybe too much intimacy (TMI),
2. has to wear a shirt,
3. sometimes fakes it and bakes Pillsbury cinnamon rolls.

Please make a suggestion below if you are an Aunt Pythia fan. If you hate Aunt Pythia then I don’t really care what you think.

Readers! Dear readers! Aunt Pythia lovers everywhere should stop what they’re doing and watch this trailer immediately, all about the female orgasm:

It looks adorable, n’est-ce pas? Aunt Pythia planning to watch it in its entirety very soon. Stand by for a review.

But enough dilly dallying, readers, Aunt Pythia has a serious job to attend to! If you enjoy her intemperate, unreasonable ramblings, then before you go,

ask Aunt Pythia any question at all at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.

——

Dear Aunt Pythia,

We had two interactions recently with the police and my wife disagrees with me if this is an example of “driving while white”, so I wanted your opinion of how this would have gone if I was African-American.

1) My 18-year old son was driving our Tesla on the PA Turnpike and was pulled over by the police. The officer walked over to the car and complemented him “Nice car.” And walked away without giving any other reason for stopping him. Do people actually get stopped to get compliments?

2) I was driving through Baltimore and looking for a rest stop. I took an exit and realizing that it was simply leading to another highway, I pulled off onto the very ample shoulder and then walked into the woods to pee. When I turned to return, I saw two Baltimore police officers with their hands on their holsters. They asked me what I was doing in the woods. When I told them they gave me a warning, saying I should “keep an empty bottle in the car for emergencies” and let me go on my way. Did I do something wrong? What did they think I was doing?

Why He Is Traffic-Stop Exempt

Dear WHITE,

Well, I’m also white, so I don’t have first hand knowledge of the counterfactual, but I’m happy to think about it with you.

I don’t know why anyone would be stopped for no reason at all, so let’s think about why your son was pulled over. I’ll wager it was because he was too obviously young to afford a Tesla, and the cop was wondering if he had stolen it. If that is the case, then the fact that your son didn’t appear overly nervous once he was stopped could have contributed to his not getting harassed further. On the other hand, imagine how it might have gone another way for the son of a black Tesla owner. The very act of being pulled over and confronted by a cop with suspicions might have freaked him out, not because he was guilty, but because he was aware of how cops treat young black men.

Similarly, why did those guys care about you peeing in the woods? I don’t know, but their hands on their holsters tells us they were ready for something violent. Again, the counterfactual is always missing, but then again unless you’ve been living under a rock it’s hard to rule out something ridiculous happening in both cases.

In summary, I can easily imagine how this could have gone differently for both you and your son had you been black, but I’m not sure that means there is something truly out of the ordinary going on in either case. That’s the thing with discrimination, it’s statistical rather than deterministic.

To answer your original question, I’d say that instead of thinking about specific events being white privilege events, think overall about how often your interactions with power are pleasant or unpleasant; after all, you have more than two data points. Personally I have gotten out of illegal driving maneuvers by crying, having kids in the car, and even once just because I had kid car seats in my car. I am most definitely a white woman and it has worked for me. Of course, those police might just have been nice to everyone; again, it’s statistics, and I can’t prove my white privilege, but I don’t doubt it one bit either.

Every now and then the situation is more clear cut. I was walking up Broadway the other day and I walked by a scene where a white man was getting a parking ticket from an black female cop. He was very upset about the ticket, and was swearing at the top of his lungs, stuff like, “This is complete shit! You’re an asshole! Fuck this ticket! I was only gone for a few minutes!” The cop said absolutely nothing while she wrote the ticket. She remained calm, and while I didn’t stay until the end, I presume she simply handed the guy his ticket at the end. I kicked myself afterwards for not whipping out my phone and recording it, but I do think it was absolutely inconceivable that if the roles had been reversed, and it had been a white male cop giving a ticket to a black women, I would have seen her acting in the same way.

Aunt Pythia

——

Hi Aunt Pythia,

A while ago, someone sent you a music video for “Dangerous” and you didn’t much like it. I just ran across this alternative video for the same song, and I think it meshes with your aesthetic much better…

But It’s Good! Don’t Argue, Try Again

Dear BIGDATA,

Very clever, I like it!

Aunt Pythia

——

Aunt Pythia,

Bad answer to the anal sex guy. I happen to like it myself, but seriously how many times does his girlfriend have to say “no” to having his dick in her ass before he listens? Your answer should have been more like “you first with a dildo.” Their tune changes very quickly when it’s their asshole being probed.

Answer

Dear Answer,

First of all, he might have been fine with a dildo up his butt; some people are just kinkier than other people. Second of all, I don’t think it’s great marital advice to tell your spouse to “shut up about their sexuality already.” That might work fine at work, or with an acquaintance, but with a spouse you’re really in it for the long haul and things could go badly if that’s your attitude. I stand by my answer.

Aunt Pythia

——

Hi Aunt Pythia,

Do you have any friends who just suck your positive energy away? I do. This friend always has something to complain about her life, with so much drama. She is a really dear friend too, so I feel quite strong sadness with her every problem in life. But when someone is just dumping their problems on you (I know she is more like sharing them, but still), how sustainable that relationship is? It is horrible to abandon people when they need you, but is there a good time/reason/way to restructure your friendship with someone?

Friendship Rest In ENjoyable Distance

Dear FRIEND,

I feel your pain. Or actually, no I don’t. But I do acknowledge your pain.

And therein lies the difference, and my advice. Make time for your friend to tell you about her problems, but don’t make them your problems. Listen to her, have compassion, smile and give her love and support, but don’t get yourself empathetic to the point of suffering yourself, because that’s not helpful to either of you. That’s the first piece of advice I have for you.

Second: indicate to your friend that your conversations are somewhat lopsided, and you’d like to discuss happier things as well as struggles, because that way it is more fun and forward thinking. Tell her three things you are grateful for from the past week, and make it part of your relationship that you talk about more than her problems.

Another possibility, if that seems unlikely or too difficult to handle: make your meetings about something, rather than straight up bitch sessions or coffee. Go to Magic Mike XXL with her, or take a walk, or take her to the yarn store, or get her engaged in a project of some kind that she would like. Make the meetings about that project and the future.

If all else fails – and do try a couple of the above first – give yourself a break and tell her you are busy when it just seems like too much work to listen to her. Wean her off of you gradually, and she will find other people to talk to.

Good luck!

Aunt Pythia

p.s. One piece of advice I didn’t give you is to give her advice on her problems. I do this by nature, so I assume you’ve already tried it, but the truth is I don’t think it helps very much. I mean, it helps me think I’m being a good friend, of course, but my experience is that people who talk a lot about their problems don’t want advice to fix them. Of course, some people do ask advice and take it, but they are typically people who talk only infrequently about their problems. And also, giving advice extends the time that you spend talking about her problems instead of something else.

——

Readers? Aunt Pythia loves you so much. She wants to hear from you, and then tell you what for in a most indulgent way. Will you help her do that?

Please, pleeeeease ask her a question. She will take it seriously and answer it if she can.

Click here for a form for later or just do it now:

Have you been reading about the shortage of workers in the trucking industry? Have you wondered why, in this crappy economy, they haven’t been able to find more workers? Here’s an excerpt from recent Wall Street Journal’s coverage of this worker shortage crisis:

Operators across the country are short 30,000 long-distance drivers, the American Trucking Associations estimates. The group projects the shortage could top 200,000 in the next decade. Average annual pay for long-distance drivers was $49,540 in 2013, according to ATA estimates. Hiring and wages in truck transportation have inched up this year, according to the Labor Department.

I’ve got a theory. Here’s what it is: they trucking companies aren’t paying enough. Funny how demand and supply and efficient markets go out the window when there’s a political point being served, though: Congress is considering passing a law that would allow 18-year-olds to be long-haul truckers. A terrible idea considering how younger drivers are much more dangerous.

Of course, $50K isn’t nothing. But on the other hand, truckers have to be trained, competent, and regularly spend many days on the road. Moreover, the current surveillance technology has severely degraded their quality of life, which I learned by reading about Karen Levy’s work on the industry. Also, new truckers probably make substantially less than $50K when they start.

Partly the surveillance arose from the very real risk of truckers driving too much per day – it was an attempt to make sure truckers were driving safely. But since the technology has been installed in many large-company fleets, the companies have used it to essentially harass their drivers, telling them when break is over and so on. This has worked, in the sense that larger companies with more surveillance have managed to lower costs, pushing out smaller and individual truckers. And that means that truckers who used to own their own business now reluctantly work for huge companies.

For an industry that has historically prided itself for its independent nature, this change does not sit well with drivers. The turnover rates are staggering:

When you make your workers lives worse, and you don’t compensate them with cash money to make up for it, you find your workers quitting. That’s what’s happening here.

Conclusion: we either need to improve truckers’ work experiences or pay them more. There’s no worker shortage, there’s simply an unwillingness, on the employers’ side, to face up to the facts.

There’s a provocative article over at Medium.com about women in tech. As the article points out in about a thousand ways, it’s not just a pipeline problem, it’s an environmental problem.

Fellow math nerd Rachel Thomas, the author, points out a bunch of sad facts about working in tech. For example, how VC’s prefer men, how men’s applications are preferred in hiring processes, how women get punished for negotiating and for being pushy whereas men get rewarded.

Having worked in tech myself, I can say the maternity policies are crap, the long hours are unreasonable, and the frat-like atmosphere exhausts me. No, I do not want to play ping pong during my lunch hour.

But having said that, I don’t think I’ve experienced the worst of it; I was already a grownup, with a Ph.D., when I entered this stuff, and as such I’m allowed to have stronger opinions than the average engineer.

The most interesting issue brought up in Rachel’s piece is the retention rates for those qualified for tech jobs. Unfortunately, both Rachel’s piece and this related NPR piece which Rachel points to only discuss the statistics for women retention, namely that about 40% of women leave engineering after they get degrees in engineering (and I think Rachel’s piece actually gets that stat wrong).

Presumably, that’s higher than men, but how much higher? And do women leave jobs more often in general, or is this a tech-related retention problem? What’s the breakdown on reasons why women and men leave? Can we address them individually?

These are important questions, and if we can figure out what is happening, we should. I’ve been thinking about how to grow the pipeline for girls and women in STEM subjects at the high school and college level, but it would be ridiculous to spend an enormous amount of time on that if, once the get a job, that job proves unattractive.

Update: In the subtitle of the piece, it says 17% of men end up leaving the field compared to 42% of women, with a link to this 100 page pdf (hat tip Ewout ter Haar). I still want to know how many women leave other fields to give more context, but it’s a good start.

Papers have mistakes all the time. If they’re smallish mistakes that don’t threaten the main work, often times the author is told to write an erratum, which the academic journal publishes in a subsequent volume. Other times the problems are more substantial, and might deserve the paper to be retracted altogether.

For example, if a paper is found to have fraudulent data, retraction is called for. Even when the claims made are outlandish, implausible, and unreproducible, but the authors hadn’t been intentionally fraudulent, there still may be just cause to seriously question their claims and retract. On the other hand, if a paper that was once deemed cutting edge and new is, in retrospect, not very innovative at all, then typically no retraction is called for; the paper is simply ignored. When exactly retraction happens, and how, probably depends on the journal, and even the editor.

Today I want to tell you a story in which that process seems to have gone badly wrong.

Elsevier, the academic publishing giant owns a journal called the Journal of Computational Science (JoCS) which published a paper called Twitter Mood Predicts the Stock Market (preprint version here) back in 2010. It got a lot of press, and even more, and according to Google Scholar has been cited 1300 times. According to media reports, the paper showed that Twitter, when it was enhanced with emotional tags, was able to predict the Dow Jones Industrial Average with an accuracy of 87% (whatever that means).

Full disclosure: I haven’t read the paper, but even so I don’t believe the results of this paper. People in hedge funds have been trolling for signal in all sorts of news and social media text-based ways for a long while, and there’s simply no way that they would have ignored such a strong signal all the way into 2008. If it was real, they wouldn’t have ignored it, and it would have faded. But I also don’t think it’s so real either.

Anyway, that’s my personal intuition about this, but I could be wrong! That’s what’s cool about academic publishing, right? That we could just be super wrong and people can say what they think and then we get to have this open conversation?

Well, sometimes. What actually happened here is that a bunch of people tried to replicate these results, which was harder because suddenly Twitter started charging lots of money for their data, and a hedge fund also tried the Twitter strategy that was similar to the one outlined in the paper, but everyone lost money*.

After a while, one of these frustrated would-be traders, who we will call LW, decides to write a letter to the editor complaining about the original paper. He even blogged about his letter here. In his letter he had two complaints. First, that the results were consistent with datamining, which is to say that there’s statistical evidence the authors cherry picked their data. Second, that if the results were true, they would violate the “Efficient Market Hypothesis,” and would surprise a bunch of traders with many decades of experience.

So far, so good. A paper is published, people are complaining that the results are wrong or extremely implausible. This is what academic publishing is for.

Here’s what happens next. The editor sends out the letter to reviewers. Two out of 3 of the reviewers respond, and I’ve got a copy their responses. The first reviewer is enthusiastic about doing something – although whether that means retracting the Twitter paper or publishing the complaint letter in the “Letter To The Editor” section is not clear – and uses the phrase “The original paper’s performance claims are convincingly shown to be severely exaggerated.” That first reviewer has minor requests for modifications.

The second reviewer is less enthusiastic but still thinks there is merit to the complaint letter. The second reviewer is dubious as to whether the original article should be withdrawn, but is clearly also skeptical of the stated claims. Finally, the second reviewer suggests that the original authors should be given a chance to respond before their article is retracted.

At this point, the editor writes to the complaint letter writer LW and says, you need to modify your letter, at which time I’ll “reconsider my decision.” The editor doesn’t say whether that decision is to retract the paper or to publish the letter.

So far, still so good. But here’s where things get very weird. After modifying the letter, LW sends it back to the editor, who soon comes back with another review, and importantly, a decision not to take further action. Here are some important facts:

1. The new review is scathing, passionate, and very long. Look at it here.
2. The new review has a name on it – possibly left there by accident – it’s the author of the original paper!
3. Perhaps this was intentional? Did the editor want to give the original author a chance to defend his work?
4. In the editor’s letter, he states “Reviewers’ comments on your work have now been received.  You will see that they are advising against publication of your work.  Therefore I must reject it.”
5. The way that was phrased, it doesn’t sound like the editor was acknowledging that this was not an unbiased reviewer, but was in fact one of the original authors.
6. In any case, before the final reviewer weighed in, it looked like the reviewers had been suggesting publication of the letter at the very least, possibly with the chance for another reaction letter from the author. So this author’s review seems to have been the deciding vote.
7. You can read more about the details here, on the complaining letter writer’s blog.

What are the standards for this kind of thing? I’m not sure, but I’m pretty certain that asking the original author to be the deciding vote on whether a paper gets retracted isn’t – or should not be – standard practice.

To be clear, I think it makes sense to allow the author to respond to the complaints, but not at this point in the process. Instead, the decision of whether to publish the letter should have been made, with the help of outside reviewers, and if it was decided to publish the letter, the original author should have been given a chance to compose a rebuttal to be published side by side with the complaint.

Also to be clear, I’m not incredibly sympathetic with someone trying to make money off of a published algorithm and then getting pissed when they lose money instead. I’m willing to admit that more than one of these parties is biased. But I do think that the process over at Elsevier’s Journal of Computational Sciences needs auditing.

* Or at least the ones that are talking. Maybe other traders are raking it in but aren’t talking?

Dear readers,

Do you ever wake up not knowing what you want to do when you grow up? And then you realize you’re far too old to feel that way? Well, that’s the way Aunt Pythia feels this morning. She’s in no position to give anyone advice.

And yet. And yet, it’s fun to give people advice! So here goes. Afterwards she’s planning to whip up a batch of delicious “Identity Crisis Crepes” to cheer herself up a bit. They’re going to look like this:

Identity Crisis Crepes have extra nutella.

Are you addicted to carbs like Aunt Pythia? Do you wish to demonstrate solidarity to the cause? If so, before you go,

ask Aunt Pythia any question at all at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.

——

Dear Aunt Pythia,

A bit lazy to sugarcoat this. I’ve noticed that your personal history/internal biases come out very strongly on some topics and result in irrational/illogical conclusions/actions including banning challenges to your logic.

Have you noticed this yourself? Do you care? If you do care – how would you (do you try to?) address this issue (which I assume every single person suffers from)?

Curious About Rational Exchange

Dear CARE,

Why, no, I hadn’t noticed! Isn’t that why they’re called internal biases? If you’d like to point out specific examples, we can discuss further.

Come to think of it, there are certain things I’m happily opinionated and even stubborn about, but that’s what it means to have a personality, isn’t it? And isn’t that why people ask an advice columnist her opinion? Because the other person is bound to have an opinion?

Of course, one is free to ignore someone else’s opinion, even if it comes from a blogger. But I wouldn’t advise it (har har)!

Aunt Pythia

——

Dear Aunt Pythia:

My work history includes math, data, operations and government analyst jobs, and direct service work. I love collaborating and sharing ideas but I find myself frustrated at a lot of the shitty attitudes that are moving into this space now that its “hot”. I loved your four political camps of big data post and felt like it was one of the few things I’ve come across that addressed this thing I am trying to get my head around.

My problems are twofold:

(1) Not punching someone in the face when they tell me they want to “hack poverty” or any number of other things that speak to a critical lack of familiarity with the context of public interest or work for social good.

(2) Feeling left behind in the job race and shut out of the bigger conversation. I’ve been doing solid research and policy work on issues I care about for quite some time and I hate the idea that the even the president (given his community organizing background) is touting corporate tech as the place to find talent to help build data capacity for the govermment. How do I get my invite to the big kids table?

For now my plan is to keep on keeping on putting data to use in communities I care about and helping community based organizations build capacity around data use and service delivery but I need some help planning ahead.

Yours Truly,

BITCHY?

p.s. Sorry there’s not a sex piece to this!

Dear BITCHY,

I feel you! How about you email me (address available on my “About page” and tell me what you’re working on, why it’s important, and then we can scheme on how to get more publicity for you. I agree that there is far too much absolute bullshit out there, and I’d like to help by promoting substantial work.

Also, one thing about the four political camps. There should have been five, I left out the academic camp which consists of people who genuinely want to make progress on stuff like medicine research and are constantly frustrated by HIPAA laws that protect privacy. Take a look at Daniel Barth-Jones’s work for a great example of this perspective.

Love,

Aunt Pythia

p.s. Nobody’s perfect!

——

Dear Aunt Pythia,

What the…

Google Chrome Listening In To Your Room Shows The Importance Of Privacy Defense In Depth

…is this for real?

Overheard, In California

Dear Overheard,

Well, I know it’s for real, because my son showed me how to use the “OK, Google” feature on his MacBook. But in order to use it, you have to activate it in your Google Chrome settings (or at least that’s what they claim!).

As for how creepy this is, it really depends on how you think about it. I mean, Siri listens too, right? Is that creepy? I think it depends on how much we trust Google and Apple. And the answer is: a fuck ton. We let Google read all our emails already, don’t forget.

As far as I know, voice transcribing still doesn’t work very well compared to actually have the text of email. So I guess if I had to list the creepy stuff in order, I’d start with gmail.

Aunt Pythia

——

Hi Aunt Pythia!

Here is my probably oh-so-familiar story. I’m a grad student in pure math, looking to get out and interested in data journalism. I’ve looked through your notes from the Lede program, and think that working at ProPublica would be AMAZING (though likely a pipe dream).

Beyond material at the level of AP exams, I have no experience in statistics, programming, nor journalism. However, I think reporting stories stemming from statistical analyses or making interactive news applications for readers to explore data themselves would be really cool. For someone in my position with these goals, would you make some suggestions for skills to pick up, people to talk to or emulate, workshops or informational events to attend?

(Addendum/Clarification to the question: Searching the web for “data journalism” and its variants, I find programs and resources for journalists to bulk up their data-science skills or calls for programmers to get involved with news agencies. However, what concrete suggestions would you give to someone starting from scratch who wants to break into this field? I am somewhat more interested in analyzing and interpreting data than in making graphics.)

Thank you in advance!

News Enformer Wanna Be

p.s. You should check out Amanda Cox’s work and talks if you haven’t!

Dear NEW B,

I happen to have some good news for you. Scott Klein of ProPublica came to the Lede Program and told us he hires people based on their webpage projects. If they are cool, innovative, and newsworthy, then he is interested. This is somewhat different from other editors who depend on your ability to get your work published by mainstream news outlets.

So in other words, I suggest you create an online portfolio of work that you think is super interesting and newsworthy, and then you start applying for jobs. To do this, you’ll need to learn statistics and computer programming, but I’d suggest starting with the project and then picking up skills you need to do it. Steal ideas from various online syllabi and such, and feel free to enroll in an actual program or do self-study. Go to hackathons and learn quick and dirty skills.

It’s a long-term plan (or at least not a short-term one), and you might not get a job at ProPublica, which I agree is a dreamy kind of dream, but you might well get another great job, and in any case you’ll learn a lot. Also definitely collaborate with journalists starting now – many great freelance journalists already have great stories and would love to work with mathy/ computer people. Go to a local journalism school and introduce yourself.

Aunt Pythia

p.s. Amanda Cox kicks ass!

——

People, people! Aunt Pythia loves you so much. And she knows that you love her. She feels the love. She really really does.

Well, here’s your chance to spread your Aunt Pythia love to the world! Please ask her a question. She will take it seriously and answer it if she can.

Click here for a form or just do it now:

I spent the morning watching this one hour lecture by David Kung, who has been studying the gender and racial achievement gaps in mathematics. Interesting stuff, with historical perspective – math has a sad history – and a call for the end to passive lecturing and much more:

Watch it if you have time. You can skip to 7:20 to start.

Are you fascinated by the “debt as moral weight” arguments you see being tossed around and viciously debated over in Germany and Greece nowadays? It seems like the moral debate has superseded the economic reality of the situation. Even the IMF has declared the current Greek deal untenable, but that hasn’t seemed to interfere with the actual negotiations.

What gives? Many point to history to explain this. Besides the whole Nazi thing, or maybe exactly because of it, the Greeks keep reminding the Germans that they (and others) forgave half of existing German debt after World War II, with the1953 London Debt Agreement. The Germans have responded vehemently that such ancient history is irrelevant, and that the Greeks are a bunch of lazy olive-eating tax avoiders. It’s a dirty fight, and getting dirtier every week.

I maintain we don’t have to examine the history of 60 years ago to understand at least some of the moral anxiety. Instead we should look a mere 7 years ago, at the enormous German bailout of their own banks, which had invested quite recklessly in all sorts of the most risky financial instruments and, most relevantly, Greek bonds.

Start with the basic facts. German and French banks invested very heavily in Greek bonds, partly because they were allowed by European Basel “risk regulation” laws to set the risk of those Greek bonds at zero, and partly because they were just investing in anything and everything with a relatively high yield. Since Greek bonds were at a higher yield than other government bonds that maybe deserved the “zero risk” designation more, they naturally bought an asston of those.

[Side note: whenever there’s a market with a spectrum of products, the ones with the biggest yield for a given risk profile will be snatched up the fastest, because people want to maximize profits. We’ve seen that this almost always is a bad thing and creates bubbles very quickly. But it’s also the reason people are constantly inventing new products that hide risk. In this case they didn’t need to “invent” anything, because it was a political decision to designate Greek bonds at zero risk.]

There are two ways to look at this story from a morality standpoint. One is that, no matter who owns this debt now, the Greek government is on the hook for borrowing it and needs to figure out how to pay it back. From this point of view it was a mistake of the Greeks to issue too much debt and to spend it unwisely, while not cracking down on tax avoiders.

The other way to look at it is that, German banks should have known better to buy this debt in the first place. After all, it’s a free market, and nobody forces you to buy things, and after all if there really were no risk at all on it there would also be no yield (beyond inflation). But the very reason Greek bonds had yield was because the market was differentiating it from German bonds. From this point of view it was a mistake of the German bankers.

Either way, when the Germans bailed out their banks, they took what was a bank problem and made it into a taxpayer problem.

Have I oversimplified? I’ll also admit that, after that whole bailout went down, a series of “Greek bailouts,” all of which were clearly insufficient, made the European governments even more involved, and the Greeks owed way more on paper to the European taxpayers, which layered on the debts while destroying the Greek economy. But most of those bailouts were simply loans which were used to pay back the original loans. Put another way, the Greeks might not have needed bailing out if the original Greek bonds had been refused by risk-averse bankers in the first place.

This is not to suggest that there was perfect planning going on by the previous Greek governments. But I do think that, if we’re looking for who deserves blame in this story, we might want to circle back to the German bankers who couldn’t resist subprime mortgages and Greek bonds back in the early 2000’s.

Last Friday I visited my high school math camp, HCSSiM, where I became a nerd. I also taught there multiple times over the years, and in 2012 I blogged my lectures.

Why the visit? You see, we loyal alums of HCSSiM have a tradition of going back every July 17th to celebrate “Yellow Pig day,” which consists of a talk where founder and director (David) Kelly talks extensively about fun facts regarding the number 17, which happens before dinner, and then after dinner we sing “yellow pig carols” and eat an enormous amount of cake in the shape of a yellow pig. You can learn more about this ridiculous and hilarious tradition here.

Anyhoo, this year we (I went with other nerds) missed the 17 talk because of traffic in Connecticut but we made it for the dinner and carols. Luckily at dinner I had the chance to talk to Kelly, and I asked him if there were any new 17 facts this year. He told me there was one, and it was slightly mysterious. This post is an attempt to explain it a bit.

The mathematical set-up is explained here. Namely, we start with something called the Ulam Spiral, which is simply a way to label the boxes of an infinite two-dimensional grid with the natural numbers. You start at some place and then spiral outwards from there. Here’s a picture:

The center of the Ulam Spiral

OK, so the first thing to say is that, when you label the plane like this, primes tend to cluster along lines. I think this is what Ulam thought was cool about his spiral:

Primes are black. This is the spiral with 200 layers.

Now comes the observation. You need to know what a triangular number is first, though. Namely, it’s a number that corresponds to counting up how many dots you need to form a triangle. We say the nth triangular number corresponds to a triangle with n rows. Here are the first few:

You can also draw these triangles so that consecutive ones fit together to form squares.

When you highlight the triangular numbers in the Ulam Spiral, instead of the primes, then you get something that looks weird:

Green dots are triangular numbers within the Ulam Spiral.

OK so if you count those spiral arms, you’ll see there are 17 of them. But does that last forever? And if so, why?

Well, the answer is going to be yes. And here’s a rough proof. Rough because it uses asymptotic limits, so technically I will not show that the above picture extends perfectly, but rather that it eventually does look like a spiral with 17 arms.

A famous story about Gauss tells us that the formula for the nth triangular number is

Also, by construction of the Ulam Spiral, the bottom right corner of each “spiral layer” is an odd square, and that if we call that number there will be boxes on the very next layer, corresponding to the 4 sides of the next layer plus the 4 corners of the next layer.

Now imagine that there’s a triangular number right on that bottom right corner. That would mean that for some

or in other words that

This is when things get asymptotic. Imagine that is very very large. That would mean that is too (everything here is a positive integer), and in particular that the term would dwarf the term above. In other words, we could approximate:

My next question is, how many triangular numbers would lie on the next layer of the spiral? Well, as we said above there are spots in the next layer, which we will approximate by  and the triangular number coming after is which is bigger than corresponding to adding one layer to a triangle with rows. We will approximate  by  again ignoring small terms.

For that matter, the next few triangular numbers after come regularly, about spots after the first. Therefore there are about triangular numbers in the next row of the Ulam spiral. That comes out to which is about 2.83.

So far we’ve figured out that, when is huge, then after meeting the th triangular number on the th row, we will see two more, and get most of the way to a third, by going one more row.

Now let’s do that 6 more times. After traveling 6 rows past a triangular number, we will meet about more triangular numbers. But

which is very close to 17. So after traveling the Ulam Spiral for 6 rows, we will just about hit 17 triangular numbers, which will be more or less evenly spaced from each other.

What this means is that we should expect to see a spiral with 17 arms, but that when the picture is enlarged to include a very large number of rows, we will see the spiral shifting very slightly to the other direction.

By the way, I didn’t figure this out immediately. First I had a most delightful time understanding when, exactly, square numbers and triangular numbers coincide. In other words, I wanted to understand when there is a and an so that:

or

I might write this up in another post, but play around with it for a while if you get bored on the subway.

The Occupy Summer School students are organizing a clever demonstration tomorrow morning to protest racial and gender inequality. From the men/women pay gap to how the police arrest black and brown people for minor nuisance crimes, the girls from the UAI have figured out thoughtful ways of raising consciousness while having fun.

The plan is to have two tables. At the first table, there will be a “bake sale” where cupcakes will be “sold” for $1 to men but 77 cents to women, to protest unequal pay. They will be handed over using a plate which details many other kinds of gender inequalities. In actuality, anybody who shows up to the protest can get a free cupcake.

At the other table, we’ll be handing out brownies with toothpick flags, which are toothpicks with facts about racial inequalities taped to them. For example one might read, “While people of color make up about ​30 percent​of the United States’ population, they account for ​60 percent​of those imprisoned.”

Everyone loves free food, of course, but given that black women like Sandra Bland get killed in this country for minor traffic infractions, there’s a deeply serious side to it as well.

If you have time, please join us. The event will take place on Cadman Plaza near Tillary, in downtown Brooklyn, tomorrow morning from 9-11am. The girls will appreciate your visit.

When I was a young idealistic mother, pregnant with my first kid, I had this crazy idea that I’d dress my kids in gender neutral clothing, like they have on Star Trek. In fact my actual goal was to get them Star Trek uniforms, but I knew that might be slightly difficult. It was 1999 and we were all worried about Y2K.

Little did I realize, until after the kiddo was born, how difficult it would be to get anything remotely gender neutral. Especially because I was rarely willing to spend lots of money on clothes I knew would be immediately outgrown, I ended up shopping at places like Toys R Us and similar, and man oh man are those clothes gendered. There’s a pink section and a royal blue and red section. Nothing in between, and no overlap.

Well, things have changed in the past 16 years, and nowadays there are clothing companies deliberately creating kids clothing that doesn’t have the awful princess/superhero dichotomy embedded into every garment. According to this Bloomberg article, there are now pink and purple clothes for boys and dinosaur, pirate, and science clothes for girls. Svaha, for example, sets itself up as a place that makes “clothes that empower your children.” Here’s an example of a girls’ shirt:

There’s also a boys’ shirt, also pink, with flowers and test tubes. That would have been great for my first son, whose favorite color was, as he described it at the time, “light red.”

A couple of things. First, these shirts are $25. That’s approximately 4 times more than these shirts that are standard issue “boy” clothes. Partly that’s just because it’s not a concept that’s really taken off, so we don’t have huge factories in Bangladesh churning out these shirts at ridiculous rates. But even so, it means that, like organic food, open-ended gender categorical clothing is firmly within the realm of the well-off parent.

Second, I don’t think it’s all that reasonable to say a shirt “empowers” a kid. Most times, when a kid is defined externally, through a shirt or a social convention or an adult’s comment, or even another kid’s comment, it’s an exercise in limiting that kid, not expanding him or her. Kids assume they can do anything until we tell them otherwise. When you say to a young girl, “You can be a scientist too, you know!” she thinks, “I never thought I couldn’t. Wait, why should I think I couldn’t?”. It’s not until they’re teenagers that they get this stuff, and can have a critical mindset about it.

In other words, I’m going back to Star Trek uniforms for everyone. The great thing about them is how utterly vapid they are of style or message. If you had to pin a message on to them, it would be an awesome (but distant) future.

Yesterday, as I was accompanying Adam Reich to the Occupy Summer School on the downtown 2 train, he pointed over my shoulder at someone reading a Metro, because the girls were on the front page:

We also were on the second page:

After I got to the UAI, the high school where we run Occupy Summer School, I found the online version of the Metro story as well, which is also exciting.

Since Occupy Summer School (OSS) is half over, I think it’s a good time to update you on what’s been happening.

• Last Monday we introduced ourselves, met the students, talked a little bit about Occupy, agreeable disagreement, and had a discussion about what they wanted to focus on using “stack.” Among the issues they came up with: inequality, Black Lives Matter, taxing the rich, how teenagers are unfairly targeted, and gender issues.
• Tuesday Ale and Mo, a high school activist, came and talked to the girls about activism. We discussed how organizing actually works, what were the props for events, like stickers, flyers, signs and banners, how to get the word out among networks via text or twitter or other social media, and so on. We ended the day by quickly planning a protest against overly lengthy standardized tests.
• Wednesday Tamir’s friends from Local 79 came and talked about unions and union organizing. The girls didn’t know much about unions, and were interested to learn how power can be created through numbers. 
• Thursday Gerald provoked a fantastic discussion on #BlackLivesMatter and related topics. This was the first time where the girls really took over the discussion and the grownups in the room were merely listening and every now and then joining the discussion.
• Friday Marni dazzled the girls with her approach to creative protests. She brought her own entourage, which ended up being how we got into the Metro. The day ended by planning a bake sale where women would be charged 78 cents and men one dollar for the same brownies, to illustrate the difference in pay.
• This Monday I spoke to the girls about “why high school is free but college is expensive,” and then about debt more generally. We ended the day by planning a protest around a $100 ticket one of the girls had gotten for “doubling up” in the subway with her cousin, who didn’t have a student Metro card. The demand was to be unlimited metro rides for high school students on all days, not just school days.
• Yesterday Adam came and talked to them about sociology, what is power (power is the opposite of dependence), and his work helping orgainze workers at Walmart. By the end of it he had two volunteers who wanted to join the cause.

I can’t wait for the rest of OSS! I’ll write another update at the end of next week when it’s over.

As you already know, Puerto Rico is in a debt crisis. It’s unsustainable – take a look at some of the numbers – and people are suffering. There’s very high prices, few jobs, and on top of that there’s a terrible drought as well. I’m trying this week to learn what some of the details are of this situation, which is incredibly complicated because Puerto Rico is a U.S. commonwealth, not a state, and has historically been ignored by the political system.

Here’s what I know. The bond markets for Puerto Rico have historically been attractive to investors because the bonds are “triple exempt,” which basically means no taxes are applicable to them. This made it too easy for Puerto Rico to borrow money and has put it in a hole, very analogous to the Greek situation. And now we have to decide how much the people should suffer for the results of the bond markets.

Yesterday I reblogged a post by Marc Joffe, who argued that the U.S. should extend Chapter 9 to Puerto Rico. Hypothetically this would allow Puerto Rico to declare bankruptcy and restructure its debts in some reasonable way. However, as Kristi Culpepper explained in this Medium piece (hat tip Tom Adams), it actually wouldn’t give Puerto Rico the relief that it needs, first of all because it would redefine Puerto Rico as a “state” but states are not eligible to declare bankruptcy, and secondly because the corporate bonds issued by Puerto Rico’s public corporations have a special status that also prevents them from restructuring.

Culpepper also notes in her piece that people who cry foul at the concept of restructuring debt after it has been issued can rest assured that there is precedent for it. Personally, I don’t even understand that complaint; surely everyone realizes that any debt might go into default, and it hardly matters exactly what that procedure looks like.

Culpepper recommends something else entirely, namely a federal financial control board. The idea is that there’s also precedent for this, in the 1990’s in Washington D.C.. However, it would essentially mean handing over control over its finances to the board. Culpepper notes that this could even happen without consent. I think the Puerto Rican people may have something to say about this. She also suggests we could provide liquidity for Puerto Rico if we wanted, although it might look something like a bailout.

The biggest problem is that, even now, no politician seems to really care about Puerto Rico, except to fight against it becoming a state.

Today I want to tell you about $heriff, an awesome new tool I learned about recently. It’s currently an add-on to the Firefox and IE browsers, but it will be compatible with Chrome soon. I mention this because I think you will want to use it, partly for the good of scientific discovery, but partly for your own good.

Because here’s what $heriff does for you. It allows you to see how prices for goods you’re interested in buying would change depending on where the request is coming from. And sometimes the answer is “a lot,” even on Staples.com or Amazon.com.

I talked to one of the creators, Nikolaos Laoutaris, who works as a computer science researcher in Barcelona. Nikos described how he came across the idea of creating $heriff. Namely, he and a friend were discussing their upcoming vacation to a town in Austria, and they were both looking into booking the same type of room at the same hotel (at the same time, since they were doing it over Skype) and they were seeing very different prices.

The way it actually works is that there’s a way to “check” prices when you come across one, and the results pop up in a separate window. Here’s a screenshot of what happened with Nikos showed me $heriff checking on the price of a fancy camera at digitalrev.com:

The list corresponds to what price showed up when a bunch of servers at academic institutions were asked to send a price request that $heriff extracted from the local webpage. To be clear, your personal request for price, coming from your browser, might depend on location as well as your cookies and referral url, among other things, but these other prices correspond to “clean browsers,” with no browsing history, and no login history, making a request from a specific location. So those price variants correspond to location changes only.

There’s also a space below for “Results from local users,” and this is where you come in. If there’s a person with a $heriff add-on in your local area, the idea would be to use the information your browser collects to allow someone to compare their price with your price. In this way we would (slowly) also learn how prices change depending on browsing history. The more people who use $heriff and donate their data to the project (third party cookies, some browsing history, and prices), the faster we will learn about how prices vary.

Among other things, this could be a way of figuring out how to intentionally set your input data so that you get the best price for a given product.

A few notes:

• Local sales tax and shipping costs are valid reasons for prices to depend on location of the buyer, but they are typically added later, after the prices are shown.
• It’s possible that tariffs come into play, but they don’t seem consistent. Also, the prices vary too much to be accounted for by tariffs.
• Nikolaos Laoutaris and his colleagues have also been doing great work looking into how people are tracked, and how such information is used to steer them into environments of “search discrimination,” which means that which products or offers they are shown changes, rather than the prices themselves.
• Also, he’s part of a cool project called the Data Transparency Lab.

Get started with $heriff or learn more about Nikos’s work, by going here.