Archive
Data Without Borders
How freaking cool is this?! I signed up today and wrote to the founder, Jake Porway. He seems fantastic. I’m very excited about his project and how we (meaning you and me, kind reader) can help use our data scientist hats to help NGOs think about what data to collect and how to analyze it once they have it. Please consider signing up!
Data mining contests
So a friend of mine came over last night and he recently became a data scientist in a New York startup too. In fact we have an eery number of things in common, although he only considered working in finance but didn’t actually go through it. It was pretty awesome to see him.
He was also pretty into the idea of this blog and making quantitative techniques more open-source and collaborative. And with that goal in mind he sent me these links:
So what do you guys say? Should we work on something together on this blog that may actually help the world/ make us some prize money? That would be filthy good.
I’m also hoping to get this guy to make a guest post on some quantitative techniques he wants to add to my list. Please comment if you have more suggestions! I will start writing about the list topics very soon.
The basics of quantitative modeling
One exciting goal I have for this blog is to articulate the basic methods of quantitative modeling, followed by, hopefully, collaborative real-time examples of how this craft works out in given examples. Today I just want to outline the techniques, and in later posts I will follow up with a post which goes into more detail on one or more points.
- Data cleaning: bad data (corrupt) vs. outliers (actual data which have unusual values)
- In sample/ out of sample data
- Predictive variables: choosing and preparing which ones and how many
- Exponential down-weighting of “old” data
- Remaining causal: predictive vs. descriptive modeling
- Regressions: linear and multivariate with exponentially down-weighted data
- Bayesian priors and how to implement them
- Open source tools
- When do you have enough data?
- When do you have statistically significant results?
- Visualizing everything
- General philosophy of avoiding fitting your model to the data
For those of you reading this who know a thing or two about being a quant, please do tell me if I’ve missed something.
I can’t wait!
Hello world! [stet]
Welcome to my new “mathbabe” blog! I’d like to outline my aspirations for this blog, at least as I see it now.
First, I want to share my experiences as a female mathematician, for the sake of young women wanting to know what things are like as a professional woman mathematician. Second, I want to share my experiences as an academic mathematician and as a quant in finance, and finally as a data scientist in internet advertising. (Wait, did I say finally?)
I also want to share explicit mathematical and statistical techniques that I’ve learned by doing these jobs. For some reason being a quant is treated like a closed guild, and I object to that, because these are powerful techniques that are not that difficult to learn and use.
Next I want to share thoughts and news on subjects such as mathematics and science education, open-source software packages, and anything else I want, since after all this is a blog.
Finally, I want to use this venue to explore new subjects using the techniques I have under my belt, and hopefully develop new ones. I have a few in mind already and I’m really excited by them, and hopefully with time and feedback from readers some progress can be made. I want to primarily focus on things that will actually help people, or at least have the potential to help people, and which lend themselves to quantitative analysis.
Woohoo!
mathbabe 