Followup: Change academic publishing
I really appreciate the amazing and immediate feedback I got from my post yesterday about changing the system of academic publishing. Let me gather the things I’ve learned or thought about in response:
First, I learned that mathoverflow is competitive and you “do well” on it if you’re quick and clever. Actually I didn’t know this, and since it is online I naively assumed people read it when they had time and so the answers to questions kind of drifted in over time. I kind of hate competitive math, and yes I wouldn’t like that to be the single metric deciding my tenure or job.
Next, ArXiv already existed when I left math, but I don’t think it’s all that good a “solution” either, because it’s treated mostly as a warehouse for papers, and there is not much feedback (although I’ve heard there’s way more in physics). Correct me if I’m wrong here.
I don’t want to sound like a pessimist, because the above two things really do function and add a lot to the community. I’m just pointing out that they aren’t perfect.
We, the mathematics community, should formally set out to be creative and thoughtful about different ways to collaborate and to document collaboration, and to score it for depth as well as helpfulness, etc. Let’s keep inventing stuff until we have a system which is respected and useful. The reason people may not be putting time into this right now is that they won’t be rewarded for it, but I say do it anyway and worry about that later. Let’s start brainstorming about what that system would look like.
That gets to another crucial point, which is that the people we have to convince are really not each other so much as deans and provosts of universities who are super conservative and want to be absolutely sure that the people they award tenure to are contributing citizens and will be for 40 years. We need to convince them to reconsider their definitions of “mathematical contributions”. How are we going to do this?
My first guess is that deans and provosts would listen to “experts in the field” quite a bit. This is good news, because it means that in some sense we just need to wait until the experts in the field come from the generation of people who invented (or at least appreciate) these tools. There are probably other issues though, which I don’t know about. I’d love to get comments from a dean or a provost on this one.
mathbabe 
Better off brainstorming for Milken and getting paid. He’s interested in for-profit education.
LikeLike
It is not clear to me how the ArXiv furthers the goal of tenure or how it encourages communication. True, I see articles in reply to other articles quickly after the original post, but it is not an everyday occurrence. Not many scientists get motivated to submit replies unless they are explicitly poked in some paper. For achieving tenure, the ArXiv probably has no effect.
LikeLike
I have been on many tenure committees and I think you’re just not right about this.
LikeLike
Great! It is good to know, since my perception is that it doesn’t have an effect. At peril of going a little off-topic, how do you value an arxiv paper? When you have a peer-reviewed paper you look at the journal it is published in, the citations, and your own expertise in the particular field. What are the equivalent parameters for the arxiv? Is a paper published on the arxiv more valuable if a similar version has gone through peer review?
LikeLike
I should mention that I am not a mathematician and my field might be significantly different from mathematics. I follow mainly the chem-ph arxiv section.
LikeLike
When somebody has papers on the arXiv that aren’t published, there is typically discussion in the letters of the relevant theorems, and how important and difficult they are. If none of the letter writers see fit to mention those results, the committee would probably conclude (in my opinion reasonably!) that the arXiv papers didn’t constitute a major part of the person’s research program.
LikeLike
I could probably go on forever on this topic. It seems to me that we have to separate different purposes served by refereed academic journals and ask whether there are alternative and maybe even better ways to accomplish them. So here is how I perceive things:
1) The original purpose of journals was to disseminate new knowledge and interpretations of knowledge, mainly to fellow experts in the field. Refereeing was probably necessary because it was expensive to publish an article, so one needed to be selective.
At this point, it appears to me that arxiv, other preprint servers, email, people’s personal web pages, and blogs now serve this purpose far better than any published journal.
2) Refereed journals have also been useful in helping new people enter into a field, because they provide a rough signal on which papers are more important than others.
Here, I see a need that is not being met by the alternatives listed above. What’s valuable about a good refereed journal is the oversight and judgement provided by a capable editor or editorial board backed by careful referee reports. We don’t really need the distribution provided by publishers, but the latter is quite valuable.
3) A person’s publication record in refereed journals is a key factor when someone who is not an expert in that person’s field needs to make a judgement about that person. The most important instance is for tenure.
This is an extremely difficult ongoing challenge faced by departments, deans, and provosts. And it’s one that’s too often handled poorly by all involved. Something really does need to be done here. The judgements implied by the editorial and refereeing process of good academic journals have definitely provided a lot of value here. We really do need to find a way to sustain something like that going forward.
If I decide to provide more thoughts on this, I’ll do it in a separate comment.
LikeLike
The department and the department chair can make the argument for tenure promotion based on arXiv contributions or any other standard. The field/department just needs to pass the resolution and have it approved up the chain along the way. This provides a clear incentive system to be written into tenure contracts with new professors.
Part of the system can be *producing* contributions on arXiv or summaries of the important work in the field (outside of the normal journal sources).
LikeLike
Increasingly in various physics fields, but perhaps most specially in high-energy physics of various types, claims of priority as well as citations stem from the archived preprint and not so much from the published paper. For this reason, the large particle physics databases such as SPIRES track citations from other preprints as well. These citation metrics, even though they may be dominated by other preprints, are increasingly being used for hiring and promotion, at least at my institution and my guess is, elsewhere. In fields closer to mine, such as condensed matter physics, breakthrough areas such as experimental developments in superconductivity, many of them originating now in China, put their articles on the archive first.
While my papers have, typically, always benefited from anonymous peer review, I have heard several horror stories from colleagues who believed that the peer review process had in some cases been manipulated to ensure that papers submitted to journals could be held up while reviewers or their friends could quickly get their research going on the same topic. The archive certainly solves that problem. Also, since all versions (including the oldest, first submitted one) are available, everyone is free to see if any claims made can indeed be supported.
The other problem I see is that journals, and I think this is a problem with all fields, except possibly pure mathematics, are increasingly deluged with submissions, many of them of indifferent quality. Unless our evaluation systems change to value quality over quantity – and I think they are slowly swiveling in that direction – this will continue to be a problem.
LikeLike
Thanks for pointing out how the ArXiv actually improves things over conventional journals. What do people thing about new journals like Plos? Also, people have mentioned other preprint servers. Can someone mention some of these?
LikeLike
LikeLike
It seems that the campaign against Elsevier’s business model is picking up a head of steam in the mathematics community and in the process this is stimulating debate about some of the issues mentioned in this post and its predecessor.
LikeLike
Now even Wired magazine has noticed:
http://www.wired.com/wiredscience/2012/01/testify-the-open-science-movement-catches-fire/
And the biologists now actually have a project setting up a proper peer reviewed open source archive
http://f1000research.com/
LikeLike
The revolution continues
http://michaelnielsen.org/polymath1/index.php?title=Journal_publishing_reform#The_cost_of_knowledge
LikeLike