Science magazine has an online feature asking readers to send their ideas on how they will be communicating science to readers in the next fifty years. Now, they gave me only 250 words to vent all my frustration, so I have not done so. But, I thought this might be a good platform to introduce my feelings. Of course, a lot of what I feel is already on the blog, so I will only touch upon the essentials here and write about what I wish would be true fifty years from now.
So, to begin with let me state as clearly as possible for me, what I think is wrong with the system we have right now:
Should peer-review reflect the consensus view?
Now this is a contentious issue that needs to be addressed. This issue also lies at the heart of the entire debate. On one hand, crackpot theories like the ‘gyre theory’ should ideally be weeded out. But then again, history is rife with cases of people thinking ahead of their time, most recent example being that of pseudo-crystals. So, a consensus approach is perhaps not a good idea in most cases, but on average it would most likely be true.
Are anonymous reviews really necessary for objectivity?
Anonymous reviews are the worst form of review. All of us care a lot about our name, and if our name does not appear on something, it makes most of us irresponsible. In the initial days, when most science communication happened within a society, it was different. Consider a weak paper written by a famous author. If anonymous review is not instituted, then obviously many such weak papers would appear in the journals, since the reviewers would be afraid to objectively criticize the paper. This was fine when there were a handful of scientists in any country. But these days with millions of scientists with fairly good and standardized training, it does not make a lot of sense. If anything, I would argue that anonymous review makes the system weak. In this regard, I like EMBO J, which makes the reviewer comments available after the paper is published. Unfortunately, this is not a big help, since we don’t know what the reasons for rejecting many papers were. If it was sent out for peer-review, then the review must be made public. This will help maintain accountability. I am reminded of a blog I read somewhere, where a neuro-scientist talks about giving a lecture on his recently published work in a conference. The Editor of Neuron came up to him and said “this is remarkable, why didn’t you send it to our journal”? And the author replied, “we did, and it was an editorial rejection”. So, a lot of good work is being rejected and a lot of bad work gets published all because of inadequate anonymous reviews.
Should publishers decide which papers to publish?
According to one of the Nature editors (I forget which one), the whole scientific publishing industry has always perplexed him. Consider the fact that scientists get the money to do the work, they actually go about doing the work, they write the results nicely, and then send it to journals for publication. The journals send it back to scientists for review and then publish it, which the scientists again pay a lot of money to even read. If this is not a scam, I don’t know what can be! The problem is, journals have prestige and they exploit it to their advantage. So, that is one reason why PLoS ONE will never overtake Science or Nature. The whole open-source thing is good, but ultimately it boils down to the prestige of doing something. Make it too easy like in PLoS ONE, and nobody cares for it. Make it too difficult and people will come in hordes to fail at it. Unfortunately, if the difficulty was just in doing a thorough job and publishing, everyone would be fine with the system. But most of these ‘prestige’ journals are looking for the ‘novelty’ factor. Now, if you ask any self-respecting scientist to predict what their field is going to look like in five years’ time, they will laugh at you. It is almost impossible to predict how things are going to be in five years’ time in any field of science. But, the ‘prestige’ journals are trying to do just that. They are deciding which papers would be ‘novel’ and ‘important’ for later generations and publishing only those papers. More often than not, it falls flat on its face. Consider the fact that close to 60% of the papers in Nature/Science are never cited. So, why do we let the publishing houses dictate terms to us? Isn’t it time to end the relevance of these publishing houses?
So, I guess I have set out what the problems really are in today’s science publishing industry. To summarize the above points:
- Peer-review should not be limited to one or two reviewers.
- Papers should be author-centric and not publisher-centric.
- Anonymous review is OK only if done on a massive scale. In most cases it is irrelevant and unnecessary.
- Money making should not be involved in the process of science. So publishing houses must be eliminated entirely.
- A rating scheme must be made available to satisfy ‘prestige’ of scientists. But the rating should not be arbitrarily decided by a few editors, but a consensus view must be taken.
Peer-review of papers should ideally be like drug trials. In a phased manner, with the threat of pulling the plug on it looming on its head eternally. And that is how science should be done. As I said in an earlier blog post "Science is an asymptote to the truth". So, why should we care for stamps of approvals which will in any case be negated to some degree later on ? For instance, does it really matter to us what the debates were, regarding classical approaches to atomic behavior in 1800s ?
On that note, let me propose the "Certificate system of self-publishing". It is simple, elegant and nobody needs to pay anything. Every department has a web-server these days, and gigabytes of data is very cheaply available. So, we should make it mandatory that all labs must make their work available on their web-page. Once you make the data available on your web-page, you can ask the regulatory authority (comprised of volunteers in the same field) to authenticate the presentation of the work. If it is not plagiarized, then a certificate will be issued and an encrypted MD5/SHA sum of the web-page as it existed at the time of authentication would be stored on the society web-page (which will be paid for by membership to said society). If you continue working on the same thing, you can add data here and there or maybe even link it to another page on your lab web-site which will again be authenticated. If your work is proven false, then the society can let your certificate expire. A periodic check through your pages to see if any links you have made to others have expired, would take care of dubious cycles of citing bad papers. And, since the department is already paying for the servers and such infrastructure, you dont have to pay anyone for 'publishing' your work. And, as it is on the web-site which can be reached by a google search, it is free for everyone to read. If enough people care about what you have to say, they will link it on their pages, and that would provide the prestige factor to you.
So, by decentralizing the entire process, you gain a lot. One, you are not at the mercy of some random 'expert' and an equally clueless editor who knows nothing about your sub-field. Two, by decentralizing the whole process of disseminating science information, you are taking away the rich gets richer scheme operative today (with the so called experts lending crap to 'prestige' journals and becoming editors elsewhere to dump more crap). Third, we have already seen in quite a few cases that a decentralized peer-review is the best form of review. So, people would comment on your work, and if they agree, your work keeps getting linked. If not, then your work is forgotten. So, by taking the focus away from all metrics, we get back into the subjective era of figuring out the importance of a work. It worked really well for centuries and only in the past few decades have we manged to screw it up by introducing idiotic metrics. So, in conclusion, down with all metrics and let us accept once for all that the only way to find the importance of a work is to let it stand the test of time.
Comments