Richard Smith does not propose a voting system, but the complete elimination of gatekeepers. It's worth emphasising that he is talking about biomed exclusively: other disciplines have different scholarly cultures. Most crucially, the commercial impact of biomed papers is higher than in other fields.
When you write
>reading research papers and judging if they are new, correct, and significant is TOUGH work.
I agree, but when you write:
>The present process makes a good effort at this tough work
I am surprised you say this. Smith points to studies that show that referees' opinions have very little correlation, which suggests the minimum level of filtering actively done by referees is very low. It does not follow from this, though, that the institution of journals considered as a whole does not perform a useful filtering function, which is to say that I agree with you about your point about successive filters.
>Publication delays
Many journals follow anonymous reviewing and forbid distribution of preprints.
The other point, which he does not make, is that referees have very little incentive, besides the opinion of the editorial board, to do their job properly.
>His point about novel, original, or innovative work being rejected is poorly considered. ... If after that the paper is also wildly 'novel', so be it; it still got published!
There's no doubt in that case that the journal has been an impediment to science.
I don't advocate abandoning peer review. I do think we should be more open-eyed about the problems with it. It is a costly system, and one that does not work the way it is intended. Science needs gatekeepers, as you argue, although they need not resemble the current journal system.
On Smith's article and a voting system, uh, I read Smith's article ASAP! I was assuming, without careful checking, that Smith's article was much like that of this thread. Looks like you got me in some too fast reading!
Yes, it looks like these articles are for biomedical work. I'm sure it's different in many small respects and maybe some larger ones.
Uh, I said that the present system makes a good "effort", and you countered with some of the problems with the "results"! You may be right!
The "correlation" is not very impressive: The review process is not quite that simple. Here's a dark, dirty little secret: Often at some point in the toughest part of the paper, some reviewer thinks, "I tried to look up the background for that, and I don't have the prerequisites even for the background. I can't take out a year to study to be ready to read this paper. The parts of the paper I can read look rock solid. The new parts I can read look nice. His references are high quality. His writing is carefully done. He doesn't make wild claims. It looks like a solid piece of work. I can't find anything wrong with it. Let him publish it: If it has a big error, then maybe that will come out and be on him. I'll give him a pass and go to dinner." or some such. So in this case correlation doesn't mean much!
"Many journals follow anonymous reviewing and forbid distribution of preprints." Sure. In most simple respects, the paper is still all locked up until it appears in print maybe two years after submission. Still work goes on! So, the author of the paper can present parts of the work in lectures and seminars. Guys down the hall know and can start working on extensions. The author can work on extensions and give talks and reference the paper as "in submission". Generally the word gets around! Net, the publication delay doesn't much slow the actual research work. That is, the actual exploitation of the work in the paper doesn't really start just with the publication and people reading that, Instead, 'the word gets around'.
"The other point, which he does not make, is that referees have very little incentive, besides the opinion of the editorial board, to do their job properly." The referees have a LOT of "incentive ... to do their work properly" if they want progress in their academic careers! Good referees get invited to be editors; good editors get invited to be good editors in chief. A referee, editor, or editor in chief is a 'gatekeeper' and, thus, a somewhat powerful guy. If the journal is good, the names are up in lights. So, professional reputation is enhanced. Professional connections are made. In terms of HN, there are connections in a professional social graph. A referee, editor, etc. gets to know who the up and comers are in their research field. They get early notice of new research results and, then, can start working on extensions. If they do a bad job, then their editor will soon not send them anymore papers to review!
"There's no doubt in that case that the journal has been an impediment to science." I don't see that. Einstein published general relativity. People are still trying to understand it, test it, and understand the implications. So, at the time of publication, just how novel it was was not at all clear. But it got published and passed the first step in the filter. Slowly people understood that the paper was very important and worked hard on it. The publication process didn't end the work on the paper but did its job. I can't believe that many of the reviewers actually understood the Riemannian geometry. Still the paper got published. If the paper was actually nonsense, then that would be on Uncle Al.
I believe that the current system is okay. The people objecting just want the system to be much more than it is; they may be asking for too much. One way to improve is to quit chopping down trees, but that change is likely on the way. But replacing the current system by putting papers on the Internet and letting nearly anyone 'vote' would be much worse.
When you write
>reading research papers and judging if they are new, correct, and significant is TOUGH work.
I agree, but when you write:
>The present process makes a good effort at this tough work
I am surprised you say this. Smith points to studies that show that referees' opinions have very little correlation, which suggests the minimum level of filtering actively done by referees is very low. It does not follow from this, though, that the institution of journals considered as a whole does not perform a useful filtering function, which is to say that I agree with you about your point about successive filters.
>Publication delays Many journals follow anonymous reviewing and forbid distribution of preprints.
The other point, which he does not make, is that referees have very little incentive, besides the opinion of the editorial board, to do their job properly.
>His point about novel, original, or innovative work being rejected is poorly considered. ... If after that the paper is also wildly 'novel', so be it; it still got published!
There's no doubt in that case that the journal has been an impediment to science.
I don't advocate abandoning peer review. I do think we should be more open-eyed about the problems with it. It is a costly system, and one that does not work the way it is intended. Science needs gatekeepers, as you argue, although they need not resemble the current journal system.