this post was submitted on 15 Aug 2024
849 points (98.1% liked)
Science Memes
10988 readers
1865 users here now
Welcome to c/science_memes @ Mander.xyz!
A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
- Don't throw mud. Behave like an intellectual and remember the human.
- Keep it rooted (on topic).
- No spam.
- Infographics welcome, get schooled.
This is a science community. We use the Dawkins definition of meme.
Research Committee
Other Mander Communities
Science and Research
Biology and Life Sciences
- !abiogenesis@mander.xyz
- !animal-behavior@mander.xyz
- !anthropology@mander.xyz
- !arachnology@mander.xyz
- !balconygardening@slrpnk.net
- !biodiversity@mander.xyz
- !biology@mander.xyz
- !biophysics@mander.xyz
- !botany@mander.xyz
- !ecology@mander.xyz
- !entomology@mander.xyz
- !fermentation@mander.xyz
- !herpetology@mander.xyz
- !houseplants@mander.xyz
- !medicine@mander.xyz
- !microscopy@mander.xyz
- !mycology@mander.xyz
- !nudibranchs@mander.xyz
- !nutrition@mander.xyz
- !palaeoecology@mander.xyz
- !palaeontology@mander.xyz
- !photosynthesis@mander.xyz
- !plantid@mander.xyz
- !plants@mander.xyz
- !reptiles and amphibians@mander.xyz
Physical Sciences
- !astronomy@mander.xyz
- !chemistry@mander.xyz
- !earthscience@mander.xyz
- !geography@mander.xyz
- !geospatial@mander.xyz
- !nuclear@mander.xyz
- !physics@mander.xyz
- !quantum-computing@mander.xyz
- !spectroscopy@mander.xyz
Humanities and Social Sciences
Practical and Applied Sciences
- !exercise-and sports-science@mander.xyz
- !gardening@mander.xyz
- !self sufficiency@mander.xyz
- !soilscience@slrpnk.net
- !terrariums@mander.xyz
- !timelapse@mander.xyz
Memes
Miscellaneous
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
For any scientific journal that's worth anything, your article has to get approved by other scientists in your field before the journal will accept it. They're mostly just looking for exactly what this post is referencing. Does it seem legit? If it passes a once-over by the other scientists, then it gets published.
This is why you should not trust any single study by itself. It's just the results from one experiment that easily could have had a consequential error no one picked up. The results could be statistical noise. Hell, even rarely, you'll get someone who's been faking data. This is not to say "science is broken," only that science has never relied on the results from a single unreplicated experiment to determine truth. If you read about scientists from the past, it's fairly common for them to publish a landmark paper and for no one to care, or even for people to argue they're wrong. Only with additional research do they get proved correct and we imagine that everyone immediately accepted this new paradigm shift off of one single paper.
Is there no journal/publishing site where other scientists can put out publicly visible peer reviews of a paper after the paper is already published?
A peer review really is just someone checking for glaring errors. If a paper gets published and someone had some real beef with it, best they can do is some of their own research to prove how shitty the other team was. After that, there are some journals that will publish letters where people comment on previous articles. But generally, most articles just get mildly ignored. It's only after a pattern of corroborating evidence piles up that people will start to say that the results of a particular early study were significant.
Mind you, the details about how this consensus process works varies from field to field. Particle physics has a different culture than hydrology. But, in general, one paper is not enough to hang your hat on.
I'm sorry, but this seems like a profoundly archaic, indirect, and unnecessary way to format it.
And with how brief you people seem to describe these peer reviews, they're apparently lower effort than a good reddit comment, yet they cannot be directly publicly visibly attached to the article they are directly reviewing?
Academia can't be too proud to take a hint of inspiration from the mitigating effects of well-informed internet comments and Twitter's community notes against low quality content?
Why would intelligent people shackle their own publications by simulating the limitations of last century? Separately published "letters"? Honestly?
The few times I've heard the processes of papers and journals described, they seem to be clinging to the logistical solutions of physical paper with some kind of demented nostalgic love for the flaws of it.
I think you're placing a lot more weight on the authority of a single scientific paper than any actual scientist ever would. If you have one paper, you have one paper. If you have a series of papers all put out by the same lab... maybe there's something there, maybe not. If other groups start publishing similar papers, okay this is sometime serious.
In some of the messier sciences, like medicine, people will publish meta-studies, where they combine results from similar, but independently published, papers and see what they can come up with using the combined data. People will also publish literature reviews, where they essentially try to summarize the state of the science in their particular little niche. To trust a single study in medicine is to hitch your horse to a wicket.
The peer review process doesn't stop wrong papers from getting published, just obviously wrong or bad ones. I'm not entirely sure what you could even do to stop wrong papers from making into journals, since often times the problem isn't in the published experimental design or analysis. Plus, there's some papers that used to be right, but have become wrong as things change.
They're not, people are being flippant. People frequently complain about having to do peer reviews specifically because it's unpaid labor. Regardless, if the paper is so wrong it would warrant a community note on Twitter, the paper would be strongly rejected. The standard for acceptance is way higher than that. Remember that it gets reviewed by fellow experts in the field. They will easily spot small errors.
Is it the best possible system? Heck if I know. It works. Moving to a different system would require everyone to recalibrate their understanding of what good science looks like. We know how to identify it under the current publication model, it would take a fair bit of time to adjust to the new one.
Edit: Oh yes, re: letters. It can take a year or more between publications. Letters might be slow, but it's not terribly important. It takes time to do science, you don't need to clap back in an instant.