The Reproducibility Crisis

•

“studies that replicate are cited at the same rate as studies that do not” [in psychology] "The majority (80%) of papers citing retracted research were published after the retraction date" [on covid] (continues below) t.co/bQI3T9sXt5 fantasticanachronism.com/2021/01/11/are-experts-real/

"Notorious fraud Brian Wansink racked up 2500 citations in 2020, two years after he was forced to resign"

"Even when they do take action, journals will often let scientists correct faked figures instead of retracting the paper!" fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/#or-Maybe-They-Don-39-t

Fraud is also a major problem. An example via @luispedrocoelho twitter.com/MicrobiomDigest/status/1266140721716719616?s=20&t=rEUemRn2rrAoQhFg4xZhCA

"And even after being caught for outright fraud, about half of the offenders are allowed to keep working: they have received over $123 million in federal funding for their post-misconduct research efforts." No skin in the game journals.sagepub.com/doi/abs/10.1177/1556264616682568

The previous article also notes another problem: "a replication of a badly designed study is still badly designed" And suggests a few solutions to the replication problem fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/#or-Maybe-They-Don-39-t

And a few more points below.

IMHO a core bottleneck to the reproducibility crisis is also that, in many academic fields, availability of funding is based on the reputation of the field more than anything else – and exposing a researcher as a fraud might lower funding for everyone else in the field.

I would suggest an academic metric indicating the ratio of studies one cited that didn't reproduce "This paper has a non-reproducible citations rate of 50%" "This academic has a non-reproducible citations rate of 50%" "This field has a non-reproducible citations rate of 50%"

Anyway, the system is currently broken. And we should stop trusting that "published" means "reliable"