Just Emil Kirkegaard Things

Share this post

Reverse publication bias: a collection

www.emilkirkegaard.com

Discover more from Just Emil Kirkegaard Things

No filter science
Over 5,000 subscribers
Continue reading
Sign in

Reverse publication bias: a collection

Emil O. W. Kirkegaard
May 7, 2020
1
Share this post

Reverse publication bias: a collection

www.emilkirkegaard.com
Share

See also Sesardić's conjecture, a related idea.

Publication bias as normally considered is really positive publication bias, i.e., the bias is away from zero, towards finding larger than reality results. There is another form, however, more rare, called reverse publication bias, or negative publication bias, where published results are biased towards the null. This pattern results from researchers' own biases towards having something noteworthy to report, wanting to publish it, and getting through peer and editorial review. All of these things are related to political ideology of the researchers and their academic environment (self-censorship). Thus, due to the massive political ideology skew in social science, we expect:

  1. Positive publication bias for left-wing friendly and results in general

  2. Negative/reverse publication bias for results that left-wing unfriendly results

Below I list the examples I know of.

GPA and intelligence

  • Roth, B., Becker, N., Romeyke, S., Schäfer, S., Domnick, F., & Spinath, F. M. (2015). Intelligence and school grades: A meta-analysis. Intelligence, 53, 118-137.

Twitter avatar for @KirkegaardEmil
Emil O W Kirkegaard @KirkegaardEmil
Reverse publication bias: when scientists want to find smaller effects. (IQ and GPA) sciencedirect.com.sci-hub.io/science/articl…
Image
12:53 AM ∙ Mar 20, 2016
32Likes15Retweets

Sex difference in spatial ability

This study was still unpublished when I asked the Jakob about it Apr 11, 2019.

Twitter avatar for @KirkegaardEmil
Emil O W Kirkegaard @KirkegaardEmil
Jakob Pietschnig: Reverse publication bias for sex difference in spatial ability. Psychologists publish as if trying to hide the male advantage. #ISIR2018 programme.exordo.com/isir2018/deleg…
Image
Image
Image
Image
1:57 PM ∙ Jul 15, 2018
45Likes15Retweets

Race difference in personality (OCEAN)

  • Tate, B. W., & Mcdaniel, M. A. (2008). Race differences in personality: An evaluation of moderators and publication bias. Preprint from before preprints were cool.

Twitter avatar for @KirkegaardEmil
Emil O W Kirkegaard @KirkegaardEmil
More reverse publication bias: race differences in personality. Altho these gaps are fairly trivial for US black-white on self-reported OCEAN, there's still evidence of suppression. Seems in general that group difference research often shows reverse bias. researchgate.net/publication/23…
Image
Image
4:18 AM ∙ Feb 17, 2019
2Likes1Retweet

Other

One can find recent discussions by searching Twitter: reverse publication bias or more strict with quotes "reverse publication bias".

A bit of a side-interest is when people are not trying to suppress some finding, but just want to avoid the result of a statistical test of an assumption being violated. This can be from testing means in post-randomization groups (these are hopefully all false positives that should happen 5% of the time), or testing for model assumptions (e.g. normal distribution). There's a lot of these kind of meta papers, two examples:

  • Chuard, P. J., Vrtílek, M., Head, M. L., & Jennions, M. D. (2019). Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?. PLoS biology, 17(1), e3000127.

  • Snyder, C., & Zhuo, R. (2018). Sniff Tests in Economics: Aggregate Distribution of Their Probability Values and Implications for Publication Bias (No. w25058). National Bureau of Economic Research.

These papers concern test of concern in studies, not tests for main findings that are undesired.

I first heard of this concept in ~2012 from Neuroskeptic:

Twitter avatar for @Neuro_Skeptic
Neuroskeptic 🇺🇦 @Neuro_Skeptic
New post: neuroskeptic.blogspot.com/2012/02/beware… The dangers of reverse publication bias - when everyone wants to publish negative results
neuroskeptic.blogspot.comBeware Reverse Publication BiasIn all the fuss over the pressure for scientists to publish positive results , we may have been missing an equally dangerous kind of publica...
7:33 AM ∙ Feb 23, 2012
14Likes18Retweets

Some have also tried to explain away replication failures as being a kind of reverse p-hacking:

Twitter avatar for @dsquintana
Dan Quintana @dsquintana
There is indeed evidence of reverse publication bias
journals.plos.orgEvidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?There is concern that the scientific literature is biased towards reporting statistically significant results. By contrast, this Perspective article presents evidence for an unusual situation in which there is a systematic bias towards publishing non-significant findings.
11:45 AM ∙ May 10, 2019
7Likes2Retweets
Twitter avatar for @page_eco
Lionel Page @page_eco
Is there a "reverse publication bias" in favour of the efficient market hypothesis? datascienceassn.org/sites/default/…
Image
6:18 AM ∙ Jan 24, 2018
1
Share this post

Reverse publication bias: a collection

www.emilkirkegaard.com
Share
Comments
Top
New
Community

No posts

Ready for more?

© 2023 Emil O. W. Kirkegaard
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing