I can hypothesis some factors which could give these results:
- It could be that high-citation-count articles have lower replicability, regardless of journal. Results that are more-surprising may be more likely to be cited than results that are unsurprising.
- High-prestige journals are more general (think Science, Nature), so the editors have less ability to evaluate articles, and less ability to figure out which reviewers have the expertise to evaluate articles. I mean, if you're the editor of "The Journal of Aquinas Studies in Metaphysics", you can be confident that your reviewers are familiar with Thomas Aquinas' metaphysics.
- High-prestige journals are good at courting prestige. At present, courting prestige seems to require high-visibility wokeness (think Science, Nature, or Scientific American). Commitment to wokeness may reduce replicability.
Question: How does research category effect replicability? Is it possible that research on sexuality and cognition beats sociology, behavioral psychology, consumerism, and endocrinology, even if they appear in different journals?
A good way to do this is probably paper topic modeling, and then see which keyword categories are the best indicators of replicability.
I can hypothesis some factors which could give these results:
- It could be that high-citation-count articles have lower replicability, regardless of journal. Results that are more-surprising may be more likely to be cited than results that are unsurprising.
- High-prestige journals are more general (think Science, Nature), so the editors have less ability to evaluate articles, and less ability to figure out which reviewers have the expertise to evaluate articles. I mean, if you're the editor of "The Journal of Aquinas Studies in Metaphysics", you can be confident that your reviewers are familiar with Thomas Aquinas' metaphysics.
- High-prestige journals are good at courting prestige. At present, courting prestige seems to require high-visibility wokeness (think Science, Nature, or Scientific American). Commitment to wokeness may reduce replicability.
Question: How does research category effect replicability? Is it possible that research on sexuality and cognition beats sociology, behavioral psychology, consumerism, and endocrinology, even if they appear in different journals?
A good way to do this is probably paper topic modeling, and then see which keyword categories are the best indicators of replicability.