Replicability Report for PSYCHOLOGICAL SCIENCE

PSYCHOLOGICAL SCIENCE is the flagship journal of the Association for Psychological Science (APS).
The journal started publishing in 1990. It has published 3,414 articles from 1990 to 2014. The average,all-time, citation rate of an article is 50.   SCImago rankings of all psychology journals ranked PSYCHOLOGICAL SCIENCE #18 with an SJR-Impact-Factor of 3.6 in 2014.

The Replicability-Report is based on 2,309 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included).  The test-statistic was converted into z-scores to estimate post-hoc-power.  The main analysis is based on z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.

PHP-Curve PsySci

The analysis is based on 16,082 z-scores (~ 7 tests per article).  The main finding is that the distribution of z-scores in the 2-4 range is consistent with 59% power for tests that produced significant results with z-scores between 2 and 4.

The same method was used to estimate power for individual years.  The number of results for the first years was too small to produce reliable estimates. Thus, the plot of time trends is limited to the years from 2000 to present.

PHP-Trend PsySci

The results show a negative trend with power decreasing from over 60% in the beginning and power below 60% in the years since 2008.  The trend shows no evidence that the replication crisis in psychology has changed editorial decisions in PSYCHOLOGICAL SCIENCE. The replicability-score in 2015 is 53.  Based on this score PSYCHOLOGICAL SCIENCE ranks in the lower half of psychology journals.  The score for 2015 can still change when more articles are published.

Advertisements

5 thoughts on “Replicability Report for PSYCHOLOGICAL SCIENCE

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s