At present, the replicability-report is based on articles published from 2000 to 2015. During this time, BJSP published 667 articles. The Replicability-Report is based on 484 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included). The test-statistic was converted into z-scores to estimate post-hoc-power. The analysis is based on 2,163 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.
Based on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 54% with a homogeneous model, which is currently being used for the replicability ranking. The average power assuming heterogeneity is 49%. This estimate suggests that about half of the published results with z-scores in this range yield significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).
The same method was used to estimate power for individual years.
The results show a decreasing time trend. In the most recent years, seven out of eight replicabilty scores were below the historic average. The average for the years 2010-2014 was 48 and the current score for 2015 is 50. Based on the actual rate of successful replications in the OSF-Reproducibilty Project, a score of 50% is expected to produce 30% significant results in actual replication studies with the same sample size as the original studies. There is no evidence that editorial decisions have changed in response to the replicability crisis in social psychology.