At present, the replicability-report is based on articles published from 2000 to 2015. During this time, JCCP published 881 articles. The Replicability-Report is based on 591 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included). The test-statistic was converted into z-scores to estimate post-hoc-power. The analysis is based on 2,193 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.
Based on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 68% with a homogeneous model, which is currently being used for the replicability ranking. The average power assuming heterogeneity is 58%. This estimate suggests that only half of the published results with z-scores in this range yield significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).
The same method was used to estimate power for individual years.
The results show a flat time trend. Due to the relatively small number of observations in a year, annual estimates vary considerably, but the average estimate in 2015 is close to the historic average of JCCP. A replicability score of 65% in 2015 places JCCP in the top-third of psychology journals. In the OSF-Reproducibilty Project, the actual rate of successful replicaitons is likely to be about 20% lower than the statistically predicted power. Thus, about 1/3 of the published results are expected to produce significant results in studies that aim to reproduce the original studies.