Replicability-Report for JOURNAL OF CROSS-CULTURAL PSYCHOLOGY

The JOURNAL OF CROSS-CULTURAL PSYCHOLOGY (JCCP) is published by Sage

SCImago rankings of all psychology journals ranked JCCP #191 with an SJR-Impact-Factor of 1.2 in 2014.

At present, the replicability-report is based on articles published from 2000 to 2015.  During this time, JCCP published 881 articles.  The Replicability-Report is based on 591 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included).  The test-statistic was converted into z-scores to estimate post-hoc-power.  The analysis is based on 2,193 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.

PHP-Curve JCCPBased on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 68% with a homogeneous model, which is currently being used for the replicability ranking.  The average power assuming heterogeneity is 58%.  This estimate suggests that only half of the published results with z-scores in this range yield significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).

The same method was used to estimate power for individual years.

PHP-Trend JCCP

The results show a flat time trend.  Due to the relatively small number of observations in a year, annual estimates vary considerably, but the average estimate in 2015 is close to the historic average of JCCP.  A replicability score of 65% in 2015 places JCCP in the top-third of psychology journals. In the OSF-Reproducibilty Project, the actual rate of successful replicaitons is likely to be about 20% lower than the statistically predicted power. Thus, about 1/3 of the published results are expected to produce significant results in studies that aim to reproduce the original studies.

Advertisements

One thought on “Replicability-Report for JOURNAL OF CROSS-CULTURAL PSYCHOLOGY

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s