Replicability-Report for the journal JUDGMENT AND DECISION MAKING

JUDGMENT AND DECISION MAKING (JDM) is an open-access journal published by the Society for Judgment and Decision Making

SCImago rankings of all psychology journals ranked JDM #149 with an SJR-Impact-Factor of 1.3 in 2014.

The journal started publishing articles in 2006.  The replicability-report is based on all articles that were posted from 2006 to 2015.  During this time, JDM published 425 articles.  The Replicability-Report is based on 305 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included).  The test-statistic was converted into z-scores to estimate post-hoc-power.  The analysis is based on 1,082 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.

PHP-Curve JDMBased on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 64% with a homogeneous model, which is currently being used for the replicability ranking.  The average power assuming heterogeneity is slightly lower with 59%.  This estimate suggests that about 60% of the published results with z-scores in this range yield significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).

The same method was used to estimate power for individual years.

PHP-Trend JDM

The results show a positive trend in post-hoc power. The highest estimates were obtained in the past two years. The current replicability score in 2015 is 70%, which is one of the highest scores for psychology journals.  Based on the rate of actual successful replications in cognitive psychology, the actual rate of successful replications is expected to be about 50%.  This is much higher than the actual rate of successful replications of results in social psychology with higher impact factors.  Based on the present results, I recommend JDM as a more credible source of scientific evidence.

Advertisements

One thought on “Replicability-Report for the journal JUDGMENT AND DECISION MAKING

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s