Replicability-Report for JOURNAL OF EXPERIMENTAL PSYCHOLOGY: HUMAN PERCEPTION AND PERFORMANCE

JOURNAL OF EXPERIMENTAL PSYCHOLOGY: HUMAN PERCEPTION AND PERFORMANCE (JEP:HPP) is published by the American Psychological Association (APA). The journal started publishing articles in 1982.

SCImago ranks JEP:HPP as #76 of all psychology journals with an SJR-Impact-Factor of 1.8 in 2014. At present, the replicability-report is based on articles published from 2000 to 2015. During this time, JEP:HPP published 1718 articles. The replicability-report is based on 1502 articles that reported one or more t or F-test in the text of the results section (results reported in Figures or Tables are not included).  The test-statistic was converted into z-scores to estimate post-hoc-power.  The analysis is based on 18,208 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed)) to 4.

PHP-Curve JEP-HPP

Based on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 71% with the homogeneous model that is currently being used for the replicability rankings. The heterogeneous model fits the actual data better and produces an estimate of 64% power in this range.  Power for all significant results is estimated to be 72%.  A power estimate of 71% implies that 71% of the published significant results in this range are predicted to produce a significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).

The same method was used to estimate power for individual years.

PHP-Trend JEP-HPP

The time trend shows  a flat line. Moreover, due to the large number of observations in each year, annual averages are close to the historic average of 71%.  The average for the years 2010-2014 is 71%, which places JEP:HPP in the top of the replicabilty rankings of psychological journals.  However, in the recent OSF-reproducibility project of 38 results from cognitive psychology the replicability estimate was 75%, but the actual replication rate was only 50%.  Thus, JEP:HPP should  aim to increase the power of studies that are published in this journal.

Advertisements

One thought on “Replicability-Report for JOURNAL OF EXPERIMENTAL PSYCHOLOGY: HUMAN PERCEPTION AND PERFORMANCE

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s