Replicability-Report for JPSP: Personality Processes and Individual Differences

The Journal of Personality and Social Psychology is considered the top journal for personality and social psychology.  The journal has three distinct sections with different chief-editors. This replicability-report examines the replicability of research published in the JPSP: Personality Processes and Individual Differences section.

Impact factors are only available for all three sections combined.  The average all-time citation of an article in JPSP since 1989 is 100 times per article.  SCImago rankings of all psychology journals ranked Journal of Personality and Social Psychology #12 with an SJR-Impact-Factor of 4.8 in 2014.

At present, the replicability-report is based on articles published from 1989 to 2015.  During this time, JPSP:PPID published 1748 articles.  The Replicability-Report is based on 964 articles that reported one or more t or F-test in the text (results reported in Figures or Tables are not included).  The test-statistic was converted into z-scores to estimate post-hoc-power.  The analysis is based on 5,331 z-scores in the range from 2 (just above the 1.96 criterion value for p < .05 (two-tailed) to 4.

PHP-Curve JPSP-PPID

Based on the distribution of z-scores in the range between 2 and 4, the average power for significant results in this range is estimated to be 55%. This estimate suggests that only half of the published significant results in this range are predicted to produce a significant results in an exact replication study with the same sample size and power (results with z > 4 are expected to replicate with nearly 100%).

The same method was used to estimate power for individual years.  The results show a flat line. However, there is some evidence that power may have increased in the years after the replication crisis in psychology that increased awareness about questionable research practices in psychological research in 2011.  The replicability scores in 2012 to 2015 were above the historic average and the replicability score in 2015 is 68.  Based on this score JPSP:PPID ranks 7 out of 27 journals in psychology.

PHP-Trend JPSP-PPID

Advertisements

2 thoughts on “Replicability-Report for JPSP: Personality Processes and Individual Differences

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s