R-Index predicts lower replicability of “subliminal” studies than “attribution” studies in JESP

PHP_subliminal_vs_attribution

This post compares articles in the Journal of Experimental Social Psychology that contained the keyword “subliminal” to articles that contained the word “attribution”.

PHP-curves based on t-tests and F-tests in these articles are compared.  Both sets of articles show signs of publication bias (fewer non-significant studies are reported than predicted based on post-hoc power).

The shape of the histogram shows clear evidence of heterogeneity (the red curve fits the data better than the green curve).

The estimated power of studies with z-scores between 2 and 4 for subliminal articles is 31%.

The estimated power of studies with z-scores between 2 and 3 for attribution articles is 42%.

The R-Index for subliminal articles is 39%, whereas the R-Index for attribution articles is 49%.

The values for subliminal articles are also lower than the values for the whole set of articles in JESP.

In conclusion, these results suggest that subliminal priming studies are less replicable than other findings in social psychology and should be the target of high-powered replication studies.  These replication studies need to take into account that reported effect sizes are inflated to achieve high power.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s