This post compares articles in the Journal of Experimental Social Psychology that contained the keyword “subliminal” to articles that contained the word “attribution”.
PHP-curves based on t-tests and F-tests in these articles are compared. Both sets of articles show signs of publication bias (fewer non-significant studies are reported than predicted based on post-hoc power).
The shape of the histogram shows clear evidence of heterogeneity (the red curve fits the data better than the green curve).
The estimated power of studies with z-scores between 2 and 4 for subliminal articles is 31%.
The estimated power of studies with z-scores between 2 and 3 for attribution articles is 42%.
The R-Index for subliminal articles is 39%, whereas the R-Index for attribution articles is 49%.
The values for subliminal articles are also lower than the values for the whole set of articles in JESP.
In conclusion, these results suggest that subliminal priming studies are less replicable than other findings in social psychology and should be the target of high-powered replication studies. These replication studies need to take into account that reported effect sizes are inflated to achieve high power.