(Preprint) Z-Curve: A Method for Estimating Replicability Based on Test Statistics in Original Studies (Schimmack & Brunner, 2017)

In this PDF document, Jerry Brunner and I would like to share our latest manuscript on z-curve,  a method that estimates average power of a set of studies selected for significance.  We call this estimate replicabilty because average power determines the success rate if the set of original studies were replicated exactly.

We welcome all comments and criticism as we plan to submit this manuscript to a peer-reviewed journal by December 1.

Highlights

Comparison of P-curve and Z-Curve in Simulation studies

Estimate of average replicability in Cuddy et al.’s (2017) P-curve analysis of power posing with z-curve (30% for z-curve vs. 44% for p-curvce).

Estimating average replicability in psychology based on over 500,000 significant test statitics.

Comparing automated extraction of test statistics and focal hypothesis tests using Motyl et al.’s (2016) replicability analysis of social psychology.

 

 

Advertisements

5 thoughts on “(Preprint) Z-Curve: A Method for Estimating Replicability Based on Test Statistics in Original Studies (Schimmack & Brunner, 2017)

  1. Very interesting!
    PS This needs editing:

    Based on the distribution of significant z-scores (z > 1.96), z-curve produced an estimate of However, the z-curve estimate is only 30%

    Like

  2. Without sampling error, results of identical studies would be identical.

    Isn’t a possible measure error to be taken into account? Change in temporary mood of population, say world championship win.

    Like

    1. Isn’t random measurement error due to uncontrollable situational factors, which is part of the first factor that can produce different results across identical studies?

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s