Most Published Research Findings are False (00:00-04:46)

• Introduction to the problem – that most published research findings are false

We screwed up the statistics (04:46 – 13:13)

• Discusses the key reasons why statistics such as the p value and statistical significance did not mean what we thought it mean and shows how we consequently grossly over-estimated the likelihood of our findings being real.
• Discusses the key statements from the American Statistical Association regarding the p value
• Demonstrates how p values are really likelihood ratios that convert a pre-test probability of a hypothesis being true (aka prior probability) into a post-test probability (aka posterior probability) where the study is the “test” and the p value represents the accuracy of the “test”.
  This is further explained in this supplementary post

We underestimated the impact of bias (13:12-19:53)

• Discusses the role of bias in creating false research findings
• Explains how bias doesn’t just affect our study findings but affects the validity of the p value itself and the post-test probability calcuations
• Explains how we have wilfully ignored the major source of bias: Invisible Bias
• Discusses how we must explicitly discount our post-test probabilities for both Invisible and Visible Bias.

EBM 2.0 – The Way Forward 19:53-24:31

• Briefly summarises the key principles of EBM 2.0 to take us forward and help protect and serve our patients best.
• Further details are provided in the full version of the talk and the supplementary posts on this webpage under EBM 2.0: More Information

Introduction (0:00 – 08:40)

• Introduction to the problem – that most published research findings are false

Bias, the underestimated factor (08:40-26:46)

• Discusses the role of bias in creating false research findings
• Provides several examples of both detectable and the more insidious undetectable forms of bias and argues that some bias needs to be “priced in” to all EBM and consequently our statistical tests and conclusions need to be discounted to account for this.

Chance, and how we screwed up the statistics so, so badly (26:46-45:20)

• Discusses the key reasons why statistics such as the p value and statistical significance did not mean what we thought it mean and shows how we consequently grossly over-estimated the likelihood of our findings being real.
• Discusses the key statements from the American Statistical Association regarding the p value
• Demonstrates how p values are really likelihood ratios that convert a pre-test probability of a hypothesis being true (aka prior probability) into a post-test probability (aka posterior probability) where the study is the “test” and the p value represents the accuracy of the “test”.
  • This is further explained in this supplementary post

The ATOM Principle (45:20-1:07:22)

• Discusses the ATOM principle recommended by the American Statistical Association to guide a path forward for EBM with several examples of applying this principle provided. ATOM stands for:
  • Accept Uncertainty and be
  • Thoughtful
  • Open
  • Modest

EBM 2.0 (1:07:22-1:20:46)

• Describes the EBM 2.0 project and movement aiming to change the way we practice EBM forever.
• Summarise the key principles of EBM 2.0.
• EBM 2.0 as an update to EBM 1.0 incorporates its useful parts (e.g. diligently examining studies for bias) and replaces the non-functioning parts (e.g. the use of “statistical significance”) with a new approach.