Verify the evidence, not the claim

Course overview

A foundational course in scientific methodology, study design hierarchies, statistical power, p-value interpretation, GRADE evidence grading, and bias detection.

Track

Evidence literacy

Level

Foundation

Lessons

10

Estimated duration

60 min

Complete course curriculum (10 lessons)

1. Lesson 1

   The Hierarchy of Evidence & Scientific Method

   Learning objective:

   Understand the evidentiary hierarchy from mechanistic hypotheses to randomized controlled trials.

   Core mechanism:

   Evidence ranges in methodological strength: expert opinion -> in vitro models -> animal studies -> observational cohorts -> randomized trials -> systematic reviews.

   Key takeaway:

   Higher evidentiary tiers reduce confounding and systematic bias, providing greater causal certainty.

2. Lesson 2

   In Vitro, In Vivo & In Silico Research Models: Scope & Limits

   Learning objective:

   Critically evaluate model systems and translational validity boundaries.

   Core mechanism:

   In vitro assays isolate biochemical pathways; animal models introduce whole-organism physiology but differ in metabolism, receptor distribution, and pharmacokinetics from humans.

   Key takeaway:

   Mechanistic findings in cultured cells or rodents cannot be claimed as proven human outcomes.

3. Lesson 3

   Observational Studies vs Randomized Controlled Trials (RCTs)

   Learning objective:

   Examine confounding, selection bias, and the power of randomization.

   Core mechanism:

   Observational studies identify correlations but are vulnerable to healthy user bias and unmeasured confounders. Randomization balances baseline characteristics, enabling causal inference.

   Key takeaway:

   Correlation in observational data does not establish biological causality.

4. Lesson 4

   Estimands, Endpoints & Surrogate vs Clinical Outcomes

   Learning objective:

   Differentiate laboratory biomarker changes from validated clinical endpoints.

   Core mechanism:

   A surrogate endpoint (e.g. LDL-C, blood pressure) is a laboratory measure intended to substitute for a clinical endpoint (e.g. myocardial infarction, survival). Surrogates do not always translate to clinical benefit.

   Key takeaway:

   A positive change in a surrogate biomarker does not automatically prove clinical outcome efficacy.

5. Lesson 5

   Statistical Power, Sample Size & Effect Magnitude

   Learning objective:

   Analyze Type I (alpha) and Type II (beta) errors and statistical power (1 - beta).

   Core mechanism:

   Statistical power is the probability of detecting a true effect when one exists. Underpowered studies with small sample sizes produce high false-positive rates and inflated effect sizes (the winner’s curse).

   Key takeaway:

   Small study sample sizes undermine reliability and inflate reported effect magnitudes.

6. Lesson 6

   P-Values, False Discovery Rates & Confidence Intervals

   Learning objective:

   Interpret p-values correctly as conditional probabilities, not effect sizes or posterior truths.

   Core mechanism:

   A p-value is the probability of obtaining test results at least as extreme as observed, assuming the null hypothesis is true. It does not measure effect size or truth probability; 95% confidence intervals convey effect magnitude and precision.

   Key takeaway:

   Statistical significance (p < 0.05) is not biological or clinical significance.

7. Lesson 7

   Systematic Reviews, Meta-Analyses & Forest Plots

   Learning objective:

   Read meta-analytic forest plots, pooled effect sizes, and heterogeneity (I^2 statistic).

   Core mechanism:

   Meta-analyses mathematically combine effect sizes from multiple studies. The I^2 statistic quantifies heterogeneity; I^2 > 50% indicates substantial inconsistency among included trials.

   Key takeaway:

   Meta-analyses are only as reliable as their constituent studies; pooling biased studies produces biased results.

8. Lesson 8

   The GRADE Framework: Rating Certainty of Evidence

   Learning objective:

   Apply GRADE criteria to evaluate High, Moderate, Low, or Very Low evidence certainty.

   Core mechanism:

   GRADE assesses five downgrading domains: risk of bias, inconsistency, indirectness, imprecision, and publication bias. High-quality RCTs start at High but can be downgraded.

   Key takeaway:

   GRADE provides an objective, transparent framework for rating confidence in effect estimates.

9. Lesson 9

   Identifying Methodological Bias, Confounding & P-Hacking

   Learning objective:

   Detect publication bias (funnel plots), selective outcome reporting, and HARKing.

   Core mechanism:

   Funnel plot asymmetry indicates missing negative studies due to publication bias. P-hacking involves testing multiple hypotheses without statistical correction until p < 0.05 is found.

   Key takeaway:

   Skeptical evidence evaluation checks for preregistered protocols and symmetrical reporting.

10. Lesson 10

    Capstone: Claim-Level Auditing & Claims Collapse Protocol

    Learning objective:

    Perform an adversarial audit of scientific literature claims, mapping statements to primary receipts.

    Core mechanism:

    Deconstructs published claims into atomic statements, tagging each as: direct human observation, derived calculation, mechanistic inference, or unproven assertion.

    Key takeaway:

    Claim-level adjudication prevents mechanistic speculation from being presented as established clinical truth.