Verify the evidence, not the claim

Course overview

A foundational course in scientific methodology, study design hierarchies, statistical power, p-value interpretation, GRADE evidence grading, and bias detection.

Track

Evidence literacy

Level

Foundation

Lessons

10

Estimated duration

60 min

Scientific review panel

Clinical Epidemiology

Evaluates study architectures: in vitro, animal models, observational cohorts, and randomized trials.

Biostatistics

Models statistical power, type I/II errors, effect sizes, confidence intervals, and p-value limits.

Evidence Synthesis

Applies GRADE methodology, PRISMA guidelines, and Cochrane risk-of-bias frameworks.

Claims Architecture

Enforces claim-level evidence mapping, distinguishing direct observations from mechanistic inferences.

Complete course curriculum (10 lessons)

1. Lesson 1

   The Hierarchy of Evidence & Scientific Method

   Learning objective: Understand the evidentiary hierarchy from mechanistic hypotheses to randomized controlled trials.

   Core mechanism: Evidence ranges in methodological strength: expert opinion -> in vitro models -> animal studies -> observational cohorts -> randomized trials -> systematic reviews.

   Key takeaway: Higher evidentiary tiers reduce confounding and systematic bias, providing greater causal certainty.

2. Lesson 2

   In Vitro, In Vivo & In Silico Research Models: Scope & Limits

   Learning objective: Critically evaluate model systems and translational validity boundaries.

   Core mechanism: In vitro assays isolate biochemical pathways; animal models introduce whole-organism physiology but differ in metabolism, receptor distribution, and pharmacokinetics from humans.

   Key takeaway: Mechanistic findings in cultured cells or rodents cannot be claimed as proven human outcomes.

3. Lesson 3

   Observational Studies vs Randomized Controlled Trials (RCTs)

   Learning objective: Examine confounding, selection bias, and the power of randomization.

   Core mechanism: Observational studies identify correlations but are vulnerable to healthy user bias and unmeasured confounders. Randomization balances baseline characteristics, enabling causal inference.

   Key takeaway: Correlation in observational data does not establish biological causality.

4. Lesson 4

   Estimands, Endpoints & Surrogate vs Clinical Outcomes

   Learning objective: Differentiate laboratory biomarker changes from validated clinical endpoints.

   Core mechanism: A surrogate endpoint (e.g. LDL-C, blood pressure) is a laboratory measure intended to substitute for a clinical endpoint (e.g. myocardial infarction, survival). Surrogates do not always translate to clinical benefit.

   Key takeaway: A positive change in a surrogate biomarker does not automatically prove clinical outcome efficacy.

5. Lesson 5

   Statistical Power, Sample Size & Effect Magnitude

   Learning objective: Analyze Type I (alpha) and Type II (beta) errors and statistical power (1 - beta).

   Core mechanism: Statistical power is the probability of detecting a true effect when one exists. Underpowered studies with small sample sizes produce high false-positive rates and inflated effect sizes (the winner's curse).

   Key takeaway: Small study sample sizes undermine reliability and inflate reported effect magnitudes.

6. Lesson 6

   P-Values, False Discovery Rates & Confidence Intervals

   Learning objective: Interpret p-values correctly as conditional probabilities, not effect sizes or posterior truths.

   Core mechanism: A p-value is the probability of obtaining test results at least as extreme as observed, assuming the null hypothesis is true. It does not measure effect size or truth probability; 95% confidence intervals convey effect magnitude and precision.

   Key takeaway: Statistical significance (p < 0.05) is not biological or clinical significance.

7. Lesson 7

   Systematic Reviews, Meta-Analyses & Forest Plots

   Learning objective: Read meta-analytic forest plots, pooled effect sizes, and heterogeneity (I^2 statistic).

   Core mechanism: Meta-analyses mathematically combine effect sizes from multiple studies. The I^2 statistic quantifies heterogeneity; I^2 > 50% indicates substantial inconsistency among included trials.

   Key takeaway: Meta-analyses are only as reliable as their constituent studies; pooling biased studies produces biased results.

8. Lesson 8

   The GRADE Framework: Rating Certainty of Evidence

   Learning objective: Apply GRADE criteria to evaluate High, Moderate, Low, or Very Low evidence certainty.

   Core mechanism: GRADE assesses five downgrading domains: risk of bias, inconsistency, indirectness, imprecision, and publication bias. High-quality RCTs start at High but can be downgraded.

   Key takeaway: GRADE provides an objective, transparent framework for rating confidence in effect estimates.

9. Lesson 9

   Identifying Methodological Bias, Confounding & P-Hacking

   Learning objective: Detect publication bias (funnel plots), selective outcome reporting, and HARKing.

   Core mechanism: Funnel plot asymmetry indicates missing negative studies due to publication bias. P-hacking involves testing multiple hypotheses without statistical correction until p < 0.05 is found.

   Key takeaway: Skeptical evidence evaluation checks for preregistered protocols and symmetrical reporting.

10. Lesson 10

    Capstone: Claim-Level Auditing & Claims Collapse Protocol

    Learning objective: Perform an adversarial audit of scientific literature claims, mapping statements to primary receipts.

    Core mechanism: Deconstructs published claims into atomic statements, tagging each as: direct human observation, derived calculation, mechanistic inference, or unproven assertion.

    Key takeaway: Claim-level adjudication prevents mechanistic speculation from being presented as established clinical truth.

Research-use boundary

Research use only. Not for human consumption.