ResearchCraft

Enrolling now · Live online workshop

Statistics & Experiment Design for ML Research

A 3-hour intensive on statistical rigour for ML research — experiment design, baselines, significance testing, and ablation studies — so your results survive reviewer scrutiny instead of being the reason for rejection.

Duration
3 Hours
Format
Online Workshop
Modules
9

Curriculum

9 modules, one session

3 Hours of live teaching, in the order you'll use it on a real project.

  1. 01

    Designing a Defensible Experiment

    Choosing datasets, splits, and evaluation protocols before you run a single experiment.

  2. 02

    Baselines That Hold Up

    Selecting and tuning fair baselines, and avoiding the comparisons reviewers flag as unfair.

  3. 03

    Significance Testing

    t-tests, bootstrapped confidence intervals, and when p-values are (and aren't) the right tool for ML results.

  4. 04

    Variance, Seeds & Reproducibility

    Reporting variance across seeds, and why a single run is not a result.

  5. 05

    Ablation Study Design

    Isolating the contribution of each component without an explosion of experiments you can't afford.

  6. 06

    Common Statistical Pitfalls

    Multiple comparisons, cherry-picked metrics, and data leakage that quietly invalidates results.

  7. 07

    Visualising Results Honestly

    Plots and tables that show variance and effect size, not just a bar chart of means.

  8. 08

    Compute-Aware Experiment Planning

    Budgeting GPU hours across baselines, ablations, and seeds before you start.

  9. 09

    Writing the Results Section

    Presenting statistical evidence in prose that a rigorous reviewer will trust.

Outcomes

What you'll leave with

01

Experiment Protocol

A pre-registered plan for datasets, splits, baselines, and metrics.

02

Significance Report

A statistical analysis template for your own results.

03

Ablation Matrix

A structured ablation plan scoped to your compute budget.

04

Results Section Draft

A drafted, critiqued results section for your paper.

Tools & frameworks you'll use

  • PyTorch
  • Python
  • Matplotlib / Seaborn
  • Weights & Biases
  • scikit-learn
  • SciPy
  • GitHub
  • SLURM
  • statsmodels
  • R

Enroll

Ready to make your results reviewer-proof?

Pay securely by card and your seat is confirmed instantly — we'll email you the joining details.

  • Full refund if you cancel before the start date
  • Small cohort — 15 to 25 people

Keep going

More live courses

See all courses