Independent investigation
Turning implementation skill into research: pick a claim, reproduce a slice of it, explain the gap, and design an experiment that could prove you wrong.
What you will learn
You can implement and evaluate. Research asks a different question: whether a published claim holds, and why not when it does not.
One claim stated precisely enough to be wrong, a bounded reproduction of its evidence, a discrepancy resolved with evidence rather than a guess, a hypothesis with a competing explanation, a protocol written before the runs, and a short honest write-up. Six exercises, no reading.
A report on a claim you tested yourself, with uncertainty, limitations and the negative results left in: the unit of work a research group deals in.
Deliverable: a claim and its evidence
One paper, one precise claim
Choose a single claim from one paper and write it down precisely enough to be wrong: what is asserted, over which tasks, at which budget, measured how. Vague claims cannot be reproduced or refuted.
Everything on this line is downstream of choosing this well.
Learning outcome: State someone else's claim precisely enough to test it.
Prerequisites
Resources
- Key Papers in Deep RL (site)
Exercise deliverables
- The claim quoted in one sentence, with the figure or table that supports it
- The protocol behind that evidence: tasks, budget, seeds, metric
- What would have to be true for the claim to generalise beyond that setting
Deliverable: your numbers next to theirs
Reproduce the smallest sufficient part
Reproduce the smallest bounded piece of that evidence, not the whole paper. Report your number and theirs with uncertainty on both, and list every deviation from their setup, including the ones you think do not matter.
Reproduction is where the real open problems surface.
Learning outcome: Bound a reproduction to something you can finish, and report it honestly.
Prerequisites
Exercise deliverables
- One bounded slice of the original evidence re-run
- Your result and theirs side by side, with uncertainty on both
- A complete list of deviations from their setup
Deliverable: candidate explanations, ruled in or out
A discrepancy is an observation, not a result
A failed reproduction is a starting point. The work is establishing the cause: implementation difference, hyperparameters, environment version, evaluation protocol, or nothing but seed noise. Rule candidates in or out with evidence before you tell anyone what you found.
This is the step that separates a complaint from a contribution.
Learning outcome: Attribute a discrepancy to a cause with evidence, or show it is within noise.
Prerequisites
Exercise deliverables
- Candidate explanations enumerated for each discrepancy
- At least one candidate ruled in or out by an experiment you ran
- An explicit check that the gap exceeds seed noise
Deliverable: a hypothesis and its rival
One falsifiable statement
Write one statement that could be false, the prediction it makes, and at least one competing explanation that predicts something different. If both explanations predict the same observation, you do not yet have an experiment.
The hinge between reproducing other people's work and producing your own.
Learning outcome: Turn an observation into a hypothesis that an experiment could kill.
Prerequisites
Exercise deliverables
- One falsifiable statement and the prediction it makes
- At least one competing explanation of the same observation
- The observation that would distinguish them
Deliverable: a pre-registered protocol
Design it before you run it
Write the protocol before the runs: factors and levels, interaction and tuning budget per arm, seeds, metric, and the analysis you will do. Pilot runs are for estimating spread, not for choosing the conclusion.
The difference between an experiment and a search for a flattering plot.
Learning outcome: Pre-register a study, then execute it without changing the question.
Prerequisites
Exercise deliverables
- The protocol written and dated before the runs: factors, budgets, seeds, metric, stopping rule
- A spread estimate from pilot runs, and the seed count it implies
- The analysis chosen in advance
Deliverable: a report and the code to rerun it
Report it, including what did not work
Method, results with uncertainty, limitations, and the negative or inconclusive parts left in. A short honest report is worth more than a long one that quietly drops the arms that disagreed.
The artefact you show a prospective supervisor or a reviewer.
Learning outcome: Produce a report someone else could act on, disagree with, or rerun.
Prerequisites
Exercise deliverables
- A short report: method, results with uncertainty, limitations
- Negative and inconclusive results kept in
- Code and configs sufficient to rerun the study
- A paragraph on what you would do next and why