A surprising number is a bug first¶
Unexpectedly good performance is a reason to inspect the pipeline.
Working notes¶
- Independently recalculate the reported metric.
- Check leakage, sample counts and incomplete output files.
- Compare against an appropriate prior and known reference.
- Challenge the claim directly; corrections were often prompted rather than volunteered.
Source: course evidence synthesis, §2, §4. These are selected observations, not general performance guarantees.
Before the course¶
- [TODO: select a compact worked example and its verification artifact.]
- [TODO: rehearse the live walkthrough and prepare a fallback.]