Neural Noise- A simulation study examines false positives in control conditions and post-hoc testing in low-powered neuroimaging trials.1
- The work improves trial design and inference for BCI and neuroimaging studies.1 1
Weekly enrichment (2026-07-20)
- The source study (Frontiers in Neuroimaging, 2025) used large-scale Monte Carlo simulations—about 100 million synthetic datasets varying sample size, treatment effect, and test–retest variability—to study post-hoc testing in low-powered RCT designs.2
- It targets the common practice of running post-hoc pairwise comparisons (e.g., in waitlist-control designs) only on brain regions where the omnibus interaction effect is significant, and shows this selective conditioning inflates apparent control-group effects.2
- Quantitatively, when interaction-test power was ~20%, 11.4% of significant findings showed false positives in the control group; at 5% power this rose to 22.0%, but at the conventional 80% power benchmark it fell to roughly 5%.2
- The false-positive rate in the control condition was strongly, inversely related to overall statistical power (well fit by an inverse-variance-type function), so under-powered small-sample neuroimaging studies are most vulnerable.2
- The authors identify Berkson’s paradox (collider bias) as the mechanism: conditioning on a significant interaction induces spurious associations in follow-up comparisons, which can misattribute placebo response, natural variability, or noise to a real effect.2
- A complementary realistic neuroimaging simulation (independent-component/spatial maps) reproduced the same inflation, reinforcing that the artifact is not an artifact of the abstract model.2
- Practical recommendation: interpret unexpected control-group effects cautiously and prioritize adequately powered designs rather than relying on standard multiple-comparison corrections, which are typically applied only to the initial interaction test.2
- Context: Berkson’s paradox/collider bias arises when one conditions on a common effect (a collider) of two variables, inducing correlation even when they are independent in the population—the same structure as selecting regions by an interaction test.3
- Context: collider-stratification bias is a recognized form of selection bias in epidemiology and neuroscience and can distort associations under any conditioning, including restriction and stratification.4
- Context: prior simulation work on multiple-hypothesis testing likewise showed that low power plus selective follow-up testing harms reproducibility in neuroimaging, motivating higher critical-value standards.5
Footnotes
-
https://news.google.com/rss/articles/CBMilAFBVV95cUxPTEItWWRBN2dCWUJDU3BwSE9WZkJtVHV1bFh1WWc2TmN3LVk5TElvRXp6Zkxxa3dmbzJZV0RvQ2wyN1J6Qk5sWHRybnNOcUgwTDUwdEhUcjllTHBiSXloQ0Z4NDdKZHRWN1p1RGhMaWU0ckw1WTJqZ2d0bHRRRmJwRXcwZ0JKUkNzcnYzQXlHN2xTNGps?oc=5 ↩ ↩2 ↩3
-
https://www.frontiersin.org/journals/neuroimaging/articles/10.3389/fnimg.2025.1637148/full ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7