#21 · ML / AI methods

Decoder evaluation shifts from held-out accuracy to threshold, robustness and validity

The common thread is that evaluation of neural decoders and simulators is shifting from aggregate accuracy to diagnostics that expose where inference actually fails. The mouse ECoG study locates anesthetic state-decoding failure at the decision threshold rather than the learned representation, and reports that a baseline-anchored threshold fixes ketamine decoding. BEND-BCI scores 23 neural decoders across 16 recordings on robustness, cost and cross-recording latent consistency beyond held-out accuracy. The neural mass model audit applies a hierarchical framework to simulation-based inference, testing observational coverage, parameter recoverability and parameter compensation before mechanistic interpretation. What is new is treating the validity of the inference pipeline, not model fit alone, as the object of assessment. The items differ in modality and scope, so the shared element is methodological rather than a common dataset or model class.