They don't, this is speculative (i.e. a theory) and almost certainly untrue (or a gross over-simplification), much like the early and now disproven serotonin theories of depression.
> but you need to amortize in all the infrastructure and training and energy costs
The average American human consumes 232kWh of all-in energy (food, transport, hvac, construction, services, etc) daily.
If humans want to get into a competition over lower energy input per unit of cognitive output, I doubt you'd like the result.
> Better? Lol no
The "IQ equivalent" of the current SOTA models (Opus 4.5, Gemini 3 Pro, GPT 5.2, Grok 4.1) is already a full 1SD above the human mean.
Nations and civilizations have perished or been conquered all throughout history because they underestimated and laughed off the relative strength of their rivals. By all means, keep doing this, but know the risks.
The meaningless NHST ritual is so harmful here. Imagine what we might know by now if all those pointless studies had used their resources to do proper science...
Yes this is ancient news for experts, but, IMO, most fMRI research outside of methodological research is quite practically useless at the moment because of deep measurement issues like these.
So if awareness of this increases the skepticism of papers claiming to have learned things about the brain/mind from fMRI, then I'd say it is a net plus.
I do not think there is much neurological understanding about ADHD at all from current fMRI research, there are far too many quality and reliability issues here, not just on the fMRI end or limited amount of data overall, but in the measurement and diagnosis of ADHD itself (i.e. ADHD subtypes, and of course ADHD is a complicated diagnosis with many components manifesting to different degrees in different individuals, which makes it very hard to cleanly link to messy fMRI signals).
Or, as I have commented elsewhere here, the idea that statements like "fMRI shows decreased activity" are ever valid is just fundamentally suspect (lower BOLD response could mean less inhibition or less excitation, and this is a rather crucial difference that fMRI simply can't distinguish). EDIT: Or to be more precise: it may well be that fMRI research suggests less metabolic activity in certain regions, but this could mean the region is actually firing more than normal, less than normal, is more efficient than normal, etc., and interpreting anything about what is functioning differently in ADHD, given this uncertainty, is what is going to be suspect.
It makes sense they wouldn't look at it, there are very few, if any, well-validated clinical uses for it. However, they might have taken it as a baseline for later comparison, and it is definitely plausible when surgery is involved that visible abnormalities could be seen in fMRI that might not show up in MRI, either now or later.
I don't think there would be much clear guidance for them on how to interpret any such fMRI abnormality on its own, but it might still be something useful for further investigations, and this might especially be the case for surgery. It might also have been done as part of research, if you consented to anything like that?
I have also published and worked for some years in this field, if that helps.
The literature is huge, and my bias is that I believe most of the only really good fMRI research is methodological research (i.e. about what fMRI actually means, and how to reliably analyze it). Many of the links I've provided here speak to this.
I don't think there is much reliable fMRI research that tells us anything about people, emotions, or cognition, beyond confirming some likely localization of function to the sensory and motor cortices, and some stuff about the Default Mode Network(s) that is of unclear importance.
A lot of the more reliable stuff involves the Human Connectome Project (HCP) fMRI data, since this was done very carefully with a lot of participants, if you want a place to start for actual human-relevant findings. But the field is still really young.
It is in fact even difficult to compare the same person on the same fMRI machine (and especially in developmental contexts).
Herting, M. M., Gautam, P., Chen, Z., Mezher, A., & Vetter, N. C. (2018). Test-retest reliability of longitudinal task-based fMRI: Implications for developmental studies. Developmental Cognitive Neuroscience, 33, 17–26. https://doi.org/10.1016/j.dcn.2017.07.001
I read that paper as suggesting that development, behavior, and fMRI are all hard.
It's not at all clear to me that teenagers' brains OR behaviours should be stable across years, especially when it involves decision-making or emotions. Their Figure 3 shows that sensory experiments are a lot more consistent, which seems reasonable.
The technical challenges (registration, motion, etc) like things that will improve and there are some practical suggestions as well (counterbalancing items, etc).
While I agree I wouldn't expect too much stability in developing brains, unfortunately there are pretty serious stability issues even in non-developing adult brains (quote below from the paper, for anyone who doesn't want to click through).
I agree it makes a lot of sense though the sensory experiments are more consistent, somatosensory and sensorimotor localization results generally seem to the be most consistent fMRI findings. I am not sure registration or motion correction is really going to help much here, I suspect the reality is just that the BOLD response is a lot less longitudinally stable than we thought (brain is changing more often and more quickly than we expected).
Or if we do get better at this, it will be more sophisticated "correction" methods (e.g. deep-learners that can predict typical longitudinal BOLD changes, and those better allow such changes to be "subtracted out", or something like that). But I am skeptical about progress here given the amount of data needed to develop any kind of corrective improvements in cases where there are such low longitudinal reliabilities.
===
> Using ICCs [intraclass correlation coefficients], recent efforts have examined test-retest reliability of task-based fMRI BOLD signal in adults. Bennett and Miller performed a meta-analysis of 13 fMRI studies between 2001 and 2009 that reported ICCs. ICC values ranged from 0.16 to 0.88, with the average reliability being 0.50 across all studies. Others have also suggested a minimal acceptable threshold of task-based fMRI ICC values of 0.4–0.5 to be considered reliable [...] Moreover, Bennett and Miller, as well as a more recent review, highlight that reliability can change on a study-by-study basis depending on several methodical considerations.
I would argue in fact almost all fMRI research is unreliable, and formally so (test-retest reliabilities are in fact quite miserable: see my post below).
EDIT: The reason being, with reliabilities as bad as these, it is obvious almost all fMRI studies are massively underpowered, and you really need to have hundreds or even up to a thousand participants to detect effects with any statistical reliability. Very few fMRI studies ever have even close to these numbers (https://www.nature.com/articles/s42003-018-0073-z).
That depends immensely on the type of effect you're looking for.
Within-subject effects (this happens when one does A, but not when doing B) can be fine with small sample sizes, especially if you can repeat variations on A and B many times. This is pretty common in task-based fMRI. Indeed, I'm not sure why you need >2 participants expect to show that the principle is relatively generalizable.
Between-subject comparisons (type A people have this feature, type B people don't) are the problem because people differ in lots of ways and each contributes one measurement, so you have no real way to control for all that extra variation.
Precisely, and agreed 100%. We need far more within-subject designs.
You would still in general need many subjects to show the same basic within-subject patterns if you want to claim the pattern is "generalizable", in the sense of "may generalize to most people", but, precisely depending on what you are looking at here, and the strength of the effect, of course you may not need nearly as much participants as in strictly between-subject designs.
With the low test-retest reliability of task fMRI, in general, even in adults, this also means that strictly one-off within-subject designs are also not enough, for certain claims. One sort of has to demonstrate that even the within-subject effect is stable too. This may or may not be plausible for certain things, but it really needs to be considered more regularly and explicitly.
Between-subject heterogeneity is a major challenge in neuroimaging. As a developmental researcher, I've found that in structural volumetrics, even after controlling for total brain size, individual variance remains so large that age-brain associations are often difficult to detect and frequently differ between moderately sized cohorts (n=150-300). However, with longitudinal data where each subject serves as their own control, the power to detect change increases substantially—all that between-subject variance disappears with random intercept/slope mixed models. It's striking.
Task-based fMRI has similar individual variability, but with an added complication: adaptive cognition. Once you've performed a task, your brain responds differently the second time. This happens when studies reuse test questions—which is why psychological research develops parallel forms. But adaptation occurs even with parallel forms (commonly used in fMRI for counterbalancing and repeated assessment) because people learn the task type itself. Adaptation even happens within a single scanning session, where BOLD signal amplitude for the same condition typically decreases over time.
These adaptation effects contaminate ICC test-retest reliability estimates when applied naively, as if the brain weren't an organ designed to dynamically respond to its environment. Therefore, some apparent "unreliability" may not reflect the measurement instrument (fMRI) at all, but rather highlights the failures in how we analyze and conceptualize task responses over time.
Yeah, when you start getting into this stuff and see your first dataset with over a hundred MRIs, and actually start manually inspecting things like skull-stripping and stuff, it is shocking how dramatically and obviously different people's brains are from each other. The nice clean little textbook drawings and other things you see in a lot of education materials really hide just how crazy the variation is.
And yeah, part of why we need more within-subject and longitudinal designs is to get at precisely the things you mention. There is no way to know if the low ICCs we see now are in fact adaptation to the task or task generalities, if they reflect learning that isn't necessarily task-relevant adaptation (e.g. the subject is in a different mood on a later test, and this just leads to a different strategy), if the brain just changes far more than we might expect, or all sorts of other possibilities. I suspect if we ever want fMRI to yield practical or even just really useful theoretical insights, we definitely need to suss out within-subject effects that have high test-retest reliability, regardless of all these possible confounds. Likely finding such effects will involve more than just changes to analysis, but also far more rigorous experimental designs (both in terms of multi-modal data and tighter protocols, etc).
FWIW, we've also noticed a lot of magic can happen too when you suddenly have proper longitudinal data that lets you control things at the individual level.
Yes on many of those fronts, although not all those papers support your conclusion. The field did/does too often use tasks with to few trials, with to few participants. That always frustrated me as my advisor rightly insisted we collect hundreds of participants for each study, while others would collect 20 and publish 10x faster than us.
Yes, well "almost all" is vague and needs to be qualified. Sample sizes have improved over the past decade for sure. I'm not sure if they have grown on median meaningfully, because there are still way too many low-N studies, but you do see studies now that are at least plausibly "large enough" more frequently. More open data has also helped here.
EDIT: And kudos to you and your advisor here.
EDIT2: I will also say that a lot of the research on fMRI methods is very solid and often quite reproducible. I.e. papers that pioneer new analytic methods and/or investigate pipelines and such. There is definitely a lot of fMRI research telling us a lot of interesting and likely reliable things about fMRI, but there is very little fMRI research that is telling us anything reliably generalizable about people or cognition.
I remember when resting-state had its oh shit moment when Power et al (e.g. https://pubmed.ncbi.nlm.nih.gov/22019881/) showed that major findings in the literature, many of which JD Power himself helped build, was based off residual motion artifacts. Kudos to JD Power and others like him.
The small sample sizes is rational response from scientists in the face of a) funding levels and b) unreasonable expectations from hiring/promotion committees.
cog neuro labs need to start organizing their research programs more like giant physics projects. Lots of PIs pooling funding and resources together into one big experiment rather than lots of little underpowered independent labs. But it’s difficult to set up a more institutional structure like this unless there’s a big shift in how we measure career advancement/success.
+1 to pooling funding and resources. This is desperately needed in fMRI (although site and other demographic / cultural effects make this much harder than in physics, I suspect).
I'm not an expert, but my hunch would be that a similar Big(ger) Science approach is also needed in areas like nutrition and (non-neurological) experimental psychology where (apparently) often group sizes are just too small. There are obvious drawbacks to having the choice of experiments controlled by consensus and bureaucracy, but if the experiments are otherwise not worthwhile what else is there to do?
I think the problems in nutrition are far, far deeper (we cannot properly control diet in most cases, and certainly not over long timeframes; we cannot track enough people long enough to measure most effects; we cannot trust the measurement i.e. self-report of what is consumed; industry biases are extremely strong; most nutrition effects are likely small and weak and/or interact strongly with genetics, making the sample size requirements larger still).
I'm not sure what you mean by "experimental psychology" though. There are areas like psychophysics that are arguably experimental and have robust findings, and there are some decent-ish studies in clinical psychology too. Here the group sizes are probably actually mostly not too bad.
Areas like social psychology have serious sample size problems, so might benefit, but this field also has serious measurement and reproducibility problems, weak experimental designs, and particularly strong ideological bias among the researchers. I'm not sure larger sample sizes would fix much of the research here.
> Areas like social psychology have serious sample size problems, so might benefit, but this field also has serious measurement and reproducibility problems, weak experimental designs, and particularly strong ideological bias among the researchers. I'm not sure larger sample sizes would fix much of the research here.
I can believe it; but a change doesn't have to be sufficient to be ncessary.
You got downvoted, but I think you are right in a way. Direct neurophysiological recording is not a panacea because either 1) you can't implant electrodes in your participants ethically, 2) Recordings usually are limited in number or brain areas. That said, I think the key is "convergent evidence" that spans multiple levels and tools of analysis. That is how most progress has been made in various areas, like autism research (my current work) or memory function (dissertation). We try to bridge evidence spanning human behavior, EEG, fMRI, structural MRI, post-mortem, electrode, eye-tracking, with primate and rodent models, along with neuron cultures in a dish type of research. We integrate it and cross-pollinate.
There is actually a field where subjects do have electrodes implanted (see https://www.humansingleneuron.org/). - this is done when they are doing pre-operative recordings in preparation for brain surgery for the treatment of epilepsy, and the electrodes are already there for clinical diagnostic purposes and they volunteer a few hours of their time while they are sitting around in the hospital, to participate in various cognitive tasks/paradigms. The areas you can go are limited for sure, but some areas are near regions of interest depicting in fMRI scans.
Then there are also papers w/ recordings done in primates...
But overall yes - you integrate many modalities of research for a more robust theory of cognition
reply