Home » Neuroscience » Multimodal measures of brain connectivity: how much should they agree?

Multimodal measures of brain connectivity: how much should they agree?

The definitive version of this post was originally published on March 3, 2015 on the PLOS Neuroscience Community website, where I serve as an editor.

A study recently published in Frontiers in Neurology started an interesting discussion on Twitter. The paper, by Stephen Jones and colleagues from the Cleveland Clinic, tackled a seemingly simple question: do measures of cerebral connectivity derived from different modalities (functional MRI, intracranial EEG, or diffusion tensor imaging) give similar results? To make a long story short, the answer is not much, as the authors report in Frontiers in Neurology and as Ged Ridgway pointed out on Twitter. Correlation coefficients (r-squared) between the connectivity metrics derived from pairs of modalities ranged from 0.001 to 0.20, which is admittedly not very high. The question is: are those observations surprising?




Whether these results are surprising, and whether they make sense at all, requires looking in more detail at what the authors did. In their study, Jones and colleagues used four distinct ways of measuring brain connectivity. Two are based on magnetic resonance imaging (MRI), and are relatively well known, while the other two center on intracranial electrodes placed into the brain of patients with severely disabling seizures during the work-up for epilepsy surgery.

  1. functional connectivity using resting-state functional MRI
  2. structural connectivity using high angular resolution diffusion-weighted MRI and probabilistic tractography
  3. cortico-cortical evoked potentials (CCEPs) evoked by direct, single-pulse electrical stimulation of the brain
  4. simultaneous functional MRI and direct electrical stimulation of the brain through the intracranial electrodes

Intracranial electrodes in human brains

I will spend some time on the last two modalities, with which most researchers are probably not familiar. As I briefly mentioned above, those modalities rely on intracranial electrodes, which are inserted surgically into an epileptic patient’s brain in order to localize precisely the site of origin of seizures. Once in place, the electrodes are recording the brain’s electrical activity for a few days to a few weeks, until the patient has had a few seizures and the physicians have determined where in the brain the seizures start.

In addition to recording the local EEG, intracranial electrodes can also be used to deliver electrical stimulation to the brain tissue surrounding them. At high frequencies (around 50 Hz) and amplitudes (several milli-amperes), and depending on where the electrode is located, direct electrical stimulation can elicit clinically observable phenomena such as muscle contractions or subjective percepts of the patient’s, such as visual hallucinations.

Cortico-cortical evoked potentials

Stimulation at lower frequencies (around 1 Hz) and amplitudes, on the other hand, is very rarely felt by the patient. The other intracranial electrodes, meanwhile, continue to record the local EEG. By stimulating one electrode with single pulses multiple times, and averaging the responses of the other electrodes, one can obtain an evoked potential in the same fashion as sensory-evoked potentials: the cortico-cortical evoked potential (CCEP). For much more information about CCEPs, check out Mapping human brain networks with cortico-cortical evoked potentials.

Metrics of connectivity between the stimulated and recording electrodes can then be derived from CCEPs. One potential limitation of such an approach is that there are many ways to record and measure CCEPs. The amplitude of the evoked response, for instance, is known to vary as a function of the stimulation amplitude; however, the authors here used a range of amplitudes across stimulation sites and patients. It is even harder to decide what to measure on the CCEP responses themselves, since the neural mechanisms that generate them remain incompletely understood. Here, Jones et al. actually extracted 2 different metrics from the CCEPs depending on what they were going to compare CCEPs against: an early-latency amplitude metric (presumably reflecting direct, cortico-cortical projections) to compare against DWI; and another metric integrating the CCEP trace over a later and longer time period to compare against RS-fMRI. Stricto sensu, they are therefore not comparing the same thing anymore. Clearly, the potential parameter space, i.e. the total number of metrics that could be extracted from CCEPs (and RS-fMRI and every other modality, for that matter), is huge; the choice of parameters in a given study could drastically affect connectivity measures.

Intracranial electrical stimulation during functional MRI

This second metric is even less widespread than CCEPs, since the authors are basically the only ones to use it. It amounts to an fMRI measurement of the BOLD signal while electrical stimulation is delivered through the intracranial electrodes. Here, however, electrical stimulation needs to be delivered at higher intensities through time in order to cause any measurable BOLD change, and the authors used 20-Hz stimulation. That of course makes comparison with “standard” CCEPs difficult, because we do not know whether the same neuronal mechanisms underlie the responses to 1-Hz and 20-Hz stimulation.

Measures of brain connectivity do not line up… should we expect them to?


The authors report rather low pairwise correlations between connectivity measures, with r-squared values ranging from 0.001 (comparing resting-state fMRI to CCEP-evoked fMRI or to diffusion tensor imaging) to 0.20 (between CCEPs and CCEP-evoked fMRI). Why would these values be so low? The authors acknowledge the above-mentioned parameter space problem. They suggest at length that the low correlations could reflect our inability to precisely localize and co-register the different measurements in brain space. They also thoughtfully write that the optimal measure of connectivity could integrate information from more than one modality and go beyond linear correlations between point-to-point, scalar metrics. In their own words: “This development would represent the next step in the evolution of neuroimaging, in which the imaging biomarker moves from being the images themselves, to a mathematical brain model that is informed by images”.

Indeed, as stated by PractiCal fMRI’s tweet, the different modalities explored here are measuring very different facets of the brain’s anatomy and physiology, and how well their results should line up is undetermined. For instance, diffusion tensor imaging reveals major fiber tracts in the white matter, whereas functional MRI highlights statistical dependences between the very slow fluctuations of blood supply to brain regions. To take but one example, functional MRI generally shows very strong connectivity between the left and right hippocampi, whereas the (direct) anatomical connections between these structures are sparse. And that’s only taking into account the temporally sluggish-to-static MRI-based approaches to connectivity; factoring in neurophysiology-based methods, with their millisecond temporal resolution, basically adds a whole other dimension to the dataset. Also, the data from different modalities were collected at different times, thus preventing the possibility of looking at the dynamic evolution of those data through time.

To conclude, the different modalities used in this study likely reflect different aspects of the brain’s structure and function, and we should probably not expect the connectivity metrics to line up perfectly.

Critical omissions

A couple of things are missing from Jones et al.’s study. First, they could have added resting-state intracranial EEG to the list of modalities that they investigated, since they would have had the data already. The authors also fail to refer to previous work that actually investigated the very same questions—and shed an interesting light on those correlations. Specifically, Conner et al. reported on the correlation between CCEPs and MRI tractography in the language system and found an average r-squared value of 0.41. Keller et al. explored the correlations between CCEPs and resting-state fMRI connectivity and obtained overall r-squared values between 0.04 and 0.1. Importantly, when they focused on the language system and only considered CCEPs whose amplitude went beyond a significance criterion derived from the baseline intracranial EEG, Keller et al. found much higher values of r-squared, ranging from about 0.25 to 0.50. Thus, pairwise comparisons of brain connectivity across modalities were much higher than reported by Jones et al. in cases where prior knowledge strongly suggests the existence of connections.

Despite those omissions, the study by Jones et al. is valuable thanks to the richness of the dataset that they generated. With this in mind, the neuroscientific community would benefit tremendously if the authors would make this vast dataset publicly available, so that others can design new ways to extract and combine connectivity metrics and thus shine further light on the structural and functional organization of the human brain.


Jones, S., Beall, E., Najm, I., Sakaie, K., Phillips, M., Zhang, M., & Gonzalez-Martinez, J. (2014). Low Consistency of Four Brain Connectivity Measures Derived from Intracranial Electrode Measurements Frontiers in Neurology, 5 DOI: 10.3389/fneur.2014.00272


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: