Functional Neuroimaging Technologies
Much of the recent progresses in neuroscience, concerning language, attention, memory, and many others that were intractable in the past, are derived from technological advancements in functional neuroimaging. These technologies are rapidly entering clinical practice and have dramatically advanced our capability in medical diagnosis and presurgical planning of various neurological diseases. Functional magnetic resonance imaging (fMRI) has excellent millimeters spatial resolution, while both electroencephalogram (EEG) and magnetoencephalogram (MEG) provides millisecond temporal resolution in researching human brain activity. Unfortunately, EEG/MEG suffer from low spatial resolutions and ambiguity in defining spatial origins of brain activity because of the volume conductor effect, while fMRI is fundamentally limited in studying the temporal aspect of brain activity at the neuronal time scale of milliseconds, which is the essence of brain function, due to its relatively slow acquisition speed and slow hemodynamic responses. We have developed a suite of innovative functional neuroimaging technologies to address these limitations in both EEG/MEG and fMRI, as well as to combine them together.
Spatiotemporal Correlations of EEG Microstates and BOLD Resting State Networks
Neuroimaging research suggests that the cerebral function during resting state is driven by large scaled functional networks. The BOLD fMRI studies at rest reveal temporally correlated spatial patterns, known as resting state networks (RSNs). However, the neurophysiological basis for these BOLD RSNs has not been fully understood. Recently, the electroencephalography microstates (EEG-ms) have been suggested to be correlated to the temporal dynamics of several BOLD RSNs. However, their spatial correlations have not been investigated. In this study, we collaborated with Laureate Institute for Brain Research (LIBR) to develop a novel approach to compare the spatial and temporal similarities between EEG-ms and BOLD RSNs by combine use of electrophysiological source imaging (ESI) and independent component analysis (ICA). Simultaneous EEG and fMRI data were acquired from nine healthy subjects at resting behavioral state. Min Zhu helped to build the volume conductor model the source model by segmenting the structural MRI data for each subject. The cortical sources of EEG-ms were obtained by solving the inverse problem estimate and then decomposed into independent components (ICs) as main microstates. The source maps associated to ICs were compared to the spatial patterns of RSNs independently derived from BOLD fMRI and IC time courses were convolved with a canonical hemodynamic response function and down-sampled to TR to compare with temporal dynamics of RSNs. Five representative networks were selected from BOLD RSNs and their correlated source maps of EEG-ms are displayed in Fig. 1. High spatial similarities were found between EEG-ms and BOLD fMRI in the default mode, sensorimotor, attention and auditory RSNs. It is worth to note that the visual network was found split into three independent EEG-ms RSNs with similarity to the visual BOLD fMRI networks. The time courses of these EEG-ms driven RSNs were all significant correlated with the time courses of BOLD fMRI RSNs (p<0.001).
figure 1
Fig. 1 Maps of RSNs obtained using EEG microstates source analysis.
Computational Sparse Neuroimaging Techniques
The goal of electromagnetic source imaging technique using EEG and/or MEG is to non-invasively reconstruct cortical electrical activity from external surface potentials and/or magnetic fields, known as the EEG/MEG inverse problem. EEG/MEG inverse problem has no unique solutions. Conventional minimum norm estimates based on L2-norm or L1-norm of the original signal domain produced bias in estimating source locations and extents. Recently, we have developed sparse source imaging (SSI) techniques, i.e. variation based sparse source imaging (VB-SSI) (Ding, Phys in Med Biol, 2009) and wavelet based sparse source imaging (WB-SSI) (Liao et al., Phys in Med Biol, 2012), which implement Compressive Sensing (CS) theory to EEG/MEG inverse problems. The proposed VB-SSI method is designed to identify source boundaries (Fig. 1(a)) by minimizing the L1-norm of variations, and the identified boundaries can naturally be used for estimation of cortical source extents, which is critical in many neuroscience studies and clinic applications. It is well-known that wavelet transform is a powerful tool to compress signals. We thus developed the face-based wavelets on multi-resolution cortical model (Fig. 2(a)) to compress cortical current density. The L1-norm minimization on compressive wavelet coefficient is expected to produce a more plausible reconstruction of cortical sources. The performance of proposed SSI methods has been evaluated in well-designed Monte Carlo simulations and neurophysiological experiments. VB-SSI shows robust reconstructions of cortical current densities with different number of sources and sources of different extents (Fig. 1(b)). Applications of VB-SSI to finger tapping experimental data demonstrated its capability in recovering multiple extended sources as compared to conventional methods, i.e. wMNE and LORETA (Fig 1(c)) (Ding et al., J Neural Eng, 2011), which provides a promising tool to study complicated neural networks. The simulation results show advancements of WB-SSI than wMNE and cLORETA in accurately localizing multiple sources (Fig. 2(b)). The successfully capture of cortical response to auditory stimuli on superior temporal cortex and medial temporal activations involved in the recognition of phonological and semantic components of words further demonstrated its feasibility for studying functional brain activations (Fig. 2(c)).
figure 1 Fig. 1. VB-SSI (or VB-SCCD):  (a) Illustration of a cortical source consisting of multiple elements and its boundary between active and inactive regions. (b) Monte Carlo simulation:  AUC metrics for VB-SSI at different SNR levels and with different source configurations. (c) Finger movement experiment: Comparison of scalp potential maps, cortical maps from VB-SSI, cLORETA, and wMNE for fingers from both left and right hands in two participants.

figure 2Fig. 2. WB-SSI: (a) Illustration of multi-resolution CCD model consisting of original cortical mesh and its compressions at four levels. (b) Monte Carlo simulation: Comparison of WB-SSI with four different levels of compression with wMNE and cLORETA using metrics of distance of localization error (DLE) and spatial dispersion (SD). (c) Auditory recognition experiment: Reconstructed cortical activations from WB-SSI at 98 ms and 232 ms after the presentation of auditory stimuli.
Clinical Neuroimaging in Epilepsy
In presurgical evaluation of patients with medically refractory focal epilepsy, it is critical to identify the location and extent of epileptogenic zone (EZ) for surgical resection. Neuroimaging techniques provide a non-invasive tool to localize the epileptic current sources through external EEG/MEG measurements, which will benefit numerous candidates for epileptic surgery. The present study is to test the feasibility of a novel neuroimaging technique, i.e., variation based sparse source imaging (VB-SSI, also known as VB-SCCD) (Ding, Phys in Med Biol, 2009), in noninvasively estimating the sites and spatial extent of epileptic current sources using MEG interictal spikes (IIS). One illustration of measured magnetic fields underlying IIS and reconstructed cortical current sources using VB-SSI are provided in Fig. 3(a). Estimated cortical sources, which are localized to the areas of MRI lesions, indicate the consistency of VB-SSI results to clinical diagnosis. Take the advantage of excellent temporal resolution of EEG/MEG at millisecond level, temporal dynamics of epileptic activities are able to be inspected on cortical source level, which is tremendous valuable to understand the mechanisms for generation and propagation of epilepsy. The reconstructed temporal dynamic of one dipole source selected from active regions shows similar transitions as the IIS. A series of snapshots of cortical sources underlying one IIS in a special case of Landau-Kleffner Syndrome (LKS) patient indicates a dynamic propagation that origins from auditory cortex to language cortex, which actually explains why LKS patients lose their language ability followed by the auditory ability loss (Zhu et al., J. Clin. Neurophysiol., in press).
figure 1
Fig. 3. (a) Illustration of the magnetic scalp map and reconstructed cortical sources by VB-SSI at a single time point (indicated by red dashed line) from one MEG IIS. The waveform of one dipole source selected from the active region showed similar dynamics as the IIS. After co-registered with MRI lesions, the reconstructed sources are located within areas of MRI lesions. (b) Dynamic propagations of epileptic activities indicated by VB-SSI source reconstructions from a case study of Landau-Kleffner Syndrome patient.
Multi-modality Neuroimaging (EEG+MEG)
Electroencephalography (EEG) and/or magnetoencephalography (MEG) have different sensitivity to cortical sources at different depths, locations, and orientations, providing complementary information for better detections and reconstructions of brain sources. We propose an integrated approach to combine the use of EEG and MEG data in a new sparse electromagnetic source imaging (ESI) technique, i.e., variation-based sparse source imaging (VB-SSI) (Ding, Phys in Med Biol, 2009; Ding and Yuan, Hum Brain Mapp, 2011). The combination of EEG and MEG are fulfilled by channel-wised SNR transformations to a common basis, and then VB-SSI is applied to reconstruct sources with larger number of measurements from different modalities. Monte Carlo simulations are conducted to investigate the performance of the proposed approach in different sensor configurations with multiple cortical brain activations (up to 10 random located activations) (Fig. 4(a)). Experimental EEG and MEG data from a face recognition task are further used to evaluate the performance of VB-SSI (Fig. 4(b)). The present results indicate remarkable improvement of integrating EEG and MEG as compared with simply increasing number of sensor in one modality (Fig. 4(a)). The source imaging results from real data further demonstrate it is capable to recover networked brain activations involving multiple cortical regions, which are consistent with results from functional magnetic resonance imaging in the same task paradigm (Fig. 4(b)).
Fig 4. (a) Whisker plots of AUC metric value for VB-SSI using EEG (black), MEG (orange), and EEG+MEG (blue) with different numbers of channels. (b) Dynamic patterns of source reconstructions with P100/M100 and P170/M170 components from a face recognition task. Dominant activities are observed in the bilaterally fusiform and laterally ventral occipital regions at 145 ms and later significant activations are found on the frontal lobe including superior frontal gyrus, orbital part of inferior frontal gyrus and medial orbitofrontal gyrus at 165 ms.
Both invasive and noninvasive neuromodulation techniques have indicated the capability of altering brain functions and processes through the use of, mainly, electrical and magnetic stimulations,which have suggested clinical diagnostic indications and/or clinical therapeutic effects. We are actively conducting research to understand modulation effects of electrical and magnetic stimulations on neural activity and networks as measured by neuroimaging technologies (i.e., EEG, fMRI, and simultaneous EEG & fMRI) and, using these discoveries, to optimize stimulation targets and parameters in maximizing treatment effects in various neurological and neuropsychiatric disorders.
rTMS in the mal de debarquement syndrome (MdDS)
The aim of this project is to evaluate the therapeutic effects of rTMS in a chromic disorder of imbalance (i.e., MdDS), by probing the resting-state neural activities and networks using EEG and/or fMRI. Specifically, we have developed a group-independent component analysis based analysis framework and successfully detected the modulation effects in neural activities and networks using resting state EEG after a five-day treatment of rTMS in 10 MdDS patients (Ding et al., under revision at IEEE TBME). As an example, significant positive correlations (p<0.05) between the connectivity changes (measured by inter-IC phase coherence, ICPC) and the clinical visual analogue scale (VAS) score changes from pre- to post-rTMS sessions were revealed mainly in parietal and sensory-related areas in the theta, low alpha and high alpha bands (Fig. 1), suggesting that the connectivity among these areas decreased as the VAS score decreased, i.e., symptoms improved. In all, both the reported VAS scores and the detected neural activities and networks changes have demonstrated the possible therapeutic effects of rTMS in MdDS.
Fig.1 Significant cross-subject correlations (p < 0.05) between ICPC changes and VAS score changes from pre- to post-TMS sessions in (a) theta, (b) low alpha, and (c) high alpha bands.
Neuroenhancement defines various techniques that directly affect on human brains and nervous system to enhance the performance of specific tasks. It can be applied to healthy individuals to promote cognitive functions, such as memory and perception.  For patients with brain impairment and associated cognitive deficiency, neuroenhancement can serve as treatment to restore brain functions.  Common techniques for neuroenhancement include, but not limited to, transcranial electromagnetic stimulation (such as transcranial direct current stimulation, i.e., tDCS, and transcranial magnetic stimulation, i.e. TMS), neurofeedback, deep brain stimulation, and all sorts of behavioral training methods.
Promoting motor development for children at risk of cerebral palsy with robot assistants
We are developing co-robotic systems that assist children at the risk of developing cerebral palsy (CP) to develop locomotory skills. The significance of the research is that the problems with mobility development in CP do not only disrupt functional independence across the lifespan, but are also associated with cognition and intellectual impairments. We propose to use robotic systems to assist self-generated goal-directed movements during the early stage of movement skill development to enhance synaptic connections in the brain. Since goal-directed movements cannot be recognized in kinematic data, we propose to use EEG data as real-time feedbacks to evaluate skill learning and experience, as shown in Fig. 1. The somatosensory rhythms in EEG measurements also provide information to monitor the motor brain development of CP children, which also serve as a tool to evaluate the effect of the robot assistant intervention.
Fig.1 (Left) Overview of robotic assistive system (Right) Baby wearing EEG sensor net on the robotic assistive system
Click here to watch full media coverage from Oklahoma News 9

NewsOn6.com - Tulsa, OK - News, Weather, Video and Sports - KOTV.com |
Neuroadaptive Interface
With the increase in technology nowadays, modern complex systems can impose high cognitive demands on human operators. This is the area concerning mental health for large workforces and safety in public workspaces. However, it is also the area that is significantly less addressed by biomedical researchers. While it has been concerned for many years in human factor studies, limited physiological data are available to provide objective, direct, and continuous measures to monitor, such as, mental fatigue, mental workload, and mental capacity. We have recognized such a need and developed several projects to build neural interfaces for studying mental workload, mental efforts, and mental fatigue using computational neuroimaging technologies based on EEG signals. These projects are also featured with translational researches.
Functional Neuroimaging Techniques for Continuous EEG Data
It is of fundamental significance to study human brain functions using neuroimaging technologies in real-world tasks. We have developed functional neuroimaging techniques to probe neural activations from continuous EEG in air traffic control (ATC) tasks simulated by C-TEAM software (see (A)). To robustly identify physiologically plausible EEG patterns related to brain activations involved in the task, a novel data-driven method, i.e., time-frequency independent component analysis (tfICA), is developed to analyze the continuous high-density (i.e., 128 channels) EEG data, which combines the time frequency analysis and complex-valued ICA method. With the analysis of EEG signals from 11 subjects using the tfICA method, six classes of independent components (ICs) of various spatio-temporal-spectral patterns (see (B) and (C)) were identified across subjects, relating to frontal, motor, premotor, supplementary motor, secondary somatosensory, and occipital cortices, which suggest a networked brain activation involving visual perception and processing, movement planning and execution, working memory, performance monitoring, and decision making to accomplish the task (Shou et al., J Neurosci Methods, 2012). The time-on-task effect related to mental fatigue was observed in most ICs. We are currently working towards the further develop of tfICA method and the application of tfICA to detect the neural substrates of different factors linked to mental fatigue evolution, such as time-on-task effect, mental workload and mental capacity, in ATC tasks. The long-term goal is to develop neuroadaptive interfaces, i.e., the online monitoring of operators’ mental appropriate for the delivery of adaptive aiding in real-world working situations such as aviation, by establishing functional neuroimaging technologies using EEG signals.
figure 1
Figure 1: Neural patterns recognition in real-world tasks using tfICA. (A) The operational interface for ATC tasks in C-TEAM; (B) Grand averages of scalp maps and spectra of 13 group-level ICs from 11 subjects, categorized into six IC classes. Red lines in spectra denote dominant frequency bands in each IC class. (C) Individual scalp maps for two IC classes from two sessions in all subjects to check the intra- and inter-subjects’ variations. For details, please refer to the corresponding publications.
Study of Mental Fatigue in Low-fidelity Simulations
It is of interest to reliably monitor the development of mental fatigue in realistic working environments that demand high cognitive workload like in air traffic controllers, so as to prevent occupational hazards. Powered by the functional neuroimaging techniques to handle continuous EEG data, we conducted a study of the time-on-task mental fatigue in a low-fidelity simulation environment, using C-TEAM. EEG and performance data are assessed simultaneously to identify neurophysiological patterns due to the time-on-task effect. Significant changes in EEG power spectra are localized to the midline regions from the frontal area and parietal area of the human brain. Significant changes in performance data are also observed in response time. Correlated changes of cognitive performance and EEG pattern are indicative of the development of mental fatigue and the mental state changes. Such information regarding mental state changes can be used to guide further work scheduling to ensure public transportation safety and mental health in air traffic controllers.
figure 2
Figure 2: These spatial plots represent relative spatial changes in all ten participants,
comparing plots from 1st 10 minutes on the task against 2nd and last (11th) segment
of 10 minutes on the task.
Study of Mental Fatigue in High-fidelity Simulations
Encouraged by affirmative results in low-fidelity study, we are presently conducting data collection efforts using high-fidelity simulator: ATCARS (Air Traffic Control Advance Research Simulator) at FAA-CAMI, Oklahoma City. Moreover, the participants in this study are experienced (~15 years) ATC instructors as compared to the student group used in low-fidelity study. This study can help us better evaluate the mental state of ATCs. Based on our neural IC precursors, we are currently developing indexes based on rhythmic spectra within each IC to assess mental state (i.e., fatigue, effort, workload etc.) of operators. Such information can augment in developing real-times systems that can reliably monitor mental state of ATCs.
figure 3
Figure 3: High-fidelity Simulations
Brain Computer Interface
Brain computer interface (BCI) is a technology to design neuroprosthetics for paralyzed people and robots for aiding people with movement difficulties, help them to control external devices, and thus restore their motor capabilities. While a large body of literature exists for BCI using implanted electrodes and/or devices, these technologies are limited to be used in human being due to not only their invasiveness, but also biomaterial compatibility issues. Meanwhile, noninvasive BCI systems, mainly using EEG signals, suffer from limited degree of freedoms for control. We aim to substantially advance BCI technologies toward noninvasiveness and natural control with more degrees of freedom (DOFs).
Decoding Individual Finger Movements
During the past decade, EEG-based BCI systems have been demonstrated to decode movements of large body parts, such as upper limbs, elbows, shoulders, and legs. However, the movements of fine body structures, such as individual fingers from one hand, have not been well studied in EEG-based BCI while they are the most dexterous part of our body and play an irreplaceable role in our daily activities. If a noninvasive BCI can decode the movements of individual fingers, it has the potential for paralyzed people to control more complicated neuroprosthetics with high degrees of freedom. We designed a series of experiments involving individual finger movements, exploring distinguishable features about them in EEG. Fig. 1 shows the experimental design. When decomposing EEG power spectra by principal component analysis (PCA), we identified a broadband power increase in EEG. By applying this feature into classification of individual finger movements from five fingers of the same hand, an average decoding accuracy significantly higher than guess rate (20% for 5-class problem) was achieved. As illustrated in the confusion matrix in Fig. 2, most finger movements were correctly classified to the respective fingers. It demonstrated the feasibility of using non-invasive EEG to decode dexterous individual finger movements for BCI.
figure 1 Figure 1. Experimental design and sensor layout. (a) Timeline of experimental trial. (b) Illustration of individual finger movements. (c) 128-channel sensor layout (black dot). Cross markers and circles represent 50-channel set and 22-channel set used for decoding, respectively.
figure2Figure 2. Confusion matrix of decoding accuracies of five fingers.
Decoding Multiple Motor Imageries of the Same Hand
Motor imagery (MI) has been widely used in the field of BCI research to provide movement-free control for people suffering from severe motor disability. While various MI-based BCIs have been developed, one remaining challenge is the limited number of features available to produce enough control signals, which largely defines the complexity of applications. We investigated the feasibility of discriminating different types of MIs on both hands using the non-invasive scalp EEG through exploring underlying features produced by MIs of thumb and fist from each hand. Variance structures in EEG spectra were analyzed by principal component analysis (PCA) at each channel and a unique spectral structure was identified on which different projections of EEG data from different MIs were suggested, as is shown in Fig. 3. When applying the extracted features in classification by simple LDA classifier, an average decoding accuracy of 48%, significantly higher than guess level (25%), was achieved, as illustrated in Fig. 4 (Xiao et al., Conf Proc IEEE Eng Med Biol Soc, 2012).
figure 3 Figure 3. (a) Average elements of the first five principal components (PC). (b) Average projection weights of left thumb (LT), left fist (LF), right thumb (LT), right fist (RF) and resting condition (Rest) on the first five PCs.
figure 4Figure 4. Average decoding accuracy for each MI type and average decoding accuracy in all trials regardless of conditions. Black solid line indicates 4-class guess level. Dashed line indicates upper boundary of 95% CI of the guess level.
Cognitive Engineering
Committing errors is the nature of human beings. The prediction and further prevention of errors is useful in real world to avoid catastrophic consequence. The dynamic adjustments after errors need behavioral flexibility and deficient error processing contributes to maladaptively rigid and repetitive behaviors in neuropsychiatric disorders. We aim to advanced functional neuroimaging techniques for the illustration of neural correlates and mechanisms behind error commissions and their applications in identify biomarkers of neuropsychiatric disorders.
Neural Correlates of Error Commissions
The work is pursuing a broad goal to probe multiple neural correlates contributing to behavior errors by applying the state of the art EEG signal processing methods. We have successfully uncovered two neural markers in a Stroop task (see Fig. 1(A)) via event-related potential (ERP) and time-frequency rhythmic activities analyses in EEG signals recorded from 18 subjects (Shou et al., under revision at Neuroimage). Specifically, elevated pre-stimulus alpha power in the central sensorimotor and occipital cortex (see Fig. 1(B) and (C)) and attenuated N2 amplitude in the frontal cortex at FCz channel (see Fig. (D)) prior to responses both linked to subsequent error commissions. The underlying neural mechanisms are ongoing sustained attention and cognitive performance monitoring, respectively, for two phenomena. The complicated relationship between these two neural mechanisms to the success of a response is also explored (see Fig. 1(E)), which indicates a correlation among them.
figure 1
Figure 1: Detection of neural markers in error commission: (A) The Stroop task paradigm, in which subjects are required to make manual response to judge the congruency between word ink color and word meaning; (B) Topography of the statistic t values of pre-stimulus alpha power (-500 to 0 ms) comparing correct with erroneous responses in congruent stimulus. The cluster of channels representing significant stronger alpha power for erroneous than correct responses is marked with dots (p<0.025; cluster-based permutation test); (C) Single trial binning analysis in congruent stimulus reveals a significant trend in error rate with increased pre-stimulus alpha power; (D) N2 amplitudes at channel FCz for four categories indicate a significant decrease in the contrast of error and correct response in incongruent stimulus (ErIg vs. CrIg), and the contrast of congruent and incongruent stimuli in correct response (CrCg vs. CrIg); (E) N2 amplitudes plotted as a function of pre-stimulus alpha power quartiles for correct trials in congruent and incongruent reveals a significant effect. *p<0.05.**p<0.01 For details, please refer to the corresponding publications.
Back to top