Towards decoding of functional movements from the same limb

using EEG
Farid Shiman, Nerea Irastorza-Landa, Andrea Sarasola-Sanz, Martin Spüler, Niels Birbaumer , and
Ander Ramos-Murguialday

Abstract— In recent years, there has been an increasing While in [7], 2-D movement control was achieved with
interest in using electroencephalographic (EEG) activity to EEG the BCI control was achieved using imagery of two
close the loop between brain oscillations and movement to different limbs. Bradberry et al. [6] demonstrated for the first
induce functional motor rehabilitation. Rehabilitation robots or time 3-D hand trajectories decoding during center-out tasks
exoskeletons have been controlled using EEG activity. offline using EEG temporal information. However, one
However, all studies have used a 2-class or one-dimensional should be careful with the interpretation and use of linear
decoding scheme. In this study we investigated EEG decoding
methods for decoding trajectories using EEG [19]. Using
of 5 functional movements of the same limb towards an online
scenario. Six healthy participants performed a three- Independent Component Analysis (ICA) as feature
dimensional center-out reaching task based on direction extraction method, has been used combined with a support
movements (four directions and rest) wearing a 32-channel vector machine (SVM) classifier to decode right versus left
EEG cap. A BCI design based on multiclass extensions of intended movement direction using EEG [20].
Spectrally Weighted Common Spatial Patterns (Spec-CSP) and In another study wavelet transform was applied to time-
a linear discriminant analysis (LDA) classifier was developed frequency representations to decode slow and fast movement
and tested offline. The decoding accuracy was 5-fold cross- using EEG signals [21] and Robinson et al. [22] proposed a
validated. A decoding accuracy of 39.5% on average for all the regularized wavelet-common spatial pattern algorithm for
six subjects was obtained (chance level being 20%). The results
multi-class EEG classification of voluntary hand movement
of the current study demonstrate multiple functional
movements decoding (significantly higher than chance level) directions. Furthermore, normalized variance of CSP-spatial
from the same limb using EEG data. This study represents first filtering of EEG signal has been applied for decoding 1-
steps towards a same limb multi degree of freedom (DOF) dimensional hand directional tasks in [23].
online EEG based BCI for motor restoration. All these studies have indicated that it is possible to
decode hand movements direction from EEG. However, in
I. INTRODUCTION
the above mentioned studies there are some limitations
In the past years, sensorimotor rhythm based Brain- regarding various parameters that should be optimized for
Computer Interface (BCI) technologies [1-4] have been selecting the type of wavelets and the number of features. In
developed to decode brain states into commands to control the interest of using a real-time BCI in stroke patients, these
rehabilitation robots. Furthermore, BCIs have proved their factors are important. For this reason, we propose in the
efficacy in restoring motor function in severely paralyzed current study to apply a modified spectrally weighted CSP
stroke patients [5]. because it can optimize simultaneous spatiotemporal
Different studies have proved the possibility of decoding filtering of motor-related EEG activity. Furthermore, CSP
movement parameters from neurophysiological signals, has been very useful in obtaining spatial filters for detecting
including EEG [6, 7], electrocorticography (ECoG) [8-10], event-related de-synchronization (ERD) and event-related
functional magnetic resonance imaging (fMRI) [11, 12], synchronization (ERS)[7].
magnetoencephalography (MEG) [13], near-infrared The objective of the current study is to decode 5 different
spectroscopy (NIRS) [14], and electromyography (EMG) movements of the same limb using EEG data towards an
[15, 16]. In particular, EEG-based decoding is of great online BCI for motor rehabilitation.
interest for non-invasive decoding in completely paralyzed
II. MATERIALS AND METHODS
patients for reaching directions [17] and continuous
trajectories [7, 18] which have been already decoded. A. Subjects
Six healthy right-handed subjects without any neurologic
disease history (three males and three females, mean age 24
years) participated in four recording sessions. Subjects were
F. S, N. I, A. SS, N. B, and A. RM are with the Institute of Medical
Psychology and Behavioral Neurobiology, University of Tübingen,
informed about the experimental procedure and signed a
Tübingen, Germany. written consent form. This study was approved by the ethical
F. S, N.I, and A.SS are with IMPRS for Cognitive and Systems committee of the Faculty of Medicine, University of
Neuroscience, Tübingen, Germany. Tübingen, Germany.
N. B is also with Ospedale San Camillo, Istituto di Ricovero e Cura a
Carattere Scientifico, Venezia, Italy
A. RM is also with TECNALIA, San Sebastian, Spain
M. S is with Computer Science Department, Wilhelm-Schickard-
Institute, University of Tübingen, Tübingen, Germany.

978-1-4244-9270-1/15/$31.00 ©2015 IEEE 1922

B. Experimental Setup
Subjects performed a center-out reaching task (see figure
1) using their right arm and hand wearing a 7-DOF
Exoskeleton (Tecnalia, San Sebastian, Spain).
The task involved four different movement directions
from a starting position (rest position) towards one of four
different target colors (see figure 1). Upon the presentation
of an imperative auditory cue specifying the target,
participants were asked to perform the movement and return
to the starting position at a comfortable pace but within 4
seconds. The auditory cues and the EEG data were presented
and acquired using BCI2000 software respectively
[www.bci2000.org].
The experiment was divided in 5 runs of 40 trials each for
each session, which gave 50 trials per class. A resting
interval was inserted between the trials for random length
Fig. 1. Subject perfoms a reaching task to wards one of the four targets
between 2-3 seconds. Four sessions on the right hand were (colored panels). The semicircular mechanical stop at the top of the picture
recorded for each subject on different days. serves as starting point position.
C. EEG data acquisition
combination pairs as only binary classifiers were trained.
EEG was recorded using a 32-channel ActiCap and a 32- Let X ∈ 𝑅𝑑×𝑇 be a CSP-filtered EEG signal of a single
channel BrainAmp amplifier (Brain Products GmbH, trial; d is the number of channels and T is the number of
Germany) at a sampling frequency of 2500Hz. Electrode samples in time by which the class label for a single trial X
impedances were kept below 5 kΩ. The cap contained the is predicted. Predicting the Log-variance feature vector is
electrodes FP1, FP2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, given as:
FC6, T7, C3, Cz, C4, T8, TP9, CP5, CP1, CP2, CP6, TP10, 𝑇
P7, P3, Pz, P4, P8, PO9, O1, Oz, O2, PO10 and was fixed by (𝑗)
φj (X; 𝜔𝑗 , 𝛼 (𝑗) ) = log ∑ 𝛼𝑘 𝜔𝑗𝑇 𝑉𝑘 𝜔𝑗 (1)
a chinstrap to avoid electrode shifts, using AFz as ground
𝑘=1
and FCz as the reference.
( j = 1, … , J ),
III. EEG DATA ANALYSIS
A. Preprocessing In this case, 𝑤𝑗 ∈ 𝑅𝑑 is a spatial projection that projects
Data analysis was performed offline. After visual the signal into a single dimension, 𝛼 (𝑗) is the spectrum of the
inspection, noisy channels (TP9, TP10, PO9, and PO10) were temporal filter, and 𝑉𝑘 are the cross-spectrum matrices [26].
removed and the Blind Source Separation (BBS) algorithm The resulting feature vector was then fed to the LDA
[24] from the Automatic Artifact Removal (AAR) toolbox as classifier as multi class classifier. It is basically for two
an EEGLAB plug-in [25] was used to remove artifacts classes extended to more classes by 1-vs-1 voting to
caused by eye-blinks and eye movements, and muscle determine the class label, where the data in each class is
activity from face, neck and shoulder movements. Data was distributed in the feature space according to a normal
downsampled to 250 Hz. distribution. In the 1-vs-1 voting, the classifier is applied to
B. Feature extraction and classification an unseen sample and the class obtaining the highest number
EEG data were band-pass filtered (0.5-70 Hz) and power of votes is selected as classifier output. The classifier is
line noise was removed using a 50 Hz notch filter. EEG trained in two steps. The first step includes the optimization
signals were then divided into trial epochs from 0 s to +2.5s of the coefficients 𝑤𝑗 and 𝛼 (𝑗) and second the training of the
with respect to movement onset triggers (t = 0s). A band Linear Discriminant Analysis (LDA) classifier.
pass filter between 7 – 30 Hz was then applied to extract mu For evaluation of the decoding model, a 5-fold blockwise
and beta frequency band data and a Spectrally weighted cross-validation process with 5 trials safety margin was
Common Spatial Pattern (Spec-CSP) [26] was applied to the performed. The data of one session was divided in 5 blocks
data. Spec-CSP is an algorithm based on simultaneous and in every fold the model was trained on four blocks and
optimization of spatiotemporal filters that helps to solve the tested on the remaining one. Decoding accuracy was
limitation of single trial EEG classification. Spec-CSP estimated according to the average over all folds for each
computes discriminative features, whose variances are session.
optimal between two classes with respect to their patterns. IV. RESULTS AND DISCUSSION
The algorithm is based on the simultaneous
diagonalization of two covariance matrices and weights of In this work we developed and tested a multi-class
the EEG channels given rows of the weight matrix. In this classification model aiming at decoding movements of the
study, the algorithm normally used for a 2-class CSP was same upper limb in four directions and ‘’rest’’ using EEG
also applied to five classes of EEG signals on all possible data.

1923
 Fig. 2. Confusion Matrix of all subjects for 5 classes

The average accuracy for all subjects when using one

session (50 trials per class) to decode 5 movement classes Fig. 3. Spec-CSP topographical patterns and the spectrum of the filter of
from the same limb was 39.5% (See Table I). subject 1 for four class pairs (Blue vs Red, Blue vs Green, Red vs Green
As the theoretical chance level (100/5 = 20%) is defined and Green vs Brown).
for an infinite number of data, we used the binomial
cumulative distribution [27] to calculate the statistical V. CONCLUSION
significance thresholds for the decoding accuracy which This study evaluated offline the decoding accuracy in a 5-
resulted to be at 24.4% (n=250, 5-class and p<0.05). class hand movement task (actual direction of hand
The grand average classification accuracy results for all movement) using an optimized spatio-spectral filter as
classes and participants are summarized in form of a feature extraction method. The obtained results show that
confusion matrix in Figure 2, being “rest” the class with the directional hand movement decoding is possible using
best decoding accuracy as expected (64.58%), followed by untrained healthy subjects EEG data. The decoding of
the other classes (all above chance level). Decoding several movements on the same limb reached significance
performance for the blue and red target were better, as levels. This work represents first steps towards the
expected, because they only have one other target next by. development of a high-dimensional EEG based BCI, that
Our decoder confused neighbor targets for the limitation of decodes movements from the same limb.
space resolution. We will be applied a combination of less In our future work, five different functional hand
confused functional movements and a probabilistic output to movements (grasp, pinching, pointing, pronation,
improve the upper limb movements classification. supination) will be integrated and new feature extraction and
The topographical pattern of Spec-CSP of subject 1 for classification methods will be used and compared.
four pairs shows that Blue versus Red has clear Although very preliminary, these results are promising
discrimination rather than Green versus Brown in Figure 3. and could help to design more functional BCIs for motor
Subject specific decoding results showed a slight variation rehabilitation.
in performance during different sessions (See Table 1). The
subject performance is directly related to the Spec-CSP filter ACKNOWLEDGMENT
used, which resulted in distinguishable features for different
directions. This study was funded by the Baden-Württemberg
Stiftung (GRUENS), the Indian-European collaborative
TABLE I. MEAN CLASSIFICATION ACCURACY (%) OF SIX SUBJECTS FOR EACH
research and technological development projects (INDIGO-
SESSIONS. DTB2-051), the WissenschaftsCampus Tübingen, the
Subjects Sess01 Sess02 Sess03 Sess04 Average
Deutsche Forschungsgemeinschaft (DFG, Grant RO
1030/15-1, KOMEG), the Volkswagen Stiftung, the Natural
S1 26% 40% 41% 38% 36.25% Science Fundation of China (NSFC 31450110072), EU
S2 48% 56% 31% 42% 44.25% COST action TD1006, Deutsche Forschungsgemeinschaft
(DFG, Koselleck), and Bundes Ministerium für Bildung und
S3 47% 42% 39% 48% 44%
Forschung BMBF MOTORBIC (FKZ 13GW0053). A.
S4 32% 33% 34% 33% 33% Sarasola-Sanz’s is supported by La Caixa-DAAD
S5 39% 39% 36% 37% 37.75%
scholarship, and N. Irastorza-Landa’s work by the Basque
Government and IKERBASQUE, Basque Foundation for
S6 41% 37% 43% 39% 40% Science.
39.5%

1924
 REFERENCES [14] S. M. Coyle, T. E. Ward, and C. M. Markham, "Brain–
computer interface using a simplified functional near-
[1] A. Ramos-Murguialday, M. Schürholz, V. Caggiano, M. infrared spectroscopy system," Journal of neural
Wildgruber, A. Caria, E. M. Hammer, S. Halder, and N. engineering, vol. 4, p. 219, 2007.
Birbaumer, "Proprioceptive feedback and brain computer [15] G.-C. Chang, W.-J. Kang, J.-J. Luh, C.-K. Cheng, J.-S.
interface (BCI) based neuroprostheses," PLoS One, vol. 7, Lai, J.-J. J. Chen, and T.-S. Kuo, "Real-time
p. e47048, 2012. implementation of electromyogram pattern recognition as
[2] K. K. Ang, C. Guan, K. Sui Geok Chua, B. T. Ang, C. a control command of man-machine interface," Medical
Kuah, C. Wang, K. S. Phua, Z. Y. Chin, and H. Zhang, engineering & physics, vol. 18, pp. 529-537, 1996.
"Clinical study of neurorehabilitation in stroke using [16] R. Boostani and M. H. Moradi, "Evaluation of the forearm
EEG-based motor imagery brain-computer interface with EMG signal features for the control of a prosthetic hand,"
robotic feedback," in Engineering in Medicine and Physiological measurement, vol. 24, p. 309, 2003.
Biology Society (EMBC), 2010 Annual International [17] S. Waldert, H. Preissl, E. Demandt, C. Braun, N.
Conference of the IEEE, 2010, pp. 5549-5552. Birbaumer, A. Aertsen, and C. Mehring, "Hand movement
[3] T. Ono, K. Shindo, K. Kawashima, N. Ota, M. Ito, T. Ota, direction decoded from MEG and EEG," The journal of
M. Mukaino, T. Fujiwara, A. Kimura, M. Liu, and J. Neuroscience, vol. 28, pp. 1000-1008, 2008.
Ushiba, "Brain-computer interface with somatosensory [18] N. Birbaumer, "Breaking the silence: brain–computer
feedback improves functional recovery from severe interfaces (BCI) for communication and motor control,"
hemiplegia due to chronic stroke," Frontiers in Psychophysiology, vol. 43, pp. 517-532, 2006.
neuroengineering, vol. 7, 2014. [19] J. M. Antelis, L. Montesano, A. Ramos-Murguialday, N.
[4] A. Ramos, S. Halder, and N. Birbaumer, "Proprioceptive Birbaumer, and J. Minguez, "On the usage of linear
feedback in bci," in Neural Engineering, 2009. NER'09. regression models to reconstruct limb kinematics from
4th International IEEE/EMBS Conference on, 2009, pp. low frequency EEG signals," PLoS One, vol. 8, p. e61976,
279-282. 2013.
[20] Y. Wang and S. Makeig, "Predicting intended movement
[5] A. Ramos‐Murguialday, D. Broetz, M. Rea, L. Läer, Ö.
direction using EEG from human posterior parietal
Yilmaz, F. L. Brasil, G. Liberati, M. R. Curado, E.
cortex," in Foundations of Augmented Cognition.
Garcia‐Cossio, and A. Vyziotis, "Brain–machine Neuroergonomics and Operational Neuroscience, ed:
interface in chronic stroke rehabilitation: a controlled Springer, 2009, pp. 437-446.
study," Annals of neurology, vol. 74, pp. 100-108, 2013. [21] G. Ofner and G. Müller-Putz, "Decoding of hand
[6] T. J. Bradberry, R. J. Gentili, and J. L. Contreras-Vidal, movement velocities in three dimensions from the eeg
"Reconstructing three-dimensional hand movements from during continuous movement of the arm," in TOBI
noninvasive electroencephalographic signals," The journal Workshop III, 2012.
of Neuroscience, vol. 30, pp. 3432-3437, 2010. [22] N. Robinson, C. Guan, A. Vinod, K. K. Ang, and K. P.
[7] J. R. Wolpaw and D. J. McFarland, "Control of a two- Tee, "Multi-class EEG classification of voluntary hand
dimensional movement signal by a noninvasive brain- movement directions," Journal of neural engineering, vol.
computer interface in humans," Proceedings of the 10, p. 056018, 2013.
National Academy of Sciences of the United States of [23] G. Clauzel, C. Neuper, and G. Müller-Putz, "Offline
America, vol. 101, pp. 17849-17854, 2004. decoding of hand movement directions from non-invasive
[8] M. Spüler, A. Walter, A. Ramos-Murguialday, G. Naros, EEG," trials, vol. 8, p. 6, 2011.
N. Birbaumer, A. Gharabaghi, W. Rosenstiel, and M. [24] A. Yeredor, "Blind separation of Gaussian sources via
Bogdan, "Decoding of motor intentions from epidural second-order statistics with asymptotically optimal
ECoG recordings in severely paralyzed chronic stroke weighting," Signal Processing Letters, IEEE, vol. 7, pp.
patients," Journal of neural engineering, vol. 11, p. 197-200, 2000.
066008, 2014. [25] G. Gómez-Herrero, "Automatic Artifact Removal (AAR)
[9] K. J. Miller, G. Schalk, E. E. Fetz, M. den Nijs, J. G. toolbox v1. 3 (Release 09.12. 2007) for MATLAB,"
Ojemann, and R. P. Rao, "Cortical activity during motor Tampere University of Technology, 2007.
execution, motor imagery, and imagery-based online [26] R. Tomioka, G. Dornhege, G. Nolte, B. Blankertz, K.
feedback," Proceedings of the National Academy of Aihara, and K.-R. Müller, "Spectrally weighted common
Sciences, vol. 107, pp. 4430-4435, 2010. spatial pattern algorithm for single trial EEG
[10] T. Pistohl, A. Schulze-Bonhage, A. Aertsen, C. Mehring, classification," Dept. Math. Eng., Univ. Tokyo, Tokyo,
and T. Ball, "Decoding natural grasp types from human Japan, Tech. Rep, vol. 40, 2006.
ECoG," Neuroimage, vol. 59, pp. 248-260, 2012. [27] E. Combrisson and K. Jerbi, "Exceeding chance level by
[11] R. Sitaram, A. Caria, R. Veit, T. Gaber, G. Rota, A. chance: The caveat of theoretical chance levels in brain
Kuebler, and N. Birbaumer, "FMRI brain-computer signal classification and statistical assessment of decoding
interface: a tool for neuroscientific research and accuracy," Journal of neuroscience methods, 2015.
treatment," Computational intelligence and neuroscience,
vol. 2007, 2007.
[12] S.-S. Yoo, T. Fairneny, N.-K. Chen, S.-E. Choo, L. P.
Panych, H. Park, S.-Y. Lee, and F. A. Jolesz, "Brain–
computer interface using fMRI: spatial navigation by
thoughts," Neuroreport, vol. 15, pp. 1591-1595, 2004.
[13] T. J. Bradberry, F. Rong, and J. L. Contreras-Vidal,
"Decoding center-out hand velocity from MEG signals
during visuomotor adaptation," Neuroimage, vol. 47, pp.
1691-1700, 2009.

1925