skip to main content
10.1145/3608164.3608188acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicbbtConference Proceedingsconference-collections
research-article

Photoplethysmogram Based Cognitive Load Recognition Using Lstm

Published: 07 November 2023 Publication History

Abstract

Physiological data are nowadays frequently used to recognize the affective state of subjects while performing different tasks. Measuring cognitive load using simple device plays an important role in everyday life such as intelligent human-computer interaction, physical health monitoring, and mental health monitoring. However, due to the nature of the experiments involving subjects, the obtained data base is often low, making it difficult to train deep learning methods from scratch. In this paper, We conducted a method to recognize the cognitive load based on the Photoplethysmogram(PPG) data and the application of Long Short Term Memory (LSTM). We tested this method on the PPG data from an experiment where 19 subjects were involved in arithmetic calculation tasks of two different cognitive load levels. The method successfully achieve btter accuracy of recognition compared with the traditional machine learning classifiers with features artificially extracted from the image and pre-trained CNN method, 92.3% binary classification accuracy was reached and about 3.8% binary classification accuracy was improved.

References

[1]
Aamir Arsalan, Syed Muhammad Anwar, and Muhammad Majid. 2022. Mental stress detection using data from wearable and non-wearable sensors: a review. arXiv preprint arXiv:2202.03033 (2022).
[2]
D Berrar. 2018. Cross-validation. Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics, 1–3, 542–545.
[3]
Karel A Brookhuis and Dick De Waard. 2010. Monitoring drivers’ mental workload in driving simulators using physiological measures. Accident Analysis & Prevention 42, 3 (2010), 898–903.
[4]
Roland Brünken, Susan Steinbacher, Jan L Plass, and Detlev Leutner. 2002. Assessment of cognitive load in multimedia learning using dual-task methodology.Experimental psychology 49, 2 (2002), 109.
[5]
Francesca Gasparini, Alessandra Grossi, and Stefania Bandini. 2021. A deep learning approach to recognize cognitive load using ppg signals. In The 14th PErvasive Technologies Related to Assistive Environments Conference. 489–495.
[6]
Luis Cabañero Gómez, Ramón Hervás, Ivan Gonzalez, and Vladimir Villarreal. 2021. Studying the generalisability of cognitive load measured with EEG. Biomedical Signal Processing and Control 70 (2021), 103032.
[7]
Shannan Guan, Haiyan Lu, Linchao Zhu, and Gengfa Fang. 2022. AFE-CNN: 3D Skeleton-based Action Recognition with Action Feature Enhancement. Neurocomputing 514 (2022), 256–267.
[8]
Joshua Harrison, Kurtuluş İzzetoğlu, Hasan Ayaz, Ben Willems, Sehchang Hah, Ulf Ahlstrom, Hyun Woo, Patricia A Shewokis, Scott C Bunce, and Banu Onaral. 2014. Cognitive workload and learning assessment during the implementation of a next-generation air traffic control technology using functional near-infrared spectroscopy. IEEE Transactions on Human-Machine Systems 44, 4 (2014), 429–440.
[9]
Woo-Hyun Hwang, Dong-Hyun Kang, and Deok-Hwan Kim. 2022. Brain lateralisation feature extraction and ant colony optimisation-bidirectional LSTM network model for emotion recognition. IET Signal Processing 16, 1 (2022), 45–61.
[10]
Jeff Klingner, Barbara Tversky, and Pat Hanrahan. 2011. Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology 48, 3 (2011), 323–332.
[11]
Long Lin and Liang Tan. 2022. Multi-Distributed Speech Emotion Recognition Based on Mel Frequency Cepstogram and Parameter Transfer. Chinese Journal of Electronics 31, 1 (2022), 155–167.
[12]
Yongqiang Lyu, Xiaomin Luo, Jun Zhou, Chun Yu, Congcong Miao, Tong Wang, Yuanchun Shi, and Ken-ichi Kameyama. 2015. Measuring photoplethysmogram-based stress-induced vascular response index to assess cognitive load and stress. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 857–866.
[13]
Nargess Nourbakhsh, Yang Wang, Fang Chen, and Rafael A Calvo. 2012. Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. In Proceedings of the 24th australian computer-human interaction conference. 420–423.
[14]
Sharon Oviatt. 2006. Human-centered design meets cognitive load theory: designing interfaces that help people think. In Proceedings of the 14th ACM international conference on Multimedia. 871–880.
[15]
Rachel L Piferi, Keith A Kline, Jarred Younger, and Kathleen A Lawler. 2000. An alternative approach for achieving cardiovascular baseline: viewing an aquatic video. International Journal of Psychophysiology 37, 2 (2000), 207–217.
[16]
Cornelia Setz, Bert Arnrich, Johannes Schumm, Roberto La Marca, Gerhard Tröster, and Ulrike Ehlert. 2009. Discriminating stress from cognitive load using a wearable EDA device. IEEE Transactions on information technology in biomedicine 14, 2 (2009), 410–417.
[17]
David Sharek and Eric Wiebe. 2012. Embedding secondary tasks in video games to measure real-time cognitive load: An approach to developing adaptive video games. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 56. SAGE Publications Sage CA: Los Angeles, CA, 2147–2151.
[18]
Lovejit Singh. 2022. Deep bi-directional LSTM network with CNN features for human emotion recognition in audio-video signals. International Journal of Swarm Intelligence 7, 1 (2022), 110–122.
[19]
John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science 12, 2 (1988), 257–285.
[20]
Aldert Vrij, Ronald Fisher, Samantha Mann, and Sharon Leal. 2008. A cognitive load approach to lie detection. Journal of Investigative Psychology and Offender Profiling 5, 1-2 (2008), 39–43.
[21]
Shiyan Yang, Jonny Kuo, Michael G Lenné, Michael Fitzharris, Timothy Horberry, Kyle Blay, Darren Wood, Christine Mulvihill, and Carine Truche. 2021. The impacts of temporal variation and individual differences in driver cognitive workload on ECG-based detection. Human factors 63, 5 (2021), 772–787.
[22]
Xiao Zhang, Yongqiang Lyu, Xin Hu, Ziyue Hu, Yuanchun Shi, and Hao Yin. 2018. Evaluating photoplethysmogram as a real-time cognitive load assessment during game playing. International Journal of Human–Computer Interaction 34, 8 (2018), 695–706.
[23]
Xiao Zhang, Yongqiang Lyu, Xiaomin Luo, Jingyu Zhang, Chun Yu, Hao Yin, and Yuanchun Shi. 2018. Touch sense: Touch screen based mental stress sense. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 2 (2018), 1–18.
[24]
Xiao Zhang, Yongqiang Lyu, Tong Qu, Pengfei Qiu, Xiaomin Luo, Jingyu Zhang, Shunjie Fan, and Yuanchun Shi. 2019. Photoplethysmogram-based Cognitive Load Assessment Using Multi-Feature Fusion Model. ACM Transactions on Applied Perception (TAP) 16, 4 (2019), 1–17.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICBBT '23: Proceedings of the 2023 15th International Conference on Bioinformatics and Biomedical Technology
May 2023
313 pages
ISBN:9798400700385
DOI:10.1145/3608164
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Cognitive load
  2. LSTM
  3. Photoplethysmogram

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • The Project of Construction and Support for high-level teaching Teams of Beijing Municipal Institutions

Conference

ICBBT 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 43
    Total Downloads
  • Downloads (Last 12 months)33
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media