Takuma Yagi, Ph.D.

Takuma Yagi, Ph. D.
Senior Researcher, National Institute of Advanced Industrial Science and Technology (AIST)

I am a Senior Researcher at Artificial Intelligence Research Center, AIST.
I got my Ph.D. degree at the University of Tokyo in 2022, advised by Prof. Yoichi Sato.
I'm especially interested in developing intelligent systems that perceives and understand our world and using them to make our life better.

（06/02更新）インターン・RA等に関する案内はこちらをご覧ください。

Email: takuma.yagi[at]aist.go.jp
Google Scholar
GitHub
LinkedIn

Resarch Interests

Computer Vision and Machine Learning for Human Behavior Understanding

First-Person Vision
Video Understanding
Hand-Object Interaction Understanding
Learning from Human Observation
Vision & Language
Assisting Human Activities with Computer Vision

News

(Oct. 2025) I have been promoted to Senior Researcher! Great thanks for all my collaborators.
(Jul. 2025) Our work on biological experiment video dataset has been accepted to IJCV2025!
(Nov. 2024) I am visiting Robotics Institute, Carnegie Mellon University as a visiting scholar until next March.
(Nov. 2024) Our work on object state recognition has been accepted to WACV2025@Tucson! Check out the project page!
(Feb. 2024) Our work on a video dataset of biological experiments has been released! Check out the preprint and dataset.

Selected Publications

Please visit Projects and Full Publication List for further details.

New!
FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation
Takuma Yagi, Misaki Ohashi, Yifei Huang, Ryosuke Furuta, Shungo Adachi, Toutai Mitsuyama, and Yoichi Sato
IJCV 2025
[preprint] [paper] [data & code]

Learning Multiple Object States from Actions via Large Language Models
Masatoshi Tateno, Takuma Yagi, Ryosuke Furuta, and Yoichi Sato
WACV2025, LPVL Workshop@CVPR2024
[project page] [preprint] [paper]

Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives.
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, ..., Takuma Yagi, ..., Michael Wray
CVPR 2024
[project page] [preprint]

Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videos
Zecheng Yu, Yifei Huang, Ryosuke Furuta, Takuma Yagi, Yusuke Goutsu, and Yoichi Sato.
WACV 2023, EPIC Workshop@CVPR2022
[preprint][proceedings]

Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, ..., Takuma Yagi, ... Jitendra Malik.
CVPR 2022 (Oral, Best Paper Finalist)
[project page] [paper]

Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction
Takuma Yagi, Md. Tasnimul Hasan, and Yoichi Sato.
BMVC 2021
[paper & talk] [preprint] [data & code]

GO-Finder: Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery
Takuma Yagi, Takumi Nishiyasu, Kunimasa Kawasaki, Moe Matsuki, and Yoichi Sato.
IUI 2021 (Long Paper), TiiS'22
[Preprint] [Paper] [Video] [Presentation]

Future Person Localization in First-Person Videos
Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani, and Yoichi Sato.
CVPR 2018 (Spotlight)
[Preprint] [Paper] [Data & Code]

Work Experience

Oct. 2025 - Present, Senior Researcher, Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST): Senior Researcher at Computer Vision Research Team
Nov. 2024 - Mar. 2025, Visitng Scholar, Robotics Institute, Carnegie Mellon University: Research on assembly action understanding from videos (Mentor: Prof. Kris Kitani)
Apr. 2023 - Sep. 2025, Researcher, Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology (AIST): Researcher at Computer Vision Research Team
Apr. 2023 - Present, Cooperative Research Fellow, Institute of Industrial Science, The University of Tokyo: Cooperative Research Fellow at Sato Laboratory
Apr. 2022 - Mar. 2023, Project Researcher, Institute of Industrial Science, The University of Tokyo: Analysis of biochemical experiment videos at Sato Laboratory
Oct. 2022 - Mar. 2023, Visiting Researcher, Nihon University: Research on Human-Agent Interaction at Osawa Laboratory
Sep. 2018 - Dec. 2018, Microsoft Research Asia: Research Internship at Media Computing Group. Advisor: Dr. Xun Guo
Jun. 2017 - Mar. 2018, Dwango Artificial Intelligence Laboratory: Curriculum supervisor/lecturer at Neuro-Inspired COmputing To AI School (NICO2AI School)
Dec. 2015 - Jan. 2016, Datasection.Inc: Machine Learning Engineer
Mar. 2014 - Mar. 2015, Datasection.Inc: Java/Javascript Engineer

Education

2022 Ph.D. in Information Science and Technology, The University of Tokyo
Advisor: Prof. Yoichi Sato
Thesis: Hand-Object Interaction Mining from First-Person Videos
2019 M.S. in Information and Science and Technology, The University of Tokyo
Advisor: Prof. Yoichi Sato
Thesis: Future Person Localization in First-Person Videos
2017 B.E. in Computer Science, Tokyo Institute of Technology
Advisor: Prof. Koichi Shinoda
Thesis: Patch-based Human Action Recognition from Video via Disrciminative Feature Mining

Awards

Aug. 2021 MIRU Presentation Award
Mar. 2019 Department Chair's Award for Excellent Master Thesis, Department of Information and Communication Engineering, the University of Tokyo

Grants

Oct. 2022 - Mar. 2025 JST ACT-X AI powered Research Innovation / Creation, 6,000,000 JPY
Oct. 2022 - Mar. 2024 JSPS Grant-in-Aid for Research Activity Start-up, 2,200,000 JPY
Apr. 2021 - Mar. 2022 JSPS Research Fellowship for Young Scientists (JSPS DC2), 800,000 JPY
2019 JASSO Scholarship for Top 10% Excellent Master Students from Japan Student Services Organization, 2,112,000 JPY
Jul. 2018 - Jun. 2023 Masason Foundation Full Member
2018 The NEC C&C Foundations: Grants for Attendees at International Conference, 200,000 JPY
Apr. 2017 - Mar. 2018 Toyota/Dwango Scholarship for Advanced Artificial Intelligence Researcher, 1,200,000 JPY

Services

Reviewer

IJCV, CVPR, ICCV, ECCV, ACCV, ICRA, NeurIPS

Organizer

First Joint Egocentric Vision (EgoVis) Workshop Held in Conjunction with CVPR 2024