Amy Pavel

Assistant Professor
UC Berkeley
Electrical Engineering and Computer Sciences

Curriculum Vitae PDF

Research Group
Ph.D. Students: Mina Huh, Karim Benharrak, Ananya Gubbi Mohanbabu, Meng Chen, Yi-Hao Peng (co-advised with Jeffrey P. Bigham)
Masters and Undergraduates: Aadit Barua, Akhil Iyer, Katie Clark, Sumaya Al-Bedaiwi, Ujjaini Das, Jerry He, Sarah Zheng
Recent Alumni: Doeun Lee, Pranav Venkatesh, Tess Van Daele, Yuning Zhang, Daniel Killough, Jalyn Derry, Aochen Jiao, Soumili Kole, Chitrank Gupta

I am an Assistant Professor in the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley and a Visiting Faculty Resesarcher at Google Research. Before joining UC Berkeley, I spent three wonderful years (Spring 2022-Spring 2025) as an Assistant Professor in the Department of Computer Science at the University of Texas at Austin.

I was previously a postdoc at Carnegie Mellon University, a Research Scientist at Apple, and I received my PhD from UC Berkeley EECS. I regularly teach a Computer Science class that covers the design and development of user interfaces (Introduction to Human-Computer Interaction). Prior versions include CS160 at UC Berkeley in 2018 and CS378 at UT Austin in 2022.

Current UC Berkeley Students: I am actively recruiting UC Berkeley students for research positions in my lab! Check out this document (CalNet ) to learn more and apply.

Prospective Students, Visitors, Interns, and Postdocs: I recruit PhD students every year. Please apply directly to the UC Berkeley EECS PhD program and list me as a potential advisor. For other questions or positions, please read these FAQs before reaching out.

Research Highlights

GenAssist

UIST 2023 – Best Paper Award

A thumbnail depicting four generated image options of a young chef cooking dinner and their descriptions.

PDF | Project Page

CrossA11y

UIST 2022 – Best Paper Award

A thumbnail depicting CrossA11y's computational approach that compares video frames with the corresponding audio.

PDF | Project Page | Video

Rescribe

UIST 2020

On one side, a set of extended audio description sentences with their corresponding frames, and in the other column a set of inline audio description sentences and the same frames. The shortened descriptions are as follows: `Shots of lavender in a farmers market' is shortened to `Shots of lavender', `Red flowers against a white house and blue sky' is shortened to `Red flowers', `a courtyard and a pool' is not shortened, `Gaby bikes along a path' is shortened to `Close up of french fries', and `Close up of tater tots and french fries' is shortened to `Close up of french fries'

PDF | Video | Talk

Long-Form VQA

COLM 2024 – Oral Spotlight

A thumbnail of longform visual question answers across humans, GPT-4V and Gemini. Each answer sentence is labeled with a discourse role (e.g., confirmation, answer) and information source (e.g., image quality, image content).

Project Page

Research Summary

As a systems researcher in Human-Computer Interaction and Accessibility, I embed machine learning technologies (e.g., Natural Language Processing) into new human interactions that I then deploy to test. Using my systems, remote content creators more effectively collaborate, video authors efficiently create accessible descriptions for blind users, and instructors help students to learn and retain key points. To inform future systems that capture what is important to domain experts and people with disabilities, I also conduct and collaborate on in-depth qualitative (e.g., AAC communication, memes) and quantitative studies (e.g., 360° Video, VR Saliency). My research goal is to make communication effective and accessible.

Research Papers

A thumbnail of the Vid2Coach system transforming instructional videos and accessible tips into interactive task assistants.

Vid2Coach: Transforming How-To Videos into Task Assistants

Mina Huh, Zihui Xue, Ujjaini Das, Kumar Ashutosh, Kristen Grauman, Amy Pavel

To Appear at UIST 2025

Project Page | arXiv

A thumbnail of the Morae system pausing agent interactions to allow for user choice.

Morae: Proactively Pausing UI Agents for User Choices

Yi-Hao Peng, Dingzeyu Li, Jeffrey P. Bigham, Amy Pavel

To Appear at UIST 2025

A thumbnail of text-based speech editing with multicolored highlights.

TalkLess: Blending Extractive and Abstractive Speech Summarization for Editing Speech to Preserve Content and Style

Karim Benharrak, Puyuan Peng, Amy Pavel

To Appear at UIST 2025

arXiv

A thumbnail of the CoSight Chrome extension interface.

CoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commenting

Ruolin Wang, Xingyu Liu, Biao Wang, Wayne Zhang, Ziqian Liao, Ziwen Li, Amy Pavel, Xiang "Anthony" Chen

To Appear at UIST 2025

arXiv

A thumbnail of VeasyGuide's highlight and zoom features.

VeasyGuide: Personalized Visual Guidance for Low-vision Learners on Instructor Actions in Presentation Videos

Yotam Sechayk, Ariel Shamir, Amy Pavel, Takeo Igarashi

To Appear at ASSETS 2025

Project Page | arXiv

A thumbnail of a Wikipedia page with varying opacity according to task relevance.

Task Mode: Dynamic Filtering for Task-Specific Web Navigation using LLMs

Ananya Gubbi Mohanbabu, Yotam Sechayk, Amy Pavel

To Appear at ASSETS 2025

arXiv

A thumbnail of two image descriptions with variations.

Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions

Meng Chen, Akhil Iyer, Amy Pavel

To Appear at ASSETS 2025

arXiv

A thumbnail of an interface comparing three video timelines.

VideoDiff: Human-AI Video Co-Creation with Alternatives

Mina Huh, Dingzeyu Li, Kim Pimmel, Valentina Hijung, Amy Pavel, Mira Dontcheva

CHI 2025

arXiv

A thumbnail of abstractive and extractive video summarization.

Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization

Aadit Barua, Karim Benharrak, Meng Chen, Mina Huh, Amy Pavel

IUI 2025

arXiv

Long-Form Answers to Visual Questions from Blind and Low Vision People

Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, Amy Pavel

COLM 2024

Oral Spotlight

Project Page

A thumbnail of an interface for browsing potentially harmful video segments.

Design Considerations for Photosensitivity Warnings in Visual Media

Laura South, Caglar Yildirim, Amy Pavel, Michelle A. Borkin

ASSETS 2024

A thumbnail of a couch listing image with a context free description that does not contain much information about the couch, and a context-aware description that focuses on the couch.

Context-Aware Image Descriptions for Web Accessibility

Ananya Gubbi Mohanbabu, Amy Pavel

ASSETS 2024

Project Page

A thumbnail of DreamStruct's UI generation that starts with a UI description, then generates HTML UI code, then generates images for the UI and outputs the final user interface.

DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation

Yi-Hao Peng, Faria Huq, Yue Jiang, Jason Wu, Amanda Xin Yue Li, Jeffrey P. Bigham, Amy Pavel

ECCV 2024

A thumbnail of 5 websites before and after using DesignChecker. The after websites have improved aesthetically according to design pricinples (e.g., color harmony, alignment).

DesignChecker: Visual Design Support for Blind and Low Vision Web Developers

Mina Huh, Amy Pavel

UIST 2024

Project Page

A thumbnail of a table of potential accessibility barriers in VR.

Barriers to Photosensitive Accessibility in Virtual Reality

Laura South, Caglar Yildirim, Amy Pavel, Michelle A. Borkin

CHI 2024

Honorable Mention Award

PDF

A thumbnail of the ShortScribe pipeline. The pipeline takes the video as input, transcribes it and describes the keyframes, then uses GPT-4 to generate multiple types of descriptions.

ShortScribe: Making Short-Form Videos Accessible with Hierarchical Video Summaries

Tess Van Daele, Akhil Iyer, Yuning Zhang, Jalyn Derry, Mina Huh, Amy Pavel

CHI 2024

Project Page

A thumbnail of the COMPA interface that features views for the AAC user and their conversational partners.

COMPA: Using Conversation Context to Achieve Common Ground in AAC

Stephanie Valencia, Jessica Huynh, Emma Y Jiang, Yufei Wu, Teresa Wan, Zixuan Zheng, Henny Admoni, Jeffrey P. Bigham, Amy Pavel

CHI 2024

PDF

GeoLatent: A Geometric Approach to Latent Space Design for Deformable Shape Generators

Haitao Yang, Bo Sun, Liyan Chen, Amy Pavel, Qixing Huang

SIGGRAPH ASIA 2023

GenAssist: Making Image Generation Accessible

Mina Huh, Yi-Hao Peng, Amy Pavel

UIST 2023

Best Paper Award

PDF | Project Page

A thumbnail of a livestream paired with a set of descriptions with timestamps.

Exploring Community-Driven Descriptions for Making Livestreams Accessible

Daniel Killough, Amy Pavel

ASSETS 2023

PDF

A thumbnail of the audio visual script that includes a transcript of the video coupled with detected errors in the video footage (e.g., camera blur).

AVscript: Accessible Video Editing with Audio-Visual Scripts

Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang "Anthony" Chen, Young-Ho Kim, Amy Pavel

CHI 2023

PDF | Project Page

A thumbnail of SlideSpecs featuring numbered slides and audience feedback on the slides.

SlideSpecs: Automatic and Interactive Presentation Feedback Collation

Jeremy Warner, Amy Pavel, Tonya Nguyen, Maneesh Agrawala, Björn Hartmann

IUI 2023

PDF | Project Page | Demo | Code

CrossA11y: Identifying Video Accessibility Issues via Cross-modal Grounding

Xingyu Liu, Ruolin Wang, Dingzeyu Li, Xiang "Anthony" Chen, Amy Pavel

UIST 2022

Best Paper Award

PDF | Project Page | Video

A thumbnail of Diffscriber's interface for reviewing slide changes.

Diffscriber: Describing Visual Design Changes to Support Mixed-Ability Collaborative Presentation Authoring

Yi-Hao Peng, Jason Wu, Jeffrey P. Bigham, Amy Pavel

UIST 2022

PDF | Video Preview

Three images of Tech Help Desk including the Tech Help Desk classroom, the Tech Help Desk sign on the classroom door and the building entrance.

Tech Help Desk: Support for Local Entrepreneurs Addressing the Long Tail of Computing Challenges

Yasmine Kotturi, Herman T Johnson, Michael Skirpan, Sarah E Fox, Jeffrey P. Bigham, Amy Pavel

CHI 2022

PDF

A thumbnail image of a chart of alt text coverage that appears in the paper.

Toward supporting quality alt text in computing publications

Candace Williams, Lilian de Greef, Ed Harris III, Amy Pavel, Cynthia L. Bennett

W4A 2022

PDF

A thumnbail image of the tutorial lens interface.

TutorialLens: authoring Interactive augmented reality tutorials through narration and demonstration

Junhan Kong, Dena Sabha, Jeffrey P. Bigham, Amy Pavel, Anhong Guo

SUI 2021

PDF

A thumnail image of SlideCho's interface for surfacing information in slide videos.

Slidecho: Flexible Non-Visual Exploration of Presentation Videos

Yi-Hao Peng, Jeffrey P. Bigham, Amy Pavel

ASSETS 2021

PDF

A thumnail image of a semantic exemplar with three corresponding examples.

Controlling Dialogue Generation with Semantic Exemplars.

Prakhar Gupta, Jeffrey P. Bigham, Yulia Tsvetkov, Amy Pavel

NAACL 2021

PDF

A slide from a lecture where most text is in black but some of the text has been colored green to represent that it has been spoken by the presenter. There is an image with a squiggly circle brush on the slide, and it has a green border to indicate the presenter described the image.

Say It All: Feedback for Improving Non-Visual Presentation Accessibility

Yi-Hao Peng, JiWoong Jang, Jeffrey P. Bigham, Amy Pavel

CHI 2021

PDF | 30s Video | Presentation (5 min) | Presentation Transcript

A YouTube search result augmented with accessibility information. A thumbnail of the video shows a woman in Rome. The rest of the video information reads as follows: Rome, Italy! Tips & Photo Spots by Ruby Keyvani. 5/7 Somewhat Accessible. 76% of the video is speech. The speech is descriptive but contains many visual references (3 per minute). Visual changes occur infrequently (5 shots per minute); few on-screen objects are described (20%).

What Makes a Video Non-Visually Accessible?

Xingyu Liu, Patrick Carrington, Xiang "Anthony" Chen, Amy Pavel

CHI 2021

PDF | 30s Video | Presentation (5 min)

A photograph taken at a workshop with a person using an AAC device, her close conversational partner, and puppeteers. All participants stand around a table full of craft supplies and are engaged in coversation.

Co-designing Socially Assistive Sidekicks for Motion-based AAC

Stephanie Valencia, Michal Luria, Amy Pavel, Jeffrey P. Bigham, Henny Admoni

HRI 2021

PDF

Rescribe: Authoring and Automatically Editing Audio Descriptions.

Amy Pavel, Gabriel Reyes, Jeffrey P. Bigham

UIST 2020

PDF | Video | Talk

Two example prototypes for making AR apps accessible. A: Foundational Accessibility. Screenshot of a virtual chair with a voice over target around it, a speech bubble shows the app announcing 'Back of chair with blue cushion'. B: Scanning. Screenshot of AR grid overlaid on a coffee table. Speech bubbles show the app announcing 'Found a new horizontal surface' and 'Scanned 2 surfaces totaling 2.3 square meters'.

Making Mobile Augmented Reality Applications Accessible.

Jaylin Herskovitz, Jason Wu, Samuel White, Amy Pavel, Gabriel Reyes, Anhong Guo, Jeffrey P. Bigham

ASSETS 2020

PDF | Video

A cartoon stylized version of a popular reaction GIF of Oprah Winfrey shrugging. She turns to look to the camera, glances to the side, stares at the camera, then shrugs with her palms up.

Making GIFs Accessible.

Cole Gleason, Amy Pavel, Himalini Gururaj, Kris M. Kitani, Jeffrey P. Bigham

ASSETS 2020

Paper | Link to Thumbnail GIF

Screenshot of a tweet by @CDCgov from April 1, 2020 3:55pm: Actions to reduce spread of the virus, such as social distancing, are key to #FlattenTheCurve. 2 of 3 (original tweet link: https://twitter.com/CDCgov/status/1245439600472084486) The tweet contains an image of the common public health infographic about “flattening the curve”, but the tweet did not include alt text for the image. The image shows an example of a common flatten the curve info-graphic. A tall peak indicates the height of the pandemic if left unchecked, and a shorter spread out curve depicts the effects of social distancing efforts.

Disability and the COVID-19 Pandemic: Using Twitter to Understand Accessibility during Rapid Societal Transition.

Cole Gleason, Stephanie Valencia, Lynn Kirabo, Jason Wu, Anhong Guo, Elizabeth J. Carter, Jeffrey P. Bigham, Cynthia L. Bennett, Amy Pavel

ASSETS 2020

PDF

A thumbnail with illegible examples of tweet images and their corresponding alt text.

Twitter A11y: A Browser Extension to Make Twitter Images Accessible.

Cole Gleason, Amy Pavel, Emma McCamey, Christina Low, Patrick Carrington, Kris M. Kitani, Jeffrey P. Bigham

CHI 2020

Honorable Mention Award

PDF

A thumbnail with a conversation between a person using an AAC device and two other people.

Conversational Agency in Augmentative and Alternative Communication.

Stephanie Valencia, Amy Pavel, Jared Santa Maria, Seunga (Gloria) Yu, Jeffrey P. Bigham, Henny Admoni

CHI 2020

Honorable Mention Award

PDF

A thumbnail of a paper figure showing three memes with descriptions.

Making Memes Accessible.

Cole Gleason, Amy Pavel, Xingyu Liu, Patrick Carrington, Lydia Chilton, Jeffrey P. Bigham

ASSETS 2019

PDF | Time article

Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References.

Prakhar Gupta, Shikib Mehri, Tiancheng Zhao, Amy Pavel, Maxine Eskenazi, Jeffrey P. Bigham

SIGDIAL 2019

PDF | Code

A thumbnail of a paper figure including nine images with heat maps indicating visual saliency in VR.

Saliency in VR: How do people explore virtual environments?

Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, Gordon Wetzstein

IEEE VR 2018

Project Page | Code | Video

A thumbnail of a paper figure indicating three different editing techniques for 360 video: traditional, viewpoint-oriented cuts and active reorientation.

Shot Orientation Controls for Interactive Cinematography with 360 video.

Amy Pavel, Björn Hartmann, Maneesh Agrawala

UIST 2017

PDF | Code | Video

A thumbnail of the VidCrit interface with a video on the left and text critiques of the video on the right.

Vidcrit: Video-based Asynchronous Video Review.

Amy Pavel, Dan B Goldman, Björn Hartmann, Maneesh Agrawala

UIST 2016

PDF | Video

SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries.

Amy Pavel, Dan B Goldman, Björn Hartmann, Maneesh Agrawala

UIST 2015

PDF | Video

A thumbnail indicating an overview of the CrowdCrit critique process.

Structuring, Aggregating, and Evaluating Crowdsourced Design Critique.

Kurt Luther, Jari-lee Tolentino, Wei Wu, Amy Pavel, Brian P Bailey, Maneesh Agrawala, Björn Hartmann, Steven Dow

CSCW 2015

A thumbnail of the Video Digests interface. Video thumbnails displayed alongside short summaries of the video content.

Video Digests: A Browsable, Skimmable Format for Informational Lecture Videos.

Amy Pavel, Colorado Reed, Björn Hartmann, Maneesh Agrawala

UIST 2014

PDF | Code

Thesis and Technical Reports

Navigating Video Using Structured Text

Amy Pavel

PhD in Computer Science, University of California, Berkeley

Advisors: Bjoern Hartmann and Maneesh Agrawala

Additional committee members: Eric Paulos, Abigail De Kosnik

Browsing and Analyzing Command Structure of Large Collections of Image Manipulation Tutorials.

Amy Pavel, Floraine Berthouzoz, Björn Hartmann, Maneesh Agrawala

UC Berkeley Technical Report, EECS-2013-167

Posters, Demos, and Workshops

Accessible Cyber-Physical Activities

Riku Arakawa, Franklin Mingzhe Li, Nandi Zhang, Mina Huh, Amy Pavel, Ryo Suzuki, Patrick Carrington, Yukang Yan

UIST 2025 Workshops

Workshop Website

A thumbnail of the creation tool workflow converting images into multiple tactile format: embossings, bas-relief models, and 3D models.

Simultaneously Generating Multiple Mediums of Tactile Graphics

Katie Clark, Amy Pavel

ASSETS 2025 Posters and Demos

A thumbnail of TaskArtisan's interface with sketch-style mockups.

TaskArtisan: Flexible Authoring and Manipulation of Task-specific Interactive Widgets via Sketch and Voice

Meng Chen, Amy Pavel

UIST 2025 Posters

Generative AI and Accessibility Workshop: Surfacing Opportunities and Risks

Kate Glazko, Mina Huh, Jazette Johnson, Amy Pavel, Jennifer Mankoff

CHI 2025 Workshop

Workshop Website

A thumbnail screenshot of a forum people use to report flashing lights.

Exploratory Thematic Analysis of Crowdsourced Photosensitivity Warnings

Laura South, Caglar Yildirim, Amy Pavel, Michelle A. Borkin

CHI 2023 Extended Abstracts

PDF

A thumbnail with an illegible system diagram of Twitter A11y.

Twitter A11y: A Browser Extension to Make Twitter Images Accessible.

Christina Low, Emma McCamey, Cole Gleason, Amy Pavel, Emma McCamey, Patrick Carrington, Jeffrey P. Bigham

ASSETS 2019 Posters

A thumbnail showing a line graph of the precision-at-one of an algorithm going up as the number of noteworthy sentences considered rises. After a certain point, the number of noteworthy sentences decreases the precision at one -- indicating that lower quality noteworthy sentences add noise rather than value to the prediction.

Extracting Structured Data from Doctor-Patient Conversations By Predicting Noteworthy Utterances.

Kundan Krishna, Amy Pavel, Benjamin Schloss, Jeffrey P. Bigham, Zachary Lipton

W3PHIAI 2020 Workshop Paper

CrowdCrit: Crowdsourcing and Aggregating Visual Design Critique.

Kurt Luther, Amy Pavel, Wei Wu, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, Steven Dow

CSCW 2014 Extended Abstracts

A thumbnail of the Sifter interface for browsing common sets of image editing commands.

Sifter: Analyzing and Exploring Large Collections of Web-Based Image Manipulation Tutorials.

Amy Pavel, Floraine Berthouzoz, Björn Hartmann, Maneesh Agrawala

TECHCON 2012

Work

Assistant Professor — University of California, Berkeley

Department of Electrical Engineering and Computer Sciences

July 2025 —

Affiliate Assistant Professor — University of Texas at Austin

Department of Computer Science

July 2025 —

Visiting Faculty Researcher — Google

Google Research

Remote, 20% appointment

October 2024 — December 2025

Assistant Professor — University of Texas at Austin

Department of Computer Science

January 2022 — June 2025

Research Scientist (50% time) — Apple Inc

AI/ML

Machine Intelligence Accessibility Group

July 2019 — January 2022

Postdoctoral Fellow (50% time) — Carnegie Mellon University

HCII

Supervised by Professor Jeffrey P. Bigham

January 2019 — October 2021

Graduate Researcher — University of California, Berkeley

Visual Computing Lab

Advised by Professors Björn Hartmann and Maneesh Agrawala

September 2013 — January 2019

Research Intern — Adobe

Creative Technologies Lab

Advised by Principal Scientist Dan Goldman

Summer 2014, Summer 2015

Undergraduate Researcher — University of California, Berkeley

BiD Lab, Visual Computing Lab

Advised by Professors Björn Hartmann and Maneesh Agrawala

June 2011 — September 2013

Teaching

Instructor — UT Austin

CS 378: Introduction to Human-Computer Interaction

Spring 2025

Instructor — UT Austin

CS 395T: Human-Computer Interaction Research

Fall 2024

Instructor — UT Austin

CS 378: Introduction to Human-Computer Interaction

Spring 2024

Instructor — UT Austin

CS 395T: Human-Computer Interaction Research

Fall 2023

Instructor — UT Austin

CS 378: Introduction to Human-Computer Interaction

Spring 2023

Instructor — UT Austin

CS 378: Introduction to Human-Computer Interaction

Spring 2022

Instructor — UC Berkeley

CS 160: User interface design and development

Summer 2018

Graduate student instructor — UC Berkeley

CS 160: User interface design and development

Summer 2017

Student project advisor — UC Berkeley

NWMEDIA 190: Making Sense of Cultural Data

Fall 2017

Instructor — UC Berkeley

CS Kickstart, intro CS for incoming freshmen women

Summer 2012

Teacher — UC Berkeley

Berkeley Engineers and Mentors

2009 - 2010