Scribd
Upload
127 views2 pages

RRL

The document describes a vision-based user interface (VB-UI) system that allows applications to dynamically reconfigure the interface based on the environment. The system sends a description of the interface as a set of widgets to the vision system. The vision system then assembles image processing components to implement the interface. It can define surfaces like camera views where the interface appears, accounting for factors like size, location, and user position. Experimental applications showed the system's ability to adapt interfaces for different display and retail environments.

Uploaded by

api-3709615
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
127 views2 pages

RRL

The document describes a vision-based user interface (VB-UI) system that allows applications to dynamically reconfigure the interface based on the environment. The system sends a description of the interface as a set of widgets to the vision system. The vision system then assembles image processing components to implement the interface. It can define surfaces like camera views where the interface appears, accounting for factors like size, location, and user position. Experimental applications showed the system's ability to adapt interfaces for different display and retail environments.

Uploaded by

api-3709615
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOC, PDF, TXT or read online on Scribd
You are on page 1/ 2

Dynamically Reconfigurable Vision-Based User Interfaces

Vision-based user interfaces (VB-UI) are an emerging area of user interface


technology where a user’s intentional gestures are detected via camera, interpreted
and used to control an application. The paper describes a system where the
application sends the vision system a description of the user interface as a
configuration of widgets. Based on this, the vision system assembles a set of image
processing components that implement the interface, sharing computational resources
when possible. The parameters of the surfaces where the interface can be realized are
defined and stored independently of any particular interface. These include the size,
location and perspective distortion within the image and characteristics of the
physical environment around that surface, such as the user’s likely position while
interacting with it.

The framework presented in this paper should be seen as a way that vision based
applications can easily adapt to different environments. Moreover, the proposed
vision-system architecture is very appropriate for the increasingly common situations
where the interface surface is not static.

1.1 Basic Elements.

A VB-UI is composed of configurations, widgets, and surfaces. Configurations


are a set of individual interaction dialogs. It specifies a boundary area that defines the
configuration coordinate system. The boundary is used during the process of mapping
a configuration onto a particular surface. Each configuration is a collection of
interactive widgets. A widget provides an elemental user interaction, such as detecting
a touch or tracking a fingertip. It generates events back to the controlling application
where they are mapped to control actions such as triggering an event or establishing a
value of a parameter. A surface is essentially the camera’s view of a plane in 3D
space. It is able to define the spatial layout of widgets with respect to each other and
the world but it should not be concerned with the details of the recognition process.

1.2 Architecture

In this system, each widget is represented internally as a tree of components. Each


component performs one step in the widget’s operation. There are components for
finding the moving pixels in an image (Motion Detection), finding and tracking
fingertips in the motion data (Fingertip Tracking), looking for touch-like motions in
the fingertip paths (Touch Motion Detection), generating the touch event for the
application (Event Generation), storing the region of application space where this
widget resides (Image Region Definition), and managing the transformation between
application space and the image (Surface Transformation).

The figure below shows the component tree of a “touch button” and the “tracking
area.”
1.3 Example Applications

One experimental application developed that used the dynamically reconfigurable


vision system is the Everywhere Display Projector (ED). This provides information
access in retail spaces. The Product Finder Application is another example. Its goal is
to allow customer to look up products in a store directory, and then guide him/her to
where the product is.

Reference:

Kjeldsen, R., Levas, A., & Pinhanez, C. (). Dynamically Reconfigurable Vision-Based
User Interface.

You might also like