Direct Manipulation

1
 Interaction versus Interface
• Recall: What is the difference between user
interaction and the user interface?
– Interaction is the dialog between user and computer
• At the I/O threshold
– Interface is the vehicle for dialog
• Fully contained in the computer
• We seek a common language for:
– Defining the interface objects users interact with
– The way we map these objects onto manipulations of
data within the interactive system/application

2
 Concepts for Describing Interfaces
• Instrumental Interaction
– Definition
– Applications
• Direct Manipulation
– Definition/Analogies
– Direct Manipulation Issues
– Representing “Special Locations”

3
 Instrumental Interaction
• Coined and described by Michel Beaudouin-Lafon in
2000
• Interface can be decomposed into instruments and
objects
– Instruments are the manipulable components that act on
objects
– Objects are the domain concepts, the knowledge or
information, being manipulated by the user through the
instruments
• The information content of the application
• The purpose BEHIND the interaction

4
 Instrumental Interaction (2)
• Goals
– Describe state of the art interaction techniques
– Provide metrics (qualitative and quantitative for comparing interaction
techniques within a UI)
– Present a design space to explore new ideas
– Provide guidance for how to integrate new techniques into GUIs
• He defined Instrumental Interaction as an interaction model
– Paraphrased from CHI 2000 paper: “A set of rules, principles, and
properties that guide the define of an interface. It describes how to
combine techniques, the “look and feel”, and provides guidance for
evaluating specific interaction designs.”
• By definitions in this course, not an “interaction model”

5
 Instruments
• WIMP interfaces present a set of instruments to user
– Activated two ways: spatially and temporally
– Examples?
• Instruments can also act on other instruments
– Reification (becoming objects)
– Meta-instruments (instruments designed to act on other
instruments)
– Examples?

6
 Instruments (2)
• Evaluating instruments
– Degree of indirection
• Spatial or temporal offset between instrument and action on
object
• Spatial
– Close = handles on rectangle to resize
– Far = dialog boxes
– Is far always bad?
• Temporal
– Real time response versus clicking OK in dialog
– Degree of integration
• Ratio of degrees of freedom of instrument to degrees of freedom
of input device
– Degree of compatibility
• Similarity of action on control device/instrument to action on
object
7
 Degree of Integration

1D 3D 6D

2D
2D

8
 Degree of Compatibility

Dragging = high Scrolling = medium Dialog = low

9
 Influences Leading to II
• Heavily influenced by Shneiderman’s
principles of direct manipulation
– Examples of direct manipulation
• Drag and drop
• Resizing via handles

10
 What is Direct Manipulation?
• By Definition:
A direct manipulation interface allows a user to
directly act on a set of objects (instruments) in
the interface.
– The instruments are indistinguishable visually from
the objects they represent.
– The actions on instrument/object entities are
analogous to actions on similar objects in the real
world.
– The actions on instrument/object entities preserve
the conceptual linkage between instrument and
object.

11
 What is Direct Manipulation?
• By Example:
– dragging a document to the trash
– changing the size of a triangle by interacting with a
visual representation of a triangle
– inserting characters in a document by pointing to
where they should go (with a
mouse/cursor/insertion point) and then typing
– “dialing” a phone number by pushing numbers on
a keypad
– playing a song using controls like a physical tape
deck or CD/DVD player
 What is Direct Manipulation?
• By Decomposition:
– an analogy (including appropriate presentation of
interface objects)
– an implementation
• By Contrast: It is one of several interaction styles
– menu selection
– fill-in forms
– command language
– natural language
– direct manipulation

13
 What is Direct Manipulation?
• By Characteristics: (from User Interface Design & Evaluation, p. 213-214)
– There is a visible and continuous representation of the task objects and
their actions. Consequently, there is little syntax to remember.
– The task objects are manipulated by physical actions, such as clicking or
dragging, rather than by entering complex syntax.
– Operations are rapid and incremental
– Their effects on task objects are immediately visible.
– Reversibility of (almost) all actions
– Users can explore without severe consequences
– Syntactic correctness – every operation is legal
• What (is/is not) direct manipulation in modern
interfaces?

14
 What is Direct Manipulation?
• By Benefits:
While interacting with DM interfaces, users
feel as if they are interacting with the domain
rather than with the interface, so they focus
on the task rather than on the technology.
There is a feeling of direct involvement with a
world of task objects rather than
communication with an intermediary.

15
 DM Issues (1/2)
• Setback for impaired users
– visually impaired users can’t see the graphics; no linear
flow for screen readers
– physically impaired may have difficulty with actions
• Meanings are indirect
– Users need to learn meaning of visual representations
– Visual representations may be misleading
• Often consume valuable screen space, forcing
valuable information off-screen.
• Switching modes, from keyboard to tracker, is
time consuming.
16
 DM Issues (2/2)
• Many commands are invoked indirectly
– Menus, dialog boxes, toolbars
• Not direct manipulation
• They are mediators that pull users away from objects of interest
• Many objects of interest are hidden
– Style sheets
– Alignment constraints are often fleeting
• Lots of object in the interface are not objects of
interest
– Toolbar pallets
(From Beaudoin-Lafon [2000])

17
 Analogies
• “attempting to make affordances in the
interface like affordances for analogous
actions in the real world”
– Build on the user’s existing experiences and
intuitions to aid learning.
– But… Not every analogy results in a DM interface.
Example: shopping analogy for on-line retailers.

18
 Analogies
• Analysis of analogies, in an attempt to come
up with a more concrete paradigm:
– Identify components of the description of the real-
world action
– Identify corresponding components of the direct
manipulation action
– Try to create a more generalized description of
each pair.

19
 Analyzing an Analogy
Common
Real World DM Interface
Language
Icon of object to be
Object to be discarded Grammatical Object
discarded
Move to object
Move hand to object Move pointer to object
location
Pick up object with
Acquire object Click to aquire object
hand
Waste Basket
Waste basket Waste basket icon
location
Move to waste Drag to waste basket
Move to waste basket
basket location icon
Release object from Release button to
Release object
20 hand discard object
 Analyzing an Analogy
• A location with special meaning (the object)
• Another location with special meaning (the
waste basket)
• Movement between locations

21
 Other Analogies
• Graphics Editors
– analogy of drawing surface and tools to paper and tools
– every location has a special meaning
– locations often have multiple meanings
• Word Processors
– analogy of screen to paper
– locations between letters have special meanings
• Desktop
– analogy to office: documents, folders, waste basket
– all of these have special meanings
• Drag ‘n Drop
• Spreadsheets

22
 Representing Special Locations
• Icons often represent a “special location”; they require:
– a location
– a visible boundary
– enough visual information for users to make a precise identification
• Many other special locations are similar
• Need to perform many inside tests to implement DM:
for (Item item : displayList)
{ if (item.contains(m.x, m.y))
{ …
}
}
• Easy for rectangles, but not for many other shapes. We need a
more general strategy…

23