ed-informatics.org

In the first of this series, I tried to persuade you that your computer is human-illiterate. We discussed ways that people try to improve this, including usability testing. I introduced you to Alan Cooper’s About Face: The Essentials of User Interface Design (you did go out and buy it and read it, didn’t you?) which included such bon mots at “No matter how cool your interface is, less of it would be better.” I introduced you to Jakob Nielsen, his book Usability Engineering, and his online columns. Next, we went on to put usability into its place in Neilsen’s schema of overall acceptability of a program, including such other factors such as cost and reliability. We then reviewed the five components of usability:

1)       Ease of learning

2)       Efficiency of use

3)       Ease of remembering

4)       Rate of error during use

5)       Subjectively pleasing

We went on to learn about Learnability and Memorability (and, I hope, in a learnable and memorable way: remember the Kiss-and-Ride signs?)

OK, enough recapitulation, and on to: Tognazzini’s Paradox. Let me now introduce you to Bruce Tognazzini, AKA “Tog,” who was the Apple Computer User Interface Evangelist (yes, this was an official Apple Computer title). In the early days of Apple, he was responsible for much of the Mac’s interface that was laudable. (We will not talk about how much of it was stolen by Microsoft for Windows, or contrariwise how much Apple stole from the Xerox Palo Alto Research Center (PARC). Anyway, Tog was and is famous for his innovations and even more for his insights about computer user interfaces. His book Tog on Interface is an updated collection of his columns – highly recommended reading. As soon as you finish Cooper’s book, start reading Tog’s book.

Let’s see what Tognazinni’s Paradox is, and how it relates to usability.

In testing the introductory program, on a floppy disk, for  the Apple II computer, Tog found that the simplest things, such as the exact wording of a single sentence, can be the hardest part of a computer program to perfect. Indeed, the question Apple used to ask users whether they had a color or monochrome monitor had to go through six iterations (modifications) to get the users to answer the question correctly. Simple and seemingly trivial changes in wording; BIG differences in whether users answered question correctly.

Specifically, here was the initial state of the introductory program before usability testing, quoting directly from Tog’s book:

Problem: In “Apple Presents … Apple, An Introduction to the Apple II Plus Computer,” find out if the user is working with a color monitor.

User profiles: New owner, customer in a computer store, or member of a class learning to use Apple computers.

Test user profiles:  Customers in a computer store, non-computerists in a classroom environment, friends, and relatives.

First design: A color graphic would be displayed.

PROMPT:  “Are you using a color TV on the Apple?”

ANTICIPATED PROBLEM:  Those who were using a monochrome monitor in a classroom or computer store situation wouldn’t know whether the monitor was black and white or was color with the color turned off.

So, they went on to test various wording, using first a color version of the Apple logo, an apple in horizontal bands of different colors.

Apple II logo

“Is the picture above in color?”

failure rate:         25%

As anticipated, but incorrectly overcome, those seeing black and white thought their color might be turned down. They didn’t answer the question wrong; they turned around and asked one of the authors whether the monitor in question was color or not. A decision was made that the authors could not be shipped with each disk.

After this failed, they put in a small graphic with the words GREEN BLUE ORANGE MAGENTA, each in color.

“Are the words above in color?”

color word display illustrating Tognazinni's Paradox

 

failure rate:

• color TV users: none
• black and white monitor users: none
• green-screen monitor users: 100%

Yes, well, you see, we hadn’t exactly thought about monochrome monitors with nice, bright green text. After all, who could have predicted that users might actually think green was a color? Silly twits! Actually, we were extremely fortunate that we accidentally got hold of a prototype green-screen monitor that day. Otherwise, we might have shipped the software with a fatal design flaw.

Next, they tried rewording slightly; I’ve added emphasis to help you pick out differences in wording:

“Are the words above in more than one color?”

failure rate:

• color TV users: none
• black and white monitor users: 20%
• green-screen monitor users: 50%

The black and white monitor users who answered incorrectly admitted that they did so on purpose. (Our methods for wringing their confessions shall remain proprietary.) 50% of the green-screen folk considered that they were looking at both black and green-two colors-and answered the question accordingly.

“Are the words above in several different colors?”

failure rate:

• color TV users: none
• black and white monitor users: 20%
• green-screen monitor users: 25%

By this time, the authors were prepared to supply everyone who bought an Apple II with a free color monitor, just so we would not have to ask the question. It turns out that around 20% of the people were not really reading the question. They were responding to the question “Are the words above several different colors?”

They finally succeeded with:

“Do the words above appear in several different colors?”

failure rate: NONE.

Note carefully how tiny changes in wording make BIG changes in the number of user errors. But, no, let’s not call these “user errors.” From the Whorf-Sapir hypothesis mentioned in the last article, we don’t want to brainwash ourselves into blaming users, we should blame the program’s design team. They’re “programming errors” or “design errors.” From the standpoint of someone designing a program for a naïve user, OR of someone evaluating a new software program for the ED, “Users don’t make mistakes.” If, like me, you’ve seen several people sit down to try out a new software program, and most of them make the same mistakes, you can be pretty sure that the designers didn’t bother with even the most basic usability testing.

Yes, Microsoft spends millions of dollars yearly on detailed usability testing. But ED software designers don’t have to spend a lot of money, they don’t have to have a big software testing lab, and often the only changes they have to make take a matter of seconds (OK, maybe minutes) to re-code.

Bottom line:  You can’t look at a program and guess how usable it is. You have to test it with naïve users. You can do your own usability testing on a program you’re thinking about buying – get two of your colleagues to try to perform several common tasks, and watch carefully and see what “user errors” they make. Don’t let the vendor representatives coach, just sit your colleagues down at the computer. You can easily do this at one of the big conferences, or during a vendor presentation at your hospital.

A sideline: Tognazzini’s Paradox applies outside of user interaction design. For instance, the NEXUS criteria, a set of simple rules, are now widely used to “clear” the cervical spine – to determine that someone who’s been in an accident doesn’t need X-rays or a CT scan of the neck to rule out a fracture. The rules are as follows (quoted directly from the original article):

• the absence of tenderness at the posterior midline of the cervical spine,
• the absence of a focal neurologic deficit,
• a normal level of alertness,
• no evidence of intoxication, and
• absence of clinically apparent pain that might distract the patient from the pain of a cervical-spine injury.

Patients who met all five criteria were considered to have a low probability of injury and not to require radiographic or other imaging.

There have been attempts to apply these criteria, designed for use by physicians in the Emergency Department, to paramedics working in the field. One study concluded that EMS personnel could not apply the criteria as well as emergency physicians in the ED. However, the version of the NEXUS criteria they studied were as follows:

• Accurate orientation to time, person and place? Responds appropriately to questions and commands? Not distracted by other pain?
• Spontaneous pain, pain on palpation, or pain on movement?
• Numbness, tingling, motor weakness, sensory or motor deficit?
• Pain with gentle range of motion?

As you can see, comparing these criteria with the NEXUS criteria is like comparing apples to kunquats, given we now know that even changing a single word can make people’s interpretation quite different. If you want to test the NEXUS criteria, you need to check the NEXUS criteria, not some mutant version of the criteria, and changing even a single word can result in vastly different results.

I hope that the quotations from Tog on Interface encourage you to go out and get a copy ASAP, there’s lots more great stuff in there. [Tognazzini B. Tog on interface. Reading, MA: Addison-Wesley; 1992.]

Enough for now.  Next time, we’ll start talking about design integrity, and in particular about abstraction and simplicity.

Tags: Computers, Alan Cooper, Tutorial, Usability, Xerox Palo Alto Research Center, PARC, Bruce Tognazzini, Tognazzini's Paradox, User Interface