UNIT 6 TEST

-Reecha Suryavanshi
USER TESTING
 Involves gathering user feedback and evaluating the user
experience of a software application, website, mobile app, or
physical device.

 User testing is conducted by having real sample users from the

target audience interact with the product while researchers or
designers observe their actions, behaviors, and feedback.

 This process aims to uncover potential usability issues, such as

confusing navigation, unclear instructions, or frustrating
interactions, that may hinder users from achieving their goals
efficiently.
KEY COMPONENTS OF USER TESTING

 Test Plan: A comprehensive test plan outlines the

objectives of the user testing, the specific tasks users will
perform, and the criteria for success. It also includes details
such as the target audience, testing environment, and the
overall structure of the testing process.

 User Recruitment: Identifying and recruiting

representative users is crucial for meaningful results. Test
users should match the target user personas for the
product or service. Recruitment methods may include using
existing user bases, hiring participants, or leveraging user
testing platforms.
 Test Scenarios and Tasks: Test scenarios are specific
situations or contexts that end users will encounter during
the testing process. Tasks are the actions or goals users are
instructed to complete. These scenarios and tasks are
designed to simulate real-world usage and assess the
usability and functionality of the product.

 Test Environment: The test environment should mimic

the actual usage conditions as closely as possible. This
includes the hardware, software, and network conditions
that users would encounter in real-life situations. Whether
testing in a controlled lab setting or remotely, the
environment should be consistent for all participants.
 Moderator or Facilitator: A moderator or facilitator
guides users through the testing process if it is
a moderated testing. They provide instructions, answer
questions, and observe users’ interactions. The moderator
ensures that the testing sessions adhere to the test plan
and gathers qualitative data through user feedback.

 User Metrics and Data Collection: Quantitative

and qualitative data are collected during user testing. This
may include task success rates, completion times, error
rates, and user feedback. Tools such as surveys,
questionnaires, or observation notes are used to document
users’ experiences and opinions.
 Recording Tools: Recording tools capture the user testing
sessions for later qualitative data analysis. This could
include video recordings, screen captures, or audio
recordings. These recordings are valuable for reviewing
user interactions and behaviors and can provide additional
context during analysis.

 Analysis and Reporting: After user testing sessions, the

collected data is analyzed to identify patterns, trends, and
issues. This analysis helps in making informed decisions
about design improvements. A comprehensive report is
often generated to document findings, recommendations,
and any necessary design changes.
 Feedback and Iterative Design: User testing is often an
iterative process. Feedback from one round of testing
informs design changes, which are then retested with
users. This iterative cycle continues until the product or
service meets user expectations and usability goals.

 Post-Test Debriefing: After completing the user testing

sessions, a debriefing session is often held with
participants. This allows the facilitator to gather additional
insights, clarify any misunderstandings, and thank
participants for their time and input.

 Usability Metrics: Depending on the goals of the user

testing, various usability metrics may be considered. These
could include metrics such as task success rate, time on
task, error rates, and subjective satisfaction scores.

User testing is a dynamic and adaptable process, and the

components listed above can be adjusted based on the
specific goals, context, and constraints of a particular
project. The key is to obtain valuable insights from real
users to inform and improve the design and functionality of
the product or service.
KEY BENEFITS OF USER TESTING
 Identifying Usability Issues: User testing helps uncover
usability problems that might not be apparent to the
development team. Observing users in real-world scenarios
can reveal stumbling blocks, confusing interfaces, or
navigation issues that need attention.
 Improving User Experience (UX): By obtaining direct
feedback from users, developers and designers can enhance
product experience through overall user experience (UX)
optimization. Understanding how users interact with a
product allows for adjustments to be made to improve
usability, accessibility, and satisfaction.
 Validating Design Decisions: User testing provides
empirical evidence to validate design decisions. Instead of
relying solely on assumptions, developers can verify
whether users understand and appreciate the design
choices made during the development process.
 Enhancing Product Accessibility: Testing with a diverse
group of users helps ensure that the product is accessible to
individuals with varying abilities and disabilities. This
inclusivity is crucial for reaching a broader audience and
adhering to accessibility standards.
 Reducing Development Costs: Addressing usability issues
early in the development process is more cost-effective than
making changes after the product is launched. User testing helps
catch issues before they become more challenging and expensive
to rectify.
 Optimizing Conversion Rates: In the context of websites or
applications with specific goals (e.g., e-commerce sites or sign-up
forms), user testing can reveal obstacles that might hinder users
from completing desired actions. Improvements based on testing
can lead to higher conversion rates.
 Increasing User Satisfaction and Loyalty: By actively
involving users in the testing process, developers can create
products that align more closely with user expectations,
leading to higher levels of satisfaction.
 Providing Objective Data: User testing generates
objective data rather than relying solely on subjective
opinions. This data-driven approach provides a more
reliable basis for making design and development
decisions.
 Guiding Iterative Design: User testing is often an
iterative process, allowing designers and developers to
make incremental improvements based on user feedback.
This iterative cycle helps refine the product continuously.
 Building Empathy for Users: Direct interaction with users
fosters empathy among the development team.
Understanding users’ needs, frustrations, and preferences
helps create a more user-centric approach to product
development.
 Enhancing Brand Reputation: Products that are user-
friendly and provide a positive experience contribute to a
favorable brand reputation. Users are more likely to
recommend and speak positively about a product that meets
their needs effectively.
 Meeting User Expectations: User testing helps ensure that
the final product aligns with user expectations. Meeting or
exceeding these expectations is crucial for the success and
adoption of any product or service.
EXAMPLE
 Online Form Completion:
 Scenario: Users need to fill out an online form, such as a
registration form for a website or an application.
 Testing: Participants are asked to complete the form while
researchers monitor their progress. Researchers pay
attention to factors like the clarity of form fields, the
appropriateness of error messages, and the overall flow of
the process. Users’ ability to complete the form accurately
and efficiently is assessed.
 E-commerce Website Checkout Process:
 Scenario: Imagine you are an online shopper looking to
purchase a product from an e-commerce website. Your goal
is to find a specific item, add it to your cart, and complete
the checkout process.
 Testing: During user testing, participants would be asked
to perform this task while researchers observe their actions
and gather feedback. Researchers might track the time it
takes to complete the task, note any difficulties
encountered, and ask users to share their thoughts about
the process.
TYPES OF USER TESTING

 1. Usability Testing
 2. Explorative Testing

 3. A/B Testing

 4. Accessibility Testing

 5. Beta Testing

 6. Remote Testing

 7. Comparative Testing

 8. Benchmark Testing

 9. Formative Testing

 10. Summative Testing

USABILITY TESTING
- Purpose: Evaluate overall usability and user-friendliness
- Process:
1. Define test objectives and metrics
2. Recruit representative users
3. Create realistic tasks
4. Conduct testing sessions
5. Analyze results and report findings
- Focus areas:
- Task completion rates
- Time on task
- Error rates
- User satisfaction scores
- Example: Testing the checkout process of an e-commerce
website
EXPLORATIVE TESTING

- Purpose: Explore user behaviors and preferences openly

- Process:
1. Set up a minimally structured environment
2. Provide users with general goals rather than specific
tasks
3. Observe natural user interactions and decision-making
4. Conduct post-test interviews for deeper insights
- Focus:
- Understanding user mental models
- Identifying unexpected use patterns
- Discovering new feature ideas
- Example: Allowing users to freely explore a new social
media app's features
A/B TESTING

- Purpose: Compare multiple versions of a product or

feature
- Process:
1. Identify the element to test (e.g., button color,
layout, copy)
2. Create two or more variations
3. Randomly assign users to different versions
4. Collect and analyze performance data
- Focus:
- Conversion rates
- Click-through rates
- User engagement metrics
- Example: Testing two different headlines on a landing
page to see which generates more sign-ups
ACCESSIBILITY TESTING
- Purpose: Evaluate accessibility for users with
disabilities
- Process:
1. Define accessibility standards (e.g., WCAG 2.1)
2. Recruit users with various disabilities
3. Test with assistive technologies (screen readers,
voice control)
4. Evaluate against accessibility checklist
- Focus:
- Keyboard navigation
- Screen reader compatibility
- Color contrast and text sizing
- Alternative text for images
- Example: Testing a government website's forms with
users who have visual impairments
BETA TESTING

- Purpose: Test pre-release version with a broader

audience
- Process:
1. Develop a stable beta version
2. Recruit a diverse group of beta testers
3. Provide clear instructions and feedback channels
4. Monitor usage and collect bug reports
5. Iterate based on feedback
- Focus:
- Real-world performance
- Bug identification
- User satisfaction and feature requests
- Example: Releasing a beta version of a mobile game to
a select group of players before the official launch
REMOTE TESTING
- Purpose: Conduct testing with geographically distant
participants
- Process:
1. Choose appropriate remote testing tools
2. Recruit diverse, geographically dispersed
participants
3. Provide clear instructions for setup and tasks
4. Use screen sharing and recording software
5. Conduct post-test interviews via video call
- Focus:
- Gathering insights from a global user base
- Testing in various network conditions
- Evaluating performance across different devices
- Example: Testing a cloud-based productivity tool with
users from different countries and time zones
COMPARATIVE TESTING
- Purpose: Compare product with competitors
- Process:
1. Identify key competitors
2. Define comparison criteria (e.g., ease of use,
features, performance)
3. Design tasks that showcase differences
4. Have users test both your product and competitors'
5. Gather quantitative and qualitative feedback
- Focus:
- Relative strengths and weaknesses
- Unique selling points
- Areas for improvement
- Example: Comparing the user experience of your photo
editing app against two major competitors
BENCHMARK TESTING

- Purpose: Establish baseline usability and performance

metrics
- Process:
1. Define key performance indicators (KPIs)
2. Design standardized tasks and scenarios
3. Conduct initial testing to establish baseline
4. Repeat tests at regular intervals or after major changes
5. Track progress and identify trends
- Focus:
- Task success rates
- Time-on-task measurements
- User satisfaction scores
- Error rates
- Example: Conducting quarterly usability tests on an
airline's booking system to track improvements over time
FORMATIVE TESTING
- Purpose: Gather early feedback to inform
improvements
- Process:
1. Create low-fidelity prototypes or wireframes
2. Recruit a small group of representative users
3. Conduct quick, iterative testing sessions
4. Gather immediate feedback on concepts
5. Rapidly implement changes between sessions
- Focus:
- Early-stage concept validation
- Iterative design improvements
- User preferences and expectations
- Example: Testing paper prototypes of a new mobile
app interface with potential users
USER TESTING METHODS
REMOTE VS. IN-PERSON USER TESTING

 During in-person testing, you’ll be in the same room as the

user while they test your prototype. This has several
advantages. Not only are you able to control the testing
environment and keep distractions to a minimum; you can
also directly observe the user. You are privy to facial
expressions, body language, and any verbal commentary
the user makes as they interact with the product—giving
you valuable, first-hand insight into their experience.
However, in-person testing can be expensive and time-
consuming.
 Remote user testing offers a less expensive, more
convenient alternative, but you’ll have little to no control
over the user’s testing environment. However if you’re one
of the growing numbers now working as remote UX
designers, this kind of user testing makes a lot of sense. If
you’re testing a digital prototype, you can conduct
moderated or unmoderated remote user tests. Let’s explore
each of these options now.
MODERATED VS. UNMODERATED USER
TESTING

 Moderated remote user testing is a good middle ground

between in-person tests and completely unmoderated remote
tests. Live remote testing allows you to observe your users
over a video call, for example. You can use a screen recording
app to capture the test, and certain programs will also track
and highlight where the user clicks in your digital prototype.
 Unmoderated tests can be conducted via user testing
platforms such as UserZoom, loop11, and usertesting.com.
If you’re short on time, such tools make it easy to conduct user
tests quickly and with minimal effort. However, you won’t
have the opportunity to observe the users or ask them
questions.
 Whether you choose to conduct in-person or remote user tests
all depends on your budget, time constraints, and the
prototype you’re testing. Paper prototypes are best tested in
person, while digital prototypes can be tested both remotely
and in-person.

 USER TESTING METHODS
 Moderated Testing: In moderated testing, a facilitator (moderator)
guides participants through a series of predefined tasks while
observing their interactions with the product. The moderator can ask
questions, probe for insights, and ensure a controlled testing
environment.
 Use Cases: This method is valuable for in-depth, qualitative insights,
especially when you want to understand user thought processes and
gather detailed feedback. It’s suitable for identifying usability issues,
evaluating prototypes, and testing specific features.
 Unmoderated Testing: Unmoderated testing involves participants
independently using the product without a moderator’s presence.
Participants follow predefined tasks, and their interactions are
recorded using specialized software.
 Use Cases: Unmoderated testing is cost-effective and efficient for
collecting quantitative data from a larger number of participants. It’s
suitable for remote testing, A/B testing, or when a facilitator’s
presence is not feasible.
 Thinking-Aloud Testing: In thinking-aloud usability testing,
participants vocalize their thoughts, feelings, and reactions as
they navigate the product and complete tasks. The goal is to
gain insights into users’ cognitive processes.
Use Cases: This method is excellent for understanding user
decision-making, uncovering usability issues, and improving
user interfaces. It’s particularly useful for testing the
intuitiveness of navigation and features.

 Remote Usability Testing: Remote usability testing allows

participants to test a product from their own location using
screen-sharing or recording software. Researchers provide tasks
and guidelines remotely.
Use Cases: Remote usability testing is convenient and cost-
effective, making it suitable for gathering insights from a
geographically dispersed user base. It’s often used for testing
websites, apps, or digital products.
 Card Sorting: Card sorting involves participants organizing content
or features into categories or groups based on their mental models.
Researchers analyze how users structure information.
Use Cases: This method helps in optimizing information architecture,
navigation menus, and content organization. It’s valuable during the
early design phase to ensure that the product’s structure aligns with
user expectations.

 First Click Testing: First click testing focuses on the first action
users take when presented with a specific task or interface element.
It helps evaluate the effectiveness of the initial interaction.
Use Cases: This method is useful for assessing the clarity of calls-to-
action, menu labels, or navigation paths. It helps ensure that users
can find what they’re looking for with minimal effort.

 Heuristic Evaluation: Heuristic evaluation involves usability

experts assessing a product against a set of established usability
heuristics or principles. They identify potential usability issues based
on their expertise.
Use Cases: This method is valuable for identifying usability problems
early in the design process and
 Mobile Usability Testing: Mobile usability testing focuses
specifically on evaluating the usability and user experience of
mobile applications or mobile-responsive websites.
Use Cases: With the increasing use of mobile devices, this type
of testing is crucial to ensure that mobile apps and websites are
user-friendly and functional on various screen sizes and devices.

 Five-Second Test: In a five-second test, participants are shown

a screen or interface for five seconds and then asked questions
about what they remember. This method helps assess the clarity
of important visual elements and messaging.
Use Cases: It’s useful for testing the impact of first impressions,
branding, and the visibility of critical information.
 Reference Testing: Preference testing focuses on gathering user
preferences and feedback regarding design elements, features, or options.
Participants express their preferences among different design variations.
Use Cases: It helps in making design decisions based on user preferences,
such as choosing between multiple interface designs or color schemes.

 Rapid Iterative Testing and Evaluation (RITE): RITE is an iterative

approach to usability testing where changes and improvements are made to
the product between test sessions. It involves quick cycles of testing and
refining.
Use Cases: RITE is beneficial when rapid improvements are required or
when addressing critical usability issues during the development process.

 Tree Testing:
Tree testing evaluates the effectiveness of a product’s information
architecture and navigation structure by having participants complete
tasks that involve finding specific pieces of content or information within a
text-based structure.
Use Cases: It helps ensure that users can efficiently locate content or
information in the product’s hierarchy.
 HOW TO CONDUCT USER TESTING
Set an Objective: Define a clear goal. What do you want to learn? Example:
For an ecommerce app, test how easily users add items to the wishlist.
Build a Prototype: Based on the stage of testing, use:
Low-fidelity prototypes for early idea testing.
Mid/high-fidelity prototypes for detailed aspects like microcopy or information
architecture.
Create a Plan: Include:
Objective
Testing method
Number of users
Equipment needed
How to document findings
A script if necessary
Recruit Participants: Ensure participants match your target audience for
relevant insights.
Gather Equipment: Prepare tools like screen recorders, note-taking
materials, and the prototype.
Document Findings: Record results during each session for proper analysis
later.
COLLECTING AND ANALYZING FEEDBACK
 Observing users interact with a prototype can help
understand its usability. Here are some key things to keep
an eye out for:
 Intuitive use: Observe if users can naturally navigate the
prototype without guidance. For example, do they struggle
to find the Submit button in your app or is it obvious?
 Error frequency: Note how often and where users make
errors, such as entering incorrect data in a form field.
 Non-verbal cues: Pay attention to body language and
facial expressions for signs of confusion or frustration.
 Interaction duration: Measure how long it takes to
complete tasks. Longer times may indicate usability issues.
 Workarounds: Notice if users create their own solutions,
indicating a gap in your design.
 Pro Tip! If possible, record the session for further analysis,
ensuring you don't miss subtle interactions.
 Encourage users to be vocal

Assure users that their honest opinions are valued and

there are no right or wrong answers. Next, ask users to
verbalize their thoughts and actions as they navigate the
prototype. For example, "I'm looking for the settings button
now," or "I expected this tab to show my profile." If users
become silent, gently prompt them with questions like
"What are you thinking about right now?" or "Can you
describe what you're trying to do?"
Remind them throughout the session that all feedback,
including negative or critical, is helpful.
Ask follow-up questions
 Clarify responses: If a user's comment is vague or
general, ask for specifics. For example, if they say, "This
feature is confusing," follow up with, "Could you tell me
what specifically about this feature is confusing to you?"
 Probe for reasons: When users express a preference or
dislike, delve deeper into their reasoning. Ask, "What
makes you like/dislike this aspect?"
 Encourage storytelling: Invite users to describe
scenarios in which they might use the product, asking
questions like, "How would you use this feature in your
daily routine?"
 Avoid leading questions: Frame queries neutrally to
avoid biasing responses. For instance, instead of asking "Do
you think this layout makes finding information easier?"
use a more neutral question like "How does this layout
affect your ability to find information?”

Stay away from biases

Let the user interact with the prototype without

interruption. For example, if a user struggles with a
feature, resist the urge to explain it. Instead, ask about and
note the difficulty they're experiencing.

Also, maintain a neutral demeanor. Avoid nodding or shaking

your head in response to user actions, as this could subtly
influence their behavior. Most importantly, save your
questions and comments for after the user has completed
their interaction. This prevents your input from altering
their natural usage patterns.
5 BENEFITS OF ITERATIVE DESIGN AND
PROTOTYPING

1. Greater Efficiency and Faster Time to Market

 Iterative design and prototyping is typically more efficient than a
traditional design process because creating new designs and
prototypes is fast and simple. The initial design process only lasts a
few days to a few weeks depending on the complexity of the design.
The goal is to get a working prototype of the design as quickly as
possible so that engineers can identify and fix potential mechanical
problems, material challenges, or other details that can’t be easily
foreseen during the design stage.

 However, although the initial design phase is relatively short,

engineers actually spend more time in total on the design when they
use an iterative design process compared to a traditional one. That’s
because the design phase never truly ends until the product is ready
for manufacturing. For example, if an aspect of the design isn’t
working, engineers can create a new design iteration and prototype in
as little as a single day. It speeds up the design and prototyping
processes simultaneously, allowing you to get your product to market
faster.
 2. Lower Product Development Costs
 Iterative design and prototyping relies on cost-effective
tools like CAD software and rapid prototyping technology,
such as 3D printers or CNC machines. These tools make it
easy to produce multiple prototypes at relatively little cost.
This is often more cost-effective than pooling most of your
resources and labor into a single prototype, especially if
that prototype ultimately doesn’t meet your end-users’
needs. You’ll also spend less time overall on the product
development process, which saves labor costs and speeds
up ROI.
3. Thorough Product Testing
 One advantage of iterative design and prototyping that can’t
be overlooked is its impact on the quality of the products you
create. This process relies on thorough testing and feedback
with every new iteration. You’ll know exactly which design
details work and which don’t. This makes it much more likely
that your end-users will enjoy your final product and you
won’t have to recall defective products after it’s on the market.

4. Fewer Redesigns
 A full redesign slows your product development process down
significantly and adds to the total cost of the project. Iterative
design helps prevent this by encouraging designers and
engineers to iron out serious flaws in the design as early as
possible. In a traditional design process, you might not catch
certain flaws until after the prototype is complete, and by this
time you’d have wasted weeks or even months on a design that
was flawed from the start. When you find and avoid major
issues in the first few days or weeks of the design process, you
can spend the remaining weeks or months making minor
adjustments to the design that increase the quality of the
product.
 5. More User-Friendly Products
 Because end users are typically involved much earlier in the
process, you’re more likely to create an end product that they
actually find useful. Sometimes it’s quite difficult to know what
end users’ pain points are, and if you overlook these pain points,
your product will be less effective. Asking them for feedback
throughout the design and prototyping process can inform your
design and give you an edge in your industry.
REFINING PROTOTYPES
 Importance of Refinement:
Refining a prototype involves improving upon initial ideas
based on feedback, testing, and evaluation.
 Key Steps:
 User Feedback: Gather insights on usability and
functionality.
 Iteration: Make incremental changes to address issues
or enhance the user experience.
 Test and Evaluate: Repeated testing ensures the
prototype is heading in the right direction.
 Benefits of Refining Prototypes
 Improves User Experience: Ensures the final product
better aligns with user needs.
 Identifies Flaws Early: Catches design flaws before full-
scale development.
 Cost-Effective: Saves time and resources by minimizing
changes during later stages.
 Increases Innovation: Encourages creativity through
continuous improvements.