Basics of Software Testing

Specification-based or Black Box

Techniques
Specification-based or Black Box Techniques
● Why do we need black box test techniques?
○ Exhaustive testing is not possible
■ Due to the constraints of time, money and resources
○ Therefore we must create a sub-set of tests
■ There must be achievable, but not reduce coverage
○ We should also focus on areas of likely risk
■ Those places where mistakes may occur
Specification-based or Black Box Techniques
● Each black box test technique has
○ A method
■ I.e. how to do it
○ A test case design approach
■ How to create test cases using the approach
○ A measurement technique
■ Except Random & Sintax
○ See BS7925-2 for detailed information
Black box test techniques
● Equivalence Partitioning
● Boundary Value Analysis
● Decision Table (Cause Effect Graphing)
● State Transition
● Use Case Testing
● Syntax Testing
● Random Testing
Equivalence Partitioning
● Inputs to the software or system are divided into groups that are
expected to exhibit similar behaviour, so they are likely to be processed in
the same way
● Equivalence Partitions (or classes) can be found for:
○ Both - valid and invalid data (e.g. values that should be rejected)
○ Outputs
○ Internal value
○ Time-related values (e.g. before and after an event)
○ Interface parameters (e.g. during integration testing)
Equivalence Partitioning
● Tests can be designed to cover partitions
● Equivalence Partitioning is applicable at all levels of testing
● IT can be applied to
○ Human input
○ Input via interfaces to a system
○ Interface parameters in integration testing
● Uses a model of the component to partition Input & Output values into
sets
● Such that each value within a set can be reasonably expected to be
treated in the same manner
● Therefore only one example of that set needs to be input to or result
from the component
Equivalence Partitioning - Test Case Design
● Inputs to the component
● Partitions exercised:
○ Valid partitions
○ Invalid partitions
● Expected results
Equivalence Partitioning - Test Case Design
● Example
● Enter car details
○ “Pret” between 0 and 200000
○ “Anul fabricației”
○ “Kilometraj”
● Identify
○ Valid Partitions and Invalid Partitions
○ Test cases including expected results
Test Cases - Expected results
All should work as expected
Specification-based or Black Box Techniques
● Boundary Value Analysis
○ Behaviour at the edge of each equivalence partition is more likely be incorrect so
boundaries are an area where testing is likely to yield defects
● Boundary Values
○ Are the maximum and minimum values of a partition
○ A boundary value for a valid partition is a valid boundary value
○ A boundary value for an invalid partition is an invalid boundary value
Boundary Value Analysis
● Can be applied at all test levels
● Its defect-finding capability is high
● Uses a model of the component to identify the values close to an on
partition boundaries
● Is often considered as an extension of Equivalence Partitioning
● Most useful when combined with Equivalence Partitioning
Boundary Value Analysis - Test Case Design
● Inputs to the component
● Boundaries to be exercised:
○ Jost below
○ On
○ Just above
● Expected results
Boundary Value Analysis - Test Case Design
● Example
○ Enter car details
○ “Anul fabricației”
○ “Kilometraj”
● Identify
○ Boundaries
○ Test cases
○ Test case category
○ Expected results
Specification-based or Black Box Techniques
● What are the valid and invalid values for Equivalence Partitioning and
Boundary Value Analysis?
○ Q1 To be eligible for a mortgage you must be between the ages of 18 and 64 (inclusive).
The age input field will only accept two digits and will not accept minus figures(“-”)
○ Q2 An input field on a mortgage calculator requires a value between 15,000 and
2,000,000. The field only allows numerical values to be entered and has a maximum
length of 9 digits.
○ Q3 The term of a mortgage can be between 5 and 30 years. The field will accept
maximum two digits and will not accept minus figures.
○ Q4 The font formatting box in a word processing package allows the user to select size of
the font - ranging from 6 points to 72 points (in 0.5 steps). The box accepts only numbers
in format ##.#
Specification-based or Black Box Techniques
● What are the valid and invalid values for Equivalence Partitioning and
Boundary Value Analysis?
○ Q5 A screen for entering mortgage applications requires information on both peoples
wages and will generate the maximum amount available for the mortgage (based on 3 ¼
times larger wage, 1 ¼ times lower wage). If the mortgage is less than $250,000 the then
the interest rate is 4.5%, if the amount is $250,000 to $1,000,000 then the interest rate is
4%.
○ Q6 Personal loan of between %1,000 to $25,000. For loans between $1,000 and $10,000
there is an interest rate of 8.5%, loans between $10,001 and $25,000 have an interest rate
of 8%
○ Q7 A grading system takes student marks (coursework 0 - 75 and exam 0 - 25) and
generates a grade based on these marks (0 - 40 Fail, 41 - 60 C, 61 - 80 B, 81 - 100 A).
Specification-based or Black Box Techniques
● Decision Table Testing
○ Uses a model of the relationship between causes and effects for the component
○ Each cause or effect is expressed as a Boolean condition
○ Represented as a Boolean graph, from which a decision table is produced
● Decision Tables are a good way to:
○ Capture system requirements that contain logical conditions
○ Document internal system design
○ Recording complex business rules that a system is to imprelement
○ Sever as a guide to creating test cases that otherwise might not be exercised
Decision Table Testing
● Contains the triggering conditions, often combinations of true and false
for all input conditions, and the retesting actions for each combination of
conditions
● Each column of the table corresponds to a business rule that defines a
unique combination of conditions, which result in the execution of the
actions associated with that rule
● The coverage standard commonly used with Decision Table Testing is to
have at least one test per column, which typically involves covering all
combinations of triggering conditions
Decision Table Testing
● The strength of Decision Table Testing is that it creates combinations of
conditions that might not otherwise have been exercised during testing
● It might be applied at all situations when the action of the software
depends on several logical decisions
● Decision Table is directly connected with the techniques called Cause
Effect Graphing according to BS7925-2
Cause effect graph
● Establish a relation between the effect (and output) and its cause (inputs)
● 4 basic symbols express the interdependency between inputs and
outputs.
Cause effect graph
File Management example

1: first char is “A”

2: first char is “B”

3: the second value is a number

70: update the file

71: message X12

72: message X13

Cause effect graph - File management example
Test Data Causes Effect

1 2 3 70 71 72

1 0 0 0 0 1 1

2 0 0 1 0 1 0

3 0 1 0 0 0 1

4 0 1 1 1 0 0

5 1 0 0 0 0 1

6 1 0 1 1 0 1
State Transition Testing
● Uses analysis of the specification of the component to model its
behaviour by state transitions
● A system may exhibit a different response depending on current
conditions or previous history (its state)
● Allows the tester to view the software in terms of its
○ States
○ Transitions between states
○ The inputs or events that trigger state changes (transitions)
○ The actions which may result from those actions
State Transition Testing
● The states of the system or object under test are separate, identifiable
and finite in number
● A state table shows the relationship between the states and inputs, and
can highlight possible transitions that are invalid
● Test can be designed to
○ Cover a typical sequence of states
○ Cover every state
○ Exercise specific sequence of transition
○ Test invalid transitions
State Transition Testing
● Is much used within the embedded software industry and technical
automation in general
● It is also suitable for modelling a business object having specific states or
testing screen-dialogue flows (e.g. for internet applications or business
scenarios)
● Identifies
○ The various states that the software may be in
○ The transitions between these states
○ The events that cause the transitions
○ Actions resulting from the transitions
State Transition - Test Case Design
● Identify possible transitions
● Identify initial state and the data/ events that will trigger the transition
● Identify the expected results
State Transition Testing - Testing

Test Case 1 2 3 4 5 6

Start State S1 S1 S3 S2 S2 S4

Input CM R TS CM R DS

Expected
D AT T T AD D
Output

Finish State
S2 S3 S1 S1 S4 S2
Use Case Testing
● Tests can be specified from use cases or business scenarios
● Use Case definition
○ Describes interactions between actors, including users and the system, which produce a
result of value to a system user
Use Case
● Has preconditions, which need to be met for a use case to work
successfully
● Terminates with post-conditions, which are the observable results and
final state of the system after the use case has been completed
● Usually has a mainstream scenario, and sometimes alternatives branches
Use Case
● Describes the “process flows” throughout a system based on its actually
useful in uncovering defects in the process flows during real-world use of
the system
● Is very useful for designing acceptance tests with customer/user
participation
● Helps uncover integration defects caused by the interaction and
interference of different component, which individual component testing
would not see
Use case example
List

Withdraw

Search

Bid

Seller
Retract Buyer
Use case example
Use Case Name Bid on an Item

Context of Use Buyer wants to purchase an item

Scope System

Success End Conditions Bid has been processed against the items

Failed End Conditions Bid has not been processed: item and its current bid price remain
unchanged

Actions 1) Buyer enters a “Bid on item” request

2) System bids up the current Bid by looping through all active
bidders on this item
3) System waits for BidTimer to expire
4) System notifies winning and losing bidders
Other Specification-based Techniques
● Syntax Testing
○ Uses the syntax of the components inputs as the basis for the design of the test cases
● Random Testing
○ Means of testing functionality with a randomly selected set of values
Summary
● Black box testing concentrates on testing the features of the system
● Techniques enable us to maximise testing
○ Create an achievable set of tests that offer maximum coverage
○ Ensure possible areas of risk are tested
● Black box testing is relevant throughout the testing process