Black Box Testing
04/20/20 Black Box Testing Slide : 2
What is Black Box Testing?
Black box testing is done without the knowledge of the
internals of the system under test.
04/20/20 Black Box Testing Slide : 3
Characteristics of Black Box
Testing
Done based on requirements
Addresses (should address) stated as well as implied
requirements
Encompasses the end-user perspective
Checks for valid and invalid conditions / inputs
May or may not know the technology aspects of the
product
04/20/20 Black Box Testing Slide : 4
Differences Between White Box and
Black Box Testing
Black Box Testing White Box Testing
Has no access to program Has access to program
code code
Requires external Requires knowledge of
perspective program code
Set of techniques Typically applies only to
applicable to all other unit testing, where code is
phases of testing involved
04/20/20 Black Box Testing Slide : 5
Principles of Black Box Testing
It is not possible to exhaustively test a product
Choose tests so that we can
– Test “as much of the external functionality as
possible”
– Uncover “as many defects as possible”
– Use “as short a time as possible”
Have methodologies that choose
tests that have
a higher likelihood of uncovering
new defects.
04/20/20 Black Box Testing Slide : 6
Techniques / Methodologies of
Black Box Testing
Requirements-based testing
Positive and negative testing
Boundary value analysis
Decision tables
Equivalence partitioning
State-based testing
Compatibility testing
User documentation testing
Domain testing (leads to ad hoc testing)
04/20/20 Black Box Testing Slide : 7
General Format of Discussion of
Techniques
Present some reasoning where applicable.
List out one or two examples.
Walk through the examples.
Summarize the process for using the
technique.
04/20/20 Black Box Testing Slide : 8
Requirements-Based Testing
Done to ensure that all requirements in SRS are tested
Difference between implicit and explicit requirements
Review requirements first to ensure they are
consistent, correct, complete and testable
Review enables translation of (some of) the implied
requirements to stated requirements
A reviewed SRS tabulates requirements, along with a
requirements id and a priority
This is the genesis of a Requirements Traceability
Matrix
04/20/20 Black Box Testing Slide : 9
Positive and Negative Testing
Positive testing to check that the product does what
it is supposed to
– Behaves correctly when given right inputs
– Maps to a specific requirement
– “Coverage” is defined better
Negative testing to show that the product does not
fail when given unexpected inputs
– Tries to break the system
– No direct mapping to a specific requirement
– “Coverage” more challenging
Table 4.4 and 4.5, Page 83
04/20/20 Black Box Testing Slide : 10
Boundary Value Analysis
Most defects come up near “boundaries”
Reasons from a white box perspective:
– Programmers tentativeness in using the right relational
operator (< or < = ?)
– Multiple ways of implementing loops
– Confusing array subscripts
Reasons from a black box perspective:
– Unclear requirements
– Ambiguous or “it depends” mindset!
04/20/20 Black Box Testing Slide : 11
Boundary Value Analysis (BVA):
Summary of Process
Look for any kind of gradation or discontinuity in data values that
affects computation – the discontinuities are the boundary values,
requiring thorough testing.
Look for any internal limits like limits on resources (like the
example of buffers given above). The behaviour of the product at
these limits should also be the subject of boundary value testing.
Also include in the list of boundary values, documented limits on
hardware resources. For example, if it is documented that a
product will run with minimum 4MB of RAM, make sure you
include test cases for the minimum RAM (i.e., 4MB in this case).
The examples given above discuss boundary conditions for input
data – the same analysis needs to be done for output variables
also.
04/20/20 Black Box Testing Slide : 12
Decision Tables
A program’s behaviour is characterized by several
decision variables.
Each decision variable specifies a boolean condition.
The distinct combinations of these decision variables
lead to different scenarios.
Each scenario occupies a row in the decision table,
and the row also has expected results.
One representative data point from each scenario
needs to be tested.
04/20/20 Black Box Testing Slide : 13
Summary of Decision Table
Identify the decision variables.
Identify the possible values of each of the decision variables.
Enumerate the combinations of the allowed values of each of the
variables.
Identify the cases in which values assumed by a variable (or by
sets of variables) are immaterial for a given combination of other
input variables. Represent such variables by the Don’t Care
symbol.
For each combination of values of decision variables
(appropriately minimized with the Don’t Care scenarios), list out
the action or expected result.
Form a table, listing on each but the last column one decision
variable. On the last column, list the action item for the
combination of variables in that row (including Don’t Cares, as
appropriate)
04/20/20 Black Box Testing Slide : 14
Equivalence Partitioning (EVP)
Generalization of BVA / decision table
Divide the (potentially infinite) set of values into a
set of equivalence classes or partitions
One element of the class can act as a
representative for the entire class
Results of the test for that one element is
extrapolated to all the other elements
04/20/20 Black Box Testing Slide : 15
Summary of Process for EVP
Choose criteria for doing the equivalence partitioning (range, list
of values, etc.).
Identify the valid equivalence classes based on the above criteria
(number of ranges, allowed values, etc.) .
Select a sample data from that partition.
Write the expected result based on the requirements given.
Identify special values, if any, and include them in the table.
Check to have expected results for all the cases prepared
If the expected result is not clear for any particular test case, mark
appropriately and escalate for corrective actions. If you cannot
answer a question, or find an inappropriate answer, consider
whether you want to record this issue on your log.
04/20/20 Black Box Testing Slide : 16
State- or Graph-Based Testing
Useful for
– Language processors or compilers
– Workflow modelling
– Dataflow modelling
04/20/20 Black Box Testing Slide : 17
Process for Language processors
Testing Using Graph Method
Identify the grammar for the scenario. In the above
example, we have represented the diagram as a
state machine. In some cases, the scenario can be a
context-free grammar, which may require a more
sophisticated representation of a “state diagram”.
Design test cases corresponding to each valid state-
input combination.
Design test cases corresponding to the most
common invalid combinations of state-input.
04/20/20 Black Box Testing Slide : 18
Requisition Approval
Approving
Authority Finance
Requestor
Budget Update &
y
Purchase Order
er
eli v
Delivery
D
m
Available Ite
Payment
Buy &
Note
Off the shelf Expense
(Transaction:
Expense Report)
Received
Item
Vendor
04/20/20 Black Box Testing Slide : 19
Workflow Modelling
Manager Feasible
Employee Verify
Ensure
Desires Leave Eligibility Approve
Feasibility
Eligible
Leave
Not Feasible
Application
Form Ineligible
Reject
04/20/20 Black Box Testing Slide : 20
General Characteristics of Graph-
Based Testing
The application can be characterized by a set of
states.
The data values (screens, mouse clicks, etc.) that
cause the transition from one state to another are
well understood.
The methods of processing within each state to
process the input received are also well understood.
04/20/20 Black Box Testing Slide : 21
Compatibility Testing
To ensure that the product works consistently with
infrastructure components
Could be parameters of hardware, OS, network…
Concept of a compatibility matrix (Table 4.11, Page 98)
Backward compatibility and forward compatibility
04/20/20 Black Box Testing Slide : 22
User Documentation Testing
To check if what is stated in the document is available
in the product
To check if what is there in the product is available in
the document
Documents include user guides, installation guides,
set-up guides, read-me files, software release notes
and on-line help
04/20/20 Black Box Testing Slide : 23
Benefits of Documentation Testing
User documentation testing aids in highlighting
problems that have been overlooked during reviews.
High-quality user documentation ensures the
consistency of documentation and product, thus
minimizing possible defects reported by customers.
Results in less difficult support calls.
Contributes to better customer satisfaction and better
morale of support staff.
New programmers and testers who join a project
group can use the documentation to learn the external
functionality of the product.
Customers need less training and can proceed more
quickly to advanced training and product usage.
Reduced training costs for customers.
04/20/20 Black Box Testing Slide : 24
Domain Testing
04/20/20 Black Box Testing Slide : 25
Domain Testing
Have “domain people” perform tests by using the
software
Business flow determines the test, not “logic” or
“steps”
Tests what the users do on a typical business day
Captures the typical problems faced by users (not
necessarily captured in SRS)
04/20/20 Black Box Testing Slide : 26
When to Use What…
When you want to test scenarios that have… The most effective black box testing technique
is likely to be…
Output values are dictated by certain Decision Tables
conditions depending upon values of
input variables
Input values being in ranges, with each range Boundary Value Analysis
exhibiting a particular functionality
Input values being divided into classes (like Equivalence Partitioning
ranges, list of values, etc), with each class
exhibiting a particular functionality
Positive and negative testing
Workflows, process flows or language Graph based testing
processors
To ensure that requirements are tested and Requirements Based Testing
met properly
To test using the domain expertise rather than Domain Testing
the product specification
To ensure that the documentation is Documentation Testing
consistent with the product
04/20/20 Black Box Testing Slide : 27
