Software Testing
Books:
1. Software testing: Principles and practices
By: Srinivasan Desikan and Gopalaswamy Ramesh,
Pearson Education
2. Software testing in the real world
By: Edward Kit, Pearson Education
3. System validation and verification
By: Jeffrey O. Grady, CRC Press
4. Software testing: A Craftsman Approach
By: Paul C. Jorgensen, CRC Press
1
Overview
In a typical testing project, someone gives you a program and
asks you to test it.
• The program may (or may not) come with a specification and
documentation.
Documents vary in accuracy and completeness.
You create tests, run them, and report the progress and
results of your work.
New versions of the program come to you. You might
reuse some old tests, check whether bugs have been
fixed, and (or) try new tests.
Eventually, the product is either cancelled or put into production.
2
What is testing?
A technical investigation of the product under test conducted to
provide stakeholders with quality-related information.
Defining Testing
A technical
means including experimentation, logic, mathematics, models,
tools (testing-support programs), and tools (measuring instruments,
event generators, etc.)
investigation
An organized and thorough search for information.
This is an active process of inquiry. We ask hard questions
(run hard test cases) and look carefully at the results.
3
of the product under test
• The product includes the data, the documentation, the
hardware, whatever the customer gets. If it doesn’t all work
together, it doesn’t work.
stakeholders
Someone who has a vested interest in the success of the testing e
Someone who has a vested interest in the success of the product.
with quality-related information
The information of interest is often about the presence
(or absence) of bugs.
Other types of information are sometimes more vital to the
stakeholders
4
Information Objectives
Find important bugs, to get them fixed
Assess the quality of the product
Help managers make release decisions
Block premature product releases
Help predict and control costs of product support
Check interoperability with other products
Find safe scenarios for use of the product
Assess conformance to specifications 5
Certify the product meets a particular standard
Ensure the testing process meets accountability standards
Minimize the risk of safety-related lawsuits
Help clients improve product quality & testability
Help clients improve their processes
Evaluate the product for a third party
6
The six essentials of software testing:
Software Testing: Means by which people, methods,
measurements, tools and equipments are integrated to test the
software product.
1. The quality of the test process determines the success of
test effort:
Quality and effectiveness of software testing are determined
primarily by the quality of test processes used.
2. Prevent defect migration by using early life cycle testing
techniques:
More than half the errors are usually introduced at the
requirement phase.
Effective test program prevents the migration of errors from
any development phase to any subsequent phase.
7
3. The time for software testing tool is now:
Testing tool for every platform:
Capture/play back tools,
Tools for client/server,
Structural coverage, Etc.
4. A real person must take responsibility for improving the test
process:
Tools must be used with test process including effective test planning
and design.
To improve the testing process someone must plan and manage the
progress.
5. Testing is a professional discipline requiring trained, skilled
people:
It is not entry level job or stepping stone to other thing.
It is a proffession.
8
6. Cultivate a positive team attitude of creative
destruction:
Testers supply the information like:
Can we make this product fail to do what we expect it to do?
It is going to break as we can always find errors.
But is it breaking at reasonable boundaries?
Is it stressing or breaking acceptably or unacceptably,
given its criticality?
Do we know that we have covered all the important
possibilities?
After addressing all the questions only we can release the
product 9
Objective of testing
• To show that a product do what it should’t
do.
• To show that a product does’t do what it
should
10
The State of art and the state of practice
SW testing yet to become fundamental component of
university syllabus
Training in the industry is there but not properly defined
Published papers giving state of art methods are unproven in
the field (many well proven methods are unused in industry)
and omit real world dimension like return on investment.
11
History Aspects:
Early days:
Testing was “debugging” (fix the bugs), performed by
developers.
Few dedicated resources for testing that too were very late in
the development process.
1957:
Testing distinguished from debugging.
Testing was still an after development activity.
1970:
Software engineering term used more often.
1972:
First formal conference on testing was held at the University
of North California.
12
1979:
Myers defined testing as “The process of executing a
program with the intent of finding errors”.
BUT IN THE REAL WORLD OF INDUSTRY: Testing got
dumped when the schedules and the budget becomes tight.
Testing –too late to be done properly
1980:
“Quality” was buzzword
Various groups formed some standards like:
IEEE (Institute of Electrical and Electronics engineers)
ANSI (American National Standards Institute)
ISO (International Standards Organization)
1990:
Testing tools came.
13
Where are we now?
1980:
SEI(S/w Engineering Institute) at Carnegie Mellon University
developed CMM for US dept. of defense. 1987: 5 level CMM was
proposed to measure level of s/w maturity.
75% of world s/w development is at level 1.
Very few are at level 5.
How should testing be positioned?
Early testing is the key to success of testing effort.
Testing has its own life cycle.
Testing is still subordinate to development.
Result: Little or no action based on tester’s input.
14
Now things are improving.
Establishing a practical perspective
Ultimate goal of testing: Quality Maintenance and Satisfaction
of customers.
What and why?
Mistake: A human action that produces incorrect result.
(Mistake produces fault or bug)
Fault: An incorrect step, process or data definition in a computer
program. The outgrowth of a mistake. (Potentially leads to
a failure)
Failure: An incorrect result. The result of the fault.
Error: The amount by which the result is incorrect
15
Where are errors?
Errors are concentrated in earlier stages of development
process. (Fig. 1.1)
Design, 27%
Design
Code
Requirements, Other
56% Code, 7% Requirements
Other, 10%
Fig. 1.1 Defect Distribution
16
The cost of errors:
All are costly.
Most expensive: Undetected errors and errors detected late.
Undetected: Migrate to other stages and cause failure in the field fin
Historical definitions of testing:
2. Establishing confidence that a program does what it is
supposed to do (Hetzel, 1973).
2. The process of executing a program or system with the intent of
finding error (myers 1979)
3. Detecting specification errors and deviations from the
specification.
4. Any activity aimed at evaluating an attribute or capability of a
program or system (hetzel, 1983)
17
5. The measurement of software quality (hetzel, 1983)
6. The process of evaluating a program or system.
• Verifying that a system satisfies its specified requirement or
identifying difference between expected and actual results.
• Confirming that a program performs its intended function
correctly.
The IEEE/ANSI definitions of testing
3. The process of operating a system or component under specified
conditions, observing or recording the result and making an
evaluation of some aspect of the system or component.
2. The process of analyzing a software item to detect the difference
between existing and required conditions and to evaluate the
features of the software items.
18
