Introduction to Software Testing

Most Common Software problems

• Incorrect calculation
• Incorrect data entries
• Incorrect matching and merging of data
• Data searches that yields incorrect results
• Incorrect processing of data relationship
• Incorrect coding / implementation of business rules
• Inadequate software performance
• Confusing or misleading data
• Inconsistent processing
• Incorrect or inadequate interfaces with other systems
• Inadequate performance and security controls
• Incorrect file handling
 Software Testing

Testing is the process of evaluating a system

(by manual or automatic means) with the
intent of finding errors prior to delivering it
to the end user
 Software Development Process Cycle
•Software Testing is an integral part of software development process, which comprises 4
components:

PLAN (P) – Devise a Plan. Define objective and determine the strategy and supporting
methods required to achieve that objective.
DO (D) – Execute the Plan. Create the conditions and perform the necessary training to
execute the plan.
CHECK (C) – Check the Results. Check to determine whether work is progressing according
to the plan and whether the expected results are obtained.
ACT (A) – Take the necessary Action. If the check up reveals that the work is not being
performed according to plan or that results are not what was anticipated, devise measures
for appropriate action
 Need of Testing
• To point out the defects and errors that were made during the development
phases.
• Software Testing is itself an expensive activity but launching a software without
testing may lead to cost potentially much higher than that of testing.
• Testing is necessary in order to provide the facilities to the customers like the
delivery of high quality product or software application which requires lower
maintenance cost and hence results into more accurate, consistent and reliable
results.
• It makes sure of the Customer’s reliability and their satisfaction in the application.
• Testing is required for an effective performance of software application or product.
• Something that works when one person is using it may or may not work when
hundreds of people are using it (100’s of people hitting a website at the same time
can lead to crashing down)
• There are lots of different devices, browsers and operating systems available (Test
it for compatibility)
 Who should do the Testing?
• It depends on the process and associated stakeholders of the project. In IT
industry, large companies have separate team. In most cases, following
professionals are involved:
•Software Tester

•Software Developer

•Project lead/ Manager

•End User

•Their objective is:

• To find bugs as early as possible and fix them

• Study the functionality in detail to find where bugs are likely to occur

• Study the code to ensure each and every line of code is tested

• Create test cases in such a way that testing is done to uncover the hidden

bugs and also ensure that software is usable and reliable.

 Basic Concepts
•ERROR: Synonym is mistake. An error is a terminology of developer. It is a mistake, syntax error, or
misunderstanding of specifications by a software developer. For example, a developer may
misunderstand a design notation, or a programmer might type a variable name incorrectly – leads to
an Error.
•BUG: Terminology of tester. It is the unknown mistake made by the programmer and is found in the
development environment before the product is shipped to customer. Programming error that
causes a program to work poorly, produce incorrect results or crash.
•FAULT or DEFECT: A Defect is a deviation from the Requirements. A Software Defect is a condition in
a software product which does not meet a software requirement (as stated in SRS) or end-user
expectations. In other words, a defect is an error that is found after the product goes into
production. This could be hardware, software, network, performance, format, or functionality.
•FAILURE: When a defect reaches the end customer then it is called failure e.g. software installation
fails
•Testbed: The test execution environment configured for testing. It is a method of testing a particular
module (function, class, or library) in an isolated fashion. A typical testbed could include software,
hardware, and networking components.

Conclusion:
•A Bug is the result of a coding Error and A Defect is a deviation from the Requirements.  A defect
does not necessarily mean there is a bug in the code, it could be a function that was not
implemented but defined in the requirements of the software.
 What should We
•
Test ?
We should test the program’s responses to every
possible input. It means, we should test for all
valid and invalid inputs.
• Suppose a program requires two 8 bit integers as
inputs.
• Total possible combinations are 28 * 28.
• We have also not considered invalid inputs
where so many combinations are possible.
Hence, complete testing is just not possible,
although, we may wish to do so.
 Test, Test Case and Test
Suite
• Test and Test case terms are used interchangeably. Test case is a document which
consists of a set of inputs, preconditions and expected outcomes developed for a
particular objective such as to verify compliance with a specific requirement.

• The process of developing test cases can also help find problems in the requirements
or design of an application.

• The set of test cases is called a test suite. Hence any combination of test cases may
generate a test suite.
Test Test case Pre- Test Test Expected Actual Status Created Date of Executed Date of
case description conditions steps Data Result Result by creation By execution
ID

Pre-conditions which need to Data required while Pass/Fail

be executed before starting executing the test
the test

Fig: Test case template

Test Case (contd.)
Test Case (contd.)
 Test Case
 Test Case is a simple pair (contd.)
of
<input, expected outcome>

 State-less systems: A compiler is a stateless system

 Test cases are very simple
 Outcome depends solely on the current input

 State-oriented: ATM is a state oriented system

 Test cases are not that simple. A test case may consist of a sequences of
<input, expected outcome>
 The outcome depends both on the current state of the system and the current input
 ATM example:
< check balance, Rs. 500.00 >,
< withdraw, “amount?” >,
< amount, Rs. 200.00 >,
< check balance, Rs. 300.00 >
 Examples
1. Basic Test Cases for Fan
It should have a hook for hanging in the roof.
It should have minimum three blades.
It should be moving once the electricity pass into it.
Speed of the fan should be controlled by the regulator.
 It should be stop once the electric switch off.
The fan should run with minimum noise.
The blades should have proper distance from the ceiling.
The fan while in motion, should not vibrate.
The color of the fan should be dark.
Fan should work in clock-wise direction
 2. Basic Test Cases for Credit Card
• Case 1: Check for invalid Characters in Credit Card.
Description: Enter invalid characters @@@@34534"asd".
Expected Result: Error message should appear informing that invalid value is entered.

Case 2: Check for wrong Credit Card type.

Description: Enter invalid Credit Card type e.g. Enter Am Ex in place of VISA.
Expected Result: Error message should appear informing that invalid Credit Card is entered.

Case 3: Check for wrong Expiry Date.

Description: Select wrong month &amp; year of expiry date.
Expected Result: Error message should appear informing that invalid Expiry date has been
entered.

Case 4: Check for CVV number with the invalid characters as well as with the alphabetic & alpha
numeric values.
Description: Enter invalid CVV number. Like: ABC or a3c. or @@" or "1".
Expected Result: Error message should appear information. Invalid characters are entered.

Case 5: Check for validation messages while enter wrong billing information.
Description: Check for Maximum & Minimum value acceptance. Check for invalid Characters.
Check for Numeric value acceptance where numeric values are required & vice-versa.
Expected Result: Error message should appear while entering invalid values.
 What is a GOOD Test?
Has
Has aa high
high
probability
probabilityofof
finding
findingan
anerror.
error.
ItItmust
mustnot
notbe
be
redundant.
redundant. Every
Every
test
testmust
must have
have aa
different
differentpurpose.
purpose.
ItItshould
shouldneither
neither
be
be too
too simple,
simple,
nor
nor tootoo complex.
complex.
 Objectives Of
Testing
Uncovering different cases of errors.
Validate the quality of a software using the minimum cost, effort and
time.
Showing that software functions work according to specification.
Generate high quality test cases, perform effective tests and issue
correct and helpful problem reports.

It cannot show absence of errors or

defects. It can only show that
errors are present.
 Example
#include<stdio.h>
void main()
{
int a, b, c;
scanf(“%2d”,&a);
scanf(“%2d”,&b);
c=a+b;
printf(“%2d\n”,c);
}
• The program is designed to add
two numbers, which we enter. Each
number should be of one or two
digits. The program will read our
entries and print the sum. Press
<enter> after each number.
 First Cycle of Testing

Step I
•Add 2 & 3
•5 will be displayed
Problem Reports After First
Test
• The program worked in the sense that it accepted 2 &
3, and returned 5. But it still has problems.
• Design Errors:- No On screen Instructions. How do
we know what to do?
Instructions could easily be displayed on the screen.
• The errors are described in problem report form.
Submit one problem report for each error. All errors
could fit on the same report but that is not a good
idea.
Problems that are grouped together might not be
fixed at the same time. The unfixed ones will be lost.
If the developer wants to group them, he /she can
sort the reports himself/ herself.
• Make some notes about what else needs testing.

• Some of the notes may turn into formal test series that is
well-documented groups of tests that may be probably
used each time we test a new version of the program.
 TESTS OF VALID INPUT
Sr. No. Test Case Expected Notes
Results
1. 99 + 99 198 Largest pair of
numbers the
program can add
2. -99 +(-99) -198 The specification
did not say we
cannot use negative
numbers.
3. 99 + (-14) 85 Contd
A large first number might affect
the program’s interpretation of
…the second
4. -38 + 99 61 Check addition of a negative to a
positive number.
5. 56 + 99 155 Large second number’s effect on
first.
6. 9+9 18 9 is the largest one digit number
7. 0+0 0 Programs often fail on zero.
8. 0 + 23 23 0 should be tested in first and
9. -78 + 0 -78 second entry position
 Step III
• Check the invalid cases and see what
happens.

• The test series in previous table only covers

valid values.

• Our next step is to create series like this for

invalid values.
 Step IV
• Always write down what we do and what
happens when we run test case.

• Try any test that seems promising, even if it is

similar to others that have already been run.
 Step V
• Summarize what we know about the program
and its problems.
• Focus on specific issues, such as coming
with boundary conditions for valid inputs.
 Summarize thoughts about the program

• The communication style of the program is very terse.

• The program does not deal with two digit negative

numbers.

• The largest sum that it can handle is 198 and the smallest is 0.

• The program accepts any character as valid

input, until we press <enter>.

• If we type anything that is not a digit it

gives a garbage value.
Problems:-

• Design Issue:- No program title on screen

• Design Issues:- No instruction on screen

• Bug: - Cannot handle two digit negative numbers

• Bug:- if third digit is entered, it is taken as second

number.

• Bug:- Crashes when we enter non numbers

 Testing Versus
Testing Debugging
Debugging

Purpose: To find bug Purpose: To fix the bug

Done by testing team Done by development team

Testing starts with known conditions, uses Debugging starts from possibly unknown initial
predefined procedures and has predictable conditions and the end cannot be predicted
outcomes. except statistically.

Testing can and should be planned, designed Procedure and duration of debugging cannot be
and scheduled. so constrained.

Can be done with design knowledge Impossible without detailed design knowledge

Testing proves a programmer's failure. Debugging is the programmer's vindication

(Justification).
Much of the test execution and design can be Automated debugging is still a dream for
automated programmers
Testing, as executes, should strive to be Debugging demands intuitive leaps,
predictable, dull, constrained, rigid and experimentation and freedom.
inhuman.
 Testing Techniques

• Top-down/ Bottom-up technique

• Box Technique
 Top-down Testing Technique
• In this, a software system is approached as a composition of sub-systems and testing is
carried out at system level and not at sub-system level
• This technique can be used even if one of the sub-systems is not ready by using stubs.
• Stubs are the programs which are written to simulate the sub-systems

System A

Sub-system Sub-system
A1/STUB A2/STUB

• STUB is the replacement of a method that has not yet been implemented or tested
• A stub has the same header as the method it replaces, but its body only displays a
message indicating that stub was called (returns a default result if a return is needed)
• In above diagram, a software system A consists of sub-systems A1 and A2. System A is
tested while A1 and A2 are not tested
 Top-down Testing Technique
Advantages
• Early testing of complete system provides confidence to
the team and customer
• Very useful if there are too many bugs at the top level

Drawbacks
• If sub-system is not ready, stubs need to be written,
which is an additional effort
 Bottom-up Testing Technique
• In bottom up approach, lower level components are created first.
• If the top level system is not ready, a driver is developed to simulate the top level
module
System A/Driver

Sub-system Sub-system
A1 A2

• A driver program declares any necessary object instances and variables, assigns values
to any of the method’s inputs, calls the method, and displays the values of any
outputs returned by the method.
• You can put a main method in a class to serve as the test driver for that class’s
methods.
• Sub-system A1 and A2 are tested. A driver can be used in place of system A, if it is not
yet developed
 Bottom-up Testing Technique

Advantages:
• Good technique if majority of bugs occur at sub-system
level
• Easier to write test cases because sub-systems are small in
size
Drawbacks:
• A driver must be written if the top system is not
developed, leading to additional effort
 Box Technique
• Black Box Testing: It is a software testing methodology
where functionality of the software is examined without
looking into the internals of the application. This
technique is used by the customers and functional
testing team
• White box testing: It is a testing methodology which
looks at the internal working of a system. It is used by
the development team
 Levels of
• There are 3 levels of Testing
testing:
i. Unit Testing
ii. Integration Testing
iii. System Testing
 Unit
Testing
• It is the testing of an individual unit or group of related
units.
• It falls under the class of white box testing.
• It is often done by the programmer to test that the unit
he/she has implemented is producing expected output
against given input
• The size of a single module is small enough that we can
locate an error fairly easily.
 Integration
Testing
The purpose of unit testing is to determine that each independent module is
correctly implemented. This gives little chance to determine that the interface
between modules is also correct, and for this reason integration testing must
be performed. One specific target of integration testing is the interface:
whether parameters match on both sides as to type, permissible ranges,
meaning and utilization.
 Integration
Testing
• Integration Test case differs from other test
cases in the sense it focuses mainly on the
interfaces & flow of data/information
between the modules.
• Priority is to be given for the integrating
links rather than the unit functions which are
already tested
 Integration
Testing
Sample Integration Test Cases for the
following scenario: Application has 3 modules
say 'Login Page', 'Mail box' and 'Delete mails'
and each of them are integrated logically.
Approaches/Methodologies/Strategies of
Integration Testing:
• Big Bang Approach
• Incremental Approach: which is further
divided into following
– Top Down Approach
– Bottom Up Approach
– Sandwich Approach - Combination of Top Down
and Bottom Up
 Integration Testing: Big-Bang Approach
All components (units) are first tested individually and then together as a single and entire
system:
♦Pros:
No test stubs and drivers are needed
♦Cons:
Difficult to pinpoint the specific component responsible for the failure
 Bottom-up Testing
•
Strategy
The subsystem in the lowest layer of the call
hierarchy are tested individually.
• Then the next subsystems are integrated and
tested from the next layer up that call the
previously tested subsystems
• This is done repeatedly until all subsystems
are included in the testing
• Only Test Drivers are used to simulate the
components of higher layers
• No Test Stubs!
Bottom-up
Integration
 Bottom-up Integration

The order of Integration by Bottom-down approach will be:

4,2 5,2 6,3 7,3 2,1 3,1

Testing Approach :

Firstly, Test 4,5,6,7 individually using drivers.

Test 2 such that it calls 4 and 5 separately. If an error occurs we know that the problem is in one
of the modules.

Test 1 such that it calls 3 and If an error occurs we know that the problem is in 3 or in the
interface between 1 and 3.
 Pros and Cons of Bottom-up Integration
Testing
Pros:
•Interface faults can be more easily found (the usage of
test drivers accomplishes a clear intention of the
underlying interfaces of the lower layer)
•No Stubs are necessary
Cons:
•Components of the User Interface are tested last
•Faults found in the top layer may lead to changes in the
subsystems of lower layers, invalidating previous tests.
 Top-down Testing
•
Strategy
Test the top layer or the controlling subsystem
first
• Then combine all the subsystems that are called
by the tested subsystems and test the resulting
collection of subsystems
• Do this until all subsystems are incorporated into
the test
• Test Stubs are used to simulate the components
of lower layers that have not yet been
integrated.
• No drivers are needed
Top-down Integration
Testing
 Top-down Integration
Testing

The order of Integration will be: 1,2 1,3 2,Stub 1 2,Stub 2 3,Stub 3 3,Stub 4

Testing
 Firstly, the integration between the
Approach:
modules 1,2 and 3

 Test the integration between the

module 2 and stub 1,stub 2

 Test the integration between the

module 3 and stub 3,stub 4
 Pros and Cons of top-down
integration testing
Pros:
•Test cases can be defined in terms of the functionality
of the system (functional requirements)
•More effective for finding faults that are visible to the
user
Cons:
•Writing stubs can be difficult: Stubs must allow all
possible conditions to be tested.
•Possibly a very large number of stubs may be required,
especially if the lowest level of the system contains
many methods.
 Sandwich Testing
•
Strategy
Combines top-down strategy with bottom-up strategy
(parallel testing is possible)
• The system is viewed as having three layers: A target layer
in the middle, A layer above the target (top layer), A layer
below the target (bottom layer)
• Testing converges towards the target layer
• No Test Stubs and Drivers are necessary for bottom and
top layer
 Pros and Cons of Sandwich
Pros:
Testing
•Top and Bottom Layer Tests can be done in
parallel
•No Stubs and Drivers (saves development time)
Cons:
•Does not test the individual subsystems on the
target layer thoroughly before integration (C in
the example)
•Solution: Modified sandwich testing strategy
 Modified Sandwich Testing
Strategy
• Tests the three layers individually before combining
them in incremental tests with one another
The individual layer tests consists of three tests:
• Target layer test with drivers and stubs
• Top layer test with stubs
• Bottom layer test with drivers
The combined Layer Tests consist of two tests:
• Top layer accessing target layer (top layer replaces
drivers)
• Bottom accessed by target layer (bottom layer
replaces stubs)
 System
• Testing
It is conducted when all the modules of a system are integrated
together and the system is ready. It is conducted before handing
over the system to the customer for acceptance testing. It includes
testing for performance, security, accountability, configuration
sensitivity, startup and recovery.

• Of the three levels of testing, the system level is closest to

everyday experiences. We test many things; a used car before we
buy it, an on-line cable network service before we subscribe, and
so on. A common pattern in these familiar forms is that we
evaluate a product in terms of our expectations; not with respect
to a specification or a standard. Consequently, goal is not to find
faults, but to demonstrate performance. Because of this we tend
to approach system testing from a functional standpoint rather
than from a structural one.
• During system testing, we should evaluate a number of
attributes of the software that are vital to the user and are
listed in Fig. These represent the operational correctness of
the product and may be part of the software
specifications.
Usable Is the product convenient, clear, and
predictable?
Secure Is access to sensitive data restricted to
those with authorization?
Compatible Will the product work correctly in
conjunction with existing data, software,
and procedures?
Dependable Do adequate safeguards against failure
and methods for recovery
exist in the product?
Documented Are manuals complete, correct, and
understandable?
 Regression testing
• Regression testing is the process of retesting the modified parts of the
software and ensuring that no new errors have been introduced into
previously tested code.

• It is an important component of incremental testing. When a new

increment is added to the system, the tests which have been run on the
previous increment are run again.

• If these tests do not produce the same outputs as before, this implies that
the new increment has, in some way, interfered with the functionality of
the existing system
 Regression testing

Problem:
•Given program P, its modified version P’, and a test set T that
was used to previously test P, find a way to utilize T to gain
sufficient confidence in the correctness of P’
 Who performs Software
Regression?
Developers - regression testing at the unit level
or integration
Test engineers - regression testing at the
function level
QA and test engineers - regression testing at
the system level
 Software Regression
Process
Step #1: Software Change Analysis
-Understand and analyze various software changes.
Step #2: Software Change Impact Analysis
-Understand and analyze software change impacts
Step #3: Define Regression Test Strategy and Criteria
Step #4: Define, select, and reuse test cases to form a regression test suite
Step #5: Perform re-testing at the different levels.
-re-testing at the unit level
-re-testing at integration level
-re-testing at the function level
-re-testing at the system level
Step #6: Report and analyze regression test results
 Different Types of Software
Changes
Requirements Requirements Spec. Changes
analysis -> add new functional features
-> change current function features
-> delete existing function features

System Design System architecture changes

-> change component interactions
-> add new components/subsystems
-> update existing components
-> delete existing components

High-level design doc. Changes

-> change state-based behaviors
-> change component interfaces
-> change database design
-> change GUI design
-> change function design
 Different Types of Software
Changes
System Design Changes - Low-level design doc. Changes
-> change algorithm logic
-> change component structure

- System - Component changes

implementation - internal data types and names
- internal structures, such as
--> class relationships
--> control flow and data flow
- internal functions

- Component interface changes

- call signatures
- message interactions
- protocol messages and
formats

- Technology and/or language changes

 Software Changes Impacts

Types of system Types of product

changes impacts

Requirements changes Affect design, coding, and testing

Document update

Design changes Affect coding and tests

Affect associated components Affect
system architecture
Affect related component
interactions
Implementation changes Affect test cases, test data, test
scripts Affect test specification.
Code change impacts
-
Test Affect other tests.
changes Affect test
documentation
Document changes Affect other documents.
 Regression testing

• Retesting requires

Test suites

Should we use existing test suite?

Should we generate new test suite?
Should we go for both?
 Regression Testing Techniques

• Test suites tend to grow in size as software

evolve

Too costly to execute entire test

suites

What to do?
Minimization Selection Prioritization
 Regression Test
Minimization
• Test suite minimization seeks to eliminate
redundant test cases in order to reduce the
number of tests to run.

• Minimization is sometimes also called ‘test suite

reduction’, meaning that the elimination is
permanent.
 Regression Test
Selection
• Test suite selection seeks to identify test cases
that are relevant to some set of recent
changes.
• Test cases are categorized as:
Reusable test cases- can be used in succeeding
regression cycles
Obsolete test cases- can not be used in
succeeding regression cycles
 Regression Test
Prioritization
• Test case prioritization seeks to order test cases in such a
way that early fault detection is maximized.

• Order the execution of a regression test suite so that the

“best” tests run first.

• Prioritize based on code coverage, fault-exposing potential,

etc.

• Hopefully, defects are revealed earlier in the regression

testing process.
 Selection and Prioritization
• These two techniques can be used in isolation or in conjunction.
• This represents an advanced, but possibly incomplete, solution
to regression testing problem.
 Test Case
Selection
• Objective is to select an appropriate subset of
the test suite to be run.
Test Case
Selection
 Prioritization of Test Cases for Regression
Testing
• Test case prioritization involves scheduling test cases in an order
that increases their effectiveness in meeting some performance
goals.

• Test suit minimization techniques lower costs by reducing a test

suite to a minimal subset that maintains equivalent coverage of
original set with respect to particular test adequacy criterion.
 Test Case Prioritization
•
Methods
General Test Case Prioritization – For a given program, P, and
test suite T, we prioritize the test cases in T that will be useful
over a succession of subsequent modified version of P without
any knowledge of modified version.
• Version specific test case prioritization – Prioritize the test
cases in T, when P is modified to P’, with the knowledge of the
changes that have been made in P.
We consider the version specific regression test selection
algorithm whose objective is to execute the modified lines of
code with maximum number of test cases.
 Example
• Let P be a procedure or program. P’ be a modified version of P. T be a set of
code coverage based tests created to test P.
When P is modified to P’, we have to find T’, which is a subset of T that achieves
maximum code coverage at the earliest and should be given highest priority
during regression testing. Thus, identify the tests that:
a)Execute the code that has been deleted so that test cases that have gone
redundant can be deleted from T.
b)Execute modified code at least once at the earliest.
Consider a hypothetical program of 60 lines of code. There are 10 code
coverage based test cases.
 Execution
History
Test Case no Execution History
T1 1, 2, 20, 30, 40, 50
T2 1, 3, 4, 21, 31, 41,51
T3 5, 6, 7, 8, 22, 32, 42, 52
T4 6, 9, 10, 23, 24, 33, 43, 54
T5 5,9,11,12,13,14,15,20,29,37,38,39
T6 15,16,17,18,19,23,24,25,34,35,36
T7 26,27,28,40,41,44,45,46
T8 46,47,48,49,50,53,55
T9 55,56,57,58,59
T10 3,4,60
 Test Case
Array
• Suppose the lines 1, 2,5,15,35,45,55 are modified
• Simplest is to execute all the test cases that have
any of the modified line of code.
• Selected test cases:- T1, T2, T3, T5, T6, T7, T8, T9
• Total = 8 test cases
Test case Modified Lines No. of Matches
T1 1, 2 2
T2 1 1
T3 5 1
T5 5,15 2
T6 15,35 2
T7 45 1
T8 55 1
T9 55 1
 Version Specific Test Case Prioritization
Contd…
• No. of matches found (N Found) is stored in an array &
elements are stored in descending order.

• The test case that has the maximum number of

matches is selected by making its candidate value = 1.
This test case will be selected first and is shown in next
slide.
Test No. of Line Candidate
cases matches Number
found
Selected for
T1 2 1,2 1 Execution
T5 2 5,15 0
T6 2 15,35 0
Removed as
T2 1 1 0 Selected
for
T3 1 5 0 execution

T7 1 45 0
T8 1 55 0
T9 1 55 0
LOC that are still to be executed are – [5, 15, 35, 45,
55]
 Version Specific Test Case
Prioritization Contd…
• T1 is elected for execution & it covers lines no 1&2 so
the modified lines are 5,15,35,45,55.

• Again we check the number of modified lines of code

covered by each test cases and sort them in
descending order and select the one with maximum
number of Matches
Test case Modified Lines No. of Matches

T3 5 1
T5 5,15 2
T6 15,35 2
T7 45 1

T8 55 1

T9 55 1
Test cases No. of Line Candidate
matches Number
found
T5 2 5,15 1 Selected for
Execution
T6 2 15,35 0
T3 1 5 0 Removed
as selected
T7 1 45 0 for
execution
T8 1 55 0
T9 1 55 0
LOC that are still to be executed are – [35, 45,
55]
Test case Modified Lines No. of Matches

T6 15,35 2

T7 45 1

T8 55 1

T9 55 1
Test No. of Line Candidate
cases matches Number
found
Selected for
T6 1 35 1 Execution

T7 1 45 1
T8 1 55 1
T9 1 55 0 Duplicate

LOC that are still to be executed are – [45,

55]
Version Specific Test Case Prioritization
Contd…

• Test Cases selected are:-

• T1, T5, T6, T7, T8
• Modification traversing test cases were 8
but for 100% code coverage for modified
lines 5 test cases are required.
 Alpha, Beta and Acceptance
Testing
• The term Acceptance Testing is used when
the software is developed for a specific
customer. A series of tests are conducted to
enable the customer to validate all
requirements. These tests are conducted by
the end user/customer and may range from
adhoc tests to well planned systematic series
of tests.
 • Acceptance Testing is a level of the software
testing where a system is tested for
acceptability.

ANALOGY
During the process of manufacturing a ballpoint pen, the cap, the
body, the tail and clip, the ink cartridge and the ballpoint are produced
separately and unit tested separately. When two or more units are
ready, they are assembled and Integration Testing is performed.
When the complete pen is integrated, System Testing is performed.
Once System Testing is complete, Acceptance Testing is performed so
as to confirm that the ballpoint pen is ready to be made available to
the end-users.
• Usually, Black Box Testing method is used in Acceptance Testing.
When is it performed?
• Acceptance Testing is performed after System Testing and before
making the system available for actual use.
Who performs it?
Internal Acceptance Testing (Also known as Alpha Testing)
• Performed mainly by the in-house software QA and testing teams.
• Alpha testing is the last testing done by the test teams at the development
site before releasing the software for beta test.
• Focus is to simulate real users
• Alpha testing aims at carrying out tasks that a typical user might perform
• Alpha testing is conducted in a controlled lab environment

User Acceptance Testing (Also known as Beta Testing)

• Performed by the end users of the software. They can be the customers
themselves or the customers’ customers.
• Developer is not present here
• Beta testing is conducted in a real environment that can’t be controlled by the
developer
Functional Testing/Black Box
Testing
 Functional
Testing
• The technique of testing without having any
knowledge of the interior workings of the
application is Black Box testing. The tester is
oblivious to the system architecture and does
not have access to the source code. Typically,
when performing a black box test, a tester
will interact with the system's user interface
by providing inputs and examining outputs
without knowing how and where the inputs
are worked upon.
 Black-box
testing
• Characteristics of Black-box testing:
– Program is treated as a black box.
– Implementation details do not
matter.
– Requires an end-user perspective.
– Criteria are not precise.
– Test planning can begin early.
 Black-box
testing Inputs causing
anomalous
Input test data I behaviour
e

System

Outputs which reveal

the presence of
Output test results Oe defects
 Performance
• testing
Performance testing is the process of determining the
speed or effectiveness of a computer, network,
software program or device.

• Performance testing is the testing, which is

performed, to ascertain how the components of a
system are performing, given a particular situation.
Resource usage, scalability and reliability of the
product are also validated under this testing.
 Performance Testing
• Goaltesting includes establishing
The primary goal of performance
the benchmark behavior of the system. There are a number of
industry-defined benchmarks, which should be met during
performance testing.

• Performance testing does not aim to find defects in the

application, it address a little more critical task of testing the
benchmark and standard set for the application.

• Main focus is on checking:

Speed: Determined whether the application responds quickly
Scalability: Determines maximum user load the software
application can handle
Stability: Determines if the application is stable under varying
loads
 Exampl
• e
We can test the application network performance on Connection
Speed vs. Latency chart. Latency is the time difference between the
data to reach from source to destination. Thus, a 70kb page would
take not more than 15 seconds to load for a worst connection of
28.8kbps modem (latency=1000 milliseconds), while the page of
same size would appear within 5 seconds, for the average
connection of 256kbps DSL (latency=100 milliseconds). 1.5mbps T1
connection (latency=50 milliseconds) would have the performance
benchmark set within 1 second to achieve this target.

• For example, the time difference between the generation of

request and acknowledgement of response should be in the range
of x ms (milliseconds) and y ms, where x and y are standard digits. A
successful performance testing should project most of the
performance issues, which could be related to database, network,
software, hardware etc…
 Types of Performance Testing
• Load testing - checks the application's ability to perform under
anticipated user loads. The objective is to identify performance
bottlenecks before the software application goes live.
• Stress testing - involves testing an application under extreme workloads
to see how it handles high traffic or data processing. The objective is to
identify breaking point of an application.
• Endurance testing – involves applying a significant load to a system for
an extended period of time to discover how the system behaves under
sustained use
• Spike testing – done by suddenly increasing or decreasing the load
generated by a very large number of users and observing the behavior of
the system.
• Configuration testing – done to determine the effect of configuration
changes to the system’s components on the system’s performance and
behavior
 Recovery
•
testing
Recovery testing is basically done in order to
check how fast and better the application can
recover against any type of crash or hardware
failure etc.
• Recovery testing is the forced failure of the
software in a variety of ways to verify that
recovery is properly performed.
 Exampl
e
Examples of recovery testing:
1.While an application is running, suddenly restart the
computer, and afterwards check the validness of the
application's data integrity.
2.While an application is receiving data from a network,
unplug the connecting cable. After some time, plug the
cable back in and analyze the application's ability to
continue receiving data from the point at which the
network connection disappeared.
3.Restart the system while a browser has a definite
number of sessions. Afterwards, check that the browser
is able to recover all of them.
 Structure Testing/White Box
testing
• A complementary approach to functional
testing is called structural / white box testing.
It permits us to examine the internal
structure of the program.
Verification and
Validation

Assuring that a software

system meets a user's
needs
 Verification vs
• Verification: Validation
"Are we building the product right"
• The software should confirm to its
specification

• Validation:
"Are we building the right product"
• The software should do what the user really
requires
 Difference between Verification and Validation
Verification Validation
It is a process that makes it sure that the software It refers to a different set of activities which ensure
product is developed in the right way. that the software built is traceable to customer
"Are we building the product right“ requirements
"Are we building the right product”
It ensures that software confirms to its predefined Validation physically ensures that the system operates
specifications, as the product development goes according to plan by executing the system functions
through different stages, an analysis is done to ensure through a series of tests that can be observed and
that all required specifications are met. evaluated.

Verification is a static practice of verifying documents, Validation is a dynamic mechanism of validating and
design, code and program. testing the actual product.
It does not involve executing the code. It always involves executing the code.
Verification uses methods like inspections, reviews, Validation uses methods like black box (functional) 
walkthroughs etc. testing, white box (structural) testing etc.

It can catch errors that validation cannot catch. It is low It can catch errors that verification cannot catch. It is
level exercise. High Level Exercise.
Target is requirements specification, application and Target is actual product-a unit, a module, a bent of
software architecture, high level, complete design, and integrated modules, and effective final product.
database design etc.
It generally comes first-done before validation. It generally follows after verification
 The V & V process
• V&V is a process of checking that a software
meets specifications and fulfills its intended
purpose

• As a whole life-cycle process - V & V must be

applied at each stage in the software process.

• Has two principal objectives

– The discovery of defects in a system
– The assessment of whether or not the system is
usable in an operational situation.
 V-Model
V-Model is an improvement over waterfall model. Left arm of ‘V’ represents the normal waterfall
model phases, right arm of ‘V’ represents the validation phases. Each phase on the left is shown
to have a corresponding phase.

This model suggests parallel work on the testing sub-phases along with other phases. It suggests
that unit test cases are written while low level design is in progress and so on
 How to test software requirements
specification (SRS)?
• “Most of the bugs in software are due to
incomplete or inaccurate functional
requirements.” The software code, doesn’t matter
how well it’s written, can’t do anything if there are
ambiguities in requirements.
 How to measure functional software
requirement specification (SRS) documents?
• We need to define some standard tests to
measure the requirements. Once each
requirement is passed through these tests you can
evaluate and freeze the functional requirements.
• Let’s take an example. You are working on a web
based application. Requirement is as follows:
“Web application should be able to serve the user
queries as early as possible”
How will you freeze the requirement in
this case?
• What will be your requirement satisfaction
criteria? To get the answer, ask this question to
stakeholders: How much response time is ok for
you?
• If they say, we will accept the response if it’s
within 2 seconds, then this is your requirement
measure. Freeze this requirement and carry the
same procedure for next requirement.
• “Requirements should be clear and consistent.”
• “Discover missing requirements.”
• For checking the requirements completeness, divide
requirements in three sections, ‘Must implement’
requirements, requirements those are not specified but are
‘assumed’ and third type is ‘imagination’ type of
requirements. Check if all type of requirements are
addressed before software design phase.
• Requirements specification (SRS) doc should
address following: Project functionality (What
should be done and what should not)
• Software, Hardware interfaces and user interface
• System Correctness, Security and performance
criteria
• Implementation issues (risks) if any
“Requirements should be clear and specific with no
uncertainty, requirements should be measurable
in terms of specific values, requirements should be
testable having some evaluation criteria for each
requirement, and requirements should be
complete, without any contradictions”
 Static and Dynamic black box
testing
• Under Static Testing code is not executed.
Rather it manually checks the code,
requirement documents, and design
documents to find errors. Hence, the name
"static".
• Main objective of this testing is to improve the
quality of software products by finding errors in
early stages of the development cycle. This
testing is also called as Non-execution
technique or verification testing.
 Static
Testing
It examines work documents and provides
review comments. Work document can be of
following:
•Requirement specifications
•Design document
•Source Code
•Test Plans
•Test Cases
•Test Scripts
•Help or User document
•Web Page content
 Static Testing Techniques

• Informal Reviews: This is one of the type of

review which doesn't follow any process to find
errors in the document. Under this technique ,
document is reviewed informally and informal
comments are provided.

• Technical Reviews: In this type of static testing

a technical round of review is conducted to
check if the code is made according to technical
specifications and standards. Generally the test
plans, test strategy and test scripts are
reviewed here.
• Walkthrough: The author of the work product explains the
product to his team. Participants can ask questions if any.
Meeting is led by the author.
• Inspection: Here the main purpose is to find defects. Code
walkthroughs are conducted by moderator. It is a formal type of
review where a checklist is prepared to review the work
documents
• Static code Review: This is systematic review of the software
source code without executing the code. It checks the syntax of
the code, coding standards, code optimization, etc. This is also
termed as white box testing
 Dynamic
•
Testing
Under Dynamic Testing code is executed.
It checks for functional behavior of software
system , memory/cpu usage and overall
performance of the system.
• This testing is also called as Execution
technique or validation testing.
• Dynamic testing executes the software and
validates the output with the expected
outcome. Dynamic testing is performed at all
levels of testing and it can be either black or
white box testing.
 Dynamic Testing Techniques

• Unit testing

• Integration testing

• System testing

• Acceptance testing
 Difference between Static and Dynamic testing
Static testing Dynamic testing

Static testing involves verification of activities of the Dynamic testing involves working with the
developed software software, giving input values and checking if the
output is as expected.
It is white box testing which is done at early stage Can be either black or white box and is done at later
of development stage of development

Static testing is done before the code deployment Dynamic testing is done after code deployment
(verification stage) (validation stage).

Done without execution of code Done with the execution of code

A checklist is prepared for testing process Test cases are executed

Static testing is more cost effective Less cost effective

Methodologies: Reviews , inspection, walkthroughs Methodologies: unit testing, integration testing,

etc system testing , acceptance testing
 Case Study on ATM
System
Aim: Take any system (e.g. ATM system) and
study its system specifications and report
the various bugs.
 Case Study on ATM System
continued..
1.Machine is accepting ATM card.
2.Machine is rejecting expired card.
3.Successful entry of PIN number.
4.Unsuccessful operation due to enter wrong PIN number 3 times.
5.Successful selection of language.
6.Successful selection of account type.
7.Unsuccessful operation due to invalid account type.
8.Successful selection of amount to be withdrawn.
9.Successful withdrawal.
10.Expected message due to amount is greater than day limit.
11.Unsuccessful withdraw operation due to lack of money in ATM.
12.Expected message due to amount to withdraw is greater than possible
balance.
13. Unsuccessful withdraw operation due to click cancel after insert card.
 Case Study on Banking Application

Aim: Write the test cases for any

known application (e.g. banking
application)
 Case Study on Banking Application
continued..

1.Checking mandatory input parameters

2.Checking optional input parameters
3.Check whether able to create account entity.
4.Check whether you are able to deposit an amount in the newly created account (and thus
updating the balance)
5.Check whether you are able to withdraw an amount in the newly created account (after
deposit and thus updating the balance)
6.Check whether company name and its pan number and other details are provided in case of
salary account
7.Check whether primary account number is provided in case of secondary account
8.Check whether company details are provided in cases of company's current account
9.Check whether proofs for joint account is provided in case of joint account
10.Check whether you are able deposit an account in the name of either of the person in a joint
account.
11.Check whether you are able withdraw an account in the name of either of the person in a
joint account.
12.Check whether you are able to maintain zero balance in salary account
13.Check whether you are not able to maintain zero balance (or mini balance) in non-salary
account.
Case Study on Library Management System

Aim: Create a test plan document for any

application (e.g. Library Management
System)
 Case Study on Library Management System
The Library Management System continued…
is an online application for assisting a
librarian managing book library in a University. The system would provide
basic set of features to add/update clients, add/update books, search for
books, and manage check-in / checkout processes. Our test group tested the
system based on the requirement specification.
This test report is the result for testing in the LMS. It mainly focuses on two
problems
1.What we will test
2.How we will test.
3.GUI test
Pass criteria: librarians could use this GUI to interface with the backend
library database without any difficulties
2.Database test
Pass criteria: Results of all basic and advanced operations are normal (refer
to section 4)
3.Basic function test
Add a student each customer/student should have following attributes:
Student ID/SSN (unique), Name, Address and Phone number.
The retrieved customer information by viewing customer detail should
contain the four attributes.
 Case Study on Library Management
System continued…
4. Update/delete student
The record would be selected using the student ID Updates can be
made on full. Items only: Name, Address, Phone number.
The record can be deleted if there are no books issued by user. The
updated values would be reflected if the same customer's ID/SSN is
called for.

5. Check-in book
Librarians can check in a book using its call number
The check-in can be initiated from a previous search operation where
user has selected a set of books.
The return date would automatically reflect the current system date.
Any late fees would be computed as difference between due date
and return date at rate of 10 cents a day.