Principle 3 – Early testing

•The efficiency gained from early testing is just as appropriate to the programming phase as it is to
other phases.
For example, problems detected during program design can be corrected more economically
than if they are detected while testing the executable program.
•Early testing finds defects in the initial phases, so cost of correcting the defect is less.
•Testing activities should start as early as possible in the software or system development life cycle,
and should be focused on defined objectives.
Principle 4 – Defect clustering
•A typical test improvement initiatives will initially find more defect as the testing improves, as the
defect prevention kicks in, the defect number drops.
•Focus may change from finding coding bugs, to looking at the requirements and design document for
defect and for process improvements so that we prevent defect in the product.
•A small number of modules contain most of the defects discovered during pre-release testing, or
show the most operational failures.
•A typical test improvement initiatives will initially find more defect as the testing improves, as the
defect prevention kicks in, the defect number drops.
•Focus may change from finding coding bugs, to looking at the requirements and design document for
defect and for process improvements so that we prevent defect in the product.
Principle 5 – Pesticide paradox
Test effectiveness is defined as :
•How well the user achieves the goals they set out to achieve using system.
•Effectiveness signifies how well the customer requirements have been meet. i.e. The final
product provide the solution to the customer problem effectively.
•Effectiveness is customer response on meeting product requirement.
•Producing a powerful effect.
Test effectiveness is calculated as,
Defect Removal in phase
Test effectiveness = -----------------------------------* 100
(Defect Injected + Defect escaped)
If the same tests are repeated over and over again, eventually the same set of test cases will no longer
find any new bugs. To overcome this “pesticide paradox”, the test cases need to be regularly reviewed
and revised, and new and different tests need to be written to exercise different parts of the
software or system to potentially find more defects.
Principle 6 – Testing is context dependent
•Testing is done differently in different contexts
• For safety, in critical software, critical part first integration is mostly used
Most commonly used approach for other software is bottom-up integration
COTS software are tested in different contexts.
For example, safety-critical software is tested differently from an e-commerce site
•Different approaches for testing will be used for different types of software
Principle 7 – Absence-of-errors fallacy
•The system must meet customer requirement and it should be fit for use
•If there is any difference between these two views, it will directly affect the quality
Finding and fixing defects does not help if the system built is unusable and does not fulfill the users’
needs and expectations
Ex: If customer need is to travel from place ‘A’ to place ‘B’. So Organization has to develop the 4-stroke
bike to meet the customer requirement, but the customer height is 2 feet that’s why the customer
can not use it although it is meeting the customer requirement.
Let’s discuss now the Fundamentals of test Process, This include the following topics
1) Test Planning and Control
2) Test Analysis and Design
3) Test Implementation and execution
4) Evaluating exit criteria and reporting
5) Test closure activities
Test Planning and Control
Test plan:
•A test plan documents the strategy that will be used to verify and ensure that a product or system
meets its design specifications and other requirements [ As per Wikipedia]
•It contains the test scope, test objectives, test team, goals and responsibilities of Testing
members, test case design techniques to be used, defect management tool To be used, risk
assessment and mitigation, test strategies, installation Scenario (optional), and various criteria like
test entry criteria, test suspension criteria, and test exit Criteria.
• It will also contains the time line of testing activities in a summarized format & Types of metrics to
be collected.
Test Planning and Control
The Test plan template includes
1)Test Plan Identifier
2)Introduction
3)Test Scope
4)Test Objectives
5)Assumptions
6)Risk Analysis
7)Test Design
8)Roles & Responsibilities
9)Test Schedule & Resources
10) Test Data Management
11) Test Environment
12) Communication Approach
13) Test Tools
Test Planning and Control
Test control is the ongoing activity of comparing actual progress against the plan, and reporting the
status, including deviations from the plan.
Following are the major tasks as part of Test Control:
1)Measuring and analyzing results
2)Monitoring progress
3)Documenting
4)Keeping History
 5)Making decisions
6)Recording Decisions
7)Test coverage and exit criteria
8)Initiation of corrective actions
•Test Analysis and Design
Identify test resources
•Testing using test cases can be as extensive or limited a process as desired.
•When time expires, testing is complete.
•Unfortunately, many programmers approach the creation of test data from a “we will do the best
job possible” perspective and then begin developing test transactions.
•The recommended approach suggests that the amount of resources allocated for the test data
test tool is determined and then a process developed that optimizes that time.
Identify condition to be tested
• A testing matrix is recommended as the basis for identifying conditions to test
•As these matrices cascade through the developmental process, they identify all possible test
conditions.
•If the matrix concept is not used, then the possible test conditions should be identified during the
use of this test tool.
•These should be general test conditions, such as in a payroll application to test the FICA
deductions.
Rank test conditions
•The objective of ranking is to identify high-priority test conditions that should be tested first.
Ranking can be used for two purposes:
•First, to determine which conditions should be tested first
•Second, and equally as important, to determine the amount of resources allocated to each of the test
conditions.
•If resources are limited, the maximum use of those resources will be obtained by testing the most
important test conditions. For example, if testing the FICA deduction was a relatively low-ranked
condition, only one test transaction might be created to test that condition, while for the higher
ranked test conditions several test transactions may be created.
Document test conditions: Both the test situations and the results of testing should be documented.
Conduct test
1) The executable system should be run, using the test conditions.
2) Depending on the extent of the test, it can be run under a test condition or in a simulated
production environment.
Verify and correct
1) The results of testing should be verified and any necessary corrections to the programs performed.
Problems detected as a result of testing can be attributable not only to system defects, but to test
data defects.
2) The individual conducting the test should be aware of both situations.
•Select condition for testing
•Based on the ranking, the conditions to be tested is selected. At this point, the conditions should
be very specific.
•For example, “testing FICA” is a reasonable condition to identify and rank, but for creating specific
 test conditions it is too general. Three test situations might be identified – such as
• Employees whose year-to-date earnings exceed the maximum FICA deduction;
•Employee whose current-period earnings will exceed the difference between the year-to- date
earnings and the maximum deduction;
•Employee whose year-to-date earnings are more than one pay period amount below the
maximum FICA deductions.
•Determine correct results of processing
•A unique number should identify each test situation, and then a log made of the correct results for
each test condition
•The correct processing results for each test situation should be determined.
•The process can also show if there are ways to extend the effectiveness of test transactions, and
whether the same condition has been tested by another transaction.
• The correct time to determine the correct processing results is before the test transactions have
been created.
•Create test cases
•Each test situation needs to be converted into a format suitable for testing.
•In some instances, this requires the creation of a test case and master information to be stored by
the program for the purpose of processing the test case.
•The method of creating the machine-readable transaction will vary based on the application and
the test rules available in the information systems department.
•The most common methods of creating test cases include:
•Key entry, Test data generator , and Preparation of an input form which will be given to user
personnel to enter
Test Implementation and Execution
Test execution objectives
1) To execute the appropriate collections of tests required to evaluate product quality
2) To capture test results that facilitate ongoing assessment of the product
Execution considerations
1) Execute tests which are deemed highest risks first.
2) Tests on which there are many dependent tests will be executed first
3) Test cases central to the architecture
4) Test execution on number of Operating system, browsers, servers etc.
5) Execution manual or automated?
Test Implementation and Execution
Execution activities include
1) Test platforms , that is, Set up test environment , Hardware, Software, Tools and Data
2) Executing the unit test
3) Executing the integration test
4) Executing the system test
5) Test cycle strategy
Test Implementation and Execution
1) Set up system test environment, mirroring the planned production environment as closely as
possible.
2) Establish the test bed.
 3) Identify test cycles needed to replicate production where batch processing is involved.
4) Assign test cases to test cycles
5) Execute the tests
Evaluating exit criteria is the activity where test execution is assessed against the defined objectives.
•All test cases are executed for each test level
Ex.: Exit criteria for unit testing
•Unit testing of the components is complete
•No open defect exists in the unit
Evaluating exit criteria has the following major tasks:
1. Checking test logs against the exit criteria specified in test planning.
2. Assessing if more tests are needed or if the exit criteria specified should be changed.
3. Writing a test summary report for stakeholders.
Test Closure Activities
Test closure activities collect data from completed test activities to consolidate experience, testware,
facts and numbers.
Test closure document is the note prepared, before you formally complete the testing process.
This note contains : Total number of test cases, Number of test cases executed , Number of defects
found , Number of defects resolved (fixed), Number of defects rejected, Total number of bugs differed,
Defect density, slippage ratio and consolidated Test report etc.
For example, when a software system is released, a test project is completed (or cancelled), a
milestone has been achieved, or a maintenance release has been completed.
Test closure activities include the following major tasks:
1.Checking which planned deliverables have been delivered, the closure of incident reports or raising
of change records for any that remain open, and the documentation of the acceptance of the system.
2. Finalizing and archiving testware, the test environment and the test infrastructure for later reuse.
3. Handover of testware to the maintenance organization.
4. Analyzing lessons learned for future releases and projects, and the improvement of test maturity.
Defect density - the number of confirmed defects detected in software/component during a defined
period of development/operation divided by the size of the software/component
Slippage ratio -the difference between the planned project schedule and actual project schedule
Test Closure Activities
One of the Test closure Activities is Lessons learned which contains
1) Successes: This section highlights successes of the project.
2) Areas for improvement : This section highlights areas for improvement of the project.
Project Kick-Off (E.g. Project Proposal) , Definition of Requirements, System Design, System
Build, Testing, Training, Implementation, Rollout, Communication, Project Management
The Psychology of Testing
"The psychology of the persons carrying out the testing will have an impact on the testing process
[Meyer 1979].“
Let's examine the psychology of the tester by describing the definition of software testing under three
circumstances.
1."Software testing is the process to prove that the software works correctly"
1. The above definition sounds good. If the tester is the one who has developed the software
he/she will try to use the above definition. This person's intentions would mostly revolve
 around the point to prove the software works.
2.This person's intentions would mostly revolve around the point to prove the software works.
He/she will only give those inputs for which correct results are obtained.
3. This above explanation is the typical psychology of testing by a software developer.
2. "Testing is the process to prove that the software doesn't work"
1.This definition sounds very good especially if the aim of the tester is to prove that the
software doesn't work for what it's supposed to do
2.This type of psychology would bring out the most of the defects in the software.
3. A software tester should therefore have such a psychology to push the software beyond its
boundaries.
3. "Testing is the process to detect the defects and minimize the risks associated with the
residual defects"
This definition appears to sound realistic, give those inputs for which correct results are obtained.
1) Practically, if at any point, the software product development starts, the testing process
should start and keep in track the number of bugs being detected while correcting them.