Five Views of Quality

„ Value-based (Engineer to price)

„ User-based (Fitness for purpose)
Software Quality Assurance „ Process-based (Conformance to requirements)
„ Product-based (You get what you pay for)
„ Transcendent (Excellence)

M8034 @ Peter Lo 2006 1 M8034 @ Peter Lo 2006 2

Software Quality Definition The Definition Emphasize

„ Conformance to explicitly stated functional and „ Software requirements are the foundation from which
performance requirements, explicitly documented quality is measured. Lack of conformance to
requirements is lack of quality.
development standards, and implicit
„ Specified standards define a set of development criteria
characteristics that are expected of all
that guide the manner in which software is engineered. If
professionally developed software. the criteria are not followed, lack of quality will almost
surely result.
„ There is a set of implicit requirements that often goes
unmentioned. If software conforms to its explicit
requirements, but fails to meet implicit requirements,
software quality is suspect.

M8034 @ Peter Lo 2006 3 M8034 @ Peter Lo 2006 4

How Important Software Quality? Software Quality Factors

„ It is important to have the understanding that the „ It can be categorized in two groups:
software quality work begins before the testing ‹ Factors that can be Directly Measured (e.g.
phase and continues after the software is delivered. errors; KLOC; unit-time)
‹ Factors that can be Measured only
Indirectly (e.g. usability or maintainability)
„ Or, it can be categorized into:
‹ Internal – Attributes which can be measured or
observed in isolation.
‹ External – Attributes which can only be
observed in relation to external environment.
M8034 @ Peter Lo 2006 5 M8034 @ Peter Lo 2006 6

McCall’s Triangle of Quality McCall’s Triangle of Quality

„ McCall’s quality factors were proposed in the

PRODUCT REVISION PRODUCT TRANSITION early 1970s.
Maintainability Portability „ They are as valid today as they were in that time.
Flexibility Reusability
„ It’s likely that software built to conform to these
Testability Interoperability factors will exhibit high quality well into the 21st
century, even if there are dramatic changes in
technology.
PRODUCT OPERATION
Reliability Integrity
Correctness Usability Efficiency
M8034 @ Peter Lo 2006 7 M8034 @ Peter Lo 2006 8
Product Operations – Product Revision –
Operation Characteristics Ability to Undergo Change
Correctness The extent to which a program satisfies its Maintainability The effort required to locate and fix an
specification and fulfills the customer's mission error in a program.
objectives.
Reliability The extent to which a program can be expected to
perform its intended function with required precision.
Flexibility The effort required to modify an
Efficiency The amount of computing resources and code required operational program.
by a program to perform its function.

Integrity The extent to which access to software or data by

unauthorized persons can be controlled. Testability The effort required to test a program to
ensure that it performs its intended
Usability The effort required to learn, operate, prepare input, function.
and interpret output of a program.

M8034 @ Peter Lo 2006 9 M8034 @ Peter Lo 2006 10

Product Transition – Adaptability to

ISO 9126 Quality Factors
New Environments
Portability The effort required to transfer the program „ These factors provide a basis for indirect
from one hardware and / or software measures and an excellent checklist for assessing
system environment to another.
the quality of the system.
Reusability The extent to which a program (or parts of ‹ Functionality
a program) can be reused in other
‹ Reliability
applications - related to the packaging and
scope of the functions that the program ‹ Usability
performs. ‹ Efficiency
Interoperability The effort required to couple one system
‹ Maintainability
to another.
‹ Portability

M8034 @ Peter Lo 2006 11 M8034 @ Peter Lo 2006 12

ISO 9126 Quality Factors – ISO 9126 Quality Factors –
Functionality Reliability
„ The degree to which the software satisfies stated „ The amount of time that the software is available
needs as indicated by the following sub-attributes: for use as indicated by the following sub-attributes:
‹ Suitability ‹ Maturity

‹ Accuracy ‹ Fault tolerance

‹ Interoperability ‹ Recoverability

‹ Compliance

‹ Security

M8034 @ Peter Lo 2006 13 M8034 @ Peter Lo 2006 14

ISO 9126 Quality Factors – ISO 9126 Quality Factors –

Usability Efficiency
„ The degree to which the software is easy to use as „ The degree to which the software makes optimal
indicated by the following sub-attributes: use of system resources as indicated by the
‹ Understandability following sub-attributes:
‹ Learnability ‹ Time behaviour

‹ Operability ‹ Resource behaviour

M8034 @ Peter Lo 2006 15 M8034 @ Peter Lo 2006 16

 ISO 9126 Quality Factors – ISO 9126 Quality Factors –
Maintainability Portability
„ The ease with which repair may be made to the „ The ease with which the software can be
software as indicated by the following sub- transposed from one environment to another as
attributes: indicated by the following sub-attributes:
‹ Analysability ‹ Adaptability

‹ Changeability ‹ Installability

‹ Stability ‹ Conformance

‹ Testability ‹ Replaceability

M8034 @ Peter Lo 2006 17 M8034 @ Peter Lo 2006 18

The Attributes of Effective Software

Software Quality Metric
A u d i ta b i l i t y
A ccuracy
T h e e a s e w i th w h i c h c o n f o r m a n c e t o s t a n d a r d s c a n b e c h e c k e d .
T h e p r e c i s i o n o f c o m p u t a t i o n s a n d c o n tr o l.
Metrics
C o m m u n ication T h e d e g r e e t o w h i c h s t a n d a r d in t e r f a c e s , p r o t o c o l s , a n d b a n d w i d th s a r e u s e d .
c o m m o n a li t y
C o m p le ten ess T h e d e g r e e t o w h i c h f u l l i m p l e m e n t a t i o n o f r e q u ir e d f u n c ti o n h a s b e e n a c h i e v e d . „ Simple and Computable
C o n cisen ess T h e c o m p a c tn e s s o f t h e p r o g r a m in t e r m s o f l in e s o f c o d e .
C o n sisten c y T h e u s e o f u n i f o r m d e s i g n a n d d o c u m e n ta t i o n t e c h n i q u e s th r o u g h o u t th e s o f t w a r e
d e v e l o p m e n t p r o je c t . „ Empirically and Intuitively Persuasive
D a ta T h e u s e o f s t a n d a r d d a ta s tr u c t u r e s a n d t y p e s t h r o u g h o u t th e p r o g r a m .
c o m m o n a li t y
E rr o r to leran ce T h e d a m a g e th a t o c c u r s w h e n th e p r o g r a m e n c o u n t e r s a n e r r o r .
„ Consistent and Objective
E x ecu tio n T h e r u n - t im e p e r f o r m a n c e o f a p r o g r a m .
e f fic ie n c y
E x p a n d a b i li t y T h e d e g r e e t o w h i c h a r c h it e c t u r a l, d a ta , o r p r o c e d u r a l d e s i g n c a n b e e x t e n d e d .
„ Consistent in its Use of Units and Dimensions
G e n e r a li t y T h e b r e a d th o f p o t e n ti a l a p p li c a ti o n o f p r o g r a m c o m p o n e n t s .
H ardw are
in d e p e n d e n c e
T h e d e g r e e t o w h i c h th e s o f t w a r e i s d e - c o u p l e d f r o m th e h a r d w a r e o n w h i c h i t
o p er a tes
„ Programming Language Independent
I n s tr u m e n t a ti o n T h e d e g r e e t o w h i c h th e p r o g r a m m o n i t o r s it s o w n o p e r a t i o n a n d i d e n ti f i e s e r r o r s th a t

M o d u lar ity
do occur.
T h e f u n c ti o n a l in d e p e n d e n c e o f p r o g r a m c o m p o n e n t s .
„ An Effective Mechanism for Quality Feedback
O p e r a b i li t y T h e ea se o f o p er atio n o f a p r o g r am .
S e c u r ity T h e a v a il a b i li t y o f m e c h a n i s m s th a t c o n tr o l o r p r o t e c t p r o g r a m s a n d d a t a .
S e lf- T h e d e g r e e t o w h i c h th e s o u r c e c o d e p r o v i d e s m e a n in g f u l d o c u m e n ta t i o n .
d o cu m en tatio n
S i m p li c i t y T h e d e g r e e to w h ich a p r o g r a m c a n b e u n d er s to o d w ith o u t d iffic u lty .
S o f t w a r e s y s t e m T h e d e g r e e t o w h i c h th e p r o g r a m i s in d e p e n d e n t o f n o n s t a n d a r d p r o g r a m m in g
in d e p e n d e n c e l a n g u a g e f e a t u r e s , o p e r a tin g s y s t e m c h a r a c t e r i s t i c s , a n d o th e r e n v i r o n m e n ta l
c o n s t r a in t s .
T r a c e a b l i lt y T h e a b i li t y t o tr a c e a d e s i g n r e p r e s e n ta t i o n o r a c t u a l p r o g r a m c o m p o n e n t b a c k t o
r e q u ir e m e n t s .
M8034 @ Peter Lo 2006 19 M8034 @ Peter Lo 2006 20
T r a in in g T h e d e g r e e t o w h i c h th e s o f t w a r e a s s i s t s in e n a b l in g n e w u s e r s t o a p p l y th e s y s t e m .
Attributes of Effective Software Metrics – Attributes of Effective Software Metrics –
Simple and Computable Empirically and Intuitively Persuasive
„ It should be relatively easy to learn how to derive „ The metric should satisfy the engineer’s intuitive
the metric, and its computation should not demand notions about the product attribute under
inordinate effort or time. consideration.

M8034 @ Peter Lo 2006 21 M8034 @ Peter Lo 2006 22

Attributes of Effective Software Metrics – Attributes of Effective Software Metrics –

Consistent and Objective Consistent in Use of Units and Dimensions
„ The metric should always yield results that are „ The mathematical computation of the metric
unambiguous. should use measures that do not lead to bizarre
combinations of unit.

M8034 @ Peter Lo 2006 23 M8034 @ Peter Lo 2006 24

Attributes of Effective Software Metrics – Attributes of Effective Software Metrics –
Programming Language Independent Effective Mechanism for Quality Feedback
„ Metrics should be based on the analysis model, the „ The metric should provide a software engineer
design model, or the structure of the program itself. with information that can lead to a higher quality
end product.

M8034 @ Peter Lo 2006 25 M8034 @ Peter Lo 2006 26

Software Reliability Reliability Metric

„ Most hardware-related reliability models are „ Reliability metric is an indicator of how broken a
predicated on failure due to wear rather than program is.
failure due to design defects.
„ Metrics are best weighted by the by severity of
„ In hardware, failures due to physical wear (e.g. the errors.
effects of temperature, corrosion, shock) are more
likely than a design-related failure. „ A minor error every hour is better than a
„ The opposite is true for software: in fact, all catastrophe every month.
software failures can be traced to design or „ These metrics are not the same as counting bugs,
implementation problems; wear does not enter into but indicate different probabilities of happening.
the picture.

M8034 @ Peter Lo 2006 27 M8034 @ Peter Lo 2006 28

Mean Time Between Failure (MTBF) Availability

„ MTBF = MTTF + MTTR „ Software availability is the probability that a program is

operating according to requirements at a given point in
„ Measures how long a program is likely to run time
before it does something bad like crash. ‹ Availability = MTTF/ (MTTF + MTTR) * 100%
‹ MTTF is the mean time to failure.  MTTF is the mean time to failure.

‹ MTTR is the mean time to repair.  MTTR is the mean time to repair.

M8034 @ Peter Lo 2006 29 M8034 @ Peter Lo 2006 30

Probability of Failure on Demand Rate of Failure Occurrence

„ Common for safety-critical systems „ This tells how many may occur in a given period
‹ e.g. a specification may be that there not exceed ‹ e.g. 2 errors per 100 minutes. This is considered
1 chance in 1010 that a missile gets through. one of the best metrics

M8034 @ Peter Lo 2006 31 M8034 @ Peter Lo 2006 32

Quality Concepts Cost of Quality

„ Controlling variation among products is what „ Prevention Costs

‹ Quality planning, formal technical reviews, test
quality assurance work is all about.
equipment, training, etc
„ Software engineers are concerned with controlling „ Appraisal Costs
the variation in their processes, resource ‹ In-process and inter-process inspection, equipment
expenditures, and the quality attributes of the end calibration and maintenance, testing, etc
products. „ Failure Costs
‹ Internal Failure Costs (rework, repair, failure mode
„ Also need to be aware that customer satisfaction is
analysis)
very important to modern quality work as is ‹ External Failure Costs (complaint resolution, product
quality of design and quality of conformance. return and replacement, help line support, warranty
work)
M8034 @ Peter Lo 2006 33 M8034 @ Peter Lo 2006 34

Cost Impact of Software Defects The Goal of Quality Assurance

„ The later in the life cycle that an error is detected the more „ To provide management with the necessary to be
expensive it is to repair.
informed about product quality, thereby gaining
„ Errors remain latent and are not detected until well after
the stage at which they are made. insight and confidence that product quality is
‹ 54% of errors detected after coding and unit testing.
meeting its goals
‹ 45% of these errors were requirements and design
errors.
„ There are numerous requirements errors.
‹ Estimates indicate that 56% of all errors are errors
during the requirements stage.
„ Requirements errors are typically nonclerical.
‹ Estimates indicate that 77% requirements errors were
nonclerical.
M8034 @ Peter Lo 2006 35 M8034 @ Peter Lo 2006 36
Software Quality Assurance Software Quality Assurance (SQA)
„ Software Quality Assurance is an essential activity for any
Process Formal business that produces products to be used by others.
Definition & Technical
Standards Reviews „ It is a “planned and systematic pattern of actions” that are
required to ensure quality in software.
„ The SQA group must look at software from the customer’s
Analysis
& Test perspective, as well as assessing its technical merits.
Reporting Planning „ The activities performed by the SQA group involve quality
& Review
Measurement planning, oversight, record keeping, analysis and reporting.

M8034 @ Peter Lo 2006 37 M8034 @ Peter Lo 2006 38

Software Quality Management Measuring Quality

„ Concerned with ensuring that the required level of „ The main factors that defined software quality
quality is achieved in a software product includes
‹ Correctness – the degree to which a program
„ Involves defining appropriate quality standards
and procedures and ensuring that these are operates according to specification
‹ Maintainability – the degree to which a
followed
program is amenable to change
„ Should aim to develop a ‘quality culture’ where
‹ Integrity – the degree to which a program is
quality is seen as everyone’s responsibility impervious to outside attack
‹ Usability – the degree to which a program is
easy to use

M8034 @ Peter Lo 2006 39 M8034 @ Peter Lo 2006 40

Software Quality – Correctness Software Quality – Maintainability

„ A program must operate correctly. „ It cannot be measure directly. So we must use

„ Correctness is the degree to which the s/w indirect measures.
performs its required operation. „ A simple time oriented metric is mean time -to-
change (MTTC), the time it takes to analyse the
„ The most common measure for correctness is change request, design an appropriate
defects per KLOC, where a defect is defined as a modification, implement the change, test it and
verified lack of conformance to requirements. distribute the change to all users.
„ On average, programs that are maintainable will
have a lower MTTC (for equivalent types of
changes) than programs that are not maintainable.

M8034 @ Peter Lo 2006 41 M8034 @ Peter Lo 2006 42

Software Quality – Integrity Software Quality – Usability

„ S/w integrity has become increasingly important in the age „ It is an attempt to quantify “user friendliness” and can be
of hackers and viruses. This attribute measures a system’s measured in terms of 4 characteristics:
ability to withstand attacks (both accidental and intentional) ‹ The physical and or intellectual skill required to learn
on its security. Attack can be made on all 3 components of the system;
s/w: programs, data and documents. ‹ The time required to become moderately efficient in the
„ The integrity of a system can be defined as : use of the system
‹ Integrity = [ 1 - threat X (1 - security) ] ‹ the net increase in productivity measured when the

„ where Threat is the probability that can attack of a specific system is used by someone who moderately efficient
type will occur within a given time. Security is the and
probability that the attack of a specific type will be ‹ A subjective assessment (using a questionnaire) of
repelled. users attitude towards the system.

M8034 @ Peter Lo 2006 43 M8034 @ Peter Lo 2006 44

 SQA Seven Major Activities –
SQA Seven Major Activities
Application of Technical Methods
„ Application of Technical Methods „ It helps analyst to achieve a high-quality
„ Conduct of Formal Technical Reviews specification and the designer to develop a high-
„ Software Testing quality design.
„ Enforcement of Standards
„ Control of Change
„ Measurement
„ Record Keeping and Reporting

M8034 @ Peter Lo 2006 45 M8034 @ Peter Lo 2006 46

SQA Seven Major Activities – SQA Seven Major Activities –

Conduct of Formal Technical Reviews Software Testing
„ It uncovers quality problems. „ It combines a multi-step strategy with a series of
test case design methods that help ensure effective
error detection.

M8034 @ Peter Lo 2006 47 M8034 @ Peter Lo 2006 48

SQA Seven Major Activities – SQA Seven Major Activities –
Enforcement of Standards Control of Change
„ SQA activity must be established to ensure that „ It contributes directly to software quality by
standards are followed. formalizing requests for change, evaluating the
nature of change, and controlling the impact of
change.

M8034 @ Peter Lo 2006 49 M8034 @ Peter Lo 2006 50

SQA Seven Major Activities – SQA Seven Major Activities –

Measurement Record Keeping and Reporting
„ An important objective of SQA is to track „ It provides procedures for the collection and
software quality and assess the impact of dissemination of SQA information.
methodological and procedural changes on
improved software quality.
„ To accomplish this, measurement must be done.

M8034 @ Peter Lo 2006 51 M8034 @ Peter Lo 2006 52

The SQA Plan The SQA Plan Standard (IEEE)

„ The SQA plan provides a road map for instituting „ Initial section describes the purpose and scope of the
document and software process activities.
software quality assurance.
‹ Management section describes organizational structure,
„ Developed by the SQA group, the plan serves as a SQA activities and tasks etc.
template for SQA activities that are instituted for ‹ The documentation section describes each of the work

each software project. products produced as part of the software process.

‹ The standards, practices and conventions section
„ The remainder of SQA plan identifies the tools describes all applicable standards and practices.
and methods that support SQA activities and tasks, ‹ The reviews and audits section describes reviews and
software configuration management procedures audits to be conducted by different group.
etc. ‹ The test section describes software test plan and
procedure and record keeping requirements.

M8034 @ Peter Lo 2006 53 M8034 @ Peter Lo 2006 54

Tradeoff between Cost and Reliability What are Reviews?

„ It demands more time and resources for testing „ A meeting conducted by technical people for
and debugging. technical people
„ It requires more internal safety checks, and this „ A technical assessment of a work product created
makes programs larger and slower. during the software engineering process
„ A software quality assurance mechanism
„ A training ground

M8034 @ Peter Lo 2006 55 M8034 @ Peter Lo 2006 56

What Reviews are Not! Software Reviews

„ They are not: „ It is important to note that any work product

‹ A project budget summary (including documents) should be reviewed.
‹ A scheduling assessment „ Conducting timely reviews of all work products
‹ An overall progress report
can often eliminate 80% of the defects before any
testing is conducted.
‹ A mechanism for reprisal or political intrigue!!
Select Complete
review team review forms

Arrange place
Hold review
and time

Distribute
documents
M8034 @ Peter Lo 2006 57 M8034 @ Peter Lo 2006 58

Types of Reviews Quality Reviews

Review type Principal purpose
Design or program To detect detailed errors in the design or „ Objective is the discovery of system defects and
inspections code and to check whether standards have
been followed. Thereview should be driven inconsistencies
by a checklist of possible errors.
Progress reviews To provide information for management
„ Any documents produced in the process may be
about the overall progress of the project. reviewed
This is both a process and a product review
and is concerned with costs, plans and „ Review teams should be relatively small and
schedules. reviews should be fairly short
Quality reviews To carry out a technical analysis of product
components or documentationto find faults „ Review should be recorded and records
or mismatches between the specification maintained
and the design, code or documentation. It
may also be concerned with broader quality
issues such as adherence to standards and
other quality attributes.
M8034 @ Peter Lo 2006 59 M8034 @ Peter Lo 2006 60
Review Results Formal Technical Reviews
„ Comments made during the review should be classified. „ Formal technical review is
‹ No action. ‹ A class of reviews that include walkthroughs,
‹ Refer for repair – Designer or programmer should inspections, round-robin reviews, and other
correct an identified fault. small group technical assessments of software
‹ Reconsider overall design – The problem identified in ‹ A planned and controlled meeting attended by a
the review impacts other parts of the design. Some group of diversified people
overall judgement must be made about the most cost-
„ Formal technical reviews can be conducted during
effective way of solving the problem.
each step in the software engineering process.
„ Requirements and specification errors may have to be
referred to the client. „ A brief checklist can be used to access products
that are derived as part of software development.

M8034 @ Peter Lo 2006 61 M8034 @ Peter Lo 2006 62

Objectives of Formal Technical

Effects of Formal Technical Review
Review
„ To uncover errors in function, logic, or „ Early discovery of software defects so the
implementation for any representation of the development and maintenance phase is
software substantially reduced
„ To verify that the software under review meets its „ Serves as a training ground, enabling junior
requirements engineers to observe different approaches to
„ To ensure that the software has been represented software analysis, design, and implementation
according to predefined standards „ Serves to promote backup and continuity because
„ To achieve software that is developed in a uniform a number of people become familiar with parts of
manner the software that they may not have otherwise
„ To make projects more manageable seen

M8034 @ Peter Lo 2006 63 M8034 @ Peter Lo 2006 64

Conducting the Review The Players
„ Be prepared – evaluate product before the review review
leader standards bearer (SQA)
„ Review the product, not the producer
„ Keep your tone mild, ask questions instead of making producer
accusations
„ Stick to the review agenda
„ Raise issues, don't resolve them maintenance
„ Avoid discussions of style – stick to technical correctness oracle

„ schedule reviews as project tasks reviewer

recorder
„ record and report all review results
user rep

M8034 @ Peter Lo 2006 65 M8034 @ Peter Lo 2006 66

Metrics Derived from Reviews Review Options Matrix

IPR * WT IN RRR
„ Inspection time per page of documentation
trained leader no yes yes yes
„ Inspection time per KLOC or FP agenda established maybe yes yes yes
reviewers prepare in advance maybe yes yes yes
„ Errors uncovered per reviewer hour producer presents product no no
maybe yes
„ Errors uncovered per preparation hour “reader” presents product no no yes no
recorder takes notes maybe yes yes yes
„ Errors uncovered per SE task (e.g., design) checklists used to find errors no no yes no
„ Number of minor errors (e.g., typos) errors categorized as found no no yes no
issues list created no yes yes yes
„ Number of errors found during preparation team must sign-off on result no yes yes maybe
„ Number of major errors (e.g., nonconformance to
IPR—informal peer review WT—Walkthrough
requirement) * IN—Inspection RRR—round robin review
„ Inspection effort per KLOC or FP

M8034 @ Peter Lo 2006 67 M8034 @ Peter Lo 2006 68

The checklists Fagan Inspection
„ The checklists are not to be comprehensive, but rather to provide a „ Michael Fagan noticed that no inspection of designs was
point of departure for each review. routinely practiced during software development in IBM
‹ System Engineering
during 1970s
‹ Software Project Planning
„ He developed a set of procedures for inspection of
‹ Software Requirements Analysis
software designs, source code, test plans and test cases.
‹ Software Design

‹ Preliminary Design Review

„ Fagan argued that software development should
‹ Design Walkthrough ‹ Clearly define the programming process as a series of

‹ Coding operations, each with its own exit criteria.

‹ Software Testing ‹ Measure the completeness of the product at any point of
‹ Test Plan its development by inspections or tests.
‹ Test Procedure ‹ Use measurements to control the process.
‹ Maintenance

M8034 @ Peter Lo 2006 69 M8034 @ Peter Lo 2006 70

Types of Inspection Inspection Team

Phases Inspections Role Task
I0 Initial Design Initial Design Specification (include Moderator In charge of the whole inspection. Chair the meeting.
functional & module specification) is He should not be involved in develop material under
inspected against Statement of inspection. A technical person.
requirements.
Author Give initial presentation. Rework to remove defect.
I1 Detailed Design Logic Specifications are inspected against
the Initial Design Specification
I2 Coding Source Code is inspected against Logic Inspector Normally two inspectors participate, usually
Specifications. members of development team that produced the
material under inspection (but not the same phase)
IT1 Test Plan Preparation Test Plan is inspected against Functional
Specification. Secretary Recording, enter data to database.

IT2 Test Case Preparation Test Case Listings are inspected against
Test Plan.
M8034 @ Peter Lo 2006 71 M8034 @ Peter Lo 2006 72
Inspection Procedure The Inspection Meeting
Steps Participants Objectives Remarks „ 3-5 people only
1. Planning Moderator Schedule Activities Include higher-level document, „ Lasts for two hours only
& Distribute Material checklists etc. „ The decision is based on standards which are specific to
2. Overview Author & others Education Author gives presentation to the organization.
other participants
„ The moderator first outlines the material to be inspected
3. Preparation All Participants Familiarization Study privately
and describes the intended function of the corresponding
4. Inspection Entire Team Find Defects A 2 hour meeting. Defects are part of the system.
recorded. Moderator decides the
material pass or not. Produce „ The reader then reads through all the material.
report. „ The inspection is extremely thorough
5. Rework Author Correct Defects
„ At the end of the meeting, the defect list is approved by all
6. Follow Up Moderator & Assure Rework is participants, and the moderator decides whether or not a
Author Correct; Improve
Development Process; re-inspection will be necessary after the rework.
Improve Inspection
M8034 @ Peter Lo 2006 Efficiency 73 M8034 @ Peter Lo 2006 74

Exit Criteria Checklists

„ To judge whether a given development phase is „ Set of questions inspectors to ask themselves.
complete. „ Different set of checklists corresponding to
„ It must be defined by individual organizations to different type of inspection & programming
meet the needs of their own development language.
environment. „ Should be continually added to improve inspection
„ Some examples: experience.
‹ I0: external specifications are completed

‹ I1: design specifications must be structured

‹ I2: module prologue up-to-date and complete

M8034 @ Peter Lo 2006 75 M8034 @ Peter Lo 2006 76

Defect Recording and Classification Inspection Database

„ Defect is record with serial number, type, category, „ Record effort required for preparation, number of
severity, line number in material, description and defect detected, size of material inspected.
estimate time to fix. „ Use to monitor and control development process,
Class Value and improve the effectiveness of the inspections.
Severity Major, Minor

Category Missing, Wrong, Extra

Type As defined in checklist

M8034 @ Peter Lo 2006 77 M8034 @ Peter Lo 2006 78

Measures for Inspection Use of Defect Type Distribution

Attributes Measures
„ Inspection improvement (Can focus on prevalent
Effort Number of person-hours
Code Size NCSS {non-Comment Source Statements}
defect)
Specification Size NCSS „ Programmer self-improvement
Rework Size NCSS {statement added, modified or deleted}
„ Development process improvement (Guide
Inspection Rate Size of material / Person-hours
management in providing training, standard and
Rework Rate Size of material / Person-hours
Defects Detected Count {can grouped by inspection, module, type, category or
procedures)
severity}
Defects Present Count {estimate number of defects before inspection}
Defects Remaining Count {estimate number of defects after inspection}
Defect Density Defects Count / Size of Module {can apply to present,
detected or remaining}
Defect Removal Efficiency Defects Removed / Defects present {in %, for an inspection
M8034 @ Peter Lo 2006 step} 79 M8034 @ Peter Lo 2006 80
Use of Defect Type Distribution Effectiveness of Fagan Inspection

„ Defect-prone modules are those in which a higher „ “Amplification and detection” – single defect in
than average number of defects are detected (or high level spec. may give rise to many defects in
estimated to remain). detailed spec.
„ It may due to badly written or more complex. „ Cost of defect detection – 6 to 8 times less than
Special attention should be paid on such modules. failure during test.
„ Inspection data MUST NOT used to assess people. „ Reduction in defect density – experiment show
Punishment will delay development process. 38% few faults than informal walk-through.
„ Improve productivity – experiment show 23%
increase due to reduction of rework time.

M8034 @ Peter Lo 2006 81 M8034 @ Peter Lo 2006 82

Statistical Software Process

Statistical SQA
Improvement (SSPI)
1. All errors and defects are categorised by origin. (e.g., flaw
• collect information on all defects in specification, flaw in logic, non-conformance to
standards).
Product • find the causes of the defects
2. The cost to correct each error and defect is recorded.
& Process
• move to provide fixes for process 3. The number of errors and defects in each category are
counted and ordered in descending order.
4. The overall cost of errors and defects in each category is
measurement computed.
5. Resultant data are analysed to uncover the categories that
result in highest cost to the organisation.
... an understanding of how
to improve quality ... 6. Plans are developed to modify the process with the intent
of eliminating (or reducing the frequency of occurrence of)
the class of errors and defects that is most costly.
M8034 @ Peter Lo 2006 83 M8034 @ Peter Lo 2006 84
Factors that Influence Software
Defect Removal Efficiency (DRE)
Productivity
„ People Factors – The size and expertise of the „ DRE is a measure of the filtering ability of quality
development organisation assurance and control activities as they applied throughout
all process framework activities.
„ Problem Factors – The complexity of the problem and the
number of changes in design requirements „ DRE can be defined as DRE = E/(E+D)
„ where
„ Process Factors – Analysis and design techniques that are
‹ E = no. of errors found before the delivery of the
used, languages and tools available.
software to the end user.
„ Product Factors – Reliability and performance of the
‹ D = no. of defects found after delivery.
computer-based system.
„ The ideal value for DRE is 1. (i.e., no defect). If used as
„ Resource Factors – Availability of CASE tools and metric, DRE encourages a software project team to apply
hardware and software resources. techniques for finding errors as many as possible before
delivery.

M8034 @ Peter Lo 2006 85 M8034 @ Peter Lo 2006 86