SOFTWARE QUALITY &
METRICS
SE303
Syllabus
Introduction
Syllabus
Unit Contents
1. Software Quality Assurance Framework: What is
Quality? Software Quality Assurance, Components of
Software Quality Assurance, Software Quality
Assurance Plan. Steps to develop and implement a
Software Quality Assurance Plan.
Syllabus
Unit Contents
2. Quality Standards: ISO 9000 and Comparison ISO
Standards, CMM, CMMI, PCMM, Malcolm Balridge,
3 Sigma, 6 Sigma, Software Quality Models.
Syllabus
Unit Contents
3. Measurement in Software Engineering: scope of software
metrics, Basics of Measurement: Measuring External
Product Attributes: Modeling Software Quality, Measuring
aspects of quality, Framework for Software Measurement,
Measuring Internal Product Attributes, Size and Structure:
Aspects of Software Size, Length, Reuse, Functionality,
Complexity, Types of Structural Measures, Modularity and
information flow attributes.
Syllabus
Unit Contents
4. Software Quality Assurance Metrics and
Measurement: Software Quality Metrics, Product
Quality Metrics, Process Quality Metrics, Metrics for
Software Maintenance, Software Quality Metrics
methodology, Object Oriented Metrics in quality.
Syllabus
Unit Contents
5. Software Quality Estimation Tools:
Desirable features in software quality
estimation tools, Study of some existing
tools for quality estimation.
Syllabus
Unit Contents
6. Computer Aided Quality Engineering
(CAQE) Concepts, Design Techniques
for CAQE.
Topic of Interests
Software metrics and anti-patterns.
Design generation, design representation, and
heuristics for good design.
Dynamic software verification: unit, integration,
regression, and acceptance testing.
Topic of Interests
Static software verification: reviews, walk-
throughs.
Standard representations for requirements, such
as user stories and interaction prototypes.
Prerequisite and References
References:
– Pressman Roger S, Software Engineering: A
Practitioner's Approach, 8th Edition, McGraw-
Hill. ISBN-13: 978-0071267823
Prerequisite and References
References:
– Software Metrics, Fenton, CRC press
– Yogesh Singh & Ruchika Malhotra, “Object
Oriented Software Engineering”, 1st Ed., PHI
Learning.
Overview
What is software quality?
How can it be measured?
– How can it be measured before/after the software
is delivered?
Quality
• Think of an everyday object
– e.g. a chair
– How would you measure it’s “quality”?
• construction quality? (e.g. strength of the
joints,…)
• aesthetic value? (e.g. elegance,…)
• fit for purpose? (e.g. comfortable,…)
Quality
• All quality measures are relative
– there is no absolute scale
– we can say A is better than B but it is usually
hard to say how much better
Examples of Metrics from Everyday Life
Working and living
Cost of utilities for the month
Cost of groceries for the month
Amount of monthly rent per month
Time spent at work each Saturday for
the past month
Time spent mowing the lawn for the past
two times
Examples of Metrics from Everyday Life
College experience
Grades received in class last semester
Number of classes taken each semester
Amount of time spent in class this week
Amount of time spent on studying and
homework this week
Number of hours of sleep last night
Examples of Metrics from Everyday Life
Travel
Cost of utilities for the month
Time to drive from home to the
airport
Amount of miles traveled today
Cost of meals and lodging for
yesterday
What is Software Quality?
Conformance to requirements.
Narrowest sense of software quality.
Lack of bugs.
High reliability (number of failures per n hours
of operation).
What is Software Quality?
According to IEEE, Software quality is
The degree to which a system, component, or
process meets specific requirements.
The degree to which a system, component, or
process meets customer or user needs or
expectations.
Software Quality:
Definition:
Conformance to explicitly stated
functional and performance requirements,
explicitly documented development
standards, and implicit characteristics that
are expected of all professionally
developed software.
Software Quality:
For example, if two cars meet their specified
speed, standard, style, and performance, then
they are said to meet the specified
requirements.
What is Software Quality:
Three important points in the
definition
Explicit software requirements are the
foundation from which quality is
measured.
Lack of conformance to requirements
is lack of quality.
What is Software Quality:
Three important points in the definition
Specific standards define a set of
development criteria that guide the manner in
which software is engineered. If the criteria
are not followed, lack of quality will most
surely result.
What is Software Quality:
Three important points in the definition
There is a set of implicit requirements that
often goes unmentioned (e.g., ease of use).
If software conforms to its explicit
requirements but fails to meet implicit
requirements, software quality is suspect.
Software Quality:
The goal of software quality is to determine:
◦ How well is the design of the software?
◦ How the software conforms to the
developed design?
Attribute domains of Software Quality
The attribute domains that are required
for a given software are:
Functionality
Usability
Testability
Reliability
Maintainability
Adaptability
Fishbone structure of attribute domains and
attributes
Functionality:
Functionality: The degree to which the purpose of
the software is satisfied.
1 Completeness The degree to which software
is complete.
2 Correctness The degree to which software is
correct.
Functionality:
Functionality: The degree to which the purpose of
the software is satisfied.
3 Efficiency The degree to which the software
requires resources to perform a
software function.
4 Traceability The degree to which requirement is
traceable to software design and
source code.
Functionality:
Functionality: The degree to which the purpose of
the software is satisfied.
5 Security The degree to which the software is
able to prevent unauthorized
access to the program data.
Usability:
Usability: The degree to which the software is easy
to use.
1 Learnability The degree to which the software is
easy to learn.
2 Operability The degree to which the software is
easy to operate.
Usability:
Usability: The degree to which the software is easy
to use.
3 User The degree to which the interfaces of
friendliness the software are easy to use and
understand.
4 Installability The degree to which the software is
easy to install.
Usability:
Usability: The degree to which the software is easy
to use.
5 Satisfaction Thedegree to which the user’s
feel satisfied with the software.
Testability:
Testability: The ease with which the
software can be tested to demonstrate
the faults.
1 Verifiability The degree to which the
software deliverable meets
the specified standards,
procedures and process.
Testability:
Testability: The ease with which the
software can be tested to demonstrate
the faults.
2 Validatable The ease with which the
software can be executed to
demonstrate whether the
established testing criteria
is met.
Maintainability:
Maintainability: The ease with which the
faults can be located and fixed, quality of
the software can be improved or software
can be modified in the maintenance
phase.
1 Agility The degree to which the software
is quick to change or modify.
Maintainability:
Maintainability: The ease with which the
faults can be located and fixed, quality of
the software can be improved or software
can be modified in the maintenance phase.
2 Modifiability The degree to which the
software is easy to
implement, modify and test in
the maintenance phase.
Maintainability:
Maintainability: The ease with which the faults can
be located and fixed, quality of the software can
be improved or software can be modified in the
maintenance phase.
3 Reada The degree to which the software
bility documents and programs are easy to
understand so that the faults can be easily
located and fixed in the maintenance phase.
Maintainability:
Maintainability: The ease with which the
faults can be located and fixed, quality of
the software can be improved or software
can be modified in the maintenance
phase.
4 Flexibility The ease with which changes
can be made in the software
in the maintenance phase.
Adaptability:
Adaptability: The degree to which the
software is adaptable to different
technologies and platforms.
1 Interoperability The degree to which the
system is compatible with
other systems.
Adaptability:
Adaptability: The degree to which the
software is adaptable to different
technologies and platforms.
2 Portability The ease with which the
software can be transferred
from one platform to
another platform.
Reliability:
Reliability: The degree to which the
software performs failure free functions.
1 Robustness The degree to which software
performs reasonably under
unexpected circumstances.
Reliability:
Reliability: The degree to which the
software performs failure free
functions.
2 Recoverability The speed with
which the software
recovers after the
occurrence of a failure.
The elements of a software quality system:
There are two goals of the software quality system
(SQS).
The first goal is to build quality into the software
from the beginning. This means assuring that the
problem or need to be addressed is clearly and
accurately stated, and that the requirements for the
solution are properly defined, expressed, and
understood.
The elements of a software quality system:
There are two goals of the software quality system
(SQS).
The second goal of the SQS is to keep that quality
in the software throughout the software life cycle
(SLC).
The elements of a software quality system:
The 10 elements of the SQS are as follows:
⚫ Standards,Processes & Metrics ⚫ Security & Safety
⚫ Reviews &Audits ⚫ Training
⚫ Testing ⚫ Supplier control
⚫ Defect analysis ⚫ Documentation
⚫ Configuration management (CM) ⚫ Risk management
The elements of a software quality system:
Requirement Analysis phase- gathering and
documentation of requirements
Design phase- preliminary and detailed design
of the software.
The elements of a software quality system:
Implementation and unit testing
phase- development of source code
and initial testing of independent units.
Integration and system testing phase-
testing the integrated parts of various
units and the system as a whole.
The elements of a software quality system:
Operational phase- delivering and installing
the software at the customer’s site.
Maintenance phase- removing
defects, accommodating changes and
improving the quality of the software after it
goes into operational phase.
Requireme Design Impleme Integration Operat Mainten
nt Analysis ntation and system ional ance
and unit testing
testing
Standards, process and √ √ √ √ √ √
metrics
Reviews and audits √ √ √ √ √
Software testing √ √ √ √ √ √
Defect management and √ √ √ √ √
trend analysis
Configuration management √ √ √ √ √
Risk analysis and √ √ √ √ √
assessment
Supplier control √ √ √ √
Training √ √ √ √ √ √
Documentation √ √ √ √ √
Safety and security √ √ √
Standards, Processes and Metrics
Standards provide procedures that must be
enforced during the software development life cycle.
Standard may be defined by
• Institute of Electrical and Electronics Engineers
(IEEE),
• American National Standards Institute (ANSI), or
• International Organization for Standardization
(ISO).
Standards, Processes and Metrics
These include the following:
1. Necessity: No standard will be observed for long if
there is no real reason for its existence.
2. Feasibility: Common sense tells us that if it is not
possible to comply with the tenets of a standard,
then it will be ignored.
3. Measurability: It must be possible to
demonstrate that the standard is being followed.
Standards, Processes and Metrics
Standards should be supported by concrete, well
defined and effective processes so that they are
effectively implemented.
Process is a collection of activities that are
required to produce a good quality product.
An effective process is well practiced, enforced,
documented and measured.
Standards, Processes and Metrics
The metrics must be used to measure the
effectiveness of software processes and
practices followed duringthe software
development.
Reviews
Reviews are conducted as a part of verification
activities.
Reviews are very effective as they are
conducted in early phases of software
development.
Reviews
They minimizethe probability of occurrence
of faults in the earlier phases of software
development i.e. before the validation activities
begins.
Reviews are cost effective and consume less
cost.
They increase the confidence about the
correctness of the software under development.
Reviews
• Reviews are of two types:
• Formal Reviews
• Informal Reviews.
Reviews
• The informal reviews are carried out
throughout software development life
cycle and are known as in process
reviews.
• Formal reviews are carried out at the
end of software development life cycle
phase.
Reviews & Audits
Reviewing
Informal reviews include walk-throughs
and inspections.
Walkthroughs are informal, but
scheduled, reviews, usually conducted in
and by peer groups.
Reviewing
The author of the subject component-a
design specification, test procedure,
coded unit, or the like-walks through his
or her component, explaining it to a
small group of peers.
Reviewing
Inspections are a more structured type of
walk- through.
Though the basic goal of an inspection-
removal of defects-is the same as that of
the walk- through, the format of the
meeting and the roles of the participants
are more strictly defined, and more formal
records of the proceedings are prepared.
Reviewing
Reviewing
10000
1000
Cost in units
100
10
1
Requirement analysis Design Implementation Integration and Maintenance
System testing
Time when fault is detected
Reviewing
Process reviews may be held at
any time. The purpose of a
process review is to examine the
success of the software process in
effect.
Reviewing
The physical audit (PA), often
included as a part of the CM
process, is an example of a formal
audit.
It compares the final form of the
code against the final
documentation for that code.
Reviewing
The goal of the physical audit is to
assure that the two products,
documentation and code, are in
agreement before being released to
the user or customer.
Reviewing
Another formal audit is the functional
audit. The functional audit ( FA),
again often a CM responsibility,
compares the test results with the
currently approved requirements to
assure that all requirements have
been satisfied.
Testing
Testing
Test planning begins during the
requirements phase and parallels the
requirements development.
As each requirement is generated, the
corresponding method of test for that
requirement should be a consideration.
A requirement is faulty if it is not testable.
Testing
Actual testing begins with the debugging and
early unit and module tests conducted
by the programmer.
An important aspect of complete testing is
user acceptance testing.
Test execution required the use of
detailed test procedures.
Testing
Test reports document the actual results of
the testing effort as it progresses.
For each test that is run, a report of the
expected results, actual results, and the
conclusions of the test conductor concerning
success of the test should be prepared.
Defect
Analysis
Defect Analysis
Defect analysis is the combination of defect
detection and correction, and defect trend
analysis.
The record of defects and their solutions
can serve to do the following:
Prevent defects from remaining
unsolved for inappropriate lengths of
time;
Prevent unwarranted changes;
Defect Analysis
The record of defects and their solutions
can serve to do the following:
Point out inherently weak areas in the
software;
Provide analysis data for development
process evaluation and correction;
Provide warnings of potential defects
through analysis of defect trends
Software Trouble Report
Code or Meaning
Field
Control Usually a sequential number
number for keeping track of the
individual STRs.
Software Trouble Report
Code Meaning
or
Field
An indication of the speed with which this STR
should be addressed:
E = emergency; work must stop until this STR is
Priority closed.
H = high; work is impeded while this STR is open.
M = medium; some negative impact on work
exists until this STR is closed.
L = low; This STR must be closed, but it does
not interfere with current work.
Software Trouble Report
Code Meaning
or
Field
The phase in which the error was made
that introduced the defect being described
Source (these will depend on the organization's
actual life cycle model and its phase
names):
•R = requirements, D = design, C = code, T
= test, O = operation
Software Trouble Report
Code Meaning
or
Field
An estimate of the impact of this defect
had it not been found until the software
was in operation:
•C = critical-persons could die or a firm
Severity could go out of business
•S = Serious = grave injury to persons or
organizations
•M = Moderate = injury or loss, but not
permanent
•T = Trivial = little or no impact
Software Trouble Report
Code Meaning
or
Field
The phase in which this defect was
detected (typically the same names as for
source):
Phase •R = requirements
•D = design
•C = code
•T = test
•O = operation
Software Trouble Report
Code Meaning
or
Field
Estimate: An estimate of the costs of
hours and correcting this defect and retesting
money the product
Actual: The actual costs of correcting
hours this defect and retesting the
and product
money
Software Trouble Report
Code or Meaning
Field
The defect detection technique with which
this defect was detected (each
Method organization will have its own set of
techniques):
•Q = quality audit
•W = walk-through
•I = inspection
Software Trouble Report
Code or Meaning
Field
•D = debugging or unit testing
Method
•T = testing
•A = user acceptance testing
•O = operation
Configuration management
Configuration management
Configuration identification is, as its name
implies, the naming, and documentation, of
each component (document, unit, module,
subsystem, and system) so that at any given
time, the particular component of interest
can be uniquely identified.
Configuration management
Configuration control is that activity that
prevents unauthorized changes to any
software product. Early in the SLC,
documentation is the primary product.
Configuration control takes on an
increasingly formal role as the documents
move from draft to final form.
Configuration management
Configuration accounting keeps track of the status of
each component.
The latest version or update of each software
component is recorded.
Thus, when changes or other activities are necessary
with respect to the component, the correct version of
the component can be located and used.
Training
Training assures that
the people involved with
software development,
and those people using
the software once it is
developed, are able to
do their jobs correctly.
Training
It is important to the quality
of the software that the
producers be educated in
the use of the various
development tools at his or
her disposal.
Training
The proper use of the
software once it has been
developed and put into
operation is another area
requiring education.
Vendor Management/Supplier Control
The following are
three basic types of
purchased software:
• Off-the-shelf;
• Tailored shell;
• Contracted.
Vendor Management
Off-the-shelf software is the package we buy at
the store. Microsoft Office, Adobe Photoshop,
virus checkers.
The second category may be called the
tailored shell. In this case, a basic, existing
framework is purchased and the vendor then
adds specific capabilities as required by the
contract.
Vendor Management
The third category is contracted software. This
is software that is contractually specified and
provided by a third-party developer.
Security
Security activities are applied
both to data and to the
physical data center itself.
These activities are
intended to protect the
usefulness of the software
and its environment.
Security
The highest quality software system is
of no use if the data center in which it is
to be used is damaged or destroyed.
Another frequent damager of the quality
of output of an otherwise high-quality
software system is data that has been
unknowingly modified.
Security
Additionally, though not really a
software quality issue per se, is the
question of theft of data.
Finally, the recent onslaught of hackers
and software attackers and the
burgeoning occurrences of viruses also
need to be considered.
Security
The software quality practitioner is
responsible for alerting management to the
absence, or apparent inadequacy, of
security provisions in the software.
In addition, the software quality
practitioner must raise the issue of data
center security and disaster recovery to
management's attention.
Safety
1. As computers and software grow in
importance and impact more and more of
our lives, the safety of the devices becomes
a major concern.
2. The literature records overdoses of
medicines, lethal doses of radiation, and
other catastrophic and near-
catastrophic events.
Safety
3. Every software project must consciously
consider the safety implications of the
software and the system of which it is a part.
4. The project management plan should
include a paragraph describing the safety
issues to be considered. If appropriate , a
software safety plan should be prepared.
Documentation
Documentation is very important part
of the quality system.
During the software development
phases, the SRS document, SDD,
test plans and test cases, user
manuals and system guides etc.
must be produced.
Documentation
The specified documentation
standards must be followed in order to
create these documents.
The documentation helps strongly in
the debugging process and hence the
maintenance of the product.
Documentation
S. No Document Name
1 Software requirement specification
2 Software design
3 Test plans
4 Test cases
5 Test reports
6 Risk management plan
7 User manual
8 Operation manual
9 Installation guide
Risk Management
There are several types of risk
associated with any software project.
Risks range from the simple, such as
the availability of trained personnel to
undertake the project, to more
threatening, such as improper
implementation of complicated
algorithms, to the deadly, such as failure
to detect an alarm in a nuclear plant.
Risk Management
Risk management includes identification
of the risk; determining the probability,
cost, or threat of the risk; and taking
action to eliminate, reduce, or accept
the risk.
