Unit-2

Software Quality Metrics Product Quality Metrics

Defect Density :
Every software is assessed for quality, scalability, functionality, security, and
performance, as well as other important factors. The defect identification procedure
guarantees that the final product meets all of the client’s requirements and standards.
To guarantee that software is flawless, software developers use the defect density
function to find the software’s quality.
Thus, in a nutshell, it can be concluded that Quality is inversely proportional to
Defects (as the no of defects increases, the software quality decreases)
Defect density is a mathematical value that indicates the number of flaws found in
software or other parts over the period of a development cycle. It is then split by the
software’s size. In a nutshell, it’s used to determine whether or not the software will
be released.
The fault density is important in the Software Development Life ycle(SDLC) cannot
be overstated due to the following reasons:
1. It is used to determine the number of software flaws.
2. The testing team will be able to hire a third-party inspection crew for re-
engineering and substitutions.
3. Developers can also use defect density to identify components that are prone to
faults in the future.
4. As a consequence, testers can focus on the proper areas and provide the highest
return on investment with little resources.

The defect density of software is estimated by dividing the sum of flaws by the size
of the software.
Defect Density = Total Defect/Size
Formula to calculate defect density:
Defect Density = Average number of Defects/KLOC
Here, KLOC is Line of codes per thousands
One flaw per 1000 lines (LOC) is deemed acceptable, according to best practices.
The KLOC defect density standard is one such example. Function Points are used to
measure the size of software or code (FP).

Steps to Calculate Defect Density

Let’s consider an example to calculate the defect density in software.

The software has the following integrated modules with respect to the specifications.
Each module has the following number of bugs discovered:
1. Module 1 = 5 Bugs
2. Module 2 = 10 Bugs
3. Module 3 = 20 Bugs
4. Module 4 = 15 Bugs
5. Module 5 = 5 Bugs
Total bugs = 5+10+20+15+5
= 55
The total line of code for each module is:
1. Module 1 = 500 LOC
2. Module 2 = 1000 LOC
3. Module 3 = 1500 LOC
4. Module 4 = 1500 LOC
5. Module 5 = 1000 LOC
Total LOC = 500+1000+1500+1500+1000
= 5500
The Defect Density is calculated as:
Defect Density = (55/5500)
= 0.01 defects/LOC
= 10 defects/KLOC

Uses of Defect Density

For the development of software and its components, defect density is regarded as an
industry standard. The following are some of the uses of the defect density:
1. It includes a development procedure for calculating the number of flaws, which
allows developers to identify weak regions that need to be thoroughly tested.
2. Organizations also desire fault density when releasing a product and comparing
performance, security, quality, scalability, and other factors. Once faults have
 been identified, developers may begin making improvements to decrease the
number of defects.
3. The defect density approach aids developers in determining the impact of a
reduction on software quality.
In many aspects, the usage of fault density is insignificant. Developers, on the other
hand, can use this model to estimate the remaining problems once they’ve built up
common defects. Developers can use this approach to create a database of
commonly used terms.

Factors Affecting Defect Density Metrics

The Defect density is calculated by dividing total faults by software size. The idea is
to find problems that are genuinely important, not just any defects. As a
consequence, it’s critical to comprehend the components that lead to a successful
outcome. Before beginning this procedure, developers and the testing team must set
up all of the essential circumstances. This enables developers to accurately track the
impacted locations, resulting in very accurate findings.
Factors affecting density are:
 Types of flaws
 The code utilized is critical and sophisticated.
 The developer’s and testing team’s abilities
 Calculating the defect density takes time.
 The software’s efficiency and performance are the most important factors to
consider.

Advantages of Defect Density

For software testers and developers, defect density has various advantages. It meets
a wide range of technical and analytical needs in addition to offering superior fault
measuring accuracy. Having precise findings on hand can assist software engineers
to maintain confidence in the quality and performance of their built program. Other
benefits of defect density include-
 The ability for developers to assure that the product is ready for launch does not
require any more testing.
 Developers and testers may estimate the amount of testing and rework that will
be needed to correct the mistakes.
 Testers can track down and identify components that pose a high danger.
 The testing team can figure out how much training is needed to finish the
procedure.
 It is possible to detect and correct areas that want improvement.

At what value of defect density does the software become unacceptable?

There are no such industrial pre-defined norms about the range of defect densities,
but a higher value of defect density results in the unacceptability of the proposed
software. Yet, a suitable range as per the current IT industry trends would be in the
range between 0.1-0.5 Defect/LOC, after this range software is not accepted as per
this standard.

Software Quality Metrics:

 Software quality metrics means the ability of a software product to be and to perform as per
the standard defined. Metric means standards.
 The software quality metrics ensures that the software product is of highest quality and
standard.
 There are three major software quality metrics. These are:

Software Quality Metrics

Software Quality Metrics : Customer Problem Metrics

 The quality metrics used in software industry that is responsible for

measuring the problems encountered by the customers while using the
product.
 From the customers point of view, all the problems they encountered
while using the software product are problems with the software
including the valid defects.
 The usability problems, unclear documentation and errors are the
problems that cannot be considered as valid defects.
 The customer problem metrics is usually expressed in terms of problems
per unit month i.e. PUM.

PUM =Total problems reported by a customer for a period of

time + Total number of license months of the software during the
period.

Here,

No of license months = Number of install licenses of

software * Number of months in calculation period.

 PUM is usually calculated for each month after the software is

released to the market, and also for monthly averages by year.

Software Quality Metrics : Customer Satisfaction Metrics

 Customer Satisfaction Metric deals with the overall quality of the product
and how much a customer is satisfied with that software product.
 Customer satisfaction is often measured by customers survey data via a
five point scale. These are:
1. Very Satisfied.
2. Satisfied.
3. Neutral.
4. Dissatisfied.
5. Very Dissatisfied.

Customer Satisfaction Metrics

1. Based on a five point scale data, several metrics with slight variations
can be constructed and used. These are:
1. Percentages of completely satisfied customers.
2. Percentage of satisfied customers.
3. Percentage of dissatisfied customers.
4. Percentage of non-satisfied customers.

Note: Generally, ”Percentage of satisfied customers is used.”

 Net Satisfaction Index(NSI) is also used by some companies for customer
satisfaction metrics.
 The net satisfaction index ha following weighing criteria:
1. Completely Satisfied = 100%.
2. Satisfied = 75%.
3. Neutral = 50%.
4. Dissatisfied = 25%.
5. Completely Dissatisfied = 0%.

Software Quality Metrics : Software Maintenance Metrics

 A software product can be called as in maintenance phase when its

development is complete and is released to the market.
 During this interval, the defect arrival by time interval and customer
problem is called as de facto metrics.
 Also, the number of defect or problems arrived mainly depends upon the
development phase and not on the maintenance phase.
 Therefore, certain metrics are required for software maintenance. These
are:
1. Backlog Management Index(BMI)
 The metric that is used to manage the backlog of open and
unresolved problems. It can be defined as:
BMI = (Number of problems closed during the month
* 100)/(Number of problems arrived during the
month)

2. Fix Response Time(FRT)

 While developing the software product, certain guidelines

regarding reporting of defect are made. It includes the
guideline for time limit of fixes that must available against
the reported defects.
 The fix response time for more severe defects is low and the
fix response time for less severe defect is high.
 The fix response time metric can be calculated as:
FRT = (Mean time of all problems from open to
closed)
3. Fix Quality
 Fix quality of software product is the number of defects
fixed.
 Every software product contains defects. But, Fix Quality
ensures that the defects in software product are fixed on
time as per the quality standard.
 Fix quality ensures that the fixes for the defects should not
turn out to be defective.
  A fix can be called as defective if it does not fixes the
problems and generates new problems in that software
product.

What is Functional Point Analysis?

 Function Point Analysis was initially developed by Allan J. Albrecht in
1979 at IBM and has been further modified by the International
Function Point User’s Group (IFPUG) in 1984, to clarify rules,
establish standards, and encourage their use and evolution.
 Allan J. Albrecht gave the initial definition, Functional Point Analysis
gives a dimensionless number defined in function points which we have
found to be an effective relative measure of function value delivered to
our customer.
 A systematic approach to measuring the different functionalities of a
software application is offered by function point metrics.
 Function point metrics evaluate functionality from the perspective of the
user, that is, based on the requests and responses they receive.

Objectives of Functional Point Analysis

1. Encourage Approximation: FPA helps in the estimation of the work,

time, and materials needed to develop a software project.
Organizations can plan and manage projects more accurately when a
common measure of functionality is available.
2. To assist with project management: Project managers can monitor
and manage software development projects with the help of FPA.
Managers can evaluate productivity, monitor progress, and make well-
informed decisions about resource allocation and project timeframes
by measuring the software’s functional points.
3. Comparative analysis: By enabling benchmarking, it gives
businesses the ability to assess how their software projects measure
up to industry standards or best practices in terms of size and
complexity. This can be useful for determining where improvements
might be made and for evaluating how well development procedures
are working.
4. Improve Your Cost-Benefit Analysis: It offers a foundation for
assessing the value provided by the program concerning its size and
complexity, which helps with cost-benefit analysis. Making educated
judgements about project investments and resource allocations can
benefit from having access to this information.
5. Comply with Business Objectives: It assists in coordinating
software development activities with an organization’s business
objectives. It guarantees that software development efforts are
 directed toward providing value to end users by concentrating on user-
oriented functionality.

Types of Functional Point Analysis

There are two types of Functional Point Analysis:

Types of Functional Point Analysis

1. Transactional Functional Type

1. External Input (EI): EI processes data or control information that
comes from outside the application’s boundary. The EI is an
elementary process.
2. External Output (EO): EO is an elementary process that generates
data or control information sent outside the application’s
boundary.
3. External Inquiries (EQ): EQ is an elementary process made up of
an input-output combination that results in data retrieval.

2. Data Functional Type

1. Internal Logical File (ILF): A user-identifiable group of logically
related data or control information maintained within the boundary of
the application.
2. External Interface File (EIF): A group of users recognizable logically
related data allusion to the software but maintained within the
boundary of another software.
 Functional Point Analysis

Benefits of Functional Point Analysis

Following are the benefits of Functional Point Analysis:

1. Technological Independence: It calculates a software system’s

functional size independent of the underlying technology or
programming language used to implement it. As a result, it is a
technology-neutral metric that makes it easier to compare projects
created with various technologies.
2. Better Accurate Project Estimation: It helps to improve project
estimation accuracy by measuring user interactions and functional
needs. Project managers can improve planning and budgeting by using
the results of the FPA to estimate the time, effort and resources
required for development.
3. Improved Interaction: It provides a common language for business
analysts, developers, and project managers to communicate with one
another and with other stakeholders. By communicating the size and
complexity of software in a way that both technical and non-technical
audiences can easily understand this helps close the communication
gap.
4. Making Well-Informed Decisions: FPA assists in making well-
informed decisions at every stage of the software development life
cycle. Based on the functional requirements, organizations can use the
results of the FPA to make decisions about resource allocation, project
prioritization, and technology selection.
5. Early Recognition of Changes in Scope: Early detection of changes
in project scope is made easier with the help of FPA. Better scope
change management is made possible by the measurement of
 functional requirements, which makes it possible to evaluate additions
or changes for their effect on the project’s overall size.

Disadvantage of Functional Point Analysis

Given below are some disadvantages of Functional Point Analysis:

1. Subjective Judgement: One of the main disadvantages of Functional

Point Analysis is it’s dependency on subjective judgement i.e. relying
on personal opinions and interpretations instead of just using clear,
measurable standards.
2. Low Accuracy: It has low evaluation accuracy as it’s dependency on
subjective judgement.
3. Time Consuming: Functional Point Analysis is a time consuming
process, particularly during the initial stages of implementation.
4. Steep Learning Curve: Learning FPA can be challenging due to its
complexity and the length of time required to gain proficiency.
5. Less Research Data: Compared to LOC-based metrics, there is
relatively less research data available on function points.
6. Costly: The need for thorough analysis and evaluation can result in
increased project timelines and associated costs.

Characteristics of Functional Point Analysis

 We can calculate the functional point with the help of the number of
functions and types of functions used in applications. These are classified
into five types:

Types of FP Attributes or Information Domain Characteristics

Measurement Parameters Examples

Number of External Inputs (EI) Input screen and tables

Number of External Output (EO) Output screens and reports

Number of external inquiries (EQ) Prompts and interrupts

Number of internal files (ILF) Databases and directories

Number of external interfaces (EIF) Shared databases and shared routines

  Functional Point helps in describing system complexity and also shows
project timelines.
 It is majorly used for business systems like information systems.
 FP is language and technology independent, meaning it can be applied to
software systems developed using any programming language or technology
stack.
 All the factors mentioned above are given weights, and these weights are
determined through practical experiments in the following table.
Weights of 5 Functional Point Attributes
Measurement
Parameter Low Average High

Number of external
3 4 6
inputs (EI)

Number of external
4 5 7
outputs (EO)

Number of external
3 4 6
inquiries (EQ)

Number of internal
7 10 15
files (ILF)

Number of External
5 7 10
Interfaces (EIF)

Functional Complexities help us in finding the corresponding weights, which

results in finding the Unadjusted Functional point (UFp) of the Subsystem.
Consider the complexity as average for all cases. Below-mentioned is the
way how to compute FP.

Weighing Factor

Measurement Total_Count Simple Average Complex

Parameter Count
 Weighing Factor

Number of
Measurement
external inputs 32 32*4=128 3 4 6
(EI)
Parameter Count

Number of
external 60 60*5=300 4 5 7
outputs (EO)

Number of
external 24 24*4=96 3 4 6
inquiries (EQ)

Number of
internal files 8 8*10=80 7 10 15
(ILF)

Number of
external 2 2*7=14 5 7 10
interfaces (EIF)

Algorithms
used Count 618
total →

From the above tables, Functional Point is calculated with the following

FP = Count-Total * [0.65 + 0.01 * ⅀(fi)]

formula

= Count * CAF

CAF = [0.65 + 0.01 * ⅀(fi)]

Here, the count-total is taken from the chart.

1. ⅀(fi) = sum of all 14 questions and it also shows the complexity factor –

2. CAF varies from 0.65 to 1.35 and ⅀(fi) ranges from 0 to 70.
CAF.

3. When ⅀(fi) = 0, CAF = 0.65 and when ⅀(fi) = 70, CAF = 0.65 + (0.01*70)
= 0.65 + 0.7 = 1.35
Differentiate between FP and LOC
function point LOC

1. FP is specification based. 1. LOC is an analogy based.

2. FP is language independent. 2. LOC is language dependent.

3. FP is user-oriented. 3. LOC is design-oriented.

4. It is extendible to LOC. 4. It is convertible to FP (backfiring)

In-Process Quality Metrics:

 Because our goal is to understand the programming process and to

learn to engineer quality into the process, in-process quality metrics
play an important role.
 In-process quality metrics are less formally defined than end-product
metrics, and their practices vary greatly among software developers.
 On the one hand, in-process quality metrics simply means tracking
defect arrival during formal machine testing for some organizations.
 On the other hand, some software organizations with well-established
software metrics programs cover various parameters in each phase of
the development cycle.
 In this section we briefly discuss several metrics that are basic to
sound in-process quality management.

4.2.1 Defect Density During Machine Testing

Defect rate during formal machine testing (testing after code is integrated into the system library) is usually
positively correlated with the defect rate in the field. Higher defect rates found during testing is an indicator that
the software has experienced higher error injection during its development process, unless the higher testing
defect rate is due to an extraordinary testing effort—for example, additional testing or a new testing approach
that was deemed more effective in detecting defects. The rationale for the positive correlation is simple:
Software defect density never follows the uniform distribution. If a piece of code or a product has higher testing
defects, it is a result of more effective testing or it is because of higher latent defects in the code. Myers (1979)
discusses a counterintuitive principle that the more defects found during testing, the more defects will be found
later. That principle is another expression of the positive correlation between defect rates during testing and in
the field or between defect rates between phases of testing.

This simple metric of defects per KLOC or function point, therefore, is a good indicator of quality while the
software is still being tested. It is especially useful to monitor subsequent releases of a product in the same
development organization. Therefore, release-to-release comparisons are not contaminated by extraneous
factors. The development team or the project manager can use the following scenarios to judge the release
quality:

 If the defect rate during testing is the same or lower than that of the previous release (or a similar
product), then ask: Does the testing for the current release deteriorate?
 If the answer is no, the quality perspective is positive.
 If the answer is yes, you need to do extra testing (e.g., add test cases to increase coverage, blitz
test, customer testing, stress testing, etc.).
 If the defect rate during testing is substantially higher than that of the previous release (or a similar
product), then ask: Did we plan for and actually improve testing effectiveness?
 If the answer is no, the quality perspective is negative. Ironically, the only remedial approach that
can be taken at this stage of the life cycle is to do more testing, which will yield even higher defect
rates.
 If the answer is yes, then the quality perspective is the same or positive.

4.2.2 Defect Arrival Pattern During Machine Testing

Overall defect density during testing is a summary indicator. The pattern of defect arrivals (or for that matter,
times between failures) gives more information. Even with the same overall defect rate during testing, different
patterns of defect arrivals indicate different quality levels in the field. Figure 4.2 shows two contrasting patterns
for both the defect arrival rate and the cumulative defect rate. Data were plotted from 44 weeks before code-
freeze until the week prior to code-freeze. The second pattern, represented by the charts on the right side,
obviously indicates that testing started late, the test suite was not sufficient, and that the testing ended
prematurely.

FIGURE 4.2 Two Contrasting Defect Arrival Patterns During Testing

The objective is always to look for defect arrivals that stabilize at a very low level, or times
between failures that are far apart, before ending the testing effort and releasing the
software to the field. Such declining patterns of defect arrival during testing are indeed the basic assumption of
many software reliability models. The time unit for observing the arrival pattern is usually weeks and
occasionally months. For reliability models that require execution time data, the time interval is in units of CPU
time.

When we talk about the defect arrival pattern during testing, there are actually three slightly different metrics,
which should be looked at simultaneously:

 The defect arrivals (defects reported) during the testing phase by time interval (e.g., week). These are the
raw number of arrivals, not all of which are valid defects.
 The pattern of valid defect arrivals—when problem determination is done on the reported problems. This
is the true defect pattern.
 The pattern of defect backlog overtime. This metric is needed because development organizations cannot
investigate and fix all reported problems immediately. This metric is a workload statement as well as a
quality statement. If the defect backlog is large at the end of the development cycle and a lot of fixes have
yet to be integrated into the system, the stability of the system (hence its quality) will be affected.
Retesting (regression test) is needed to ensure that targeted product quality levels are reached.

4.2.3 Phase-Based Defect Removal Pattern

The phase-based defect removal pattern is an extension of the test defect density metric. In addition to testing,
it requires the tracking of defects at all phases of the development cycle, including the design reviews, code
inspections, and formal verifications before testing. Because a large percentage of programming defects is
related to design problems, conducting formal reviews or functional verifications to enhance the defect removal
capability of the process at the front end reduces error injection. The pattern of phase-based defect removal
reflects the overall defect removal ability of the development process.

With regard to the metrics for the design and coding phases, in addition to defect rates, many development
organizations use metrics such as inspection coverage and inspection effort for in-process quality
management. Some companies even set up "model values" and "control boundaries" for various in-process
quality indicators. For example, Cusumano (1992) reports the specific model values and control boundaries for
metrics such as review coverage rate, review manpower rate (review work hours/number of design work
hours), defect rate, and so forth, which were used by NEC's Switching Systems Division.

Figure 4.3 shows the patterns of defect removal of two development projects: project A was front-end loaded
and project B was heavily testing-dependent for removing defects. In the figure, the various phases of defect
removal are high-level design review (I0), low-level design review (I1), code inspection (I2), unit test (UT),
component test (CT), and system test (ST). As expected, the field quality of project A outperformed project B
significantly.

FIGURE 4.3 Defect Removal by Phase for Two Products

4.2.4 Defect Removal Effectiveness

Defect removal effectiveness (or efficiency, as used by some writers) can be defined as
follows:

Because the total number of latent defects in the product at any given phase is not known, the denominator of
the metric can only be approximated. It is usually estimated by:

Defects removed during the phase + defects found later

The metric can be calculated for the entire development process, for the front end (before code integration),
and for each phase. It is called early defect removal and phase effectiveness when used for the front end and
for specific phases, respectively. The higher the value of the metric, the more effective the development
process and the fewer the defects escape to the next phase or to the field. This metric is a key concept of the
defect removal model for software development. (In Chapter 6 we give this subject a detailed treatment.) Figure
4.4 shows the DRE by phase for a real software project. The weakest phases were unit test (UT), code
inspections (I2), and component test (CT). Based on this metric, action plans to improve the effectiveness of
these phases were established and deployed.

FIGURE 4.4 Phase Effectiveness of a Software Project

Difference between quality measures, quality metrics &

quality indicators.
Software development & testing experts rightly declare “We cannot
control what we cannot measure”. Thus in order to successfully
measure quality of any software application, first of all we need to
understand the key differences between the elements of chain of
three interconnected quality terms like; measures, metrics &
indicators.

Quality Measure Quality Metric Quality Indicator

An indicator is a device or
A metric is a quantitative variable, which can be set to
A measure is to ascertain or measure of the degree to
a prescribed state, based on
appraise by comparing to a which a system,
standard. component, or process the results of a process or the
possesses a given attribute. occurrence of a specified
condition.
A standard or unit of
measurement covers:
The extent, dimensions, It is a calculated or composite An indicator usually
capacity of anything, indicator based upon two or compares a metric with a
especially as determined
more measures. baseline or expected result.
by a standard.

Indicator help the decision-

A measure gives very little
A metric is a comparison of makers to make a quick
or no information in the
two or more measures like comparison that can provide
absence of a trend to follow
defects per thousand source a perspective as to the
or an expected value to
lines of code. “health” of a particular
compare against.
aspect of the project.
Software quality indicators
Software quality metrics is act as a set of tools to
Measure does not provide used to assess throughout the improve the management
enough information to make development cycle whether the capabilities of personnel
meaningful decisions. software quality requirements responsible for monitoring
are being met or not software development
projects.
SOFTWARE QUALITY INDICATORS:
The software quality indicators address management concerns, take
advantage of data that is already being collected, are independent
of the software development methodology being used, are specific
to phases in the development cycle, and provide information on the
status of a project.
Software Testing experts like ISTQB advanced certified Test
Managers prescribe following quality indicators for use during the
software testing & development life cycle.
1) Progress: Measures the amount of work accomplished by the
developer in each phase. This measure flows through the
development life cycle with a number of requirements defined and
baselined, then the amount of preliminary and detailed designed
completed, then the amount of code completed, and various levels
of tests completed.
2) Stability: Assesses whether the products of each phase are
sufficiently stable to allow the next phase to proceed. This measures
the number of changes to requirements, design, and
implementation.
3) Process compliance: Measures the developer’s compliance
with the development procedures approved at the beginning of the
project. Captures the number of procedures identified for use on the
project versus those complied with on the project.
4) Quality evaluation effort: Measures the percentage of the
developer’s effort that is being spent on internal quality evaluation
activities. Percent of time developers are required to deal with
quality evaluations and related corrective actions.
5) Test coverage: Measures the amount of the software system
covered by the developer’s testing process. For module testing, this
counts the number of basis paths executed/covered, & for system
testing it measures the percentage of functions tested.
6) Defect detection efficiency: Measures how many of the
defects detectable in a phase were actually discovered during that
phase. Starts at 100% and is reduced as defects are uncovered at a
later development phase.
7) Defect removal rate: Measures the number of defects detected
and resolved over a period of time. Number of opened and closed
system problem reports (SPR) reported through the development
phases.
8) Defect age profile: Measures the number of defects that have
remained unresolved for a long period of time. Monthly reporting of
SPRs remaining open for more than a month’s time.
9) Defect density: Detects defect-prone components of the
system. Provides measure of SPRs / Computer Software Component
(CSC) to determine which is the most defect-prone CSC.
10) Complexity: Measures the complexity of the code. Collects
basis path counts (cyclomatic complexity) of code modules to
determine how complex each module is.
Following table illustrates the software quality indicators
used during the development phase.
Conclusion: The quality indicators discussed above have certain
characteristics related to quality measures. Accordingly software
testing experts like Test Managers have made few conclusions like.
1) Quality measures must be oriented toward management goals.
One need not have extensive familiarity with technical details of the
project.
2) Quality measures should reveal problems as they develop and
suggest corrective actions that could be taken.
3) Quality measures must be easy to use. They must not be
excessively time consuming, nor depend heavily on extensive
software training or experience. Measures that are clearly specified,
easy to calculate, and straightforward to interpret are needed.
4) Quality measures must be adequately flexible.
5) We must not use quality measurement just for the purpose of
quality assurance & software testing managers & engineers must
come out of the narrow notions they happen to have about the
constituents of the quality.