Software Metric
&
Software Estimation
Metric
• Numerical measures that quantify the degree to which software, a process or a
project possesses a given attribute
• Metrics help the followings
– Determining software quality level
– Estimating project schedules
– Tracking schedule process
– Determining software size and complexity
– Determining project cost
– Process improvement
Software Metric
Without measure it is impossible to make a plan, detect problems, and improve
a process and product.
A software engineer collects measure and develops metrics so that indicators
will be obtained
An indicator provides insight that enables the project manager or software
engineers to adjust the process, the project, or the product to make things better
The five essential, fundamental metrics:
– Size (LOC, etc.)
– Cost (in dollars)
– Duration (in months)
– Effort (in person‐month)
– Quality (number of faults detected)
Size metrics
• Ideally, we want to define a set of attributes for software size analogous
to human height and weight. That is, we want them to be fundamental
so that each attribute captures a key aspect of software size
• Length (LOC)
• Functionality
• Complexity
SLOC/LOC (Source lines of code)
• It is a software metric used to measure the size of a
software program by counting the number of lines in the
text of the program's source code.
• SLOC is typically used to predict the amount of effort that
will be required to develop a program, as well as to estimate
programming productivity or effort once the software is
produced.
• Simplest and most widely used metric.
Lines of Code Calculation
There are two major types of SLOC measures:
• Physical SLOC (LOC) and
• Logical SLOC (LLOC)
Specific definitions of these two measures vary, but the most common definition of
physical SLOC is a count of lines in the text of the program's source code including
comment lines. Blank lines are also included unless the lines of code in a section
consists of more than 25% blank lines. In this case blank lines in excess of 25% are not
counted toward lines of code.
Logical LOC attempts to measure the number of "statements", but their specific
definitions are tied to specific computer languages (one simple logical LOC measure for
C-like programming languages is the number of statement-terminating semicolons).
Example:
Consider this snippet of C code as an example of the ambiguity encountered when
determining SLOC:
for (i = 0; i < 100; i += 1) printf("hello"); /* How many lines of code is this? */
In this example we have:
1 Physical Lines of Code (LOC)
2 Logical Line of Code (LLOC) (for statement and printf statement)
1 comment line
Depending on the programmer and/or coding standards, the above "line of code"
could be written on many separate lines:
for (i = 0; i < 100; i += 1)
{
printf("hello");
} /* Now how many lines of code is this? */
In this example we have:
4 Physical Lines of Code (LOC): is placing braces work to be estimated?
2 Logical Line of Code (LLOC): what about all the work writing non-statement lines?
1 comment line: tools must account for all code and comments regardless of
comment placement.
LOC‐based Estimation - Example
Usage of SLOC measures
• SLOC measures are somewhat controversial, particularly in the way that they are
sometimes misused.
• Experiments have repeatedly confirmed that effort is correlated with SLOC, that
is, programs with larger SLOC values take more time to develop. Thus, SLOC can
be effective in estimating effort.
• SLOC is particularly ineffective at comparing programs written in different
languages unless adjustment factors are applied to normalize languages
Problems with LOC
• Lack of a Standard definition for line of code.
• Counting types of lines.
– Executable lines
– Data definition
– Comments
– Blank line
• Application written in multiple language.
• A report, screen, or GUI generator can generate thousands of lines of code in minutes
• Size variation due to individual programming style.
• Focuses on coding activity alone.
• Correlates poorly with quality and efficiency of code.
Functionality
• Many software engineers argue that length is misleading and
that the amount of functionality in an inherent product paints
a better picture of product size
• In particular, those who generate effort and duration estimates
from early development often prefer to estimate functionality
rather than physical size
Function points (FP)
• A function point count is a measure of the amount of
functionality in a product
• Analogy: For a given house, we can say how many square meters
it has (LOC) or we can say how many bedrooms and bathrooms it
has (FP).
• Derived using an empirical relationship based on countable
(direct) measures of software’s information domain (i.e., things in
the external behavior that will require processing)
• The idea was first put forward by Allan Albrecht of IBM in 1979.
Function Oriented Metric - Function Points
• Function Points are a measure of “how big” is the program, independently from the
actual physical size of it.
• It is a weighted count of several features of the program.
• Product size
• Use-Case evaluation –Each use-case screened for:
– External Input
– External Output
– External Inquiries
– Internal Logical Files
– External Interface Files
• For each such concept –> complexity
– LOW, AVERAGE, HIGH
• Number of F-points using formula
Information domain values
Number of external inputs (EI) : Originates from a user or is transmitted from another
application and provides distinct application-oriented data or control information
• A set of related inputs is counted as one input.
Number of external outputs (EO): Derived within the application and provides information to
the user.
• A set of related outputs is counted as one output.
Number of external inquiries (EQ): Interactive inputs requiring a response
• Each user query type is counted. Number of inquiries is the number of distinct interactive queries
which can be made by the users.
Number of internal logical files (ILF): A logical grouping of data that resides within the
application’s boundary and is maintained via external inputs
• Each logical file is counted. A logical file means groups of logically related data. Thus, logical files can
be data structures or physical files.
Number of external interface files (EIF): A logical grouping of data that resides external to the
application but provides data that may be of use to the application
Computing function points: step 1
• Establish count for information domain values
• Number of external inputs
• Number of external outputs
• Number of external inquiries
• Number of internal logical files
• Number of external interface files
• Associate complexity value with each count
• Simple, Average, Complex
• Determined based on organisational experience based criteria. Nonetheless, the
determination of complexity is somewhat subjective
• Compute count total
Computing function points: step 2
weighting factor
measurement parameter count simple avg. complex
number of user inputs X 3 4 6 =
number of user outputs X 4 5 7 =
number of user inquiries X 3 4 6 =
number of files X 7 10 15 =
number of ext.interfaces X 5 7 10 =
Unadjusted
Unadjusted Function
count-total Function Points
Points (UFP):
(UFP):
Assuming
Assuming
complexity all
all inputs
inputs with
multiplier with the
the same
same weight,
weight, all
all output
output with
with the
the same
same weight,
weight, …
…
function points
Complete
C omplete Formula
Formula for
for the
the Unadjusted
Unadjusted Function
Function Points
Points (UFP):
(UFP):
Wi Output Wo InquiryWin InternalFiles Wif ExternalInterfacesWei
Inputs
Computing function points: step 3
• Calculate complexity adjustment values (Fi, where iÎ[1..14])
‒ Does the system require reliable backup and recovery?
‒ Are data communications required?
‒ Are there distributed processing functions?
‒ Is performance critical?
‒ Will the system run in an existing, heavily utilised operating environment?
‒ Does the system require on-line data entry?
‒ Does the on-line data entry require the input transaction to be built over multiple screens or
operations?
‒ Are the master files updated on-line?
‒ Are the inputs, outputs, files or inquiries complex?
‒ Is the internal processing complex?
‒ Is the code designed to be reusable?
‒ Are conversion and installation included in the design?
‒ Is the system designed for multiple installations in different organisations?
Computing function points: step 3 (cont..)
• Answer each question using a scale of 0 (N/A or Not
Important) to 5 (absolutely essential/Very Important)
(No Influence = 0; Incidental = 1; Moderate = 2
Average = 3; Significant = 4 Essential = 5 )
• Add the 14 numbers
• This gives the total degree of influence (DI) = S Fi
• Sometimes called Complexity Measure(CM)
CM ComplexityMultiplier FComplexityMultiplier
Computing function points: step 4
Compute the technical complexity factor (TCF)
– TCF = 0.65 + 0.01 × DI
(0.65 and 0.01 are empirically derived constants)
– The technical complexity factor (TCF) lies between 0.65 and 1.35
The number of function points (FP) is then given by
FP = UFP × TCF
FP Template
Function Points: An Example
Spell Checker Specification
The spell checker accepts as input a document file, a dictionary file and an optional user
dictionary file. The checker lists all words in the document file not contained in either of
the dictionary files. The user can query the number of words processed and the number
of spelling errors found at any stage in the process.
A = #inputs = 2
B = #outputs = 3
C = #internal files = 1
D = #external files = 2
E = #queries = 2
UFC = 4 A + 5 B + 7 E + 10 D + 4 E = 58
Exercise Problems
• A target product has 7 simple inputs, 2 average input, and 10 complex inputs.
There are 56 average output, 8 simple inquires, 12 average master files, and 17
complex interfaces.
Determine the unadjusted function points (UFP).
• If the total degree of influence for the product of the question above is 49,
determine the number of function points.
Function Point Metric (CONT.)
• Suffers from a major drawback:
• the size of a function is considered to be independent of its
complexity.
• Extend function point metric:
• Feature Point metric:
• considers an extra parameter:
• Algorithm Complexity.
Function point (FP): Advantages
• Programming language independent
• Used readily countable characteristics that are determined
early in the software process
• Does not “penalize” inventive (short) implementations that
use fewer LOC than other more clumsy versions
LOC vs. FP
• Relationship between lines of code and function points depends upon the
programming language that is used to implement the software and the quality
of the design
• Empirical studies show an approximate relationship between LOC and FP
Software
Estimation
Software Cost Estimation
• Three main approaches to estimation:
• Empirical
• Heuristic
• Analytical
Software Cost Estimation
• Empirical techniques:
• An educated guess based on past experience.
• Heuristic techniques:
• Assume that the characteristics to be estimated can be expressed in
terms of some mathematical expression.
• Analytical techniques:
• Derive the required results starting from certain simple assumptions.
Why Estimate?
• At the beginning of a project customers usually want to know:
– How much?
– How long?
• Accurate estimates for “how much?” and “how long?” are critical to
project success.
– Good estimates lead to realistic project plans.
– Good estimates improve coordination with other business functions.
– Good estimates lead to more accurate budgeting.
– Good estimates maintain credibility of development team.
What to Estimate?
• System Size (LOC or Function points)
• Productivity (LOC/PM)
• Effort
• Duration
• Cost
How Long?
System Size Duration
Effort
Productivity Cost
How Much?
Effort
• Effort is the amount of labor required to complete a task.
• Effort is typically measured in terms of person months or person hours.
• The amount of effort it takes to compete a task is a function of developer and
process productivity.
• Productivity = LOC or function points (or unit of product) per month or hour.
Duration
• Duration is the amount of calendar time or clock time to complete a project or task.
• Duration is a function of effort but may be less when activities are performed
concurrently or more when staff aren’t working on activities full time.
Probability distribution of an estimate
• With every project estimate there is an associated probability of the estimate
accurately predicting the outcome of the project.
• Single-point estimates aren’t very useful because they don’t say what the probability
is of meeting the estimate.
How to Estimate?
• Techniques for estimating size, effort and duration:
• Analogy
• Ask an Expert
• Parametric (algorithmic) models
Estimating by Analogy
• Identify one or more similar past projects and use them (or parts of them) to
produce an estimate for the new project.
• Estimating accuracy is often improved by partitioning a project in parts and making
estimates of each part (errors cancel out so long as estimating is unbiased).
• Can use a database of projects from your own organisation or from multiple
organisations.
• Because effort doesn't scale linearly with size and complexity, extrapolating from
past experience works best when the old and new systems are based on the same
technology and are of similar size and complexity.
• Expert Judgement
• Delphi Estimation
Expert judgement
• Experts divide a software product into component units:
• e.g. GUI, database module, data communication module,
billing module, etc.
• an expert makes an educated guess of the problem size
after analyzing the problem thoroughly.
• Usually, the expert estimates the cost of the different
components (i.e. modules or subsystems) of the system
and then combines them to arrive at the overall
estimate.
Delphi Estimation:
• Team of Experts/stakeholders and a coordinator.
• Experts carry out estimation independently
• Share project information
• Each participant provides an estimate independently and anonymously
• All estimates are shared and discussed
• Each participant estimates again
• Continue until there is consensus, or exclude extremes and calculate average
• Experts re-estimate.
• Experts never meet each other to discuss their viewpoints.
Parametric (Algorithmic)
Models
• Formulas that compute effort and duration based on system size and other cost
drivers such as capability of developers, effectiveness of development process,
schedule constraints, etc.
• Most models are derived using regression analysis techniques from large
databases of completed projects.
• In general the accuracy of their predictions depends on the similarity between
the projects in the database and the project to which they are applied.
Heuristic Estimation Techniques
Heuristic techniques assume that the relationships among the different
project parameters can be modeled using suitable mathematical
expressions.
• Single Variable Model:
• Parameter to be Estimated=C1(Estimated Characteristic) d1
• Multivariable Model:
• Assumes that the parameter to be estimated depends on more
than one characteristic.
• Parameter to be Estimated=C1(Estimated Characteristic)d1+ C2(Estimated
Characteristic)d2+…
• Usually more accurate than single variable models.
Product Metrics
External Product Metrics
Measures of the Software in its Environment
• External metrics are those we can apply only by observing the
software product in its environment (e.g., by running it)
• Includes many measures, but particularly:
Reliability
Definition of Reliability
• Reliability is the probability that a system will execute without failure
in a given environment over a given period of time
• Implications:
• no single number for a given program - depends on how the
program is used (its environment)
• use probability to model our uncertainty
• time dependent
Reliability
Definition of Failure
• Formal view: Any deviation from specified behaviour
• Engineering view: Any deviation from required, specified or expected behaviour
(by environment) (by user)
Errors, Faults and Failures
Definitions
• A (human) error is a mistake or oversight on the part of a designer or programmer,
which may cause ...
• A fault, which is a mistake in the software, which in turn may cause ...
• A failure when the program is run in certain situations
Defects
• A defect is usually defined as a fault or a failure:
Defects = Faults + Failures
(or sometimes just Faults or just Failures)
Defect Density Metric
Defect Density
• Defect density is a standard reliability metric:
Defect Density = Number of defect found
System Size
• Size is normally measured in KLOC (1000’s of lines of code), so units of defect
density are defects found per 1000 lines
• Widely used as an indicator of software quality
Internal Product Metrics
Measures of the Product Itself
The vast majority of metrics in practice are internal product metrics,
measures of the software code, design or functionality itself, independent of
its environment
These measures are easy to make and easy to automate, but it’s not always
clear which attributes of the program they characterize (if any)
Code Metrics
Software Size
• The simplest and most enduring product metric is the size of the product, measured
using a count of the number of lines of source code (LOC), most often quoted in 1000’s
(KLOC)
• It is used in a number of other indirect measures, such as
• productivity ( LOC / effort )
• effort / cost estimation ( Effort = f(LOC) )
• quality assessment / estimation ( defects / LOC )
• Many similar measures are also used
• KDSI (1000’s of delivered source instructions)
• NLOC (non-comment lines of code)
• number of characters of source or bytes of object code
Better Size Measures
Fundamental Size Attributes of Software
• Length - the physical size of the software (LOC will do as measure)
• Functionality - the capabilities provided to the user by the software
(how big / rich is the set of functions provided)
• Complexity - how complex is this software?
• Problem complexity - measures the complexity of the underlying problem
• Algorithmic complexity - measures the complexity / efficiency of the solution
implemented by the software
• Structural complexity - measures the structure of the program used to
implement the algorithm (includes control structure, modular structure, data flow
structure and architectural structure)
• Cognitive complexity - measures the effort to understand the software
Code Complexity Metrics
Better Measures of Source Code
• Realization that we need something better approaching cognitive complexity led
to work on complexity metrics for code
• Early explorations measured characteristics such as:
• number / density of decision (if) statements
• number / depth of blocks / loops
• number / average length of methods / classes
• and many more…
• Best known and accepted source code complexity measures are
• Halstead’s “Software Science” metrics
• McCabe’s “Cyclomatic Complexity” and “Data Complexity” metrics
Halstead’s “Software Science” Metric
Operators and Operands
• Program source code considered as a sequence of tokens, each of which is either
an operator or an operand
n1 = number of unique (different) operators
n2 = number of unique (different) operands
N1 = total number of operator uses
N2 = total number of operand uses
Halstead metrics: Example
Halstead metrics are sensitive to…
Halstead’s “Software Science” Metric
The Software Science Predictive Theory
• Using n1, n2, N1 and N2 as a basis, Halstead formulated a theory of software complexity and effort
Length of program N = N1 + N2
Vocabulary of program n = n1 + n2
Volume of program V=N log2 n
Difficulty D = ( n1 / 2 ) * ( N2 / n2 )
Cyclomatic complexity
• Cyclomatic complexity is a software metric that provides a quantitative
measure of the logical complexity of a program
• It was proposed by Tom McCabe in 1976
• This number, based on a graph-theoretic concept, counts the number of
linearly independent paths through a program (we have seen this in the
context of testing with basis path testing)
• McCabe felt that the number of such paths was a key indicator not just
of testability but also of complexity
McCabe’s Cyclomatic Complexity
• Determines the logical complexity of a graph
• typically the graph is a flow graph of a function or procedure
• can also be a graphical representation of an FSM.
• Define:
• independent path: any path that introduces at least one new set of
statements or a new condition; must move along at least one edge that
has not been traversed before.
• Basis Set: a maximal set of independent paths (covers all conditions and
statements).
Basis for McCabe's metric
Uses a Program Control Graph
• Measure of complexity is number of different paths through the program control graph
• Number of basic paths (all paths composed of basic paths)
Cyclomatic Number is the number of basic paths.
V(G) = Cyclomatic Complexity
= Number of predicates nodes in program + 1
(A predicate node is a node in the graph with 2 or more out going arcs.)
= E (#of edges) - N (#of nodes) + 2 x P
(where P is the number of connected components (i.e P= the number of
nodes that have exit points)
= R (Closed Region)+1
Example: set of independent paths which comprise a basis set for the
following flow graph:
Path1: a, c, f R1 a
Path2: a, d, c, f R2 R5
Path3: a, b, e, f b c d
Path4: a, b, e, b, e, f R3 R4
Path5: a, b, e, a, c, f e f
E=2 N=4 P = 2 CC = 2-4+2*2=2
McCabe’s “Cyclomatic Complexity:
Metric
Flow Graphs Again
• If the control flow graph G of program P has e edges and n
nodes, then the cyclomatic complexity v of P is
v (P) = e – n + 2X1
• v (P) is the number of linearly independent paths in G
• Example
e = 16 n = 13
v (P) = 16 – 13 + 2 = 5
• More simply, if d is the number of decision nodes in G then
v (P) = d + 1
• McCabe proposed that for each module P
v (P) < 10 Source: Fenton, Agena Corp. 2000
Example
McCabe’s complexity: Example
Final Word
“The models are just there to help, not to
make the management decisions for you.”
--Barry Boehm
