Process Improvement

CIS 376 Bruce R. Maxim UM-Dearborn

Process Improvement Goals

Understanding existing processes Introduce process changes to improve quality, reduce costs, or accelerate schedules Industry is demanding increased attention to quality in general Most process improvement work focuses on defect reduction and prevention There are other process attributes that deserve our attention

Process Improvement Attributes - part 1

Understandability - degree to which a process is well defined and understood Visibility - process activities have results that are externally recognizable Supportability - process activities supported by CASE tools Acceptability - defined processes are used and accepted by software engineers

Process Improvement Attributes - part 2

Reliability - process is defined so that errors are avoided or trapped before product errors result Robustness - process can continue despite unexpected problems Maintainability - process can evolve to reflect changing organizational requirements or identified process improvements Rapidity - the time required to complete a system from specification to delivery

Process Improvement Stages

Process analysis
modeling and quantitative analysis of existing processes

Improvement identification
quality, cost, and scheduling bottlenecks located

Process change introduction

modify process to remove bottlenecks

Process change training

train staff involved in process revision proposals

Change tuning
process improvements are revised and allowed to evolve

Process Improvement Activities

Introduce process change Analyse process Identify improvements Train engineers

Tune process changes

Process model

Process change plan

Training plan

Feedback on improvements

Revised process model

Process and Product Quality

Closely related to one another Good processes are usually required to produce good products In manufacturing applications, process is principle determinant of quality For design-based activities, the capabilities of the designers are also important

Product Quality Factors

Development technology
for large projects with average capability this is the main determinant of product quality

Quality of people involved

for small projects the developer capability is the main determinant of product quality

Process quality
significant for both small and large projects

Cost, time, and schedule constraints

unrealistic schedules can doom the quality of most products

Process Analysis and Modeling

Process analysis
study of existing processes to understand relationships among process components allows comparisons with other processes

Process modeling
documentation of process in which the tasks, roles, and entities used are recorded best to represent models graphically several different perspectives may be used (e.g. activities, deliverables, etc.) model should be examined for weaknesses, this involves discussion with stakeholders

Process Model Elements - part 1

Activity - (round edged rectangle)
has clearly defined objective, entry, and exit conditions

Process - (round edged rectangle with shadow)

set of coherent activities with agreed upon objective

Deliverable - (rectangle with shadow)

tangible output of an activity predicted by project plan

Condition - (parallelogram)
process or activity pre- or post-conditions

Process Model Elements - part 2

Role - (circle with shadow)
defined and bounded area of responsibility

Exception - (double edged box))

description of how to modify the process if anticipated or unanticipated events occur

Communication - (arrow)
exchange of information between people and/or machines

Process Model Example

Rle Post-cond ition Pre-condition Module compiles without syntax errors Test engineer Responsible for Test module Process All defined tests run on module

Outputs Signed-of f test record Module test data

Input Module specification

Process Exceptions
Process models cant represent how to handle exceptions
key people are lost prior to a critical review failure of e-mail server for several days organizational reorganization request to respond to change requests

General procedure is to suspend the process model and follow RMMM plans augmented with the managers own initiatives

Process Measurement
Wherever possible quantitative process data should be collected Organizations without process standards may have to be define processes before measurements can be made (since they wont know what to measure) Process measurements should be used to assess process improvements Organization objectives drive process improvement, not measurements

Process Measurement Classes

Time taken to complete process activities
e.g. calendar time to complete a milestone

Resources required to complete processes or activities

e.g. person months

Number of event occurrences

e.g. number of defects found

Goal Question Metric Paradigm

Goals
What is the organization trying to achieve? Process improvement deals with goal satisfaction.

Questions
Concerned with areas of uncertainty related to goals. You need process knowledge to derive questions.

Metrics
Measurements collected to answer questions

SEI Process Maturity Model

Level 1 - Initial
essentially uncontrolled

Level 2 - Repeatable
project management procedures defined and used

Level 3 - Defined
process management strategies defined and used

Level 4 - Managed
quality management strategies defined and used

Level 5 - Optimizing
process improvement strategies defined and used

SEI Process Model Problems

Focuses on project management rather than project development Ignores the use of strategies like rapid prototyping Model is intended to represent organizational capability and not practices used on particular projects There may be wide variation in the practices used in a single organization Capability assessment is questionnaire-based

Capability Assessment Process

Select projects for assessment Distribute questionnair es Analyse responses Clarify responses

Identify issues for discussion

Interview project managers

Interview engineers

Interview mana gers

Brief managers and engineers

Present assessment

Write report

Process Classification
Informal
No detailed process model, developers created their own way of doing things

Managed
defined model drive development process

Methodical
processes supported by standard development method

Supported
processes supported by automated CASE tools

Process Tool Support

Informal process Managed process Methodical process Improving process

Generic tools

Configuration management tools

Project management tools

Analysis and design workbenches

Specialized tools

Defect Removal Effectiveness

Defect removal is central to software development One of the top expense items Affects project scheduling Improves product quality

PSP - Defect Density

This is the primary defect measure used in PSP
Dd = 1000 * D/N D = total number of defects found in all phases of the process N = number of new and changed lines of code in the program

Defect Density Example

For a program with 96 new or changed lines of code and 14 defects
Dd = 1000 * (14/96) = 145.83 defects/KLOC

Defect Metrics - part 1

Error Detection Efficiency
100%*(#errors found in 1 inspection)/(#errors in product before inspection)

Defect Removal Efficiency

100%*(#defects found now)/(#defects found now + #defects found later)

Error Detection Percentage

100%*(#inspection errors)/(#inspection errors + #valid discrepancy reports)

Defect Metrics - part 2

Total Defect Containment Effectiveness (TDCE)
(#prerelease defects)/(#prerelease defects + #post-release defects)

Phase Containment Effectiveness (PCE)

(#phase(i) defects)/(#phase(i) defects + #phase(i+x) defects)

Effectiveness (E)
100%*N/(N + S) N = #defects found by an activity S = #defects found in subsequent activities

Phase-based Defect Removal Model

Defects present at exit of each development phase are estimated This allows us to set realistic targets and assess the costs of reducing error injection rates This is a quality management tool and not a device for estimation of software reliability How would this work in practice?

Assumptions
Suppose we decide to create two broad defect removal classes
activities that handle defects before code is integrated into the system library (design reviews, inspections, unit testing) formal machine tests after code integration

Also assume the same defect removal effectiveness for each phase

Example - part 1
MP = major problems found in before integration PTR = errors found during formal machine tests mu = MP/PTR the higher the value of mu the better Q = defects found after release to customer TD = (MP + PTR + Q) total defects for life of software

Example - part 2
Phase 1 effectiveness E1 = MP/TD MP = E1 * TD Phase 2 effectiveness E2 = PTR/(TD - MP) PTR = E2 * (TD - MP)

Example - part 3
Some equations that can be useful in quality planning (assuming that E1 = E2)
Q = PTR /(mu - 1) Q = MP / [mu * (mu - 1)] Q = TD / (mu * mu)

These equations work with either raw or normalized defect values

PSP Phase Yield

Phase yield = 100 * (defects removed during phase)/ (defects in product at phase entry)
Note: cannot be computed until project is completed

Phase Yield - Example

5 defects found during code review 3 defects found during compile 2 defects found during unit testing 2 defects found during integration testing

Phase yield for compile =

100 * 3 / (3 + 2 + 2) = 42.9 %

Phase yield for code review =

100 * 5 /(5 + 3 + 2 + 2) = 41.7 %

Seven Basic Software Quality Tools

Checklists (paper forms)
used to gather data for later analysis used to confirm that process tasks are complete both simple yes/no and branching questions

Seven Basic Software Quality Tools

Pareto Diagram
bar chart sorted in descending height order vertical axis labeled with # defects horizontal axis (nominal) labeled with defect cause types software defects tends cluster near related causes

Seven Basic Software Quality Tools

Histogram
frequency bar graph vertical axis is # defects horizontal axis has ordinal or interval type labels

Seven Basic Software Quality Tools

Flowchart
pictorial representation of a process breaks down process into its constituent steps can be useful in identifying were errors are likely to be found in the system

Seven Basic Software Quality Tools

Scatter diagram (point plots)
used with correlation, regression, or statistical modeling vertical axis is # defects horizontal axis some metric (e.g. McCabes index)

Seven Basic Software Quality Tools

Run chart
line graph showing performance of dependent variable (y) over time (x) best used for trend analysis (e.g. arrival of defects during formal machine testing) can plot cumulative dependent variables (S curves)

Seven Basic Software Quality Tools

Control chart
advanced form of run chart where capability is defined upper and lower control limits (dashed lines) are drawn to alert the user when dependent measure is out of control can plot cumulative dependent variables (S curves) C chart based on # conforming or not R chart based on subgroup ranges (max min) X bar chart based on subgroup means

Control Chart (C)

Seven Basic Software Quality Tools

Cause and effect (fish bone) diagram
not widely used in software development, but can be useful shows effect between quality variable and the factors affecting it

Fishbone Diagram