Software Quality Assurance Activities

What is "Quality"?

"Quality of Design" and "Quality of Conformance"

-- Roger Pressman

User Satisfaction =
compliant product + good quality + delivery within budget and schedule
-- Robert Glass

Standard Definition
Software Quality - Conformance to:
explicitly stated functional and performance requirements, 2. explicitly documented development standards, and 3. implicit characteristics that are expected of all professionally developed software.
1.

SQA Activities
1.
2. 3. 4. 5.

6.

Pre-Project Activities Development and Maintenance Activities Error Reduction Infrastructure SQ Management Components SQA System Assessment Human Components

Software Quality Assurance by Galin

SQA Activities
1.

Pre-Project Activities
1. Contract Review 2. Development Plan 3. Quality Plan

2. 3. 4. 5. 6.

Development and Maintenance Activities Error Reduction Infrastructure SQ Management Components SQA System Assessment Human Components

Contract Review Process

RFP
SOW

1st Draft

Revisions

Final Contract

Reality Check...
Q: Why should the software geeks worry about the contract?
A: Because the software team must do the work and assure the product's quality.

loosely defined requirements unrealistic budgets unrealistic schedules

A: Contract review is required by ISO 9001

What to look for in 1st Draft

customer reqs clarified and documented?

alternative approaches examined?

risks identified? costs and time estimates reasonable? both customer and creator have capacity? subcontractor participation clear?

proprietary rights?
relationship between customer and creator specified? Galin section 5.3

Components of the Quality Plan

Quality Goals

hopefully the SRS is some help

Review Activities

schedule, type of reviews, scope, responsible persons

type of unit tests and coverage, integration plan

Software Test Plan

Acceptance Tests for Sub-contract Software Configuration Tools and Procedures

Galin section 6.3

SQA Activities
1.

Pre-Project Activities

2.
1. 2. 3.
3. 4. 5. 6.

Development and Maintenance Activities

Formal Technical Reviews Code Inspections Unit Testing, Integration Test, etc. Error Reduction Infrastructure SQ Management Components SQA System Assessment Human Components

Design Review

Types of Evaluations

Formal Reviews
conducted by senior personnel or outside experts uncover potential problems

Inspections and Walkthroughs

done by peers detect problems, adherence to standards, etc.

Verification
Unit Test Integration Test Usability Test

Formal Reviews

Reviewers should be senior personnel and/or outside experts Outcome:

approve approve pending changes reject

Review Leader should not be Project Leader Usually done at the end of the phase.
very appropriate for SRS and Design sometimes appropriate for code

Sample Design Review Checklist

Well-structured
Simple Efficient Adequate Flexible Practical Implementable

Peer Reviews

guided by:
checklists, standards, past problems

attendees:

review leader the author scribe folks with domain knowledge possibly an SQA team member (for standards)

Why schedule a meeting with so many people? Why not just have two people review the item without a meeting?

Peer Review Process

pre-meeting

read the document ahead of time

meeting
author presents overview review team asks questions and express opinions

after meeting
scribe prepares summary team approves summary follow up

 1.
2. 3. 4. 5. 6. 7.

General: Does the architecture convey a clear vision of the system that can be used
for further development? Is the architecture structured to support likely changes? Does the architecture describe the system at a high level of detail? (No interface or implementation details.) Does the architecture cleanly decompose the system? Is the architecture independent of the infrastructure used to develop the system? Has maintainability been considered? No duplicate functionality in the architecture?

1. 2. 3. 4.

Complete:
Are software requirements reflected in the software architecture? Is effective modularity achieved? Are modules functionally independent? Does each module/class have an understandable name? Is each association well named? Is each associations and aggregations cardinality correct?

5.

1. 2.

Correct:
Does each association reflect a relationship that exists over the lives of the related modules/classes? Does the architecture have loose coupling and good cohesion?
www.cs.trincoll.edu/~hellis2/CPSC240/Project/Design Review Checklist.doc

Sample Design Walkthrough

1. 2. 3. 4. 5.

6.
7. 8. 9.

10. 11.

Does the algorithm accomplishes desired function? Is the algorithm logically correct? Is the interface consistent with architectural design? Is the logical complexity reasonable? Have error handling and "anti-bugging" been specified? Are local data structures properly defined? Are structured programming constructs used throughout? Is design detail amenable to implementation language? Which are used: operating system or language dependent features? Is compound or inverse logic used? Has maintainability been considered?
stolen from Pressman

Inspection Guidelines

Review the Product, not the person! Find errors, don't try to solve them! Keep Records

Take written notes. Review your earlier reviews. 3 to 5 people Conduct training for reviewers limit debate and rebuttal

Allocate resources and schedule time for FTRs.

Keep it short

Set an agenda and keep it.

small portions only narrow focus increases likelihood of finding an error

no more than two hours preparation

meeting duration less than two hours

SQA Activities
1.
2. 3.

Pre-Project Activities Development and Maintenance Activities Error Reduction Infrastructure

1. 2. 3. 4. 5. 6.

work procedures templates and checklists staff training preventive actions configuration control document control

4. 5.

6.

SQ Management Components SQA System Assessment Human Components

Definitions

procedure - general methodology work instructions - details of how to implement those procedures for this project template - a standard Table of Contents checklist - list of items that are part of an activity

Which Philosophy is correct?

True greatness is measured by how much freedom you give to others, not by how much you can coerce others to do what you want.
Larry Wall Creator of Perl March 1999

Standardization is the key.

paraphrase of ISO 9000-3

Example types of Work Instructions

audit process for subcontractors design documentation templates C++ programming instructions coordination and cooperation with the customer follow-up of beta site reporting monthly progress reporting

Example Templates and Checklists

See Course Web Site for

DoD

DIDs Work Instructions and Guidelines

NASA

http://sw-assurance.gsfc.nasa.gov/disciplines/quality/index.php

NWS

Coding Standards

SQA Activities
1.
2. 3.

Pre-Project Activities Development and Maintenance Activities Error Reduction Infrastructure

4.

QA Management Components

Project Progress

schedules, budgets, risk analysis,

Quality Metrics Quality Costs

5. 6.

SQA System Assessment Human Components

Statistical SQA
1. Info about defects are collected and

categorized. 2. Defects are traced to their root cause. 3. Isolate the vital 20% 4. Move to correct problems that caused defects.
SSQA is similar to "Six Sigma" Pareto Principle: 80% of defects can be traced to 20% of causes.

Statistical SQA

SQA Activities
1.
2. 3. 4.

Pre-Project Activities Development and Maintenance Activities Error Reduction Infrastructure SQ Management Components

5.

SQA System Assessment

Quality Management Standards

SEI CMM ISO 9001

Details o'plenty Next Week

6.

Human Components

SQA Activities
1.
2. 3. 4. 5.

Pre-Project Activities Development and Maintenance Activities Error Reduction Infrastructure SQ Management Components SQA System Assessment

6.

Human Components Management SQA Unit

The SQA Group

Participates in the development of the project's software process description. Reviews software engineering activities to verify compliance with the defined process. Audits designated software work products to verify compliance with those defined as part of the software process. Ensures that deviations in software work and work products are documented and handled according to a documented procedure. Records any noncompliance and reports to senior management. Coordinates and controls change. Helps collect and analyze metrics.

Summary

"Quality" includes explicit and implied properties

as well as the development process. FTRs are a key part of SQA. An SQA Group is highly recommended. QA must be part of the plan and schedule.

SSQA is a commonly used formal approach to quality assurance and process improvement.

Next Time
SEI CMM ISO 9001