Understanding the Sources of Variation in Software Inspections

Adam Porter, Harvey Siy, Audris Mockus, Lawrence Votta

Research output: Contribution to journalArticle

80 Citations (Scopus)

Abstract

In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other factors must be affecting inspection performance. The nature and extent of these other factors now have to be determined to ensure that they had not biased our earlier results. Also, identifying these other factors might suggest additional ways to improve the efficiency of inspections. Acting on the hypothesis that the "inputs" into the inspection process (reviewers, authors, and code units) were significant sources of variation, we modeled their effects on inspection performance. We found that they were responsible for much more variation in defect detection than was process structure. This leads us to conclude that better defect detection techniques, not better process structures, are the key to improving inspection effectiveness. The combined effects of process inputs and process structure on the inspection interval accounted for only a small percentage of the variance in inspection interval. Therefore, there must be other factors which need to be identified.

Original languageEnglish (US)
Pages (from-to)41-79
Number of pages39
JournalACM Transactions on Software Engineering and Methodology
Volume7
Issue number1
DOIs
StatePublished - Jan 1998

Fingerprint

Inspection
Defect detection
Experiments

Keywords

  • Empirical studies
  • Experimentation
  • Software inspection
  • Software process
  • Statistical models
  • Testing and Debugging - code inspections and walk-throughs

ASJC Scopus subject areas

  • Software

Cite this

Understanding the Sources of Variation in Software Inspections. / Porter, Adam; Siy, Harvey; Mockus, Audris; Votta, Lawrence.

In: ACM Transactions on Software Engineering and Methodology, Vol. 7, No. 1, 01.1998, p. 41-79.

Research output: Contribution to journalArticle

Porter, Adam ; Siy, Harvey ; Mockus, Audris ; Votta, Lawrence. / Understanding the Sources of Variation in Software Inspections. In: ACM Transactions on Software Engineering and Methodology. 1998 ; Vol. 7, No. 1. pp. 41-79.
@article{e6dd6421c352490eb7bff5d3b3792cb8,
title = "Understanding the Sources of Variation in Software Inspections",
abstract = "In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other factors must be affecting inspection performance. The nature and extent of these other factors now have to be determined to ensure that they had not biased our earlier results. Also, identifying these other factors might suggest additional ways to improve the efficiency of inspections. Acting on the hypothesis that the {"}inputs{"} into the inspection process (reviewers, authors, and code units) were significant sources of variation, we modeled their effects on inspection performance. We found that they were responsible for much more variation in defect detection than was process structure. This leads us to conclude that better defect detection techniques, not better process structures, are the key to improving inspection effectiveness. The combined effects of process inputs and process structure on the inspection interval accounted for only a small percentage of the variance in inspection interval. Therefore, there must be other factors which need to be identified.",
keywords = "Empirical studies, Experimentation, Software inspection, Software process, Statistical models, Testing and Debugging - code inspections and walk-throughs",
author = "Adam Porter and Harvey Siy and Audris Mockus and Lawrence Votta",
year = "1998",
month = "1",
doi = "10.1145/268411.268421",
language = "English (US)",
volume = "7",
pages = "41--79",
journal = "ACM Transactions on Software Engineering and Methodology",
issn = "1049-331X",
publisher = "Association for Computing Machinery (ACM)",
number = "1",

}

TY - JOUR

T1 - Understanding the Sources of Variation in Software Inspections

AU - Porter, Adam

AU - Siy, Harvey

AU - Mockus, Audris

AU - Votta, Lawrence

PY - 1998/1

Y1 - 1998/1

N2 - In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other factors must be affecting inspection performance. The nature and extent of these other factors now have to be determined to ensure that they had not biased our earlier results. Also, identifying these other factors might suggest additional ways to improve the efficiency of inspections. Acting on the hypothesis that the "inputs" into the inspection process (reviewers, authors, and code units) were significant sources of variation, we modeled their effects on inspection performance. We found that they were responsible for much more variation in defect detection than was process structure. This leads us to conclude that better defect detection techniques, not better process structures, are the key to improving inspection effectiveness. The combined effects of process inputs and process structure on the inspection interval accounted for only a small percentage of the variance in inspection interval. Therefore, there must be other factors which need to be identified.

AB - In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other factors must be affecting inspection performance. The nature and extent of these other factors now have to be determined to ensure that they had not biased our earlier results. Also, identifying these other factors might suggest additional ways to improve the efficiency of inspections. Acting on the hypothesis that the "inputs" into the inspection process (reviewers, authors, and code units) were significant sources of variation, we modeled their effects on inspection performance. We found that they were responsible for much more variation in defect detection than was process structure. This leads us to conclude that better defect detection techniques, not better process structures, are the key to improving inspection effectiveness. The combined effects of process inputs and process structure on the inspection interval accounted for only a small percentage of the variance in inspection interval. Therefore, there must be other factors which need to be identified.

KW - Empirical studies

KW - Experimentation

KW - Software inspection

KW - Software process

KW - Statistical models

KW - Testing and Debugging - code inspections and walk-throughs

UR - http://www.scopus.com/inward/record.url?scp=0031599398&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031599398&partnerID=8YFLogxK

U2 - 10.1145/268411.268421

DO - 10.1145/268411.268421

M3 - Article

AN - SCOPUS:0031599398

VL - 7

SP - 41

EP - 79

JO - ACM Transactions on Software Engineering and Methodology

JF - ACM Transactions on Software Engineering and Methodology

SN - 1049-331X

IS - 1

ER -