Spreadsheet bibliography

Title Hitting the wall: Errors in developing "simple" spreadsheet models
Authors Raymond R. Panko
Year 1996
Type Proceedings
Publication 29th Annual Hawaii International Conference on System Sciences
Series January, pages 356-363

Undergraduate MS business students developed and debugged a spreadsheet model from a word problem. This model consisted of a bid to build a wall. The problem was designed to be relatively simple and domain-free, to address the concern that past spreadsheet experiments may have used problems that were too difficult or that required domain knowledge that subjects did not have.

During the development phase, 72 subjects created the spreadsheet model. Even with this rather simple problem, 38% of the models contained an error.

This high number of incorrect spreadsheets was not due to subjects making many errors. They only made 0.4 errors per spreadsheet. In addition, their cell error rate (CER) was only 1.7%, meaning that only 1.7% of their cells contained errors. Unfortunately, spreadsheets tend to have long cascades of cells leading to the bottom line. This means that even tiny cell error rates will multiply into high rates of bottom-line errors.

In a debugging phase, subjects tried to debug their own models. Of 19 subjects with incorrect models who did the debugging part of the experiment, only three (16%) found and corrected their errors. So even with a relatively simple model, development and debugging were problematic.

This is a lower rate of finding errors than Galletta, et al. [1993, 1996] found when subjects debugged models created by the experimenter. This may mean that people are not as good at debugging their own models as they are at debugging models created by others.

Full version Available
Also see Hitting the wall: Errors in developing and code inspecting a 'simple' spreadsheet model
At a glance

Development phase:

Number of subjects 72
Spreadsheets with errors, number 27
Spreadsheets with errors, percent 38%
Errors per spreadsheet 0.4
Total errors 30
Omission errors 17
Logic errors 13
Mechanical errors 0
CER (cell error rate) overall 1.7%
CER for omission errors 0.9%
CER for logic errors 1.4%

Debugging phase:

Debugged spreadsheets with errors 19
Correctly fixed, number 3
Correctly fixes, percentage 16%