Authors
Daniel Kulesz & Sebastian Zitzelsberger
Abstract
Spreadsheets are software programs which are typically created by end-users and often used for business-critical tasks. Many studies indicate that errors in spreadsheets are very common. Thus, a number of vendors offer auditing tools which promise to detect errors by checking spreadsheets against so-called Best Practices such as "Don't put constants in fomulae".
Unfortunately, it is largely unknown which Best Practices have which actual effects on which spreadsheet quality aspects in which settings.
We have conducted a controlled experiment with 42 subjects to investigate the question whether observance of three commonly suggested Best Practices is correlated with desired positive effects regarding correctness and maintainability: "Do not put constants in formulae", "keep formula complexity low" and "refer to the left and above".
The experiment was carried out in two phases which covered the creation of new and the modification of existing spreadsheets. It was evaluated using a novel construction kit for spreadsheet auditing tools called Spreadsheet Inspection Framework.
The experiment produced a small sample of directly comparable spreadsheets which all try to solve the same task. Our analysis of the obtained spreadsheets indicates that the correctness of "bottom-line" results is not affected by the observance of the three Best Practices. However, initially correct spreadsheets with high observance of these Best Practices tend to be the ones whose later modifications yield the most correct results.
Sample
Observations about the experiment:
- Phase 1 (development): 71% of spreadsheets were complete.
- Phase 2 (modification): 79% complete.
- Phase 1: 12% of sheets contained data entry errors. Phase 2: 17%. Cell error rate (CER) was 1% to 1.6%.
- Defect rates of modified spreadsheets are strongly influenced by the defect rates of their input sheets.
- Subjects claimed to have advanced spreadsheet skills, but all of the solutions for input sheet 1 failed in all test scenarios.
- Phase 1: Only 33% of spreadsheets produced a correct result. Phase 2: 21% were correct in all three evaluation scenarios.
- Phase 1: Violation of best practice was common, but not related to correctness.
- Phase 2: Use of best practice leads to higher chance of maintenance success.
Publication
2012, EuSpRIG
Full article
Investigating effects of common spreadsheet design practices on correctness and maintainability
