Authors
Marc Fisher, Mingming Cao, Gregg Rothermel, Curtis Cook, & Margaret Burnett
Abstract
Spreadsheet languages, which include commercial spreadsheets and various research systems, have had a substantial impact on end-user computing. Research shows, however, that spreadsheets often contain faults.
Thus, in previous work, we presented a methodology that assists spreadsheet users in testing their spreadsheet formulas. Our empirical studies have shown that this methodology can help end-users test spreadsheets more adequately and efficiently; however, the process of generating test cases can still represent a significant impediment.
To address this problem, we have been investigating how to automate test case generation for spreadsheets in ways that support incremental testing and provide immediate visual feedback. We have utilized two techniques for generating test cases, one involving random selection and one involving a goal-oriented approach. We describe these techniques, and report results of an experiment examining their relative costs and benefits.
Sample
We consider the ability of our techniques to generate test cases to cover all the feasible du-associations in the subject spreadsheets (we refer to this as their ultimate effectiveness).
This table lists, for each of the subject spreadsheets, the ultimate effectiveness of Random and Chaining with and without range information, averaged across 35 runs:
- Chaining without range information achieved over 99% ultimate effectiveness on all but two of the spreadsheets.
- Random without range information behaved much differently than Chaining. In only one case did Random without range information achieve an ultimate effectiveness greater than 90% (Budget), and in six of ten cases it achieved an ultimate effectiveness less than 60%.
- In contrast to the results observed for Chaining, addition of range information to Random did affect its performance, in all but one case increasing ultimate effectiveness, and in seven of ten cases increasing it by more than 20%.
- Addition of range information to Random also helped its performance in comparison to Chaining. However, this difference, though statistically signifcant, was less than 1%.
Publication
2002, 24th International Conference on Software Engineering, May, pages 141-153
