Authors
Justin Mow Wah Goo
Abstract
One of the problems documented in behavioral research and cognitive sciences is overconfidence in users. Overconfidence is when the prediction of the degree of success exceeds the actual degree of success achieved by the user. Some studies suggest that systematic feedback would decrease users' overconfidence.
This study was designed to determine if direct feedback on the spreadsheet developer's previous performance would have an effect on their subsequent performance. The hypothesis is that such feedback would increase both their performance and their confidence calibration.
In this dissertation, an experiment has been conducted, utilizing performance feedback to study overconfidence and performance changes in spreadsheets. In the first two weeks of November 2001, 193 subjects were given three spreadsheet tasks and questionnaires that measured their confidence levels before and after each task. An experiment group was given feedback on their performance after each task, while the control group was given no such treatment.
At the conclusion of this experiment, the results indicate that there was not enough evidence to statistically accept or reject the main null hypotheses. However, while the experiment did not provide the anticipated results, it was not without relevant findings.
It was found that:
- Statistically insignificant changes did occur in the predicted direction.
- Subpopulations respond differently to treatment and some do respond as hypothesized.
- While certain stereotypical gender based differences are dispelled, others are supported.
- Confidence calibration in the spreadsheet developmental subject domain may be affected with appropriate stimuli.
The results indicate that feedback of this form is not sufficiently effective to improve overall spreadsheet development. Because breakout groups reacted differently to the treatment, different training and development techniques may be warranted for different cross sections of the population. The few noteworthy observations suggest avenues of further research.
Sample
In several instances the subjects reacted as the hypotheses predicted but not at a statistically significant level.
Though only a small number of results were statistically significant, many positive results encourage belief in the hypotheses.
Publication
2002, Ph.D thesis, University of Hawaii, August
Full article
Not available
