Toward Building Self-Sustaining Groups in PCR-based Tasks through Implicit Coordination: The Case of Heuristic Evaluation

Usability flaws found in the later stages of the software development process can be extremely costly to resolve. Accordingly, usability evaluation (UE) is an important, albeit usually expensive, part of development. We report on how the inexpensive UE method of heuristic evaluation (HE) can benefit from collaborative software (CSW), implicit coordination, and principles from collaboration engineering. In our study, 439 novice participants were trained in HE methods and then performed HE. Our results show that traditional nominal HE groups can experience implicit coordination through the collaborative software features of group memory and group awareness. One of the key results is that CSW groups had less duplication of effort than traditional nominal groups; these differences were magnified as group size increased from three to six members. Furthermore, because they coordinated less, traditional nominal groups performed more work in the overall process of HE. We attribute the reduction in duplication for CSW-supported groups to the implicit coordination available to them; CSW-supported groups could see violations input by other group members, but could not directly discuss the violations. These findings not only show the power of implicit coordination in groups, but should dramatically change how HE is conducted. These results may also extend to other evaluation tasks, such as software inspection and usability assessment tasks.