Previous | Table of Contents | Next |
6.1. USABILITY TESTS
The activity of testing and evaluating the interface is intimately connected to the prototype design. By testing the prototype, the design group will find out how well it meets the requirements. Also, the users are given the opportunity to give their opinions about the proposed design. Representatives from the various user categories participate in the testing by being invited to carry out a number of tasks or test cases. using the prototype.
The design group records how well users carry out their tasks and compare the result to the usability requirements for the system. All problems that the users experience during the tests are addressed in the subsequent prototype, which, in its turn, is tested and revised. This process continues until the usability requirements are met or until a specified number of iterations have been performed. The result of the test and evaluation activity are recorded for each user.
When we performed usability tests of the paper prototype for the case study, we ran short of prepared scenarios! This resulted from a combination of factors. The paper prototype did not implement the entire user interface, and we tried to steer the users away from those parts that were incomplete. The users also managed to solve the tasks too quickly. When the test cases were created, we were concerned that they would be too difficult and complex. As a result, we oversimplified them. Finally, we had too little time available for development of the tests. It is difficult and time consuming to design effective test tasks, and when the users arrive to participate in testing, it is necessary to use what is available, even if it isnt optimal. To gather additional usability information, we often used the debriefing session following the usability test for a more comprehensive interview. During these interviews, the users were encouraged to talk freely both about the paper prototype and the tasks that were not yetsupported by the tool.
6.2. COMPUTER PROTOTYPE
A computer simulation or on-line prototype was developed using a user interface management system. The work with the computer prototype began before all parts of the user interface had been designed on paper. The two prototypes were developed in parallel, moving the stable parts of the paper prototype to the computer version. The computer prototype eventually caught up with the paper prototype and it was then abandoned. Figure 7.7 contains a computer prototype showing the main window of the application.
Additional problems with this version of the layout were discovered during the tests. The users liked the hierarchical organization of the test cases, but they had difficulty handling large numbers of test cases. The concept of Test Suites, consisting of several related test cases, is currently being evaluated.
The usability tests of the computer prototype have been very successful, and the user interface has been viewed favorably by the users, the customers, and the responsible management personnel within Ericsson. Attention to usability issues increased dramatically following a series of demonstrations of the prototype. Using the computer prototype as visual proof of our work increased the understanding of what we attempt to accomplish, or as a colleague outside the project expressed it, Now I understand the purpose of the questionnaires, interviews and graphs, but why didnt you start out with designing the prototype?
7. CONCLUSIONS
The way of working that we have described does not eliminate the magic or creativity needed to bridge the design gap, but usability engineering enables us to structure the work and make better use of the skills in the design team. We believe that one part of the magic is the learning process that the individual designer and the usability team take part in during analysis and design. The joint discovery of the user information and the subsequent discussions within the team enable them to manage more complex information and to develop solutions that encompass more aspects of the work tasks.
One of the initial restrictions of the study was that the scope was limited to improving the user interface of the tool. There was very little understanding of the need for an open-ended usability study. This turned out to be a smaller problem than one might expect. During the early phases of the usability work (user and task analysis), this constraint did not become an issue. During the conceptual design and prototyping we had gained enough credibility to be able to suggest changes that would affect the underpinnings of the system. Having one of the system architects in the usability group allowed us to suggest solutions that were inexpensive and easy to implement, but which improved the usability of the system considerably. Still, the unwillingness of project managers to allow us to design from scratch affected what services we chose to support and how we wanted to implement them in the prototype.
We believe that the need for a general usability questionnaire will diminish or change over time. As the Delta Method is used as an integrated part of system development, we will be able to build a repository of user information that can be reused in later studies. This will allow us to focus on other issues such as internationalization and localization of the user interface.
Previous | Table of Contents | Next |