Previous | Table of Contents | Next |
We do not have a list of questions that we ask the user. The team member doing the interview must listen very carefully to what the user says and construct each successive question based on previous input. However, we do learn from previous interviews we conduct. Therefore, if a previous interview has uncovered some surprising information, we will probe in that direction with future users to see if that information is validated. If, during an interview, we uncover some information that we havent heard previously, well probe in depth in that direction.
Facilitators and obstacles are often volunteered by the users. When they describe how they do something, they often include phrases such as we try to do this but its not always easy because..... If facilitators and obstacles are not volunteered, we do ask is that easy to do? or is that difficult to do?.
We use the same two people do conduct all the interviews if possible. This is just to minimize mind sharing between interviewers during analysis. If the time or number of interviews prohibit this, we use several teams and switch around so that all possible pairs are formed to do interviews. Frequent discussions are then encouraged among the interviewers. Ongoing analysis is encouraged, as is frequent viewing of the structured data, while teams are in the interview phase.
In the best situation the interview team is able to analyze one interview before conducting the next. If a complete analysis is not possible, the team has a short discussion after each interview to summarize any new information, to identify any conflicts with previous information, and to note any information that verifies what they have already collected. This produces several possible areas to probe in succeeding interviews. However, this has to be evaluated with respect to the overview information supplied by the next user. If any of these areas appear to be within that users job description, then probing is possible. However, the interviewer needs to determine whether other information supplied by the user is more interesting.
We also switch the roles for the interview team members. We usually try to switch every other interview. We feel that this helps us to maintain better balance. The team member structuring the data gets a better sense of the users tasks than does the team member conducting the interview. The interviewer has to listen for and recognize new information and probe in that direction if anything new is identified.
It is important to note that all our interviews are conducted at the users workplace. In addition to conducting the interview, we do a careful observation of the users office. Often we find items there that we ask about. For example, a user may have a large whiteboard schedule on his wall. We would ask (if the user doesnt mention it) what role this object plays in his daily tasks. Users sometimes open up file cabinets and explain how they store information or take us on short tours to explain how a certain process flows within their organization.
We take audio tape recorders along if companies will permit us to tape our interviews. This allows us to listen to prior interviews to obtain or clarify information we may have missed. We also ask for permission to take a still photo of the people we interview. Taking a photo back to incorporate with the structured data helps to make the information more real to the product team.
The interviewing process is not an easy one. Both members of the team must be skilled interviewers. They need to know how to listen and quickly interpret what the user says and how to probe for information rather than just asking a set of predetermined questions. Structuring the data is also difficult as it needs to be done quickly. We take breaks at times and assess how were doing with the data. The interviewer cannot keep track of all the information and needs feedback from the team member, structuring the data, about points that were not adequately covered.
4.2. ANALYZING THE DATA
It is best to conduct an interview, analyze that data, and incorporate it into the accumulated data, then do the next interview. When we analyze the data collected during an interview, we normalize it. That is, we try to use the same terminology to describe the goals, objectives, and tasks for all our users. As the interview phase continues, we are able to structure the data using normalized terms instead of specific terms. Each interview builds on the one before it. Once we find the same type of goals and supporting tasks in successive interviews, we probe lightly only to ensure that there are no differences. We probe more deeply in areas where we have little or no data.
When do we stop interviewing users? We continually incorporate the data from each interview into what we have accumulated. When we find that we are getting little new data, we discontinue our interviews and go onto the next phase of our work. An easy way to identify the amount of new information added to the framework is to simply color code the information. Only new contributions are added for each interview conducted. When the new contributions appear in a different color, it is easy to see the amount of new information being added. When the new information is relatively small, it is safe to assume that sufficient information has been collected for product definition. The number of users will depend on the homogeneity of the target market.
As we collect the data, we structure it, using an outline form of a word processor. We use the different levels in the outline to represent the different data classes we collect. The first level is the objective. The next level lists goals that support this objective. At the next level, goals are expanded to show the tasks needed to support the goal, followed by subtasks. Facilitators and obstacles are expanded under the tasks. The next level contains actions and objects needed for the task or subtask. At this point, we move the data to spreadsheets. We sort the data in different ways, producing a sheet for each view. We have a sheet sorted by objectives, another sheet sorted by goals, etc. We use a numbering system to tie each data class back to the original objective and goal. We also note where the objective and goals came from human factors input or marketing input. Using this method we can easily ask questions during design. For example, we can see which actions are performed on the same objects, or we can view all the goals that a task supports.
Previous | Table of Contents | Next |