Steve Portigal

Usability testing: Typically done in a controlled environment such as a lab, users interact with a product (or a simulation of a product) and various factors (time to complete a task, error rate, preference for alternate solutions) are measured.

• A/B testing: Comparing the effectiveness of two different versions of the same design (e.g., advertisement, website landing page) by launching them both under similar circumstances.

• Quantitative survey: A questionnaire, primarily using closedended questions, distributed to a larger sample

to obtain statistically significant results.

• Web analytics: Measurement and analysis of various data points obtained from Web servers, tracking cookies, and so on. Aggregated over a large number of users, Web analytics can highlight patterns in navigation, user types, the impact of day and time on usage, and so on.

• Focus group: A moderated discussion with 4 to 12 participants in a research facility, often used to explore preferences (and the reasons for those preferences) among different solutions.

• Central location test: In a market research facility, groups of 15 to 50 people watch a demo and complete a survey to measure their grasp of the concept, the appeal of various features, the desirability of the product, and so on.

• Ask for a demonstration of an activity that might not otherwise take place.

• Observe a behavior or a task as it happens to occur naturally.

• Use a mapping exercise to create a tangible representation of something abstract that you can refer to repeatedly throughout the interview (and then take away with you at the end).

• Show provocative concepts at varying levels of fidelity and create concepts that will generate discussion around the issues at hand (rather than testing your best guess at the best solution).

• Use images as stimuli to prompt a deeper discussion. When mounted on cards, they can be sorted, grouped, annotated, referred to later, and so on.

• Assign homework (for example, take a few pictures, save some artifacts, complete a questionnaire, and document a set of activities) to give you some data before the interview and to prime the participant about the interview topics

Generally speaking, I find the ideal size for the field team to be two people: one to lead the interview, and one to back up the other person.

You can provide him with sticky notes to write his questions on as he thinks of them (so even if the asking is deferred, at least capturing the question provides some—albeit muted—immediate gratification).

Ask about sequence. “Describe a typical workday. What do you do when you first sit down at your station? What do you do next?”
• Ask about quantity. “How many files would you delete when that happens?”
• Ask for specific examples. “What was the last movie you streamed?” Compare that question to “What movies do you stream?” The specific is easier to answer than the general and becomes a platform for follow-up questions.
• Ask about exceptions. “Can you tell me about a time when a customer had a problem with an order?”
• Ask for the complete list. “What are all the different apps you have installed on your smartphone?” This will require a series of follow-up questions—for example, “What else?” Very few people can generate an entire list of something without some prompting.

• Ask about relationships. “How do you work with new vendors?” This general question is especially appropriate when you don’t even know enough to ask a specific question (such as in comparison to the earlier example about streaming movies). Better to start general than to be presumptive with a too-specific question.
• Ask about organizational structure. “Who do the people in that department report to?”
Questions that probe what’s been unsaid:
• Ask for clarification. “When you refer to ‘that,’ you are talking about the newest server, right?”
• Ask about code words/native language. “Why do you call it the bat cave?”
• Ask about emotional cues. “Why do you laugh when you mention Best Buy?”
• Ask why. “I’ve tried to get my boss to adopt this format, but she just won’t do it....” “Why do you think she hasn’t?”
• Probe delicately. “You mentioned a difficult situation that changed your usage. Can you tell me what that situation was?”
• Probe without presuming. “Some people have very negative feelings about the current government, while others don’t. What is your take?” Rather than the direct “What do you