Some methods to gather qualitative data are:

• Interviews
• Focus Groups
• Diary studies
• Surveys

These methods should all sound familiar to you, as they were discussed in iteration 1 and in the Analysis section. Feel free to revisit that content if you want to!

Tips on how to make the most out of your qualitative evaluation

1. Ask the right questions

Make sure you ask all the most crucial questions or get all the important answers. Also, in case you are conducting an interview, you may have a list of other less relevant questions to ask if there is time left in an interview.

Keep the questions open-ended and bias-free. You don't want to push the participant in any direction, you want an honest evaluation of your idea. If you feel that it's hard purposefully phrasing questions that way, write them down in full sentences beforehand.



2. Take notes efficiently

Take note of any direct mention of keywords that are relevant to the prototype or discussions you had in the team.

It's always good to have some direct quotes for the evaluation report. Highlight them, e.g. with “...”, to make sure you and your team members recognize them as direct quotes when summarizing and writing the report.

Write down anything surprising that does not meet expectation. Don't only think of comments, try to also observe the emotional and non-verbal reaction to certain parts of your prototype.



3. Make sure you don't miss anything important.

Don’t be afraid to ask a participant to clarify a statement, if the evaluation takes place with e.g. an interviewer. Additional questions can make an otherwise unusable statement much clearer. Also, don't be afraid to simply ask "Why?".

Remind yourself that understanding the participant's view on your prototype is what matters most! If the participants asks you directly about a feature of your prototype, take the opportunity to not answer straight away but to answer with another question "Would you appreciate to have that feature?"/"What do you expect to happen if you were to press that button?". Afterwards, you can of course give the participant a clear answer about how you envisioned the prototype to work.

It can be quite helpful to record participant’s voice and/or behavior. That way, you can for example focus on just asking questions and observing non-verbal cues and write down the participant's comments later with the help of the recording. Alternatively, you can also just record in case your notes are not enough to understand an issue and you and your team want to revisit the context of a statement.

Evaluating your prototype

In this iteration, conduct a qualitative evaluation of your prototype (either the web app or the visual prototype, you decide which to pick!) with 2-5 participants based on your user scenarios and test cases. Define criteria for your evaluation. Document/record the test in a suitable way (photos, log, etc.). For inspiration, read Chapter 7 ("Usability Assessment Methods beyond Testing") in Nielsen (1993), or Chapter 4 ("Evaluating Interface Designs") in Shneiderman and Plaisant (2005).

Write/draw a very brief evaluation report on 1 single page, summarizing the documented evaluation. Describe your evaluation approach in a schematic manner: include your test cases, participant pool, identified problems & suggestions, etc. as well as your interpretations.