This blog post was originally published on the Brandwatch Engineering Blog
As you have probably heard, user involvement is vital for product- and service development. There are myriad books, articles and courses about the importance of user research, user participation, usability testing and so forth, and all of them have their raison d’être and scope.
It’s a long way to Utopia
But one of the fundamental issues you’ll encounter when you want to establish user involvement as an ongoing part of your development process is often overlooked – that is candidate recruitment. Academic writings often assume a somewhat utopian state, where potential candidates are simply available. It is assumed that there is a bulging candidate database, where you can cherry-pick the most fitting ones for your project. And of course in the realm of research teaching this has to be a given.
But there is a lot of work to do before you reach this luxury state. Until you have integrated candidate recruitment as an inherent part of your processes and the database fills itself like a well-oiled machine, there will be a time where you only have a handful of candidates who are willing to take part in user research. Within Brandwatch we also face the issue of a smaller user base in general, since our product serves a special kind of use case and it’s not accessible publicly, compared to e.g. Spotify. This scarcity led to an approach where I tried to get the most insights out of our limited user resources and combined usability tests with user interviews.
Necessity begets ingenuity
Usability testing is one of the core methods we use at Brandwatch to involve users in our development process. We test prototypes or early stages of the product to get user insights, thoughts and concerns about our approach. As already briefly described in my previous blog post, a usability test uses a basic structure to guide as a common thread through the test. This structure is written down in the usability test script. The script contains the assumptions about the feature we want to test, tasks to validate or falsify these assumptions and further questions to give a better understanding of why an assumption was right or wrong. You don’t have to follow this script rigidly, but in general you will follow the designated path, since the tasks might be consecutive.
User interviews, however, are a separate method which can be used to get user insight. Interviews are useful in all phases of the development process. One of the use cases is to get detailed insights about the current state of something, i.e. a process. The direct communication enables the researcher to collect complex and detailed data, since you have the ability to address ambiguities instantly, which is an advantage over other survey routes, e.g. questionnaires.
Preparation is key
What I did next was merge user interview questions into the setting of a usability test to use one test session for two purposes. This sounds fairly easy and obvious, but there are a couple of things you need to consider to get the most out of the test session.
Without going into too much detail, it’s fair to say that a lot of the current work my team is doing belongs to a bigger theme. There is a broader vision that has been chunked into smaller projects but are highly interrelated and build up on each other. This factor put me in the convenient position to know upfront, that the next project would be of concern for the same users as the current one. This thematic overlap is one of the requirements that needs to be in place to merge the two user research methods.
The next thing you need to think about is when is the right time to ask your interview questions. Obviously you could just append them at the end of your usability test, but this would shoot down one of the advantages of baking interview questions into usability tests, that is context. You want to leverage the primed mindset of the user. Therefore it’s better to lead into the interview questions within a natural conversation. And since you are the one who is writing the test script, you have all the power to put things on the right track.
I’ll give an example here: Let’s say you are going to test a new feature that will improve the onboarding process of a user. And you also want to get user insights about how their process of winning new users can actually be improved. For that you have prepared the interview questions. You now can direct the conversation towards the topic of new users, since it is part of your test setting. And this is the perfect spot to switch from ‘testing’ to ‘interviewing’ mode.
Smooth shift
After the task is completed, you follow up with questions about the general impression of the new feature, it’s use case and so forth. Thus an obvious follow up question to ask, while you are already talking about new users with your candidate, is how they are actually getting new users for their service? The question appears in the right context and is a natural part of the conversation. It doesn’t force your candidate to adjust their current thinking.
You shifted smoothly from your usability test script to the interview part. The transition was subtle and natural. You can now ask your questions, or adapt to the actual conversation and get first-hand user insights about the topic. The questions won’t feel inappropriate or odd since they are semantically related to the topic.
Certainly, you should not forget about the still ongoing usability test and overstrain this interview part. After you have asked your questions or when you feel that the conversation is leading towards an end, you should shift back to the actual test. Since you are still in the same broader context, the shift will be easy. You can then just continue with the actual usability test.
With this interview blended in within a usability test, you are able to kill two birds with one stone. You can get user insights about two different (but related) topics within just one test session. This will save you and your user’s time, which is one of the most valuable resource within usability testing, and it helps you to keep your sanity when you have to cope with a limited candidate database.