By Dr. Markus Funk, Senior UX Project Manager
In my 10 years as a UX researcher, one of my favorite elements has always been working with people. Inviting them to the lab, showing them new technology, and observing how they use and react to exciting new experiences.
Over the last year, with global lockdowns and uncertain times due to the COVID-19 pandemic, the (what I would now consider) luxury of working with people directly in-person has no longer been possible. This has led to a global shift in the UX community, as research moved from in-person studies to online studies [1].
While there are certainly some positive aspects to the move online – quick turnaround times from study to result, access to millions of users online, etc. – there are some downsides, the most significant of which is the immersion. When it comes to creating the feeling of being in a car, immersing a user in a computer-based simulation with keyboard control of a vehicle might not create the same atmosphere as actually driving a car. Especially with highly specialized hardware setups, it is next to impossible to get accurate results remotely. While for some scenarios, it is okay to send hardware to users[2], for highly complex setups, it just isn’t feasible.
In addition, we lose one of the most valuable aspects of in-person research: the ability to establish a personal connection with the participants to fully understand why an experience was perceived as “good” or “bad.” Sometimes getting to the bottom of why a technology is perceived in a certain way is key to improving products and experiences. Without this personal connection and understanding of the user, a promising technology might be discarded, as the reason for not fully accepting or embracing the technology was not easily understood in an online interview format.
Therefore, in-person studies are still crucial to developing and improving our products. But how can we conduct them during a global pandemic? Our research approach has always had a balance between realism, cost, and feasibility. In these challenging times, we need to add another factor to the mix: keeping our participants – and facilitators – safe and healthy.
Take a recent study at our Ulm, Germany, DRIVE Lab about our Just Talk technology as an example. Since speech input can be challenging to get right while wearing a mask, we needed to create a safe environment for participants to interact with the technology while being immersed in a driving simulator. From our experience, we developed these best practices for conducting in-person studies while keeping everyone involved safe:
1. “Can we do this remotely?”
Even before the pandemic, we have all had our moments of “this meeting could have been an e-mail.” The equivalent in today’s world of user research should be the first question asked when considering conducting a user study: “Does this have to be in-person, or could this study be done online?” If it is opinions or general feedback that we are looking for, an online study is likely the better choice. When there is special hardware or technology or experience involved that cannot be used remotely, we should consider an in-person study.
2. Allow an in-person study to take more time.
It is important to factor in more time for user studies these days. In pre-pandemic days, running a study with approximately 30 participants would take three or four days (8-10 participants per day). In today’s world, we found that two participants per day is a good number that allows for adequate time for the pre- and post-study cleaning and sanitation tasks – meaning that same study will take three weeks.
3. Design for not being in the room.
This seems obvious, but it is important compared to how we designed studies before: the facilitator cannot be in the room at any time! Study participants should be in their own room, with the facilitator in another room to observe from a distance. This can be done through a video connection or, in our case, through a one-way mirror. In any case, a voice connection between the facilitator’s room and the study room is critical for communication. We also need to be prepared for technical issues: if a prototype fails and you need to restart something, plan to be able to do that remotely. If something brakes and one needs to physically press a button, kindly ask the participant to push it. We have learned that participants are fully understanding of the situation and will be more than happy to help so as to not put anybody at risk.
4. Continuously test and stay up to date with legal regulations
As we adapt to our new normal, we must stay apprised of current safety measures and adapt our protocols accordingly. For example, recently in Germany, rapid COVID-19 tests were made available to the general public. As such, future in-person studies will require a negative rapid test result for both participants and facilitators. In addition, it’s important keep up with current legal regulations as they can change frequently.
While the pandemic has definitely not made our job of doing user research easier, if we are cautious, careful, and reasonable in planning and conducting our research, we are still able to do what we love: improving the UX and usability of our products directly and in-person with our users.
References:
[1] UserTesting.com CX Industry Report - https://info.usertesting.com/2021CXIndustryReport.html
[2] Boll, S., Ihmels, T., Lunte, T., & Muller, H. (2020). Making, Together, Alone: Experiences from Teaching a Hardware-Oriented Course Remotely. IEEE Annals of the History of Computing, 19(04), 35-41.