Confessions of a User Researcher
I write this reflective thinking article because I was invited to host a workshop at IXDC. My original plan was to write a very brief intro to wrap up what brought me to land this job in the first place. However, this process actually brings me to look back on my whole career as a user researcher and upon all my confusions, struggles, and of course enjoyment while doing this job.
I am not a “silver spoon” in my career. I did not know the term “user research” prior to my job interview at Baidu, where I started my career after graduating from college. In fact, I define myself as a “scrapper”, who learnt all the necessary skills through real work projects. After the first year, my manager had a one-on-one review with me. Luckily he gave credit to my work, but half joking half seriously he said: “you won’t be doing this work for long.” “Then what will I do?” I kindly responded. I was quite nervous hoping this is not a sign of being fired because I clearly remember he spoke highly of my work just now. “You might be a saleswoman.” He stated. What a surprise! However, I did not ask what I should sell at that point because I have always been a bit skeptical about my job. Can I be a good user researcher? Does this job really matter to a design team?
Why am I being so skeptical? As times goes by, I summarized the seven deadly sins that I need to confess as user researcher. Since this is a reflective thinking article, I will also try to give my solutions on how to resolve these issues.
Lag in Phase
Working as a user researcher, I often encounter the Phase II problem: after rounds and rounds of user research, the features and ideas for the next phase of work/product are gone after they leave the “lab”, and never to be heard from again. Do these ideas disappear because they are flawed? Is it because they don’t really meet user needs and business goals? As far as I see it, for most of the time, it’s because these features simply run out of time. Conventional user research is time consuming and since product teams are using techniques such as agile software development and continuous deployment, they tend to radically reduce cycle time. Even a two-week long user research seems too time consuming because it is highly possible that within this period of time, the tested version has already been replaced by a new one. Therefore, I think one of the solutions to solve this Phase II problem in user research relies on rapid experimentation.
To confront this sort of dilemma, I led a cross-functional team that consist of user research and developed an online system called UBA (User Behavior Analysis), aiming to use rapid experimentation and measurement as a supplement to conduct user research. UBA collects user online behavior data (click, move, scroll and view), computing these data in real time and visualizes them in a high-fidelity, data-rich heat-map. It is a fast way to reveal user behavior needs and intent by using online big data (every page visit will be collected as a sample). In my other article I described how this system works.
One of my worst nightmares at work is being invited to participate in product team parties. The reason is not because I don’t-fit-in or I’m a weirdo but because I don’t really know these guys. As a user researcher, I’m a kind of third-party gal, or to say a “contractor” in this team. I receive a top-down research request from manager/VP level, then assigned to a product team temporarily doing research, hand in my report, and then I am gone. Sometimes, I get lucky and my findings work out. As a result, the products end up doing well and that’s how I get invited to product team parties. However, I always feel like I am a supporter and am never really involved that much in the product development process. I hate to say this but it seems like a “one-night-stand,” I never develop a “relationship” with any of them.
I want to feel like I’m needed sort of like being needed in a long-term relationship sort of way. Instead of being a supporter for product teams, I want to be a coordinator. TakeJD.com as an example. If the shopping experience is an integrated process as a whole, each product team is required to be responsible for their one task. In hinesight it creates a very fragmented structure after connecting the parts but lacks in having a person to take care of the bigger picture. This is where user researchers come into play. They understand the process in each product team and are also experts in taking care of the user-experience. Therefore, I think user researchers should not just focus on product design, but service design as well because the latter is more focused on experience as a whole.
First of all, I want to make it clear that I like Shelton. I use this picture because I think he is like the “Know-it-All” guy and as user researchers, sometimes do label ourselves as the “physicist” and the only make decisions when there are user data or theories backed up. As for product managers, designers, and programers, we label them as the “engineers”, who are practical but short-sighted, as Shelton does. However, the longer I stay as a user researcher, the more I feel those product managers, designers and programers are the guys who really “crack the cases”, and we are the people who just point fingers. The reason we think we seem to be the “Know-it-All” guys is because we live in our own bubble or to say “utopia.”
I like the idea of observing before you make statements or judge others. My philosophy is that you need to not only walk in their shoes, but also walk in them for 10 miles long. Coding might be a little difficult for a user researcher, but with the help of some design tools, such as Axure, Sketch, Canvas, we user researchers can design mock-ups or low fidelity prototypes. The point of this process is not to just deliver some documents and make our research insights look more concrete but to make this is a process that forces us to really think and be detailed oriented through product design.
Members in the product teams seem to always be confused with the terms of “user researcher” and “customer services”. I want to convince myself that the reason is due to the fact that these two terms are quite similar when pronounced in Chinese, but actually they are totally different from each other. Then why does it happen? As a user researcher, I interpret it as these two terms have a similar mental structure in people’s mind. Accepting this fact really upsets me, but I have to face the reality of it.
When conducting interviews, sometimes we simply use the feedback from users and transfer them into words where it is used in our research report to help aid us in improving our products. When the product team has a product idea to be tested, we simply ask users if they like or dislike the feature that is being tested. As a matter of fact, as user researchers, we need to think even deeper.
For example, if we have complaints from our users that our promotion information through push notifications, text messages, EDMs are too disturbing then our marketing team needs to problem solvers and utilize a different strategy of sending out shopping triggers to consumers. As a slothful “customer service” user researcher, the research focus would be landing on when to send those notifications in order to be less disturbing; a less slothful “senior customer service” user researcher would focus on what kind of information would be less disturbing. And for a real user researcher, the question would be in which shopping scenarios, what kind of information, would be useful to which particular type of online shopping users.
Narcissism is defined as a word for me as a user researcher and as a word to describe me the actual me. Let’s skip the part about the actual me, and focus on the working scenario. I have to confess that I really spent a lot of time on thinking what kind of methods I was going to use before tapping into the research. Taking on a methodology which is heavily imposed should not be looked down upon. This was the reason why I was considering it such a serious topic that should be questioned. Honestly, I was in deep thought because I always tried to use some fancy or never-tried-before method in order to showcase my research skills instead of solving the real product problems at first.
When I crafted Personas, I would definitely use both qualitative and quantitative method to emphasize that cluster analysis and corresponding analysis were both applied. When conducting satisfaction research, I needed to make sure that structure equation was included. However, as time goes by, when I look back on this behavior, I knew it was narcissism that made me focus so much on the deliverables of work because portray a self-image of myself.
It’s something like if I purchased a Tesla, instead of taking a ride to showcase how sexy it is, I spend days and nights reading the product manual. Is it possible? Does this make any sense? If what I want is to make the products I worked on become better, I think the deliverables of my work should not be user reports, but should impact on those insights that shed on the product teams. One of the ways to make this happen would be having our product teams be more involved in user research process. In my article Doing Research as an Evangelist, I wrote how I conduct user research to involve them more.
Above is the picture that we usually use to illustrate the importance of the way we frame our interview questions. Had Henry Ford only asked people what they wanted, he would not have invented a car but a “faster horse”.
I mention dogmatism here not only to confess the way we talk with users, but the way we design the whole research, from recruiting respondents to conducting tests, analysing research data, and working out the final research reports. Here are some ways to make user research lean and agile from InVision’s blog. Below are three valuable suggestions I get directly from them.
– Desk research:
In an era when access to the world’s knowledge resources is commonplace, there are ways to bypass the traditional primary research approach and look for studies that have already been conducted by somebody else.
There are a number of sources to look to for scientific knowledge. Here are a selection of the ones we consider the most useful. But we have no doubt you’ll uncover many more, including those specific to your field of study.
– Guerilla research
Whether you’re exploring a particular research problem or testing a design solution that you already have, guerilla research can be a huge time-saver. Guerilla research means getting out of the building and spontaneously recruiting and interviewing research participants.
In order to find people relevant to your research problem, target places where your audience might regularly spend time. You don’t want to discuss speed dating with couples shopping for wedding rings, for example. But if you research problems that are common across the demographic spectrum, you might want to visit places where people will have spare time to help you: train stations, parks, restaurants, etc. People love to tell their stories—you just need to listen. During her UX Cambridge workshop in 2015, Dr. Emmanuelle Savarit described her successful efforts to conduct quick interviews with people waiting for trains on the King’s Cross station.
– Remote interviews
Interviews are usually time-consuming—recruiting people, preparing scripts, scheduling, conducting, analyzing findings. But by using smart, online participant recruitment methods, you can quickly build a pipeline of participants you want to interview.
Conducting interviews via Skype or Google Hangouts is one of the ways of doing remote interviews. Recently, we tried out a new way by using instant-message tool such as WeChat and QQ to do it.
I used to say that my vocation as a user researcher is to understand users. I loved presenting my research reports in front of product teams and hearing the “wows” from them. I enjoyed every bit of what they told me. Product teams never imagined users would act in such a way where user insights in my reports are really insightful. I almost hated product team participating into research process because then it will restrict my report of having this sort of awe.
In my current position, as user researcher, one of the main roles is to help product teams understand users. Instead of being a psychic, a user researcher should work as a evangelist. One of the reasons why I define my job as providing product teams with the ability to communicate with users is because they will have more empathy on users. This is not just for solving a single product design problem in the fastest time, but to build their understanding on users for the long term.
Another reason can be explained by Dan Ariely’s IKEA Effect Theory. When product teams are actually being more involved in user researches, there are higher possibilities that research insights we receive from researches have more impact on the products they are building. Just as David Travis writes in his blog about how to evangelize user research, “no presentation will be more persuasive than having the design team observe users as they do their task”.
Now back to the prediction that I might be a saleswoman one day. I don’t think my ex-boss’s prediction is incorrect, but as of right now, my job title stands as user researcher. In fact, everyone sells, it just depends on what your selling. Some people sell products, their companies services, but I sell user research ideology to product teams and in order to make it a best seller, I need to watch out for my 7 deadly sins and make them become righteous virtues.