Conducting User Interviews

"Don't use the data alone to figure out who the marginal user is. It'll give you a clue where they are and what might be wrong, will give you some hints. It's not going to give you the answer. You have to go watch them to find the answer."

"We were literally on the ground in India every three months took a team of engineers with me. I'm talking like we're in Delhi in people's homes and Mumbai, and we went to go investigate what was going on, watch people make friends, watch them use people you may know."

"The first thing we do is we try to extract the story from the customer. And it doesn't have to be my product, it could be somebody else's product if I haven't built it yet. It's like, what are people going to fire when they hire me?"

"I only talk to people who have already tried to make the progress... I need to talk to people who did something and tried and though they might've failed, what made them try?"

"The other tip I have is to not have a discussion guide. It drives people crazy because everybody wants to ask the same set of questions, but the problem happens is when you ask the same set of questions, you actually don't follow the ones that actually have the most meaningful information in it"

"Speaking as a former PM, I would not ever give up spending time with customers and watching their pain. That's how I fell in love with product was I saw my internal customer 12 years back now fighting with the keyboard, fighting with the mouse, and I was just like, 'Oh, my gosh. What's this guy doing?'"

"I cannot see product managers saying, 'I don't want to be a part of that conversation.' Then you know what? I'm going to say it then don't be in product."

"I would start pretending that the product existed in the interviews. Only to find out, at the end, people was like, 'Oh, it's not out yet.' And they're like, 'What I want to use...' You could really feel the like, 'No, no, no, no, this has to be out. I want to start using it.'"

"I just said, 'Listen, let's get some older people who are hosts in here to see how well they will be versed in managing their listing purely on mobile.'"

"I actually went into the homes of these hosts all around the world... I think I was lucky because Brian did less of that than he did with other people."

"we spent a lot of time talking to users directly, and a lot of guerrilla user research techniques. Literally, sometimes going outside and just talking to runners in passing to understand what were the challenges in finding motivation"

"I usually tend to really enjoy my conversations with people who decided not to use HubSpot and to really try to unpack what drove that decision. Was it as calculated, as I think sometimes we can all maybe assume... And then you often learn that they're sometimes kind of emotional, really instinctual and visceral"

"We do consumer research or customer research at that stage... putting the names in a series of drills... seeing how these names fire their imagination. And that's the most important thing in research, not is the name popular, are they comfortable with it, does it fit to concept."

"You have to get out in the world and talk to people in person. And you can't just hide behind your keyboard and call that talking to customers. Right? I think a lot of folks, the inclinations are to build a landing page and buy some Instagram ads and try to get people to sign up for something. And again, maybe that's something, but I think a lot of the reason people do that is they're just shy and they don't want to put themselves out there because it's a little awkward to go talk to people."

"I think it's look at your calendar and there should be 20 or 30% of your time that the calendar says something like customer meeting, customer call, meeting with who, meeting with this person. And when the calendar is not that or it's all... Again, what you're actually doing is just buying ads to try to validate your idea, I don't think that's talking to customers."

"When you delegate such an important thing as customer research and collecting their feedback to someone, you will get a refined filtered version. You won't get these nuances of how people describe things, which emotions they feel like and so on, and where they think stuck. It's very easy to lose all those important bytes of information in this process. So I'm a strong believer that it's very important to have direct communication channel to customers."

"Just like going to a doctor, you don't want the doctor to come and say, 'Hey, can I show you a movie of the operation I just did?'... What I would like to do is talk to you about this pain I have in my side. And then when the doctor asks you good questions about it, you go, 'Ah, this is a good doctor. I'm going to trust this.'"

"The problem is, which anyone who hasn't done sales before that joined YC, they realize this, is that if you take the average customer group in the world, 90% are not early adopters... Those 10 percents are the early adopters. They're The ones that you actually want to reach. But that means you have to reach 10 to find one."

"I think people just have this fear that, 'If I get a lot of rejection, that means everything is bad.' But the rejection should be put in context to the early adopter idea and that most people who don't care are not early adopters... you just need to overcome that fear and just learn that there's nothing that's really that bad that can happen when people don't use the service."

"The best way for you to figure out what is the intensity of the problem is not to ask them but to watch them or to watch them solve the thing that they do... have them screen share, have them walk you through their daily workflow about the area where you're doing some discovery. That is the best thing."

"We did some UXR in India many years ago... the idea of keeping a flame lit to show your commitment to something makes less sense... it was a metaphor for a mechanic, and when we redesigned it... we actually focused the design on the screen to show your number going up."

"These conversations with customers are so much more fruitful and pointed when you've got the context of I know exactly what my hypothesis is and you have prototypes to show them."

"The moment the customer felt compelled enough to go out of their way to talk about some problem, that's a unbelievable gift. I will leave a meeting to just get one message back to them. If you're text message friendly with five or 10 of those, you are going to have so much direct signal that is infectious."

"You have to carefully listen and not pitch your customer... people don't want to be pitched. I'm sometimes on a UXR call... and the first thing that they start talking about when we get on the call is, 'Hi, I'm the CEO of X, Y, and Z company. We do one, two, and three. I want to show you a demo.' It's like, 'Hold up, I'm a customer. I have...' What a wasted opportunity you've just done here."

"I'll prompt our customers, I'll say, 'Hey, do you mind just opening up your email? What's in there?' Or, 'If you weren't talking to me right now, what would you be working on?' Or, 'Hey, last week, what grinds your gears? What are you not looking forward to?' Or magic wand, 'What do you wish you could just have off your plate immediately?'"

"I will have them practice charging me. I'll just say, 'Hey, I'm just a friend. I'm trying to help you out... send me an invoice or a payment link for $1 right now, right now. That way, when it comes time to actually charge your first customer, it won't be your first time.'"

"I would be very open and honest with where you are. Hey, I'm an early stage startup. We have a lot to learn. Can we kind of gain your insight into how this problem is manifesting on your side?"

"The team, data scientists included, just sat and made phone calls. ... that's where qualitative research is superior to quantitative research, it's asking for the context, to actually talking to people to figure out what was the motivation, what worked, what didn't for them."

"One of my big mantras was, 'We don't validate, we falsify. We are looking to be wrong.' Many PMs, many designers are not in that place. They do not want to be wrong. They're looking to validate, and that's user-centered performance."

"A researcher who's going to ask customers what they want is a bad researcher. You need a different researcher. I've never done that in my career. No one on my team has ever run a study that's like that."

"We showed those to people across the political spectrum. We saw, hey, people really like these. Whether they're on the right or left, they seem very open to reading these community notes even when they're critical to people of their own side."

"I think I just wanted to understand what that role was like to build a lot more empathy towards the job and make sure that our product was catered towards what those needs were... this is eventually what led to, I think a lot of the support I gave to our driver teams when they were pushing for, Hey, we need more waiting fees, we need to add multiple stops in order to make sure that hey, a lot of this extra work was actually compensated."

"PMs need direct exposure to raw material. End of story. And so that's where I think you just need to constantly be talking to or interacting with whoever is your customer... you can set up workflows either via Zapier or something else to say, 'Who was the customer? Pull their email, put that into a sequence, drop in my Calendly' and you just have interviews showing up automatically on your calendar."

"If you go and ask someone, 'How much should I charge for this product?' You're actually going to get garbage back... People are absolutely meaningless, relatively super smart."

"I tell this to user researchers at the companies that I coach, if the product manager and the designer are not available to be there during their products test, cancel the test. They need to be there. This is what makes them useful to their team."

"The first is they have to really get to know the users and customers. They have to be considered pretty much one of the experts on the users and customers. I remember when I was an engineer wanting to become a product manager, the person coaching me said explicitly that I was not allowed to make any decisions for the team until after I visited 30 customers."

"I know that I was not allowed to take the product manager role until I had visited 30 customers in person, 15 in the US, 15 in Europe. That was just the person who was coaching me. That was their rule. And all I know is those 30 customers changed my life because I thought I knew our customers and I really didn't."

"It's also about finding participants who want to opt into research. So making sure that you have customers aware that, 'Hey, we might contact you to do customer interviews. This is why. Do you want to participate in alphas and betas? If so, this is what it entails.'"

"Once a month, ideally, you should just have some sort of a schedule for yourself where if you're at a larger company and you have a user researcher who can recruit people for you, that's great, but if not, just go find people who either fit the demographic for your user or even are your user and have them sign up for an account and walking through it."

"Something that I find incredibly useful is to not just ask users of your product what they think about your product, but to ask non-users about your product, why they're not using your product. And actually I think that those are the most insightful conversations."

"You would spend maybe Monday to Thursday and you would actually go into the building where the customer worked and you would work alongside them. You would literally get a desk there... you learn to live and breathe the customer's problems and you learn to speak their language."

"Monday, you go in. You do your meetings. Monday night, you build something. Tuesday, you show it to somebody. Tuesday, you get the feedback. Tuesday night, you iterate on it. Wednesday, you show it to somebody. Wednesday night, you iterate on it. So, you get four of these, five of these cycles every single week."

"My goal is to feel bad in the same way that customers feel bad. They come to us with a request, 'Hey, we want X,' and it's like there's something motivating it... What is the actual emotional valence that is motivating whatever you're telling me?"

"You can ask people directly like, 'How do you feel?' And they're not necessarily going to tell you, but if you have a long enough and deep enough conversation with them, you start to level with them, and you're starting to see stuff from their perspective."

"Imagine building a startup, like a product from day one and not being able to sit right next to your customer and being like, 'Show me how you do this or show me how you do that.' It's incredibly hard. You're looking for such faint signal. The idea that you would try to get it through layers of indirection and games of telephone is crazy."

"What I used to do with people was map out their social network, all the people in it, their family, their friends, how they communicate. We'd map on all the channels, we'd talk about what worked well, what didn't. And, we did this with dozens and dozens of people over the course of maybe 18 months. And the same pattern emerged every single time."

"We identified something quantitatively that was really interesting... Our assumptions about why that use case was that use case were wrong. And when we ended up talking to users, we had some really surprising and fun conversations as a result, and we were also able to recalibrate and understand what those people were solving for."

"Once you interview between 7 and 14 people, you stop learning new things. Less than 7, you don't learn enough, more than 14, you start learning anything new. And so if you interviewed two people, you probably don't have enough data. If you interviewed 22, you probably had too much, so they don't even right size their efforts."

"Then they go into these conversations asking leading questions, which really are designed to get the customer to say what they already want to be true, which is so they haven't done enough research or they've done too much and then they've blown up all of the results before they've even heard anything."

"Lighthouse Users Program... we work with 10 and we prove that the problems that they had are the things that we solved... put these people in front of the whole team, not just the PMs, PM, designer, engineering. We meet on Zoom, we chat, we work with the same ones over months to build a product. They are with us on Slack."

"I think interviewing is a grossly underestimated skill. Grossly underestimated skill. So that's the first thing is that if you're not collecting rich stories in your interviews, it's going to be really hard to identify opportunities."

"The most common strategy is to allow your customers to opt in while they're using your product or service... instead of saying, 'Would you recommend our product or service to a friend or colleague?' it says, 'Do you have 20 minutes to talk to us?' If they say yes, you send them some scheduling software, they pick a time on your calendar, and voila."

"I teach in our interviewing class, you really don't have to think about what to ask. You could run an entire interview by asking them one question... 'Tell me about the last time you watched something on a streaming entertainment service.'... All I have to do is just be curious about your experience. What I'm doing with my questions is just helping you tell the timeline."

"We refer to it as dollar-driven discovery. We get very specific about not just the normal way of doing customer conversations and customer discovery, but how do you find that a customer is willing to pay money for this thing and a lot of money?"

"I need your help. It is very important to me that this company succeeds and does not fail. So I don't want you to be nice to me. I want you to tell me is Persona a necessity for your company? If we went away, how painful would that be? If a competitor came along that charged half as much as us, would you switch to them?"

"Back in the day, we were doing studio visits with sellers. We were going to their workshops, we were going to their homes. We were seeing how they make items, we were seeing how they package and ship them out. We were bringing them into the office when we were running hack weeks to say, 'Hey, we've got this crazy idea. Is this interesting?'"

"If you speak with a hundred people, but if you actually speak with 20 people that you don't know, you will get validated whether or not this problem is real or not."

"Getting to know your customer always pays off. So it's user interviews and getting to know your market, your customers, and your prospects always pays off. Customer research, inside surveying, interviewing panels incredibly useful. And I found that to be very especially early stages. The amount of clarity at momentum that it can create inside of a seed Series A up to Series B company when you first do some proper research push."

"Literally had a Slack integration that we pulled in a whole bunch of information about anyone who signed up for Airtable... and literally would sit there and had a little button in each of the records that came in that would allow us to email them immediately... 'Hey, we'd really love to get your feedback.'"

"The simplest way to do this is to write a template email for yourself... 'I would really love to hear about your experience so far and get your feedback. Do you have 10 minutes to talk on the phone?' I know it is important that it's on the phone and not a survey because you get way more instinct from unstructured conversation."

"I ran him through the user study, I knew we'd need a bottle of wine to finish and it took hours. The type of sentence in Figma was so slow... Payam wrote a very long doc for us about all the things that he wanted to see in Figma after we did a user research study with him."

"Become best friends with a researcher, and spend time watching customers use the product firsthand because what they maybe report on or are trying to do a study about might be very different from what you observe, but you really just need that front row seat with customers."

"I still remember distinctly flying down to LA with Kevin Systrom to go do a user research study... there's no substitute for that... you can't empathize with the summary. You have to be in the room fully immersed, no phones, just actually hearing the words and the intonation."

"The most humbling moments for me as a product leader have always been shoulder to shoulder to customers. Watching them actually go about their day, not just use my product but go about their day because what you end up getting is context for a lot of data that you're missing."