Evaluating Trade-offs

"Trying to ignore it by trying to pin it down with fake numbers that you just made up for yourself at great expense is a really bad idea... It doesn't really matter if it's 1,000 or 1,001, who cares? It orders a magnitude larger than the alternative, and so it is better."

"In my opinion, when you have to make a choice, think more and ship better. Most experiments should be thought experiments. They should not even be tried out because they're obviously going to fail which is contrary to, 'Let's try it out, and then let's see.' I think that wastes a lot of company time."

"Waste is a prospective problem, not a retrospective one... if you wouldn't start this today, then that means that everything that you're putting into this going forward is the actual waste."

"Tenets are really decision-making tools and it's sort of like... A classic one is paper versus plastic. It's just too complicated to reconsider that every time you're at the grocery store. So you sort of make a rule for yourself and you're just a paper person or a plastic person, you move on from there."

"If you're a great engineer, the answer to almost every problem in your business is engineering... you probably by default should question it."

"But then you have all these teams that are really managing some tension in the business, which is totally different than a funnel. So say if you have a marketplace quality team... if I let on a bunch of new supply to a marketplace, probably the first thing that happens is our GMV or our revenue goes up... but if those are on average lower quality, it's going to degrade the kind of customer experience and reduce retention over time. And so the model that they're trying to build is how to manage that tension."

"I would look at how many more ideas the founder has on how to make it grow. If it's not going well and you're out of ideas, that is usually a good time to pivot. But when you have half a dozen or a dozen really good growth ideas that you haven't tried yet, try them. Right? Hey, give it a shot."

"Andy cautions in the book that, 'Look, most of the time CEOs want options. They want to hedge their bets, but what you really want to do in these strategic inflection points is go all in one thing.' ... I just killed Carousel, killed Mailbox. Went all-in on productivity."

"Run it through three filters... head, heart, hands is what I like to think of, which is head, okay, how's this going? What are my thoughts? ... Emotionally, how is this going in your heart? ... But then how does it feel in your body? Our bodies are ultimately where we take in our stimuli and then store all of our experiences."

"It should really be this thoughtful strategic decision-making process of how much will this cost me all in and is this the right use of funds... if you took that exact same million dollars and put it into brand ads, or they put it into influencer campaigns... would you make that million dollars or $1,000,001 back faster?"

"Instead of spending $15,000 on SEO, you can get... you can go to all these shows for the exact same budget, and you get users who are interested, they're in market, they try your tool out at the booth, and then they leave and they're leads that you could follow up with."

"Be very clear with the tradeoffs that you need to make and present those tradeoffs back to your leadership team. So here's what we're doing and here's what we're not doing and why and which one would you pick? Give them a menu of items."

"The hardest part in that model is, how do you balance delight versus margin? ... In this case, both right, right? We saw a very small change in retention. That was the surprise. And so, it went from something like 4.5% canceled in the control. It was 4.45. Very small change."

"If as a startup, whether it be early stage, but as a startup you focus too much on those marginal differences between groups of people, you can run the risk of making big mistakes, which is making too many changes too soon and learning very little and adding crazy amounts of code complexity and overall organizational complexity to what you're building... It's important to keep things super simple... every new A/B test you want to run or every new hypothesis you have, if you now have a version of your app that's different in Mexico, in China, in India, whatever, you're going to have to run that test and whatever change you have in all of those versions... and time is almost more important than money when you're a startup."

"Everything you want is on the other side of worse first... If I'm optimizing for tomorrow and I just want to have a great day tomorrow, I'm going to stay exactly where I am, because my life will be better tomorrow if I don't make any changes... So if you realize this and instead ask the question, the version of myself five years from now, what would they wish I was going to do right now?"

"I think the time to quit is when you can no longer see the vision and you can no longer really believe the vision."

"The saying is that's like peeing in your pants in cold weather. It feels really warm and nice to begin with. And then after a while, you start to regret it. It's about being short term, basically."

"There's no free lunch. You don't get something for free. You have to give up something. Then what do you give up? It's essentially you give up the right thing that market or your user wants at that given space and time. It's just the craft of building a business or building a product."

"This idea of flexibility versus perfection, and then regardless, celebration, is core to how we think about the streak. Because I think for the streak for us, it's very much a bend not break."

"We'll plot those options on these axes of easy to use versus hard to use and perfect solution to the customer problem versus this is just an okay solution... we call this activity magic lenses."

"We spend a lot of time quantifying things in terms of a common currency. ... if we have, say, a dollar to spend, we know what we get depending on where we put it, over what timeframe."

"In some environments there's this should/can divide... some people are just locked into the can. They're uber pragmatic people... And then there's people who for some reason go in and say, 'Should we do it? What should we do here? If those things were not an issue, what should we do?'"

"What you really want is you want to diagnose with data and treat with design. Data is not a tool that's going to tell you what you should build... but it can tell you if you have a problem and where that problem or opportunity might be."

"The difficulty in making product decisions comes down ultimately to making these trade-off decisions and you have to look at things through the lens of the customer. You have to balance that with what's driving the right business outcomes."

"In FinTech, every trade-off is existential. When you found a company, every trade-off is existential. You may not exist as a business if you make a wrong decision."

"If you know that you have a bridge to move up market, then giving up the long tail of individual users can be very worthwhile... Figma is a great example of that. This was a company that took a while to monetize. And even having free usage at the individual level, that was the way to just drive insane community and love for this product."

"It comes down to really clearly defining what your criteria is. What's important, and then among that criteria what's most important... identify the criteria that are most important to you... for each criteria, what's their relative weight, what's their importance? And I make it add up to 100%. And then I... give everything a score, and I just multiply it out."

"I always ask, 'What are you optimizing for?' That's the question, it's what are you optimizing for? And it's the short, medium, lon-term in product, but it's what are you optimizing for today, this quarter, this year?"

"Marketplaces are a little bit like a game of whac-a-mole... Many of the changes that are most consequential create winners and losers. And rolling with those changes is about recognizing whether the winners you've created are more important to your business than the losers you've created in the process."

"What's that cost? Well, when they unsubscribe, we can't mail them. So we did some data science study on the side and we said, 'What is the value that we're losing from an unsubscribe?' And we came up with a number, it was a few dollars. But the point was, now we have this countervailing metric. We say, 'Here's the money that we generate from the emails. Here's the money that we're losing on long-term value. What's the trade-off?'"

"Facebook may be the most extreme example of trying to solve so many different problems... a big part of the product manager's job in a situation like that is going to be managing trade-offs... In Facebook's case, sometimes it's, 'Oh, if we do this, we just can't do this.' ... Whereas at a startup, a lot of it is time. Time is the main variable."

"Now, we can have a really cool conversation about trade-offs. So, let’s say we’ve got three different integrations here... If we just did this on one of those branches, would it be a win? And if we did it on all three, how much more time would we have to negotiate for and would it feel worth it?"

"If our models are smarter, more people can use them to solve harder tasks and make more money. And then on the security side, it's like, or we can invest in security and they're more robust, but not smarter. And you have to have the intelligence first to be able to sell something."

"We use data when it favors us, we use anecdotes when it favors us... we exit the QBR, you high-five each other. Well, 'Good job, team, great job,' et cetera, et cetera. But now, you have signed up for even more work for a feature you should not have built in the first place. That's why we're busy. And through a product leader's life, what happens is we just accumulate all of this debt, feature after feature."

"The difference that you could possibly achieve between having this feature and not having this feature is like this much... The cost of doing the analysis was this much. So it's guaranteed to be a loser."

"Systems thinking. Think of all the players in the system, think of all of their incentives and how they interact with each other."

"The driver performance metrics, they span across safety, the compliance to the road rules, our ability to make adequate progress as in not get unduly stopped or stranded in dense traffic situations... you can be very safe if you're not moving at all. That's not what we're building. We need to make sure that the riders get to their destination on time."

"I think the best way to guard against what you think is the obvious solution is to work with multiple solutions for the same opportunity. Compare and contrast. We already know this intuitively... When you're looking for a job, you don't talk to one company. We know if we want to make good decisions, we need options and we need to evaluate the pros and cons of each."

"In general, I would say letting people go is probably better than reducing salary for everyone. But let's say that you need cost reduction of 30%. One of the options is let 30% of the people go. Another option is actually reduce salary by 30% for everyone... If the organization feels that they are committed to each other, they will prefer the second way. If this is more of individuals, then obviously they will prefer the first way."

"If you have a choice and you choose the easy thing and it works great. If you choose the hard thing and it works great, you did more work, but if it doesn't work and you chose the easy thing, you've actually not learned anything... But if you do the hard path and it doesn't work, actually you still win because you've now done something hard."

"Oftentimes we'll use a traffic light. That means that the three options are three rows. The columns in the table will be criteria by which to evaluate the options... And then obviously it should be color-coded, red, yellow, green based on how it stacks up against those criteria."

"It's like 70/20/10 across horizon one, two, and three. Horizon one business... generally there's about a 70% allocation... Horizon two, which is an adjacent thing... usually that tends to be around 20%... Then there's horizon three, which is three years out... and that's about 10%."