AI Builder Playbook
Building products in the age of AI
Who This Is For
PMs building AI products, engineers using LLMs, founders in AI space
Any stage
You're incorporating AI/ML into your product
AI is a tool, not a product. Users don't care about AI—they care about outcomes. The best AI products make AI invisible. Don't sell 'AI-powered'—sell what it enables.
1. AI Strategy
Not everything needs AI. Know when it creates value and when it's a distraction.
AI Product Strategy
AI strategy should focus on using algorithms to scale human expertise and judgment rather than just...
View Skill → →Evaluating New Technology
Be skeptical of 'out-of-the-box' AI solutions for enterprises; real ROI requires a pipeline that acc...
View Skill → →Problem Definition
Directly experiencing the product as a user or provider reveals fundamental flaws in problem definit...
View Skill → →2. Build with LLMs
LLMs are a new paradigm. Learn the patterns that work.
Building with LLMs
Using LLMs for text-to-SQL can democratize data access and reduce the burden on data analysts for ad...
View Skill → →AI Evaluation (Evals)
The guests explicitly define this as a 'new skill' that is distinct from traditional software testin...
View Skill → →Usability Testing
AI tools allow PMs to bypass engineering bottlenecks to create functional prototypes for real-world...
View Skill → →3. Ship & Iterate
AI products need fast iteration. Ship, measure, improve.
Shipping Products
When shipping an acquired product, prioritize technical integration and feature parity (like stats/s...
View Skill → →Measuring Product-Market Fit
Product-market fit is not static; changes in the external economic environment can effectively reset...
View Skill → →Analyzing User Feedback
Unfiltered community platforms like Reddit provide higher-signal feedback on product flaws and 'real...
View Skill → →4. Scale Responsibly
As you scale, build guardrails and handle edge cases.
Platform Strategy
Platform and ecosystem success comes from identifying 'gardening' opportunities—projects with inhere...
View Skill → →Managing Tech Debt
When a technical solution lacks the operational flexibility required by the business, a full rebuild...
View Skill → →Planning Under Uncertainty
Managing through a crisis requires 'wartime' humility to accurately diagnose problems before attempt...
View Skill → →Common Mistakes to Avoid
- Adding AI because it's trendy, not because it solves a problem
- Shipping AI features without evals
- Promising capabilities you can't deliver reliably
- Ignoring the 'unhappy path' (hallucinations, errors)
- Building custom models when APIs would work