---
title: "Lenny's Podcast — 2025 Q4 合集"
date: "2025-01-01"
source: "Lenny's Podcast"
url: "https://www.lennysnewsletter.com/"
---

<div v-pre>


# Lenny's Podcast - 2025 Q4 (17 episodes)

This file contains 17 articles/episodes.

---

## [1/17] How to find hidden growth opportunities in your product | Albert Cheng (Duolingo, Grammarly, Chess.com)

**Albert Cheng** (00:00:00):
Growth as the job is to connect users to the value of your product. Growth sometimes gets this reputation that it's just pure metrics hacking.

**Lenny Rachitsky** (00:00:08):
You've worked at three of the most successful consumer subscription products in the world. What do you think is the biggest missing piece that people don't get about building a successful consumer subscription product?

**Albert Cheng** (00:00:18):
User retention is gold for consumer subscription companies. If you don't retain your users, then a lot of the onus is on getting them to pay on day one.

**Lenny Rachitsky** (00:00:26):
Noam Levinsky, he said that I need to ask you about the biggest monetization win that you found at Grammarly.

**Albert Cheng** (00:00:31):
The lived product experience for most of the free users was that Grammarly was just a product to fix your spelling and grammar because those were the free suggestions. What if we actually sampled a number of different paid suggestions and interspersed them to free users across their writing? All of a sudden, people were seeing Grammarly as a much more powerful tool than they were before.

**Lenny Rachitsky** (00:00:50):
What's the most counterintuitive lesson you've learned about building teams?

**Albert Cheng** (00:00:54):
I saw some of the highest performers just being people that had very high agency, had that clock speed, had that energy, but they didn't necessarily need to have deep experience on that matter. Sometimes experience could be a crutch, especially in this world where the grounds are shifting so fast with AI. A lot of your learned habits actually need to be intentionally discarded.

**Lenny Rachitsky** (00:01:13):
Today my guest is Albert Cheng. Albert is known as one of the top consumer growth minds in the world. He led growth and monetization at three of the most successful and beloved consumer products in the world, Duolingo, Grammarly, and now Chess.com. Earlier in his career at YouTube, he worked on streaming and gaming features used by over 20 million people.

**Lenny Rachitsky** (00:01:32):
His unique approach to growth blends marketing, data, strategy, and product management, and in our conversation, we cover a lot of ground, including his explore and exploit framework to find growth opportunities. His biggest and most interesting growth wins at Duolingo, Grammarly and Chess.com, how he uses AI to accelerate his growth work, what he's come to realize about the power of brand and community in your growth work, his top experimentation, best practices, why his goal at every company is to run 1,000 experiments a year and so much more.

**Lenny Rachitsky** (00:02:02):
A huge thank you to Erik Allebest, Noam Levinsky, and Jorge Mazal for suggesting topics for this conversation. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube, it helps tremendously. Also, if you become an annual subscriber of my newsletter, you get 15 incredible products for free for an entire year, including Lovable, Replit, Bolt, n8n, Linear, Superhuman, Descript, Wispr Flow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast, ChatPRD and Mobbin.

**Albert Cheng** (00:04:33):
Thanks for having me, Lenny. Excited to be here.

**Lenny Rachitsky** (00:04:35):
I'm even more excited to have you here. So as I do for every podcast conversation, I reached out to a bunch of people that you've worked with that know you well to find out what to ask you about and what topics to spend time on. Jorge Mazal, who is famous in my world for writing, what was for the longest time, the most popular newsletter post on my newsletter, it's actually people have usurped it now, but it was stuck there for a long time. So here's what he wrote. "It is a mystery to me how Albert is able to do what he does. I am actually eager to listen to this episode and learn from him."

**Albert Cheng** (00:05:10):
That is super nice. Thank you, Jorge. I've learned so much from him. I'm the type of weird person that likes to wake up before their kids and pull up a bunch of browser tabs and look at experiments. So it was perfect that Jorge brought me into the growth world at Duolingo, learned a ton of best practices, and he's just a great guy. Thanks, Jorge.

**Lenny Rachitsky** (00:05:27):
We're already getting into these tactics. I love it. Let me just give a little framing on what I want to do with this conversation. What I want to try to do is to help people learn tools and mental models for finding growth opportunities for their own products and essentially learn the growth mentality that you bring into the companies and products that you work on.

**Lenny Rachitsky** (00:05:48):
What I want to start with is to give us a little insight into how you became what you became. There's an interesting pattern I found across a bunch of recent guests, which is many people were very good at piano when they were younger and were very serious piano players. For example, Head of ChatGPT, Nick Turley was almost going to become professional jazz pianist. You were very serious as a piano player earlier in your career. How did you go from pianist to one of the top growth minds in the world briefly?

**Albert Cheng** (00:06:18):
Well, that's very flattering, but I appreciate it. Yeah, I grew up playing a lot of piano. My parents were immigrants from Taiwan and I was the oldest kid that they had, and so I definitely felt that strong encouragement, if you will, to learn a bunch of things, take them seriously, study hard, and so I did. And my parents, even though they weren't musically proficient, they had a deep love for classical music.

**Albert Cheng** (00:06:45):
So I was the stereotypical baby that would listen to Mozart, I guess when I was sleeping type of thing. And I still vividly remember we had this upright Yamaha piano, and at the very top of the piano we had this countdown clock from 90 minutes. Literally every single day of my childhood, just practice really, really consistently.

**Albert Cheng** (00:07:04):
At first, I really was irritated by that thing, but as I grew older, I started to appreciate music quite a bit more. But anyway, I think what really accelerated my interest and abilities in piano was I feel like I hit the lottery. I had perfect pitch, and so I was able to quickly understand whether I was playing the right stuff or the wrong stuff and just pick up music pretty rapidly.

**Lenny Rachitsky** (00:07:29):
What does perfect pitch even mean? Does that mean which note is playing?

**Albert Cheng** (00:07:29):
Exactly.

**Lenny Rachitsky** (00:07:29):
Okay.

**Albert Cheng** (00:07:34):
Exactly.

**Lenny Rachitsky** (00:07:34):
Wow.

**Albert Cheng** (00:07:35):
So I could listen to a song and then just a very, very clear understanding of which note I'm supposed to start with and if I'm playing something wrong. So it's very helpful.

**Lenny Rachitsky** (00:07:35):
Unfair.

**Albert Cheng** (00:07:44):
It's unfair. Definitely. So anyway, yeah, I got quite good as a teenager in high school and even considered studying at a music conservatory. My intrinsic motivation for music wasn't necessarily as strong at that point, and so I decided to go to engineering school instead, but that would've been an incredibly different career. And to your original point around the relationship between music and growth, I didn't really reflect on this until recently.

**Albert Cheng** (00:08:12):
I have a four-year-old and I'm starting to teach him how to bang on the keys a little bit, but a couple things stand out. One is that I think music and growth, they both rely on this just consistent repetition. You're constantly making mistakes. You have this super tight feedback loop. You have to get really resilient to just making mistakes all the time. And you know that the way of learning is through those mistakes. So that's a thing that I learned very early, and the second thing that occurred to me is that they both have this structural underpinning to them.

**Albert Cheng** (00:08:45):
With growth, you have a growth model, you have metrics, you have experiments, you have channels, things like that. But you also need on a day-to-day basis to have creativity, you got to come up with interesting solutions or hypotheses to test. And the same is true on the music side. You have music theory of scales and stuff, but to create beautiful music, you need that passion, that emotion, that flow. So I think that's the beautiful combination between the two.

**Lenny Rachitsky** (00:09:09):
Fun fact, my wife bought me piano/singing lessons for Father's Day recently, and I've gotten really into this stuff. So I'm learning how to play very basic piano now and learning to identify notes and hit notes with my voice.

**Albert Cheng** (00:09:24):
Nice.

**Lenny Rachitsky** (00:09:25):
What a weird new thing.

**Albert Cheng** (00:09:25):
Could be your next act.

**Lenny Rachitsky** (00:09:26):
It could be. I could go the reverse, I could become a professional piano player. Oh man. No, it's so fun, so hard though. I'm just like, my fingers are like, how do you do four freaking keys at once? I'm just like "What is going on here?" Okay, so let's get into the meat of it. I want to talk about growth.

**Lenny Rachitsky** (00:09:42):
There's a very specific framework that as we were chatting that I think would be really helpful for people to hear and learn from you. You call it explore and exploit. I think there's a bunch of different ways to think about this. Talk about this framework and how that informs the way you think about growth.

**Albert Cheng** (00:09:56):
Yeah, I initially came up or heard with, heard about explore and exploit through my engineering partner at Grammarly, Nermal, and I think he actually had taken some reforged classes. So maybe the original inventor of it might be Brian Balfour, who I know has been on your pod. But anyway, it's a great concept.

**Albert Cheng** (00:10:13):
The gist of it is that when you're in exploratory mode, think of it as finding the right mountain to climb. And then when you're in exploitation mode, it's like focusing your resources on climbing that mountain effectively. And certain companies, I think the warning is to basically spend too much of your time on one end of the spectrum. If you do too much exploration, you can have your team feel a little bit too scattershot, just trying a hundred different random ideas.

**Albert Cheng** (00:10:40):
What's the through line? What's the strategy? How do you pattern match successes across them? And if you do too much in exploitation, which is often the MO of growth teams, it can lead to this saturation and stagnation where you're just locally maximizing a thing. And even though this principle of explore and exploit, it's typically thought of as a macro thing. I like to work with my teams more on the insight level. So I'll give you a concrete example.

**Albert Cheng** (00:11:07):
So I work at chess.com and one of our priorities is to encourage chess players to improve, to learn and improve. So one of the PMs that we have, Dylan, he works on all the learning features. The most used learning feature in our product is called game review. So you play a game of chess, after the game is over, we have this virtual coach that teaches you about your worst moves, best moves, et cetera. And his job is to improve user engagement and retention.

**Albert Cheng** (00:11:34):
And so he's in this exploratory phase trying to figure out how do I drive more of that type of activity? And what he observes is that 80% of people that review their games actually do so after a win. And that's really counterintuitive to when we initially built the feature. We thought that people would want to use it after losses or to see their mistakes such they could work on their mistakes. That turned out not to be the truth when it came to the human psychology and the actual data of it. And so we made some changes in the product experience.

**Albert Cheng** (00:12:04):
When you lose a game now as opposed to surfacing your blunders and your horrible stuff that you did, we flip it on its head and so we show you your brilliant moves, your best moves, and we have coach say something encouraging, "Losing, just part of learning, keep it up." That type of thing. That change alone was pretty dramatic for us.

**Albert Cheng** (00:12:22):
It grew game reviews by 25%, subscriptions by 20%, user retention by a lot as well. So that was fantastic, but the point is that it doesn't just stop there. You have to take that insight, share it broadly across the company. Now, adjacent product managers like the PM working on puzzles can now think about, "Okay, how do I audit these cold patterns in my product and think about making them more positive?"

**Albert Cheng** (00:12:48):
I can change the success rating, I could tweak some copy, change the color of some buttons, and so you now can take this experiment win and expand it out 10X across your organization and that's the kind of exploitation phase of it. So when done right, you can oscillate between the two until you saturate out of exploitation mode and then you encourage the teams to brainstorm and get more creative again.

**Lenny Rachitsky** (00:13:11):
Amazing. Okay, so there's a lot here to follow up on. One is the core piece of advice when you find something that works really well, find ways to build on that learning. One is here's an insight, it can apply to other parts of the product. "Hey teams, here's something we learned unexpected. Maybe this can help you. Also, just keep find more, run more experiments in the same zone." I imagine is a part of that.

**Albert Cheng** (00:13:34):
Yeah, exactly right. In my experience, the typical win rate, and I hate to use that term for experiments, is often something like 30 to 50%. Usually you're trying a bunch of things, a lot of hypotheses turn out not to be true, consumer products are very unpredictable like that, but when you do find a thing that breaks through the noise, and it could actually be a hugely losing experiment too, those are also super valuable.

**Albert Cheng** (00:13:58):
Surfacing those across the company, the original PM running that experiment doesn't necessarily need to be the person that figures out what you should do for all the other parts of your product experience, but the onus is on them to clearly articulate what their hypothesis is, what they found such that then as a growth leader, I can encourage people to swarm around that and try a bunch of different ideas such that the success rate is up and the impact is up. So it's just oscillating back and forth between the two. That is the magic bullet.

**Lenny Rachitsky** (00:14:30):
I think another takeaway here/something that I think about when I hear what you're saying is there's often a lot more wins in an area than people expect that you can continue to find wins and growth in something for a long time.

**Albert Cheng** (00:14:46):
Exactly right. Yes. At the end of the day, users, I think within a company sometimes you can have this siloed approach where you break apart the product experience in 50 different ways and distribute them across different teams, and you assume that users interact with each of the different features with a different mentality, but oftentimes that's actually not necessarily the case. And so sometimes, you can surface an insight that's more human psychology based that can resonate across the entire product experience. And so I think when you can find that, you can double down.

**Lenny Rachitsky** (00:15:19):
People hearing this might feel like, "Okay, yes, find big wins and then find more." Is there something you find that helps you figure out when to explore versus when to exploit when you've exploited too far? Just like any heuristics or I don't know, ways of helping people guide them along this process of exploring and exploiting?

**Albert Cheng** (00:15:41):
One thing that I try to focus on at a company of our scale of a chess.com, right? We're running roughly 250 experiments a year. So we're not the highest in the industry, but we run a decent volume. And so when that happens, I invest in these experiment explorer tools and we can talk about AI as well as another way to uncover and pick out these nuggets of wisdom, but basically, these explorer tools can allow me to look across the spectrum of experiments that are going on.

**Albert Cheng** (00:16:08):
Try to figure out if there are patterns between the hypotheses and the learnings that are happening. And if I'm starting to see more and more experiments that are not statistically significant, that may be a signal to me to say, "Okay, we might've tried to exploit a little bit too far. There might not be as much juice to squeeze. Hey guys, let's get back to the table and brainstorm and be a little bit more divergent with our thinking."

**Lenny Rachitsky** (00:16:34):
Well, let me follow this thread on AI and how you're using AI to help you figure this out. That is very cool. Talk about that.

**Albert Cheng** (00:16:40):
I think one of the latest things that we've been tinkering around with is this text to SQL capability. It's actually pretty powerful. We have this data request Slack channel where for the longest time, and this is still true today, people will toss in all sorts of just one-off questions. How many subscribers do we have in South Africa? Or how long did somebody play puzzles last month or something?

**Albert Cheng** (00:17:07):
And these ad hoc questions, they often take a lot of human time to just go in and a data analyst needs to prioritize it and find time to go run the query. And yes, you can invest in self-serve tooling to improve at this, but also I found that AI is quite good at doing that first pass answer as well. And so we're working on training some of these Slack bots to essentially be the first party provider of a lot of these answers, which makes the company as a whole lot more data informed, I guess.

**Albert Cheng** (00:17:39):
And I think what's also kind of interesting is that just human nature is that if you have a question that you feel like you might be a bit embarrassed to ask or you don't want to bother someone, you just don't ask the question. And so by the nature of having these tools, you get actually a pretty large explosion of questions being asked. And I think you see this in ChatGPT too, right? It's like just having a thing that you can converse with that you feel comfortable in makes a huge difference.

**Lenny Rachitsky** (00:18:03):
Okay, this is extremely cool. So is this something you build basically it's a Slack bot that gives you the SQL query or does it actually do the analysis for you?

**Albert Cheng** (00:18:12):
No, it does the analysis. Yeah.

**Lenny Rachitsky** (00:18:13):
Whoa, so cool. Okay. Is this something you guys are going to release or is this just like somebody, you guys should just build this at every company?

**Albert Cheng** (00:18:20):
We should. It's a good idea.

**Lenny Rachitsky** (00:18:21):
Okay. Okay. Well, there's an episode where everyone in the comments is like, "Open source this." So we'll see if that happens again. That is very cool. Are there other examples of that kind of stuff that you've done or seen?

**Albert Cheng** (00:18:32):
An adjacent example is a lot of the product managers, we're tinkering around with all sorts of different prototyping tools right now. It's just like go from an idea to a representative solution. Today, there's a lot of humans involved in taking an idea, writing up a spec, doing a review, doing design, et cetera. I'm sure you've interviewed plenty of people that have talked about this specific problem.

**Albert Cheng** (00:18:52):
And so for us, we've invested a bit in at least carving out the main screens of our product experience, things like our onboarding flow, our home screen, our chessboard as an example, and building essentially AI prototypes of those using tools like a V0 or a Lovable. And when you have those foundational pieces, you can then share them with the rest of the company and they can use that as a starting point and then they can try to put their ideas on top of that and then they become a lot more discussable and hopefully testable relatively soon.

**Lenny Rachitsky** (00:19:25):
What's in your AI stack along those lines?

**Albert Cheng** (00:19:27):
The PMs are mostly using V0. The designers love Figmas, they're using Figma Make. The engineers are using a combination of tools right now. So Cursor, Cloud Code, GitHub, Copilot. Marketing teams use all sorts of tools for translation, subtitles, content adaptations, et cetera. Customers support uses Intercom then. So there's quite a lot of tools that are used across the company.

**Albert Cheng** (00:19:50):
I would say though that something that is annoying to me is that we haven't yet figured out the bridging from the tinkering to the workflow quite as seamlessly as I would like. And so each sub-function, even though the common I guess wisdom now is that AI is going to strip away these functional titles. It is true that based on your experience, you may gravitate to using a type of tool more. And if that tool isn't as interoperable with some of the other tools that you need to pass down the chain to actually ship it into production, at least at our scale.

**Albert Cheng** (00:20:24):
I think for smaller startups, sure, PMs should just go ship it, but for us, we are still doing some handoffs between functions. I expect that to change over time and we are investing in some of design system components and MCPs and stuff to make it a little bit easier. But yeah, it's an investment and it takes time to smooth things out.

**Lenny Rachitsky** (00:20:42):
I want to come back to this topic of how things have changed and how you work as a product person, as a growth person across the companies you've been at. But first of all, I want to talk about another example of finding growth wins and monetization wins. Noam Levinsky, who is Chief Product Officer at Grammarly, you worked with him for a while while you were at Grammarly. He said that I need to ask you about the biggest monetization win that you found at Grammarly and how you discovered the opportunity.

**Albert Cheng** (00:21:10):
I had the pleasure of working with Noam and his product team at Grammarly. Some context first for those that don't use Grammarly. So Grammarly is an AI-powered writing assistant. And so typically, people will use it as a Chrome extension or a downloadable desktop client. And basically what it does is it overlays your writing with a bunch of different-

**Lenny Rachitsky** (00:21:28):
I use it. I'm a big fan. I use it-

**Albert Cheng** (00:21:30):
Correction, so you're a big fan.

**Lenny Rachitsky** (00:21:30):
... And it saves my life.

**Albert Cheng** (00:21:32):
Fantastic. Glad to hear that. Grammarly is a freemium business model, which means that over 90% of our users are on the free service and the rest of it pay for subscriptions essentially, right? And so one of the teams, they work on subscriber conversion, PM there is Kayla, that team is great and their job is to figure out the free to paid subscription path.

**Albert Cheng** (00:21:54):
And so one of the realizations, one, is that we weren't actually tracking the events that well for the types of essentially suggestions that people were getting and how often were users seeing paywalls and stuff like that. That's kind of step number one. We have to put that instrumentation in. Step number two is that, "Hey, we noticed, actually first let me explain some of the logic."

**Albert Cheng** (00:22:18):
So as a free user, you basically get these underlines across your writing and if you accept all of them, then you see the paywall and that encourages you to subscribe for more nuanced features. As a free user, the main things you get are spelling, grammar, they're basically correctness things. And as a paid user you get that, how do you improve your tone to be more empathetic? How do you improve your writing to be more clear?

**Albert Cheng** (00:22:40):
How can you rewrite entire sentences, that type of thing. And so the observed behavior from all that tracking and data was that actually a very small percentage of our free users was deciding to accept all of their suggestions. They were more picking and choosing as they go, and I wonder if your experience is kind of similar too.

**Lenny Rachitsky** (00:22:59):
Definitely, yeah. I'm always like, "Wait, stop rewriting everything." Just like this part is wrong. I will fix it. Yeah, I'm very much a pick and choose.

**Albert Cheng** (00:23:07):
That's right.

**Lenny Rachitsky** (00:23:07):
Correction person.

**Albert Cheng** (00:23:08):
And then the second thing, which is I think equally if not more interesting is that I was at this company during this generative AI transformation, which is obviously still going on. And quite frankly, both the company brand as well as the lived product experience for most of the free users was that Grammarly was just a product to fix your spelling and grammar because those were the free suggestions we were showing people.

**Albert Cheng** (00:23:34):
And so we decided to flip that on its head entirely and we said, "Okay, what if we actually sampled a number of different paid suggestions and interspersed them to free users across their writing?" Such that they were intermingled and we would provide a limited taste of what the paid offering had to provide. And on the surface, even though it's rational, the concern is that if we give too much of this away, then will people want to subscribe?

**Albert Cheng** (00:23:58):
And we found completely that was not the case all of a sudden, people were seeing Grammarly as a much more powerful tool than they were before and our upgrade rates nearly doubled just through this change. And so I think this is interesting, just modernization learning that especially if you work on a freemium product, try to have your free product be a reflection of everything that your product can offer you. Obviously to an extent there's some costs involved with some of the paid features and things like that, but it generally will pay for itself if you're able to put your best foot forward and go do that. So that really worked well for us there.

**Lenny Rachitsky** (00:24:36):
I think this is what converted me to being a paid Grammarly subscriber. Wow, what a genius move. So essentially, it's here's a bunch of improvements, but you get three, I think max, and then it's like, "Okay, now you get upgrade."

**Albert Cheng** (00:24:51):
It's basically a reverse free trial but in real time while you're writing as opposed to a time-based one. So we adopted some patterns that are in the industry, but molded it to Grammarly's specific use case.

**Lenny Rachitsky** (00:25:04):
Right. I was going to ask, so it's not like a full trial, it's like a capped trial where you get a certain number of things and then you run out and then they get refreshed. I think once a day or something like that is what I found.

**Albert Cheng** (00:25:16):
Yeah, you got it.

**Lenny Rachitsky** (00:25:18):
Yeah. Grammarly is the best/most devious at their upsells. I'm always just like, "God damn it, I'm so close to seeing an improvement, I just have to upgrade." And it's right there, it's right there where my mouse is.

**Albert Cheng** (00:25:32):
Yeah, well, I'm not proud of being devious, but.

**Lenny Rachitsky** (00:25:33):
In really getting me to buy the thing. Good job. What was it? Kayla? Okay, nice job Kayla. It's very effective. I love that. Okay, so in terms of the free trial, I don't know, is there anything there of just, there's always this question of freemium, give things away and then there's pro account, there's like trial versus time. Some features are limited. I don't know, do you have for consumer subscription products like here's the way to go?

**Albert Cheng** (00:26:00):
Yeah, I think first of all, why do freemium subscription in the first place is a common question that I've joined all these companies that are freemium subscription. What do I like about it I guess? Well one, I think it ties really nicely to mission orientation of a lot of these companies. It's often like you want to spread the product as wide as possible because that's why the founders built the thing, right?

**Albert Cheng** (00:26:22):
You're trying to improve education with Duolingo or Grammarly or Chess.com, these are meant to be widespread products with a really wide value proposition that fits globally. And so obviously, the lowest friction to that is going to be a free product. So that alone is part of it. Another part of it is that a lot of these products primarily grow through word of mouth and especially if you can build network effects in the product, like Duolingo has a bunch of social features or with Grammarly, they have a bit of a B2C2B play as well.

**Albert Cheng** (00:26:54):
So you see Grammarly being used by teams and by companies and whatnot, and even if users are on the free plan, they still provide quite a lot of value in making sure that Grammarly can be purchased by a coworker or by a team member or whatever. So I think these things are usually why I lean toward make sure that the core value proposition that you're providing users is free and is permanently free and then you layer on a sampling or a taste of some of the premium features that are on top of it. That's usually the sweet spot that I've seen.

**Albert Cheng** (00:27:26):
As to the trials, reverse trials type of thing, I think it largely depends. I think if you have especially a B2B feature where you may have some lock-in, reverse trials can be super powerful. You just want to get people in there. You don't need to ask for their credit card because they're using your CRM or they're investing quite a lot of time in building out material and content. And so by the time that window drops, you actually feel, "Oh man, I probably should keep this and start paying." I think for a lot of consumer products it's a little bit harder for that to work. And so I've typically seen more just normal free trials be the norm.

**Lenny Rachitsky** (00:28:03):
Let me follow this thread of just consumer subscription products. I feel like this is the category that every indie developer dreams of building a product in because it's easy to build. Cool, I'll build an app, I add a paywall, and then they realize this is a lot harder than I thought. From a perspective of distribution and CAx and growth like that, is that the biggest missing piece that people don't get about building a successful consumer subscription product?

**Albert Cheng** (00:28:31):
Yeah, user retention is gold for consumer subscription companies. If you don't retain your users, then a lot of the onus is on getting them to pay on day one, that's super hard. Then you're dealing with totally different business models where you're paying for users, you're trying to aggressively upsell them before they hit any habitual usage patterns with your product.

**Albert Cheng** (00:28:53):
A lot of apps naturally do that because that's how they break the mold and get their first users to do it, but I don't know, I've been fortunate to join companies after that initial phase, but especially take Duolingo and Chess.com, these are organic word of mouth driven businesses and in both ways, they grew the market from a much smaller market and as opposed to it being a very competitive space where you're competing and taking market share from others and bidding for higher terms and stuff like that. So I don't know, there's something to that.

**Lenny Rachitsky** (00:29:26):
So what I'm hearing here is you need to find a way to grow through word of mouth for this to have any chance of success and also retention needs to be very high. Do you have a heuristic of what retention needs to be for you to have a chance building a successful consumer subscription business?

**Albert Cheng** (00:29:42):
I think consumer companies tend to track essentially two main types of user retention. There's more of the new user, one, D1, D7, et cetera. I think when you have your D one retention somewhere around the 30 or 40% mark, that's quite solid I think for a consumer app. If it's much lower than that, then sometimes I might question the intent of the user or the ability for that, you to I guess acquire just mathematically acquire enough users such that you can grow a big enough daily active user base.

**Lenny Rachitsky** (00:30:14):
That's surprisingly low.

**Albert Cheng** (00:30:16):
Yeah.

**Lenny Rachitsky** (00:30:17):
So it feels achievable in theory.

**Albert Cheng** (00:30:18):
It's achievable. It's achievable in theory, but there are so many options out there in the market and people are feeling a lot of app and product bloat.

**Lenny Rachitsky** (00:30:26):
And so just to be clear, you're saying 20 to 30% of people come back the next day?

**Albert Cheng** (00:30:30):
Yeah, 30 to 40.

**Lenny Rachitsky** (00:30:30):
30 to 40.

**Albert Cheng** (00:30:32):
40%. I think you're an okay place. I think even more importantly, and you mentioned Jorge to kick this off, but he wrote that very, very popular article about the growth model and how current user retention rate was the biggest thing for them. And I think especially if you have a product that has daily frequency, that's actually the retention that matters the most is that of your existing user base that has developed a habitual pattern, how sticky is your product? And it's that retention rate that really compounds and builds that daily habit.

**Albert Cheng** (00:31:04):
So over time, especially when companies mature a little bit, you actually focus most of your energy on the existing user retention mechanics. You find that that's a much, much bigger lever. One exception is that Grammarly was a different type of product and that you install it and you don't proactively open it every day. So that was interesting to me because I assumed that you should always just focus on existing user retention, but for a product like Grammarly, it's actually the activation installation aha moment that's really, really critical and will carry the user for a very, very long time.

**Lenny Rachitsky** (00:31:37):
That makes sense. Yeah, the stats would show someone's a daily active user because they're typing things and that's not an accurate step for Grammarly. The other interesting trend I've noticed across successful consumer subscription products is they always start very scrappy and very cost-efficient and spend efficient because I think it's because it takes them a long time to find something that's working and they're surviving on that margin of retention to growth cost essentially.

**Albert Cheng** (00:31:37):
Yeah, that's right.

**Lenny Rachitsky** (00:32:06):
Yeah, and the retention piece, that's such a good point. My newsletter is very much along these lines. It's just like how many people are joining every day, how many people are leaving? And it's a difficult treadmill to be on because people, they want to save money, they want to spend on Netflix and things like that. So as amazing as you are, people are always going to leave. So the trick is how do you find more people coming than going?

**Albert Cheng** (00:32:26):
Yeah, and I think just to take Chess.com example, I think probably 80% of our daily or weekly active users, I'll check the numbers, but something like that would be a current user or an existing user and then a new and a reactivated or resurrected user. Those are actually about similar size for a company of our sale. So even though there's a lot of attention on that new user experience, it's actually pretty interesting that the components of your active user base are actually not heavily weighed in the new user set after you mature to a certain degree.

**Lenny Rachitsky** (00:33:00):
Can you explain that a little bit more?

**Albert Cheng** (00:33:01):
Yes. So after some period of time, you stack up a lot of inactive users in your product and you also stack up sporadic users, people that may not have a daily habit, but they will use it once or twice a week or once or twice a month type of thing. And so eventually that math adds up where you have, let's say hundreds of millions of dormant users that are coming back and it's actually worth spending some time making sure that that resurrected, for lack of a better word, experience inside the product is really excellent and that you find novel ways to try to bring them back.

**Albert Cheng** (00:33:38):
Duolingo as an example, they did a good job of using social notifications. And so if people would use contact sync or something, you might get a push notification that one of your best friends just started using Duolingo and that might encourage you to come back and resurrect into the product. And whether you resurrected in the product, it might be the case that your proficiency of the language you were learning, you were learning French three years ago, but now you for forgot most of it. And so when you open the app again, it encourages you to essentially replace yourself, do another placement test and put you in the right spot. And so some of these types of mechanics for a more mature company can lead to pretty good ROI guess is what I'm trying to say.

**Lenny Rachitsky** (00:34:19):
Got it. Essentially, so many people have already tried in the past that to grow, you need to resurrect people that have been there. And so thinking through, it's almost like a user experience for resurrected users.

**Albert Cheng** (00:34:34):
Exactly.

**Lenny Rachitsky** (00:34:35):
Okay. Let's zoom out a little bit. You've worked at three of the most successful consumer subscription products in the world. What is the difference between how these three operate? I think there's many ways to be successful. It feels like these companies are very different. What's the gist of each of these, how they operate?

**Albert Cheng** (00:34:53):
Well, first of all, there's obviously a lot of similarities, but I'll just focus my answer on the differences. So I think Duolingo, what struck me most working there is they're very particular, they have an approach of product development that is infused across everyone in the company. And they actually wrote a playbook about this. It's called the Green Machine if you look it up. That was one of my most successful tweets ever really.

**Lenny Rachitsky** (00:35:15):
I just tweeted something about Duolingo just released their playbook and I screenshotted the owl's butt and screened like a page and it was like 5,000 likes.

**Albert Cheng** (00:35:26):
That's hilarious.

**Lenny Rachitsky** (00:35:26):
Yeah. So yeah, keep going. Sorry.

**Albert Cheng** (00:35:29):
But yeah, the ethos of the company. They hire a lot of intelligent, energetic people out of college basically, and they give them a lot of amazing experimentation, tooling, and they care a lot about the clock speed of the company. So it's a lot of creativity, a lot of ideation.

**Albert Cheng** (00:35:46):
The product experience of dual legal actually changes multiple times per day for each user, which is pretty shocking. And so I'd never worked in a place like that before, but it really struck me about how consistently the company operated and they had specs and processes for doing each of those steps in their product development cycle and they were really, really tight about it.

**Lenny Rachitsky** (00:36:08):
Okay, so that's still lingo.

**Albert Cheng** (00:36:09):
Yeah, that's still lingo. Grammarly. This is an interesting company because they started as a paid product oriented at students. Then they expanded into more of a freemium model tailored to everyone gradually focusing more on the professional base. And then as they accumulate a lot more professionals, they realize, "Hey, there's patterns." We're seeing that a bunch of marketing teams or a bunch of sales teams or a bunch of customer support teams or whatever, particular functions within particular companies were really adopting Grammarly at scale.

**Albert Cheng** (00:36:41):
And so they were able to then layer on much more of a managed enterprisey motion. And while I was there, I was focused on the consumer self-serve motion, but they weren't siloed. They were intermixed with each other. And so a big part of my job was not just to grow the self-serve revenue and self-serve active users, but it was also how do you uncover the right teams, the right functions, the right companies for demand gen and sales to go reach out to?

**Albert Cheng** (00:37:12):
So that was a very interesting, it's a product-led sales work, and it's really fascinating thing for me to learn. And then on top of that, with all the transformation going on with generative AI, and even recently with them acquiring CODA and Superhuman and becoming more of a productivity suite, the company is just evolving pretty rapidly. It's a really exciting thing for me to be a part of and to see from the sidelines, but that just made it at its core of a different growth job than Duolingo for sure.

**Lenny Rachitsky** (00:37:40):
Essentially a B2B business versus a very consumer business?

**Albert Cheng** (00:37:43):
Yeah, and a lot more meaningful strategic decisions as well.

**Lenny Rachitsky** (00:37:44):
Mm-hmm.

**Albert Cheng** (00:37:47):
And then the core product team also, I'm used to in growth, laying out the entire user journey that a user go through acquisition, activation, engagement, so on and so forth. And typically, growth teams, if they're well-resourced, they can do enough to move each one of these various levers. And it's just a matter of the sequencing of them and what you want to prioritize first. But Grammarly was unique in that the core product experience itself was what drove repeated activity.

**Albert Cheng** (00:38:18):
It's that I previously mentioned that current user retention thing, what most drives that is the frequency and the quality of the suggestions that you get every day. And so it was an interesting learning in that I staffed up a growth team, tried to work on this metric, and then I realized actually I'm just getting in the way. This is really a thing that the core product team most influences. Let me have a conversation with the core product leader and then shift that over to them. So yeah, just a super interesting experience.

**Lenny Rachitsky** (00:38:45):
And then Chess.com.

**Albert Cheng** (00:38:47):
The thing that's most unique about chess.com is that they're super fanatical about chess.

**Lenny Rachitsky** (00:38:54):
Makes sense.

**Albert Cheng** (00:38:55):
Crazy. You shouldn't be surprised. Obviously the name of the company is like this, but they've always hired people from around the world. The company's always been globally remote. They just hire people that love chess. They play all day, they watch the streams. Our Slack is always blowing up with people's chess moves and games and whatnot. I think I want to say this a little bit delicately, like Duolingo, even though the product they're providing is around language learning, I think the original ethos of how to start the company was really around motivation.

**Albert Cheng** (00:39:29):
The hardest thing to its habits, it's how do you build that daily habit? And I actually in many ways see language learning as their first vehicle. And what they have a superpower in is that, again, the motivation, the habits, et cetera. So that's Duolingo, and Grammarly actually similarly. People know them for the spelling and grammar corrections, but what's really unique about them is they're integrated across tons and tons and tons of applications.

**Albert Cheng** (00:39:55):
There's not many products that work like that, that's really unique. And so now if you hear Shishir, their new CEO talk about the AI super highway and all that type of stuff, they can now use that technology to provide a lot more than just grammar writing. And so my point is just that Chess is about chess 100%. It's in the ethos. People are crazy passionate. That just means we're always dogfooding the product. There's just an amazing energy in the company to just use the product all the time, come up with ideas, and I love that environment. I think that's fun for me.

**Lenny Rachitsky** (00:40:28):
That is so cool. What I love about what you're saying is there's no right or wrong answer. All of these companies are killing it. I think Duolingo is worth like $10 billion, something like that, and keeps growing. I'll look it up in a second. And Grammarly is worth a ton, and then Chess.com is doing super well. So I think that's a really interesting takeaway here is you can succeed in a lot of different ways.

**Albert Cheng** (00:40:51):
Yeah.

**Lenny Rachitsky** (00:40:52):
What's really cool about Duolingo, I was just thinking as you were talking, is yeah, it's just interesting that this very structured, methodical way of building is working so well because you could listen to that and be like, "Oh, I don't want to work." This is rigid way. But the fact that it is killing, it tells us this actually works really well. If you find something that works, lead into it.

**Albert Cheng** (00:41:11):
Yeah, that's right. Yeah, the structure is rigid, but the ideas are the farthest away from rigid as possible. You have seen their, I don't know, Superbowl commercials, they're memes, gamification, tactics. It's a super fun creative environment. So rigid is the farthest possible word to use, but what I just mean is they're consistent. They have for everything, and their product reviews are 10 or 15 minutes. It's just people go in and out. So it's just this kind of a surreal environment about how rapidly and consistently they work.

**Lenny Rachitsky** (00:41:42):
Awesome. They're worth $12 billion, and they were much higher actually, not too long ago. They're coming down a little bit. So speaking of Duolingo, when people think Duolingo, they think of the brand and the owl and the success they had on TikTok and things like that. I'm curious to get your take on as a very growth-oriented person watching that work and your take on growth, experimentation data versus marketing, viral TikTok videos, mascots, things like that.

**Albert Cheng** (00:42:09):
Yeah, I used to think it was versus, but now I realize that they combine really well. It could be rocket fuel for your growth. Yeah, being a product person. I joined a lot of these companies literally on the home screen on my phone, and I like using them. And I consider myself someone that's not easily swayed by ads or TV commercials telling me what to buy.

**Albert Cheng** (00:42:28):
So I always had an element of skepticism on the marketing side for much of my career. But then, yeah, you join a place like Duolingo and you see how Duo the owl has developed a personality through the push notifications and the product experience, and then seeing the marketing team leverage that personality in their TikTok and in their YouTube and all throughout social media and just feed into those memes. And then we would track back in the product experience, how did you hear about us?

**Albert Cheng** (00:42:58):
And put all those channels in there. And some days, it would be like, holy, it's bringing in 20, 30% of our new users and any given day. So those two things really go hand in hand, and that feeling has only been reinforced by Chess.com over the last five years. The first 15-ish years of this company was really under the radar. 800 million people play chess around the world, but most of that is over the board.

**Albert Cheng** (00:43:25):
Until recently, there wasn't actually that much online, but five years ago, everything changed. You had the pandemic, you had Queen's Gambit, you had a lot of YouTube and Twitch streamers, you had a bunch of kids playing it in school, et cetera. And so it's really the combination of those two things that make it take off. And it's like the growth experimentation is more the slow and steady or fast and steady, I should say, approach where you're just continually iterating, you're making the product experience better, but then every so often, there's a big wave that comes in. You can quadruple your registrations overnight and you'd be a fool not to take advantage of that.

**Lenny Rachitsky** (00:44:03):
I was actually speaking at Chess.com and playing chess. I was at a coffee shop this weekend. There's a family, a dad and mom and a daughter ordering, and the dad's sitting at the table and he's just on his phone, just opened up Chess.com secretly and just plain while he is waiting. Oh man.

**Albert Cheng** (00:44:18):
I will not admit or deny that I've done that before.

**Lenny Rachitsky** (00:44:22):
But if I can think of anything more wholesome, I can't. That's an amazing thing to be doing while you're just sitting.

**Albert Cheng** (00:44:32):
My 4-year-old can actually set up the pieces, which is pretty great. So he enjoys the game quite a bit.

**Lenny Rachitsky** (00:44:36):
Oh man, this 4-year-old already a pianist, playing chess.

**Albert Cheng** (00:44:39):
That's right.

**Lenny Rachitsky** (00:44:40):
**Albert Cheng** (00:46:13):
Yeah, I'll tackle them in sequence. I'll start with the chess one just because I have maybe a slightly unique take on that one. So chess and AI, they've been intertwined for almost a century. Some of the early computing pioneers, they just figured, "Yeah, chess is an interesting game. We can test machine intelligence and write some algorithms or not." And then fast-forward to 1997, and you had IBM, they had their DeepBlue application who actually beat the world champion back then, which was Garry Kasparov.

**Albert Cheng** (00:46:43):
And that was a huge moment of shock and reckoning of like, "Oh man, is AI going to take over? Humans are, we're going to have jobs and all this stuff." And this is 30 years ago, and thankfully we're all still here and more people are playing chess than ever. And so the game of chess and chess.com specifically have learned how to augment, I guess the human playing experience with the power of chess engines, which are definitely a powerful form of AI. It's not LLMs to be clear, but there's engines like Stockfish these days that are just dramatically better than the top grand masters in the world.

**Lenny Rachitsky** (00:47:22):
Is that where we're at? I remember when it beat humans and now it's just dramatically better.

**Albert Cheng** (00:47:26):
It's dramatically better.

**Lenny Rachitsky** (00:47:28):
Wow.

**Albert Cheng** (00:47:28):
Yeah, I think there's a rating system that compares relative skill level and an average chess player somewhere like a thousand, maybe 1,500 on the high end, a top grandmaster like Magnus Carlsen, it's like a 2,800 and then Stockfish and similar engines are like 3,600.

**Lenny Rachitsky** (00:47:45):
Wow.

**Albert Cheng** (00:47:45):
And so to put that in comparison, yeah.

**Lenny Rachitsky** (00:47:48):
At least it's not 10,000 or a million. I don't even know if that's possible.

**Albert Cheng** (00:47:51):
No, it's not 10,000. But it's similar to if the chess engine was playing without a major piece like a rook or something, they would still be competitive against the best players.

**Lenny Rachitsky** (00:47:59):
And this is the Elo score? Is that the term?

**Albert Cheng** (00:48:00):
Yeah, the Elo score, Elo rating.

**Lenny Rachitsky** (00:48:01):
Magnus is what you said about 2,800, and then the Stockfish is would you say 3,600?

**Albert Cheng** (00:48:04):
Yeah, and really it's because computing power is so amazing and there's so many techniques for how to do deep evaluation on specific chess lines. They can calculate tens of millions per second. So it's not realistic for a human to compete against that. But yet, watching some of these chess engines played has opened up a lot of creativity, new strategies, new lines, new appreciation for the game. And our chess.com approach is that we can bring this technology for every user.

**Albert Cheng** (00:48:36):
Even people that have never moved a piece before. I talked earlier about that game review product, that's exactly what this does. So behind the scenes, we're running chess engines to basically spit out evaluations for every move that you make. And then we translate that and make that approachable to the user using their native language and plain approachable style, and even with audio and things like that as well. And that part of it, the personality, the speech back to the user, that part is LLMs.

**Albert Cheng** (00:49:07):
And so I guess my point is that, again, chess and AI have been intertwined forever, but for us, what's most important is that we keep the customer at the North Star of it. We're not just applying LLMs just because the new hot thing, you've got to apply the right technology for the right feature to provide value to the user. And so we try not to ever lose sight of that and let hype get us too carried away.

**Lenny Rachitsky** (00:49:31):
It's just really surprising. I think people would not have expected AI and cannot beat every human alive ever. And chess is at an all-time high. People want to keep playing and are playing more and more than ever played, not unexpected.

**Albert Cheng** (00:49:46):
Interestingly, LLMs themselves are quite bad at playing chess. They hallucinate moves, they look at patterns. They're very good at pattern recognition, but not so good at going super, super, super deep on a specific chest thing. And if you've even tried to create or look at chessboard images on ChatGPT, a lot of them have the wrong number of squares. They're not set up properly, and so I don't want to be too dismissive.

**Albert Cheng** (00:50:09):
I'm sure it's going to get much stronger at reasoning. And actually, Google recently sponsored a tournament where all the top LLMs played a tournament against each other. So that was pretty fun to watch. They're improving, but chess is specifically a game that having a trained deep, deep computing engine is just going to be much, much, much more powerful than LLMs.

**Lenny Rachitsky** (00:50:30):
And not to go down this track too far, but AlphaZero famous for beating the Top Go player. Was that trained specifically for Go? Obviously not in LLM, but that was a Go specific model.

**Albert Cheng** (00:50:42):
Yeah. My understanding is that the one, that documentary is incredible, by the way. I don't know if you've watched AlphaGo, it's amazing how they took something so technically deep and made it so emotional and human. But I think that's the crux of how we feel, I guess, about AIs and the products that we build, actually. But to your point, my understanding is that the way AlphaZero is primarily trained is that it just plays a bunch of games against itself. And so through the neural network, it just gets smarter every time. And because it can have that repetition times a billion or a trillion, I don't know exactly what number, but it's going to get pretty damn good.

**Lenny Rachitsky** (00:51:19):
Okay. Let's go back on track to where we were going. So this was how AI is impacting chess.com. How is AI changing just the work of a growth person?

**Albert Cheng** (00:51:30):
I like to describe growth as the job is to connect users to the value of your product. And in order to do that, what I like to do is think about that user journey again, and essentially, staff teams that are oriented around each element of that user journey. And those teams have specific metric goals, they have roadmaps, et cetera. And then they go run against them.

**Albert Cheng** (00:51:52):
So that's how it's structured. AI, I think can be applied to speed up some elements of that essentially experiment cycle that you get through. So one example is in product discovery. As opposed to core product, which tends to have longer timeframes, and you might do thorough user research or market research. It's more foundational, more for first principles, et cetera. Growth is a little bit less like that.

**Albert Cheng** (00:52:18):
It's like you're running a lot of experiments and the output of any given experiment is the input to your next idea. And so historically, I don't even mean historically, but just a few months ago, we were operating in a, that's history, I suppose, but there would be a lot of manual writing of these analysis docs. You'd have to read them, you'd have to understand what insight you want to grab from them and then write another spec to translate that idea. That's still happening to some degree, but I think that's a spot where even tools like ChatGPT are super helpful.

**Albert Cheng** (00:52:56):
You can just plug in like an analysis that another person wrote and just have it summarized for you and give you advice on ideas to go try. And so that ideation, that research cycle was much, much faster. I talked a little bit about prototyping also just becoming much, much faster than before. We have not yet gotten to the point where product managers themselves are actually shipping the code into production, but it's dramatically shortened the amount of time it takes to conceive of especially a bolder idea that you might have.

**Albert Cheng** (00:53:27):
And so when I talked earlier about explore and exploit, a lot of the explore was harder to do, but now it's a little bit easier to do. You can take a broader concept and visualize it, and when you can visualize it, send it around the team, get people to click around it, that makes a world of difference. So those are just a couple examples that come to mind.

**Lenny Rachitsky** (00:53:47):
Awesome. I want to go back to this phrase right at the beginning of this answer that you shared that I think is really helpful that you see growth as simply your job is to connect users to the value of your product.

**Albert Cheng** (00:53:58):
Yeah.

**Lenny Rachitsky** (00:53:59):
Can you speak more to that? Because I think that's such a nice way clarifying what is growth's role?

**Albert Cheng** (00:54:03):
Yeah, it resonates deeply with me because I feel like growth sometimes gets this reputation I guess that it's just pure metrics hacking, like we're cold people that just are trying to move a particular metric up and we're going to do whatever it can to throw walls and pay walls and add friction in all these spots. And even though that could theoretically work at a micro level on a specific feature or a specific metric, I think what's most healthy for a company, and I want to work at durable companies is to think about the user holistically.

**Albert Cheng** (00:54:42):
And when you take that framing of connecting users to the value of your product, that value can change for a user over time, and that also lines up really nicely to the journey. What someone that's not even a user yet needs to understand about the value proposition is super different than what a habitual user of three plus years might need. And so the teams working on them should think from that perspective and then from there, then ladder into specific problems to solve hypotheses, et cetera.

**Lenny Rachitsky** (00:55:15):
Following that thread a little bit more, people listening to this are imagining, "How do I get better at experimentation? How do I run more experiments? How do we do this better?" What are two or three tips and best practices that you think people need to hear maybe are not totally aware of when they think about getting better at experimentation on our teams?

**Albert Cheng** (00:55:36):
I think the first thing is just start somewhere. I just read this Atlassian state of product report and it was like 40% of product teams basically don't run experimentation at all. And there may be some good reasons for it. It could be philosophical or maybe you're more B2B oriented or whatever. So I get it, but I think for a lot of, especially if you work on a consumer product that has some degree of scale, some degree of frequency with your product, you can collect enough data.

**Albert Cheng** (00:56:05):
And also I have found I can pattern match all day long. I've worked a lot of companies, right? But I'm wrong all the time. And I think consumer behavior can be very fickle and especially when you work at a company, you become a power user naturally. So sometimes you may forget what the actual user experience is for a brand new user, and so you leave a lot of opportunities on the table if you don't even try to experiment.

**Albert Cheng** (00:56:27):
So I just encourage taking that first step, just run an A/B test, find a third-party tool or something that you can integrate quickly or even just work with your engineers to spin something up. Just get in the practice of crawl then walk then run type of thing.

**Lenny Rachitsky** (00:56:40):
Do you have a favorite tool, by the way? Just to throw out? Is there a go-to tool for you?

**Albert Cheng** (00:56:44):
We used Statsig at Grammarly and I saw that they recently got acquired, so that was exciting news. Duolingo and Chess.com both have an in-house experimentation approach.

**Lenny Rachitsky** (00:56:54):
Sweet.

**Albert Cheng** (00:56:55):
Pros and cons to either. Obviously Duolingo is an experimentation machine, and so it's been a huge accelerant to have our own thing specifically tailored to be excellent at that. But no, I typically don't encourage companies to build experimentation in-house from day one. At a certain scale it can make sense. And some of these companies, they were started 15 years ago when these tools weren't out. So it was just something they had to do.

**Lenny Rachitsky** (00:57:22):
Something that you mentioned to me at Chess.com, your goal is to run a thousand experiments a year. You said you were at 250. Talk about just that as a North Star.

**Albert Cheng** (00:57:32):
Yeah, so part of having team members that are fanatical about Chess is that the company can get pretty far just building for themselves, building for the community, and not actually being very experimentation and data oriented. The problem with that is that you can have relatively lumpy growth. And so part of the excitement of me joining the company was to help smooth that out and bring in that experimentation mindset.

**Albert Cheng** (00:57:56):
So prior to 2023, the company practically didn't experiment at all. Last year they did about 50, this year they're on pace for about 250. And then next year we have that ambitious target of a thousand. Did I make it up? Yes, absolutely, I made it up, but it's still a target and a thing for the teams to think about and a thousand experiments by itself. If you just did that but you didn't learn, you didn't make an impact, that's kind of a waste of time.

**Albert Cheng** (00:58:25):
The whole point of setting a goal is that you can have conversations about what would need to be true to actually hit that goal, and so that leads to insights. Actually we need not just product management or engineering to be running these experiments. We can experiment with lifecycle marketing, changing copy of push notifications and emails. We can experiment with app store screenshots and keywords and stuff like that. We have all sorts of content marketing teams, et cetera. We could have engineering enable no code for specific screens.

**Albert Cheng** (00:58:58):
Think about our home screen or our pricing screen where we might want to do a lot of just tests that are configurable without engineering support. We might want to just track our progress and look at it from time to time and make sure that we have the right observability around this. So anyway, that's the stuff that really matters as opposed to the hitting that goal itself. So don't tell the team, but I don't actually care that much if we actually hit a thousand, but I think if we get pretty close and we accomplish some of these things, we'll be in really good shape.

**Lenny Rachitsky** (00:59:27):
Okay, we'll make sure none of them watch this. I think chess.com is in, this is just such a cool example of a culture shifting dramatically from zero experiments to sounds like two years later, a thousand, which is three a day. There's many teams running experiments in parallel, but that's a lot. What has helped you most shift that culture? Is it just the CEO being like, "This is the way we're going to go." What have you learned about helping shift to culture from No, we're not doing experiments to a thousand experiments a year.

**Albert Cheng** (00:59:58):
Yeah, definitely a lot of credit to the CEO and co-founders like Erik and Danny, they're amazing. It's not their intuitive way of thinking about growing companies, but their mental flexibility and encouragement to evolve and add this as a tool for the company has been awesome, and they've been on the front lines preaching product-led growth and experimentation just as much as I have.

**Albert Cheng** (01:00:20):
So I'm glad that you brought that up because I think that is critically important for me, joining a company to not be at odds with the co-founders and the existing approach of the company. I think that's absolutely, absolutely critical. I think I started this podcast with the example of the game review and the positivity and how that was shared. I think those types of things are really what motivate people. They need to see this working in practice.

**Lenny Rachitsky** (01:00:45):
Wins.

**Albert Cheng** (01:00:46):
Yeah, you need wins, you got to celebrate them. People feel good about the learning. It's applied across the board. Who's not going to be energized by that, I think, right? So you can't just set goals in a vacuum and create it from top, right? People have to see it working and when it works, the metrics move and you learn faster and you ship faster, and that's a great environment to be part of.

**Lenny Rachitsky** (01:01:07):
What was the first experiment you guys ran? Do you remember?

**Albert Cheng** (01:01:10):
I don't know, before my time actually.

**Lenny Rachitsky** (01:01:13):
Okay. Okay. Got it. So they're already going down this track before they brought you in?

**Albert Cheng** (01:01:17):
They had run some.

**Lenny Rachitsky** (01:01:19):
Okay, sweet. Are there any other key lessons that you think people need to know to be successful running experiments at scale?

**Albert Cheng** (01:01:30):
The system matters just as much as any given experiment, probably even more, right? I think starting with a growth model, so you have an understanding of how your company grows in the first place and which channels you're going to leverage is critical. You need to make sure that you are instrumenting your product in and out. Otherwise, you're going to run experiments and have wonky results.

**Albert Cheng** (01:01:53):
I won't name which company, but I was part of a company that had an in-house experimentation tool. It's about three months into the company, we're running some experiments and we realized that user retention was actually configured backwards. So all positive results were negative results.

**Lenny Rachitsky** (01:02:09):
Geez.

**Albert Cheng** (01:02:10):
So that was kind of embarrassing and that will never happen again.

**Lenny Rachitsky** (01:02:13):
You just go through and undo all those experiments and just drive up retention.

**Albert Cheng** (01:02:17):
It's kind of weird. We're seeing people use the features a lot more. Why is user retention going negative? So I have plenty of horror stories around that type of stuff, but yeah.

**Lenny Rachitsky** (01:02:25):
Oh my God. On the flip side of horror stories, you've shared a bunch of cool examples of experiment wins. Is there another that comes to mind of one you're really proud of or that was really trajectory changing either at Duolingo or Grammarly or Chess?

**Albert Cheng** (01:02:38):
So I already shared one of Chess.com and one of Grammarly. I could talk a bit about Duolingo as well. Duolingo and you had Jackson on the podcast, right? Where you talked about the streaks.

**Lenny Rachitsky** (01:02:38):
Yes, talked about the streaks.

**Albert Cheng** (01:02:52):
So I also don't want to steal his thunder because I was going to think about that, but the amount of learning through commitment and putting streaks on a calendar and just getting people started as opposed to achieving some large milestone, that was huge. I think we did something interesting. We spun up a virality team and virality is this really amorphous thing to me.

**Albert Cheng** (01:03:16):
I think it's really hard to generate virality in your product, but Duolingo is a product that is shared quite a bit. And so we invested actually in some time to essentially add screenshot tracking for a brief period of time in the app just so we could find out the hotspots of where users were doing screenshots. And you see this in other apps too, it's not necessarily some horrible thing, but we did this for some period of time and we were able to basically articulate and say, "Okay, streak milestones is the obvious one."

**Albert Cheng** (01:03:46):
Really funny challenges that you get in the Duolingo experience is also super highly shared. Advancing in the top three of a leaderboard is another thing. Anyway, so you can find these different moments where that's the case. And then we staffed those moments with illustrators and animators and created these really delightful experiences around them, and that worked amazingly well.

**Albert Cheng** (01:04:06):
So as opposed to going against I guess human intuition and trying to get them to share stuff that they otherwise wouldn't on the margins want to share, lean into it more, actually grab the moments where users are already organically screenshotting and make those much, much, much better. And you can 5X or 10X and drive a lot of growth that way too. So that's not so much an experiment, it's more a core product thing, but it just resonated with me that that was interesting.

**Lenny Rachitsky** (01:04:33):
Well, it connects to your explore and exploit methodology. Just find or explore where things are happening and then try to exploit in a nice positive way.

**Albert Cheng** (01:04:41):
You got it.

**Lenny Rachitsky** (01:04:42):
Speaking of that, you mentioned this with Duolingo is just very good at habit formation and motivation behavior. It feels like chess is good at this too. You've worked at both these companies. What have you learned about how to motivate people? How to create habits?

**Albert Cheng** (01:04:57):
Again, Duolingo would not have started without this insight from day one. They aim to focus on motivation and build a lot of these tactics. Jorge actually had this model of gamification patterns having essentially three pillars to it. You have the core loop, you have the metagame, and then you have the profile. And so we actually thought about it that way too, where your core loop is your lesson that you go through. You do a lesson, you get some rewards, you extend your streak, and then the next day you get a push notification.

**Albert Cheng** (01:05:29):
It's the core loop of the product and making that really tight is super important because people need a habit to stick to. Then you need a metagame, which for Duolingo is the path, but it's also the leaderboard achievements. It's long-term things that you're going to strive to such that you have long-term, I guess, motivation to continue doing the thing. And then the profile is also critical because you build up a profile over time.

**Albert Cheng** (01:05:51):
It's a reflection of your investment inside the product experience. And so when you nail those three things, you can end up with a long-term learning journey that can be quite successful. And then to flip over to the Chess.com side, what we see is that over 75% of our new users, they classify themselves as like, "I'm completely new to chess." Or, "I'm a beginner." And unfortunately, if you're new to chess and you're a beginner, you're not going to have that fun of a time playing live games, and we see this in the data. It's like less than a third of those users actually win their first game. And when you lose a game, user retention is 10% worse than when you win a game.

**Lenny Rachitsky** (01:06:29):
That's not so bad, but at scale, that's bad.

**Albert Cheng** (01:06:31):
Yeah, and it could be worse. That's true. And so typically what a lot of mobile games will do is they'll just create a super simplified version of the game. It's harder for us to do at chess, and so without changing the rules of that, I think that's, I don't know, it's just very eye-opening to me when you're trying to learn something, whether that be language learning or chess or whatever, usually those first steps are fraught with a lot of self-doubt and reinforcement that you're not good at the thing. And so it pays to be very intentional to craft experiences that guide the user around that.

**Lenny Rachitsky** (01:07:09):
Well, I can't help but ask, is there anything that helped that along?

**Albert Cheng** (01:07:12):
Yeah, so something we're experimenting right now is just like purely if you say that you're new to chess, we're going to craft a more delightful learn how to play experience as opposed to dropping into a live game, that's an example. Another is hiding your ratings for the first five times such that you're not seeing your rating plummet. So there's a lot of tips and tricks you can do.

**Lenny Rachitsky** (01:07:29):
I'm just imagining a little guide that's like, "Here's how you win."

**Albert Cheng** (01:07:32):
Yeah, or play against a coach, play against a friend, play against a bot. There's a bunch of different avenues you could take.

**Lenny Rachitsky** (01:07:38):
Well, what I'd love is play against someone real and here's where you should move. Just like, "Hey, here's we're going to help you win."

**Albert Cheng** (01:07:45):
Like a hint in real-time?

**Lenny Rachitsky** (01:07:46):
Yeah, yeah, yeah.

**Albert Cheng** (01:07:47):
I don't want to be playing with you then.

**Lenny Rachitsky** (01:07:50):
Okay. Let me ask you a couple more questions. One is just zooming out a little bit, what's the most counterintuitive lesson you've learned about building products or building teams across the many companies you've worked at?

**Albert Cheng** (01:08:03):
Yeah, I've talked a lot about products. So maybe I'll flip to the team side for a bit. I think the standard way to hire and build a team is you fill out a JD, it's got a whole bunch of different characteristics that you're looking for. You typically will find a short list of companies that are kind of similar to yours, and then you try to hire for that, right? I think that's the typical default path that a lot of companies take.

**Albert Cheng** (01:08:27):
And I was really struck by my experience working at some smaller startups or take Duolingo as an example, where over and over and over, I saw some of the highest performers just being people that had very high agency, had that clock speed, had that energy. Yes, they cared about the mission, but they didn't necessarily need to have deep experience on that matter. And in fact, sometimes that experience could be a crutch in certain ways, especially in this world where the grounds are shifting so fast with AI, a lot of your learned habits actually need to be intentionally discarded.

**Albert Cheng** (01:09:01):
You need to have a beginner's mind on this type of stuff. So I think this is more true than ever, looking for people that respond and move quickly and think just faster and move faster. I think the fastest speed of learning, those types of companies are the ones that I want to bet on. I think those will end up surviving and thriving.

**Lenny Rachitsky** (01:09:25):
So just to double click on this idea of high agency is very trending these days of just higher high agency people. To unpack that a little bit, you mentioned a few of these traits, so let's just help people see what you see. So one is clock speed, just they think fast, they move fast, they learn fast. What else? What else do you look for that helps you see that there are high agency people?

**Albert Cheng** (01:09:49):
Yeah, a lot of it actually happens outside of the interview process interestingly. So a lot of it is the types of questions they asked, "Have they actually tried your product and gone deep into it?" A lot of it is, it's the references, it's the communication that they have to even set up your interview, it's the energy they bring into the conversation.

**Albert Cheng** (01:10:09):
You can actually pick up a lot of soft signals on some of these traits over time. You've got to pick up on some of these patterns. I don't know that I'm perfect at it, but I've learned to balance those things quite a bit more than I did in the past when I would just purely read from my questions and my rubric and not care about anything else.

**Lenny Rachitsky** (01:10:27):
Yeah, there's like a vibes component to it. This is also support for the work trial way of interviewing versus just a talk interview where you have them actually work with you for a week or whatever.

**Albert Cheng** (01:10:36):
That's a great point.

**Lenny Rachitsky** (01:10:38):
Okay. One other question I wanted to ask you. You've worked at a bunch of different sizes of companies from startup to Grammarly, I don't know, you call it a big company, bigger company. Duolingo, I don't know how big is Duolingo?

**Albert Cheng** (01:10:50):
There are about a thousand people.

**Lenny Rachitsky** (01:10:50):
Okay, cool.

**Albert Cheng** (01:10:53):
But I worked at Google too to start my career.

**Lenny Rachitsky** (01:10:55):
Oh, right, okay. What have you learned about just the size of company that makes you happy? What have you learned about just helping other people that you talk to decide what size of company is good for them?

**Albert Cheng** (01:11:05):
I definitely believe that everyone has a company stage that they shine best at. I've personally gone through this journey of big tech to tiny, tiny, tiny startup, then landed in the middle, which I consider my own goal lock zone. I talked earlier about what actually gives me personally a lot of energy is seeing across a company's efforts, but also the company being small enough that I can get into the details, I can work with the specific teams.

**Albert Cheng** (01:11:33):
I can read experiment results, I can look at the pixels. And so I find that the balance of those two things tends to fit best with medium-sized companies, but that's me, right? I think at big companies like a Google, you're dealing with immense scale, which is interesting by itself. You learn a lot of best practices from your peers. They have all the tools and functions that you would possibly want to go learn from, but they can tend to move slower and it's harder to ship things and get them out the door, which eventually drove me nuts a little bit.

**Albert Cheng** (01:12:06):
On the flip end of the spectrum, these tiny startups, they move incredibly fast, but I grew all my gray hair from those tiny startups because no one knows about your company, and so you're recruiting people one by one. You're trying to get users one by one. So yeah, you can learn fast and ship a lot of things, but if you're trying to make a big impact on the world, it can be actually pretty grueling to do so at really, really, really small startups.

**Albert Cheng** (01:12:29):
Now, some of them do hyperscale and make it out, and obviously, I am not one to trash that because the path that I tried for quite a while. But for me, I really like the zone where I can contribute at scale, but also execute at a pace that's more on the daily and weekly scale as opposed to monthly and quarterly.

**Lenny Rachitsky** (01:12:50):
And when you say medium, what size of company is that roughly?

**Albert Cheng** (01:12:53):
Yeah, so these companies that we've talked about in the podcast are about 500 to a thousand people. Typically, these companies who have been around let's say 10 to 20 years. They're durable, ideally profitable, have a good leadership team, but there's still a lot of dimensions to go figure out. A lot of them are in key inflection points, so they're certainly not stagnant. You need to find a place that's dynamic too.

**Lenny Rachitsky** (01:13:17):
Interesting, 10 to 20 years old, I don't know if that's a, not many people would feel like that's where I want to be. I love that you found a number of companies like that that you enjoyed working at. The last question, and this is going to be taking us to a recurring segment on the podcast that I call Failed Corner.

**Lenny Rachitsky** (01:13:35):
People hear all these stories of all these experiments and all these companies that worked at, they're all killing it up into the right. In reality, you've touched on this, a lot of things don't work out great. So can you share a story when something went wrong, when you failed and what that taught you?

**Albert Cheng** (01:13:50):
First of all, in the growth world, you're failing all the time. So I'm not going to pick a specific growth story because those don't actually hit my ego too much. But earlier in my career I did a lot of core product work. I worked for this startup called Chariot. I don't know if you ever lived in San Francisco, but.

**Lenny Rachitsky** (01:14:05):
Yes, it was like the bus super thing.

**Albert Cheng** (01:14:06):
The blue commuter shuttles, like 15-person shuttles, they would essentially drive from various neighborhoods into downtown San Francisco. It's a commuting use case across between the public bus system and an Uber and Lyft. So I was there for some time. I led product there and the core service was really loved by its users. It was reliable and fast and affordable enough, but we got pretty interested in this idea that maybe we can improve utilization, maybe we can make the service a little bit more innovative if we offer dynamic routes more similar to Uber and Lyft.

**Albert Cheng** (01:14:46):
How could the drivers are driving these fixed routes? But if they have spare time, they can go out of their way, go pick up somebody at their house or something and keep going. So we tried this, we called the chair direct, really interesting attempt, but I learned a lot of lessons there because ultimately it didn't work out. One lesson is like this was kind of a solution searching for a problem. You never just purely want to chase A, it wouldn't it be nice if we did this as opposed to this is our user and this is the problem that we're solving, this is why it's going to delight them, et cetera, that's one.

**Albert Cheng** (01:15:20):
Second is you got to consider, especially in these more marketplace type businesses, there's more than just one end user and we focus so much of our attention on the writer app without realizing, oh yeah, the drivers are carrying a lot of the brunt of this experience and our operations team is as well. And so when the drivers are confused or disgruntled, that can lead to a challenging overall experience for the product. So that's definitely another one.

**Albert Cheng** (01:15:50):
And the third one is we did a lot of actually PR, prior to the service going out just to get the word out. And PR has its time in place, but I think doing it before you have validation that customers definitely want, the thing is quite risky and it can lead to a lot of sun cost once you get it out because you need to see it through, you want to see it succeed. So yeah, this is a decade ago, honestly, I had a great time at that company, but I still remember that vividly because it contained three or more key lessons that carried forward as I have built many products since then.

**Lenny Rachitsky** (01:16:28):
Yeah, it feels like you went to the complete other end run experiments of everything before you tell anyone about it.

**Albert Cheng** (01:16:33):
That's right.

**Lenny Rachitsky** (01:16:34):
Yeah, I remember the chariot bus showing up at the Airbnb office and people getting, I'm like, "What the hell is this?"

**Albert Cheng** (01:16:39):
That's right.

**Lenny Rachitsky** (01:16:40):
Very cool. I didn't know you worked there. Albert, we've covered so much ground everything I was hoping we'd cover. Is there anything else that you wanted to cover, anything else you want to leave listeners with before we get to a very exciting lightning round?

**Albert Cheng** (01:16:56):
No, this is great. I hope it was useful for your listeners. I will say over the last few days, as I was prepping for this, I was honestly a little bit anxious about do I have enough deep independent frameworks that I need to come up with? But just being authentic to my actual experience at these companies, a lot of my lessons learned have been off of the backs of other people that have tried similar things and have succeeded or failed.

**Albert Cheng** (01:17:21):
And I think what's important is that you have that your mental sponge. You can try a bunch of different things, you can absorb them and then put them in practice right away, discard the things that don't work and evolve them for yourself and for the company's needs. And so I don't know, I think that was just a realization that I had as I was thinking through this podcast, and I think that's partly why I haven't done too much public speaking.

**Lenny Rachitsky** (01:17:45):
I know exactly what you mean. When I left Airbnb, I was just like, and that was the first time I ever took a break in my career of 30 years of just working straight in school. I was just like, what have I actually learned? I've never just sat down and thought about, here's the thing I've learned. And that led me to writing this medium post that did really well what I learned at Airbnb, and then that basically led to what I do now. So there's a lot of power and I love that this is the excuse to make you think through what have I learned concretely that I can share.

**Albert Cheng** (01:18:17):
That's right. Thank you for that.

**Lenny Rachitsky** (01:18:18):
Yeah, and so at the beginning of this podcast, before I started recording, I always like to ask guests, what is your goal? What do you want to get out of this conversation? And usually, it's like we're hiring. We want to make sure people know about our company or we want to get the users. And your answer is just, I just want to give back things I've learned, which I love.

**Albert Cheng** (01:18:36):
That's it.

**Lenny Rachitsky** (01:18:37):
And you've done that. With that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

**Albert Cheng** (01:18:44):
I'm ready.

**Lenny Rachitsky** (01:18:45):
What are two or three books that you find yourself recommending most to other people?

**Albert Cheng** (01:18:51):
Yeah, so the truth of it is I have a, not just the four-year-old, but I also have a one-year-old. So most of the books that I'm reading these days are kids' books, trying to make them laugh in all.

**Lenny Rachitsky** (01:19:00):
Wait, any favorite kids books? Because I have three or two year olds already.

**Albert Cheng** (01:19:03):
Well, you said that you started singing. There's a book called Snuggle Puppy that has a song in it that just makes my daughter crack up.

**Lenny Rachitsky** (01:19:11):
Oh my God.

**Albert Cheng** (01:19:11):
That is heartwarming for me. But no, a book that I recommended recently at work is Ogilvy on Advertising. Do you know this book?

**Lenny Rachitsky** (01:19:21):
I don't know the book. I've seen these tenants of marketing or whatever.

**Albert Cheng** (01:19:23):
Yeah, it's interesting. So it's 40 years old, but it's just packed with a bunch of different practical examples about copy and creative that work in, these are old school ads, but he took a very experimentation-oriented approach to just try a lot of things.

**Albert Cheng** (01:19:38):
I think in the book, it makes a good reminder that what ultimately matters is to compel your users to some action for him as buying a product, right? It's not about just creating clever ads or sexy creatives, it's to do things that compel that action. I think that's very true for many of our product and life cycle teams. And so I shared that around as an interesting recommendation.

**Lenny Rachitsky** (01:20:02):
Is there a movie or TV show? Sorry, were you going to share another book?

**Albert Cheng** (01:20:08):
Yeah, actually.

**Lenny Rachitsky** (01:20:10):
Oh yes, please.

**Albert Cheng** (01:20:11):
Our co-founder at Chess.com, his name's Danny Rensch, and he is quite well known in the chess circles. He's releasing a memoir called Dark Squares, and it is super fascinating. He grew up in an abusive cult and was a chess prodigy. And so it is just this unbelievable story and I'm about halfway through it, it's a reminder that sometimes the people that you work with, you don't realize how deep their pasts go, but this is something else, and I think it should be out by the time this podcast releases.

**Lenny Rachitsky** (01:20:45):
And it's called Dark Squares?

**Albert Cheng** (01:20:46):
Dark Squares.

**Lenny Rachitsky** (01:20:47):
Which is a reference to the chess board and also I imagine the difficult past.

**Albert Cheng** (01:20:52):
Exactly.

**Lenny Rachitsky** (01:20:53):
Wow. How cool. Okay. Are there movie or TV shows you really enjoyed that you've recently watched?

**Albert Cheng** (01:21:00):
These days it's football season, so I'm consumed by all the hot takes of my favorite teams that I love and the teams I love to hate as well, so.

**Lenny Rachitsky** (01:21:11):
Who's your team?

**Albert Cheng** (01:21:13):
The 49ers. I have season tickets and I go all the time. We had a rough season last year, so hoping to turn around.

**Lenny Rachitsky** (01:21:20):
Okay, very cool. Okay. Is there a product you've recently discovered that you really love?

**Albert Cheng** (01:21:25):
Yeah, so last 20 years of my life roughly, I've moved around a lot, but I've always been within walking distance of a coffee shop. It's just like a ritual that I go and get coffee and it starts my day, right? Two years ago, I bought a house and for the first time ever in my life I'm like not buy a coffee shop, and I was so depressed about this for a little while.

**Albert Cheng** (01:21:45):
So my favorite product is the bread bowl barista, and it just starts my day off. I like making horrible latte art with it, and I think it's just a reminder. I don't know. The products that most impact me, I guess are the ones that I use all the time, and it's a daily habit-

**Lenny Rachitsky** (01:22:05):
And have the most caffeine.

**Albert Cheng** (01:22:06):
Then the most caffeine. You got it.

**Lenny Rachitsky** (01:22:08):
Amazing. Do you have a favorite life motto that you find yourself using in work or in life?

**Albert Cheng** (01:22:14):
As I was thinking about my piano stories, I also remember that my mom used to have a quote. She just said, "Nothing is more important than your reputation." And she used to say this, and I think the charitable understanding of this is that a lot of the small decisions that you make each day, how do you treat people? How do you show up? What's your character, et cetera. They can compound and they open doors for you in many surprising and amazing ways.

**Albert Cheng** (01:22:41):
A lot of these companies that have actually joined have come through relatively light connections. And even just being on this podcast, I think I've seen a number of folks that I've worked with before beyond the show. And so I think doing the right thing, building a good reputation, they can carry you a long way. And the flip side of that is reputations are fragile too, right? So if you do the wrong thing, take a long time to repair that. So I don't know, it just stuck with me my entire life. I thought that was an interesting life motto.

**Lenny Rachitsky** (01:23:13):
Last question. You work at Chess.com, how's your chess?

**Albert Cheng** (01:23:16):
Terrible compared to serious, serious players, but quite compared to the casual ones, yeah. My yellow rating is about 1,800 for a rabbit games.

**Lenny Rachitsky** (01:23:25):
It sounds really-

**Albert Cheng** (01:23:27):
And about 1,500 for blitz. Yeah, but I play many times every day.

**Lenny Rachitsky** (01:23:31):
Blitz is like fast chess?

**Albert Cheng** (01:23:32):
Blitz is like faster chess, three minute games. Rapid is more like a 10-minute game, which is still pretty fast.

**Lenny Rachitsky** (01:23:38):
And you say you play multiple times a day? Do they make time? Is this like-

**Albert Cheng** (01:23:42):
They do.

**Lenny Rachitsky** (01:23:44):
Okay. At Patagonia, there's a famous book, the founder wrote called Let My People Go Surfing, and the rule at Patagonia is you can go surfing if the waves are great. Is that how it works at Chess.com?

**Albert Cheng** (01:23:53):
Absolutely.

**Lenny Rachitsky** (01:23:54):
Okay.

**Albert Cheng** (01:23:55):
Chess is always fun. So we play all the time and they even have chess coaches on staff.

**Lenny Rachitsky** (01:23:59):
Staff, just like you can book to do?

**Albert Cheng** (01:24:01):
You can book. So I get bi-weekly lessons and it's helping me improve.

**Lenny Rachitsky** (01:24:04):
Wow. Okay. This is going to drive a lot of hiring for you guys. Saved it for the end. Albert, this was awesome. Thank you so much for doing this. Thanks so much for giving back and sharing all these stories. Two final questions, work and folks find you if they want to follow up on some of this stuff, and how can listeners be useful to you?

**Albert Cheng** (01:24:22):
Yeah, thanks for having me. This was great. You can find me on LinkedIn or Twitter. Not a super active poster, but I read it all the time. If there's something that I said today that resonates with you and you just want to get in touch, trade notes, feel free to reach out.

**Lenny Rachitsky** (01:24:36):
And can they play with you on, can they find you on Chess.com to play?

**Albert Cheng** (01:24:39):
They can.

**Lenny Rachitsky** (01:24:40):
Okay. Do you want to share your username or you don't want that?

**Albert Cheng** (01:24:44):
I am happy to.

**Lenny Rachitsky** (01:24:45):
No. Okay.

**Albert Cheng** (01:24:45):
I just mentioned that I'm a 49ers fan, so my username is Go9ers, so.

**Lenny Rachitsky** (01:24:49):
Wow.

**Albert Cheng** (01:24:50):
I'm sure I'll get a lot of game requests now.

**Lenny Rachitsky** (01:24:52):
Here we go. Here we go. 1,800. Okay. Albert, thank you so much for being here.

**Albert Cheng** (01:24:56):
Yeah, thank you so much.

**Lenny Rachitsky** (01:24:58):
Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcast, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [2/17] First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason Droege

**Lenny Rachitsky** (00:00:00):
There's been a lot of talk these days about AI not delivering on the promise that we hear, especially at enterprises.

**Jason Droege** (00:00:06):
These things take 6 to 12 months to get them truly robust enough where an important process can be automated. Like with any of these major tech revolutions, headlines tell one story and then on the ground, laying broadband means you need to dig up every single road in America to lay it. Someone's got to dig up the road or someone's got to run the undersea cable.

**Lenny Rachitsky** (00:00:26):
Is there anything you think people don't truly grasp or understand about where AI models are going to be in the next two, three years?

**Jason Droege** (00:00:31):
The general trend right now is going from models knowing things to models doing things. The next question becomes, what can it do for me? How does the agent make decisions for you?

**Lenny Rachitsky** (00:00:39):
Let's talk about Scale and this whole world of AI that you're in, you essentially pioneered data labeling, trading data, creating evals for labs.

**Jason Droege** (00:00:46):
18 months ago, you would get a short story and it would say, "Is this short story better than this short story?" And now you're at a point where one task is building an entire website by one of the world's best web developers, or it is explaining some very nuanced topic on cancer to a model. These tasks now take hours of time and they require PhDs and professionals.

**Lenny Rachitsky** (00:01:07):
I've talked to a bunch of people that have worked with you over the years, and I heard a lot about just how high of a bar you set for new businesses.

**Jason Droege** (00:01:13):
From an entrepreneurship standpoint, it truly is about what insight do I have? Why am I so lucky to have this insight? Why in a world of a million entrepreneurs who are thinking, who are smart, who are trying everything, why am I in the position where I likely have an insight that others do not?

**Lenny Rachitsky** (00:01:31):
Today, my guest is Jason Droege. Jason is the new CEO of Scale AI. This is the first interview that he's done since taking over for Alex Wang after the Meta deal. Alex now leads the super intelligence team at Meta. Prior to Scale, Jason co-founded a company with Travis Kalanick. Before, he started Uber, worked at a couple startups. Most famously, Jason launched and led Uber Eats, which went from an idea that he and his team had to what is now a multi-billion dollar run rate business and one that basically saved Uber during the pandemic when nobody was taking rides. This interview is following a theme that I've been following through a bunch of interviews, which is the evolution of how AI models actually gets smarter. Along with scaling, compute and improving the actual model code, much of the improvements we're seeing in ChatGPT and Claude and every frontier AI model is these labs hiring experts to filling gaps in their knowledge and correcting their understanding of how things work, and basically showing them what good looks like in every domain that consumers are using models.

**Lenny Rachitsky** (00:02:35):
Scale was the pioneer in this space. They created the category, and in our conversation we talk about what is happening at Scale and just how this deal with Meta worked, what experts like doctors and software engineers are specifically doing to help models get smarter, how the whole market of data labeling and evals and data training has changed from when Scale entered the market to today, and also just how long will we need humans to keep helping AI get smarter. We also get into where Jason sees models going in the next few years because they have such a unique glimpse into the future. We also talk about a ton of really unique and really important product lessons from the course of Jason's career, including a bunch of advice on how to start a new business, both startups and within existing companies, and also a bunch of advice on hiring and leadership and so much more.

**Jason Droege** (00:06:08):
Yeah, thanks for having me. Excited to be here.

**Lenny Rachitsky** (00:06:10):
As I was researching your background and prepping for this podcast, I learned a really interesting fun fact about you that I don't think a lot of people know. So Travis Kalanick, he had a startup before Uber. It was called Scour. It was a peer-to-peer file sharing app, and then I think got shut down. You were his co-founder. This was the early part of your career. I'm guessing there are hours of stories we could talk about during this experience. So let me just ask you this one question. What's just a lesson that has stuck with you from that experience that you've taken with you to future places you've worked and built product at?

**Jason Droege** (00:06:44):
I mean, there's so many lessons. I like to pick one. I think that the main lesson is that in business and in startups, everything's negotiable. I think that's the main thing. Because we were 19 at the time, 19, 20 at the time, we built this search engine in a dorm room and we were running it out of the dorm room and our first URL was scour.cs.ucla.edu. These things were not necessarily in fractions at the time, but we were just being practical. It was basically a project that we had started, and so we built the search engine and people started using it and we thought we would get in trouble, but it turned out the computer science department was excited about it even though we had basically parked a domain on their servers and we were using our own computers in the dorms to serve up this website and product.

**Jason Droege** (00:07:46):
And then, when we got into financing, the financing process was fascinating, and this is where the everything is negotiable lesson came from, which is, it was Ron Burkle and Mike Ovitz, who are the initial investors in the business. We were in LA, so we were at UCLA, so we were not quite wired into the entire Sand Hill Road scene. And as we were doing the deal, the terms kept changing on us. We thought you went and raised money and it was like, "Oh, we'll get a few million dollars at a $5 million valuation." This is back when that was actually a series A valuation. And then over the course of the deal, it was like, "We're doing the deal. We're not doing the deal. Oh, you should give us 50% of the company. Oh, you should give us 75% of the company. Oh, if you want to sign the document today, this person's going to show up for breakfast and if you don't sign today and give us 80% of the company, the person's not going to show up."

**Jason Droege** (00:08:38):
It was just completely wild, the things that we saw from day one of what can happen in business, and we thought there was a way to do things, and at a very young age we realized there is no way to do things. There is just the way that you can negotiate your way through the world, which I actually think influenced Travis heavily and then me later heavily at Uber in terms of if you can imagine it and it makes sense and you can align incentives, then it can happen. But there is no way. And to learn that at 19 or 20 years old I think was highly imprinting.

**Lenny Rachitsky** (00:09:12):
That is an amazing lesson. What happened to Scour? It got shut down, I think. What happened there?

**Jason Droege** (00:09:16):
Well, yeah, so basically what Scour was was it was a multimedia search engine and then peer-to-peer file sharing network. But what it was used for was finding free content. And at the time, the laws were on this were pretty ambiguous because we weren't, mix tapes were legal, but this was like a hyperversion of that. But we were eventually sued for a quarter of a trillion dollars. So I guess if you're going to experience something that's potentially as life devastating as that, doing it when you're, I think we were 21 or 22 at the time is the time to do it, but it was just this very cold splash of water about how the real world really works, because the MBAA and the RAA were the ones who sued us, the entertainment industry sued us or the associations that represent the entertainment industry, and then they settled it for $1 million.

**Jason Droege** (00:10:08):
So we're like, "Wait, you wanted a quarter of a trillion dollars and then you settle for $1 million." And of course they were just trying to drive us in a bankruptcy, drive us out of the market, and these are established companies. So we're like, "If these guys don't have a playbook to follow, they just make up numbers, then wow, how should we navigate the rest of our lives?"

**Lenny Rachitsky** (00:10:28):
Let's talk about Scale and this whole world of AI that you're in. This is the first interview that you're doing since taking over CEO at Scale. I'm honored to have you here to talk through this stuff. This is also the first interview you're doing since the whole Meta deal, which is very complicated, confused a lot of people. So I'm just curious to hear the current state of Scale, what people should know. For example, what's your relationship with Meta? What's your relationship with Alex? What is the current state of Scale?

**Jason Droege** (00:10:55):
Yeah, so Scale is a fully independent company. The transaction was Meta invested a little bit over $14 billion to get 49% of the company, non-voting stock, didn't take a new board seat. Alex fills the board seat. So the board is the same, the governance is largely the same. There's no preferential access to anything that Meta has. There's no preferential relationship. I mean, we've had a longstanding relationship with Meta on the data side of the business for a long time and even on some business development related things to maybe working on things in government together, et cetera. And so, those might get bigger just as we're closer now, but there's nothing that prevents us from doing things with other parties and they have no access to anything that they wouldn't have had otherwise. All the privacy still in place, all the data security still in place that was there before.

**Jason Droege** (00:11:47):
And in fact, only about 15 people went over in the transaction. So Scale has about 1,100 employees or so now, and we have two major businesses. Each of those businesses, each of them has hundreds of millions of revenue. So we have two unicorns inside the company today that sustains. The business has grown every month since the deal happened, which I've read, the reporting is not consistently reported. We haven't talked about it, so this is part of getting the word out and we're excited to continue to build, deliver data, and do what we did before.

**Lenny Rachitsky** (00:12:20):
So the company today, independent, its own company. Alex, just to be clear, he works at Meta now. He's no longer at Scale.

**Jason Droege** (00:12:26):
Yeah, that's right. Excuse me, I should have talked about that more.

**Lenny Rachitsky** (00:12:30):
I think that's really interesting. So basically, it was an investment. Some people left to join Meta, the company continues, you're running the ship. Let's talk about this whole space that you guys essentially pioneered, I don't know best way to call it, data labeling, training data, creating evals for labs. You guys were at this before anyone even knew this was a thing. I know even Scale pivoted into this market from other things. I think there was a bunch of stuff they tried with self-driving cars and all these things, and then it's like, "Oh shit, AI labs need this data."

**Lenny Rachitsky** (00:12:57):
One of the main stories I've been hearing is, and I've had a bunch of CEOs from this space on the podcast, is that there's been this big shift from the way, from what Scale had pioneered and had been doing for a long time, which is generalists, low-cost labor training. From that to now, labs mostly need experts, lawyers, doctors, engineers doing training, writing evals, things like that. I'm curious just what you're seeing, how that's impacting you guys, where you think things are heading, what people should know about this whole market of data training data.

**Jason Droege** (00:13:27):
Yeah, totally. I think the current positioning out there from competitors is just bogus. So I'll start with that and then maybe talk a little bit about, I'll explain what I mean by that in a second. But I think it's important to just give 30 seconds on what the history of Scale is and what's the thread going back to 2016. So Alex had this insight in very early days that the important thing to models was data. And I think he was 19 or 20 years old at the time as well. And so, he's like, "Okay, well what business would I create around this?" And the business that he created around it was, okay, let's do labeling for autonomous vehicles, because if you label the data that they have, the cars do better. And then, that wave turned into the computer vision wave, which we have a relationship with the Department of Defense where we do labeling for them, and that was in 2020.

**Jason Droege** (00:14:21):
And then, you move forward and the models have gotten better over this period of time. And so, as models get better, they need different types of data. So we've constantly been adapting to the type of data that models need to be successful. And so, then the gen AI wave hit, and this went through the moon or to the moon. And so, as part of that, that industry is changing constantly too. So it is correct that when the models came out two or three years ago, I mean we remember using them, they would hallucinate all the time, they would get basic answers wrong, they didn't know which poem was better, this poem or that poem. And that was the state of labeling a couple years ago. And things have changed quickly and we've changed with it. And now the state for everyone, and we've been at the forefront of all of this, is expert data labeling, more sophisticated tasks.

**Jason Droege** (00:15:15):
So to give you a sense of what the task was 18 months ago, I've been here about 13 months. So I was interviewing and I remember seeing it. You would get a short story and it would say, "Is this short story better than this short story?" And then you would edit it and be like, "Yeah, it would be better if it was this," and you would give some preference ranking to it. It was pretty basic 18 months ago, and you had the rise of some experts, but the models were so far behind that they needed just even the basic stuff they needed. And now, you're at a point where a task is, one task is building an entire website by one of the world's best web developers, or it is explaining some very nuanced topic on cancer to a model. And these tasks now take hours of time and they require PhDs and professionals.

**Jason Droege** (00:16:01):
So to give you a stat to back this up, 80% of the people that we have on our expert network have a bachelor's degree or greater, which is very contrary to some of the positioning that's out there and some of the understanding of this industry. About 15% have a PhD that's greater, and we have PhDs on the network earning significant amounts of money doing labeling, contributing their expertise to these models. So we've been doing expert data labeling ever since the models need it. I mean, this game is keeping in touch with the researchers, knowing what they need, coming up with ideas internally. In some ways, we drove this because we were seeing that the models were not sufficient in more expert ways. And so, we would go to the model builders and say, "Hey, we noticed that this is a problem. If you would like to fix it, this cadre of experts can do that for you." So the counter positioning is out there, but I think that's just what competitors say sometimes. It has nothing to do with reality.

**Lenny Rachitsky** (00:17:02):
Okay. That was extremely interesting. So what I'm hearing is yes, there has been a big shift to labs need more expert folks involved in training, labeling, writing evals. You guys are very aware of that and have evolved with that. One of the, I don't know, allegations I guess in the market is that it's hard to find these experts. So all these companies have their proprietary network of experts and how they find them. Is there anything you could share about just how you guys go about that because that feels like the hardest part is finding these experts and keeping them from other companies?

**Jason Droege** (00:17:33):
They are hard to find. You have to have many, many tactics. So we get, as you would expect, there's not one way you do it. The largest way is that they refer each other because when you are enjoying what you're doing and you are using your expertise to contribute to AI, which is pretty cool. If you're a PhD on this pretty specific topic and you're using a model and you're frustrated that, oh, it doesn't interact with me in the way that I want, this is a paid way to have an outlet for that and to make hundreds or thousands of dollars doing that. And so, a lot of times they refer each other.

**Jason Droege** (00:18:13):
We also have campus programs where we will literally go onto the campus and talk to the professors, talk to the students, ask about who would like to do this type of work. And then, of course, there's the more traditional scaled ways of LinkedIn and places like that. But the best ones come from these grassroots and referral networks. And the only way you get that is providing a great experience to these people, because these people, they're doing it partly for money, but they're also doing it because they think that their contribution to the AI models is important and interesting, and in many times it solves a problem for them.

**Lenny Rachitsky** (00:18:48):
So something that I've been seeing on Twitter just this week as I was preparing for this is there's the information headline. This came out and this mirrored something that Brendan from Workhorse said that over time the entire economy is going to move towards just reinforcement learning and everyone's just training AI is basically the jobs that will be left. Thoughts on that? Is that where you think things are going? Is there another perspective?

**Jason Droege** (00:19:12):
Reinforcement learning is very important, and I think this is a broader comment about the move to environments. There's these things called RL environments that effectively are sandboxes for AI agents to play in to accomplish a goal so that they can learn how to accomplish that goal. We've been doing this for over a year. So for example, you have a Salesforce instance. How does an AI agent navigate that instance? That instance has data that it needs to recognize, it has configurations. Salesforce is a highly configurable product. It has configurations, it needs to understand how to navigate. You're asking the agent to do a business process that needs very high reliability, and then the agent needs to know, "Hey, if I can't accomplish what I'm going to accomplish, or I think if there's a low accuracy of what I'm about to accomplish, how do I pop it up to a human being for feedback so I can get guidance?"

**Jason Droege** (00:20:08):
All of those things need to be trained and there's no alchemy to it. You just have to put the AI agent in an environment that represents what a human being would be doing. And you can imagine the number of environments in the world and the number of goals within each environment is enormous. So the question is, and the research that we have done over the past year to try to be a good partner to our model builders, our model builder customers, is how generalizable is each individual task or each individual environment. So if you imagine the world of environments of software systems, configurations, data types, sizes, user counts, complexities, it's like the permutations are endless. So what you need is you need a strategy that allows a lab to collect data that is generalizable enough across a broad spectrum of use cases so that they don't have to collect 45 trillion combinations of what should the agent do in this particular situation.

**Jason Droege** (00:21:20):
So sometimes the work and the data is highly generalizable, and by generalizable I mean you have it accomplished in a simple way. The task might be find the meeting on my calendar for my interview with Lenny, and the agent goes and it looks through all my calendar and then it pops it out, very simple example. That needs to be generalizable to any calendar search potentially or potentially any calendar action. And the more generalizable it is, the more valuable the data is. So our job is to provide the most valuable data to model builders that accomplishes the goal of making agents as useful as possible for their end users.

**Lenny Rachitsky** (00:22:03):
I love that you've been sharing these examples of what this stuff is specifically that these people are doing, the data you're providing to labs. So just to mirror back a few of the examples you've shared, one is an engineer building a website, sharing the code essentially with the model. And here's how I would do it. And in that example, is it just like here's the code or is it a recording of them building it? What is the data?

**Jason Droege** (00:22:30):
It could be both. So in some cases, it's just the website and here's an example, and then they design it. In some cases, it needs to be annotated in such a way that's like, I made this decision for this reason or this decision for that reason, or here's how I would think about it. So it depends on what the model builders are trying to accomplish. And so, it can get quite nuanced in terms of what they're trying to train on.

**Lenny Rachitsky** (00:22:54):
Got it.

**Jason Droege** (00:22:54):
So it's not like here's a website and then it's created doing websites. It's like, here's a website, here's why I made this decision, here's why I didn't make this decision, or here's a broken website and here's why it's broken if they're trying to accomplish, I don't know, a debugging tool for a website builder or something like that.

**Lenny Rachitsky** (00:23:09):
And another example you shared is a short story where it's like, here's one short story, here's another I imagine generated by a model. And then it's like, which is better, and then how would you make it better? The other example you just shared is a Salesforce agent where it's like, Hey, book a meeting with a prospect and then teach it how that happens. I love just how concrete these are because it's like, okay, I get it. This is the stuff that these companies do. Is there another maybe one or two examples just to give people a sense of what this data looks like?

**Jason Droege** (00:23:33):
Absolutely. I can actually give you an example from, so we have two sides of our business. One, we supply data to model builders. We sell the data, and then the other is we actually do solutions. We sell applications and services to healthcare systems, insurance systems, et cetera. I actually think it would paint a more colorful picture if I gave you an example of one of those because it involves data, but it involves the use of data, the manipulation of data for a very, very specific goal. And so, one example there is we work with a healthcare system and health systems have lots of problems. This particular healthcare system has experts that see very rare cases on a regular basis. So you go there only if no one else can figure out your problem, and there's a huge backlog. So there's a productivity element to this implementation tier.

**Jason Droege** (00:24:27):
So there's a huge backlog. They want to be able to see more patients, they want to be able to provide better care, and they want to prevent the number of revisits because they want to give the accurate diagnosis day one and what the treatment should be. Well, to do this today without the help of AI, the doctor really needs to read 200 to 300 pages of documentation and it's rolled into one document, but in different formats. And so, if you're a doctor, how are you going to read 200 or 300 pages of everything? So what they do is they do the best they can. They scan it, they ask a nurse to look at it, they ask maybe a more junior doctor to take a look at this case. They want to treat the patient well, obviously this is why they became a doctor. And then, they go into the room and they talk to the person and then they make a diagnosis.

**Jason Droege** (00:25:14):
Well, we basically built a tool that will read that document for them and point out the top 5 to 10 things that they should take into consideration, either allergies that might not be obvious is one example where we actually, we picked up on an allergy that a patient had that would not have been obvious from reading the document and that allergy actually would've had a conflict with the medication that they were going to be prescribed. And so, the AI tool basically pulled out this correlation that would've even been hard for a human being to do. To make this tool better and better, you get to a certain limit with the models off the shelf, and actually the people inside of this healthcare system have to do their own labeling.

**Jason Droege** (00:25:54):
So we talk about labeling for model builders, but we are starting to see the labeling move into enterprises and into governments because you can only get so far with off the shelf plus rag plus some fine-tuning based on recorded data. One thing people often miss about these systems is we assume because you hear these numbers of like, "Oh, this bank in just 200 petabytes of data a year or whatever fantastical number." What we miss is is that the right data? Which of that data is useful to the models? And most of it is not useful. Some of it is, but a lot of what we do when we're talking about knowledge work, when we're talking about making judgment is human judgment based on synthesizing how would this doctor in this case or how would this banker in this case make this decision and how would they make decision in the context of their overall enterprise? And that might be different bank to bank, healthcare system to healthcare system, because of the culture, the objectives, the incentives, et cetera. And so, we're getting to the point now where we see that digitizing judgment, human judgment, true subject matter, deep expertise is becoming a bottleneck that we're unblocking for our customers.

**Lenny Rachitsky** (00:27:05):
That's really interesting. It's like the spectrum went from just low skill generous labor to experts to now the specific expert at this one company who needs to do this work, this labeling.

**Jason Droege** (00:27:16):
Absolutely. I mean understanding what, there's this broad narrative. We have two narratives. We have the AGI, everything is just going to become AGI, and then there's the skeptics, which is like, "Hey, this is all bunk, this is a bubble, et cetera." And of course, my view is most things are kind of like there's truth in between and some of the extreme parts of the extreme probably correct, but the reality is is that it's very hard to get machine critical use cases in agentic systems where agents are talking to agents to a level of accuracy that is necessary to accomplish a goal. And one of the main issues is that a one document, think about the problem of even understanding a document, a document that reads the exact same words in company A will have a different meaning and importance in company B. So how do you have a system that knows that? So this is all got to be built. So if you're going to make good decisions.

**Lenny Rachitsky** (00:28:18):
This is a good segue to this question that is always on people's minds when they look at companies like yours and the other folks in the space is just how long do we need people to be doing this? At what point will AI be smart enough to do it themselves? I know your incentives are to say we'll never run out of people because it's aligned with your growth, but just how should we think about just why do we need people, I don't know, in 10 years? How long do we need these experts telling AI things it doesn't know?

**Jason Droege** (00:28:42):
First off, the history of data labeling is a history of new beginnings. Autonomous vehicles do not need as much data labeling as they did in the past. I mean, Scale is a company that believes that data will always be important at the point at which you don't need external data, human data in models. I think we've gotten to a level of advancement in the world that is almost like unfathomable because you're effectively saying that no new human skill and no new human knowledge is important enough to put into these models. That feels like pretty far out there. And so, for a business like ours, we're constantly looking at how do you build operations that can constantly find the new needs and then work with the contributor network we call the experts contributors to unearth that data, to unearth that information. And sometimes it's new people, sometimes within our existing base we find that existing people have expertise that we didn't know about that maybe wasn't useful to a model a year ago, but now is useful.

**Jason Droege** (00:29:47):
So this is a constant progression of getting more and more data into these models. Yes, we are financially incentivized to believe that humans will always be in the loop, but that's not just a business belief, it is a personal belief. These systems need to work for us, and if these systems work for us, then we will need to be on the loop or in the loop on any of the decisions that these systems make. As to the broader point around labor, which I think comes up around white collar apocalypse and these things that come up, I'm definitely on the more maybe practical side of this, possibly just because of my nature, possibly because I see what's going on on the ground actually in these customers where supposedly this transformation is going to happen in the next one to two years. And I just think that it might happen. The space is moving super fast, but I don't think it's going to happen.

**Jason Droege** (00:30:39):
It is definitely not going to happen in the next year. The idea that it happens in the next two years I think is very far-fetched, but nothing's impossible here. And long-term, I think that if you go back through, I don't know, pessimist archive or whatever, these accounts that post, the radio was invented and then all of this will be eliminated. There will be change, but the change, I think humans are very good at adapting. So I think what we're underestimating in all of the doom and gloom is we believe in human adaptability. We as a company are highly adaptable and I think the history of technology has shown that people are adaptable.

**Lenny Rachitsky** (00:31:14):
I really like that takeaway. I'm an optimist as well, so I'm always looking for reasons to be optimistic. I want to follow that thread before I get there, something very tactical I want to ask about is evals seems to be coming up a lot, especially with companies in your space. I'm still learning a lot about just what this all is, especially in your market. How much of what you or experts are providing are evals versus other types of data?

**Jason Droege** (00:31:40):
A lot of it's evals, and within enterprise customers and government customers, it's mostly evals because somebody's got to establish the benchmark for what good looks like. That's the simple way to think about evals. What does good look like and do you have a comprehensive set of evals so that the system knows what good looks like? It's as simple as that.

**Lenny Rachitsky** (00:32:00):
So in the case maybe of the healthcare example you shared, essentially this doctor would be sitting there looking at all these reports, creating evals that are like, this is what this should be discovering in this report, in this record. Is that a way to think about it?

**Jason Droege** (00:32:15):
Yeah, that's a very big part of it, which is what does good look like?

**Lenny Rachitsky** (00:32:19):
Awesome, okay.

**Jason Droege** (00:32:20):
I have to reduce things down to simple terms.

**Lenny Rachitsky** (00:32:24):
It's interesting you say good versus correct. Is that a specific term you like to use good versus just this is the correct answer.

**Jason Droege** (00:32:31):
I didn't intentionally use that word, but these are probabilistic systems and so depending upon... Yeah, so I can get into some nuance here about the right types of problems that AI is good at solving. So if you have a human process that is 10 or 20% accurate or 10 or 20% liked, AI is awesome. Because if you get to 50, 60, 70, 80% accurate, you're in the money, you're in the green, everybody's happy. Now, the system then has to know, hey, for the remainder, how do I make sure that humans are involved for the remainder of the decision making? But from a net value add standpoint, the humans are pumped in that scenario.

**Jason Droege** (00:33:13):
If you have a human process, a workflow that is 98% accurate, and you expect an AI system to get you the remaining 2%, not totally there yet. And so, when I say what does good look like? A lot of the processes and a lot of the things that people are asking these systems to do and systems for us to build are making judgments on their behalf. And so, just like we would ask a human being, "Hey, what do you think we should do in this scenario?" What you're looking for is you're looking for the best recommendation or course of action given the current information.

**Lenny Rachitsky** (00:33:48):
To you, this is so obvious and to people in your market that I think a lot of people think about AI being trained on just here's a bunch of data, check it out, learn everything you can from all of human history and all of written record. But what's wild is basically people are sitting around teaching AI things it doesn't know, filling gaps. That's how AI is getting smarter now. There's no more real data for it to feed on. It's just like, here's what I don't know, or here's what an expert found you're wrong. I'm going to teach you this. And the fact that it scales and that's keeping models improving is so mind-boggling.

**Jason Droege** (00:34:21):
Yes. No, yeah, I agree. I mean, like with any of these major tech revolutions, the headlines tell one story and then on the ground, laying broadband means you need to dig up every single road in America to lay it. There is the, yeah, it's as simple as that. Someone's got to dig up the road or someone's got to run the undersea cable. There's always some operational chiseling that's going on in all of these industries. I mean, if you think about how magical these models are, they're remarkable that if you've been in technology long enough, it blows my mind even today that they get the punctuation right consistently. I mean, that sounds like almost daft to say at this point in the market, but if you were to go back three years and think about that from a technological standpoint, a lot of things that we think are trivial now are very sophisticated, and it's a combination of, I mean, the real answer is it's a combination of computational power, model improvement, and data, and all three are getting better at once.

**Lenny Rachitsky** (00:35:25):
Let's follow that thread. You've been at Scale for a long time, CEO for, you said, 13 months. I feel like you see a lot more about where things are heading because you work with labs on things they haven't even announced yet. You see more than most people, and I know there's only so much you can share about what companies are doing, but just is there anything you think people don't truly grasp or understand about where AI models are going to be in the next two, three years?

**Jason Droege** (00:35:47):
Look, there's so much talk. I think it depends on how much X or news you consume. So I think it's like what sort of our perspective. The general trend right now is going from models knowing things to models doing things. And we're pushing the boundaries of knowledge, like the benchmarks that we put out and that others put out are showing that the knowledge that these models have is getting, it's quite robust. And then, the next question becomes, well, what can it do for me? And as soon as you get into that world, that's where the environments we were talking about start to come into play. How do you navigate a Salesforce instance? How do you navigate a healthcare system? How do you navigate even a weather app on your phone, and how does the agent make decisions for you?

**Jason Droege** (00:36:36):
We're just getting into the beginning of that. It'll be very interesting to see how quickly that happens. And I think that's where a lot of the speculation has a wide variance because we're at the beginning of it. People take different trajectories on how that's going to improve. And so, if you take a trajectory of the most aggressive trajectory, which is like, oh, it's actually going to be quite easy to train on these things, and then it's just a change management exercise in the economy, which by the way, change management exercises are not to be underestimated.

**Jason Droege** (00:37:06):
There's still people in the world without an email address. And so, the adoption curve then becomes a human and policy issue, not a technological issue. We're not there from the technology standpoint, but I do think in the next two to three years, if I take the bait and have to make a guess is the technology will get to a point where it will push the change management and policy makers to say like, "Oh, what do we do with this because it's getting pretty close?" That's probably two or three years away.

**Lenny Rachitsky** (00:37:35):
There's been a lot of talk these days about AI not delivering on the promise that we hear, especially at enterprises. There's this MIT study that just showed that there's all these pilots that people are excited about and then they don't work and companies aren't adopting these tools. There's data showing engineers are not actually as productive with tools. It actually slows them down sometimes. You work with a ton of companies implementing all kinds of AI. What are you seeing on the ground? What kind of gains are you seeing? Do you feel like it's overhyped, underhyped?

**Jason Droege** (00:38:03):
There's a lot of hype out there, and our job is to actually build products that work, that deliver value for our customers and figure out where the rubber hits the road. And to get a sophisticated, my healthcare example is one, we do other sophisticated workflows, claims management for insurance companies. This is a financial decision that's happening, but it's an automatable process. But basically what happens is the POCs get to 60 or 70% of the way there, and the human mind goes, oh, the rest is no big deal. But it's like uptime in data centers where every nine is an order of magnitude investment in terms of reliability, backups, et cetera. One nine is basically a web server in a dorm room like we had at UCLA, and then five nines is this crazy high bar, but it just seems like a very small movement.

**Jason Droege** (00:38:55):
So you have a similar dynamic going on here where you have a bunch of people, one of the reasons why the POCs have failed, one, there's a denominator effect because it's so easy to do, "Hey, I spun up a project, I spun up a project, I spun up a project." So it's really easy for people to try. So I don't necessarily know that the 95% number, I think is a bit of clickbait in a way. It tells the right story, but it is a little bit hyperbolic because if you take the efforts that happen in the company where they actually get a quality partner like we are, or if you do it yourself, if you have engineers who've worked with models before and they put in the time, and I'm talking about months, not like minutes like you see in these videos to actually get legal approval, policy approval, regulatory approval, change managements like an accuracy that everybody's comfortable with. If you actually do that, these things take 6 to 12 months to get them truly robust enough where an important process can be automated.

**Jason Droege** (00:40:00):
So I think that's where the hype is right that when you do it, the impact is like, whoa, I never would've figured that out myself, and I'm one of the most educated doctors in the world as an example. But the time to get there is just longer than what people are selling.

**Lenny Rachitsky** (00:40:17):
It's such a good point that it's not only is it easy to try these things, it's just like everyone's doing it so everyone's feeling FOMO like, "I got to try these things. I got to try all these prototyping tools, Cursor, all these things." Just goes, "Everyone's doing it," and then you just rush into it and it doesn't actually work out.

**Jason Droege** (00:40:31):
Easy to learn, hard to master. That's my summary.

**Lenny Rachitsky** (00:40:36):
**Jason Droege** (00:42:33):
I mean, I probably fall in the category of what you just described, which is maybe part of the hubris you need to start anything new. But I mean, I don't think it's a clean process. I think my process is I'm constantly questioning every single thing that I'm hearing at the beginning of anything. I don't take what a customer says literally. And there's been a lot talked about on this topic from a product management standpoint in terms of like, oh, don't do what they say, do what they mean, and look at the real problems and underlying things. I think the way that I look at it that might be additive to the discussion is I look at the underlying incentives of the customer. And the underlying incentives of customers are not always financial. Sometimes it's ego, sometimes it's career growth.

**Jason Droege** (00:43:24):
If you're selling enterprise software to someone, there's an executive sponsor as an example, that person needs to trust that you're going to do a good job for them. How do you get them to jump with you on this big project? Well, that's part of the journey of not just the product, but what do they need to hear from us? What do we need to supply them? What do we need to do to actually unlock the opportunity to implement the product? So I think there's an incentives alignment baseline. I'm a big believer that it's cliche, but show me the incentive and I'll show you the outcome. I think that's absolutely true. And even when customers will tell you things, I'll give you an example. I've been out of the game for a while so I can be open about it, Uber Eats.

**Jason Droege** (00:44:06):
So when we launched Uber Eats, I looked at the business in terms of being close to the customer. We actually couldn't get a restaurant tour. I knew nothing about this industry. So at Uber, my job was to figure out what other businesses we should get into. And so, we looked at a billion businesses and Uber Eats, food delivery was the one that we thought was most interesting, which turned out to be right so good for us.

**Lenny Rachitsky** (00:44:26):
Very right.

**Jason Droege** (00:44:28):
And we couldn't get a restaurant tour to help us understand their unit economics. And they'd say like, "Oh, it'd be this percentage or that percentage, or Why do you want to know?" And then we'd go to a different restaurant tour and they would explain it, but they were a little suspicious of why are these Uber guys talking to me about how much my ham costs? And so, what we did is we ordered just a bunch of food from these places, and then we got a restaurant supplier to give us a base catalog, and we just matched up how much does the ham weigh? How much does the cheese weigh? How much does the bread weigh? How many pieces of lettuce were on there? And we tried to actually just compose our own independent view of what's the ingredients cost versus what's the labor cost? And then, we triangulated what was our ground truth, and then what are we being told by restaurant tours, and then what is the site guys telling us about restaurant economics?

**Jason Droege** (00:45:16):
And if those things all overlapped, and we're like, okay, we have an insight about what to do here and how does this relate to Uber Eats? Well, what we found as part of this is that roughly a restaurant pays 20 to 30% of every meal to ingredients, and they pay roughly 20 or 30% to labor, and they pay roughly 10% to real estate and a bunch of other, anyway, so goes down the chain. But the important parts is what's the value of incrementality?

**Jason Droege** (00:45:41):
And so, we came in and we said, "We're going to charge you 30% of the bill." And they were like, "Oh my God, is this group on all over again? This is way too high. Oh my gosh." And we explained the economics to them and they were like, "Okay, we'll give it a try, but this is way too high." And they were right, the real number, the real clearing prices aren't 25%, but we weren't that far off. And so, when you go to find product market fit or be close to the customers, it's a combination of what's the most valuable thing. Well, in a restaurant tours case, give me incremental demand. Because if you were to take a restaurant location and triple demand based on the same labor but you're just scaling ingredients, you've got a 70, 80% incremental gross margin product.

**Jason Droege** (00:46:27):
Restaurant tours would hate when we would say this because it doesn't work out exactly like that in reality. But because we had that insight, we had confidence that we could go to market with, we need to charge you this so that the delivery fee can be that. And then, if the delivery fee is that and we charge you this, then we think the consumers will adopt, and that's what you need to get your incremental demand, and then we could pay the driver this. And so, you fit this whole puzzle together without totally satisfying, in the case of a marketplace, you're not totally satisfying any individuals 100% of their needs. What you're satisfying is is you're getting a clearing rate for them to participate in the market in the case of a marketplace. So that's one example.

**Lenny Rachitsky** (00:47:07):
Yeah. I love this example as you almost you figure out how to help them with something they don't even fully themselves know yet. So as you think through their goals for them as if you were them, break down the economics and then here's the solution versus, hey, what can we do for you guys?

**Jason Droege** (00:47:25):
Yeah. I mean, if you walked into a restaurant, they would tell you a bunch of things. They would say, "Oh, labor schedule is an issue." They would say, "My rent is an issue." They would say, "All these, my ingredients prices are an issue, that's 20 or 30%." If you could shave off 3% of that, that would be huge. You might then take that and go, "I'm going to go build a business. It's going to save you 10% of your ingredients costs."

**Jason Droege** (00:47:45):
Well, but that doesn't actually get into their head on what's truly important day-to-day. That might be important for them on an annual basis, but on a daily basis, what are they doing? They're looking at their numbers, they're looking do people show up. Did I make money yesterday? Am I going to make money tomorrow? So the urgency, I think the biggest thing people miss when they're building new products is the urgency of the buyer part of it. You can build something that provides a lot of value, but if it's not the top thing that the customer is thinking about in their busy days, then you're just going to have a long road to a small town.

**Lenny Rachitsky** (00:48:19):
This touches on just the theme I heard a lot about, this idea of independent thinking and how much you value that, and this feels like a really good example of that. Is there anything else along those lines of just why this way of thinking is so critical?

**Jason Droege** (00:48:30):
I think as a founder's job, and I mean I stretched that term because at Uber we had all of the benefits of Uber so I wasn't really a founder. I just started the business there. But there are some elements of founding there is you're looking for alpha in the market. When we started our first company in '97, it wasn't that cool. It might've been cool in Silicon Valley, but it was definitely not cool in LA. Now, it's super cool to start a business. So as a result, everyone's trying everything. So how do you get alpha on that market. If your research is highly influenced by what the world is saying around you, you're not going to have an independent insight. You have to go off and do your own thing.

**Jason Droege** (00:49:14):
And this is why from an entrepreneurship standpoint, I have very strong feelings about what the approach to founding a company should be and is probably very particular to me. But it truly is about what insight do I have, because why am I so lucky to have this insight? Why in a world of a million entrepreneurs who are thinking, who are smart, who are trying everything, why am I in the position where I likely have an insight that others do not? And then, why am I the one to do it?

**Jason Droege** (00:49:46):
And the answer might be I'm in this narrow, far-flung place. The other answer might be, I am inherently a contrarian personality type, so I'm just constantly looking for the thing that's true that people don't believe is true, which sometimes worked. But then, the second part of that's super important, which is why do I want to work on this problem for 5 to 10 years? And people get this wrong all the time. They go and talk to a customer and they go, "They have a problem. I'm going to go solve it." And it's just not a great way to start a business. You really have to have this burning desire to constantly be questioning yourself.

**Jason Droege** (00:50:20):
The other thing about independent thinking is is that you can't fall in love with your ideas. And I do not proclaim to be the world's greatest thinker for what it's worth, this is what you've been told, but is just part of that is basically throwing away who you are, who you've been, all your ideas for the mission that you're on, which is trying to accomplish something for our customer.

**Lenny Rachitsky** (00:50:43):
This is great. I'm glad you went here. This touches on the other theme I heard often about you is just how high of a bar you set for new businesses. And I think this advice is useful both for founders, as you said, and also people starting companies within companies, new business lines. So you've talked about this a bit already, but is there anything more there, just how high that bar needs to be for it to likely work out when you're starting something new?

**Jason Droege** (00:51:07):
Look, if you want to give yourself the best chance, and this isn't always how it works, but if you're in my position 25 plus years in their career, if you want to give yourself the best chance, I think there's two ways that companies end up working out. And the first way, which is probably the most important, quite frankly, is that the founder is just a force of nature over a long duration of time. Because you're going to have to pivot, you have to have that energy to pivot. You have to go years and years and years with it being hard, and that's probably the most important thing.

**Jason Droege** (00:51:36):
But the second most important thing is that you can easily educate yourself on what are good business models, what are bad business models, what are good markets, what are bad markets? And even if you're this force of nature, having the knowledge, if you're going to go into a bad market with all your energy, you should at least know, maybe ignorance is bliss because you just throw yourself into it and it just works out with time. But that's not how I would operate, which is marketplaces are good businesses. SaaS, at least historically, we'll see how this changes, but SaaS, historically, great businesses, recurring revenue businesses, sticky businesses, network effect businesses.

**Jason Droege** (00:52:12):
And if you look at what the top VCs invest in, yes, there is a lot of portfolio building, but there are similarities in terms of the types of business models that they believe could be worth tens of billions of dollars. And they have network effects, they have lock-in. They are more valuable at scale, a big scale than low scale. So if you just take a filter on a new business, this is what I did at Uber, which is like if you just have a filtering mechanism on a new business, it doesn't take that long to eliminate the bad ideas. And then, of what's left, you can pick, oh, I'm very passionate about this, even though it might have more problems than this other thing that on paper looks better. And then, you have to have passionate about it. But I think people just miss a basic understanding of what businesses even have a chance of being worth $100 billion.

**Lenny Rachitsky** (00:53:04):
So you launched Uber Eats, you figured out this is the place to go and bet. As an outsider, feels obvious, of course this is going to be a massive success. Of course, food delivery, such a good idea. I know you looked at a ton of ideas in that process. Can you just talk about what you explored and why you ended up picking Uber Eats?

**Jason Droege** (00:53:23):
I am definitely not the smartest person in the room when it comes to figuring these things out. And so, I keep a very, very wide aperture on ideas for as long as I can until I'm like, okay, everything is coalescing. And I think there's a bunch of reasons why you have to keep an open aperture on considering ideas that might seem bad at the start, but you just keep digging and see if you're right that they're bad or you're wrong. So just as a general philosophical principle, I'll start there. We looked at, we did some crazy stuff. I went walking around San Francisco one day and I looked down Market Street and there was a CVS, a 7-Eleven, a CVS, a Walgreens, a 7-Eleven, and I'm like, "How many SKUs could possibly be inside one of these things that people want and couldn't you just put that into a van and you hit the button on the van and the van comes around and you get whatever convenience items you have, and they're convenience items, so why would that be a problem?"

**Jason Droege** (00:54:14):
And we launched that in DC. We put 10 of these trucks on the road, we put 250 SKUs in them. And I mean, crickets is an understatement of how bad it was. I mean, we couldn't get an order to save our lives. And what we realized was that we hadn't really done the research on what convenience stores really were. It was if you didn't have cigarettes, you didn't have beer, you didn't have Slurpees, you didn't have these things, for example, you didn't bring people in to sell all the other things. So we didn't know anything about retail. We were clueless. So that's one idea. We looked at grocery, but honestly the unit economics just terrified me of all the pick packing and everything like that. I think Instacart did a remarkably good job at getting the unit economics to a good spot and probably the hardest operational problem you could tackle.

**Jason Droege** (00:54:56):
We did generalized delivery, point to point delivery, what's now, I forget what Uber's product is called, but Uber Direct I think it's called, where you have something that needs to go point to point in a city. That was a flop from the beginning because the truth is is how consumers don't really have this need, business sort of have this need, and in 2014 when we were doing this, no one had this need. But we tried 15 versions of all these things before we eventually just said, "Okay, the food delivery thing is just popping off on all signals and we can make the unit economics work. People seem to want it. It's a super cool problem because we can enable independent restaurants with all these tools and allow them to compete with the big guys. We can take the real estate out of the equation. So you can have a real estate location that's non-prime, but if you have prime food, then you get to compete." So we're like, "Oh, this is a very interesting problem and we can really help local economies."

**Lenny Rachitsky** (00:55:49):
And this ended up being, if I remember correctly, this basically saved Uber during COVID. Lyft didn't have something like this. And how big is this business at this point? Anything you share about just how important this turned out to be for Uber?

**Jason Droege** (00:56:02):
Yeah, of course. Well, we launched it in December of 2015 in Toronto and within two hours we had done $20,000 for the sales. It was crazy how quickly we saw that it was the right idea and the unit economics were good. And then, four and a half years later, I was at Uber for about six years, but it took us about a year and a half to figure this out. Four and a half years later, it was about $20 billion. So it was 0 to 20 billion in four and a half years, which is pretty good. Uber was very good at scaling things, but competitive market. Others did well. We beat a lot of people. Some people beat us. And then, now I think it's pushing 80 billion, and that's been for another four and a half years since I left. I think COVID turned it from 20, I left right before COVID, total coincidence, 20 to 50 in a year. So I mean, ride-sharing went this and food delivery just went to Pluto.

**Lenny Rachitsky** (00:56:52):
What luck. Well done.

**Jason Droege** (00:56:55):
Luck is part of the game. That's the other thing that's important to realize. Luck is part of the game, so do not begrudge people for luck. This industry is hard. All these things we're doing are really, really hard. Luck is just part of the game.

**Lenny Rachitsky** (00:57:08):
Maybe speaking that maybe not. One of your colleagues, Stephen Chau, who I am an investor in his new company, he worked with you at Uber Eats for a long time. He told me to ask you about the McDonald's story. I imagine that was just a big milestone, a big moment enough for you guys. So why'd you decide put McDonald's in Uber Eats and there's apparently a story of how you won that deal.

**Jason Droege** (00:57:27):
So it was interesting, and this just goes to maybe where sometimes ignorance leads you to accidentally the right answer. So we had launched Uber Eats and Uber had a global footprint and we were the only food delivery network with a global footprint excluding China. Everything at Uber needed to be launched globally. That was a very big part of the culture, et cetera, which is a lot of work and you can spread yourself too thin and cause other problems. But in this way it was good. My vision was, okay, let's help the little guy compete with all these chains. They have these systematized food systems and food is what makes a city amazing. And no one talks about the chain restaurant that they visited in Paris. They talk about the local place that they found and let's be part of that. That's who we want to be.

**Jason Droege** (00:58:17):
And so, McDonald's actually approached us and they said, "Hey, we'd love to do food delivery with you." And I said, "No." And they're like, "Hold on a second. We have 80 million consumers a day. You don't want to do this together?" I'm like, "It's not really our vibe right now." And so, I pushed them off for four or five months until my team is like, "You're insane. These people are going to put marketing behind it. They really want to do this. They want to lean in." So we actually had, because of that, I think it's hard to correlate these things, we ended up with this exclusive relationship with them, got an insane number of customers of... Chains at this point actually weren't really on food delivery networks because everybody was so worried about the unit economics, because they're so sensitive to the basket size.

**Jason Droege** (00:59:00):
And my approach was like, eh, figure it out, which is a very Uber culture thing. Okay, the basket's $17, it's our job to make that work, reduce the radius on the delivery, figure out the economics, maybe mark up some of the food someplace. There's always a way to figure it out. So we did it and then three months later the business just started hockey sticking again at a different level. And my team is just like, "Dude, you were so stubborn on this point," but I think it actually ended up being in net benefit because we got a great deal with them.

**Lenny Rachitsky** (00:59:34):
So the fact that you pushed him out helped you get a better deal is what I'm hearing. That's amazing.

**Jason Droege** (00:59:37):
Yeah, I think that's the story he would be referencing. And then, the onboarding of it was crazy because we basically went global with them in six months, and at this point the business was less than two years old. So activating this, I don't even know, an 80-year-old company that expects processes to be in place and we have two of our office managers in New York managing it. It's just mayhem.

**Lenny Rachitsky** (00:59:59):
I'm still sad In-N-Out is still not on any of these apps.

**Jason Droege** (01:00:04):
Yeah, me too.

**Lenny Rachitsky** (01:00:05):
I remember someone was hacking it. There's all these ways people found a way around and they're like, "No, no. Okay, you're Postmates. We know we're not going to give you any food."

**Jason Droege** (01:00:12):
Yes, love In-N-Out.

**Lenny Rachitsky** (01:00:13):
You've touched on this idea of gross margins and margins, how obsessed you are with this. I wanted to spend a little time on here. I've heard just you're obsessed with understanding gross margins before going in on anything. Most founders have no idea what they're doing here. What have you learned about just what people should be paying attention to, what they might be forgetting when they think about just the feasibility of a business?

**Jason Droege** (01:00:33):
Yeah, look, it's one filter like many filters. There are certainly businesses that have low gross margins that are great businesses. Costco, Walmart, et cetera. Amazon talks about this all the time of there's companies that increase prices and there's companies at lower prices. But I would say that by and large, high gross margins combined with healthy churn curves are a very healthy sign for the business. I mean, think about it. If I were to sell you something and I can't mark it up a lot, how much value am I adding beyond what's in my hand? And if I'm not adding that much value, then what am I in the business of doing? And I'm in business of adding value. And it's not quite that simple. This is just a litmus test of when someone comes to me and they go, especially in a new business, and we deal with this. I dealt with this at Uber, I've dealt with it everywhere.

**Jason Droege** (01:01:26):
Someone comes up with an idea and they go, "We can get into this business and I think we can charge this and it'll get us to a 40% gross margin." And then, my next question is start at a 60% gross margin. Why does that not work? And they go, "Oh, well, the customer..." And immediately, you short circuit to what the real problem is. Oh, the customer has an alternative. Oh, okay, well who's the alternative? Oh, it's some offshoring company. Well, what's their gross margin? Oh, we don't know. You go find out. It's like 20% and they've been around for a long time and they have scaled operations. And you're like, okay, so your gross margin is going to go from 40 to 20 quicker than you think, and you're going to be in a world of hurt unless you do something to differentiate.

**Jason Droege** (01:02:06):
So I take gross margin is just a very coarse instrument, not a perfect instrument to think about, am I adding enough value? Am I differentiated? It's not perfect, but it's a very quick short circuit filter to even to see if someone's pitching you an idea, have they thought through this dynamic? Because if the response is gross margin is super low right now, but here's the dynamic I'm going after. And then you're like, "Oh, okay." And sometimes it's like, we'll just make it up with volume and then the gross margin will go negative for a while and you're like, "Wait, this doesn't work."

**Lenny Rachitsky** (01:02:35):
So what I love about this is just a lens into is my idea good enough if studying, can I keep a high gross margin? Is there a reason why people in this space haven't been able to have a higher margin?

**Jason Droege** (01:02:48):
Yeah, exactly. And like I said, it's meant to disqualify just you're doing these large for larger companies and everybody has ideas. And so, it's a way to cut through. Do you understand the machine that is going to need to be in place in two or three years? You might have a 70% gross margin now because the next question is why can't someone else do this? And if you have an answer of like, "Well, they can now, but they can't in two years, if we run really fast." Okay, we might have something. If they can now and they will be able to in two years, you're going to have margin compression.

**Lenny Rachitsky** (01:03:29):
Along these lines I was just listening to, I think it was the a16z podcast. Alex Rampell I think was sharing this story about Costco, how as you said, their strategy is actually to keep margins very, very low because all their revenue comes from their membership. So they have something like 50 million members paying 100 bucks a month and that's their entire business. And so, they don't plan and they don't want to make money off the products.

**Jason Droege** (01:03:55):
Yeah, that's right. I mean, they're playing a slightly different game, not an expert on Costco, have spent some time with the company, but they use price as a way to get to scale. And so, they're basically saying if we discount, same with Walmart, we will get so much volume that we will just take the air out of the room for all of our competition. And so, then the question of, okay, so if you have a low gross margin today, in two or three years, once you land one of these centers in a market, why won't your margins to get eroded? The answer is because we will have already absorbed all of the demand. You try to go to 8% versus 10% gross margin, which I roughly think is what their gross margin is. That's going to be a really hard business. If you already have a habit with a customer, they have already built their weekly trips around you, you already have relationships with suppliers, you already have general managers that know how to stock inventory, that's not a straightforward exercise. So they're first to scale and then good luck competing with them.

**Lenny Rachitsky** (01:04:49):
Okay. Just a couple more questions. One is there's this term that I've heard that you often say and believe in is this idea of not losing is a precursor to winning.

**Jason Droege** (01:05:00):
Yes, yes.

**Lenny Rachitsky** (01:05:01):
Talk about that.

**Jason Droege** (01:05:03):
Tech is a culture where portfolios are built by investors, and a lot of the narrative is controlled by investors frankly. Founders obviously participate, but this idea that you should just go for it is consensus. Just go for it. Who cares? Well, I don't know, if it's my life and I only have one moment to take a shot, I might want to just not just go for it. I might want to think for a little bit, and I think the best entrepreneurs, I have no data to back this up, but just these are my friend, this is my friend group. I think the best entrepreneurs and the best business owners look at the risk profile of the decisions that they're making and they try to make asymmetrically positive decisions all along the way.

**Jason Droege** (01:05:51):
And so, oftentimes I feel like we forget about the risk of a decision, and there's more to unpack there because I actually think taking highly risky decisions and then having it work out is a weird cultural thing too, because then how do you train people to do that? Because it's a very hard thing to take high risk decisions and be right enough because it creates a lot of volatility. But it goes back to my comment about the most important thing in founders, which is just this ability to persevere through. Survival is just part of the game, and most people just give up before they get their timing right, before they get the right insight with the customer before they get the right product in the market. And life can change quickly in tech. You can go from being a dog to being a hero in a very short period of time, but you're on this very, very long journey, but you have to survive for that condition to be met.

**Jason Droege** (01:06:38):
And so, then the question is is when you're in a hype cycle, I would argue that we are right now, everyone wants to go for it and then go for it more and then go for it more and go for it more and you don't realize, guys, all of our customers are going to be around in five years. They just want us to solve their problems. We have to be around to solve their problem for them. And so, survival is a precursor to that. So let's not put ourselves in position that could potentially compromise the enterprise along the way. It doesn't mean don't take risks, but think about how you calculate it.

**Lenny Rachitsky** (01:07:10):
I love how clear it is that this lesson and many of the lessons along these lines have come from just failure and things not working out and things breaking, which is the best outcome.

**Jason Droege** (01:07:19):
If you ever get on the other side of a high reward, high risk decision, it is so painful because you are just cooked. You are done, and often there's no way out.

**Lenny Rachitsky** (01:07:32):
Is there a story along those lines that comes to mind or an example of that?

**Jason Droege** (01:07:35):
Well, this is where it is together on why I try to be so I think you can spend a little bit of time thinking upfront to save yourself a lot of pain downstream. I had this business not worth detailing it, but after the bubble burst in 2001, I'm like, "I'm going to self-fund a business. I'm going to build a profitable business. I want to prove that I can do this." And we had started Scour, which had all the things we talked about. And so, what I did is I'm like, I was a golfer and frankly, there was nothing to do in tech.

**Jason Droege** (01:08:03):
So I started selling golf clubs on the internet and I was making real money and I might've learned more from this business than any other because I started on eBay and I was 22, and I didn't really understand that my margins would come down because anyone can do this, but I was one of the first ones to do it, so I was making a ton of money and then I built this business and then I just failed to recognize I had a lot of hubris. I was like, "Oh, if I could just buy all the used golf clubs in America, I can be the market maker for prices," and don't people do that?

**Lenny Rachitsky** (01:08:34):
I love this ambition. That's great.

**Jason Droege** (01:08:36):
And it's just like it's madness to actually think about the practicality of that. And so, I just didn't spend the time thinking and then I ended up in this business. The business was profitable, it got to a couple million of revenue, whatever, paid me a dividend for a while, but it was painful the entire way.

**Lenny Rachitsky** (01:08:51):
I love the spectrum of experiences you've had. You've sold golf clubs, you're helping achieve AGI, you could say. There's also a whole part of your career. We haven't talked about where you built tasers and body cams and drones and all these things. Also, peer-to-peer file sharing before anyone else. Final topic I just want to spend a little time on based on this experience is hiring and building teams, something that I know you have a really strong take on. That I've been hearing a lot on this podcast recently is this idea of it's more important to build the right team than find the most optimal top talent. Talk about that, why that's so interesting and important.

**Jason Droege** (01:09:31):
As of late, I've developed a more nuanced view of this, which is for certain roles, you absolutely need the right experience in this current market. You see this with researchers, because the market's moving so fast, you don't have time to train up some people, so you actually have to go find people either who have the right relationships with customers that you want to get or you have to, who might not check other boxes but are awesome at that, might not check the classic boxes that I think you're referencing of they're a problem solver, they can grow with the company, they have a high trajectory, et cetera. I would say that's 5% of the roles in the company, but very important whenever speed to market is important.

**Jason Droege** (01:10:09):
And then, for interviewing, I just interview for three things and I have to interview across all kinds of expertises, which is hard. I can't be an expert in everything. And so, I reduce it down to just three things, which is like, are you a curious problem solver and can you articulate that verbally? Can you work across people? Are you humble enough to work across and are you a good leader? And if you just do those three things, I think you have a pretty high chance of success, at least in an organization that I'm running, because the world's changing. So you do need people that are adaptable. So all the experience is not necessarily one-to-one relevant.

**Jason Droege** (01:10:53):
And then, the working across to your team point, this actually came up at Uber Eats. So when I was building the Uber Eats management team, I'm not sure if this was mentioned to you from that group, but whenever I would hire people, I was trying to compose almost like an organism of strengths and then minimize the conflicts. That management team for the most part outside of some of the operations side, but for the most part, that management team was the same management team from day one when we had nothing to $20 billion. And I just believed that the team, knowing each other's strengths and weaknesses and being able to compensate for each other was more important than the classic advice you get around, "Well, that person hasn't seen this much scale." And you're like, "Well, yeah, but can they learn it?" I learned it. So you do have to kind of believe in people a little bit, which is my job, not necessarily their job. And so, I mean, these are people systems. They're not straightforward rules-based things you can apply.

**Lenny Rachitsky** (01:11:49):
And I especially love this advice because there's all this talk about what skills will matter in this world of AI doing all our jobs, and it feels like these three buckets are maybe the same thing, just are they good at solving problems? Are they good leaders? Can they collaborate well with other people?

**Jason Droege** (01:12:02):
Yeah, I don't think that the core rise of humanity, it will change, and I think that these things are pretty core to how humans have been successful for a long time.

**Lenny Rachitsky** (01:12:11):
Speaking of that, I'm going to take us to a recurring segment of this podcast that I call AI Corner, where I ask folks this question, what's some way that you've found a use for AI in your day-to-day life in your work that makes you more effective, get more done, get better stuff done?

**Jason Droege** (01:12:28):
Honestly, when I came into Scale, so my history was in consumer and I've done some application level stuff with government, and this space is moving so quickly. AI is my, I use it as a tutor. As these new concepts come up, I have a lot of people in the company who can educate me on the nuances of the technicals of all of, excuse me, the technical nature of the data and the products, but they only have so much time. And honestly, there's new concepts coming up all the time and I need to stay on top of it.

**Jason Droege** (01:13:01):
So it might sound crazy, but a large percentage of my job is not dealing with the engineering issues related to AI. I'm managing an organization, but I love understanding it. It's one of the most enjoyable, rewarding parts of my job is to learn from all these AI researchers, but they don't always have the time to do it, so I use it as a tutor. I turn on voice mode and talk to it on my way into work. So I think that's probably the most impactful thing that I use it for that's also relevant to this topic.

**Lenny Rachitsky** (01:13:32):
I do exactly the same thing, especially when I'm prepping for this podcast. What exactly is this? I think about when you say this, I did an interview with the founders of Perplexity a few years ago asking about how they work at Perplexity, and the founders said that before, they were ruled, before they ask a question of anyone on the team, they have to ask AI first. And I was just like, "That's crazy." Now, it's so obvious. But back then, I was like, "That's an insane way of working. I've never heard of this before." Just a sign of how ahead of the curve they were.

**Jason Droege** (01:14:04):
Yeah, I think number two would be I'll take internal documents and I'll ask, what's the most important thing in this document? And I'm shocked, and then I'll read it and just double check, but I'm shocked at how good it is at just pulling out. There's so much in organizations that is like, I don't know what you want me to say and I don't know what I need to know, but we each have our own agendas, and so this matching of, and so then there's this huge broadcast problem where it's like, of all of the information you might want to receive, what's actually important to you? And so, I use it a lot for that too.

**Lenny Rachitsky** (01:14:39):
Amazing. That's a really good tip. I use it for legal documents, just like what do they know about what they're trying to do here for me or against me? Jason, is there anything else you wanted to share or leave listeners with, maybe double down on a point before we get to a very exciting lightning round?

**Jason Droege** (01:14:54):
Yeah, absolutely. I mean, I think the really important, the reason why I'm doing this, the reason why want to spend time here outside of wanting to be on the show for a while and being a long-term listener is, our long-time listener, excuse me, is there's a lot of amazing work going on at Scale. The teams are working super hard, we're delivering a ton of value for our customers. The public narrative has not represented the work that the people here are doing and the work that our customers are doing with what we're doing for them. And I just think that deserves the respect and reward that all those people are putting in, and we'd like people to know that.

**Lenny Rachitsky** (01:15:34):
I appreciate you saying all that. With that, we've reached our very exciting lightning round. We've got five questions for you. You ready?

**Jason Droege** (01:15:38):
Yeah, let's go for it.

**Lenny Rachitsky** (01:15:39):
What are two or three books that you find yourself recommending most to other people?

**Jason Droege** (01:15:44):
Some of this is going to sound interesting. The Selfish Gene is one of my favorite books.

**Lenny Rachitsky** (01:15:48):
Love that book. I don't know if anyone's ever mentioned, it was one of the most influential books for me too. So sorry, keep going.

**Jason Droege** (01:15:54):
Yes. I think Selfish Gene. Road Less Traveled, I've read more than once. I mean, it's just one of the classic human psychology book. And then, I think in business, I think Good to Great. It's not the read that you're going to be most excited to enjoy on a vacation, but it's pretty much right, and I think we should take advice from people who have analyzed these business problems before because not a lot's changed, but we keep acting like everything's changed.

**Lenny Rachitsky** (01:16:24):
What's crazy about that book, you look at all the companies they talk about, I haven't read in a while, but just the whole book is about companies that last, I believe, or maybe that's the other book, I don't know. But anyway, all the companies that they talk about, I don't know if they're still around. It's so hard for a business to last a long, long time.

**Jason Droege** (01:16:38):
I would also recommend Thinking Slow and Fast, that's the... Yes.

**Lenny Rachitsky** (01:16:38):
Thinking, Fast and Slow.

**Jason Droege** (01:16:43):
Thinking, Fast and Slow. Excuse me, sorry. It's been like a decade since I read it, but just in terms of point there being human biases are very important to understand.

**Lenny Rachitsky** (01:16:51):
What's really crazy to me about that book and Kahneman in general, someone asked them just, how's your life been impacted by learning all these biases humans have? He's like, "Not much. I have the same biases. Knowing them doesn't really help me avoid them."

**Jason Droege** (01:17:05):
See, I find myself checking myself. Whenever I get super convicted on something now I will be like, okay, what is the list of things that I'm inclined to do to try to catch myself? Because I think we're most inclined to have these bad decisions impulsively, which is what I think the book is largely about. I mean, it's a long book.

**Lenny Rachitsky** (01:17:28):
So long. Oh, my God. It feels like that's where AI can help us in the future. Just like, "Hey, Jason, are you sure this isn't framing a fact or whatever?"

**Jason Droege** (01:17:38):
Yes.

**Lenny Rachitsky** (01:17:38):
Okay. Next question. Do you have a favorite recent movie or TV show that you've really enjoyed?

**Jason Droege** (01:17:43):
Most of the movies I watch are with my kids, so I wish I had something deep and profound.

**Lenny Rachitsky** (01:17:50):
No, kids content also is a very acceptable-

**Jason Droege** (01:17:52):
The Formula 1 movie I thought was really good. I mean, it's a classic action movie. I don't think it informs anything in AI or business, but it's good to check out from the craziness of tech once in a while.

**Lenny Rachitsky** (01:18:04):
Is there a product you recently discovered that you really love? Could be an app, could be clothing, could be a kitchen gadget, anything along those lines?

**Jason Droege** (01:18:13):
VO3. Not totally new, but when I was in high school, I wanted to be a screenwriter. I actually grew up in the Bay Area and everybody was an engineer, but I wanted be a screenwriter. And so, I went back and I got the first page of one of my old scripts, which not good scripts, but I got the first page. I took a picture of the script and I fed it to VO3, and I said, "Make this scene," and it got it right.

**Lenny Rachitsky** (01:18:38):
Wow.

**Jason Droege** (01:18:39):
I was shocked. I was just absolutely shocked that you could just take a picture of a script. And so, now I'm thinking about that for how do I use these tools for family videos? Some of the grad tools now with making live images more active, I think are really interesting. I think they need one more step of iteration, but I think those are going to be really emotionally life-changing for people because just a little bit of movement in an image from a grandparent or a relative or whatever you haven't seen in a while, it really does make a big emotional impact on you.

**Lenny Rachitsky** (01:19:20):
I love that when you play with these tools, you probably can think about, oh, here's the people that help train this thing. Here's the people that helped on the problem that it had.

**Jason Droege** (01:19:26):
I was actually talking to someone who was working on VO3, and I told him the script thing and he goes, "Oh, actually scripts. Yeah, no, the way the data is formatted in a script, that would actually be very good." Because they start with set looks dark interior, this character says it in this raspy voice, and so it gives you all the instructions in the script.

**Lenny Rachitsky** (01:19:45):
Oh, man, just unlocked a whole new business unit right there. Two more questions. One is do you have a favorite life motto that you often think about, find useful in work or in life?

**Jason Droege** (01:19:57):
Yeah. The end is never the end. That's my favorite internal saying, and it goes to the comments before about survival being a precursor, surviving being a precursor to thriving. You got to survive before you thrive, which is your brain tells you, and along these entrepreneurial journeys, I think this is most applicable. I mean, this is the hardest journey anyone can go on. If you go on this journey for five years, you are mentally harder than 99.9% of the population. People don't understand the Chinese water torture of having self-doubt and having things go wrong, et cetera.

**Jason Droege** (01:20:31):
And so, more tactically, you get this when you're working out like in a day like, "Oh, I'm too tired. I need to stop." But the truth is is you can keep going and the world's going to keep spinning. So I find in the moments where it's just the hardest or you have this hard decision that seems impassable and your body, you're having this visceral reaction to this is impassable, just to remind yourself that I'm going to wake up tomorrow. This isn't the end. There's another end somewhere. I just find that to unlock me to be like, okay, there might not be a perfect solution, there might be an imperfect solution, but it's a solution so let's just keep going.

**Lenny Rachitsky** (01:21:06):
Final question. You helped create Uber Eats. I imagine you're still a power user of Uber Eats. You have a favorite restaurant on Uber Eats that maybe people should know about, maybe that you order most from?

**Jason Droege** (01:21:16):
I order a shocking amount of McDonald's actually. Despite my original story, it's the family treat in the house. I would say that that's probably the top thing that we order.

**Lenny Rachitsky** (01:21:31):
Oh, man, I'm worried for your health, but I love, I haven't had McDonald's so long. This is like, maybe I should give it another-

**Jason Droege** (01:21:37):
I mean, more practically we will order mixed greens or tender greens or something like that on a day-to-day basis, but I think that the more notable, surprising thing is is that despite my initial aversion to working with a global chain, it's a good treat once in a while. You just shouldn't have it all the time.

**Lenny Rachitsky** (01:21:57):
Jason, this was incredible. I really appreciate you making time for this. I'm really honored to be the first chat you've had since taking over at Scale. Where can folks find you online if they want to maybe reach out, learn more about what you're, I don't know, maybe join Scale. Where do you want to point people to and how can listeners be useful to you?

**Jason Droege** (01:22:14):
Yeah, absolutely. I'm @jdroege, J-D-R-O-E-G-E on X. That's probably the easiest way to follow me, keep up with things and you can shoot me a DM if you like. And so, I think that's how you would keep in touch and, sorry, what was your other question? Sorry.

**Lenny Rachitsky** (01:22:31):
If you're hiring, I don't know, where should people go check it out if you are, and then also just-

**Jason Droege** (01:22:34):
Absolutely. Just go to scale.com, go to our careers page, and we have 250 open roles. To the point about we're in business and we're growing, we're hiring a ton of people. Our data business is growing, our applications and services business is growing like crazy, and so we're going to need a lot of people to help us on that journey.

**Lenny Rachitsky** (01:22:54):
You guys just signed some insanely large contracts with the government I was reading.

**Jason Droege** (01:22:58):
Two $100 million contracts.

**Lenny Rachitsky** (01:23:01):
$100 million contracts.

**Jason Droege** (01:23:02):
100, yeah. We didn't sign just one. We signed two in one month, so yes, no, our federal business is doing well. Our enterprise business is doing well. Our international government's business is doing well. There's a lot of demand out there.

**Lenny Rachitsky** (01:23:16):
Some salespeople are getting some great commissions. Good job. Jason, thank you so much for being here.

**Jason Droege** (01:23:21):
Yeah, thank you. Honor to be a guest here. Super excited to be with you, especially so early in the journey, or at least my journey here leading Scale.

**Lenny Rachitsky** (01:23:32):
Appreciate it. Thanks for coming. Thanks for joining us. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [3/17] Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)

**Lenny Rachitsky** (00:00:00):
It feels like something has changed internally at Google. Just last week, Google Gemini hit the number one app in the App Store. I feel like nobody saw this coming.

**Robby Stein** (00:00:08):
Google's mission around have any information be universally accessible, this very enduring, very motivating thing, and it feels like with the AI moment, we can actually achieve that more than ever before. What I'm feeling now is just an incredible sense of focus and urgency. Things have hit a tipping point where these models are now truly able to deliver for consumers.

**Lenny Rachitsky** (00:00:26):
As ChatGPT emerged over the past couple of years, as Perplexity emerged, a lot of people were just like, "Google is dead. Nobody wants to sit through search results and click links."

**Robby Stein** (00:00:35):
The core Google search isn't really changing, in my opinion. We're not seeing that people come to search for just ridiculously wide set of things. They want a specific phone number, they want a price for something, they want to get directions. I think the vastness of that is underappreciated by many people. AI is expansionary. There's actually just more and more questions being asked and curiosity that can be fulfilled now with AI.

**Lenny Rachitsky** (00:00:54):
You've built a lot of very successful products. You used this phrase: embodying relentless improvement.

**Robby Stein** (00:00:59):
You need to be the physical manifestation of two pieces of things. One is just relentlessness, just complete effort that is always exerted in a direction of positive productivity. And then the second is make things better. You have to always make things better. You're never content.

**Lenny Rachitsky** (00:01:12):
You build and launch Stories at Instagram back in the day is quite controversial because it basically took what Snapchat was doing really well and then like, "Hey, let's bring it to Instagram."

**Robby Stein** (00:01:21):
Not every great thing is going to be invented by you. Facebook probably created the modern feed, but there's a feed for every single product. At the end of the day, you're just robbing your user base of the opportunity to have a better product.

**Lenny Rachitsky** (00:01:33):
Today my guest is Robby Stein, Robby's VP of Product for Google Search and is responsible for essentially the entire Google search experience, including the new AI Overviews, AI Mode, multimodal AI experiences like Google Lens, the ranking algorithm, and a lot more. He's at the forefront of one of the biggest shifts in Google's history, and has already made a massive dent in Google's trajectory. He's also made a massive dent in the trajectory of Instagram where he was head of product, and led the launch of Instagram Stories and Reels and Close Friends, and through that, grew Instagram to half a billion daily active users. He's also on the founding team of Artifact with Mike Krieger and Kevin Systrom. Started two companies of his own. Very few people have had this level of impact on two global consumer products at this scale. And Robby shares all of the biggest lessons that he's learned about building great and successful consumer products, along with a bunch of insights into where Google is headed in the world of AI.

**Robby Stein** (00:04:53):
Thanks so much for having me.

**Lenny Rachitsky** (00:04:54):
This is such a cool week to be recording this podcast. So just last week, Gemini, Google Gemini hit the number one app in the App Store. I have it right here, it's still number one in the App Store. It's above ChatGPT. I feel like nobody saw this coming. I feel like everyone's always like, "Google, what have you guys been doing? You guys build all this amazing tech and why didn't you have anything working in consumer? Why is ChatGPT doing? Why are all these amazing companies doing better than Google?"

**Lenny Rachitsky** (00:05:20):
So first of all, let me just say congrats on, I know this isn't all you. I imagine you had some part in this, so just congrats.

**Robby Stein** (00:05:26):
Many, many more people, yes.

**Lenny Rachitsky** (00:05:28):
It feels like something has changed internally at Google. It feels like things are starting to really work, especially on the AI consumer side. So in terms of the growth, is Nano Banana the source of a lot of this recent growth or is there something else?

**Robby Stein** (00:05:40):
People are really excited about Nano Banana to be clear, very much so, but I think also people are recognizing that there's just so many cool things that you can do across the Google set of products and they've become quite powerful. I'm always shocked, even for things in search, people, we think they're very obvious. They sit right in the core search experience and then on X, I'll go look and like, "Oh, I just found out about this AI thing," and it seems very obvious, but I think a lot of people are just discovering quite how powerful these tools are.

**Lenny Rachitsky** (00:06:07):
Now. So to go one level deeper, to your point, there's been all this incredible tech. You guys wrote the original transformers paper that have powered so much of the innovation and it's just like, "Where's Google been? And actually, why aren't they building the thing that's winning?"

**Lenny Rachitsky** (00:06:20):
What has changed? Is it just like, okay, has there been major reorgs? Has there been new leaders put in place? Is there just a new philosophy in the past couple of years that have led to this moment where Gemini is now the top app in the world?

**Robby Stein** (00:06:32):
Yeah, I mean, look, I've been to Google now, this is my second time at Google, so I started at Google in 2007, done a bunch of things in between, and I've been back at Google now, so I can't speak to that whole period for many, many years back to today. But what I can tell you about what I'm feeling now is just an incredible sense of focus and urgency to deliver great products quickly. I think that that is in part leadership for sure. I think the people who are, we work very closely with our partners at DeepMind and Google DeepMind. We work very closely obviously across the organization and there's just an incredible group of people and also an incredible group of researchers and technical thinkers who've been thinking about this for a while. When you have that energy, and I think the product teams and the tech, the research groups are working really closely together, we're able to move and we're getting a lot done.

**Robby Stein** (00:07:19):
I don't think there's any one thing that has happened. I think that a lot of times people ascribe a lot of momentum to a one time change or a single person. I find a lot of this is actually this compounding effect when you think about just every month ruthlessly improving the product or the models and just every day getting better, and then it just hits this tipping point where people just like it, they use it more, they enjoy it. And that's more of the feeling that I've had is just we've had, I think the right investment and focus and then it just hit a moment where people are seeing the effects of that now.

**Lenny Rachitsky** (00:07:52):
As ChatGPT emerged over the past couple of years, as Perplexity emerged and all these other chatbots, a lot of people were just like, "Google is dead. Nobody wants to sit through search results and click links. Why not just get your answer right there?"

**Lenny Rachitsky** (00:08:06):
And it feels like that's not at all happening. It feels like you guys are doing just fine. What can you share about just the, I don't know, the state of Google search specifically, and then we'll talk about AI Mode. Just how is traffic going, how is search going considering all these things are out there, and just what are you seeing in the data since the launch of say ChatGPT?

**Robby Stein** (00:08:24):
Yeah. Well, what's interesting is people come to search for just ridiculously wide set of things, like all kinds of things. They want specific phone number, they want a price for something, they want to get directions, they want to find a payment web page for their taxes. Every possible thing you can imagine. I think the vastness of that is underappreciated by many people. And what we see is that it's not changing. AI hasn't really changed those foundational needs in many ways, and what we're finding is that AI is expansionary, and so there's actually just more and more questions being asked and curiosity that can be fulfilled now with AI. And so that's where you get the growth.

**Robby Stein** (00:09:00):
All the core Google search isn't really changing, in my opinion. We're not seeing that, but you're getting this expansion moment. What we're seeing is a few examples is you can now take a picture of something and ask about anything you see. And Google Lens, one of the fastest growing products out there, it's growing 70% year-over-year increase in visual searches, which is already at a massive scale. It's billions and billions and billions of searching in that way.

**Robby Stein** (00:09:24):
But you can take a picture of your shoes, say, "Where can I buy this?"

**Robby Stein** (00:09:27):
Or take a picture of your homework, say, "I'm stuck on question two."

**Robby Stein** (00:09:29):
And then just take a picture of your bookshelf and say, "What are the books I should get based on these books?" And AI can help you with those things now, just an example of I think why there's so much growth left and why we're so excited.

**Lenny Rachitsky** (00:09:41):
Okay, so you're not seeing the death of search.

**Robby Stein** (00:09:45):
No.

**Lenny Rachitsky** (00:09:45):
And along the same lines, you guys recently launched AI Mode, which I don't think enough people are talking about. I think you get there at google.com/ai, is that the right URL?

**Robby Stein** (00:09:53):
Yep.

**Lenny Rachitsky** (00:09:54):
Okay, cool. I've been playing with it as we were prepping for this conversation. It's really incredible. I asked it what is the best newsletter on product and growth and it's very smart. Said Lenny's Newsletter. So that's my eval.

**Robby Stein** (00:10:06):
Fantastic. Okay, one of one, perfect eval.

**Lenny Rachitsky** (00:10:10):
It's perfect. Also, just if you go to it, there's these recommendations for things to ask it that are just like, "Wait, how did you know I care about this stuff?" So it's like, "Help me switch to product management," just on the front page.

**Lenny Rachitsky** (00:10:21):
I'm like, "How did you know?" And it tells you that it's based on your Google activity. Talk about just what people should know about AI Mode, maybe what they don't really understand about the power of this thing.

**Robby Stein** (00:10:31):
I can tell you there's three big components to how we can think about AI search and the next generation of search experiences. One is obviously AI Overviews, which are the quick and fast AI you get at the top of the page many people have seen, and that's obviously been something growing very, very quickly. This is when you ask a natural question, you just put it into Google, you get this AI now, it's really helpful for people.

**Robby Stein** (00:10:51):
The second is around multimodal. This is visual search and lens. That's the other big piece. You go to the camera in the Google app and that's seeing a bunch of growth. And then really with AI Mode, it really brings it all together. It creates an end-to-end frontier search experience on state-of-the-art models to really truly let you ask anything of Google search. You can go back and forth, you can have a conversation and it taps into and is specially designed for search. What does that mean?

**Robby Stein** (00:11:16):
And one of the cool things that I think it does is it's able to understand all of this incredibly rich information that's within Google. There's 50 billion products in the Google shopping graph, for instance. They're updated 2 billion times an hour by merchants with live prices. You have 250 million places in maps. You have all of the finance information, and not to mention, you have the entire context of the web and how to connect to it so that you can get context but then go deeper. You put all of that into this brain that is effectively this way to talk to Google and get at this knowledge. And that's really what you can do now.

**Robby Stein** (00:11:52):
You can ask anything on your mind and it'll use all of this information to hopefully give you super high quality and informed information as best as we can, and you can use it directly at this google.com/ai. But it's also been integrated into our core experiences too. We announced you can get to it really easily if you can ask follow-up questions of AI Overviews right into AI Mode now. Same for the lens stuff. Take a picture takes you to AI Modes, you can have this back, you can ask follow-up questions and go there too. So it's increasingly integrated experience into the core part of the product.

**Lenny Rachitsky** (00:12:24):
I imagine much of this is wait and see how people use it, but what's the vision of how all these things connect? Is the idea continue having this AI Mode on the side, AI Overviews at the top and then this multimodal experience, or is there a vision of somehow pushing these together even more over time?

**Robby Stein** (00:12:40):
I think there's an opportunity for these to come closer together. I think that's what AI Mode represents, at least for the core AI experiences, but I think of them is very complimentary to the core search product. You should be able to not have to think about where you're asking a question ultimately, you just go to Google. Today, if you put in whatever you want, we're actually starting to use much of the power behind AI Mode right in AI Overviews. So you can just ask really hard, you could put a five sentence question right into Google search. You can try it and then it should trigger AI at the top. It's a preview, and then you can go deeper into AI Mode and have this back and forth. So that's how these things connect.

**Robby Stein** (00:13:15):
Same for your camera. So if you take a picture of something, "What's this plant?" Or, "How do I buy these shoes?" It should take you to an AI little preview. And then if you go deeper, again, it's powered by AI Mode. You can have that back and forth, so you shouldn't have to think about that. It should feel like a consistent simple product experience ultimately, but obviously this is a new thing for us, and so we wanted to start it in a way that people could use and give us feedback with something like a direct entry point like google.com/ai.

**Lenny Rachitsky** (00:13:41):
I recently had Brian Balfour on the podcast and he showed this quote that's really stuck with me that I think about as you talk about all this, it was by Alex Rampell, this idea that startups is a game of getting distribution before incumbents can innovate fast enough.

**Lenny Rachitsky** (00:13:55):
And it feels like you guys are finally there where it's like, "Oh man, now here comes Google." I don't know if I have a question here, but it just feels like there's been all this time for people to find distribution, and now it's like, okay, now Google is coming.

**Robby Stein** (00:14:07):
What we found is that people are asking these questions in Google. They're trying to get this out of Google. And so if you can just have an AI that's powerful enough to answer a really hard calculation someone's trying to figure out, or take a picture of multiple choice homework question for a chemistry question, people are doing this. And so now that you have this really sophisticated AI that's based on our frontier models, we can just handle increasingly more and more stuff for people and so hopefully that's the more natural on ramp here. And then we just need to make it easy enough for people to use, because these are new products, and people are used to using Google in a specific way.

**Robby Stein** (00:14:38):
They type in keywords, we call it sometimes keyword ease, but you can actually use natural language in Google. That's the biggest shift. We're seeing people asking real long, hard, complex questions. You just don't think, "Oh, I can go to Google and type in what's a great place for a date night? I already went to these four restaurants. I'm looking for outdoor dining and my friend has this allergy." You could put that into Google. And I think that's the kind of thing that we're excited to continue to make easy for people.

**Lenny Rachitsky** (00:15:03):
It's interesting, and we've come around to back in the day there was Ask Jeeves, which was this whole just ask a question as if you're asking a human and then it'll give you a really good answer.

**Lenny Rachitsky** (00:15:12):
And then we moved into Google just, "No, no, just type the thing you want and figure out how Google likes it."

**Lenny Rachitsky** (00:15:17):
And now we're back to, "Okay, just ask your question and it'll give you a really good answer."

**Robby Stein** (00:15:20):
Yeah, Ask Jeeves was surprisingly prescient on that, huh? They had material, they had something way before its time that we think looks to rally around now.

**Lenny Rachitsky** (00:15:29):
Oh, man. What's your take on this whole rise of AEO, GEO, which is this evolution of SEO? I'm guessing your answer is going to be just create awesome stuff and don't worry about it, but there's a whole skill of getting to show up in these answers. Thoughts on what people should be thinking about here?

**Robby Stein** (00:15:47):
Sure. I mean, I can give you a little bit of under the hood how this stuff works because I do think that helps people understand what to do, but when our AI constructs a response, it's actually trying to, it does something called query fan-out where the model uses Google search as a tool to find to do other querying. Maybe you're asking about specific shoes, it'll add up and append all of these other queries like maybe dozens of queries and start searching basically in the background. And it'll make requests to our data back end, so if it needs real time information, it'll go do that. And so at the end of the day, actually something searching, it's not a person, but there's searches happening and then each search is paired with content. And so if for a given search your web page is designed to be extremely helpful and you can look up Google's human rater guidelines and read, it's a very long document that's been thoughtfully crafted for decades now around what makes great information.

**Robby Stein** (00:16:40):
This is something Google has studied more than anyone, and it's like, do you satisfy the user intent, what they're trying to get? Do you have sources? Do you cite your information? Is it original or is it repeating things that have been repeated 500 times? And there's these best practices that I think still do largely apply because it's going to ultimately come down to an AI is doing research and finding information. And a lot of the core signals, is this a good piece of information for the question? They're still valid, they're still extremely valid and extremely useful, and that will produce a response where you're more likely to show up in those experiences now.

**Robby Stein** (00:17:14):
I think the only thing I would give advice to would be think about what people are using AI for. I mentioned this as an expansionary moment. It seems to be that people are asking a lot more questions now, particularly around things like advice, or how to, or more complex needs versus maybe more simple things. And so if I were a creator, I would be thinking, what kind of content is someone using AI for? And then how could my content be the best for that given set of needs now? And I think that's a really tangible way of thinking about it.

**Lenny Rachitsky** (00:17:45):
It's interesting your point about how it goes in searches. When you use it, it's searching a thousand pages or something like that. Is that just a different core mechanic to how other popular chatbots work because the others don't go search a bunch of websites as you're asking?

**Robby Stein** (00:18:00):
Yeah. This is something that we've done uniquely for our AI. It obviously has the ability to use parametric memory and thinking and reasoning and all the things a model does, but one of the things that makes it unique for designing it specifically for informational tasks, we wanted to be the best at informational needs, that's what's Google's all about, and so how does it find information? How does it know if information is right? How does it check its work? These are all things that we built into the model, and so there is a unique access to Google. Obviously, it's part of Google search, so it's Google search signals everything from spam, what's content that could be spam? And we don't want to probably use in a response all the way to, wow, this is the most authoritative helpful piece of information. We're going to link to it and we're going to explain, hey, according to this website, check out that information and then you're going to go probably go see that yourself. That's how we've thought about designing this.

**Lenny Rachitsky** (00:18:51):
You've worked on a lot of AI products at this point, and it's not just Google or Artifact and Instagram, you did a lot of AI stuff. What's something you've learned about building AI products that you find maybe people don't truly understand, maybe something that's surprised you by building successful AI products?

**Robby Stein** (00:19:07):
I think the most recent one, and this is true, something even within the last week or two, is that it's so obvious how human-like the interface is becoming with how you can communicate and steer AI. I think it used to be even just months back that you had to do a lot of work to get the AI to do the thing you're trying to get it to do, right? You had to do these incantations, you had to prompt in a really specific way. People would have all these hacks like, "Hey, act like you're a coach and you do these things," and you have to really push it, or to use a tool more on the technical side. You had to do post-training, you had to take this foundational model and you had to show it data, you had to train it and actually update its weights to do more sophisticated things.

**Robby Stein** (00:19:51):
Tell it, "Hey, here's documentation for an API. If you ever have a problem, ping this API. Here's the data," as if it's an engineer that you had that you could talk to and it would have no idea what to do with that, or it would have some idea and wouldn't really do it.

**Robby Stein** (00:20:05):
But increasingly, you can just use language. Almost if you were to write up an order, you could be like, "Wow, I'm a new startup. Here's my data internally. Here are the APIs to it. Here's the schema and the URL. Here's when to use it. By the way, make sure that if you get this kind of a question, you really make sure to get it right." And that'll end up doing a lot in the model.

**Robby Stein** (00:20:28):
The model's been now encoded to be able to say, "Okay, I'm going to use more reasoning or thinking budget for that kind of a question."

**Robby Stein** (00:20:35):
Or, "I'm going to use tools or code, use code execution in order to connect to this API I'm told about." That's a relatively new thing. So I think it's going to open up a lot of this democratization of accessing these models and building incredible things because you don't even need to do a lot. To get the most sophisticated outcomes increasingly, I don't think you need to do a lot of this heavy duty fine-tuning.

**Lenny Rachitsky** (00:20:58):
It makes me think about, I had this recent guest, Nesrine Changuel, on the podcast. She was a PM at Google, she worked on Google Meet, she was a delight PM working on at making products more delightful. And she talked about the reason Google Meet did so well and is now feels like it's killing Zoom is they compared the experience of Google meet to a human meeting versus making it the best possible video conference, make this as good as a human experience. And that's interesting what you're talking about, how that's almost the goal here with AI is just make you feel like you're just talking to a person.

**Robby Stein** (00:21:27):
Exactly.

**Lenny Rachitsky** (00:21:28):
Might be obvious, but think about that. Okay, let me zoom out and talk about, and let's talk about just broader lessons you've learned over the course of your career. You've built a lot of very successful products, which I've shared in the intro at this point.

**Robby Stein** (00:21:44):
Many also on the other side of the spectrum, we got the whole portfolio.

**Lenny Rachitsky** (00:21:48):
Okay, perfect. We'll talk about some of that. I asked you as we were getting ready for this conversation, what's one thing you wanted to get across in this conversation? What's something you think would be really helpful for product builders to hear to help them build more successful products? And you used this phrase: embodying relentless improvement. Can you just talk about that? What does that mean? Why is this so important?

**Robby Stein** (00:22:08):
Of course, I mean, I think that you need to be the physical manifestation of two pieces of things. One is just relentlessness, just complete effort, but is always exerted in a direction of positive productivity. And then the second is make things better. You have to always make things better, you're never content. And I think this actually came out of a story, a little bit of a funny story where I was at Instagram at the time doing a big all team meeting, one of my first, and they had this icebreaker, what's one word to describe yourself?

**Robby Stein** (00:22:35):
And so in the backstage area, I texted my wife really quick. I was like, "Hey, just one word to describe me, first thing that comes to your mind."

**Robby Stein** (00:22:42):
And she just wrote back, "Dissatisfied."

**Robby Stein** (00:22:45):
I was chuckling in the back room because I was first of all kind of offended because I was like, "It's not loving, caring, something good?" And then I saw her little bubble thing.

**Robby Stein** (00:22:56):
She's like, "Okay, there's more." And then she wrote me this really thoughtful thing that was like, "It's not that you're just unhappy. It's like you want the world to be better. You're driven out of a deep desire. It's that you feel this sense of dissatisfaction with what the world gives you. You want to make it better, and you're pushed and motivated to do that."

**Robby Stein** (00:23:17):
And I thought about that after. And it wasn't until we built a bunch of products, some that didn't do well, some that have had a lot of really large success now, billions of people use them, where it felt like one of the big differences, obviously a lot of it is just the conditions of the product and a little bit of luck here and there too. But for the things that went well, there was always this spirit of just we're going to get it eventually if we just make two more moves to make it better. And then eventually, as I talked about before earlier in our conversation, you get this tipping point where it just tips over into being net useful to people because of just that amount of compounding effort that you put into something because you're just always so... You're the harshest critic and the most dissatisfied person in the room about your own work basically.

**Robby Stein** (00:23:56):
And I think that's really meaningful. And there's this other incredible story that Tony Fadell told on a TED Talk 10 years ago. You can look it up. I think it's something around Think Younger as a title. And he talks about what it means that as we grow up in age and become grownups, I have two little kids so that's something I think about a lot. We habituate to everything. We accept and we tolerate what the world gives us everywhere, and we just go, "Oh, that kind of sucks. Oh, well," we shrug our shoulders and we move on.

**Robby Stein** (00:24:27):
But if you don't do that and you ask, "Why? This sucks, why am I tolerating this and how do I make it better?" He has this incredible story about going grocery shopping, and he goes on for 10 minutes about this story almost it felt like where he talks about getting a piece of fruit like a plum or a peach, and how it has that sticker on it and it's got that sticker and who put that sticker there?

**Robby Stein** (00:24:51):
And then when you get home, you take your fruit out of your bag, you're ready to eat it, you're all excited, you stick your thumb under the sticker, it punctures the flesh. He goes into just incredible detail about how it punctures the flesh of the fruit. The sticker comes off now, the fruit's bleeding, then you flick the sticker. The sticker misses the garbage, you bend over and pick it up, you put the sticker back in.

**Robby Stein** (00:25:17):
And I was like, "Wow, that is embodying this mentality of just why is this here? How can this be better?" And I think the best product people, the best thinkers in the space, that's how they think, in my opinion.

**Lenny Rachitsky** (00:25:32):
I imagine there are many examples of you doing this in the many products you worked on. Is there one that comes to mind as a good example of this inaction of this actually working really well and delivering something really huge?

**Robby Stein** (00:25:44):
I mean, honestly, a big thing is working on AI Mode. I think a lot of it was we saw in AI Overviews that people were trying to ask harder questions and we weren't able to answer a bunch of them, or AI Overviews just didn't show up. And so a bunch of us sat around and we're like, "Why can't you just do this for everything?"

**Robby Stein** (00:26:04):
Instead of saying, "Oh, we don't need to solve for that," or, "That's not something that's in the most addressable next thing."

**Robby Stein** (00:26:12):
It's like we actually saw people in the query stream putting the words AI at the end of their queries because they're trying to get the AI to do the thing. We would look at that and be like, "This is ridiculous. We need to build something here."

**Robby Stein** (00:26:27):
And that was one of the big motivations, was actually identifying that user problem, being very disgruntled on behalf of the user. We're just failing the user every day. We are not helping them actually get their thing better understood, and we're going to go build a whole thing because of it, because that's hard to do by the way, to build all of that. But it just was so obvious that that's what we needed to do.

**Lenny Rachitsky** (00:26:52):
There's two buckets of people. Let's say hypothetically, one bucket is just make things better, make amazing experiences, you're going to do great. There's another bucket that's like drive metrics, drive goals, hit our KPIs. I know what you're not saying is just work on things, just make things better, relentlessly, make things better. How do you just think about, I guess that overlap of okay, makes things better, but also here's what we really, here's the strategy, here's the vision. How do you think?

**Robby Stein** (00:27:18):
Yeah, I don't think of them as an or. I think they have to be intersected because basically the way I think about it is you actually start with a problem or the inverse of that, which is a vision, but they're connected. Most great companies, most great products come out of a problem, but out of the problem becomes like, "Here's a better way." What if instead of this crappy thing or way of living or thing that we all tolerate and accept, some entrepreneur comes up and says, "What if we did this other thing?" So it comes out of this dissatisfaction and this sense of better that you need to make things better, but then you're going to build, and at the end of the day, you need your instrumentation to know if you're on the right track.

**Robby Stein** (00:27:58):
And that's where you bring tools like, okay, you build your first version of the product, do people like it? And then each product goes through its journey. The way understand that people like it is you scrutinize. Typically, you talk to people, but you also add some analytical tools there. You might look at something like a J-curve. This is the retention, the percentage of people still using the product day seven, day 30, day 90, and does it flatten or do people just drip out of there? Over time, it's just not exciting people. And that would go to zero if on a long enough timeline, no one's going to use it. You don't get past that, you toast right then. Okay, some people are doing it, okay, great. We need more people to do it, and it needs to be good enough that people talk about it and then it grows. And so that's another gate.

**Robby Stein** (00:28:44):
And then there's another one which is, well, how big can this get actually, is it a small thing? Is it a medium thing? And I think most companies, you have an aspiration of being big, but you can't start big. Everyone's got to go through that journey. No product has started big. Even ones that get big really quickly, even a week quickly, they had something. And then even internally, they started small. They started small with a hundred to 100 people, and so you have to be metrics focused, I think in order to know if you're doing the right thing.

**Robby Stein** (00:29:09):
And then the other thing is, on the other side of the spectrum, you're running a big thing, and there, you need metrics to be your guide. If your product, let's say, let's say our core metrics down 5% this week, it's like, well, what's going on? And so you be really close to root cause analysis there and say, "Well, actually it turns out that it's an issue. Is it in a region? Is it on a device? Is it in a demographic? Is it in a use case? Where does my problem lie?"

**Robby Stein** (00:29:33):
And then when you get to it, you understand the problem and then this improvement thing comes back where it's like, "Okay, I'm going to fix that thing. What's the treatment for that disease?" And then you're back to growth again, and so you need this and you always are looking at what's the system that I'm working on and what are my instruments? I'm a pilot to know if this thing is going and flying correctly, but then it doesn't tell you exactly what to do, you have to thank for yourself how to make it better. I can just show you a little bit of the way.

**Lenny Rachitsky** (00:30:05):
I love that you just gave a master class on just how to prioritize and pick what to work on. I want to go on a quick tangent. Speaking of products that have done really well and become really big, Stories, you build and launched Stories at Instagram. It's quite an infamous product launch back in the day, it was quite controversial because it basically took what Snapchat was doing really well and then, "Hey, let's bring it to Instagram," and it was not great for Snapchat. Now that it was so long ago and just, it's so far in the past, I'm so curious just to hear about that time reflecting on just that decision, what you guys talked about, how you decided to go ahead with that and anything just, I don't know, you think about looking back at that.

**Robby Stein** (00:30:47):
I think there's a couple of really important lessons from that launch. And I mean we went on afterwards to launch Reels, a bunch of updates to direct messaging, we had feed rank game. There was just a huge era there when I was there between 2016 and 2021 or so where just so many new products got built. I think an interesting lesson in all of those, and particularly in Stories was you have to really understand why someone uses your product and know when something is actually an existential question because there's just a better format or a different way of doing something that has worked and works and you need to figure out what that might mean for you, because not every great thing is going to be invented by you. But I think that a lot of these things are, they can become formats that you can make your own and you need to learn from the world and what's happening out there in order for your product to always give the best thing to its users.

**Robby Stein** (00:31:41):
And so for Stories, we looked at Instagram like, what's the point of Instagram? It is sharing your life and connecting with people ultimately. And if there's a way to do that, that lowers the pressure because it doesn't have likes or it's just ephemeral format and it's optimized well for mobile because it's this full screen experience. It's a really great format and kudos to Snapchat for inventing it. We didn't think of that as a deterrent, that we had to go make Instagram photo clock. And actually, there were early versions of this idea where you try to take the core Instagram feed and make it ephemeral. And whenever you try to mix a core product that's very cemented in someone's mind and physically looks a specific way and you're trying to make, contort it to do something new, it's usually a bad recipe. And so we knew we needed to do something new and then it was so clearly was critical to the core essence of what the product could do, could fit in naturally.

**Robby Stein** (00:32:39):
But the question was how do we make it our own? And how do we build on this? And so if you think, there were a bunch of things that we did that made it Instagram. For example, it had different creative tools and it had things like neon drawing and these really sophisticated filters that people loved. We also looked at this talk about being dissatisfied. People took, a lot of times they want their main camera to take a picture of something and then they want to upload it to Instagram because they want to save it and they want it to be in a very high quality, high resolution photo, because it's a memory. And Snapchat at the time didn't allow you to upload photos, it was like you have to use the Snap camera. And so we made a bunch of decisions like that where why don't you just let people upload their photo? This is back to the dissatisfied point, that's frustrating.

**Robby Stein** (00:33:22):
Or there's another example where you couldn't pause if you were consuming a story. You couldn't pause it, it just would go through and be done because it was this ephemeral thing and you wanted to create safety. Why can't you just pause? It goes by too fast. So we added this pause, it's such a small thing, but you put your finger down to pause the story now. And so there were a whole set of those things that were shipped that made Stories feel Instagram. It wasn't like you just had some other thing. And then it turns out that worked incredibly well, and so much to the fact that someone on the team mentioned that they always felt like at the time, they didn't realize it, but it was almost like it was missing the story size holes at the top of the page and it completed the product in some weird way for them. And so that was, I think an important lesson.

**Lenny Rachitsky** (00:34:05):
Instagram definitely got a lot of hate for that moment from a lot of founders. It was just like, "Hey, you guys just stole this idea and that sucks."

**Lenny Rachitsky** (00:34:13):
How did you guys just deal with that internally? It was just this is, "We got to do this. We got to focus on our shareholders and grow this thing," and that's how it goes sometimes?

**Robby Stein** (00:34:19):
I mean, I think it's more that we're focused on our users and the people who are loving Instagram and it's denying them the opportunity to have an easy way to just share a photo and have the thing go away. I mean, that's ultimately what we were trying to add. At the end of the day, that is a format that people adopt. In the same way that you think about feeds, I think we talked about this at the time too when we shipped it. Facebook probably created the modern feed, but there's a feed for every single product. There's a LinkedIn feed and there's a feed for DoorDash.

**Robby Stein** (00:34:53):
These things become core primitives quickly and formats, and then at the end of the day, you're just robbing your user base of the opportunity to have a better product if you're not making the best possible product for your use cases. And for Instagram, it's used differently. People use Instagram differently than they use other products. And it turns out that there were these experiences in WhatsApp and in Messenger and in many other social products over time, and they all were used differently actually, which is fascinating.

**Lenny Rachitsky** (00:35:21):
Something else I want to talk about is you came into two products that were already doing really well, Instagram and Google. And on the Instagram side, a transformative growth and improvement. Google is happening, we're in the middle of the improvement and growth you're driving. Not a lot of people get to do this where they go into an existing product, make it grow significantly. A lot of people want to do this. They have a product that's been around for a long time. Hey, how do we make this grow and be more successful? Is there anything specifically that you've learned about just coming into an existing product, figuring out where the big opportunities are and then just hockey-sticking growth? Because this is what everyone wants to do.

**Robby Stein** (00:35:57):
There's a couple lessons here. And I think, by the way, the first lesson is to be humble always because it's extremely incredible to be able to work on products that have such impact on people. I view product like golf, you're always one stroke away from shanking. And as soon as you think you're good, you're not, you don't know anything. The world changes quickly. You have to always be a servant to your user base and the people that are out there and learn from them. The first thing I always do and think about is you get in touch in terms of why are people using this product, and where are the areas of growth? And so usually even in a big product or a mature in a complex system, there's a part of it that's growing. There's a part of it that's mature, there could be a part of it that's declining or isn't growing as much.

**Robby Stein** (00:36:42):
Certainly in Instagram, there's been a big shift over the years of sharing into public very large broadcast posts and feed into these more lightweight formats like Stories and DM actually private sharing as well. And so you have to observe that because every month, every year, the world changes, people's needs change. First thing you do is you get a sense of what do people want out of this product? What's its true essence? I think a lot about this job to be done framework, which is one of the things that I'm a big fan of and Clayton Christensen's book on Competing Against Luck is one of my favorite books on this topic where you have to really be a student of causation. Why is someone using this product? What are they doing with it and what are they trying to get done with it?

**Robby Stein** (00:37:26):
And that usually leads you to do bigger next stage ideas, and it removes this belief that you need to solve the problem with the current tools. In the Instagram version, it was like you have to make a square photo do more for people. That would be how you increment the product. Or in Google's example, there's something very specific with the core search experience that needs to change, it's a subtle tweak. You have to think, well, what's the big thing? Someone's trying to ask a really hard question out of Google? What's the best way to do that for them? And so it makes you think more first principled and that's the first basis of this.

**Robby Stein** (00:38:04):
And then once from first principles, you're like, "Oh, this newer thing." And it could be a shift, it could be a new form. In many ways, the AI version of Google and Stories and Reels, they're all similar in that they're new formats in the world that people are expecting and wanting more of.

**Robby Stein** (00:38:18):
And by adding them, it becomes complementary, not replacement. And in both cases, Stories didn't replace Instagram, it expanded in the same way we're seeing for AI. And so what's interesting is then you think, well, how do I bring that into my world? You have this big mature product. The best way I've seen is by making it complimentary, having it be a core part of the experience, but clearly defined as a distinctive thing that has its own attributes associated with it because people think spatially. So if you have a feed and you have holes with pictures, they expect those holes to do things. And so if you make one of those holes with a little clock and that one goes away the next day or you can't like it or it operates differently than the other parts of your feed, it's going to be super confusing for people. It sucks.

**Robby Stein** (00:38:59):
And so you have to add product carefully, but it needs to feel coherent but different. Stories, it has similar aesthetic. It obviously uses your camera roll in the same way it works that you can share it in DM, it works in the system, but it has a different primitive in the same way Google AI, it's a full page experience that you can pop out now. You can have follow up conversation with it. People have a set of expectations you need to snap to for those use cases. And then you are constantly learning how to best make these new products work within your world.

**Robby Stein** (00:39:31):
You never just want to snap in something that's working, you have to make it work for your users, your expectations, and what people are trying to do with your product. It's actually one of the things I see people fail on the most is they assume something working for one system will work in your world, but someone else's system is on totally the types of users they have with the consumer expectation of that product, that's totally different set of expectations. You have to respect that and say, "What can we learn from that," and bring it here. I guess if you were to talk about the method that I've seen now or twice, I guess that's how these products have developed.

**Lenny Rachitsky** (00:40:09):
I love this topic. It makes me think about just this balance. People always try to find between optimizing something they've already got versus trying to take a big bet on something. You've had so many examples where you've taken a big bet on something totally new and it's worked out incredibly well. Do you have just a heuristic in how you structure teams and prioritize across, okay, we have amazing Google experience today, what percentage of resources go into improving that versus trying something totally new?

**Robby Stein** (00:40:35):
That's one where I actually do feel like the more analytical, systematic thinking helps a lot because you're trying to produce value in the world, you want to quantify it some way. And so if you're seeing this growth curve and you're trying to understand, wow, people are using it more and more to liken this product. And when products are young, they grow, and then eventually things mature. You can break out product suites and different features of products all along the same way. Certain features that are growing fast, other features that are not. You get to these points of just diminishing marginal return in every system where it feels like you could put 50 people on this project, it's just not going to dramatically move the needle. Part of it is this bottoms up thing with your own team being really thoughtful about what is the expected value of that investment, and knowing when it's starting to approach zero or diminishing marginal return.

**Robby Stein** (00:41:23):
And then when that happens, these are these moments that usually coincide with something fundamental changing. Either people's expectations, externally, market saturation, there's something happening where you need to adjust. You then find your next growth driver or set of drivers. That's where you need to go more first principled and try these new things more. Then when you land a new thing that creates this new little growth engine and then you put people on it and you optimize it because each change is like 10% win, 20% win, 4% win.

**Robby Stein** (00:41:57):
It's clearly still has so much value in headroom and to make it better for people, and you can see that in the data. And so that becoming, I talked about this instrumentation, it becomes your guide for knowing if you're making good calls. Otherwise, if you don't know where you're headed and you don't have a goal of what you're trying to do more quantitatively, it's really hard to know if the thing you're doing is mattering to anyone. I think I made the product better, but is anyone using it? Does anyone care? Or are we just congratulating ourselves? Ultimately you want to have impact on people and that's what matters.

**Lenny Rachitsky** (00:42:29):
So it says essentially tracking S-curves on every product and understanding if you're in the plateau and if it's time to invest heavily somewhere else.

**Robby Stein** (00:42:36):
Yes.

**Lenny Rachitsky** (00:42:37):
**Robby Stein** (00:43:55):
I mean, I think it probably started earlier on with AI Overviews actually, which was the first way we brought generative AI to search. And in that world, we noticed that people were asking these questions and many people were actually trying to put natural language questions into search. And so how can you provide helpful context links to go deeper and make an AI that made sense for Google? That was our first version of these models that could do this for people. And then by building into that and seeing this observation around people wanting more of it, direct access to it, and then being able to ask follow-up questions. You need a new modality. It's going to be really hard to build all of that within the construct of the core search experience. And so that led us to have form a small team of folks, a few people that were technical leaders, a couple designers very small to just prove out what if there was on, almost blank screen, delete, make a little fresh doc with a blinker.

**Robby Stein** (00:44:53):
What if there's a new page and you can ask the question, you can ask whatever you want of it. You can tap right into the AI that was originally powering this top of the experience in search. But we invested in making it much more powerful in the ways I described before was in it could search for you. It had reasoning as a part of its model capability, it had multi turn context, so if you had a conversation with it could keep track of that context so it had some unique pieces to it. And what would happen if we tried that quickly. And we basically got, I mean, this was probably five to 10 people worth of people originally.

**Lenny Rachitsky** (00:45:29):
And how long ago was this team formed?

**Robby Stein** (00:45:31):
This was probably over the last year, last summer basically, into the fall.

**Lenny Rachitsky** (00:45:34):
Wow, so about a year ago.

**Robby Stein** (00:45:37):
Yeah, maybe about a year ago. It was where maybe it started. We were really plugging away on it, and then we saw this little version of it emerge that wasn't very good, but it had this moments of brilliance. It's actually, again, it's kind of like golf where you hit the perfect shot and you're like, "Oh my God." You get that feeling where it's just everything worked. And I asked it a question about, I forget, I was doing something with my daughter and I was planning an experience and it found all this incredibly useful information about park information. It had links to go to the site and confirm a bunch of things. It had Google Maps information that for my daughter, you could walk up, it was walkable. There was early examples like this where it just, it blew me away of what it could find and how helpful it was.

**Robby Stein** (00:46:27):
It gave us conviction that we should go and go further. And obviously there's lots of people involved in this type of a decision, tons of support from leaders across the organization. But it just says a little working team that initially, you got to build something and then you have to feel it yourself and it is very entrepreneurial in that way. And then when you see it tangibly, you're like, "What's a version of that? That's good and that could work?" And that gave you hope. And so then we basically built it out and built the first version that launched in Labs basically.

**Lenny Rachitsky** (00:47:01):
So the first big milestone was this is working. It was just a qualitative experience of, "Oh wow, this has really, there's magic here."

**Robby Stein** (00:47:09):
Yes, it's working. And then we did bring it before labs actually to trusted tester group. There were maybe 500 people externally that we added onto it, and we had pings with them. Some of them were, we actually had friends and family. We tried to treat it a little more like a startup where, because we feel like you got to have people test it to tell you the truth, and tell you when it sucks, because it probably does.

**Robby Stein** (00:47:27):
And then they'd message you. So I had a friend who was loving it, but also hating it for lots of good reasons and would just be messaging me all the time, screenshots, "This broke, this broke, this makes no sense."

**Robby Stein** (00:47:37):
We had that for a while, and then we got to a point where it was feeling good, the trusted testers were liking it, reporting good stuff, and then we it to this Labs moment where anyone could turn it on and then we used that to make it better with real query data. We could actually see what people were using it for at more scale and so that could tune it to make it better. And then we launched it out to everyone, or at least in the US, and then we've now been on this journey to expand it to all countries and languages and have more people be able to access it.

**Lenny Rachitsky** (00:48:05):
It's incredible that Google went roughly in a year from idea to a significant change to the search experience that's AI powered. I think this is not what people imagine Google is like, and it feels like things are different and things have changed in how you guys operate. What has allowed this to happen so quickly? What's changed? Is it just top-down leadership, we need to get shit done, or is there something more?

**Robby Stein** (00:48:30):
No, I mean I think it's interesting how organizations change. I think when you feel like there is a moment in time that is clearly critical to deliver for people, people are trying to get information from Google. We are not able to answer certain things or help people in certain ways and there's this technology that can do it, that creates urgency, and obviously there's lots of people building lots of things and the market's crazy and there's lots of things shipping all the time.

**Robby Stein** (00:48:56):
There's a really exciting and healthy moment for us to build and build quickly and I think it's just exciting to be able to capture that opportunity because I think people believe, and I certainly believe that the next year or so of product is going to establish how people use the next wave of products for many years. And so at least I can only speak for myself, I feel this obligation to our users to give them the best version of Google that's powered by AI and that gives them the full knowledge of everything Google knows about the world and information to people and accessible with AI. That's driving a lot of the excitement.

**Lenny Rachitsky** (00:49:34):
Yeah, it's such a good point that people are building their new habits. It's wild how many people just now rely on ChatGPT and how quickly that happened. And I could see Google being worried that, oh, shit, everyone's changing their habit from searching Google to searching ChatGPT. And the fact that now Gemini is number one. I was actually looking at the list of top, so in the top 15 apps, Google is I think five of them, a third. It's out of control, killing it. When people look at AI Mode versus ChatGPT or Claude or let's even say Perplexity, what's the way you think about the positioning of AI Mode versus these other tools? Is it trying to be a direct competitor or is it just like, "No, it's actually pretty different and here's what it's for?"

**Robby Stein** (00:50:15):
Yeah, I mean AI Mode's a way to ask search anything you want. It's designed and specially created for information. And so really, it should give incredible helpful responses for the things that people come to Google for. Think about you're planning a trip, you're trying to buy something, you're working through a question for your research project. It needs information and that's really, it's less focused on things like creativity, although there's things that can do that are nice there. It can help you. Just like any kind of core AI product, you can ask it to rewrite something for you, it'll do that. But we are less focused on creativity, productivity, upload a spreadsheet and output graphs for me, we're not focused on that.

**Robby Stein** (00:50:57):
We're really focused on what people use Google for, and making an AI for that so that you can come to Google, ask whatever you want and get effortless information about that and context and links to then also verify, dig in and go to the authoritative sources ultimately that people want, and we hear from people. So those ends up becoming the distinct qualities of this product versus more of a chatbot. Maybe you would talk to it like you maybe even have a bit of a, "Hey, how are you doing today," with that chatbot that we have some of that, we see that a little bit, but people are usually coming for information. They're trying to learn something and we focused our product on that.

**Lenny Rachitsky** (00:51:30):
Got it. Okay, AI Mode is not your therapist. Maybe zooming out again a little bit and reflecting on all the amazing products you've worked on, all the places you've worked, if you had to pick two or three just core product principles or philosophies that have helped you build such amazing and successful products, what would those be? What comes to mind?

**Robby Stein** (00:51:53):
I mean, there's typically three things I think about. If I were to write a book about how to build great products, there'd be three chapters. I mean there'd probably more than that, but three chapters.

**Lenny Rachitsky** (00:52:08):
I love that. I love how short that would be. That's the ideal book.

**Robby Stein** (00:52:08):
I've thought about these three areas now for a while and it's like they're always consistently the three things. The first is deeply understand people, and I think we talked about this a little bit with the jobs to be done point and Clayton Christensen's book, which I loved around Competing Against Luck. It really helps you be a student of why someone ends up, in his words, hiring a product. Don't think of users as using your product. Think of users as hiring you to do something for them.

**Robby Stein** (00:52:35):
There's this famous quote, I think it's Theodore Levitt had, "People don't want a quarter inch drill, they want a quarter inch hole." So what is someone trying to do? You have to understand that deeply and then you can build an amazing product. And also by the way, when you go back, why someone not using your product?

**Robby Stein** (00:52:57):
And so it focuses on these techniques to extract causation. So he actually talks a lot about this interview. He calls it an interrogation where you talk to a user like, "Hey, why do you use my product? Where were you? Were you in bed? Were you at work? What were you doing?"

**Robby Stein** (00:53:11):
"Oh, I was talking to my wife in the morning."

**Robby Stein** (00:53:13):
"Okay, well, what brought it up?"

**Robby Stein** (00:53:15):
"Well, I guess I was reading the newspaper."

**Robby Stein** (00:53:16):
"Okay, well why?"

**Robby Stein** (00:53:17):
And then you have this aha moment like that when they first decide to use your product, he calls it the big hire. That is information that you obtain ends up becoming the most critical because that is what caused someone to use your product. And if you can study that and understand it, you'll be much more on your way than just building things that sound cool. And so that's the first chapter is deeply understand people.

**Robby Stein** (00:53:37):
Second is really around analytical rigor and understanding your problems. You have to understand your problems. And this got is a little bit of what we were talking about about root cause analysis and understanding, okay, the metrics are dropping. Why? If someone's not using your product, why? And really being able to dissect that to get to true root causes. It's like, well, they went all the way to the end and then bailed, and then you understand what turns out that it was most, we actually learned about this and there's a story in Close Friends at Instagram where it just totally failed at first in a bunch just when we shipped it. And it turned out that we looked at the data and people were only adding one close friend to their list because it was mistranslated as best friend in many markets. So people just put one person and then the probability that person saw it and wrote back to you was zero. It's a product which is broken. So it's like you got to understand your problems.

**Robby Stein** (00:54:30):
And then the third one's around really designing for clarity instead of cleverness. A lot of people are like, "Oh, we're going to differentiate the design," and we talked about this a little bit with Stories. We're going to make a new version of something, but if something's a standard and people understand it, if you lean into it, you're going to get so much leverage than if you reinvent it, and you have to be really thoughtful around when you reinvent and where you don't.

**Robby Stein** (00:54:54):
And I think on this one, there's this great, Don Norman's book. Obviously, Design of Everyday Things is a big one, but he has this incredible chapter in there about doors, and why is it that after all of these years you walk up to a door, and based on how they're designed at times, people still don't know if you should pull or push that door because if you try to build the as beautiful symmetric two handles on each side on a glass door, it doesn't communicate in for any information to you.

**Robby Stein** (00:55:20):
And there's lots of, I've seen all the time we've designed new icons when we could have used global icons like, "Oh, wouldn't it be so cool if we used a camera that's kind of a camera but is mostly an AI looking thing and then is mostly, but then has this dots in it that connects it to this other product?"

**Robby Stein** (00:55:37):
And you're like, people just, it's a camera. Just put the camera in. Maybe you could add a little thing to it, and that's how you get people to use your products. And if you do those three things, I think you typically can do well.

**Robby Stein** (00:55:49):
And then, sorry, the fourth one would be more of the coda is be humble. Constantly and always question yourself. Listen to others, listen to users and be open to being wrong.

**Lenny Rachitsky** (00:56:00):
I love these. On that third point, I feel like AI Mode as the name is such a good example of clarity. What is this? This is AI Mode.

**Robby Stein** (00:56:07):
We talked about it internally. If you look at it in the tab, it's like everyone know, it's like you see it and you'll know what it is or we could call it something random, but then what is that? And now you're working against yourself.

**Lenny Rachitsky** (00:56:20):
If I were to reflect back these three pieces of basically this is the book you would write to help people build more successful products, it's understand the problem you're solving for people deeply. What's the job they're hiring you to do? I love the, it's lowercase jobs to be done. It's not like the rigorous whole thing that everyone-

**Robby Stein** (00:56:41):
Exactly. Lowercase for sure.

**Lenny Rachitsky** (00:56:41):
Okay. This is just like why are people hiring your product to solve a problem for them? What problem are they solving? So it's like basically figure out what problem they're having then very, through data, understand the problem and whether you are solving it. And then it's just keep it really simple. Clarity over cleverness essentially.

**Robby Stein** (00:57:02):
Exactly, yes. And be humble.

**Lenny Rachitsky** (00:57:05):
And be humble. Yes. Okay, important. Is there an example that we haven't talked about that shows this in action of just, cool, here's the problem we found. Here's how we figured out this is the solution and if we're succeeding, and then here's a very simple way of solving it?

**Robby Stein** (00:57:19):
I mean honestly, this Close Friends example, I can give you more from Instagram days was really wild. It took two or three years to get Close Friends to work, and I think people, it totally failed originally. This is the product that lets you add a private list of people and then you can post to your story and then only those people see it. It's like this very exclusive private space so you can feel really comfortable sharing maybe more.

**Lenny Rachitsky** (00:57:39):
Oh, green circle.

**Robby Stein** (00:57:40):
Green circles, yes. It's one of the most popular, at least when I was there, was one of the most popular features of Stories and did really well, but it totally failed. And I think what we found out was that you actually used a bunch of these techniques here. So one was we first thought about it as an overall system problem and you could add a Close Friends post for anything. So you could do a feed post or a Stories post, and you also had a close friend's profile. You could see, if Lenny went to Robby's page, we were Close Friends, you would just be like, "Oh, you get to see extra stuff from me on my profile too."

**Robby Stein** (00:58:18):
So we shipped it, we thought it would be great. This is the be humble part, wasn't great, had a bunch of, it was just super confusing. You would see this really beautiful photo and then in the feed right after it, this blurry, very vulnerable moment someone's trying to share with their friends, just felt so out of place and weird for the reason people use feed. And then it was just confusing because it had an extra little green thing on it, but it was like that got a green thing and the Stories one didn't. If you open the story, it had a green thing inside the story, and people were just so confused.

**Robby Stein** (00:58:49):
And it had this other issue with the list where you're like, "Okay, the list doesn't work because it's mistranslated and people don't get it." I think it was actually called originally favorites, I want to say, and that encouraged people to just do two people on it. But then the way that it worked was, so this gets to the framework, I guess. So deeply understand people. What are people trying to do with this?

**Robby Stein** (00:59:10):
What they're trying to do is share a vulnerable thing and be like, "Hey, I'm lonely. Hey, what's going on? Are people up?" And it feels very much like a friend group thing.

**Robby Stein** (00:59:18):
And if you only have two people on it, the job that we're doing is actually connecting you to your friends. And if you don't get a DM back, it's broken. And so really what we're doing is getting you a DM and we're getting you connection. We're getting you a sense of being connected to your Close Friends. That is the job.

**Robby Stein** (00:59:33):
It's actually everything Clayton Christensen talked about in the book is there are utility jobs and there are emotional jobs. People usually discount the emotional ones a lot. This was really an emotional thing as much as it was utility one, and so product's broken, right? And people don't even know that it's a close friend story, they just see the little head because you have to click on it to see the thing. And so it just, people stopped using it.

**Robby Stein** (00:59:56):
We went through and we did these revs where we would simplify it and we would update it and we would go through this change list. Okay, take this out, take this out, change the name, here. And then we saw it was that it was working really well for people who added 20 to 30 people to their list. Because what would happen is you put 30 people on your list and then two of them would write back to you on DM and now you have closed the loop and you feel connected to those people. It's a winning thing. And so we designed the whole system around that, and also only worked in Stories. We were looking at the data, we were trying to understand where it was working and where it was failing, and then we updated the name to Close Friends so it didn't feel like favorites. So it wasn't three people, it's 20.

**Robby Stein** (01:00:34):
In the list, we built this list builder where we recommended a set of people based on some cool algo that was created by an engineer. And then we updated the design to put the green ring on the outside of the story so that this was the design for clarity. We were being cute. We thought, I think at the time it was like, "Oh, it's a secret story or something, and if you open it, you see it."

**Robby Stein** (01:00:56):
It just was not clear to people. And so we put the green ring on the outside so that users would see it in the tray and be like, "Ooh, what's that little green guy?"

**Robby Stein** (01:01:04):
And then they'd click on it and be like, "Oh, this is a private story for me." That system worked and did incredibly well, and that was the process we followed from a total flop to something that was very successful.

**Lenny Rachitsky** (01:01:16):
That is an awesome example. And this took two or three years, you said this process?

**Robby Stein** (01:01:19):
Yeah, it took a while. That was actually one of the longest projects we worked on, but that actually came, the reason we did it was when we asked people to understand people like, "Why aren't you posting to your story? What's preventing you from doing it?"

**Robby Stein** (01:01:32):
And everyone had some version of, "Well, my ex is on it. I have a teacher on it. Oh, a friend that kind of is judgy is on it."

**Robby Stein** (01:01:39):
It was like this commonality was audience problem. Someone had an issue with people watching them. And so that gave us conviction to go this hard at it for so long because we knew that that was a core problem with the product.

**Lenny Rachitsky** (01:01:51):
Was this connected to the Finsta, Rinsta trend also?

**Robby Stein** (01:01:55):
It was actually. I think that informed us. Everyone had a Finsta and there was a Binsta.

**Lenny Rachitsky** (01:01:58):
Was is a Binsta?

**Robby Stein** (01:02:00):
Best friend Insta.

**Lenny Rachitsky** (01:02:03):
I see.

**Robby Stein** (01:02:03):
Different, it's this layering of people 20 Finstas down to your partner, Pinsta, and then it's basically like, I made that up. I don't know if it's true, but I'm sure it was out there somewhere. We were like, "Wow. People clearly are trying to hack Instagram basically to create these private smaller group settings, and so we should just make a product."

**Lenny Rachitsky** (01:02:23):
How did you actually do this testing? Was it rolled out to some percentage? Was it rolled out in New Zealand or whatever?

**Robby Stein** (01:02:27):
Yeah, we rolled it out in a few other countries, exactly.

**Lenny Rachitsky** (01:02:29):
Okay,

**Robby Stein** (01:02:29):
Got it. We had a basket of countries that we tried it in and then we would do research. I think it was Australia was one of the first ones for that one.

**Lenny Rachitsky** (01:02:37):
Okay. I was going to ask if you can share the country. So Australia.

**Robby Stein** (01:02:40):
I think that was one of the earlier ones, yeah, but every time you ship something there's a slightly different reason why.

**Lenny Rachitsky** (01:02:46):
Oh, interesting. So it's not always Australia gets all the new stuff.

**Robby Stein** (01:02:49):
No, although it sometimes is. Australia and Canada get a lot of stuff just because easier for the teams to see feedback from them.

**Lenny Rachitsky** (01:02:57):
Yeah, speak English.

**Robby Stein** (01:02:59):
Yeah, exactly.

**Lenny Rachitsky** (01:03:00):
Awesome, okay, let me go in a different direction and talk about something that you have a hot take on. There's a lot of talk these days about lean teams, small teams, just creating limited resources, not hiring at all. You have an opposite perspective of you actually need a lot of resources to build really big breakthroughs. Talk about your experience there.

**Robby Stein** (01:03:19):
Yeah, I mean I think there's obviously, depends on what you're trying to build and there's been famously small teams building big impact products, but I think there's this cult of lean, scrappy, fast, throw away your product quickly, keep moving. And I think at some level it's true for internal conviction, but to build a product that works for a lot of people that is based on a technological breakthrough. A lot of times, I see teams just give up to early or under invest in the product, and obviously the space matters. And if you're building a single product that is a way to, I don't know, do something with a digital app that's fairly straightforward, that's going to be different than building a robotics company. So what you're building does change.

**Robby Stein** (01:04:02):
But even for software, I mean I think for really hard technical problems, think about the amount of time and effort it took for teams to build a foundational model, and how many years and hundreds and hundreds of people that were needed for that to happen. And you think about these large companies that have had huge impacts on people, and I think particularly for bigger companies internally, something I've seen is it's almost too scrappy because it never gets enough momentum. The product never gets good enough internally and then it just dies on the vine. Whereas if you put more people on it, you have to be careful not to put too many too soon. But I see the opposite more true where people hold on to small teams too long and then you, either takes forever to get to the thing you're looking for.

**Robby Stein** (01:04:46):
This Close Friends example I mentioned this actually was a small team. One of the reasons it took us forever was it kept the team so small and scrappy. That loop cycle was so short and by a startup age you'd be dead probably. So you can maybe do that in a bigger company, but as a startup, I don't know if you have that leisure. And so I think you need to actually think what is the group I need to build a version that's great. And from first principles, really think about it instead of just embracing blindly, okay, we're going to be the two of us until this thing has escaped velocity market fit, which it's not always true.

**Lenny Rachitsky** (01:05:19):
This is definitely counter to the narrative we see on Twitter. Anything you can share about just the heuristic you use to decide here's how long to keep it small? I know there's not going to be this step 1, 2, 3, but just like what I'm hearing is start small to prove out the concept designer PM engineer maybe. When do you find that makes sense to go big?

**Robby Stein** (01:05:40):
Yeah, I think that it's mostly when you've hit the conviction moment. I think there's two big milestones. There's internal conviction. For yourself, do you believe in it? And you believe in it because there's some external validation, your friends, you put 20 friends on it. And by the way, I found out very quickly building startups that if you put 20 friends on something, they're not going to do you that many favors. They're not going to use a product every single day because they're your friend 30 days in, 60 days in, 90 days in. They're not using your product unless you're doing something that's useful to them. And so you get all of this feedback and you're seeing people really enjoy it. You get to that moment.

**Robby Stein** (01:06:17):
And then I think that's not a product that would win externally because if you were to ship it, it's broken, doesn't work great. And then you need to, I think invest enough to make the best version of it or as good a version as you can to get it out the door and to ship it. And I think that that, it's like you want to build the right product eventually is the mentality and you can only really do that with the right group.

**Lenny Rachitsky** (01:06:39):
I'm going to take us to a recurring segment on the podcast that I call AI Corner.

**Robby Stein** (01:06:43):
Okay.

**Lenny Rachitsky** (01:06:44):
What's some way that you've found use for AI in your work, in your life that is really interesting, really helpful, maybe other people can be inspired by?

**Robby Stein** (01:06:53):
I think one of the coolest trends ever is how AI is affecting multimodal visual and inspirational needs for people. And we're early in this and I think this is something that I'm actually working on as a project as well, but right now if you think about what AI has done in large part, it was born and grew up in this text modality, it was chat. And so for a long time, if you were to ask it to help you, what's a cool way to redecorate your bookshelf behind you? It's going to describe that to you in text, because that's what it knows. But increasingly, AI is going to be liberated to help in every possible modality.

**Robby Stein** (01:07:29):
This is something that we've seen a lot with this explosive use of Google Lens and our image search and image features and with this deep understanding, and what I'm actually starting to use internally and some things that we're excited about more coming up that we actually announced at I/O that we're going to going to be building more of was how AI can help with inspiration, how AI can help with shopping and helping you really get things done that are more in the inspiring bucket of needs versus these core utilities like code, math, homework side of things.

**Robby Stein** (01:08:04):
And I'm really excited for things that are coming where you can ask it for inspirational tasks and it's starting to do really fascinating things in terms of what I'm seeing and hopefully we'll share more on that soon. But I think the one thing I can share is there's a visual version of AI Mode that basically we talked about at I/O, and so you can reference some of those keynotes, but that's in the process of being rolled out.

**Lenny Rachitsky** (01:08:34):
Mysterious.

**Robby Stein** (01:08:34):
And so you're going to be able to now ask what's a mid-century modern beautiful office design with dark themes? It'll be able to produce this image board that's inspirational and you can do multi-turn with it. And so you'll be able to go and say, "Actually, I want more of a light theme, more creamy, more California, more coastal vibe." And it'll do that and it'll understand that and it'll actually see the images and be able to turn with you in the way that text works, which is going to be really cool. So I think that's going to be one of the more exciting things that will be new to AI soon.

**Lenny Rachitsky** (01:09:10):
What I'm hearing is Nano Banana integrated into AI Mode. Recipe for success.

**Robby Stein** (01:09:14):
Well, it's a little different than Nano Banana because Nano Banana is an image editor. This is more like helping you find images on the web, so it's a little bit more like AI inspiration, AI image search, and allowing you to then talk with two effectively visual responses with natural language. So that's going to I think, be a little bit different than edit this photo so that it changes it. Although potentially an interesting idea too, to have an ability to take a picture of your living room. And I think AI will help with that too ultimately.

**Lenny Rachitsky** (01:09:48):
Pinterest is in trouble, feels like this is what people use Pinterest for. Here's all the inspiration. Now it's just AI doing it all. By the way, Nano Banana, where does this name come from?

**Robby Stein** (01:09:58):
I don't actually, I forget that. There's a story somewhere. I forget it now honestly. But the team I think came from a scrappy, fun group of people building this and they wanted to go for something fun for folks to-

**Lenny Rachitsky** (01:10:13):
Yeah, it feels like that's a part of the reason things have started to work. There's just more fun and delight and random crazy stuff coming out.

**Robby Stein** (01:10:20):
It does. It feels a little more like when I was at Google the first time through right now where you just have so much stuff and this kind of fun curiosity happening where people want to try things and ship things and yeah, hopefully that continues.

**Lenny Rachitsky** (01:10:31):
Yeah, it feels like Veo 3 would be even more successful if it had a wacky name. And I like that this is the opposite of your advice of clarity. I don't know what Nano Banana is, but it worked.

**Robby Stein** (01:10:42):
Yeah, it's the other thing. No advice is right universally, right? But yeah, Nano Banana.

**Lenny Rachitsky** (01:10:49):
Robby, is there anything else that you wanted to share? Anything else you want to leave listeners with as a final nugget of wisdom before we get to a very exciting lightning round?

**Robby Stein** (01:10:58):
This concept: be curious. I think of embodying everything as like it's really about curiosity. It's about wanting to know why everything is the way it is. Why is someone doing something? Why does someone have a different opinion than I do? Why might this not be working? And the people who really have that level of intense curiosity and they chase things down until they know, I think you're well served by that. That would be my only parting thought.

**Lenny Rachitsky** (01:11:20):
Let me follow that thread actually, because it's maybe the most trending term on the podcast over the past few months is curiosity. It comes up a lot when I ask people, what are you teaching your kids and embracing with the rise of AI and curiosity comes up all the time. Is there anything that helps you? Is it just like I am good at this and I am curious innately and I'm just, "This is valuable." Is there anything you can share that helps you or others around you embody that and actually be curious?

**Robby Stein** (01:11:48):
Well, I mean AI is obviously the ultimate curiosity engine, and that's what's so cool is you can now ask anything and just get information. And so I find that people just appreciate just how much they can learn about whatever they want. But also, I think that a lot of this also comes down to studying what you want to know about, and knowing where the branches of knowledge live there. A lot of times I'll read old papers and PDFs that are free online on a statistics thing if I want to learn about that and I think people under appreciate those. There's analog old school great learning and AI can help you discover them. I'm using AI, I'm particularly at Google to help discover all these cool links and things to read, but I find that that is an interesting hybrid where it's not just AI but really going to original sources more. I find that these books I mentioned on the chat here, I find that you need a blend of all of those things to ultimately really get to the bottom of things ultimately.

**Lenny Rachitsky** (01:12:46):
Actually reading the thing, not just reading the summary of the thing.

**Robby Stein** (01:12:48):
Yes.

**Lenny Rachitsky** (01:12:49):
Let me actually ask you this question I've been asking all these people that are at the cutting edge of AI. You have kids, is there anything you're thinking about and leaning into helping them learn, develop as AI emerges and becomes a big part of the world?

**Robby Stein** (01:13:04):
The biggest thing I'm doing, I have younger kids, so the biggest thing I'm doing is they're using live versions of AI that they just talk to now much more. And so funny enough, we actually just launched search live actually out of Labs this week. And so you can talk to search in a live AI setting, which is conversational voice. Voice on when you're driving, you can just talk all the knowledge I talked about where you can do with Google, you can talk to it in a normal conversation with your voice. And I found that to be incredibly accessible for kids.

**Robby Stein** (01:13:31):
And I hear all my kids come home, they're like, "Can I talk to Google about something?"

**Robby Stein** (01:13:34):
"What do you need? What do you need to say?"

**Robby Stein** (01:13:36):
And then they go to my app, they hit the live button and they just start talking to it. They want to know about animals, they want to know about certain, I don't know, history things. They learn about something in school, and it's so natural to learn in that way that I think that that's helping them become much more AI native than any other thing I'm doing.

**Lenny Rachitsky** (01:13:56):
Life as a parent is going to be way too easy now whenever kids have questions, "Just go talk to the AI," but I don't think that's bad. So this is within the Google search app. There's a live, how do you access this?

**Robby Stein** (01:14:07):
Yeah, that's exactly right. You go to Google app, so there's one of the apps in the App Store you mentioned. You open Google and there's a button now that's live on it, right on the home screen. And if you tap on, it's a live version of AI Mode that you can just talk to. It's a full screen experience, and we'll say start talking.

**Lenny Rachitsky** (01:14:22):
In the show notes, I'm going to link to this project that somebody built, Eric Antonow, which I love. It basically shows you how to put a little speaker into a little stuffed animal and you connect the speaker to, it could be Google Live or it could be ChatGPT, whatever you like, in voice mode. And you put it on your shoulder, you get a little magnet that attaches, and your kids could talk to this parrot, for example, and you could tell it, "Talk in a pirate voice," and so they're talking to his pirate.

**Robby Stein** (01:14:49):
Oh, that's really funny. Okay, that's really cute.

**Lenny Rachitsky** (01:14:51):
It takes 15 minutes. You could get an X-Acto knife and sew it and stuff and it's fun. I made one for my nephew and he was looking for treasure with this parrot.

**Robby Stein** (01:14:59):
That's really adorable, I'm definitely going to look into that.

**Lenny Rachitsky** (01:15:02):
Robby, with that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

**Robby Stein** (01:15:07):
All right, I'm ready.

**Lenny Rachitsky** (01:15:08):
What are two or three books that you find yourself recommending most to other people?

**Robby Stein** (01:15:11):
I mean, definitely the two I mentioned here. Clayton Christensen, Competing Against Luck. Don Norman, Design of Everyday Things. But I also really love this for fiction, Aurora, which is this book David Koepp wrote. It's about electromagnetic pulse in the sun that knocks out, it's fiction for just fun. And it was a really fun beach read and apparently it was going to be made into a Netflix show, it didn't work out. I don't know. It was sad to see that fall apart, but so it's a really fun book.

**Lenny Rachitsky** (01:15:39):
There's a book along those lines that I love, they're making a movie of it right now called Hail Mary.

**Robby Stein** (01:15:43):
Oh, I'm in the middle of reading that right now.

**Lenny Rachitsky** (01:15:45):
Okay, awesome.

**Robby Stein** (01:15:46):
Yes.

**Lenny Rachitsky** (01:15:46):
Of the same mind.

**Robby Stein** (01:15:48):
Yes.

**Lenny Rachitsky** (01:15:48):
Yeah, they're making a movie of it. How about that?

**Robby Stein** (01:15:50):
In the middle of reading it. It's getting wacky where I am right now, but I'm excited to see where it goes.

**Lenny Rachitsky** (01:15:54):
It gets wackier. The ending especially wacky.

**Robby Stein** (01:15:55):
Oh, really? Okay.

**Lenny Rachitsky** (01:15:56):
Just prepare yourself.

**Robby Stein** (01:15:57):
Okay.

**Lenny Rachitsky** (01:15:59):
What is a recent movie or TV show you've really enjoyed?

**Robby Stein** (01:16:02):
I love The Bear. I think that's just absolutely awesome show. Dune, of course. And I thought the new Top Gun is a little old now, but I think the new Top Gun was so fun and awesome.

**Lenny Rachitsky** (01:16:13):
Is there a product you've recently discovered that you really love? It cannot be AI Mode.

**Robby Stein** (01:16:17):
I'm going to use a non-digital product.

**Lenny Rachitsky** (01:16:19):
Perfect.

**Robby Stein** (01:16:20):
I'm super into this new pillow that I got called Purple Pillow, and I've been recommending it to everyone at work. We're on a pillow chat now. It's a thing. It's like you talk about what pillows we're getting, but it's this really cool thing where it's got this new technology of this honeycomb polymer that's inside and so it supports you and it has these little micro holes so it doesn't get hot. It's really cool. Big fan. Strongly recommend Purple Pillow.

**Lenny Rachitsky** (01:16:50):
I've never heard of this thing, I am excited. I recently got an avocado pillow, focusing on low toxins.

**Robby Stein** (01:16:57):
Oh, those are good. I've heard good things about those too, yeah.

**Lenny Rachitsky** (01:17:00):
Okay, I got to join this pillow. Pillow talk is a great name for it by the way.

**Robby Stein** (01:17:04):
You're into pillows too. That's great.

**Lenny Rachitsky** (01:17:05):
Huge.

**Robby Stein** (01:17:06):
I love bedding.

**Lenny Rachitsky** (01:17:06):
No, I'm just joking.

**Robby Stein** (01:17:07):
Yeah, great.

**Lenny Rachitsky** (01:17:08):
But I did upgrade my pillow. This is not Mr. Pillow, whatever that guy is, right? Is that guy that, there's like a controversial pillow guy. Okay.

**Robby Stein** (01:17:17):
No.

**Lenny Rachitsky** (01:17:17):
Okay. Purple Pillow. I'm going to ask AI Mode.

**Robby Stein** (01:17:20):
Yeah, you should.

**Lenny Rachitsky** (01:17:20):
This.

**Robby Stein** (01:17:20):
Definitely.

**Lenny Rachitsky** (01:17:22):
Next question. Do you have a favorite life motto that you find yourself coming back to in life?

**Robby Stein** (01:17:28):
This is be curious. I think I almost named a company Curious. I just think it's a really awesome, there's one thing in life. It's that in terms of getting things done, in terms of understanding the world, people, your kids, your family. You always just want to know more and question things outside yourself, not feel like you have all the answers. I think that's really important.

**Lenny Rachitsky** (01:17:49):
I love that. Final question, okay, so speaking of startups, you started a company called Stamped back in the day, it got acquired by Yahoo. I hear there's a story where you got Justin Bieber on your app and that was a big deal and a big inflection in the success of the app. Can you just tell that story?

**Robby Stein** (01:18:06):
Yeah, it's a wild story. Just to scene set a little bit. I was 25 right after Google being an IC PM in New York with some Google friends building this company. So very early on, and maybe in a good way and no idea what I was doing. But basically we decided that the concept of Stamped was to put your stamp on your favorite things, get recommendations from friends and from people that you trust. And so you think of a Twitter feed, but it's all stuff that people think is cool.

**Lenny Rachitsky** (01:18:34):
Which products.

**Robby Stein** (01:18:35):
It's like books, restaurants, food. Products, exactly.

**Lenny Rachitsky** (01:18:37):
Pillows, possibly.

**Robby Stein** (01:18:37):
Pillows could be on there. I would totally stamp this pillow and then you could discover it. And one of the cold star problems was obviously you want a group of people that are on it that are already using it, that could have some tastemaker type folks. We had a bunch of people that were chefs and we had people who were literary folks. And then we wanted to get a couple people that were more musicians, artists, and these influential folks.

**Robby Stein** (01:19:00):
My co-founder and I just basically got the contact of Scooter Braun, who's Justin's manager, and we just sent out an email and we were like, "Hey, we're in New York. We're going to be in LA tomorrow." I think we said something, I don't remember all the details, but it was something like tomorrow.

**Lenny Rachitsky** (01:19:15):
And you were not going to be in LA tomorrow.

**Robby Stein** (01:19:16):
No, no.

**Lenny Rachitsky** (01:19:17):
Okay.

**Robby Stein** (01:19:17):
"Do you happen to be there?"

**Robby Stein** (01:19:19):
And he just wrote back some one line thing like, "Meet me at this hotel for breakfast at something."

**Robby Stein** (01:19:25):
And we're like, "Oh, okay."

**Robby Stein** (01:19:28):
We literally went immediately to the airport. I just remember just basically going straight to the airport, flying to LA meeting with him. We gave him the whole pitch, we showed him the product, and then he was like, "Okay, I think this would be super cool. We can be involved and maybe you can help be an advisor."

**Robby Stein** (01:19:44):
And we ended up going back and meeting with Justin and showing him the product and even filming some little clips with him. It was actually really funny and it was a really fun moment. And obviously he was using it to stamp his favorite stuff. And so people would go, "Oh, Justin's into this song, or he is into this stuff," and would post that.

**Robby Stein** (01:20:02):
It was one of the ways that we got lots of people to try out and see what we were doing. That's a little extra scrappy moment in time, but I think it embodies a good lesson. Just do it now, be scrappy, be immediate. Intense urgency usually wins over thinking about it for a long time, and that's certainly proved to be true on that one.

**Lenny Rachitsky** (01:20:20):
Incredible story, thank you for sharing that. So many lessons to take away. Two final questions, where can folks find online if they want to reach out, maybe learn more about what you're doing and how can listeners be useful to you?

**Robby Stein** (01:20:31):
Yeah, I think on X @rmstein is probably the best single place. And then to be helpful, send me feedback. DM me, just mention me, ping me, let me know problems with Google products, with AI in general, but also just anything. As I said before, you have to always listen to people understand their experiences, so ping the ideas and feedback. That's the best way to be helpful.

**Lenny Rachitsky** (01:20:52):
Wow. What an onslaught you're about to receive of feedback on the search experience.

**Robby Stein** (01:20:56):
No problem. Yes, please do.

**Lenny Rachitsky** (01:20:58):
"Robby, why is this link second? Why is my site not at the top?" I can only imagine the kind of stuff people complain about. Robby, thank you so much for being here.

**Robby Stein** (01:21:08):
Thank you, it was great.

**Lenny Rachitsky** (01:21:09):
It was great. Bye, everyone.

**Robby Stein** (01:21:11):
Take care.

**Lenny Rachitsky** (01:21:13):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating, or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcasts.com. See you in the next episode.

---

## [4/17] How to measure AI developer productivity in 2025 | Nicole Forsgren

**Nicole Forsgren** (00:00:00):
Starting with what is your problem or what is your goal? I would say this is a bigger challenge than most people recognize or realize. 80% of the folks that I work with, this is their biggest problem. Even at executive levels, teams will have gone off for several months, and they're tackling something, and they'll come back with uncertainty, and they'll say like, "Well, you told me to improve developer experience."

**Nicole Forsgren** (00:00:20):
I'm like, "Okay, what do you mean by this? Are you talking about inner and outer loop? Are you talking about friction? Are you talking about culture? But if you're talking about culture, this is totally different than if you're talking about friction in toolchains. If you're on different pages, you're heading in completely different directions."

**Lenny** (00:00:37):
Welcome to Lenny's Podcast where I interview world-class product leaders and growth experts to learn from their hardware experiences, building and growing today's most successful products. Today my guest is Nicole Forsgren. This is actually my first recording back since going on pat leave for the past couple months, and what an awesome episode to get back into the swing of things. Nicole is the developer productivity expert, having written the award-winning book Accelerate, and she's been the co-author of the State of DevOps Report year after year.

**Lenny** (00:01:06):
She's currently a partner at Microsoft Research, leading developer productivity, research, and strategy, and she's helped some of the biggest companies in the world move faster, improved product quality, and transform their cultures. In our conversation, we get into the weeds of how to go about measuring and improving your engineering team's productivity and experience. We talk about the DORA framework and the SPACE framework and how to actually implement them to understand how your engineering team is doing.

**Nicole Forsgren** (00:02:54):
Thank you so much. I'm excited to be here.

**Lenny** (00:02:56):
I'm excited to have you here. I actually skip this question usually with guests, but I thought it'd be actually really valuable to spend a little time on your background. You have such a unique role and unique set of experiences. Could you just talk briefly about the things you've been up to in your career, where you've worked, and then what you're up to now and what you focus on these days?

**Nicole Forsgren** (00:03:15):
Sure, and I appreciate the question because you're right. I sort of had this choose your own adventure background. So I started as a software engineer at IBM. I was writing software for large enterprise systems, which meant I ended up running them. So I was also a CIS admin. I was racking the stacking. I was running these really, really large labs, and then I kind of stumbled into this seven-day march for several years, and I was like, "There has to be a better way." And we're hearing rumors of it, but management was not buying in. And so I decided to win this battle with data, and I was like, "I should go do a Ph.D."

**Nicole Forsgren** (00:03:55):
And so ended up kind of taking a slight pivot into Ph.D. and management information systems, which some people are less familiar with, but it's basically a cross between tech and business. And I ended up getting a fairly technical Ph.D. So I went to a school, I went to University of Arizona, which has a very, very technical degree, but I liked that it crossed with business because then I had the ability to make these strong business case statements, right. So it was like, how is or is the way that we develop and deliver software tied to outcomes at the individual level, right?

**Nicole Forsgren** (00:04:33):
Can I be more productive? Can I have better work-life balance? And the team level, is the team more productive? Is the team more efficient? And the organizational level, right. This is what I was really interested in originally. Do I see better ROI? Do I see better efficiency because then I could sell it to people, right? And so that was really kind of what I originally went into. And then I was a professor for a handful of years because if you're doing research, traditionally, that's the job in academia. I also had a master's in accounting because that really helped me make that kind of financial tie and understanding financial statements.

**Nicole Forsgren** (00:05:07):
And then, after a handful of years kind of walked away from tenure because academia was not convinced that DevOps was a thing, right, that DevOps wasn't real, and State of DevOps Report who I was doing with DORA, DevOps research and assessment in collaboration with Jez Humble and Gene Kim. And we started that work with Puppet. So shout out to Alanna Brown for starting that, and Nigel Kersten and the team there. We kind of pivoted away. And Chef, that little configuration management startup at the time, hired me, and they're like, "We'll give you halftime to do research and halftime to help our engineering practices improve."

**Lenny** (00:05:45):
That's cool.

**Nicole Forsgren** (00:05:46):
Yeah. I mean, they were incredible because what startup is going to be like, "Yeah, do research." So I was there for a year and a half and then left to do DORA full-time. We actually had a SaaS offering, so we continued the State of DevOps Report just under the DORA banner, and we had a SaaS offering because so many large companies were like, "I want my own customized measurement, reading, and report." And then the joke there when we met with Gartner, they were like, "Your superpower here was that you tricked people into strategy," which was not only how do I benchmark? That was kind of our top-of-the-funnel. Everyone wants to know how they compare.

**Nicole Forsgren** (00:06:22):
But the important thing is what should you do next? What's the most important next step? So it's how do I measure? What do I do next? And that gave me this incredible view into advising large organizations into this transformation journey. And then, we built out this amazing partner network. We weren't actually consulting. We just had this SaaS piece, but then how do you act on it? We were then acquired by Google, so I was CEO. And so I kind of led that acquisition and then the integration and building out these teams in Google. And after that point, I joined GitHub, which is the largest developer network.

**Nicole Forsgren** (00:07:01):
So I had this amazing opportunity to do more grounded and applied research again. I was VP of research and strategy and then went over to MSR, where I kind of wear a couple of hats. So right now, I have a research lab there with an incredible team. It's the developer experience lab where we do a bunch of work across productivity, community, and wellbeing. And then I also help with Microsoft's kind of cross company effort to improve their developer infrastructure. So it's sort of this round effort into how do I really remain engaged in measuring, applying, thinking about this work, both in very applied concrete pieces and incredibly forward-looking work with MSR.

**Lenny** (00:07:48):
Amazing. And just to clarify, MSR, is that Microsoft? That's a division-

**Nicole Forsgren** (00:07:52):
Ah.

**Lenny** (00:07:52):
... of Microsoft.

**Nicole Forsgren** (00:07:53):
Yeah. Thank you. Yeah, MSR is Microsoft Research.

**Lenny** (00:07:54):
Okay, cool. So you've shared a couple of these terms, DevOps, developer productivity. I'm curious what the term you like to use for this area you focus on. Developer productivity, developer experience, DevOps, what's kind of the way the best way to think about this?

**Nicole Forsgren** (00:08:10):
I really love that you asked this question because I think they're very related concepts that people sometimes conflate, but I see them as being different. So related, but different. So productivity, I think, is basically how much we can get done and how much we can do over time. And I think that's why it's so important to have this holistic measure because we can't just brute force it, right. And so that's why when my team and a bunch of my peers study productivity, we include this community effect because software is a team sport. We joke, right.

**Nicole Forsgren** (00:08:40):
And also why wellbeing is so important, right. Because we see that when you do productivity the right way, we see sustainability, we see wellbeing, we see reductions in burnout. Now, developer experience is very related and very tied to this, and it contributes to productivity. But developer experiences, if you think about who your users are, developers really are you your users in this software engineering, in the software development piece. And so developer experience is sort of like what is it like to write software? Is this a friction-free process? Is this a very predictable and certain experience? Can we reduce this uncertainty and increase the predictability here to contribute to productivity?

**Lenny** (00:09:26):
And then how does DevOps fit into that just so that we kind of have the mental model of these terms?

**Nicole Forsgren** (00:09:31):
People have sort of co-opted the term. So some people name their tools DevOps. I'm maybe a little more old school. So when I was doing a bunch of my DevOps research, it was the capabilities and tools and processes that we can use to improve our software development and delivery end to end so that it's faster and it is more reliable. So DevOps was kind of this technical, architectural, cultural practices that enable us to do this work better so that it is, yes, more productive, we have a better developer experience. It was kind of, again, this very holistic picture.

**Lenny** (00:10:06):
So what I love about this topic is that I've never met a founder or a leader who is not thinking, "We need to move faster. We need our engineers to be more productive. We need to get things out the door quicker. We want engineers to be happier." Nobody doesn't want that. And so that's why I'm excited to dig into a lot of these things. Is that roughly what you find as well, that nobody's ever like, "We're good. We don't need any of this. We don't need to focus on this area?"

**Nicole Forsgren** (00:10:32):
You know what, I'll say yes and, right. So and it kind of goes back to why I got into this because, on the one hand, you won't say anyone who's saying, "Uh, we don't really need to go faster. Everything's fine." But at the same time, very often, I will come into scenarios where I'll find myself in scenarios where people are like, "I mean, it would be nice if we were going faster, but do we really need to. Show me the business case. What's the ROI?" Or if we go too fast, we'll have an instability, right. What are our safety measures? Are we going to lose reliability? What is happening?

**Nicole Forsgren** (00:11:13):
And when I first started ITIL and ITSM, right, the old school kind of change management processes, the common knowledge was that you had to have at least a two-week wait for change approvals in order to get that stability. Turns out that's not right. It was just kind of an old wives' tale, right. And so we kind of have this weird balance of I want to move faster, but is it worth the investment? What am I going to get for it? Are you sure this is the priority? Or I've been in meetings where it's like, "Oh yes, absolutely, right. This is a priority, but it's the lowest priority."

**Nicole Forsgren** (00:12:00):
And I'm like... Right. So then what we want to do is we want to have these kind of pointed conversations or these kind of Socratic type questions and conversations where it's like, "Help me understand more what your concerns are. Are your concerns around reliability when you move faster?" We're not just trying to all the guardrails down and sprint for no purpose of sprinting. And this is where kind of the DORA and DevOps research program comes into play where it's we don't just want to move fast and take all guardrails down.

**Nicole Forsgren** (00:12:33):
We want to implement good technical practices like automated testing, good architectural practices so that when you move fast, you are also more stable, right. We want to be thinking about improving the developer experience so that when we are faster, we are also saving time. And then, we can highlight a handful of statistics. Like, what is your typical time for feature delivery? What is your typical time to first PR? What is your typical time to steady state productivity?

**Nicole Forsgren** (00:13:04):
What is your typical time for code review and PR process? And if we are to do back-of-the-napkin math, what sorts of time are you spending here? And if we do a rough look at industry, what are your peers spending here? And are we losing time, right? And if we could turn this into a value calculation, what does that look like? So that we can think about the priority and the strategy here. And I think that's where it becomes a more focused conversation.

**Lenny** (00:13:43):
This is a great segue too, something I was going to get to a little bit later, but let's just get into it, which is the DORA framework, and then there's also the SPACE framework. Could you just talk about what these two are when you use one versus the other, and then how that essentially helps you measure and then improve productivity and developer experience?

**Nicole Forsgren** (00:14:01):
Sure, sure. Absolutely. And I'm so glad you brought this up. So DORA is, it's an entire research program. Now, many people, when they hear DORA now they think of the Four Keys or the DORA four or the four metrics, and I think that's what the research program and the company ended up becoming most known for. And so that was the software delivery performance metrics, and those are, there's two speed and two stability metrics. So the speed metrics are lead time, so how long does it take to get from code committed to code running in production? Deployment frequency. How often do you deploy code?

**Nicole Forsgren** (00:14:40):
And then the stability metrics are MTTR, mean time to restore. So if something happens, how long does it take you to come back? And then change fail rate. For every change that is pushed, what's the rough percentage of incidents or that require human intervention, right? Now the thing that was really interesting is when we started measuring these, we found that they move in tandem now with very strong significance from a statistical standpoint. Now what this means is now, we say speed and stability move together.

**Nicole Forsgren** (00:15:19):
Most people only think about this from the speed standpoint, which means when you move faster, you are more stable, which means you're pushing smaller changes more often, right. Because if you're pushing all the time, it's going to be very, very small changes, which means you have a smaller blast radius, which means when you push, you have an error in production. It's going to be easier to debug, right. It's going to be much easier to figure all of that out your mean time to restore and mitigate. It's going to be much faster.

**Nicole Forsgren** (00:15:44):
But that also means is the reverse. When you push changes less frequently, you will have more unstable systems because when you push less frequency, you will have very, very large batch changes, which means you'll have a very high, very large blast radius, which means when you do have a resulting bug error, you will have to disentangle this big ball of mud and figure out which piece actually caused the error, figure all of that out. That ended up being a big surprise because, refer to my prior comment about ITIL and ITSM, if you're forcing a two-week pause for change approvals, you're causing this batching up of changes.

**Nicole Forsgren** (00:16:36):
And sometimes people were waiting, "If two weeks is good, a month must be better, or three months must be better, or six months must be better." And I mean, just think about the merge conflicts you're causing, right. You're just causing so many challenges in figuring out how to push this code into production. So many people think of those four metrics, one, because we found that speed and stability move together, and two because we started publishing benchmarks on what this looks like for low, medium, high, and then elite performers for many times.

**Nicole Forsgren** (00:17:06):
This, I believe, may have been interesting. I'm not sure if it was useful or helpful, but I think it was interesting because it gave people at least something to shoot for, something to aim for. I will definitely say what's most important is knowing where you are and the progress that you're making, right. It doesn't matter if, frankly, you're a high performer or you're an elite performer. It matters that where you are and you're making progress, right. Can you push daily or on demand, or is your only technical capability that you can push twice a year, right? Just know where you are, and is it a business decision or a technical capability? That's basically what it comes down to.

**Lenny** (00:17:47):
I'm going to jump in real quick just to highlight what you're talking... what you just said, which I think is extremely important and powerful, and people might kind of move on too quickly. I also want to ask you what actual benchmarks are, if you can share those, whatever you want to share there.

**Lenny** (00:18:01):
But before I ask that, essentially, what you're sharing right now is just I feel like the 64,000 dollar question of this episode is just how do I move faster as a team? And what I'm hearing is essentially it's ship smaller things is kind of at the core of it. And also, if we're... if quality is low, you're also saying the answer is ship more often, ship smaller things. Is that roughly the message?

**Nicole Forsgren** (00:18:26):
Yes, absolutely. It ends up being much, much safer.

**Lenny** (00:18:29):
Amazing. So I think that's an extremely important takeaway that I think people would... I don't know. That's surprising to me to hear that it's quality comes from shipping faster and then also to ship faster, and move... help your team move faster. It's ship smaller things and just deploys more often.

**Nicole Forsgren** (00:18:47):
Yep.

**Lenny** (00:18:48):
Amazing. Okay, great. I know we'll talk more about this, but let me go back to the question I was going to ask is-

**Nicole Forsgren** (00:18:53):
Yeah.

**Lenny** (00:18:55):
... are there benchmarks you can share just right now that you think would be useful to people? I know you said it was interesting and maybe not as useful to people as you mentioned.

**Nicole Forsgren** (00:19:00):
Yeah. So I will admit I only have the 2019 benchmarks top of mind. The team at Google has continued that work since I left. It's been led by Dr. Dustin Smith. Nathen Harvey continues the work, so huge shout out to that team. Many others participate. You can go to dora.dev and find all of the continued reports. They've integrated all of this work.

**Nicole Forsgren** (00:19:26):
But I will say they've remained fairly consistent. So really quickly, I'll share the elite performance. So deployment frequency, you can deploy on-demand, lead time for changes takes less than a day. Time to restore is less than an hour, and your change fail rate is between zero and 15%.

**Lenny** (00:19:44):
Amazing. Okay, I'm writing these down. These are extremely valuable.

**Nicole Forsgren** (00:19:48):
And I will mention people will say, "Well, this is kind of a chunk of time, right. It's not super precise." Precision isn't really super important here, right. It doesn't really matter if your lead time is... If it's less than a day, it's less than a day, right. That's fine from a business perspective. It doesn't matter if it's four hours or four hours and two minutes.

**Lenny** (00:20:14):
Mm-hmm.

**Nicole Forsgren** (00:20:15):
Right. General categories are fine. Now I will say the next category for lead time for changes by the day is if lead time is between a day and a week.

**Lenny** (00:20:28):
And this is for good.

**Nicole Forsgren** (00:20:29):
For high between elite and high. Elite is less than a day, and high is between a day and a week. And then it goes between a week and a month and between a month and six months, right. So you can ask people, and they can tell you, right. They can kind of hunch it.

**Lenny** (00:20:44):
Mm-hmm. And this is from committing code into the repo and it going out into production.

**Nicole Forsgren** (00:20:51):
Mm-hmm. To about ring zero. So you don't... Don't worry if it's like, "Oh, well, now we need to think about the global deploy and which is the final endpoint." And it's like, how long does it take to get through your deployment pipeline? Because are you going to be surprised? Do you have fast feedback loops? How does your deployment pipeline work? Does your deployment pipeline work, right? Or are you going to commit code? Are you going to wait for that final review for about three months?

**Nicole Forsgren** (00:21:27):
Is something going to happen or break? And when it comes back to the developer because something happened or broke because that kind of happens, are they going to have to insert themselves back in the code, re-review all the things that happened three months ago, all... so many other things happened that's incredibly difficult. Which to your prior question, this is how it relates to the developer experience. If something happened less than a day and it's a surprise, and it's not great, but whatever, something happened downstream, and I got to fix it, I'm still sitting in my code in my head.

**Nicole Forsgren** (00:22:01):
I've got that mental model. I know what happened. Maybe it's not great, but it's fine. If it happened three months ago and I get interrupted. First of all, interruptions suck. That's not fun. Second of all, now I've got to re-remember, reread all of this code, maybe reload an entire new workspace and set up libraries and everything because I... maybe it's a whole quarter ago, and we thought we were done, and I got to do the whole thing all over.

**Lenny** (00:22:33):
If a listener is working at a startup, I imagine they're hearing this, and they're like, "Takes a day to ship that. We ship it all day, a thousand times a day." I imagine these benchmarks are more valuable for larger companies. Is there kind of buckets you think about for here's the size of company this is meant for? And then, do you think about anything differently for a startup, say, I don't know 10 people?

**Nicole Forsgren** (00:22:55):
If anyone is only listening to this? I just got the biggest smile because we saw no statistical significance between small companies and large companies. The only statistically significant difference was with retail. I'll come back to that. It's so funny because large companies would say, "Oh, but this isn't fair for us. We have more complex code bases. We have so many things to do. Small companies just don't have to deal with this." Small companies would come to me, and they would say, "Oh, but this isn't fair. Large companies have so much money. They have so many resources. They don't have to deal with all the things. This doesn't apply to me."

**Lenny** (00:23:30):
Wow.

**Nicole Forsgren** (00:23:30):
So it's either way. And on days when I was feeling real snarky, I'd be like, "Pick your excuse. You've got your dropdown." Now when I say, "Retail was a bit of an outlier," they had a statistically significant difference. Their difference was that they were actually better. Why? Now, I can't tell you why. I can... In a research paper, we'd have a discussion section, and this is where you get to guess. I would surmise, and we do this in the report, that it's probably because retail went through the retail apocalypse, right.

**Nicole Forsgren** (00:24:09):
If you didn't survive, if you weren't just killing it, you did not survive. So many retail firms just did not make it through. You had to be at the top of your game. There was no such... Black Friday, there's no such thing as not having systems that are performing incredibly well. There's no such thing as not being in the cloud because if you cannot make it through bursting on demand, bursting like magic, sometimes I joke, right, you're not going to make it. And so I suspect if I were to guess, if you're not already a high performer in the retail space, natural selection got rid of you.

**Lenny** (00:24:53):
That is really interesting. That makes a lot of sense. So I'm looking at these thresholds again, and I'm thinking from the perspective of a founder who's just like, "I wish my engineering team would faster." Essentially you're saying if deploy times... if they deploy more than once a day if their deploy frequencies on demand or I think it was hourly was kind of the other bucket, was that part of it?

**Nicole Forsgren** (00:25:14):
Mm-hmm.

**Lenny** (00:25:14):
Yeah. And then, their mean time to their fail rate is less than 10%, and their mean time to recovery is less than an hour. Basically you're doing great. That's kind of the message of this framework, at least.

**Nicole Forsgren** (00:25:29):
And if you're not doing it through brute force and killing yourself. Now can I jump in here, please? Because then people are like, "But how do I do this?" So let's say that you're not in that category, and you're like because this is the next... this is the piece of criticism I'll get about DORA, right. People are like, "Well, all you've done is make me feel bad. You gave me these metrics. You've judged me. Now I feel bad." And then I'm like, "So there's DORA there." I wrote a book called Accelerate, which is the first four years of the research compiled and put together and expanded in a few things.

**Nicole Forsgren** (00:26:06):
And I'll joke, "There's a whole rest of the book, right." DORA is best known for the four metrics, but there's an entire research program supporting it. So it's not just these four metrics. What we find is that if you improve a set of capabilities... I loved your question around what is DevOps? DevOps is not a toolchain you buy. Marketing teams labeled toolchains DevOps because they wanted your money. DevOps is a set of capabilities. They're technical capabilities. They're architectural capabilities.

**Nicole Forsgren** (00:26:34):
They're cultural capabilities. They are lean management practices that predict speed and stability. And then speed and stability gives you money, right, because it's your ability to create these features that give you money. So when you work backwards, if you want money, you get the features fast. If you want the features fast and stable, you do the things. And the things are technical capabilities like automated testing, CI/CD.

**Lenny** (00:27:04):
And CI/CD is continuous integration/continuous deployment, is that right?

**Nicole Forsgren** (00:27:08):
Yes. Mm-hmm. Trunk based development using a version control system, right. So do you have good technical practices? Do you have good architectural practices? Do you have a loosely coupled system? Are you using the cloud? Or if you're not in the cloud for whatever reason, are you using the underlying architectural pieces that enable good cloud to do the cloud, right? Or if you're in the cloud and you're not realizing benefits, is it because doing the cloud wrong, right? Do you have a good culture?

**Nicole Forsgren** (00:27:48):
So you don't just magically go fast and have stability. So working backwards, which pieces are you struggling on? Now you kind of noted down the benchmarks. If you go to dora.dev, the team at Google was lovely. We worked really closely with the team, and they're keeping this updated. You can take a quick check. There's a button there that says, "Quick check." And you can plug in where you kind of think you are.

**Nicole Forsgren** (00:28:13):
Like I said, you can hunch it, and it'll tell you where you are in the benchmarks today and what industry you're in. And then the cool part is it'll say... Now you'll want to ask yourself, "Where am I struggling?" But it'll say, "For your performance profile and for the industry that you're in, statistically over the last several years, these are probably your constraints, AKA, these are probably the things that you're struggling in right now." For people in finance who are high performers, they tend to struggle with these four things, whether it's like culture or continuous integration or whatever.

**Lenny** (00:28:54):
I love that you're getting tactical with how to actually improve these already, which is the bread and butter of this podcast.

**Nicole Forsgren** (00:28:54):
Mm-hmm.

**Lenny** (00:28:59):
And so we'll link to this quick check. [inaudible 00:29:02] dora.dev/quickcheck.

**Nicole Forsgren** (00:29:03):
And by the way, they do not collect your name. They do not collect your info. There's no lead, lead gen, anything. Everything's just there. And then, there's deep dives into every single one of the capabilities.

**Lenny** (00:29:15):
Amazing. And also your book talks about all these things. So people should go check out the book. Obviously it's on Amazon. Search Accelerate. Is that right?

**Nicole Forsgren** (00:29:22):
Yep.

**Lenny** (00:29:23):
Okay. So we were talking about DORA. This may be a good time to talk about SPACE, which I think is a different framework you recommend. What is that all about?

**Nicole Forsgren** (00:29:31):
Okay, so SPACE is a way to measure, we say productivity, developer productivity, but it's a little bit more than that. SPACE is a good way to measure any type of complex creative work. Now, how do they relate? Let's say you go through the quick check. It points out four things, and you decide you want to improve continuous integration and culture, right. Well, now you're like, "Cool, but how am I going to actually measure them?" This is where SPACE comes in because SPACE helps you figure out SPACE gives you a framework to pick the right metrics.

**Nicole Forsgren** (00:30:12):
Now some people are like, "Well, SPACE you didn't give me the exact metrics." People love DORA because it's like, "Here's the exact four you need." Well, SPACE is like when you want to measure something that's complex, creative work, maybe like developer productivity. There's also an example at the bottom for incident management. When you have something you want to measure, it says, "Within your context, within the metrics you have available to you, here's how to pick."

**Nicole Forsgren** (00:30:40):
That's what SPACE is good for. Now we called it SPACE because it stands for the five dimensions that you want to measure. So S is satisfaction and wellbeing. So satisfaction wellbeing is kind of self-explanatory. Now, some people might jump in here and say, "Oh, well, you're just touchy-feely." This actually matters because we find that satisfaction wellbeing ends up being incredibly highly correlated with all of the other dimensions of productivity and doing things well.

**Nicole Forsgren** (00:31:09):
And as soon as satisfaction and wellbeing, things like sustainability, if you're satisfied, as soon as that starts falling off, other things start to break. So this can be an incredibly strong and important signal. P is performance. This is going to be the outcome of a process. So reliability within DORA, the MTTM or change fail rate. Those are both performance metrics.

**Lenny** (00:31:34):
And so you pick one to kind of measure as performance.

**Nicole Forsgren** (00:31:37):
Yep. A is activity. Anytime you have a count or a number of something. And these we see all the time because they're super easy to instrument and automate, right. Number of pull requests, number of check-ins, number of something, that's A. C is communication and collaboration. This can be how people work and talk together. It can be meetings. It can be collaboration. It can also be how our systems communicate together. It can be the searchability of a code base. And then ease efficiency and flow. So this is going to be the flow through the system.

**Nicole Forsgren** (00:32:11):
It can be the time through the system. If we think about SRE or incident management, it can be the number of hops a ticket takes until it reaches the right person. Now, to use SPACE correctly, we want to use at least three dimensions at a time because that helps us balance. Turns out DORA is actually implementation of SPACE. So DORA would be SPACE for mostly that outer loop. So again, once you've found something that you want to improve, find the metrics that make sense to you, try to have them be in balance or intention so you don't throw something out of whack, but pick three.

**Lenny** (00:32:52):
So when you say DORA is an implementation of SPACE, one has five buckets, one has four. How do you actually think about that?

**Nicole Forsgren** (00:32:59):
So SPACE is there to help you think about how you want to pick metrics, right. So a lot of time I see people... So [inaudible 00:33:06] half step back. I used to advise people on how to pick metrics. For years people would pull me in to advise on DORA or Accelerate. They would ask me questions, but it ended up being metrics questions a lot. "How do I pick the right metrics to improve what I'm doing?" Like I said, they had the DORA numbers. They would pick their constraints, and they wanted to improve.

**Nicole Forsgren** (00:33:29):
"But how do I improve? How do I measure this? How do I show improvement?" And so we would start thinking really critically about which metrics were the right metrics to pick. And I would always say make sure you pick balanced metrics. Make sure you pick metrics that are intention. And I could say it, but people have a hard time wrapping that around their heads because they kept picking things like number of lines of code, never picked number of lines of code.

**Lenny** (00:33:29):
Oh, wow.

**Nicole Forsgren** (00:33:58):
Number of... Still every month I get email about this.

**Lenny** (00:34:00):
Wow, [inaudible 00:34:00].

**Nicole Forsgren** (00:34:00):
Number of pull requests, number of commits. And I was like, "These are all activity metrics." And so finally I pulled a few of my friends together and I was like, "Let's come up with a framework to help people think about it." And so there are five broad categories, pick three because that will help force you through the mental exercise of, "What could I possibly pick?" You don't need all five, right. This isn't... We're not playing-

**Lenny** (00:34:00):
I see.

**Nicole Forsgren** (00:34:28):
... bingo. We're not playing blackout bingo. You don't need all of them but try to have at least three across different dimensions. Now one example here. I was working with a group that wanted to improve their pull requests very generally. They just said improve pull requests. So they were thinking about pinging someone every 15 minutes, and I was like, "Oh, this is going to be bad."

**Nicole Forsgren** (00:34:48):
Because we know from other literature and research like nursing, you'll get alert fatigue, where people will just start tuning out alerts. Either they'll turn them off, or they will just stop hearing them. So number of alerts, right. They're like, "Let's just think about number of alerts." And I said, "Well, but if we think about efficiency and flow, how much time do you have to work on your coding?" So those two are balanced. So we need to protect time to work as well as code review time.

**Lenny** (00:35:25):
Mm-hmm.

**Nicole Forsgren** (00:35:25):
Like pull request time. And so, sometimes we can think about those, and then I think we added a satisfaction metric. Are you satisfied with the pull request process and the selection of the reviewer?

**Lenny** (00:35:39):
How do you go about actually capturing and measuring the say satisfaction?

**Nicole Forsgren** (00:35:43):
So for satisfaction, I would generally ask, right. Go ahead and ask now the ones that you instrument, you can instrument and pull out of systems all the time, right. Go ahead and grab that string. For satisfaction metric, I would only pull that periodically once every few months.

**Lenny** (00:36:01):
Like a survey to your engineering team.

**Nicole Forsgren** (00:36:02):
Like a survey. Yep, absolutely.

**Lenny** (00:36:04):
Awesome.

**Nicole Forsgren** (00:36:05):
And don't discount what people say, right. Sometimes I hear... Actually not sometimes. A lot of times, I hear people say, "Oh, but people lie." First of all, what is their incentive to lie? Why would they lie about having a bad system? Because it's bad, and they want it fixed, right. If it's absolutely a hostile work environment, they might lie and then tell you it's good. Then you have bigger problems, right. Also, do we ever see bad data from our systems or incomplete data from our systems? That's a lie, but we find ways to deal with it. We see it. We acknowledge it. We look for holes in our system data.

**Nicole Forsgren** (00:36:46):
We try to deal with it, right. That's also a lie. So I think there are better ways to think about and deal with that and then try to work with it because and I wrote a paper with Mik Kersten on this several years ago on how data from people and data from systems are really important compliments because we can get certain insights from people that we'll never get from systems. Let's look at lead time from changes, for example, from commit to deploy. The speed might be fine, but people might tell you it's taking absolute heroics. It's some ridiculous Rube Goldberg machine.

**Lenny** (00:37:27):
Mm-hmm.

**Nicole Forsgren** (00:37:27):
The system will never tell you that. Or you could get data on your version control system. I worked with a company several years ago, and we found out that there was a significant portion of code that was just not going into any version control system. You're never going to find that out from your systems because it's not in the systems, and it was mission-critical.

**Lenny** (00:37:52):
I can see why people come to you asking for advice on metrics because you have this framework of, "Here's the type of metrics you want." And then I think, and especially from an engineering team, there's going to be this like, "How do I optimize and make sure I'm doing the right thing and measuring the right things." For someone that wants to do this, and an hour long podcast isn't going to give them all the answers, what do you recommend they go read or go do or look at to help them figure that out?

**Nicole Forsgren** (00:38:17):
One, I hate to be this person, but I'll point to a few of my papers. I will say I write things down because I get asked them so often, and I want to make sure it is broadly applicable or broadly available, I guess. This SPACE paper, for sure. It's an ACM, and I think the year we published, it was the most read paper at ACMQueue. Yeah. So we tried to make it as readable as possible. So the SPACE paper is nice because it outlines this framework and it gives examples of metrics in every single category. And so hopefully people can look at this and they can [inaudible 00:38:54], "Okay, here's an example to use here, right. Here are some of the things that I could possibly use."

**Nicole Forsgren** (00:39:00):
And we're seeing that SPACE is being used lots and lots of different places. Another good one could be the paper that I mentioned with Mik Kersten, and it was about... we talked about using data from people and data from systems. We wrote it up in the DevOps context. I want to say this was written in 2016 or 2017 or something. But it helps you think through what types of data are good in which situations because you will never find yourself in a situation when you don't want both types of data. Even teams that I've worked with that are the most advanced. They have absolute instrumentation in every possible scenario.

**Nicole Forsgren** (00:39:46):
In the most detailed way, they will still survey their developers at least once a year because you can get new insights, right. One book that I love. It's a little dense, but it's really interesting that I love is How to Measure Anything. And it's by Hubbard, and there are parts of it that are real stats heavy, but he has this portion in the front that's in covering intangibles. So it's like, what happens when you don't have data? You have no data. You're starting from nothing. What are good ways to hunch data? And I really love that because he covers some really good ground there.

**Lenny** (00:40:34):
Today's entire episode is brought to you by DX, a platform for measuring and improving developer productivity. DX is designed by the researchers behind frameworks such as DORA, SPACE, and DevX, including Nicole Forsgren, who is my guest for this very episode. If you've tried measuring developer productivity, you know that there are a lot of basic metrics out there and a lot of ways to do this wrong, and getting that full view of productivity is still really hard.

**Lenny** (00:40:59):
DX tackles this problem by combining qualitative and quantitative insights based on the very research Nicole and her team have done, giving you full clarity into how your developers are doing. DX is used by both startups and Fortune 500 companies, including companies like Twilio, Amplitude, eBay, Brex, Toast, Pfizer, and Procter & Gamble. To learn more about DX and get a demo of their product, visit their website at getdx.com/lenny. That's getdx.com/lenny. You also mentioned offline that you might be working on a book that will answer a lot of these questions. Is that something you're up for chatting about?

**Nicole Forsgren** (00:41:36):
Yeah, absolutely. So as I mentioned, I tend to write things down when I get asked questions on it a lot. And so this is one in particular. So we'll be covering... I'm starting to go through, and I'm covering some of these, and I think some of the important topics in particular are starting with what is your problem or what is your goal and being super, super crisp on it, right. What is it that we're trying to answer? And I would say this is a bigger challenge than most people recognize or realize.

**Nicole Forsgren** (00:42:11):
I'm making this set up, right. 80% of the folks that I work with, this is their biggest problem. Even at executive levels, they'll ask their team, or teams will come back with uncertainty, and they'll say like, "Well, you told me to improve developer experience." I'm like, "Okay, great. What do you mean by that?" And then teams will have gone off for several months, and they're tackling something, and they'll come back, and they'll be like, "Oh, well, wasn't what I meant."

**Nicole Forsgren** (00:42:38):
And I'm like, "Okay, what do you mean by this? Are you talking about inner and outer loop? Are you talking about friction? Are you talking about culture?" Because sometimes they're talking about culture. "And if you're talking about culture, this is an incredibly valid answer. But if you're talking about culture, this is totally different than if you're talking about friction in toolchains, right. And if you're on different pages, you're heading in completely different directions."

**Nicole Forsgren** (00:43:05):
So that's one thing we cover, which seems obvious, but trust me, it's not. And then even how do you... We're going to do kind of a rough version of how do you start measuring from nothing and also the measurement journey, right. How do you think about the trade-offs between and the proportion of measurement between subjective data, right, data from people. So you have Azure interviews, and you have surveys and objective data, stuff you get from systems. Because when you first start off, you'll be relying much more on data from people.

**Nicole Forsgren** (00:43:42):
You can get a relatively quickly. But as you kind of transition through this measurement journey, you'll get more and more data from your systems because it's scalable. It can be engineered. You can be doing much more with it. And also, you should be thinking about, "Don't let the perfect be the enemy of the good." So how do we think about this very, very strategically? How do we transition through this? How do we think about what each piece of data is for?

**Nicole Forsgren** (00:44:09):
And also lots and lots of examples. So I have included example interview scripts. How do you select people? How do you screen people? Example survey scripts. What are some of the analyses we should do? And trying to make this incredibly accessible, so basically anyone can do this. So you do not need to be a data scientist, but if you have one on staff, you can hand them some of this and just let them run.

**Lenny** (00:44:32):
I think this book is going to do extremely well. Definitely come back on when it is out. I think you said maybe year-ish kind of timeframe?

**Nicole Forsgren** (00:44:39):
Yeah, probably about a year by the time we get-

**Lenny** (00:44:39):
Okay.

**Nicole Forsgren** (00:44:40):
... all the way through.

**Lenny** (00:44:41):
If people want to be notified when it's out, can they sign up on your site for newsletter or anything like that? Is there any way to be in the loop as it approaches?

**Nicole Forsgren** (00:44:49):
Oh, yeah, absolutely. Yeah, I'll add a link for that. Also, if anyone is doing some of this work now, if they have major questions that they would love to [inaudible 00:45:00] me to answer if they have success stories, if they have case studies, if they have anything that they would love to be included.

**Nicole Forsgren** (00:45:08):
I remember when I wrote Accelerate before, there were a couple folks that reached out after, and they were like, "Oh, I wanted to have something included." Now I've learned. Today I've learned, right. If there's anything that folks would love to be in discussion with me about, I'm always eager to chat and nerd out about DevX and especially measurement and measurement journeys.

**Lenny** (00:45:28):
Awesome. I usually ask this at the end, and I have more questions, but while we're here, how would people reach out to you? What's the best way to contact you?

**Nicole Forsgren** (00:45:35):
On my website, I've got info.nicolefv@gmail [inaudible 00:45:40]-

**Lenny** (00:45:35):
Okay.

**Nicole Forsgren** (00:45:35):
Yeah.

**Lenny** (00:45:40):
Awesome. Okay, a few more questions.

**Nicole Forsgren** (00:45:42):
Awesome. Thanks.

**Lenny** (00:45:43):
What are the most common pitfalls that companies run into when they're trying to roll out any sort of developer experience, developer productivity, system measurements, improvements?

**Nicole Forsgren** (00:45:53):
I think one I just mentioned, right. Not being clear or not understanding what it is that they're looking for because then you can have a thousand flowers bloom, and everyone's kind of running in a different direction. I think another one is not pursuing this in both a top-down and a bottom-up structure, right. And I think that can really help drive success, and having good communication throughout is super, super important, right.

**Nicole Forsgren** (00:46:26):
So getting your ICs bought in and helping them understand that this is for them. We want to understand what they're doing. Knowing what vocabulary they use, what terminology they use is super important. And then chatting with leaders and understanding what their motivations are or helping them understand what the motivations could be. This kind of hearkens back to one of our earliest chats on why I even got into this and how I see two different sides to the conversation on why is DevOps even a thing? Why should we even ship faster?

**Nicole Forsgren** (00:47:03):
There are so many people that I talk to that are super passionate about DevX right now, and they're like, "How can I convince my executive team this is important?" Because their developers are just completely burning out, or they use computers and anger every day. And so it's like, "How can we have the right tools to socialize this to our leaders as well?" Because this should be a priority. This needs to be a strategic piece, and how can we help pull together the right value points to communicate this and to understand what their priorities are so that we can see how this fits in, right.

**Lenny** (00:47:43):
You've been working in this space for a long time, probably longer than anyone that has ever worked on this area of developer experience productivity. What have you seen change most from the time you started working in the space to today? What kind of progress has been made?

**Nicole Forsgren** (00:47:58):
We have these increasingly large complex systems, right. So 10 or 15 years ago, the internet was around, but things were really different. Now, almost every company has a really large complex system, right. We also have a shortage of developers, or at least a reported perceived shortage of developers. More companies are technology-driven, or at least they understand they're technology driven. It's like I understand a handful... I remember a handful of years ago when I met with a financial institution whose CTO insisted to me that he was not a tech company like that. That's not real anymore. That doesn't happen anymore, at least very, very rarely.

So all of these things come together, and suddenly, many more companies are like, "We have to be better at this." And that was not always the case five to 10 years ago. I used to have to really explain why this was a pressing concern and why it would continue to be a oppressing concern. And now, in the last six to nine to 12 months, we have this AI moment happening, and it just poured gas on top of everything because now what's important... We've always said that [inaudible 00:49:14] execution's important, but now this is absolutely true because it's not just about what it is that you build.

**Nicole Forsgren** (00:49:23):
It's about creating absolutely novel, incredibly new experiences and doing them at a speed that no one has seen before. And the only way to do this is to have this software pipeline that is fast and is safe and is stable, and is reliable. And that's where we're seeing this really interesting convergence, and pressure isn't quite the right word, but it's really forcing the discussion and strategy and prioritization, right.

**Lenny** (00:50:07):
I'm glad you touched on AI. That was actually exactly where I was going to go next.

**Nicole Forsgren** (00:50:11):
Perfect.

**Lenny** (00:50:12):
Yeah, obviously productivity, AI, engineering, something that's top of mind for a lot of people. There's a lot of layoffs that have been happening. There's a lot of talk of we don't need as many engineers. I actually had dinner not too long ago with few, I'd say 10 X engineers, and those are folks that people sometimes say they don't need Copilot. They're not going to use any of these tools. They're already amazing. And they were the opposite.

**Lenny** (00:50:34):
They're like, "This is making me 100% more effective and efficient and I love it." So clearly good things are happening there. I don't know what the question is specifically, but I guess have you seen the impact of AI on engineering productivity? And has that shifted how you think about developer experience and productivity beyond what you already just shared?

**Nicole Forsgren** (00:50:54):
Absolutely. So yes and, right.

**Lenny** (00:50:57):
Mm-hmm.

**Nicole Forsgren** (00:50:57):
I think this is a super interesting open question. So can I answer it just with a whole bunch of questions?

**Lenny** (00:51:03):
Absolutely.

**Nicole Forsgren** (00:51:04):
We're absolutely seeing an impact, and we continue to explore this. So I have an interesting question to see how it'll change the SPACE framework. What's open here? I think a few things will remain, right? Satisfaction's still going to be there, performance is still going to be there, activity's still going to be there. How you communicate with people and with the tool efficiency and flow is still going to be there. I believe it will change and add a dimension like trust or reliability. How do I rely? Can I rely on it?

**Nicole Forsgren** (00:51:35):
Will I have an over-reliance on it? And what we're seeing is that, probably unsurprisingly, people really fundamentally shift the way they work when they work with an AI-enabled tool like GitHub, Copilot, or Tabnine or others because now, instead of just writing code or having a short auto-complete, you spend more time reviewing code than writing code, right. There's this wonderful paper out that uses the Cups model. I'll share it with you. A team at MSR did it. Defines it, "About 50% of your time now is spent reviewing versus writing." But it'll be interesting to see how that changes things longitudinally, right.

**Nicole Forsgren** (00:52:20):
Because other... Some of my colleagues also did a paper that showed that you can do certain tasks, like build an HTTP server 50% faster, but I don't think that's what productivity's about when you're using an AI tool, frankly. Anyone who's looking at that and dear CEOs or whoever who are like, "Now I can lay off half my workforce." That's not what this is about, right. It's not about taking a task and cutting your time in half because now what we've enabled is your ability to do certain things faster and then free up some of your cognitive space so that you can do harder things with this new co-pilot sidecar or something, right.

**Nicole Forsgren** (00:53:03):
But also, because now you're accepting text and then reviewing it, we've changed what your mental model is. So we've changed the friction model that you expect. We've changed the cognitive load of what you expect. We're changing reliance on code. So what does this mean for reliance or overreliance? What does this mean for learning? What does this mean for novices versus experts? How do we measure productivity, right? There are a handful of us that are having these discussions on what does this mean and how do we communicate it thoughtfully?

**Nicole Forsgren** (00:53:35):
Again, we really need to have these kind of holistic, balanced metrics because if it's an oversimplification, we really risk losing the forest for the trees, right. But it's also super interesting and super compelling, I think. How can we think about learning or onboarding to new code bases or new languages for folks who already know computational learning? I think it's also very different for folks who are just learning programming languages and don't already know things like computational thinking.

**Lenny** (00:54:05):
If someone was excited to kind of go down this road of, "We're going to focus on developer experience. We're going to focus on helping our engineers be more productive," what are the next step or two that they should take in your opinion just broadly knowing that you don't know any specifics about, say, the company that's thinking about this right now?

**Nicole Forsgren** (00:54:22):
I think if you're walking away from this podcast and you're like, "I'm already working on this, or I think this is a thing that's happening," I would say just go check your work basically, right. Has this been written down? Is there a clearly defined challenge, problem, something? Start there.

**Nicole Forsgren** (00:54:45):
Absolutely because that is going to be the thing that reduces confusion the best, right. Absolutely. And then see if there's any data. And data can be very loosely defined, right. Is there any signal that is related to the problem? I'd start there. And you can do that. You can do that in a week. You can hunt something down.

**Lenny** (00:55:09):
Sounds like something you could do in a day.

**Nicole Forsgren** (00:55:10):
Yeah. Well, depending. Depending on how scattered things are.

**Lenny** (00:55:16):
Are there any companies that you look at as good models of, "They do this really well."

**Nicole Forsgren** (00:55:20):
I think Google does this incredibly well, and sometimes I hesitate to mention Google because they're like... some people are like, "Well, we can't be Google, and we aren't really advanced." But the thing I love about Google's approach is that they've really taken kind of this measurement phase approach to things, even when they roll it out in new places. They're very systematic in how they measure things.

**Nicole Forsgren** (00:55:41):
They have incredible telemetry and tooling and instrumentation, and they continue to invest time in developer experience surveys and they triangulate them. And one thing that I also love being able to point out here is if there is ever a disagreement between the surveys and the instrumentation, which is incredibly advanced, almost every time, every that I've ever heard of the surveys are correct and not the instrumentation.

**Lenny** (00:56:11):
Amazing. I have just a couple more questions unrelated to this topic. Is there anything else that you thought you think would be useful to share or leave people with around this general space?

**Nicole Forsgren** (00:56:22):
I would say that thinking about what it is you want to do is always important, right. Like getting crisp, the ability to communicate clearly is always one of the best things. I think one of my superpowers and one of the things that I've been working with my teams on doing and kind of teaching them is, and one of the things that's really leveled up our work in general, is making your work incredibly accessible.

**Nicole Forsgren** (00:56:49):
And accessible, not necessarily in the accessibility definition of the word, but making it very easy to understand what you're doing for your key audiences. And so thinking about doing that for anything that... anyone who's listening for all of your work is super important, right. So who is it that your audience is? What's their role? What words resonate with them? And then always being able to translate your work into a few sentences or a paragraph or left... or less.

**Lenny** (00:57:21):
I love it. A lot of the listeners of this podcast are product managers, and this is so core to the work of a PM, so I think this is [inaudible 00:57:28]-

**Nicole Forsgren** (00:57:28):
Perfect.

**Lenny** (00:57:29):
... speaking verily directly to a lot of the listeners. Okay. So just a couple more questions. Before this podcast, I asked you a few questions, including just like what are people asking you for advice often around, and are there any other frameworks that you find really useful? And so there's a couple of things I just want to touch on, see if there's something interesting there. The first is you have this framework that you call the four-box framework. I'm curious what that is and what it's all about.

**Nicole Forsgren** (00:57:53):
Yes. I love this four-box framework. I've used it for years. I actually pulled it out first when I was a professor, and I still to this day get LinkedIn messages from my students saying that it's like the most useful thing they've ever used. So here's what it is. I literally pull this out on napkins, at bars, at conferences to this day.

**Nicole Forsgren** (00:58:12):
So here we go. Draw four boxes on a piece of paper, two on the top, two on the bottom. So they'll be kind of aligned. The first two to the left of them write the word words. And below them, write the word data and then between the two on the top, draw an arrow between them. So it'll say words, box, arrow, box, right. Is that making sense?

**Lenny** (00:58:12):
Mm-hmm.

**Nicole Forsgren** (00:58:42):
And then on the bottom it'll say data, box, arrow, box.

**Lenny** (00:58:46):
Mm-hmm.

**Nicole Forsgren** (00:58:47):
Okay. So on the top half, this is where, if you want to think about measuring something or testing something, you have to start with words. So as an example, let's just say, I think that customer satisfaction gets us more money or customer satisfaction gets us return customers. Let's do customer satisfaction. So the first box, you'll put customer satisfaction inside the box, and you'll put return customers in the second box. Now always start with words. You do not start with data. You always start with words.

**Nicole Forsgren** (00:59:22):
And then you'll go around to a couple of people, stakeholders, managers, others, and you'll say, "Do you agree with this? Is this actually what we're doing?" And it can turn into a sentence. And then, in the boxes below it, this is your data. How are we going to measure customer satisfaction? It could be a survey.

**Nicole Forsgren** (00:59:44):
And so this is where you'll go, and you'll say, "What data points do we have that could proxy for? What could be our data points for customer satisfaction?" And this is where it gets tricky. You could say, "Well, customer satisfaction could be return customers. But we think it leads to return customers, so we can't use that here." But return customers could be... So that's where you kind of roll this out. So how else would we measure customer satisfaction? I made this hard on myself.

**Lenny** (01:00:10):
Like a CSAT score or NPS score.

**Nicole Forsgren** (01:00:10):
CSAT. Yeah.

**Lenny** (01:00:10):
Yeah.

**Nicole Forsgren** (01:00:10):
CSAT, NPS.

**Lenny** (01:00:10):
Mm-hmm.

**Nicole Forsgren** (01:00:16):
We could say the amount of money that they spent. It's a stretch.

**Lenny** (01:00:20):
Mm-hmm.

**Nicole Forsgren** (01:00:21):
Okay. Now, return customers. Let's go to the next box. How are we going to measure return customers? Depending on our context, let's say that this is an online business. We could say that it's return customers as measured through the website. We could say that it's returned customers. We could just ask them, right. Maybe we have a follow-up survey. Return customers. Maybe we're going to do a stretch here. Maybe we say it's a referral link. This helps us get super clear on what it is we're going to measure. Now, the reason I like this is because if some of our... Now this data analysis. We'll just do correlations here, right.

**Nicole Forsgren** (01:01:00):
If we have longitudinal overtime, that's fine. You can hand this to a data scientist. You can hand this to someone and you can say, "What data do we have? Let's go run this." If something here falls apart, now you can point to the data boxes, and we can get mad about the things in the data boxes, and we can say, "What's wrong? Is the data poor quality? Are we missing data? Was this a bad proxy?" Proxy stands in for something else. "Was this ridiculous?" One of the things I made up. It was just a bad idea.

**Nicole Forsgren** (01:01:32):
Instead of getting mad at Lenny for his really stupid idea or getting mad at Nicole because this was a really bad idea, we can say, "This was problematic. What's wrong here?" Or we can go back up to the words at the top, and we can say, "This is not actually something that is probably going to hold, or this is not something we want to test right now, or this is something instead..." And it makes things incredibly clear. It helps you communicate what it is you want to do fairly quickly.

**Lenny** (01:02:04):
I love it. [inaudible 01:02:05].

**Nicole Forsgren** (01:02:05):
Now...

**Lenny** (01:02:06):
I'll check it out. It's ugly.

**Nicole Forsgren** (01:02:08):
Nice.

**Lenny** (01:02:08):
I'll Zoom in, right. Okay.

**Nicole Forsgren** (01:02:10):
Now I will say advanced mode. You can start with the same four-box framework and you can say, "What data do we have available? What do we think the relationships are?" But then you have to go back up to words and then say, "For these data points..." And we think that they represent something, and we think this is the relationship between them. "What do they represent? If I turn this into a sentence, what do they represent?"

**Nicole Forsgren** (01:02:39):
And then you want to double check because spurious correlation's one of my favorite websites instead of charts. So you'll want to go chat with someone, interview, make sure things are actually right. But the challenge is I will see people run every correlation they could think of, but they haven't turned it into a word or a sentence that you can communicate to someone else.

**Nicole Forsgren** (01:03:01):
They don't do the check, and they don't do that before... one, before running the correlations. And two, if it's there, right. All of our data is so interrelated that we quite often will find spurious correlations. But it can be really helpful just to have that laid out, even if it's just on a post-it, to say, "What are the things I expect to see? What is this actually testing? What relationship do I suspect is there?"

**Lenny** (01:03:27):
Mm-hmm. Amazing. There's actually... I have a newsletter post, a guest post on how to do a correlation analysis and a regression analysis so folks can [inaudible 01:03:36]-

**Nicole Forsgren** (01:03:36):
Awesome. Oh, that's so great.

**Lenny** (01:03:38):
... templates and plug and play, all kinds of... makes it easy for you. So what I'm take away from this is an awesome framework, especially for thinking about a hypothesis you may have. In this case, it's like, "Customer satisfaction's going to lead to more return customers. Here's how we're going to measure it." And then you basically run the test and see if it's true. And if it's not, maybe you need to pick different metrics. Maybe you need to pick a different conclusion.

**Nicole Forsgren** (01:03:59):
And within the DORA framework, we would say, "If we want to improve our speed and stability, we think improving build time would help. And then how would I measure build time?" These are the-

**Lenny** (01:03:59):
Makes them circle back.

**Nicole Forsgren** (01:04:10):
... data points that I have available to us. Yep, to circle back.

**Lenny** (01:04:13):
I love it. It's all connected. Okay. And then, last question. I asked you what advice people often ask you for, and you said that it's around making decisions. And I'm curious, what advice do you give people about making decisions?

**Nicole Forsgren** (01:04:28):
Yes. So this one comes up in business but also comes up personally and among my mentees. So many times, it starts with being very crisp about your objectives and definitions, but then it comes down to really clearly defining what your criteria is. What's important, and then among that criteria what's most important. Some of my friends know I have a decision-making spreadsheet that I have shared out with a handful of friends on should you take a job? Where should you move?

**Lenny** (01:05:03):
Wow, [inaudible 01:05:04]-

**Nicole Forsgren** (01:05:03):
What are the different things-

**Lenny** (01:05:04):
... really useful.

**Nicole Forsgren** (01:05:05):
... you should do. It is. Well, it's funny, though, because what's interesting is many times I will... I'll share it with someone, and I've got a couple that are just funny, right. But walking through the spreadsheet is often all you need to do in order to know what the decision is. And by that, I mean, so we walked through the decision. I had one where it was like, "Where should I move next, or what job should I take?" So when I started DORA, I did this. Starting DORA, I thought, was my lowest. Once I walked through the spreadsheet, it became my high. So what you do is you outline of all of your options. What do you want to do? And then you say, "What are the criteria that are important to me?" So if it's for a job, is it something like total comp, cash money, prestige, team, job predictability, work-life balance. Identify the criteria that are most important to you.

Now it's really interesting because sometimes I will only get that far when I'm working with someone I'm mentoring or coaching, and they will say, "I know what my answer is." We don't even get to the next step. But just identifying the criteria that are important is it... Now, when I was thinking about where I wanted to move next, it was proximity to an airport, the relative tech scene, the food scene. That was real high for me, a handful of things that was important. Now the next thing I do is, for each criteria, what's their relative weight, what's their importance? And I make it add up to 100%. And then I... This is the easy part, right. You just put it in a little spreadsheet, and then I give everything [inaudible 01:06:51] score, and I just multiply it out. Now this is where I'm data informed, and I'm not data driven.

**Lenny** (01:06:56):
Mm-hmm.

**Nicole Forsgren** (01:06:56):
There have been times I make a decision where the whole flip a coin and whatever it's... wherever it lands on, what your reaction is tells you what it should actually be. There have been times where I multiply it out, and then I'll actually fudge the numbers to get what I want, but it's still slightly off. That's per your data informed. Same thing in business. There are many times where you actually run the numbers, and it'll give you a class or a category of things, and then you choose. Now this is where one of my favorite quotes I heard somewhere about strategy comes into play and that's that, "The key to having a good strategy is knowing what not to do."

**Nicole Forsgren** (01:07:40):
And the key to executing a good strategy is actually not doing it. So you can have many options, right. As a leader and as an executive, we have many options, and we only fund some of them. If you fund everything, things are going to fail. So being able to think through and identify what your criteria are, identifying that criteria, what's your selection criteria, what's your evaluative criteria, ranking them, and then deciding what the cutoff is is important. You can't fund everything. You don't get to pick everything.

**Lenny** (01:08:19):
Amazing. I love the spreadsheet idea. I've made versions of it, but it's always... I think, like you said, a lot of times, the exercise is just tell you what you already think and just gives you like-

**Nicole Forsgren** (01:08:28):
Yep.

**Lenny** (01:08:29):
"... All right, you're right. You probably should just do that thing you already thought you should do."

**Nicole Forsgren** (01:08:32):
Yep.

**Lenny** (01:08:33):
Have you thought about making a public template of this spreadsheet? Even though it is simple, I bet it would be really helpful to people.

**Nicole Forsgren** (01:08:39):
I have, and this actually might be a good forcing function. Maybe [inaudible 01:08:43]-

**Lenny** (01:08:42):
Okay, awesome. So if you do it, I'll put in the show notes. It'll probably be near the bottom at the end of the episode, but that'd be awesome.

**Nicole Forsgren** (01:08:50):
Perfect.

**Lenny** (01:08:50):
Is there anything else that you want to share before we get to our very exciting lightning round?

**Nicole Forsgren** (01:08:55):
No, I think that's it.

**Lenny** (01:08:56):
Well, welcome to our very exciting lightning round. I've got six questions for you. Are you ready?

**Nicole Forsgren** (01:09:01):
Absolutely.

**Lenny** (01:09:03):
All right. First question, what are two or three books that you've recommended most to other people?

**Nicole Forsgren** (01:09:08):
We actually had the perfect segue because the book I've recommended absolutely the most is called Good Strategy Bad Strategy by Richard Rumelt. Another one is Designing Your Life by Bill Burnett and Dave Abbott... Dave Evans. And the last one is probably Ender's Game, Orson Scott Card. No comment right now on some of his political commentary, but I used to have extra copies in my office when I was a professor and I would just hand it out to my students. It's a fun, just easy nonsense read, but...

**Lenny** (01:09:41):
I absolutely love it. Such a good pick. Haven't read in a long time. And are they making [inaudible 01:09:48] show of that at all? That'd be something.

**Nicole Forsgren** (01:09:49):
They made a movie, and I was afraid I wasn't going to like it, so I just didn't read it. I didn't want it to ruin the book, but at least Harrison Ford was in it.

**Lenny** (01:09:58):
Okay. I'm not going to check it out. They're making a movie of Three-Body Problem. I don't know if you've read that, but that is... I'm really excited for that.

**Nicole Forsgren** (01:10:03):
It's on my list.

**Lenny** (01:10:05):
Oh, man. Best sci-fi ever. Next question actually very correlated. What is a favorite recent movie or TV show?

**Nicole Forsgren** (01:10:13):
I think going through some real just easy, fun watches lately. I'm rewatching Suits again, but Ted Lasso is a favorite, and I just tore through Never Have I Ever, which is fun because John McEnroe narrates it, which is hilarious.

**Lenny** (01:10:30):
John McEnroe, the tennis player?

**Nicole Forsgren** (01:10:31):
Yeah. It's a riot. Yeah, it's so funny.

**Lenny** (01:10:35):
I love it. Next question, what's a favorite interview question that you like to ask people when you're interviewing them?

**Nicole Forsgren** (01:10:42):
I love questions that I can kind of spin around hard decisions that people have had to make and how they made them. I love hearing their thought process. And I get a little nervous when people just [inaudible 01:10:54] and shoot from the hip constantly.

**Lenny** (01:10:56):
So what is it you look for there that gives you a sense that they're someone you may want to hire or work with?

**Nicole Forsgren** (01:11:01):
I just hearing if they have some sort of process, right. If they have some kind of decision-making process, if they have criteria, if they have... How do they do evaluation?

**Lenny** (01:11:11):
What is a favorite product you've recently discovered that you love?

**Nicole Forsgren** (01:11:14):
I have a big one and a little one. My big one is probably Sleep Eight. So I live in Arizona. It gets hot here sometimes.

**Lenny** (01:11:21):
Oh, Eight Sleep.

**Nicole Forsgren** (01:11:23):
Or yeah, Eight Sleep. Yeah, the other way around.

**Lenny** (01:11:24):
Yeah.

**Nicole Forsgren** (01:11:25):
Yeah, so one's fun because it-

**Lenny** (01:11:27):
Cool.

**Nicole Forsgren** (01:11:27):
... it makes the bed cold and also gives me some data, which is probably a little bit off, but in the approximation's fun. And then Korean face masks. They're just fun. Yeah. And you can get some pretty good ones for just a couple dollars, and that's always fun. Self-care.

**Lenny** (01:11:44):
Wow. First mention of that one, of Korean face masks.

**Nicole Forsgren** (01:11:47):
Right. Listen, everyone get on board.

**Lenny** (01:11:50):
I just did the TikTok. There's a filter now where you could see how you look when you age, and I'm not happy with how it turned out. And so I might look into this.

**Nicole Forsgren** (01:12:00):
I had some basal cell cancer on my forehead a few years ago, and so I am much more careful with my skin, and you can get... One of my favorites is COSRX. You can get 10 for like $15. So it's fun to just chill at the end of the day with a good face mask.

**Lenny** (01:12:15):
I was going to ask you for a specific pick, and so we got one.

**Nicole Forsgren** (01:12:18):
Yep.

**Lenny** (01:12:19):
Amazing. This next question I ask everyone, and it's especially appropriate to you, but I don't know if you'll have an answer. What's something relatively minor you've changed in your product development process that has had a big impact on your team's ability to execute? And I feel like you have a big perspective on this, so I'm curious what you have as an answer.

**Nicole Forsgren** (01:12:38):
I think I alluded to this earlier. I would say that it's helping everyone. So I've done this before, but I think it's helping everyone to ask who's our audience and how will we share this now. And it's sort of interesting because, right now, I'm wearing two hats. One is at MSR, Microsoft Research. We lead very ambitious research, right. So like H2, H3.

**Lenny** (01:13:03):
I mean, what is H3? The third half?

**Nicole Forsgren** (01:13:06):
Oh, Horizon 3. And so it's-

**Lenny** (01:13:09):
Oh, Horizon 3. Oh, okay.

**Nicole Forsgren** (01:13:10):
... supposed to be five to 10 years out, which-

**Lenny** (01:13:10):
Got it.

**Nicole Forsgren** (01:13:12):
... right now is like, who even knows, right? [inaudible 01:13:14]-

**Lenny** (01:13:14):
And we're going to be in computers, living [inaudible 01:13:16]-

**Nicole Forsgren** (01:13:16):
Yeah. AI has completely upended how we kind of think of Horizons. And so when we're thinking really ambitiously and very, very, very forward-looking, what's our check-in? How do we evaluate this? And then how can we easily communicate it to our core audience? And so here, who's our audience, and how do we bring the far near?

**Nicole Forsgren** (01:13:37):
And then for the other hat I'm wearing, I'm working with OCTO kind of across all of Microsoft to take a data-informed approach to really improve and uplevel our central developer infrastructure. And so, as we're thinking very, very tactically, what is our long-term vision, and how do we align with several of our broad stakeholders? And so there it's who's our audience, and how do we bring the near, far?

**Lenny** (01:14:04):
I love that. Final question. What is one tactical piece of advice that listeners can do this week to help improve their developer productivity or developer experience and move it in the right direction?

**Nicole Forsgren** (01:14:17):
If you walk away from this podcast right now, you could take a look at what's happening in your org today. Is it written down? Is it clear? Do you have any existing data and efforts? And if not, go find a handful of developers and ask them how they feel about their work tools and their work process and what the biggest barriers to their productivity are.

**Lenny** (01:14:41):
Also, pick up a copy of Accelerate on Amazon or your local retail establishment. Nicole, this was amazing. I think we're going to help a lot of companies move faster, have better and happier engineers, which is going to create infinite value in the world. Thank you so much for being here. Two final questions. Where can folks find you online if they want to reach out, and how can listeners be useful to you?

**Nicole Forsgren** (01:15:02):
I am on Twitter and Bluesky, Nicolefv. And my website is nicolefv.com, and my... all my contact information is there. And as we mentioned previously, I'm working on a new project and a new book digging into exactly these ideas, right. How can we measure better? How can we improve? And what does that measurement process look like?

**Nicole Forsgren** (01:15:24):
Both for kind of one time, really quick unofficial measurement pieces, and if we want to do kind of very formal longer term measurement pieces. So if anyone is interested in that or has any success stories they'd love to share, I would love to hear more about it. So please reach out and share. I'd love to hear more.

**Lenny** (01:15:44):
Awesome. Nicole, thank you again for being here.

**Nicole Forsgren** (01:15:47):
Okay, thank you Lenny.

**Lenny** (01:15:49):
Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [5/17] Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

**Chip Huyen** (00:00:00):
A question that get asked a lot and a lot is, "How do we keep up to date with the latest AI news?" Why do you need to keep up to date with the latest AI news? If you talk to the users who understand what they want or they don't want, look into the feedback, then you can actually improve the application way, way, way more.

**Lenny Rachitsky** (00:00:15):
A lot of companies are building AI products. A lot of companies are not having a good time building AI products.

**Chip Huyen** (00:00:19):
We are in an ideal crisis. Now, we have all this really cool tools to do everything from scratch and have new design. It can have you write code. You can have new website. So in theory, we should see a lot more, but at the same time, people are somehow stuck. They don't know what to build.

**Lenny Rachitsky** (00:00:33):
All this AI hype, the data is actually showing most companies try it, doesn't do a lot. They stop. What do you think is the gap here?

**Chip Huyen** (00:00:38):
It's really hard to measure productivity. So, I do ask people to ask their managers, "Would you rather give everyone on the team very expensive coding agent subscriptions or you get an extra head count?" Almost every one, the managers will say head count. But if you ask VP level or someone who manage a lot of teams, they would say, "Want AI assistant." Because as managers, you are still growing, so for you having one HR head count is big. Whereas for executives, maybe you have more business metrics that you care about. So you actually think about what actually drive productivity metrics for you.

**Lenny Rachitsky** (00:01:11):
Today, my guest is Chip Huyen. Unlike a lot of people who share insights into building great AI products and where things are heading, Chip has built multiple successful AI products, platforms, tools. Chip was a core developer on NVIDIA's NeMo platform, an AI researcher at Netflix. She taught machine learning at Stanford. She's also a two-time founder and the author of two of the most popular books in the world of AI, including her most recent book called AI Engineering, which has been the most read book on the O'Reilly platform since its launch.

**Lenny Rachitsky** (00:01:41):
She's also gotten to work with a lot of enterprises on their AI strategies, and so she gets to see what's actually happening on the ground inside a lot of different companies. In our conversation, Chip explains a lot of the basics like, what exactly does pre-training and post-training look like? What is RAG? What is reinforcement learning? What is RLHF? We also get into everything she's learned about how to build great AI products, including what people think it takes and what it actually takes. We talk about the most common pitfalls that companies run into, where she's seeing the most productivity gains and so much more.

**Chip Huyen** (00:04:34):
Hi, Lenny. I've been a big fan of the podcast for a while, so I'm really excited to be here. Thank you for having me.

**Lenny Rachitsky** (00:04:40):
I want to start with this table/chart that you shared on LinkedIn a while ago that went super viral, and I think it went super viral because it hit a nerve with a lot of people. Let me just read this and we'll show this on YouTube for people that are watching. So it's this very simple table you shared of what people think will improve AI apps and what actually improves AI apps. What people think will improve AI apps, staying up to date with the latest AI news, adopting the newest agentic framework, agonizing about what vector databases to use, constantly evaluating what model is smarter, fine-tuning a model. And then you have what actually improves AI apps, talking to users, building more reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Why do you think this hit such a nerve with people? If you had to boil it down, what do you think people are missing about building successful AI apps?

**Chip Huyen** (00:05:30):
[inaudible 00:05:30] question that get asked a lot and a lot is that, "How do we keep up to date with the latest AI news?" I'm like, "Why? Why do you need to keep up to date with the latest AI news?" I know it sound very counter-intuitive, but there's just so much news out there. A lot of people also ask me questions like, "How do I choose between two different technologies?" Maybe like recently, MCP versus agent-to-agent protocol? And it was like, "Which one is better or this or that?" I think it's a [inaudible 00:05:59] question you should ask them is like, "First, how much of the improvement could you get from optimal solutions versus non-optimal solutions?" Right? And sometimes they were like, "Actually, it's not much." Right?

**Chip Huyen** (00:06:10):
I was like, "Okay, if it's not much improvement, then why do you want to spend so much time debating something that doesn't make that much difference to your performance?" Another question they ask is like, "If you adopted a new technology, how hard it could be to switch that out to another?" And sometimes they will like, "Oh, I think it could be a lot of work switching it out." And I'm just like, "Hmm, let's say here's a new technology. It hasn't been tested by a lot of people, and if you would adopt it, you would be stuck with it forever. Do you actually want to adopt it?" Maybe you want to think twice about over commit to new technologies that hasn't been better tested.

**Lenny Rachitsky** (00:06:49):
I love your just broader advice is just simple like, to build successful AI apps, talk to users, build better data, write better prompts, optimize the user experience, versus just like, what is the latest and greatest? What's the best model to use right now? What's happening in AI? Let me follow this thread of this idea of fine-tuning and basically post-training. There's all these terms that people hear in AI, and I think this is going to be a really good opportunity for people to learn what we're actually talking about, since you actually do these things, you build these things, you work with companies doing these things. There's a few terms I want to sprinkle in through the conversation, but let's start with this one. What's the simplest way for someone to understand? What is the difference between pre-training and post-training and then just how fine-tuning fits into that, just what fine-tuning actually is?

**Chip Huyen** (00:07:34):
Chip disclaimer, I don't have full visibility on what this big secretive frontier labs are doing. But right from what I heard, so I think it's like one is, supervised fine-tuning when you have demonstration data, and you have a bunch of experts, "Okay, here's a prompt, and here is what the answer should be like." You just train it to emulate what the human expert could be like. That's also what a lot of people would like, so open-source models are doing as they do it by distillation. So instead of having human experts to write really great answers to prompts, they get very popular, famous good models to generate a response to it and getting this train smaller models to emulate.

Sometimes you see people just like... So, that's because I really appreciate open source community by the way, but going from being able to train models that can emulate a existing good model. It's very different from being [inaudible 00:08:38] trained good models, like an output for existing good model. So, it's a big step there. Yeah, we have my supervised fine-tuning, and another thing that's very big, I'm not sure you have guests talking about it already, but reinforcement learning is everywhere.

**Lenny Rachitsky** (00:08:52):
Let's pause on that because I definitely want to spend time on that, and that's such cool topic that's merging more and more in my conversations. But just to even summarize the things you just shared, which I think is really, really important stuff. So, the idea here is a model, essentially this algorithm piece of code that someone writes and say the frontier models are feeding it just like the entire internet of content, and basically, it's trying to test itself on predicting across all that data the next word, essentially. Token is the correct way of thinking about it, but a simpler way to think about it is the next word in text. As it gets it wrong, it adjusts these things called weights, essentially. Just like, is that a simple way to think about it, even that's just very surface level?

**Chip Huyen** (00:09:35):
So, I think of language modeling as a way of encoding statistical information about language, right? So, let's say that we both speak English, so we get a sense of what is more statistically likely. If I say my favorite color is, then you would say, "Okay, that should be another color." The word blue would be much more likely to appear than the word like [inaudible 00:09:59], right? Because statistically, blue is more likely to [inaudible 00:10:02] my favorite color is. So, it's a way of encoding statistical information.

**Chip Huyen** (00:10:07):
So when language modeling, when you train a large amount of data, you see a lot of languages, a lot of domains. So it can tell, okay, your basic size is standard. Then the user do the prompts and it could come with the next most likely token. So by the way, it's not a new idea actually. So it's the idea comes very, very old, from the 1951 papers like English entropy. I think it's by Claude Shannon, it's a great paper. And I think it reveals a story I really like is from... Did you read Sherlock Holmes by the way?

**Lenny Rachitsky** (00:10:39):
Yeah, I read a few Sherlock Holmes books. Yeah.

**Chip Huyen** (00:10:41):
Yeah. So this is story of when Sherlock Holmes says using this statistical information to help solve a case. So this is his story. There is somebody left a message with a lot of stick figures. So Sherlock Holmes was like, okay, he knows that in English, the most common letter is E. Then the most common stick figure must be E. And then he goes, he stopped like that, [inaudible 00:11:07]. So the code... So I think there's language. So in a way, it's simple language modeling, but instead of at a work level, he does this as character level and token is something in between, right? A token is not quite a word, but it's bigger than a character. So let's say we say token because it would help us reduce vocabulary because which character is smallest amount of vocabulary right now. So alphabet has 26 character, but words can have millions and millions, right? Whereas tokens, you can be able to get the sweet spot between the two.

**Chip Huyen** (00:11:44):
So let's say that we have the new word, how to say it, like podcasting, right? Let's say it's a new word, but it can divide a podcast and ing. So people understand, okay, podcast, we know the meaning. We know that ing is a verb, gerund, whatever it is. So we even know the word podcasting so that's why the token comes in. But yeah, the pre-tuning is basically encoding statistical informations of language to have you predict what is most likely. I think that most likely is the most simple way of doing it because it's more building a distribution of, okay, so the next token could be more 90% of the the time it could be a color, 10% of the time could be something else. So it basically distribution so language could pick, depending on your sampling strategy. Do you want it to always pick the most likely token or do you want it to pick something more creative? So I think my sampling strategy, I think is something extremely important. It can have you boost a performance in a huge way and very, very underrated.

**Lenny Rachitsky** (00:12:49):
Okay, awesome. So essentially, a model is just code with this whole set of weights, essentially the statistical model that has learned to predict what comes next after certain words and phrases?

**Chip Huyen** (00:13:03):
Yeah.

**Lenny Rachitsky** (00:13:03):
And then post-training and fine-tuning, specifically, is doing that same thing. So pre-training you get GPT5. Fine-tuning is someone taking GPT5 and doing the same sort of thing, adjusting these weights a little bit for specific use cases on data that they find is necessary to do their very specific use case. Is that a simple way to think about it?

**Chip Huyen** (00:13:24):
Yeah, I think weights is functions, right? So let's say you have... Maybe it has a functions of maybe Lenny's height is maybe 1X plus something or 2X [inaudible 00:13:38] and plus something is the weights, right? So you change it until you fit the correct data, which is my height and your height. So you can think it's a weight, as just a weight, say function. So you train, adjust the weights so they can fit the data, which is the training data.

**Lenny Rachitsky** (00:13:54):
Awesome. Okay. So we're talking about pre-training, post-training, fine-tuning. Is there anything else here that's important to share about just what this is exactly? What people need to understand about these parts of training?

**Chip Huyen** (00:14:06):
So the vast majority of time, we don't touch on pre-training model. As users, we don't use it at all.

**Lenny Rachitsky** (00:14:12):
Right. It's already done for us.

**Chip Huyen** (00:14:13):
Yeah. So I think my [inaudible 00:14:15] is a bit of fun process when my friend's training model is they try to play with their pre-training model and they're horrendous. They're saying things like [inaudible 00:14:23] "Oh, my gosh." Yeah, it's crazy. So it's very interesting to look at how much of post-training can change the model behavior and I think that's where a lot of time, is a lot of people are spending energy on nowadays, their frontier lab, is on post-training. Because pre-training, I think... So pre-training have been used to increase the general capacity of capabilities of a model. And it needs a lot of data and model size to increase the model capabilities. And at some point, we are actually have kind of maxed out on the internet data. And people text data max out. I think a lot of people are doing with other data like audios and videos, and everyone's trying to think of what is the new source of data, but where like post-trading, but middle course of this is more of everyone have very similar pre-training data, is that post-training is where they make a big difference nowadays.

**Lenny Rachitsky** (00:15:21):
This is a good segue to, you talked about supervised learning versus unsupervised learning. I love, we're getting into this, by the way. This is super interesting. So you talk about labeled data. Basically, supervised learning is AI learning on data that somebody has already labeled and told it, here's correct versus incorrect. For example, this is spam versus not spam. This is a good short story. This is not a good short story. We've had the CEOs of a lot of these companies that do this for labs, Mercor and Scale, Handshake, there's Micro, there's a few others. So is that essentially what these companies are doing for labs, giving them labeled data, high-quality data to train on?

**Chip Huyen** (00:15:57):
It is in a way, but I think it's more like a product of big equations. So there are a lot more different components than that. So that's why I was talking about reinforcement learning. I'm not sure if your CEO [inaudible 00:16:09] interview bring up that term. So the idea is that once you [inaudible 00:16:14]... So let's say you have a model, give the model a prompt and it produce an output. You want to buy, once you reinforce, encourage the model to produce an output that is better. So now it comes to how do we know that the answer is good or bad? So usually, people relies on signals. So one way to get a first one good or bad is human feedback. They happen to be have two responses. You can, okay, this one one's better than the other. And we do that is because as humans, we tend to, it's very hard to give a concrete score, but it's easier to do comparisons.

If you ask me, okay, give this song a score, I'm not a musician and don't know how hard it is. It's like yeah, I don't know what, out of 10 I going to remove six. And if you ask me again a month from now and I completely forgotten, okay, maybe now seven, only four, I don't know. But then if you ask me, okay, here are two songs and which one would you prefer to play for the birthday party? I was like, "Okay, I can prefer this song." So comparisons a lot easier. So [inaudible 00:17:18] have a human, you have human feedback and then you use this human feedback to treat a reward model to tell which and then the reward model help you like, okay, it's a model that produce this response.

It's [inaudible 00:17:30] can score, is this good or bad? And you try to bias toward producing better model, the better responses. Another ways you can, instead of using a human, so you can use AI because the response and say good or bad, right? Or in fact the thing is that people are very big on nowadays, verifiable rewards, which it's natural. So basically, they give it a math problem and then math solutions is a model app a solution. Okay, it's expected response should be 42 and if it doesn't provide 42, then it's wrong. Now it's not a good response. So yes, a lot of time, people using this human laborer, human laborers should produce, how to say, expert questions and I say expected answers and in the ways that [inaudible 00:18:16] systems that verifiable so that the models can be trained on. Yeah.

**Lenny Rachitsky** (00:18:19):
Okay, I'm really glad you went there. This is essentially RLHF reinforcement learning with human feedback, which is exactly what I wanted to also talk about, right?

**Chip Huyen** (00:18:29):
Yeah. So I think it's general, it's a way of learning. It's training is [inaudible 00:18:33] learning and whether it learn from human feedback or AI feedback or verifiable rewards, I think I say it's just different way of collating signals.

**Lenny Rachitsky** (00:18:44):
Awesome. Yeah. We had the CEO of Anthropic on the podcast and he talked about their version of RLHF, which is AI driven reinforcement learning. I love the way you phrased it where basically you want to help the model, you want to reinforce correct behavior and correct answers, and this is the method to do it, whether it's say an engineer seeing an output from a model being like, "No, here's how I would code it differently." And it's training a different model that the original model works with to tell it, am I correct or not correct? Is that right, roughly?

**Chip Huyen** (00:19:15):
Yeah.

**Lenny Rachitsky** (00:19:16):
Okay.

**Chip Huyen** (00:19:16):
I think that's a way of looking into it. I think that's a space is so exciting nowadays because there are so many domain expert task that the model developers want models to do well on, right? Let's say you're accountant. Maybe you want to use a model to have accounting task and need a lot of accounting data examples from accountant. So you need to hire a lot of them, should I do it or everyone [inaudible 00:19:41] physics problems, everyone should do, I don't know, legal questions and stuff or engineering questions or somebody was telling me they want to do, using coding to source scientific problems and not just coding to build product, which is another different whole realm of things. And I also using very specific toolings. I'm not sure what apps you use, but maybe like a [inaudible 00:20:04] app or QuickBooks or Google Excel. They have very specific tools, specific expertise. So you want the model [inaudible 00:20:13] learn.

So they need a lot of humans expert in this area should create data to train them and it's a massive thing people because everyone wants a lot of data and wants [inaudible 00:20:25] unlimited budget. But whether, I think this is also a little bit of low-key, interesting economics. I'm not sure you've talked to the guests about, I thought it's very interesting [inaudible 00:20:35] think about because it's very lopsided, right? Because they're only a very small numbers of frontier labs and they want a lot of data and there's a massive amount of startups or company providing related data. So you can see these companies like this startup doing data labeling. They have maybe some massive AR, but if you ask them, "Okay, so how many customers you have?" And they could be very small numbers, I'm not sure. I'm not sure you... I saw you smiling.

**Lenny Rachitsky** (00:21:03):
Yeah, yeah, yeah, we chatted about that.

**Chip Huyen** (00:21:05):
Yeah, so I'm a little bit like [inaudible 00:21:08] uneasy. I have a company's growing crazy, but it's heavily dependent on two or three companies. And at the same time, if I was this company, frontier labs, what could be the right economical things for me to do? Now I want a lot of startups. I want to have a lot of providers so I can pick and choose, and as this providers can also to compete each other to lower the price and it's so dependent on [inaudible 00:21:34] regardless. So I feel like, yeah, so this whole economics is very interesting to me and I'm curious to see and how it plays out.

**Lenny Rachitsky** (00:21:42):
What I'm hearing is you're bearish on the future of these data labeling companies because as you said, they don't have a lot of leverage over pricing because they have so few customers and there's so many people getting into the space. So basically, even though there's some of the fastest growing companies in the world, you're feeling like there's a challenge up ahead.

**Chip Huyen** (00:22:00):
I'm not sure if I'm bearish on it. I think I'm curious because I think things has a way of work out in ways that I don't expect. So I think that maybe these companies, they have a lot of data, maybe they wouldn't be able to use that to have some insight that helps them stay ahead of the curve. So I don't know.

**Lenny Rachitsky** (00:22:22):
A very fair answer. Okay, while we're on this topic, I want to chat about evals, which is a very recurring topic in this podcast. This is the other piece of data content these companies share that AI labs really need. Can you just talk about what an eval is, the simplest way to understand it and then how this helps models get smarter?

**Chip Huyen** (00:22:41):
So I think people approach eval, I think they're two very different problems. One is a app builder and can I say have an app that do maybe a chatbot? Very simple answer first thing that came to my mind and I want you to know if chatbot is good or bad. So it needs to come away with evaluate the chatbot. Another thing is, I think of this as a task-specific eval design. So let's say I'm a model developer and I want to make my model better at code writing. And it was like, "Okay, but how do I even measure code writing?"

So I need someone to understand code writing and think about what makes a story good and then design the whole dataset and then criteria to evaluate code writing. So yeah, I think there's that. I think it's more like eval design that is very interesting [inaudible 00:23:39] work criteria, [inaudible 00:23:42] work guidelines, how to do it and then also train people how to do it effectively. So I guess, [inaudible 00:23:49], I think eval is really, really fun because it's extremely creative. I was looking at different eval people built and it was like, "Wow." It's not dry at all. It's just super, super, super fun.

**Lenny Rachitsky** (00:24:01):
We had a whole podcast on evals with Hamel and Shreya. That's exactly what they talked about is just, it's actually really fun to create evals for companies, especially. So let's still dig into that one a little bit more. There's this kind of debate online that, I don't know how big of a deal this debate is, but it feels like people spend a lot of time thinking about this, this idea of, do we need evals for AI products? Some of the best companies say they don't really do evals, they just go on vibes. They're just like, "Is this working well? Can I feel it or not?" What's your take on just the importance of building evals and the skill of evals for AI apps, not the model companies?

**Chip Huyen** (00:24:39):
You don't have to be absolutely perfect, I think, to win. You just need to be good enough and being consistent about it. Okay, this is not a philosophy I follow, but I have worked with enough companies to see that play out. So when I say, why a company don't eval? Let's say you are an executive and you want to have a new use case. So here's a use case you started out, built and it's like it works well. The customers are somewhat happy. You don't have the exact metric for it.

**Chip Huyen** (00:25:05):
So the traffic keeps increasing, people seem happy, people keep buying stuff and now here's our engineer coming like, "Okay, we need eval for it." And it was like, "Okay, how much effort do we need to go into eval?" And they were like, "Okay, maybe two engineers, this much, this much." And it could maybe would improve that and it was like, "Okay, so how much expected gain can I get from it?" And the engineer would be like, "Oh, maybe you can improve it from 80% to 82%, 85%."

And it was like, "Okay, but [inaudible 00:25:35] that two engineers and we going to launch a new feature, then it could give me so much more improvement." So I think it's one of them is eval. Sometimes people think of eval as like okay, this is good enough, just don't touch it. If you do spend a lot of energy on eval, it would only incremental improvement where it spends the energy on another use case and maybe [inaudible 00:25:55] good enough that you can vibe check it.

So I do think maybe that's a debate is about. I do think that a lot of time people just get things to the place where it's like, okay, good enough, people run. But in the end, but of course there's a lot of risk associated with it because if you don't have a clear metric, you have a good visibility to [inaudible 00:26:17] applications or models performing it might do something very dumb or it can cause you, I know something crazy can happen. So yeah, so I do think eval is very, very important if you have, if you operate a scale and where failures can have catastrophic consequences.

**Chip Huyen** (00:26:38):
Then you do need to be very tyrannical about what you put in front of the users, understand different failure modes, what could go wrong and also maybe in a space when that it's a feature, the product is a competitive advantage. You want to be the best at it. So you want to have a very strong understanding of where you are and where you are with the competitors. But it's just something that's more a low-key, okay, this is like something is like, okay, that's not the core but it helps with our users.

**Chip Huyen** (00:27:04):
Then maybe you don't need to be so obsessed or theoretical about it. It's like, okay, that's good enough for now and if it fails, then it fails. Okay, I know it's so terrifying. But yeah, I think it's all about the question of return investment. I'm a big fan of eval, I love reading eval. And I says, I understand why some people would choose to not focus on eval right away and choose bringing on new functionalities instead.

**Lenny Rachitsky** (00:27:32):
Awesome. That is a really pragmatic answer. What I'm hearing is evals are great, very important, especially if you're operating at scale, but pick your battles. You don't need to write evals for every little feature. Something that Hamel and Shreya shared is that people need just, I don't know, five or seven evals for the most important elements of their product. Is that what you see or do you see a lot more in production that people build and need?

**Chip Huyen** (00:27:54):
I don't think of just a fixed number on the evals. What was the goal of eval? The goal of eval is to guide the product development. So you see eval, because I think I'm a big fan of eval, is that it helps you uncover opportunities where the progress are doing well. So sometimes, we've seen a very obvious [inaudible 00:28:15] where you look at the eval and we realize it's like, okay, it performed really poorly on this specific segment of users and then we look into it's like, okay, what's wrong with it? And it turns out, it's like we just don't have a good messaging to it. So people should just focus on the things that we're doing poorly, can improve significantly. Yeah, so I kind of like the number of eval is really depends. We have seen product with hundreds of different metrics.

**Lenny Rachitsky** (00:28:40):
Oh, wow.

**Chip Huyen** (00:28:41):
People going crazy, this is because that product is general, have different names, have one eval for, I don't know, verbosity, have one eval for user sensitive data and another is for length but has a number of, okay, let's just give a good example, concrete example, like deep research. So you have the application, you have views and model to do deep research for you. Okay, have a prompt. Let me say, okay, do me a comprehensive research on only Lenny's Podcast and help me propose, show me report on what kind of topics he's interested in, what kind of videos could get the most views or what topics that he's missing on that he should be covering, right? Have that prompt. Then how do you evaluate the result? I don't think there's one metrics that would help. Maybe it's like maybe you have a hundred, I think somebody has a benchmark and is get a hundred expert, write a bunch of prompts and they go through, on the answers on AI and do it. And it's extremely costly and slow.

But [inaudible 00:29:47] might have something else. First of all, one way I was thinking about it, I was talking to a friend about it and one way it's like, how would you produce the result of the summary? At first you need to, what you do, gather informations and to gather informations you need to do a lot of search queries. You gather, grab the search results and then some of the search results you aggregate and then maybe say, okay, I'm still missing on this. You have to go another route and on another route, [inaudible 00:30:17] have the summary. So every step of the way, you need evaluations. You don't [inaudible 00:30:21] end-to-end. Maybe it was a search query in my first thing about, okay, now I write five search queries. I might look into how good are the search queries? Do they as they similar to each other because you need five search queries are very similar? Okay, Lenny Podcast, Lenny Podcast last month, Lenny Podcast two months ago.

It's not very exciting. But if the query is a podcast, the keywords are more diverse and then look at the results of the search query and then say you enter the search query. Lenny Podcast data labeling and they come up with 10 pages, 10 results. And then you come up with like, oh, Lenny Podcast on, I don't know, frontier labs, and you have 10 results. [inaudible 00:31:06] different webpages. Okay, how much of them overlapping... Are we doing both the breadth, getting a lot of page, but also, do we have depth and also do you have relevance because if we come up with a search query, it's completely irrelevant to the original prompt. So I feel like every aspect of it, it would need a way of evaluating. So I don't think it's how many eval should I get, but how many eval do I need to get a good coverage, a high confidence in my application's performance and also to help me understand where it is not performing well so that I can fix it.

**Lenny Rachitsky** (00:31:43):
Awesome. And I'm hearing also just especially for the very core use case, the most common path people take in your product is where you want to focus.

**Chip Huyen** (00:31:51):
Yeah, yeah.

**Lenny Rachitsky** (00:31:54):
Okay. There's one more term I want to cover and I want to go a somewhat different direction. RAG? People see this term a lot, R-A-G. What does it mean?

**Chip Huyen** (00:32:04):
So RAG stands for Retrieval-Augmented Generations [inaudible 00:32:08] not a specific true generative AI. So the idea is just for a lot of questions, we need context to answer. So I think it came pretty, I think it's from the paper 2017. So someone was like, so they realized it's for a bunch of benchmark. When the question answering benchmarks, they realized it's like, okay, if we give the model informations about the questions, the next answer can be much, much better. So what they do with that is try to retrieve information from Wikipedia. So for question [inaudible 00:32:39], just retrieve that and then put it into the context and answer. It does much better. So I feel like it sounds like a no-brainer, right? I mean, obviously. So I think that's what RAG is, as a simplest sense, it's just providing the model with a relevant context so that it can answer the questions. And it's where things get really more interesting because traditionally, when it started out, RAG is mostly text.

So we talk about a lot of way of how to prepare data so that the model can retrieve effectively. Let's say that not everything is a Wikipedia page. A Wikipedia page is pretty contained and you know, okay, everything about it is about a topic. But a lot of time, you have documents of like [inaudible 00:33:19] and they have a weird way of structures of documents. Let's say that you had documents about Lenny Podcast and in the future, in the beginning a document it's like, from now on, podcast wouldn't refer to Lenny's Podcast. So let's say somebody in the future is like, "Okay, tell me about Lenny. Lenny's work." And because as a [inaudible 00:33:40] document does not have the term Lenny, you just don't know, you might not retrieve it. And if the document is long enough that it's chunked into a different part, so the second part doesn't have the word Lenny, so you cannot reach it. So you have to find a way to process data. So that makes sure it's like... It can retrieve, the information is just relevant to the query even though it might not immediately obvious that it's related.

So people come up with only thing of, I think, contextual visual, like giving X chunk of the data, the relevant, maybe in a summary metadata so that it knows or some people use as hypothetical questions. It's very interesting for even the chunk of documents, I must generate a bunch of questions that the chunks can help answer so that when I have a query, it's like okay, does it match any of the hypothetical questions? It can fetch it. So it's very interesting approach. Okay, so maybe before I go to the next thing, I just want to say this data preparations for RAG is extremely important. And I would say this in a lot of the companies that I have seen, that's the biggest performance, in their RAG solutions coming from better data preparations, not agonizing over what [inaudible 00:34:51] databases to use because [inaudible 00:34:53] database, of course is very important to care about things like latency or if you have very specific access patterns like read-heavy or write-heavy, of course, it's like it matters. But in term of pure quality answers, I think the data preparation is just [inaudible 00:35:07].

**Lenny Rachitsky** (00:35:06):
When you say data preparation, what's an example to make that real and concrete for us to understand?

**Chip Huyen** (00:35:16):
So one way is mentioned as in you have chunks of data. So we have think about how big of each chunk should be. Because if it's sort of think about it's a context you want to maximize, maybe you can, it's very simple example. You want to retrieve a thousand words. So if a data chunk is long, then it's more likely to contain more relevant metadata so it can retrieve more. But if it's too long then you have a thousand word. And so chunk is like a thousand words, you can reach one chunk. So it's not very useful. But if it's too short, then you can retrieve more relevant information also. It can retrieve a wider range of documents and chunks, but at the same time each chunk is too small to contain relevant information.

So we have very nice chunk design, how big each chunk should be. You add contextual informations like summary, metadata, hypothetical questions. Somebody was telling me just a very big performance they got is that from rewriting their data in the question-answering format. Instead of having... So they have a podcast instead of just chunking the podcast, you just reframe, rewrite it into here's a question, here's answers and produce a lot of them. It can use AI for that as well. So that's one example of data processing. A lot of example I see is for people helping, using AI to help specific [inaudible 00:36:40] use and documentations. And we write documentation. Usually a lot of documentation today is written for human reading and AI reading is different because it's different because humans, we have common sense and we kind of know what it is. So one things are, even for human experts, they have the context that AI doesn't quite have.

**Chip Huyen** (00:36:59):
So somebody told me that what's a big change they have is let's say, that you have a function. The documentation for this, maybe the library. As a library said okay, the output of this one is maybe talking for, I don't know, some crazy term, maybe some temperature or something on the graph. It should be like one zero or minus one. And as a human expert maybe understand the scale, what one in the scale mean, but for AI, just really doesn't understand what that means. So actually, have another annotation layer for AI. It's like, okay, good temperatures equal one means like that. It's not like it's a actual temperature. It's associated with the scale over there. So just saving all this data processing to make it easier for AI to retrieve the relevant information to answer the questions.

**Lenny Rachitsky** (00:37:45):
**Chip Huyen** (00:39:32):
For GenAI in company, I think there are two types of GenAI toolings that have been, I've seen ones is to internal productivity, like have coding tools, Slack chatbot, internal knowledge. A lot of big enterprises have some a wrapper around models, so with access to maybe some different type of a RAG solution. I think we talk about data or kind of like text-based RAG. We haven't talked about agentic RAG or I haven't talked about multi-modal RAG yet. But this, yes, it's a whole very exciting area around that. So basically, it should allow the employee to access internal document. Somebody ask, okay, I'm having a baby. What could be the maternal or paternal policy or am I having these operations with the health benefit cover that or I want you to interview, I want to refer my friend. What will be the process for that? So a lot of this having chatbot, internal chatbot to help with internal operations.

**Chip Huyen** (00:40:35):
And another things, another category is more customer facing or partner facing. So product customers support chatbot is a big one. If you're a hotel chain, you might have a booking chatbot, which is somehow massive. A lot of booking chatbot because I guess it's... I do have this theory of a lot of applications companies pursue because they can't measure the concrete outcome. And I feel like booking or a sales chatbot, it's very clear. There was a conversion rate right now with that chatbot with human operators and what could be conversion rate with a chatbot and certain, somehow I think it's very clear outcomes and companies are easier to buy into these solutions. So a lot of companies have that customer facing chatbot.

So that is another category of tool and I think that for customers or external facing tools, because people are driven to choose applications with clear outcomes. So the questions of adopting them is really based on whether they see the outcome or not. Of course, it's not perfect because sometimes the outcome can be bad not because the idea or the application's idea [inaudible 00:41:52] is bad. It's just because the process of building it is not that great. Yeah. So it's tricky. For the internal adoptions of toolings or internal productivities, that's where it gets tricky. I would say a lot of companies [inaudible 00:42:08] think of AI strategy. I think of AI strategies usually have two key aspects. It's like use cases and the second is talent. You might have great data for great use cases, but you don't have talents and you cannot do it.

So a lot of time at the beginning with GenAI and sometimes I'm really admire a lot of companies for that, it's just like [inaudible 00:42:28] was like, okay, we need our employees to be very GenAI aware, very AI literate. So what I do is I start maybe adopting a bunch of tools for the team to use. They have a lot of up-skilling workshops, they encourage learning and then it's a really, really good thing. And it's also willing to spend a lot of money into adopting, giving people chargeability, subscriptions, purchase subscriptions, [inaudible 00:42:56] subscriptions to get the employees to be more AI literate. And that's the thing is a lot of... There's a [inaudible 00:43:05] may say, okay, we spend a ton of money on this tooling, but then we don't see because you can see the usage, but people don't seem to use them as much and what is the issue. So yeah, so I think that is tricky. Yeah.

**Lenny Rachitsky** (00:43:20):
What do you think is the issue? Is it just they don't know how to use them? What do you think is the gap here? Do you think we'll get to a place of just like, wow, work is completely different because of AI for a lot of companies?

**Chip Huyen** (00:43:32):
The main thing is it's really hard to measure productivity again. So I talk to a lot of people on their website. First of all, [inaudible 00:43:40] is coding. A lot of companies not using coding agents or coding [inaudible 00:43:45] coding. And I was asking, I was like, "Do you think that it helps with your productivity?" And a lot of times, the questions are very [inaudible 00:43:56] okay, I feel like it's [inaudible 00:43:59] better. And I said, okay, because we have more PRs, we see more code and then immediate [inaudible 00:44:04]. Okay, but of course, code, number of live code is not a good metric for that. So it's really, really tricky and it's something funny. So I do ask people to ask their managers because I work with usually VP level, so they have multiple teams under them. So I asked them, okay, do you ask some managers, okay, would you rather have access...

Would you rather give everyone on the team very expensive coding agent subscriptions or you get an extra headcount? Let's say maybe and almost everyone could say the managers could say headcount. But if you ask VP level or someone who manage a lot of teams, they would say just like [inaudible 00:44:48] good one, AI, a system as tools. And the reason is that we could say okay, because as manager is right, because you are still growing. You're not as a level when you manage hundreds of thousands of people. So for you, having one HR headcount is big. So you want that not for productivity reasons, but because you just want to have more people working for you. Whereas for executives, you care more about, maybe you have more business metrics that you care about. So you actually think about what actually drive productivity metrics for you. So it is tricky and I think that the question of productivity. I'm not sure it's fundamentally is the [inaudible 00:45:32] more productive, but it's just like we don't have a good way of measuring productivity improvement.

Another thing is also very [inaudible 00:45:40]. And I think it's like people do tell me that they notice different buckets of employees, different reactions to AI assist tools. First of all, I keep going back to coding because coding is big and it's easier to reason somehow. So it says I have different reports. One team would tell me that... One of people tell me, okay, amongst on his engineers, he thinks senior engineers would get the most output, would be more productive because it's like, okay, so that person's very interesting. So he actually divided his team to three buckets, but he didn't tell them, obviously. He was like, okay, here's more currently best performing, average performing and lowest performing. And then there's a randomized trial. So they give half of each group access to Cursor. And then [inaudible 00:46:31] noticed over time it was like, okay, something funny. The group that get the biggest performance boost, in his opinion, he was very close to his team.

The biggest performance boost [inaudible 00:46:41] the senior engineer, the highest performing. So the highest performing engineer get the biggest boost out of it. And then the second group is the average performing. So his opinion is like, okay, the highest performing engineers is also normal practice. They also know how to solve problems. So they have some solved problem better. Whereas the people who have the lowest performing, they only don't care much about work. So it's easier to just go on autopilot, get it to generate that code and just do it or just don't know how to do it. Another company, however, they tell me just actually, senior engineers are the one most resistant to using AI as this tooling because they said it's like, okay, but AI, because they are more opinionated and they have very high standard. It was like, okay, but AI code, [inaudible 00:47:30] code just sucks. So just very, very resistant in using it. So I don't know, I haven't quite been able to reconcile very different reports on that yet.

**Lenny Rachitsky** (00:47:39):
This is so interesting. So just to make sure I'm hearing what the story, so there's a company you work with, that did a three bucket test with their engineering team where they created three sorts of groups, the highest performing engineers, mid-performing engineers, lowest performing engineers, and gave some of them, so they gave some of them access to say, Cursor. Was it Cursor or what did they give them access to? It was Cursor, right?

**Chip Huyen** (00:48:03):
I think it was Cursor.

**Lenny Rachitsky** (00:48:04):
Okay, cool. And so within-

**Chip Huyen** (00:48:05):
I didn't work with them. This is more like a friend company.

**Lenny Rachitsky** (00:48:08):
Okay. It's a friend's company.

**Chip Huyen** (00:48:09):
Yeah.

**Lenny Rachitsky** (00:48:09):
So did they give half of the higher performing engineers Cursor and half not or how did they do the split there?

**Chip Huyen** (00:48:14):
Yeah, so they give half of the entire company but half of each bucket. Yeah.

**Lenny Rachitsky** (00:48:18):
Whoa.

**Chip Huyen** (00:48:19):
And then they observe the difference in productivity.

**Lenny Rachitsky** (00:48:21):
I see. So how do they even do that? They're just like, "Okay, you get cursor, you don't get cursor." How did they do that? That's so interesting.

**Chip Huyen** (00:48:31):
Yeah, I didn't get into the mechanics of it, but I was like, "I respect you for doing a randomized trial on that."

**Lenny Rachitsky** (00:48:33):
That is so cool.

**Chip Huyen** (00:48:33):
Yeah. Yeah.

**Lenny Rachitsky** (00:48:34):
Okay. Wow. How large was this engineering team? Was it like hundreds of people?

**Chip Huyen** (00:48:38):
It's not that large. It's about maybe 30 to maybe 40. Yeah.

**Lenny Rachitsky** (00:48:43):
30 to 40. Okay.

**Chip Huyen** (00:48:44):
Yeah.

**Lenny Rachitsky** (00:48:44):
Wow. Okay. So they found that the highest performing engineers had the most benefit from using AI tools and then behind them was the middle tier engineers and the worst performers or the lowest performers. Okay.

**Chip Huyen** (00:48:59):
But it's also not the same everywhere.

**Lenny Rachitsky** (00:48:59):
Right. Right. Right, right.

**Chip Huyen** (00:48:59):
Some companies are different.

**Lenny Rachitsky** (00:49:03):
Right. This other example you shared of just senior engineers in this one example are most resistant to changing the way they work, which I get. I do feel like the most valuable people right now other than ML researchers and AI researchers like yourself, are senior engineers because it feels like junior engineers are just, so much of this is now done by AI, but an engineer that knows what they're doing that understands how things work at a large scale with AI tools, just basically infinite junior engineers doing their bidding, feels like an extremely valuable and powerful asset.

**Chip Huyen** (00:49:39):
Yeah, I definitely really appreciate, as you see companies, we appreciate engineers who have a good understanding of the whole systems and be able to have good problem solving skill are thinking holistically instead of locally. Or when our company have seen the way they work, as they told me is we're completely different now. So they actually restructured engineering org so that they get more senior engineers should be more in the peer review because they get writing guidelines on what is a good engineering practices, what is the process would be like.

**Chip Huyen** (00:50:12):
Or maybe like okay, so they write a lot of processes on how to work well. And then they have more junior engineers just produce code and submit PR, but senior engineer more in the reviewing case. So I think it might be prepared for the future. So another company actually told me something very similar. So preparing for the future once they only need a very small group of very, very strong engineers to create processes and reviewing code to get into production but get AI or junior engineers to produce code. But then the question becomes just like, how does one become a very strong senior engineer.

**Lenny Rachitsky** (00:50:54):
Right. That's right. That's right. That's the problem. Yeah.

**Chip Huyen** (00:50:57):
Yeah. So I don't know what's the process I was thinking about, yeah.

**Lenny Rachitsky** (00:51:01):
No one's thinking about it. It's a problem. We won't have any more in 10, 20 years. There'll be no more engineers because no one's hiring junior engineers. Although I could make the case. Junior engineers, people just getting into computer science right now, are just AI native. And in theory, you could argue they will become really good really fast if they're curious, aren't just delegating, learning and thinking to AI, but learning how to actually, using it to learn how to code well and architect correctly. You could argue they'll be the most successful engineers in the future.

**Chip Huyen** (00:51:33):
I do think that what I mentioned said relating to architect. I think I grouped that in my system thinking. I do think it's very important skill because I think AI can help automate a lot of disjointed skills, but knowing how to utilize the skills together to solve problems is hard. So that's a webinar between Mehran Sahami who is one my favorite professors. He was a chair of the curriculum at the CS Department at Stanford. So he spent a lot of time thinking about CS educations, what should students learn nowadays in the area of AI coding. And then the other person is Andrew Ng, which is of course, is a legend in the AI space. And Mehran Sahami, Professor Sahami, said something very interesting. He said a lot of people think that CS is about coding, but it's not. Coding is just a means to an end.

**Chip Huyen** (00:52:27):
CS is about system thinking, using coding to solve actual problem and problem solving will never go away because what AI can automate more stuff. The problem is just get bigger. But as a process of understanding what caused the issue and how to design step-by-step solution to it, will always be there. So I think an example of, I actually have a lot of issues with AI for in the way of it's debugging. So I'm not sure you use a lot of AI for coding, but something I have noticed and also seen from my friends, it's like it is pretty good when you have very clear, well-defined tasks. Maybe write documentations, fix specific features or build an app from scratch. Doesn't have to interact with a large access in code base, but you added something a little bit more complicated, maybe required interaction with other components and stuff. It's usually not that good.

And for example, I was using AI to deploy an applications and it was testing out a new hosting service I was not familiar with. It was like, okay. Usually they inform me, so working AI does give me is confidence to try a new tool. Before what AI is like trying new tools has written, not documentations for the beginning, but I was like, okay, just try it out and learn. So I was testing out this new hosting service and it kept getting a bug, so was very, very annoying. And it was like, okay, I asked [inaudible 00:53:51], fix it. And it kept changing the way, maybe change the environment variable, fix the code, maybe not change from the function to this function, maybe change the language, maybe it doesn't process JavaScript, I don't know, whatever. And it didn't work. And it was like, okay, that's it.

I'm just going to read documentation myself and see what's wrong. And it turns out, it's like I'm on another tier, the [inaudible 00:54:16] I want did not, is not available in this tier, right? So I feel like, okay, so the issue with [inaudible 00:54:22] was just trying to focus on fixing things from a different component versus the issue is from a different component. So I think of, okay, be understanding how different components work together and where the source of issue might come from. You need to give a holistic view of it. And it's made think is like, okay, how do we teach AI system thinking that I have all the human experts having very much [inaudible 00:54:46] scaffold just like, okay, for this kind of problem, look into this, look into that, look into that, and then stuff. So [inaudible 00:54:53] that could be one way, but that's also made me think is, how do we teach humans, system thinking? Yeah. So yeah, I think it's very interesting skill. I do think it's very important.

**Lenny Rachitsky** (00:55:04):
That's exactly the same insight Bret Taylor shared on the podcast. He's the co-founder of Sierra. He created Google Maps. He was CEO of Salesforce, Quip, a few other things. And I asked him just like, should people learn to code? And his point is exactly what you said, which is taking computer science classes is not about learning Java and Python. It's learning how systems work and how code operates and how software works broadly, not just, here's a function to do a thing.

**Lenny Rachitsky** (00:55:32):
One thing that I wanted to help people understand, you wrote this book called AI Engineering, which is essentially helping people understand this new genre of engineer and you have this really simple way of thinking about the difference between an ML engineer and an AI engineer, which has a really good corollary to product managers now, of just an AI product manager versus a non-AI product manager. The way you describe it and fill in what I'm missing is just ML engineers built models themselves. AI engineers use existing models to build products. Anything you want to add there?

**Chip Huyen** (00:56:05):
One thing I really dislike about writing books is that it has to define this and I think it's like no definitions would be perfect because they always be edge cases. But yeah, in general, I think it's just like GenAI as a service, more as a service, when somebody build the models for you and the base model performance is a pretty [inaudible 00:56:26]. So it's like it's enabled people to just like, okay, now I want to integrate AI into my product. I don't need to learn [inaudible 00:56:34] even though knowing that could really help. But yeah, it makes an entry barrier really low for people who want to use AI to build product and at the same time, AI capabilities are so strong. It's also increased the possibilities, the type applications that AI can be used for. So I think yes, both entry barriers' is super low and a demand for AI applications a lot bigger. So it feels, it's very, very exciting. It's opens up a whole new ball of possibilities.

**Lenny Rachitsky** (00:57:04):
Yeah. It's like now you don't have the time, now you don't have to spend time building this AI brain. Now you can just use it to do stuff, such an unlock. Okay. Maybe just a final question. You get to see a lot of what's working, what's not working, where things are heading. I'm curious just if you had to think about in the next two or three years, just where things are heading, how do you think building products will be different? How do you think companies working will be different if you had to think of maybe the biggest change we expect to see in the next few years, in terms of how companies work?

**Chip Huyen** (00:57:40):
I think in a lot of organizations they don't move that fast, but at the same time, they move faster than I expected because again, I think it's like bias and don't work with dinosaur companies who don't care. I think a lot of executives who come to me are very forward-looking. So maybe for me, I'm very biased towards organizations is move fast. So yeah, I think one big change I see just in organizational structure. I think this a lot of value plays in... So before we have a lot of disjointed teams. We have very clear engineering team, product team, but then there's a question of who should write eval? Who should own the metrics? And it turns out, eval, it's not a separate problem. It's a system problem because you need to look into different components, how they interact with each other. You need user behaviors because you need to know what users care about so that you can write eval reflect what users care about.

**Chip Huyen** (00:58:44):
So all of that you can sort it from you look into different component architectures, place guardrails and stuff. So it's just engineering, but understanding users is what product. So because of a lot of things and eval is extremely important. So the kind of bring product team and engineering team, even marketing team like user acquisition, very close to each other. So yes, since in a ways if people are structuring, so that's more communications between previously very distinct functions. Another thing is I also see as teams, of course, I think about what can be automated in the next few years and what work cannot be automated. And I seen that people already shedding, actually it's a little bit scary to think about it, but I also think it's the teams, they would've told me, it's just like okay, this is good and you and me, but we have got rid of these functions for a lot of things like previously outsourced, for example.

**Chip Huyen** (00:59:37):
Traditionally, it's a business outsourcing that's not core to them and can be in a more systematized. So with that, you can actually use AI to automate a lot of that. And so as a separation people thinking more of what is the value of junior engineers or senior engineers, how should we restructure engineering org for that? Yeah, so I do definitely think that is one thing to successful organization. People are just moving pieces around and thinking about use cases, whether you need to spin out new use cases and who would lead a new effort. That is one big change. Another thing in terms of AI, I think there's, I'm not sure how true this is. I guess, I'm also on the camp of thinking that it has merit, is a camp of okay, base models we have probably not quite maxed out, but we're unlikely to see really, really strong, crazily strong model.

So you remember when we have GPT, right? And then GPT2, which is a big step up, an [inaudible 01:00:49] better than GPT and then GPT3, which much, much bigger than GPT4, much, much bigger. And then of course, GPT5, but it's GPT5, that scale of much bigger step jump compared to the previous, I think it's debatable. So I think that we had disappointment, the base model performance improvement is not going to be mind-blowing. It was in the last three years. So I think there's a lot of improvements when I see in the post-training phase, in the application building phase. And yes, also I think that's where I feel I would see a lot of improvement there. I also very interest in multimodality. So we've seen a lot of text base, but I think there's a lot of audio, videos use cases that is very, very exciting.

And I think audios is not quite as solved. Well, I think because I do work with a couple of voice startups and when it comes to, think about voice, it's an entirely different beast. So let's say have chatbot. We go from a text chatbot to voice chatbot. It's like the consoles are completely different because now with voice chatbot, we need to think about latency because I think multiple steps, first have voice to text, text to text, text question into text answer and then text to voice answer. So you have multiple hops and latency become very important. And there's a question, what does it make you sound natural? So for example, people think of in AI and humans, when humans talk to each other, if I say, you try to interrupt me and say, Chip [inaudible 01:02:36]. I would pause and I try to hear you out.

**Chip Huyen** (01:02:38):
But sometime even if I just like say some word, like acknowledge when I, mm-hmm, mm-hmm, that I shouldn't stop. It's just continue. So the question of forced interruption and whether it's, should I stop or not, it's a big in what perceived as natural conversations. And that's also regulations because a lot of time, people want to build AI chatbot, voice chatbots that sound like humans, try to trick users into thinking that they're talking to humans, but also maybe potential regulation saying okay, you have to disclose to users when you talk, if the bot is talking to is human or AI. So I think this a whole space, I think it's not quite as solved as you think. But it's not quite like an AI foundation model problem because a human interruption detection, it's actually a classical machining problem.

It's a different framing, but you can give classifier for that. Or the question of latency, actually a massive engineering challenge, not an AI challenge. Of course, it can be an AI challenge because people are trying to build voice-to-voice model. So instead of having to firstly transcribe the voice from me into text and then get a model [inaudible 01:03:54] text answer and get another model should turn from text to speech, you can just do voice-to-voice directly. So that is something we're working on, but it's very hard. Yeah. So yeah, so even audio, I think of it's the easier than video because video have both image and voice. It's already pretty hard. So I think there's a lot of challenges in that space.

**Lenny Rachitsky** (01:04:16):
That was an awesome list of things. Let me mirror them back real quick. So what you're predicting in the next few years, things that will change in the way we work, and these actually resonate with so many conversations I've had on this podcast. So says, just kind of doubling down on where things are heading. One is the blurring of lines between different functions instead of just design engineering. Everyone's going to be doing a lot of different things now. Two is, just more of work being automated with agents and all these AI tools and just in theory, productivity going up. Third is, a shifting from pre-training models to post-training, fine-tuning and things like that because to your point, models maybe are slowing down in how smart they're getting.

Although, I'll point folks to the, I had a chat with the co-founder of Anthropic. He made a really good point here. He's like, we're really bad at understanding what exponentials feel like when we're in the middle of that. And also, models are being released more often. So the difference between them we may not notice because they're just happening more often versus GPT3 came out a year before after GPT2. Maybe true, maybe not. And then the fourth point you made is this idea of multimodal, investing in multimodal experiences. I cannot wait for ChatGPT voice mode to get better at interruption, exactly what you're saying. I'm just talking to it and then someone makes a little sound and it's like [inaudible 01:05:33]. Okay. And then you have to, and then it's like, and then it stops talking. It's so annoying.

**Chip Huyen** (01:05:36):
I'm shocked that we don't have better voice assistant at home yet. I think I have been testing out a bunch, honestly. I keep hoping, oh my God, that could be the one and then I know how many of them I just had to give away because they're not that good.

**Lenny Rachitsky** (01:05:49):
I think it's coming. I hear it's coming. Anthropic's working with someone that, I don't know if it's launched or not yet.

**Chip Huyen** (01:05:54):
Yeah, [inaudible 01:05:55] want to bring back to what you mentioned about your guest from Anthropic, mentioned about the performance improvement. I think there's a big change, I think this difference between a model-based capability. So I'm talking about the pre-trained model versus the perceived performance perform. So let's say, I'm not sure you thought about, are you familiar with the term test time compute?

**Lenny Rachitsky** (01:06:20):
I don't think so. Help us understand.

**Chip Huyen** (01:06:26):
So this idea is like okay, you have a fixed amount of compute. So you're going to spend a lot of compute on pre-training or training the model. Pre-training and then I've spent a lot of some compute fine-tuning and the ratio of pre-training to the post-training compute is crazy, varies between different lab. And also, since then has a spend compute on generate inference. When I have a trends and fine-tuning model and now you want to serve it to users. So I might type a question in a prompt and if generate, do inference and that requires a compute. And I guess, I feel about discussion of should I spend more compute on pre-training or fine-training or inference because inference and people thought I was just like test time compute. So spending more compute on inference is like calling test time compute as a strategy of just allocating more resources, compute resource to generate inference when I shouldn't bring better performance and how does that do it?

**Chip Huyen** (01:07:22):
Let's say you have a math questions and maybe instead of just generate one answer again generate four different answers and say okay, whichever is the best according to some standard or okay, I have four answers and then maybe three of them say 42 and one of them says 20. You say okay, three of them in agreement. So the answer should be 42. So just people shouldn't generate a bunch of it. Or another thing is a lot of time like reasoning, thinking, it just be able to generate more thinking tokens, like spend more time thinking before showing the final answers. It's like require more compute but also give more better performance. So yeah, so I think it's like from the ease of perspective when the model spend more time exploring different potential answers, thinking longer, it can give you much better final answers. But the base model itself does not change.

**Lenny Rachitsky** (01:08:16):
Awesome.

**Chip Huyen** (01:08:17):
Does it make sense?

**Lenny Rachitsky** (01:08:18):
Yes, that does. Absolutely.

**Chip Huyen** (01:08:18):
Yeah?

**Lenny Rachitsky** (01:08:19):
That is a good corollary to Ben Man's point.

**Chip Huyen** (01:08:23):
Yeah.

**Lenny Rachitsky** (01:08:23):
Chip, we covered a lot of ground. I've gone through everything I was hoping to learn and more. Before we get to a very exciting lightning round, is there anything else that you wanted to share? Anything else you want to leave listeners with?

**Chip Huyen** (01:08:34):
So I do work with a few companies that does these things of they want employees to come up with ideas. So there's a big debate on what is a better way for AI strategy, should they be top out or bottom up, should executives come up with one or two killer use case and everyone allocate resource to that, should you give engineers and PMs and smart people come up with ideas. And I think it's a mixture of both. So some companies it was like, okay, we hire a bunch of smart people, let's see what they come up with and they organize more hackathons or internal challenge to get people to build product. And one thing that I noticed, a lot of people just don't know what you built. And it shocked me why I feel like we are in some kind of an idea crisis, right?

**Chip Huyen** (01:09:21):
Now, we have all this really cool tools to have. You do everything from scratch, can have you design, it can have you write code, it can build website. So in theory, we should see a lot more, but at the same time, people are somehow stuck. They don't know what to build. And I think it's like, maybe you see a lot of had to do with maybe society expectations because we have gone into this phase of specializations, people very highly specialized and people are supposed to focus on one thing really well instead of being a big picture. And we don't have a big picture view. It's hard to come up with ideas of what you build.

So I know what, when I work with this company on this hackathon, we do work on come up with a guideline, how to come up with ideas. And usually what we think of is like, okay, one tip is go look from the last week. For a week, just pay attention to what you do and what frustrates you. And when something frustrates you, think about, is there anything we can do? Can it be done a different way? So it's not frustrating and you can talk, people can swap to accept [inaudible 01:10:27] or teams, and I even see they come on frustrations. Maybe there's something you can think about just to build something around that. So yeah, so I feel like just notice how we work, thinking of ways, constantly ask questions, how can this be better? And then I just build something to address the frustrations, I think it's a good way to learn and adopt AI.

**Lenny Rachitsky** (01:10:46):
I think people have felt exactly what you're describing every time they open up one of these vibe coding tools where you could just describe anything you want. I'm like, "I don't know, what do I want?" And I love this very tactical piece of advice, just like what frustrates you, just pay attention to where you're frustrated. For example, I just built a very cool little vibe coded app. I was working on a newsletter post inside Google Docs and I pasted all these images into the Google Doc, from screenshots and stuff and then I forgot, oh yeah, you can't take images out of Google Docs. It's like this Hotel of California experience where you can paste stuff into it, very hard to get images back out. So I just went to all the vibe coded tools and just built an app that I can give you a Google Doc URL and it let me download all the images automatically. And it worked amazingly well and I made it really cute. And I'll link to it in the show notes.

**Chip Huyen** (01:11:33):
OH, I would love to see that. I'm very bullish on using AI, just create micro tools. It's just something just make your life a bit easier.

**Lenny Rachitsky** (01:11:41):
A hundred percent. I feel like that's one of the main ways people are using these tools, just a little niche problem they have. With that, Chip, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

**Chip Huyen** (01:11:54):
Yeah, always. No, no, no. It depends on how hard the questions are.

**Lenny Rachitsky** (01:11:58):
They're very consistent across every guest. So I imagine you've heard them before. First question, what are two or three books that you find yourself recommending most to other people?

**Chip Huyen** (01:12:10):
I'm really terrified of book recommendations because I feel like what books [inaudible 01:12:15] you should read really depends on what they want and where they're in life and where they want to get to. But just several books that I do think's have really changed the way I think and see the world. So one thing is The Selfish Gene, that's to understand, it actually helped me with the question whether I want to have kids or not because it's understanding more of a lot of our functions, the way we operate is the functions of our genes and genes want to do one thing, to procreate.

So yes, in a way, the book also proposed another thing is so everyone wants to live forever and maybe it's not consciously, but subconsciously, we do want that. And there are two ways. One is via genes. Genes [inaudible 01:13:00] want to continue forever, but [inaudible 01:13:03] two ideas. I think there's something [inaudible 01:13:05]. It's just like being able, if you have some ideas out there and then it's last for a long time, it's going to live on. I know it's a little bit abstract, but I thought it's very interesting.

The other books I really, really like is from the book from Singaporean previous, I think he is [inaudible 01:13:24] as a Father of Singapore, I don't know, Lee Kuan Yew. I'm not sure what's the title is, but he was the one who led Singapore from, he's changed Singapore from a Third World country to a first world country within 25 years. And I have never seen any country leaders spent so much effort into putting down his thought of how to build a country like that.

**Chip Huyen** (01:13:47):
And as I talk a lot about public policy, how to create policies that encourage people to do the right things that is good for the nations and also talking about foreign affairs, foreign policies, the liberation of the country, but other. So it's a really good book to think about. For me, it's a system thinking, but it's a different kind of system which a country, which a lot of us don't get a chance to ever experiment in our life. So it's good to learn about that.

**Lenny Rachitsky** (01:14:13):
What was the name of that second book?

**Chip Huyen** (01:14:15):
It's called From Third to First World. Actually, I think I have it somewhere here. Yeah.

**Lenny Rachitsky** (01:14:20):
There it is.

**Chip Huyen** (01:14:21):
It's a very heavy book.

**Lenny Rachitsky** (01:14:21):
Show and tell.

**Chip Huyen** (01:14:23):
Yeah.

**Lenny Rachitsky** (01:14:23):
That's awesome. I definitely want to read that. That's a really good [inaudible 01:14:26]. I've heard a lot about just the impact he's had and I've seen all these videos on Twitter of just his really wise insights into how to build a thriving society. And clearly, it works.

**Chip Huyen** (01:14:34):
Yeah. Can you believe, how does he time to write such a thick book? It's insane.

**Lenny Rachitsky** (01:14:39):
That is. Claude, please summarize. I'm just joking. By the way, Selfish Gene, I also absolutely love that book. That is such a good choice. It's such an under the radar kind of book that really changed the way I see the world as well. So really good pick. Okay, next question. Do you have a favorite recent movie or TV show you really enjoyed?

**Chip Huyen** (01:14:56):
So I watched a lot of movie and TV shows as a research because I working on my first novel and I recently sold it. So I'm interested what makes, it's a drama. It's not a science fiction or anything that tech people usually read. So it very, I know it's a very out of left field and very, so it's almost like reading, watching TV to see what kind of stories become popular, trying to understand the trope and stuff like that. So I'm not sure if the audience will like...

**Lenny Rachitsky** (01:15:28):
Well, what's one? What's one that taught you something about writing?

**Chip Huyen** (01:15:35):
I think like Yanxi Palace. It's a Chinese TV show.

**Lenny Rachitsky** (01:15:37):
Cool. Okay. I haven't heard that one on the podcast before. Okay, cool.

**Chip Huyen** (01:15:37):
Yeah.

**Lenny Rachitsky** (01:15:43):
Next question. Do you have a life motto that you often think about, come back to when you're dealing with something hard, whether it's in work or in life?

**Chip Huyen** (01:15:51):
This sounds very nihilist. I think to say, in the end, nothing really matters. Usually, I think of in the grand scheme of things, in a billion years, nothing will, no one would ever be there. I think okay, someone will argue with me about that. [inaudible 01:16:05]. So my theory's like, in a billion years, none of us would ever exist. So whatever messy things, like crazy things we do or how bad we do it, I mean, no one would be remember, wouldn't be there to remember it. And I think in a way, it sounds scary, but it's very liberating because it just allows me say, okay, let's just try things out, right? Why does it matter? And there's a story of recently, so I have some family member who passed away recently. And I was talking to my dad because I couldn't be home for that.

**Chip Huyen** (01:16:36):
I was asking my dad like, "Okay, os there anything I can do to make the person..." Something like comfort. So anything that you can get the persons. And my dad was just like, "What can he possibly want at this moment?" It just made me feel at the end of life, there's nothing that can bring you, like material can bring you joy. There's no money, no product, nothing. And in way, it makes me feel like, okay, what really do I really care about at the end of the day? So I guess it's like I think about it. It's just like, okay, maybe I fail it, maybe I don't get that contract. Maybe those things, but in the end of life, I don't think that actually really matters. So in a way, it's quite liberating.

**Lenny Rachitsky** (01:17:15):
I know you said it might be nihilistic. This is what Steve Jobs shared too in one of his most famous speeches. Just we all die someday day, so don't take things so seriously and it is freeing. Absolutely. It just makes you appreciate every moment, every day you have. Just like, yeah, let's just do something hard and scary. Okay, final question. You talked about how you're writing a novel. Most people in tech have never written something creative and fiction. What's just one thing you learned in the process about how to write better stories, better fiction?

**Chip Huyen** (01:17:46):
A lot of time when we read, we get tripped up by some small things. So I think I want to do creative writing because I just want to go a better writer and it tells us maybe try a different audience could have me become better at anticipating what this different type of audience would want to hear and what they care about. So it's a way for me to get a... So I think if I write it or even any kind of content creations is about predicting the user's reactions, right?

**Lenny Rachitsky** (01:18:14):
The next token.

**Chip Huyen** (01:18:15):
You do a podcast.

**Lenny Rachitsky** (01:18:15):
Just kidding.

**Chip Huyen** (01:18:17):
Yeah. Yeah, so you do a podcast, it's like, okay, what kind things that the users could find engaging, right? And I find this a little bit and a lot of companies you have launch a product, you have a narrative coming out and say, okay, how do we position this product in a way that users would want? So I feel like I have done technical writing for a while and I felt like I had some experience trying to predict what engineers would want to hear or care about. But then I don't have any experience like this, completely different type of audience. So that's what I want to, creative writing, writing a story. And that's why I was doing a lot of research [inaudible 01:18:55]. I mean, doing research [inaudible 01:18:56] enjoyed a lot, watching a lot of dramas. I just see what people like. So one thing that I care about is, I think I learned what emotional journey was from a editor.

So when we write something we care about how users would feel across a story. We want something in the beginning, we want something, we need to have a hook so that people continue reading. But we also don't want too much of drama because we'll get too tired because you're emotionally exhausted because it's like you're being emotionally manipulated a lot of time. So it gave a emotional journey, maybe have some climax or something more chill, maybe like... And also care about another thing I didn't realize is, for me, for technical writing, you entirely focus on the content, the argument. It's very impersonal. For example, people like ML compilers, doesn't matter if they like the person telling them about compiler or not because it's just objective [inaudible 01:19:56]. But for a novel, people care about character likeability.

**Chip Huyen** (01:20:00):
So in the first version of my story, it makes the characters a little bit more, very logical, very rational, and just does everything just very rationally. And then the feedback I got is, I have a very good friend read it and he was like, he's an amazing person, he's a great person. And he was like, "Chip, I'll be honest, I hate that person." So it doesn't matter as a story, it's just like the person is so unlikeable, that's why he doesn't want to continue. So is a second version. It makes that person, the character more likable. How she makes that character more likable is that you put in some vulnerability sometimes it's like okay maybe it's person have setback because sometimes we can relate to it. So in a lot of ways, it's very interesting. A lot of it is about understand the emotional bit, like how the users feel, not just about the story but also about the characters.

**Lenny Rachitsky** (01:20:50):
That is so interesting. Wow. I learned a lot more there than I thought. That was awesome. Really good example. Chip, two final questions. Where can folks find you online, if they want to reach out and maybe work with you or maybe even just share the stuff that you offer if folks want to reach out. And then how can listeners be useful to you?

**Chip Huyen** (01:21:08):
I'm on social media, LinkedIn, Twitter. I don't post a lot, but I keep telling myself that I should do more because I kind of like the conversation with readers. So I'm actually about to I start a Substack. So I have a placeholder for Substack right now and I'm thinking of doing it for more system thinking because I think it's a very interesting skill. I'm also thinking of doing a YouTube channel on book reviews and basically books than help you think better. So I think it's the first book I'm a review is probably like this book because it's my favorite book growing up and I've been keep on reading it. So yeah, so how can you be helpful? Send me books that you like, books that help you have changed the way you think or change you the way you do anything. So I would appreciate it.

**Lenny Rachitsky** (01:22:00):
Amazing. I'm excited to read that book.

**Chip Huyen** (01:22:02):
Mm-hmm.

**Lenny Rachitsky** (01:22:03):
Chip, thank you so much for being here.

**Chip Huyen** (01:22:05):
Thank you so much, Lenny, for having me.

**Lenny Rachitsky** (01:22:07):
Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [6/17] How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna

**Lenny Rachitsky** (00:00:00):
There's a lot of talk about productivity gains through AI. There's this camp of people that are so overhyped, nothing's working, nobody's actually adopting this at scale.

**Dhanji R. Prasanna** (00:00:07):
We see a significant amount of games. We find engineering teams that are very, very AI forward are reporting about eight to 10 hours save per week. Whenever I hear a stat like this, I think an important element is this is the worst it will ever be. This is now the baseline. The truth is the value is changing every day, so you need to ride that wave along with it.

**Lenny Rachitsky** (00:00:27):
There's a story I heard you share on a different podcast where there's an engineer who has Goose watching.

**Dhanji R. Prasanna** (00:00:31):
You'll be talking to a colleague on Slack or an email, and they'll be discussing some feature that they think is useful to implement. Now a few hours later, he'll find that Goose has already tried to build that feature and opened a PR for it on Git.

**Lenny Rachitsky** (00:00:43):
What level of engineer is most benefiting from these tools?

**Dhanji R. Prasanna** (00:00:47):
What's been surprising and really amazing, the non-technical people using AI agents and programming tools to build things, the people that are able to embrace it to optimize for their particular workday and their particular set of tasks are really showing the most impact from these tools.

**Lenny Rachitsky** (00:01:07):
How do you think things will look in a couple of years in terms of how engineers work that's different from today?

**Dhanji R. Prasanna** (00:01:12):
All these LLMs are sitting idle overnight and on weekends, while humans aren't there. There's no need for that. They should be working all the time. They should be trying to build in anticipation of what we want.

**Lenny Rachitsky** (00:01:24):
What's maybe the most counterintuitive lesson you've learned about building products or building teams?

**Dhanji R. Prasanna** (00:01:29):
A lot of engineers think that code quality is important to building a successful product. The two have nothing to do with each other.

**Lenny Rachitsky** (00:01:37):
Today my guest is Dhanji Prasanna. Dhanji is Chief Technology Officer at Block, where he oversees a team of over 3,500 people. With Dhanji's leadership, Block has become one of the most AI-native large companies in the world and has basically achieved what many eng and product leaders are trying to achieve within their companies.

**Lenny Rachitsky** (00:01:55):
In our conversation, we chat about their internal open source agent called Goose, that by their measure is saving employees on average eight to 10 hours a week of work time, and that number is going up, how AI specifically making their teams more productive and the teams that are benefiting most. Interestingly, it's not the engineering team, what it took to shift the culture to be very AI-oriented, the very boring change they made internally that boosted productivity even more than any AI tool.

**Lenny Rachitsky** (00:02:24):
Also, lessons from building Google Wave and Google Plus and Cash app and so much more. This episode is for anyone curious to see what a highly AI-forward technology-driven large company looks like and can act like. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It helps tremendously.

**Dhanji R. Prasanna** (00:05:34):
Thank you Lenny. It's a great pleasure to be here.

**Lenny Rachitsky** (00:05:37):
I want to start with a letter that I hear you wrote to Jack Dorsey to convince him that he and that Block needed to take AI a lot more seriously. I think you called it your AI manifesto and it seems like it really worked. We're going to talk a lot about the changes that came as a result of that. So let me just ask, what did you say in this letter and what happened right after you sent that letter to him?

**Dhanji R. Prasanna** (00:06:00):
So about two and a half years ago or so, Jack really felt like things needed to change. I think he had a sense that the industry was going in a different direction. So he got about 40 of the company's top executives into a room on a weekly basis, and they all used to sort of talk everything through that was going on and he added me to that group.

**Dhanji R. Prasanna** (00:06:26):
So at some point, I observed that we were talking about lots of deep things, lots of relevant things, but no one was really paying attention to AI, and so that's when I wrote that letter. And to be honest, it's I think taken on a life of its own, but there wasn't much to the letter other than I think we should do this. I think we should do it centrally and it's important for us to be ahead of the game and be an AI native company because that's where the industry is heading.

**Lenny Rachitsky** (00:06:55):
Let me just say it's important to note you were not CTO at this point. You were just a senior engineer kind of person?

**Dhanji R. Prasanna** (00:07:00):
No, yeah, in fact, I was part-time at the time because I had just had a kid and I was coming back in and I was helping out one of the engineering teams and then Jack came over to Sydney and spent two days with me and both of us like long walks. So we walked all around Sydney and talked it through up and down, and then yeah, he offered me the job and I thought it was a great opportunity once in a lifetime, so I took it.

**Lenny Rachitsky** (00:07:30):
It's like be careful what you're good at sort of situation. Okay. So what were some of the bigger changes that you made after Jack is on board and Block execs are on board of are, "Cool, this is completely right. We need to go much bigger and think much more deeply about how AI is changing, how we build and how we should build." Or some of the bigger changes that you made from a perspective of other companies listening to this, trying to think about what they should be doing?

**Dhanji R. Prasanna** (00:07:53):
At the start, my main focus was to get block to think like a technology company. And for a long time we had had a little bit of, I'm going to call it identity drift, maybe. We were talking about ourselves as a financial services company. Some people called us FinTech, all of this stuff. But when I started working at what was then known as Square, we were always thought of as a technology company just like Google or Facebook or any of the others.

**Dhanji R. Prasanna** (00:08:25):
And so I wanted to get us back to that. And so the first thing I did was to try and institute a number of programs that focused on that. So everything from getting the top ICs in the company together to talk to each other, to starting a whole bunch of special projects. So we got about two to five engineers per project. There were about eight or nine different projects and we had reinstituted, the company-wide hack week.

**Dhanji R. Prasanna** (00:08:55):
And so all of this just kind of created a little bit of a spark of, "Hey, we're building technology again, we're trying to push the frontier again." And that's how it started, and then there were a whole number of steps after that where we went from a GM structure to a functional org structure, which was I think the key to making our transformation into being more of an AI-native company.

**Lenny Rachitsky** (00:09:19):
Okay, talk more about that. What does that mean? What does that look like? Why is that so important?

**Dhanji R. Prasanna** (00:09:23):
Absolutely. So when we were in our mature phase, so when Square was working quite well, it was a very large business, and then we had started Cash App and that also followed suit. We had spun them out almost as what we call a GM structure. So they were effectively run as a portfolio of independent companies and they had their own CEOs who all reported to Jack and it was still one single executive team, but they had separate engineering practices, they had separate design teams.

**Dhanji R. Prasanna** (00:10:00):
They were kind of separate in almost every way except for some shared resources like our foundational resources like legal and some platforms and things like that. So I think that that was very useful for us for the stage of company that we were in, but when you really want to go deep in technology, when you really want to connect with these things that are industry changing events that are happening, you need a singular focus, and we changed the organization.

**Dhanji R. Prasanna** (00:10:32):
So all engineers report into one single team now, all designers report into one single team and there's single head of engineering, single head of design, et cetera. And so that was the big transformation that we made, and that meant we could really drive forward AI, we could drive forward platform and just technical depth generally.

**Lenny Rachitsky** (00:10:51):
For companies that are struggling with this potentially or trying to figure out how to do this, two things I'm hearing here is start to see yourself as a technology company. It doesn't necessarily apply to every company, but seems like an important element is like we're building technology, we're not a financial company, we're not a real estate company, we're not a technology company. And then two is organize the team such that say engineers report up to an engineering leader versus a GM who maybe doesn't understand engineering as well or doesn't take it as seriously as they should.

**Dhanji R. Prasanna** (00:11:19):
Yeah, I think that's pretty much what we did. And not to lean too heavily on this, but this is what jobs did when he came back to Apple as well. He reorganized Apple to be functional, and it wasn't like we were following a playbook. We discovered this as we were investigating what it's going to take to make these teams more tech-focused and to bring our DNA back to our roots, which really was putting engineering and design first, which is what technology first means to me. So yeah, I would say to companies, find your DNA and really try to optimize for what that is in a very simple and clear way.

**Lenny Rachitsky** (00:12:00):
Okay, so you made a bunch of changes, you had this manifesto, everyone's on board, you made a bunch of changes. Functional technology first, comparing the way that your say engineering team works today versus two or three years ago, what is most different?

**Dhanji R. Prasanna** (00:12:13):
Not everyone was on board, I'll tell you that. It was quite a painful transformation. I think that one of the things that I learned the most throughout this process is that Conway's Law can be really, really powerful. So it's the law that basically says you ship your org structure. So what you're organized as in terms of teams, in terms of collaborating groups and your operating model matters a lot to what you build.

**Dhanji R. Prasanna** (00:12:45):
And so I think that that was essentially the biggest change is we had a lot of momentum in each of these silos, be it Cash App, be it Afterpay, be it Square or even TIDAL or music streaming service. And no one was really talking to each other, no one was really aligned on technical strategy on what we even wanted to be five years from now as a collective team. And so all those things are different now. I'm not saying it's perfect, there's still a long road ahead of us, but we at least speak the same language.

**Dhanji R. Prasanna** (00:13:24):
We're all have access to the same tools, we share the same policies. So a certain level of senior engineer means the same thing across the whole company. People can move from one team to another's into an area of need. All of these things are very different. But to sum it up, I would say we're technically focused and we're focused on advancing technical excellence as a goal. And that just really wasn't that true two to three years ago. There were other things we were optimizing for then.

**Lenny Rachitsky** (00:13:58):
Maybe going one level deeper in terms of how people actually work at a day. So if you're looking at an engineering team, say the average engineering team and maybe also the top most optimal engineering team, how is the way they work today different from a couple of years ago?

**Dhanji R. Prasanna** (00:14:12):
In the small, certain teams that are very, very AI natives or teams that are building AI first everywhere are working much differently than before because they're using vibe code tools and they're essentially building without writing lines of code by hand, and that just wasn't true through the three years ago. I don't think it was true anywhere in the world. So that's dramatically different in teams that are still working with very heavy legacy code bases.

**Dhanji R. Prasanna** (00:14:47):
It's less true, but they're also encountering these background AI processes. So we have these tools that run 24/7 or run in the CI pipeline and they're analyzing vulnerabilities. They're looking at even bugs filed on tickets and trying to build patches while engineers are asleep. So they come in the next day and look at it. So I would say there are a number of ways in which they're different, but different teams have adapted in different ways depending on how close they are to the tools.

**Lenny Rachitsky** (00:15:25):
Okay, so let me lean into that AI piece, which is I think where you guys are most ahead of a lot of other companies. You guys built your own agent I think is how you describe Goose. So there's a lot of talk about productivity gains through AI. There's this camp of people are like, you don't understand how much productivity there is to gain from AI. It's the future, this is the way it's all going to work.

**Lenny Rachitsky** (00:15:46):
We're all accelerating 10X. There's also this camp where people are like, I'm so overhyped, nothing's working. People talk about it. All these pilots are failing. Nobody's actually adopting this at scale. I feel like you're probably in that first camp. What sort of gains have you seen practically from AI tools on your teams?

**Dhanji R. Prasanna** (00:16:03):
Our number one priority is through automate Block, which means getting AI and getting AI forms of automation through our entire company. And we feel that that's just at the beginning of where the utility is with all these large language models, and I think we're going to continue to see that improve. But even now, we find engineering teams that are very, very AI forward that are using Goose every day are reporting about eight to 10 hours saved per week, and this is self-reported. And then we also have a number of check metrics to try and validate that.

**Dhanji R. Prasanna** (00:16:45):
So we look at PRs, we look at throughput of features, we look at a whole bunch of things and we have our data scientists come up with a complicated formula that tries to distill it all into something meaningful. And we feel across the whole company, we're probably trending towards 20 to 25% of manual hours saved. And I think that's just the start of all of this. I do feel that the more AI-native companies are doing a better job of realizing this.

**Dhanji R. Prasanna** (00:17:17):
So companies that started just with AI startups mostly, but there is some truth to this notion that AI isn't a panacea and it's growing as well in capability. So you need to ride that wave along with it. And I think a lot of the companies aren't realizing this. They're like, "Well, where's the value?" And the truth is the value is changing every day. And so you need to be adaptable and look at what the value is today and plan for what the value will be tomorrow and then slowly expand to the areas where it's most efficacious.

**Dhanji R. Prasanna** (00:17:54):
I'll give you an example. One area in which we find that it's really good is for non-technical teams to be able to build little software tools for themselves. So this has been one of the most surprising and energizing uses of Goose within Block is we'll have our enterprise risk management team build a whole system for self-servicing enterprise risk, and this is compressing weeks of work into hours, or ordinarily, they would be waiting for an internal apps team or something to go and build that and they would put that on their Q2 roadmap and everyone would be twiddling their thumbs until it all clicked into place, but now you can just go and do it.

**Dhanji R. Prasanna** (00:18:42):
And so a lot of these kinds of use cases we're seeing an enormous amount of productivity gain in the other area, which I'm really excited about is we have this other tool called Gosling, which is a goose for mobile effectively. So it operates your Android OS at a native level using the accessibility API. And we use that for automating UI tests.

**Dhanji R. Prasanna** (00:19:09):
So before, you would have to hire an army of contractors or QAs who would go and click through every screen, but now we can just bake those into automated tests and then give you a report at the end. So we're seeing a lot of advantages in those types of areas, but where you have a lot of depth and a lot of really strong people come together is where AI, I think still underperforms humans. And that's something that's probably going to get better over time, but it's also something where we should lean into as humans.

**Dhanji R. Prasanna** (00:19:45):
So when you have some very senior engineers and they're thinking about things like architecture and design and race conditions, orchestration, things like this, that's still an area where AI isn't quite there. And so I think the companies that aren't feeling the success in AI are trying to just throw these tools at their giant code bases and hoping good things will happen, and that's not how it's playing out. Eventually, I do think it'll get there, but right now we're still in the early utility phase.

**Lenny Rachitsky** (00:20:18):
Holy moly, there's so much there in what you just shared. There's like five things I want to follow up on. Okay, so one is this metric you kind of alluded to, which is how you measure the impact of AI in your team. So it was human manual hours saved, is that how you describe it?

**Dhanji R. Prasanna** (00:20:35):
That's correct. Yeah.

**Lenny Rachitsky** (00:20:36):
So it's roughly a fourth of an engineer's time currently is being saved by AI tooling.

**Dhanji R. Prasanna** (00:20:43):
That metric is across all teams. So that would be our support teams, our legal teams, our risk teams, all of them together.

**Lenny Rachitsky** (00:20:51):
Wow.

**Dhanji R. Prasanna** (00:20:52):
And then on the engineering side, it's very variable because like I said before, it matters how big and how complex the code base is. And so if you're building a totally new Greenfields code base or you're building an app for a new platform, then we're seeing those pretty aggressive gains, but in very complex code bases that already exist, those gains are not quite there yet.

**Lenny Rachitsky** (00:21:18):
That's amazing. And whenever I hear a stat like this, I think an important element that people need to think about is this is the worst it will ever be. This is the lowest, this is now the baseline. And so it may not sound that crazy yet, but it's going to get crazy. Okay, the other thing that you talked about is Goose, you haven't explained what Goose is. This is a huge deal. Explain what Goose is and how important this has become to you guys.

**Dhanji R. Prasanna** (00:21:49):
So Goose is a general purpose AI agent. So you can think of it as a desktop tool or a program that you can download and install on your computer and then it has a UI. You can talk to it just like a chatbot and you can say anything from, "Hey Goose, organize my photos by category, and it has the ability to look within your photos and if there are a lot of trees, it'll organize them as nature photos. And there are a lot of people, it'll organize them as portraiture." All of this sort of stuff to writing software for you.

**Dhanji R. Prasanna** (00:22:30):
So it can do all of these tasks, and the way we've been able to do this is through something called a model context protocol or the MCP, which a lot of your listeners might've heard. And this is something that Anthropic came up with that we were a very early contributor to. And the model context protocol is very simply just a set of formalized wrappers around existing tools or existing capabilities. So if you have tools that you use in the enterprise, be it Salesforce or be it Snowflake or SQL, any of these things, you can wrap them in the MCP and then it exposes them to your LLM to be able to manipulate.

**Dhanji R. Prasanna** (00:23:16):
So until that point, the LLMs were not really able to do much other than chat, but Goose gives these brains arms and legs to go out and act in our digital world, and that's where we find it's had most impact and it's built on this fairly open protocol that anyone can implement. There have been an explosion of MCPs. Goose is entirely open source, by the way, so any of you can download it and extend it, write your own MCPs, and that's been our core successes through Goose.

**Lenny Rachitsky** (00:23:54):
Okay. So essentially like Claude code with a UI, desktop app sort of thing built on top of Claude and OpenAI ChatGPT and a bunch of open source models. Is that right?

**Dhanji R. Prasanna** (00:24:06):
Yeah, it can use any model. So we have a pluggable provider system and you can either bring your own API keys and use the Claude family models or OpenAI's family models, or you can use open source models and you can download them and use them directly or via Ollama and other, there are several tools that help you do that, but essentially it's taking the capability of these models to generate text and to interpret text and applying them to real world situations.

**Dhanji R. Prasanna** (00:24:42):
So one example that I really like is you can ask Goose to go and build your marketing report and it has MCPs to connect to Snowflake and Tableau and Looker. So it'll write SQL to pull out data from there, it'll do some analysis and a CSV so it can write Python code on your desktop to do all that. It will generate some graphs using some JavaScript charting library that it knows about.

**Dhanji R. Prasanna** (00:25:11):
And then finally, it'll put this all into a PDF or Google Doc or whatever and it can even email it for you or upload it somewhere. And it's doing all of this on its own, by the way. No one's sitting here telling it that, you're just saying, "Hey, I want this report, I want this emailed here, I want these pretty charts." And it's orchestrating across all these systems.

**Lenny Rachitsky** (00:25:31):
So essentially at Block, instead of using Claude or ChatGPT directly or even Cursor and all these apps, they use Goose?

**Dhanji R. Prasanna** (00:25:40):
Yeah, we allow our engineers and our general employee population to use any tools that they want. Goose is the one that's most well-integrated into all of our systems because it's built on the MCP and it's so easy to create an MCP for an existing system. So for example, if you're using a issue tracking tool and you want some AI automation added to it, before Goose, our teams would have to wait for the vendor to build that AI capability in there, or maybe there's some way in which OpenAI or Anthropic or Google would provide a general purpose capability where we could plug those in. But with Goose, that's no longer necessary with a few lines of code that an MCP represents. All these systems are orchestratable with AI basically overnight, and Goose can write its own MCPs. So it's pretty bootstrappable as well.

**Lenny Rachitsky** (00:26:43):
And this is open source and basically you've spent all this time building this thing, any other company can now implement it and build on all the work you've done?

**Dhanji R. Prasanna** (00:26:51):
Yeah, and we have a lot of companies using Goose pretty actively. I don't want to name too many names, but from our competitors to our close partners, a lot of them are using Goose pretty regularly on their teams. I know Databricks talks about it a lot, but everyone you can think of in this mid-tech tier is using Goose in some form.

**Lenny Rachitsky** (00:27:15):
That's insane. This feels like it could've been a massive business of its own, some of the fastest growing companies in the world, basically this is their product and you've built it and given away.

**Dhanji R. Prasanna** (00:27:23):
Yeah, we believe in the power of open source and one of our core missions is to increase openness, and that means contributing to open protocols and contributing to open source. And as a tech company, we're built on a lot of open source software. I think pretty much every tech company is whether you're talking about Linux or Java or MySQL or any of these essential components, and so we feel like we have a strong imperative to give back.

**Dhanji R. Prasanna** (00:27:57):
We want to build things that not only are good for us and our customers, but that outlast Block and outgrow Block, that's certainly a core value for us and has been from the beginning even long before this whole AI phase. So yeah, Goose follows in that proud tradition and yeah, we're very excited that its had the success it's had.

**Lenny Rachitsky** (00:28:18):
What's the story with the name Goose, by the way? Can't help but ask.

**Dhanji R. Prasanna** (00:28:21):
Goose is a Top Gun reference. So our engineer that came up with it. He also looks exactly like Goose, so it's kind of crazy if you put them side to side, he's going to be really embarrassed with my sharing this, but that's the reason why they call it Goose, and then we lent into the whole bird theme after that.

**Lenny Rachitsky** (00:28:41):
That's incredible. There's a story I heard you share on a different podcast where there's an engineer who takes this to the extreme and has Goose watch him. Talk about that, share that story.

**Dhanji R. Prasanna** (00:28:50):
Yeah, absolutely. So he is very, very AI-focused and he's trying to extract all these crazy ideas from Goose and Goose can do all of the things that I described through specific interactions with tools, but it can also just watch your screen so it understands how to process images and process the things that it's looking at through screenshots. And so he built this system where it's essentially just watching everything he does all the time and he'll be talking to a colleague on Slack or an email and they'll be discussing some feature that they think is useful to implement.

**Dhanji R. Prasanna** (00:29:30):
And then a few hours later he'll find that Goose has already tried to build that feature and opened a PR for it on Git and all sorts of other wacky things like that. So it'll try to nudge him out of a workflow. If he's running over on a meeting and he's late for something else, it comes up with these creative things that he didn't program or he didn't write prompts for, but that it thinks will help him improve his productivity or improve his work day. So yeah, it's pretty crazy. You have to have the stomach for it to be that level of tied into your working tools, but it kind of shows you what's possible with tools like this.

**Lenny Rachitsky** (00:30:14):
Clearly this is where things are going. Once this gets good enough, I love this guy is just trying it. So it's basically watching him work and anticipating what he should be doing and does the work for him as a first draft so that he's like, "Oh, the PR is already done on this thing. We were just talking about it at this meeting." That's incredible.

**Dhanji R. Prasanna** (00:30:31):
Exactly.

**Lenny Rachitsky** (00:30:32):
How good is it? Where's it at? If you had to go zero to a hundred of like, "Okay, going to, all you have to do is now think and talk and that'll just do your job."

**Dhanji R. Prasanna** (00:30:41):
Yeah, so voice is the other big part of it. It has voice processing capability, so it's always listening to what he's saying as well and trying to interpret that. I would say that this is mostly an experiment, given that he's on our core Goose team and he contributes to Goose, so he has a day job. This is a kind of thing on the side that he was developing.

**Dhanji R. Prasanna** (00:31:04):
So once this evolves into more of a native feature of Goose itself or other tools that we use in the enterprise, I think it can have a lot of legs, but it's already pretty good. It's probably cutting down enormous amounts of busy work that he has to do. So for example, one thing he'll do is he'll say, "Oh, I have a meeting conflict. I can't make it that time, or I have to go pick up my kid."

**Dhanji R. Prasanna** (00:31:32):
And Goose will automatically reschedule that meeting without him ever sitting in front of his calendar and clicking through 10 times. Yeah, so these are things that I think we were waiting for the calendar vendor to build as features into calendar, but we don't need to do that anymore because AI is able to orchestrate this for us.

**Lenny Rachitsky** (00:31:53):
This isn't that guy that had four jobs at four different startups that he was able to paralyze all his work and hire people.

**Dhanji R. Prasanna** (00:31:58):
No, it's not. He's someone that I've worked with for a long time and he's been at Block for a long time. He just loves experimenting and he embodies that culture of experimentation just like our creator of Goose who did the same thing.

**Lenny Rachitsky** (00:32:16):
So let me pull on that thread a little bit. You're kind of seeing a glimpse of where things are going. You're very ahead of the curve in a lot of ways at Block. How do you think things will look in a couple of years in terms of how engineers work, how product teams work that's different from today?

**Dhanji R. Prasanna** (00:32:32):
I think a lot of it is dependent on the improvement of LLM performance, but I can tell you the way I'm trying to change how I work and how I'm trying to change our immediate team's way of working. So I think vibe coding has been an interesting, exciting thing, which is you talk to a chatbot essentially and it goes and builds software for you, but I think this is highly limiting.

**Dhanji R. Prasanna** (00:32:59):
It's very ping pong. You do something, you wait for three or four minutes and it comes back with something sort of half-baked and you have to nudge it and guide it and massage it to get where it needs to be. I think that we're going to see much more autonomy. So where we're working on a couple of experiments with Goose, with the next version of Goose where we're really trying to push it to work not just for two or three or five minutes at a time, our median session length is five minutes and on average, seven, but we're trying to push it to hours.

**Dhanji R. Prasanna** (00:33:36):
We're trying to say, "Hey, all these LLMs are sitting idle overnight and on weekends while humans aren't there, there's no need for that." They should be working all the time. They should be trying to build in anticipation of what we want if we go back to the earlier part of the conversation. But also I think that they should be able to build in ways that were never possible before.

**Dhanji R. Prasanna** (00:34:04):
Before as humans, we had limited resources, limited bandwidth, and a lot of coordination overhead. So we would have to choose the best path to try in an experiment, and I don't think we need that anymore. We need instead to be able to describe multiple different experiments in a great amount of detail. And then maybe we go to sleep and then in the morning, all those experiments are built and we can sort of throw away five or six of them.

**Dhanji R. Prasanna** (00:34:34):
So one of the things that I do regularly, so I write code every day, but one of the things that I do regularly is just throw away huge, huge amounts of code, and it's kind of hard for me because I've never done that before. I mean obviously engineers love deleting code, but this is different. You build a whole new system or a whole new feature and you're like, "Ah, it doesn't feel exactly right. I'm just going to delete and start over."

**Dhanji R. Prasanna** (00:35:00):
So I think you're going to see a lot more of that way of working. And I think that you're going to see instead of us, for example, refactoring an app to have a different UI or to evolve into its new version, we're just going to rewrite that app from scratch. And one of the things I'm really pushing our teams to think about is what would our world look like if every single release, RM minus RF deleted the entire app and rebuilt it from scratch? And so we can't really do that today, but I think this shows you some of the direction of what's possible and where these tools are taking us.

**Lenny Rachitsky** (00:35:42):
What's interesting about that is that there's this common rule in software engineering and just product, don't ever just rewrite. Don't try to rewrite your thing. You're going to forget all of the small improvements and tweaks and bug fixes people have made over the years, and you think it's going to be the simple straightforward thing. It ends up being now it's like a year or more of just getting it back to where it was. And so interesting that AI now can make that possible, and what you're saying is that's actually maybe the way you should be working.

**Dhanji R. Prasanna** (00:36:09):
I think so. And I think that the trick is getting the AI to respect all of those incremental improvements, yeah, and sort of bake those in as a part of the specification, if you will. Yeah.

**Lenny Rachitsky** (00:36:25):
Also, the point you made about this agent, just you give it a bunch of ideas that builds them overnight and then you could see, I imagine it goes even further up the stack and comes up with the ideas and then starts building them and then you're like, "Okay, oh, that was a great idea. Now I can see it immediately in the same workflow."

**Dhanji R. Prasanna** (00:36:39):
Yeah, that's true. I was actually literally trying what you're saying just last week. And so I have this new version of Goose that we're working on and I was asking it to come up with ideas to improve itself and implement it overnight. And sometimes-

**Lenny Rachitsky** (00:36:57):
Slip problem.

**Dhanji R. Prasanna** (00:36:59):
... Sometimes it kind of goes off the script entirely and you have to sort of pull it back a bit. So I think we're not quite at that era where it's completely self-improving and completely autonomous, but I do think we're in a transition phase where we can give it that nudge and say, "Hey, here's my wishlist of 10 things that I wish you could do. Go and figure out the best way to do them." And it's successful I would say on 60% of those things, if the features are well enough described and it struggles on the remaining 40 where you have to kind of intervene and massage it. Yeah.

**Lenny Rachitsky** (00:37:43):
Oh man, I'm just imagining this feature where you give it the goal of drive revenue and growth and then it's just like, "Okay, everyone's fired. Here's your paychecks. I'll take it from here."

**Dhanji R. Prasanna** (00:37:56):
I don't think we're going to be there. I do think we're going to need a lot of human taste to anchor these AIs so they don't go off script to be honest. And that's really where our design lead and our design teams are pushing us to think, and that's a differentiator that I think will push us beyond this era of AI slop that everyone's talking about. So yeah, it's very much anchoring it into a thing that matters to people and the thing that's tasteful and useful and has value.

**Lenny Rachitsky** (00:38:30):
To make that even more concrete, is there an example of something maybe AI was trying to, or a team was trying to pitch where you had to just know this is where humans are going to step in and keep things on track?

**Dhanji R. Prasanna** (00:38:43):
I'd say it was more around things like process automation or a lot of times I'll get this sort of request where a team will say, "We need to buy this new tool from this vendor because our current tool is entering X, Y and Z." Another team will say, "No, no, no, we can just use Goose to build an app that will do the same thing for us in half the time or less." And then as a human, you're sitting there thinking, "Is any of this necessary? If we just change the process, do we even need to think about building tools?" And this is the thing that AI isn't good at, it's not able to have this portfolio judgment or judgment across a global sense of what's important and what matters.

**Dhanji R. Prasanna** (00:39:40):
So a lot of times, I tell teams just question the base assumption, particularly our InfoSec teams because they'll twist themselves into knot sometimes trying to secure something and you'll be like, "We'll just ask the team that's building it to do it differently or to not build that at all if it doesn't matter, and then you won't have to increase your surface area of securing it." So I think those are the areas where it's better for a human to use judgment and AI has not done a great job.

**Lenny Rachitsky** (00:40:11):
You make this point about building your own software, your own tools instead of buying stuff. This is a big question with AI, is it's going to replace all these SaaS apps to Salesforce over. Is there a sense of just either how much money you guys have maybe saved building your own stuff, or have you built a new-found respect for the existing SaaS software that everyone's using and pays lots of money for?

**Dhanji R. Prasanna** (00:40:31):
I think there's a trap in getting away from your core purpose as a company. And our core purpose is economic empowerment. So getting customers or merchants or artists the ability to make a sale or pay their rent or upload their latest creation to TIDAL. And I think that anything that serves that purpose, we should encourage and we should invest in, but if we're just purely looking at dollars versus dollars, then that's pulling us off that purpose.

**Dhanji R. Prasanna** (00:41:12):
The savings and costs that there might be in replacing a vendor tool by something you build in-house is probably not worth it in the mental bandwidth that you've lost and the amount of the team's technical focus that's being taken away. So yeah, I would say it just keep coming to the thing that matters to you as a company and then the rest will follow from that.

**Lenny Rachitsky** (00:41:38):
Yeah, I think people forget just how much maintenance it takes to keep something you've built. Like, "Okay, cool, we built it in a weekend and now it's years of endless maintenance and requests and support." And also to your point, it feels like it comes back to the always motto of just focus on your core competencies and then buy everything else.

**Dhanji R. Prasanna** (00:41:57):
Yeah, it's the classic 80/20 problem, and we have that enough with the apps that we build for our customers. We'll build some great experiments that really resonate, and then we have to spend a lot of time ironing out the long tail of problems. So in Cash Card, for example, we built the entire functionality of Cash Card, I would say pretty much in a weekend or maybe a week of integration and work.

**Dhanji R. Prasanna** (00:42:26):
And then it took a really long time to iron out all these edge cases where someone would tip twice the value of the bill and then it would completely break something in the back end, or people would use it as a gas station and they have a different way of billing your card. So yeah, it's very much that. And to your point, I would always come back to what is the reason we're doing this? Why does it matter to us and to our customers? And if it doesn't clearly satisfy that, I would just push it off as a not interesting thing.

**Lenny Rachitsky** (00:43:04):
**Dhanji R. Prasanna** (00:44:34):
I don't think that things have progressed far enough that it's really impacted in a fundamental way how many people you would need to build an app of the scale of Cash App, for example. I think what's changed for us is much different and it has nothing to do with AI, it's what we talked about earlier is moving from our GM structure to a functional structure. And in our GM structure, our incentives were always to think of engineering headcount as a commodity.

**Dhanji R. Prasanna** (00:45:09):
And so we would just add more engineers if we wanted to build more features and the classic mythical man person month trap or whatever it's called. And I think that moving to a functional structure completely changes that and you're like, "Well, we can leverage common platforms, common modules, we can bring in experts from across the company to advise us on how better to do this."

**Dhanji R. Prasanna** (00:45:38):
And so those kinds of things I think have made it much different and how we hire and we no longer see engineers as a commodity to just add 100 people to go and build the next product in Cash App. But on the AI side, we're very much looking for people that are embracing these tools and that are eager to try and learn from it. We're not looking for people who are amazing AI practitioners on the get-go.

**Dhanji R. Prasanna** (00:46:14):
I think we have those people and we're interested in those people if they ever want to work with us. But I'm much more keen on looking for that college grad who just really is eager to learn about these tools and open to it, or even the veteran who has embraced these tools and figured it out. And that's kind of where we're optimizing for who we look for rather than a specific set of skills.

**Lenny Rachitsky** (00:46:44):
So essentially the biggest change is just looking for people that are embracing AI, not being like, "No, I don't need this stuff. I'm an amazing engineer. I don't need to use Cursor or Goose or all these things."

**Dhanji R. Prasanna** (00:46:55):
Yeah, a learning mindset is how I would put it. This is something that Jack our CEO talks about a lot is he wants us to be a learning first company. So everything we do, every experiment that we ship, what can we learn from it and did we feel that we gave it our best shot? And I think that that's more important to him than even sort of coming up with the right business answer every time.

**Lenny Rachitsky** (00:47:25):
What about when you're interviewing? Are you encouraging engineers to use AI tools as they're doing exercises? How did that change over the past year or two?

**Dhanji R. Prasanna** (00:47:33):
Yeah, we're starting to do that now. So traditionally we would just use CoderPad or something like that to wipe boards or a problem or even program it in Pseudocode or near Pseudocode. But now we're looking at can you use Vibe code to build something? How comfortable are you with these tools or how are you thinking about evolving with them as well?

**Dhanji R. Prasanna** (00:48:04):
But it's early days yet I would say that it's not clear to me that necessarily how someone knows how to use, be it Goose or Cursor or any of these other tools matters that much to whether they're a good engineer. I still think that things that we interviewed for in the past, a critical mindset, the ability to really understand deeply the technical nature of a problem is still much more important than whether you're a fully AI native programmer or not.

**Lenny Rachitsky** (00:48:37):
Another question that I've always been thinking about a lot of people wonder is what level of engineer is most benefiting from these tools? You could argue it's the junior engineers now, they could just get all this work done. You could argue it's senior engineers because they know so much more about how things work and now they could just orchestrate thousands of agents doing their bidding. What have you seen in terms of which level is benefiting most?

**Dhanji R. Prasanna** (00:48:56):
Yeah, so two answers to that. One is you're definitely right that the more senior and the more junior they are, the more comfortable or the more eager they are to adopt these AI tools. And I think that's for a variety of reasons, including some of them that you named. I think the senior people really understand in great depth how everything works.

**Dhanji R. Prasanna** (00:49:18):
And so they're almost relieved that this tool exists that can go and do all these things that they've done a million times before and couldn't be bothered. And then the junior people are like my niece and nephew on a BlackBerry or something, they're just blitzing through things, not BlackBerry in the early days and iPhones now, they're blitzing through a text message when I'm still seek and destroying through my keyboard, shows you how old I am.

**Dhanji R. Prasanna** (00:49:50):
So I think there's that, but I think the non-technical people using AI agents and programming tools to build things is really what's been surprising and really amazing. And I think that speaks to how these roles are going to evolve in the future. The lines are going to be blurred between whether you're in legal or in risk or in engineering and design even. And so I think that the people that are able to embrace it to optimize for their particular work day and their particular set of tasks are really who are showing the most impact from these tools.

**Lenny Rachitsky** (00:50:36):
It's interesting. No one talks about that element of engineering productivity, which is the reduction of asks from all the other parts of the company to build random one-off things. That feels like a huge productivity gain for engineers.

**Dhanji R. Prasanna** (00:50:47):
It is massive, although I think that it's a little bit like the analogy of if you build a bigger highway, you'll just get more cars on the road. So I think the fact that everyone's building software means that there's more software to be built, more coordination to happen, and everyone's more eager to ship things faster and with greater results. And so we're just seeing an overall uptake in velocity and the ask for more features, if that makes sense. Yeah.

**Lenny Rachitsky** (00:51:22):
Absolutely. And it connects to your point about you're not slowing hiring. What I'm hearing is just headcount, hiring desires for more engineers, more product people is not slowing at all. You're basically, it's as if AI wasn't really there.

**Dhanji R. Prasanna** (00:51:37):
We're being more thoughtful about it. So like I said, we were looking at as a commodity in the GM era, and now that we're functional, it's much less about how many engineers we need as a function of the number of features we have in Square or Cash App and in the functional org structure, we think of it much more as what are the areas of optimization? Where can we build depth and what really accelerates our priorities through things like modularization reuse and going deep into platforms.

**Lenny Rachitsky** (00:52:12):
I love this hot take of if you're trying to be more productive, forget AI, just re-org into a functional structure.

**Dhanji R. Prasanna** (00:52:21):
It's not wrong in some ways. So here's another really interesting example where we are trying to improve our build times and you were using Goose and a lot of other tools to help us with this too, and they've done remarkable things. So we have this really cool tool that analyzes our test suites and selects the right test to run for changes that were made.

**Dhanji R. Prasanna** (00:52:50):
So we cut down basically 50% of test runs this way, which is pretty great, and we're not warming the planet as much with all these unnecessary CPU cycles being wasted on tests. But then things like offloading tests to the cloud or simply just deleting tests that don't make sense anymore, probably save you two to three times that.

**Dhanji R. Prasanna** (00:53:14):
So there is still a portfolio approach that you need to take for lack of a better term. It's like that example I told you earlier about should we buy a vendor tool? Should we build this in-house? It's like, "Well, do we even need to do this process at all?" So in some ways, structure matters more than the efficacy of the tools you have.

**Lenny Rachitsky** (00:53:34):
Wise words makes me think about Elon has this whole process for optimize stuff and one of the steps is like, "Do we even need this thing before we start out optimizing and automating it?" Before I zoom out and ask about just general lessons that you've learned over the course of your career, is there anything else that you think might be really valuable or useful to folks that are trying to lean in further into AI or just help their teams think a little bit more forward thinking?

**Dhanji R. Prasanna** (00:54:04):
I would say really try and use these tools yourself. So the way in which I think we've been able to drive most of the adoption is Jack uses Goose, I use Goose, our executive team all have used Goose and use it regularly and use other AI programming tools and assistance as well, and we do it every single day.

**Dhanji R. Prasanna** (00:54:31):
And so we learn a lot about how our own workflow can change, and that's going to tell you so much more about how are you going to change your organization's workflow than if you're reading a bunch of think pieces on LinkedIn or Harvard Business Review or whatever it is, and then trying to get your teams to follow suit. So I think we do this with everything. It's feel it, use the product yourself, feel it, understand its strengths and weaknesses and its ergonomics, and then figure out how to apply it to your teams.

**Lenny Rachitsky** (00:55:06):
Something I've found helpful in doing that, which I completely agree with, which is stop reading about it, stop listening to us talking about it, just build some stuff. The thing that I found really helpful there is have a specific task or problem you want to solve for yourself because that really motivates you and makes it very real.

**Lenny Rachitsky** (00:55:21):
For example, just the other day, I was trying to pull images out of a Google Doc. Google Doc, it's like I think of it as Hotel California. You put images in there, but there's no way to get them back out unless you do some crazy stuff. So I just went to Lovable and like Bill an app, or I can give you a Google Doc URL and let me download the images real easily and bam. Perfect.

**Dhanji R. Prasanna** (00:55:41):
Yeah, great example. I did something like this a couple months ago as well, where my son has a whole bunch of therapies, he has additional needs, and so I was trying to gather the receipts for all these therapies and share them with my wife and she will claim it from our insurer, and I was really struggling to do this because they're in various forms.

**Dhanji R. Prasanna** (00:56:05):
There are screenshots in some cases or PDFs or whatever. So I asked Goose to do this and it was all sitting on my laptop and Goose figured out that it could put all of these receipts into my Apple Notes app into a single note. It converted it to HTML so it would sync seamlessly to my phone and then I could email it or share it with her from there.

**Dhanji R. Prasanna** (00:56:30):
And that's just something I just never would've thought of. And it did this using Apple Script, so it just controlled my computer for me in the background. Yeah, so these are surprising ways in which these tools help us, and the more you use them to solve real problems to your point, the more you understand what their strengths are and where you can deploy them.

**Lenny Rachitsky** (00:56:51):
I love this example. So did you just go to Goose and be like, "Here's the problem I have, how would you solve it?"

**Dhanji R. Prasanna** (00:56:56):
Yeah, pretty much. I said, "I have all these receipts there in Google Drive, so we have similar origin problem there and I need to get them into a single form and I need to collate the totals and do all this." So it tried a few approaches first. It tried to download them and it tried to read them using a PDF reader and this and that. And then the thing about Goose that I think a lot of the other AI agents learn from us as well is if it tries a few things and fails, it'll back up and it'll try a different route and it'll just keep going until it makes some progress.

**Dhanji R. Prasanna** (00:57:33):
And that's what it did. Then it picked Apple Script as a way to do it because it had the MCP extension to control my computer, and this is the same thing that our engineer, we were talking about the other day uses to watch his screen and things like that, but this was a very focused problem and it managed to do that. So yeah, it's surprising what these tools can do and allowing them the flexibility to do that is a big part of learning how to use them.

**Lenny Rachitsky** (00:58:02):
That's cool. I love the, by the way, can you run Goose as a regular person? Can you just download Goose and use that instead of Claude?

**Dhanji R. Prasanna** (00:58:08):
Yeah, absolutely. Yeah. You can just download it from our URL. We can share it in the show notes for you and yeah, you can install it. It comes for Mac and Windows and Linux I believe. It's an electron app, so it'll work on all of them. It also has a command line, so for people who are more comfortable using that, we have that UI as well.

**Lenny Rachitsky** (00:58:32):
Wow, you really are competing with these massive foundational model companies building. What's the simplest way to compare Goose to something else? Is it like this Claude code, this simplest comparison or something else?

**Dhanji R. Prasanna** (00:58:43):
I think it's a bit different than Claude Code because at its core, Goose is a platform that implements MCPs. So MCPs give it this dynamically extensible nature so it can do all of these things for you, whether it's automating things like we were talking about with Google Docs and notes and things like that, or it can do straight up programming tasks for you using other MCPs.

**Dhanji R. Prasanna** (00:59:11):
It can index code and do it that way. So it's really more of an extensible platform. So I would say it sits somewhere between your classic AI assistant where you just ask it, "What's the weather today? Can you calculate how many months it's been since this date?" Or whatever it is, to the more focused cursors and Claude codes of the world.

**Lenny Rachitsky** (00:59:39):
Basically, it's everything combined wholly and free. You pay for the LM tokens, but yeah.

**Dhanji R. Prasanna** (00:59:46):
Yeah, there's not like an open source models which-

**Lenny Rachitsky** (00:59:50):
Oh my God, this is crazy. What a cool team to be on building Goose at Block. Must having must be having so much fun. Oh man. Okay. Let me zoom out a little bit. So you've been CTO in LinkedIn right now for just about two years. What's something that you wish you knew before you stepped in this role? If you could go back a couple years and just whisper a few tips and tricks or lessons into your ear, what would they be?

**Dhanji R. Prasanna** (01:00:13):
I think maybe two different things. One is just the power of Conway's Law, like we talked about before. It's like how difficult it is to change outcomes without changing the structure of relationships between people in an organization. And I think I always kind of knew that at some level, but really appreciating it in a visceral way is big. The other thing that I really learned the hard way maybe is you only hear about it when things are going wrong.

**Dhanji R. Prasanna** (01:00:49):
So when things are going well, you kind of have this eerie silence and you're like, "Well, am I doing the right things here? Am I focusing on the right problems?" So having a bit of judgment, having a bit of time to step back and look at things holistically, those are things that you really need to make time for and do on a regular basis, which I wish I had known when I took up the role.

**Lenny Rachitsky** (01:01:15):
Looking back at your time at Block, I keep trying to, I almost say Square because I'm so used to that over the air, but I know Block is the name of the broader company and Square is just one. Just so people understand, Square is one business unit, one product within Block.

**Dhanji R. Prasanna** (01:01:28):
Correct, yeah, we have Square, Afterpay, Cash App and TIDAL are four major brands, and then we also have Bitkey and Proto that are focused on Bitcoin for us and we chip hardware in those two brands.

**Lenny Rachitsky** (01:01:44):
Okay, great. I think that some people are like, what are you guys talking about? Okay, cool. So reflecting back on your time at Block, what's maybe the most counterintuitive lesson you've learned about building products or building teams that goes against what most people believe, say common startup wisdom?

**Dhanji R. Prasanna** (01:02:02):
I think code quality is one. Being an engineer. I learned this very early on and it keeps coming true over and over and over again. A lot of engineers think that code quality is important to building a successful product. The two have nothing to do with each other, but my favorite example is YouTube. I was working at Google around the time YouTube was acquired and I just remember there was this whole wash of angst about how horrible the YouTube code base is and how terrible their architecture is, and they're storing videos as blobs in MySQL and whatnot.

**Dhanji R. Prasanna** (01:02:41):
And you could argue that YouTube is the most successful product at Google by a long way, maybe more successful than many of their others combined. And so it really has very little to do with how well it was architected because the flip side of that Google video, which is product that I don't know if people remember, it existed before YouTube. It supported more formats, it supported higher resolution.

**Dhanji R. Prasanna** (01:03:11):
You could upload hour long videos, YouTube had none of this. It just had the one or two minute quick video thing and it's far and away, blown away its competition. And so I think just keeping that front and center is why are we building these tools or these apps or these products? They're for people to solve a specific problem. So in our case, it's for a square merchant to make a sale, to sell coffee to you or to sell something they've made. And that's really what's important.

**Dhanji R. Prasanna** (01:03:49):
It's not really important how well our Android platform performs unless it's serving that need. And so I think that's been a really hard one for me over my career. And I continually encounter engineers who think we need to refactor, we need to do this in a better way. And then I'm like, "No, all this code could be thrown away tomorrow. So just focus on what we're trying to build and whom we're trying to build for."

**Lenny Rachitsky** (01:04:19):
That is an incredible insight and lesson. This YouTube story is so fun and such a good example. You're saying they were storing the video content in a MySQL set like row and column as a blob data.

**Dhanji R. Prasanna** (01:04:34):
Yeah, this is what, I didn't actually look the code so I couldn't verify it, but this was the common wisdom. And then they had an entirely Python stack that was incredibly slow compared to the state-of-the-art C++ and Java servers that we had hyper-optimized at Google back in those days.

**Lenny Rachitsky** (01:04:57):
That is hilarious. It makes me think about also companies when you look inside a company, if you work at a company, you're just like, "This is just pure chaos. No one knows what's going on. This is just about to all fall apart." And that's basically what it's like at every successful hyper-growth company.

**Dhanji R. Prasanna** (01:05:14):
Yeah, there's some truth to that for sure. Yeah.

**Lenny Rachitsky** (01:05:17):
And so I think again, it's just there's so much more that is more important to the success of a business. And it's what you said is are you solving a real problem for people? Can you get in their hands? Can you continue solving real problems for them? It's not about the quality of the code, it's not how well you operate internally.

**Dhanji R. Prasanna** (01:05:32):
Absolutely. I think on Cash App we had that as well. So in the early days of Cash App, I was head of engineering from when we were about 10 engineers to 200 plus and took us to about 10 plus or 20 million users thereabouts. And there was a very similar thing there. From the outside it looked like everything was really chaotic. It's like people would build random experiments and ship them and it just didn't look like we were following strict policies on things like software life cycle and stuff like that, and it was kind of true.

**Dhanji R. Prasanna** (01:06:13):
And my philosophy was always, we have all these brilliant engineers and I'm going to do more harm than good by trying to harness them into very strict blinkered areas. If they want to spin their wheels building something that is a complete waste of time for a little bit. But at the same time, if they're delivering these amazing things on the flip side, then I'll almost allow that. I'll be okay with that.

**Dhanji R. Prasanna** (01:06:44):
And it's a fine balance because engineers can really go off and into rabbit holes if you let them. But yeah, there's a certain amount of creativity that chaos breeds and you have to know how to build controlled chaos in some ways. So you have to create a foundation that isn't liable to rupture. You have major liability problems or something like that, or you're going to lose money in our case. And so as long as those things are bedded down and you allow your engineers to have the freedom to experiment and iterate and do the things that energizes them, that's the ideal.

**Lenny Rachitsky** (01:07:26):
Speaking of controlled chaos, one of your titles during your time at Block, I guess this was while you were actually at Square, was Mad Scientist for four and a half years.

**Dhanji R. Prasanna** (01:07:38):
Yeah, that was a time when I was working part-time, mostly because I had very young kids with lots of additional needs and I was a consultant on various different projects and I was trying to help some wacky things get off the ground. And yeah, I'm really grateful to Block that they afforded me the freedom to have that role in my career as well.

**Lenny Rachitsky** (01:08:08):
Maybe one more question before I take us to Fail Corner, which I'll explain. So you've shared a few lessons of things you've learned over the course of your career. Are there any other, just let's say core leadership lessons that you've learned that you think have been important to you being successful at the work that you've done?

**Dhanji R. Prasanna** (01:08:26):
I think start small with everything. If you try to boil the ocean to make a cup of tea, I don't know who said that, but it's a really a useful phrase that I keep coming back to. You'll never get there. So if you're making a cup of tea, just make the cup of tea. You don't need to boil all the water that there is.

**Lenny Rachitsky** (01:08:47):
That sounds like really not delicious tea. Ocean water.

**Dhanji R. Prasanna** (01:08:52):
Yeah, I think there's another one of, I think Carl Sagan said, "If you want to make an apple pie from scratch, you have to first invent the universe." So it's like narrow your scope to the thing that's in front of you and that's achievable. And so that I think is really important and that's one of our core tenets and always has been even when we were just Square in the early days, start small.

**Lenny Rachitsky** (01:09:19):
Is there an example that maybe worked really well or maybe didn't work?

**Dhanji R. Prasanna** (01:09:23):
Yeah, Goose started small. It was just an engineer working on their own time trying to build something that was useful and that satisfied a thesis that they had. So Brad, our creator of Goose, believed very early on, I think long before we heard the buzzword going around that agents would be how we unlock value from LLMs. And he built a proof concept and he shared it with a bunch of people. He shared it with Databricks and Anthropic, got them excited and learned a lot from them.

**Dhanji R. Prasanna** (01:09:59):
And so it just sort of built momentum from there. And even internally, it was quite a similar thing. Cash App itself was like that and Cash App started more or less as a hack week sort of idea and grew into a bigger and bigger and bigger thing. So a lot of our projects start with these small experiments that we try to then build on top of. We became the very first company that was a public company to launch a Bitcoin product. And that was again a hack week idea that actually Jack and me and another engineer worked on.

**Lenny Rachitsky** (01:10:40):
That was the hackathon team? You and Jack Dorsey and an engineer?

**Dhanji R. Prasanna** (01:10:44):
Yeah, it was the three of us.

**Lenny Rachitsky** (01:10:46):
Unreal.

**Dhanji R. Prasanna** (01:10:48):
Yeah, and it was great. We went and bought a cup of coffee, a blue bottle, and it was bought using Bitcoin over cash card. And I'll tell you those in hindsight, probably the most expensive cup of coffee.

**Lenny Rachitsky** (01:11:02):
What was Bitcoin at? 20,000?

**Dhanji R. Prasanna** (01:11:02):
I think it was 6,000 or 7,000 back then. I don't know.

**Lenny Rachitsky** (01:11:07):
It's like 120,000 now. Great.

**Dhanji R. Prasanna** (01:11:12):
But yeah, it's an example of how you get to a working useful product to people if you focus on a small thing first in a build.

**Lenny Rachitsky** (01:11:22):
And just to double down on this counter too. "Okay, we have a big idea, we're just going to put a bunch of resources on it and go big immediately."

**Dhanji R. Prasanna** (01:11:29):
Yeah, absolutely. And I've been part of teams like that too. So in my career, I worked at Google on this product called Google Wave, which was trying to be everything to everyone and we were 70, 80 engineers building this thing before it even really had any users outside Google. And so I think that's an example of something that started big, tried to go big on day one and probably lacked some of that meeting the earth where reality lies and adapting accordingly.

**Lenny Rachitsky** (01:12:08):
I remember Google Wave. Absolutely. It was beautiful. A lot of hype. I don't remember what it was for specifically, but it looked really nice.

**Dhanji R. Prasanna** (01:12:15):
Yeah, a lot of learnings from that one for me. Yeah.

**Lenny Rachitsky** (01:12:19):
What else? Any other big lessons?

**Dhanji R. Prasanna** (01:12:21):
Those two are the big ones, but I would also say question base assumptions on everything. Sometimes we get into traps where we are as professionals, hyper focused on what we're building that day, that week, that month. And we don't stop to think should we even build this at all? Or what's the purpose of building this?

**Dhanji R. Prasanna** (01:12:46):
Could we build something completely different that would matter more to our core reason for being? So I would say, yeah, question the sort of base assumptions. It's somewhat of a cliche, but you really need to remind yourself to apply it over and over and over again.

**Lenny Rachitsky** (01:13:03):
I had a colleague of yours on the podcast back in the day, IO, who worked with you on Cash App.

**Dhanji R. Prasanna** (01:13:08):
Yeah.

**Lenny Rachitsky** (01:13:09):
He's a friend of mine, he's amazing. He had a quote along those lines of just like, I forget exactly what it was, but it was just get to the bare metal of the thing that you're working on, just touch the thing that you're building and go to the base of it to really understand what's going on. And I imagine that was really important with Building Cash App and Cash Card.

**Dhanji R. Prasanna** (01:13:26):
Yeah, IO is one of the best product people I've ever worked with and one of my closest friends actually. So absolutely with him, and you on that one, yeah.

**Lenny Rachitsky** (01:13:38):
Okay. I'm going to take us to a recurring segment on the podcast I call Fail Corner. You already shared one example of a product that failed that you worked on. I'm curious if there's another, and the question is just what's the product you worked on that did not work out? Because people listening to this hear all these amazing successful people come on the podcast, share all these stories of success, endless success, but they don't hear the stories when things don't work out. And so this question is just, "What's a product you worked on that didn't work out and what did that teach you?"

**Dhanji R. Prasanna** (01:14:08):
It's a very valuable point. My career has basically been a string of failed product on top of failed product. And I think that, "Yeah, the Google wave example's there." I worked for Hot Minute on Google+, which was another epic failure.

**Lenny Rachitsky** (01:14:23):
Good one.

**Dhanji R. Prasanna** (01:14:23):
I worked at this social networking startup called Secret, which burned hot for a bright minute and then blew up. And then there was an email startup that we did, and that was, again, very promising, and then that fizzled. So the co-founder of Canva and I worked on that one. So there's been a whole string of failures, but at each point, I think I learned something and I learned that I need to never make that class of failures or errors again.

**Dhanji R. Prasanna** (01:15:01):
And so Cash App was probably the big success for me that a product that I worked on that was very early on and grew to be this giant business and product that people love. So yeah, been my career is essentially taking the learnings from all these failures, getting some humility out of it in the process too, coming into things, willing to listen to other people's points of view, critical points of view, and not just thinking that I have all the answers, yeah.

**Lenny Rachitsky** (01:15:36):
And I bet all these products that failed had really beautiful code. A lot of really good architecture decisions were made. Some of them, some of them were awful in every way. So many reasons for it to fail. Incredible. Dhanji, is there anything else that you wanted to share or I don't know, double down on before we get to our very exciting lightning round?

**Dhanji R. Prasanna** (01:15:59):
I would say I think that we're in this era of a lot of change and people are scared or reticent or uncertain about where things are going. And I think that look at the things that matter to you. For us, it's open source, open protocols, improving access for everyone. I've been very lucky in my career to only work on products that are either free or almost free to anyone or they have a free tier and then you pay for some premium services and that are usable by everyone. So anyone can become a Square seller.

**Dhanji R. Prasanna** (01:16:42):
I remember even in the early days, people used it to pay each other as a peer-to-peer money transfer system and that's why we built Cash App and that was really successful on the back of that. So I think it's really look at the things that are important to you and optimize for them. It's not really that important that the technology trends are growing in a certain way because technology is here to serve us, and if we have an important reason for being and an important purpose, then we can make that technology serve us. And that's much more important than being deep with the technology or being at the forefront of every trend.

**Lenny Rachitsky** (01:17:27):
Such great advice when there's so much to pay attention to and so much happening. So stressful to feel like I'm just not aware of all the things. I'm not as good as all these people I'm seeing on social media, but what's happening with AI, I'm just so behind. What I'm hearing from you is just like what is actually important to you? And just do that. Don't feel like you need to be the best at everything that's happening on top of all the latest AI news.

**Dhanji R. Prasanna** (01:17:51):
Yeah, exactly. And if it's not meaningful and fun, then you shouldn't be doing it probably.

**Lenny Rachitsky** (01:17:58):
With that Dhanji, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

**Dhanji R. Prasanna** (01:18:03):
Okay. Shoot.

**Lenny Rachitsky** (01:18:04):
I see so many books behind you. So I love this first question. I'm excited to see what you pick. What are two or three books that you find yourself recommending most to other people?

**Dhanji R. Prasanna** (01:18:12):
Yeah, so I very much of the opinion that you shouldn't read books that are about your daily work or your professional life. I read fiction, I read the classics, I read poetry, philosophy, history. These are the books I really enjoy. And I think it expands your mind and gives you creative ideas and helps you question things about the human condition.

**Dhanji R. Prasanna** (01:18:39):
And that's much more valuable than some self-help book or some get good at being an engineering manager book. So yeah, having said that, the Master in Margarita by Mikhail Bulgakov is one that I really love. It's a masterpiece of Russian literature. And then I've always been drawn to Tennyson's poetry and I find that in the times when I'm most uncertain or grieving, Tennyson's poetry has always resonated with me and helped me find a center.

**Lenny Rachitsky** (01:19:16):
Wow. Never heard these recommendations before. I'm really excited to check these out. Very cool for a CTO of a big tech company. What is a favorite recent movie or TV show you've really enjoyed?

**Dhanji R. Prasanna** (01:19:30):
Alien Earth I think is pretty awesome. It's by Noah Hawley who did the Fargo TV series. And so it's someone with all of these incredible skills in high art filmmaking who's doing a pulp sci-fi show, and it just looks stunning and it feels stunning and it captures all of that essential alien pulpiness that makes it so interesting and fun. So I really like that, and I'm also watching Slow Horses, which I think is one of the better shows on TV.

**Lenny Rachitsky** (01:20:04):
Love Slow Horses. The new season's Out, think the fifth episode just dropped the day we're recording this. So I love that show. Alien Earth also just watched it, so creepy and just like all these slimy, gooey little creatures just crawling around.

**Dhanji R. Prasanna** (01:20:16):
Yeah, I just love the aesthetic and they captured something essential about the original Alien and yeah, they do it, but every scene in Alien Earth feels like you're watching a painting or something or someone's reading a novel to you. It's really unfolds very thoughtfully.

**Lenny Rachitsky** (01:20:36):
I've never watched any alien content in my life and I really enjoyed Alien Earth. I will say the ending, I was just like, it felt like it kind of slowed down a bit. I'm just like, "All right, I guess I see where it's going now." But it was really fun to watch. Okay, next question. Do you have a favorite product you really enjoyed? Sorry, a favorite product you've recently discovered that you really enjoy? It could be an app, could be an gadget, could be some kitchen thing.

**Dhanji R. Prasanna** (01:20:58):
Well, I'm a gamer. I love playing games. So for me it's the Steam Deck, the Steam Deck OLED, which is their latest version. It's like this gorgeous piece of hardware that lets you play the best games out there, but it's totally extensible and customizable. And in this era where we're constantly told by big tech companies that we need to lock everything down, we need to lock down the user experience and customizability in order to have things work for people.

**Dhanji R. Prasanna** (01:21:28):
I think Valve showed that's totally unnecessary and totally wrong, and you can build the Steam Deck. You can install competing app stores, you can install Windows on it. You can treat it like a computer, write programs, which I have done to run on it. So yeah, I think it's an incredible thing and it looks beautiful and it works great. So yeah, big fan.

**Lenny Rachitsky** (01:21:50):
Do you have a favorite life motto that you find yourself coming back to often in work or in life?

**Dhanji R. Prasanna** (01:21:55):
If you're not waking up in the morning feeling energized about what you're going to do that day in your professional life, then change something, quit if that's what it comes down to, or find a new way of doing what you're doing. Just don't accept what's meted out to you. So that's how I've tried to do things, and sometimes it works, sometimes not, but yeah, it's a good thing to ask yourself.

**Lenny Rachitsky** (01:22:25):
I really love this advice. It's really hard to do that for a lot of people. Is there anything that has helped you get over that fear of just like, "Oh man, I'm going to quit this thing. I don't know where I'm going to go next."

**Dhanji R. Prasanna** (01:22:36):
The main thing is telling yourself that a year from now, you're going to look back on what looks like a monumental problem, a life-changing thing, and you're going to be like, "Oh, that was so trivial." A lot of times we get into these traps where we're overthinking something or really nervous about making a change, but in hindsight, those don't seem that big.

**Dhanji R. Prasanna** (01:23:05):
And all the time that's passed since and all the events that have happened teach you that there's more to the world and it's never too late to do something useful or never too late to do something that's for yourself and improving yourself. So yeah, I think just kind of remembering that things are not as big or bleak or decisive as they seem in the moment is always important.

**Lenny Rachitsky** (01:23:30):
Final question. So you were a mad scientist at Square for many years. Do you have another favorite mad scientist from pop culture or real life?

**Dhanji R. Prasanna** (01:23:41):
That's an interesting one. I think the image that always comes to my mind is Doc Brown from Back to the Future. I feel like he's the canonical mad scientist of my generation anyway, but there've been a lot in video games and stuff too, but he was the one that was like, "I'm just going to do this crazy thing because I almost have this burning desire, need to do it, and whether I want to or not, I must build this time machine." And he spends the entire movie trying to fix the problems that it creates. But yeah, he has always been a really fun character for me.

**Lenny Rachitsky** (01:24:20):
You know what? I think about Pinky from Pinky in the Brain.

**Dhanji R. Prasanna** (01:24:24):
Oh yeah, that's a good one too. Yeah.

**Lenny Rachitsky** (01:24:27):
Oh man. Dhanji, this was awesome. You were wonderful. Thank you so much for being here. Two final questions before we actually wrap up. Where can folks find you online if they want to reach out, learn more about say Goose or anything else going on at Block? And how can listeners be useful to you?

**Dhanji R. Prasanna** (01:24:43):
Check out our GitHub pages for Goose and all of the other open source projects we have at Block. So there's a lot that's useful there. We do a lot on Android open source as well, so check that stuff out. You can always find me on LinkedIn, so feel free to connect. I'm very happy to be contacted.

**Dhanji R. Prasanna** (01:25:06):
And I would say the way people can be useful is, again, going back to this era we're in of a lot of change and uncertainty, I think people that demand more of their companies, of their employers, of their teams, demand something better. At Block, we always ask, "Can we default to making this open source? Can we build this for people that are not just us or our customers? Can everyone benefit?"

**Dhanji R. Prasanna** (01:25:37):
And I think that's particularly important in this era of AI where everyone's locking themselves in walled gardens and trying to capture parts of the platform that are emerging. So yeah, just demand more of people. The internet was created as a promise for open sharing of information to the benefit of all, and I think that AI should realize that for us. And so yeah, just demand that of people.

**Lenny Rachitsky** (01:26:07):
A really beautiful way to end it. Dhanji, thank you so much for being here.

**Dhanji R. Prasanna** (01:26:11):
Thank you, Lenny. I really appreciated it. Thank you.

**Lenny Rachitsky** (01:26:14):
I appreciate you.

**Dhanji R. Prasanna** (01:26:15):
Cheers.

**Lenny Rachitsky** (01:26:15):
Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [7/17] The woman behind Canva shares how she built a $42B company from nothing | Melanie Perkins

**Lenny Rachitsky** (00:00:00):
There's a very famous story about Canva. Early on, you pitched over a hundred investors and over a hundred investors said no to you.

**Melanie Perkins** (00:00:06):
It was really clear in my mind that it was the future and I thought the investors were wrong, frankly. But investors also gave really helpful feedback and feedback. Often in the form of rejection, they would say, "Oh, your market's not big enough," and I would say, "It's going to be huge." And I'd add a new page in my pitch deck that said how big the market I believe was, and then they'd say, "You're the same as some of other company." And I would say, "Hey, now I've got a new slide in my pitch deck that shows all the players and the huge gap in the market that we believe we're going to fill."

**Lenny Rachitsky** (00:00:33):
One of your values, Crazy Big Goals. I love that as a value.

**Melanie Perkins** (00:00:35):
The thing that I love about a crazy big goal is that you feel completely inadequate before it. You want to work really hard to will it into existence. I really like to start by just imagining what is the future that you actually want Right now? I have a wall in my house in my office, which is my vision for what I'd like the world to look like in 2050.

**Lenny Rachitsky** (00:00:52):
I heard from one of your team members, Melissa Tan, there's a deck like this for every project you kick off. There's this big vision deck.

**Melanie Perkins** (00:00:59):
So we have this concept of chaos to clarity. Every idea starts in the chaos side, and then you have to work all the way to the other side, which is clarity. That very first step at the far end of chaos was quite an embarrassing step actually, because you don't have mastery at that point. You don't have all the answers.

**Lenny Rachitsky** (00:01:14):
A lot of people think of Canva as like design graphics for social media and marketing and things like that, but you also have spreadsheets, whiteboards, charts, AI coding tool.

**Melanie Perkins** (00:01:24):
Was funny, looking back from really old decks. We were trying to do AI before AI was actually a thing.

**Lenny Rachitsky** (00:01:30):
Today my guest is Melanie Perkins, CEO and co-founder of Canva. Melanie is on track to be the most successful female tech founder in history and one of the most successful founders, period. Canva is currently valued at over $42 billion, making over $3.3 billion in revenue a year. They've been profitable for eight years straight and are one of the hottest private tech companies in the world right now. But it wasn't always this way. Melanie was rejected by over 100 investors when she was trying to raise her first round. Their team spent two years rewriting their entire code base and were unable to ship any new features for over two years, something they expected to just take six months, and they even went through a big pivot early on from a yearbook publishing platform to the Canva that you know today, Melanie does not do a lot of podcasts. She shares stories that I've never heard before and lessons that I'm still thinking about.

**Melanie Perkins** (00:04:53):
Thank you so much for having me. I'm excited to be here.

**Lenny Rachitsky** (00:04:55):
I'm even more excited. It's such an honor to have you here. I am such a fan of yours. I'm such a fan of the company that you've built. Also just everyone I meet from Canva is just so nice and so awesome and so smart, and so clearly you've built something really special. I'm really excited to use this hour to learn as much as I can from you about how you did that. We were actually chatting ahead of this about what would make the best use of this hour. I asked you what you believe has been the biggest factor in the success of Canva. You described something called building a Column B company and Column B thinking, I've never heard of this before, so let us start there. What is building Column B company, what does that mean?

**Melanie Perkins** (00:05:35):
Really great place to start. So I guess there's two ways of planning. The way that you can plan is you can dream of what is the perfect vision of the future, what future do you want to exist in, what would you like the world to look like? What would you like companies to look like? And then going from there, which is completely improbable, a completely crazy big dream, and then working hard to turn that into reality. And the alternate is, so just imagine you are building a castle on the hill and you're like, "What would be the most magical, wonderful, mythical experience?" And the other thing you can do is you can look at the bricks around you and you can say, "What can I do with these bricks? How high can I stack them? What can I do?" And I think most planning is often done by looking at the bricks and trying to stack them, and then you can create only so much. And so I guess the column B thinking is thinking about what is that magical wonderful future that you then want to invest years and decades of your life actually building? And so that's column A and column B in a nutshell.

**Lenny Rachitsky** (00:06:37):
So column A is the traditional just work from today's world. Column B is work from this dream reality and work backwards from how to achieve that.

**Melanie Perkins** (00:06:44):
Exactly that.

**Lenny Rachitsky** (00:06:45):
This is exactly the way actually Brian Chesky thought. I worked at Airbnb for a long time and it was always just, "Think about the world, the dream and then work backwards from that," so there's a lot clearly also worked out. So clearly this is an important lesson in the example of Canva, just what would've been column A for what Canva could have been and how did you think about the column B approach of what Canva in a dreamland could be?

**Melanie Perkins** (00:07:07):
So column A would've been nothing frankly, because the reality was when I was a university student with no company and no business or product or software experience, the reality would've been not very much. And so it was all column B, it was all thinking about the wild future that we wanted to create. Imagine what would be publishing in the future, what would communications look like in the future? And it seemed really impossible that it would stay on the desktop, it would stay really complicated. And it just seemed so apparent to me that in the future it was going to be completely different. Could I build that future? I had absolutely no idea, but the idea, it seemed completely likely, completely improbable that it wouldn't be the case that in the future, design would be online and collaborative and really simple. And so starting from that, we then took that concept and applied it to the school yearbook market in Australia with our first company Fusion Books. And then we applied it to Canva where we wanted to take it much, much bigger.

**Lenny Rachitsky** (00:08:08):
Let's talk about just how to actually go about building a Column B company. Say a founder is listening to this and they're just like, "Okay, I want to do this." What do they do? What are the steps? Where do you start?

**Melanie Perkins** (00:08:17):
I really like to start by just imagining what is the future that you actually want? What is the world that you want to live in? What is the future of transportation? What is the future of healthcare? What is the future that you want to live in and exist in? And for example, right now I have a wall in my house in my office, which is my vision for what I'd like the world to look like in 2050. And so it's not necessarily that you can bring that into existence or you can will that into existence, but just to start to get clearer on what you would like that world to be like. Would you like it to be more inclusive? Would you like it? For me, one of the things I desperately want is everyone on this planet to have their basic human needs met.

**Melanie Perkins** (00:08:56):
What are those things that you believe are so important that you would love to see exist in that future? And I think an exercise we often do is what is wild success for X, or what is wild success for Y? And then equally, what is terrible failure for those things? And you can apply that just to abstract thinking in different industries. You can do that, apply that. We do that for the whole company for different areas of the company. And I think just taking that very long timescale of 10 years and getting a really crisp idea of what you want, of what you don't want, that's sort of the first step. And I think a lot of people don't spend quite enough time imagining that. And then the next part is you don't want to just have this crazy big dream and then do nothing about turning it into reality.

**Melanie Perkins** (00:09:43):
You kind of want to have a ladder that goes all the way up to the moon, which is your crazy, wild vision. And then you want to have rungs that just work its way up step by step. And so you want to get that really clear picture of the future that you would like and then just take little step, after little step, after little step. And it doesn't matter how small that first step is or how seemingly inconsequential if it is working towards that future that you want to will into existence, then you'll keep on climbing up that ladder in the right direction.

**Lenny Rachitsky** (00:10:10):
I think a lot of people hearing this might feel like, "I do this. Yeah, I have a vision. I know where I'm going. I have this big idea." What do you think they might be missing about just what this actually means and why they're probably not thinking big enough, they're not making the time to think this.

**Melanie Perkins** (00:10:23):
I think it is easy to be discouraged by the two, because they're completely two complete odds. They're completely different ends of the spectrum. So one is dreaming about the future, not that you think you can will into existence, it's just the future that you want. And then the next part is taking the tiny step that might be extremely microscopic and it feels a little embarrassing to be like, "I want a future that is," whatever it might be. And then to take such a microscopic step because I think you often have the future in one side or don't spend much time thinking about that. You're just thinking about the bricks before you. And so I think it's also naturally we all get distracted by day-to-day. It's your email, your Slack, the things that are kind of in your face, the reality that lives around you every day that kind of pulls you into this moment right now.

**Melanie Perkins** (00:11:14):
And so I think actually just making time to spend thinking about that is probably one of the most critical pieces. Just literally dreaming what is wild success in 10 years, what is terrible failure in 10 years is a really great place to start, is just spending some time there. And then even if that is so big and so vast and so wild, having that very first step is so important because then you take that little tiny first step and then the next step then that compounds for us, it's been compounding over a decade as we continue to work towards that same mission and vision.

**Lenny Rachitsky** (00:11:49):
I want to hear how you operationalize this. I heard from one of your team members, Melissa Tan, that there's a deck like this for every project you kick off. There's this big vision deck. Talk about what that looks like, because I think that's where people are like, "Okay, how do I actually do this?" Talk about that deck.

**Melanie Perkins** (00:12:04):
So we have this concept of chaos to clarity and every idea starts in the chaos side, and then you have to work all the way to the other side, which is clarity. And so chaos can be an idea, it can be a problem, it can be a philosophy or a belief. And I've got a joke that I find funny. I'm not sure if you will, but how do you go from chaos to clarity? You add clarity. And so the idea is that each little step from chaos to clarity is the very first step might be literally writing it down. So rather than it being in your head, you've written it down, then the next step might be starting to create a pitch deck on it. And the next step might be starting to refine that, turning it into some designs, turning into a prototype. And then as it kind of goes from chaos to clarity, it starts to become more and more real and more and more people can see it. And so just taking those little incremental steps that adds clarity with every single step, then starts to help will it into existence rather than it being something that's completely amorphous and just stays in your head. So I think that's why visual communication for us is so important, is because otherwise, if it's just in your head, no one else can see it and you can't will it into existence.

**Lenny Rachitsky** (00:13:13):
This makes me think about this concept of an ugly baby from the book, Creativity Inc., by I think Ed Catmull where he talks about how new ideas or this ugly baby that nobody wants to look at and deal with that I think he says they want to kill. I don't know why you would do that with an ugly baby. But there are these very soft, fragile things and it's really important to not kill them early and give them a chance to survive. And it's kind what I'm hearing here is have this big vision that many people are be like, "No, wait. This is completely absurd." And I love this idea of, "Okay, but here's one step we could take there to see if this could be a thing."

**Melanie Perkins** (00:13:48):
Yeah, I completely agree. And I think that's the thing is that that very first step at the very far end of chaos, it's very embarrassing because you're like, "I have this idea that is so big and so wild and how the hell would I do that? I have no idea." And so it's quite a embarrassing step, actually, because you don't have mastery at that point. You don't have all the answers. In fact, you have likely none of the answers, but you just have the idea that you think would be cool. And ideally you get the idea that it would be so cool that you want to work really hard to will that into existence. And so I think one of the really key parts is not only just having the idea but thinking it's so cool that you're going to work for years to will it into existence. Actually Melissa did a really amazing pitch deck recently about the vision of, I won't go into the details right here, but the vision of her space and I was really excited about it. And so I think that that's the great thing about a pitch deck, is that other people can see your thinking and your thinking actually gets clarified as well as you go through that process.

**Lenny Rachitsky** (00:14:50):
I talked to Melissa, I talked to a bunch of other people that work at Canva, that have worked at Canva, and something else I heard along these lines is this phrase, Crazy Big Goals, which I think is one of your values. Crazy Big Goals. I love that as a value. Why is that so important? How does that fit into this? And just talk about the power of having Crazy Big Goals.

**Melanie Perkins** (00:15:08):
Yeah, so right from the start of Canva, it was truly a crazy-big goal. We're like, "We want to empower the world of design and take all these things that are super complicated and put them into one platform and make it accessible to the whole world. And we want to, rather than it be super expensive and unaffordable, we want to empower everyone everywhere in the world to design." So I mean, that was the epitome of a crazy big goal. And if we macro out even further, we've got this two-step plan, build one of the world's most valuable companies and do the most good we can do. So again, a rather crazy big goal. And I think the thing that I love about a crazy big goal is that you feel completely inadequate before it. Another crazy big goal. We'd love to see everyone's basic human needs met on the planet. Completely crazy big, that truly shouldn't be. It's kind of absolutely absurd that that's the case, but we can go into that later.

**Melanie Perkins** (00:16:02):
But I think with a crazy big goal, then you want to work really hard to will it into existence. And so if you start with a reasonable goal or a realistic goal, then you kind of get to it and you're like, "Oh cool, whatever." Or more importantly, if something happens, all the problems and roadblocks come along as they always do, then you're like, "Okay, I won't bother with that." And then you can just go and choose another course. And so a crazy big goal is both crazy big, but it's also something that you think is incredibly important that you actually want to will into existence because it is so much work to will a crazy big goal into existence. So it better be one that you want to actually achieve.

**Lenny Rachitsky** (00:16:45):
Is there a crazy big goal that you set that comes to mind maybe as a good example of, "This is what I'm talking about?" Maybe a product you launched or feature back in the day?

**Melanie Perkins** (00:16:53):
Yeah, I mean so many things. So we have our mission to empower the world to design and we break it down into mission pillars. So empower everyone to design anything with every ingredient in every language on every device, obviously a mouthful. But then what we do is we take successive goals every year towards this mission. And so for designing anything, we started off with social media and presentations and docs and websites and whiteboards and video. And so every year we're just launching more and more things to fulfill that part of the mission of empowering everyone to design anything. And then equally in every language we started in English and then it was Spanish and then it was 20 languages and then a hundred languages and then hard languages like Arabic and Hebrew and Urdu and right to left languages. And now we're in a hundred plus languages now and now we're really doubling down on the localization experience to make it feel truly local in every market around the world.

**Melanie Perkins** (00:17:47):
And so you can see how having these very big, audacious goals that you then just take a step after step towards helps then will it into existence or on every device. We started off obviously just with a web platform and then we launched our iPad app and then iPhone and then Android and then we spent years investing in cross-platform. So we have the same feature set across every device. And so you can see how these big amorphous things that seem very outlandish, you can then just will into existence after continual investment for a decade and it compounds over time.

**Lenny Rachitsky** (00:18:20):
I see how all this is starting to fade together. There's this big crazy ambitious vision and below there are the mission pillars that are feeding into this vision and then the Crazy Big Goals within each of those mission pillars to measure your progress towards all these components of the mission.

**Melanie Perkins** (00:18:32):
Exactly, exactly.

**Lenny Rachitsky** (00:18:34):
Okay, so with all that in mind, there's also this kind of trade-off you have to make of just how ambitious you get because oftentimes sometimes maybe you never miss the goal, but many times people miss these very ambitious goals. How do you just find that balance between ambitious crazy but doable enough where people don't get discouraged?

**Melanie Perkins** (00:18:51):
I think with a crazy big goal, the thing we have been really great at is achieving them. The timeframe that we achieve them on has not always been very reliable. We have certainly not been able to have pin dart. What's it?

**Lenny Rachitsky** (00:19:07):
Bullseye?

**Melanie Perkins** (00:19:08):
We have not been very accurate with timing, but it's really interesting. I look back at a 2021 vision deck that we made obviously in 2021, but it was about 2026 and it was fascinating to see how much we'd actually been able to achieve from that vision deck and how many things were currently still in flight. And so by having that, I thought in 2021, some of those things may have happened a little quicker, but over the last five years they've really been coming into reality. And so we might think things are going to take six months and they take a year. We might think things are going to take six months and they take two years. This has been the case. We might think something's actually going to take our entire lifetime as some of those really truly Crazy Big Goals. And in fact, I don't even know if we can ever achieve them frankly. But they're such an important goal that even if you make a little step in their direction, they're worthwhile nonetheless.

**Lenny Rachitsky** (00:20:01):
Whether you like him or not, Elon, this is similar to his, if you watch him, he sets these really ambitious goals and then often is far late on achieving them, but clearly it has worked out in achieving the crazy things that he's achieved. Something else I hear is you celebrate these goals in a really unique way. Talk about that.

**Melanie Perkins** (00:20:18):
So when we have these Crazy Big Goals, we also have couple them with really fun celebrations. And so we attempt to make the Crazy Big Goals happen in a moment in time. So then when we achieve them, we actually have a really fun moment because if you're just trying to plod towards the top of the mountain always and you never take a little moment to pat yourself on the back, it would feel a little arduous. And so over the years we had all sorts of fun little celebrations where we have smashed great plates and released doves and had a La Tomatina festival.

**Melanie Perkins** (00:20:51):
All sorts of fun things just to take a moment with the team to celebrate that huge achievement. And so I think that you want to celebrate what you want to really focus on and what you want everyone to take that moment. So when you achieve that crazy big goal when we launch in Spanish, when we hit a hundred languages and then the so forth across the company, taking that moment to actually pat yourself on the back and pat the teams on the back and say, "Hey, we did this thing. That thing that seemed really hard, we've now achieved." And the mission is often each of the different mission pillars, they're obviously a long area of investment that's going to take a long time to get to, but being able to celebrate each of the rungs on that on the way there I think is extraordinarily important. And then everyone works extraordinarily hard to bring them to life and then it gives everyone a little moment to feel proud of themselves.

**Lenny Rachitsky** (00:21:43):
It'd be very motivating to get to just break a bunch of plates. I really like that. Celebration strategy.

**Melanie Perkins** (00:21:47):
We need to bring that one back. We haven't done that one in a while.

**Lenny Rachitsky** (00:21:49):
That's a good one. I just love how concrete you're making this. So it's set these Crazy Big Goals, confine the component of the goals, set numbers there and then figure out the kind of steps you take to achieve these things. All fail is very easy. On the flip side of that, this is a segue. People hear these stories, they hear your story, they see Canva over the years and it's just this up into the right huge success story, one of the most successful companies in history. I imagine there have been many periods where things weren't going so great and when maybe things didn't look like they would work out. So let me just ask you this over the course of building Canva, once it started to click and started to feel like it was going to be a thing, was there a point where it started to again feel like, "Wow, maybe this may not work out. Maybe there's a huge setback that we may not get over?"

**Melanie Perkins** (00:22:35):
I think it's just a constant evolution. Every time the company doubles in size, pretty much all your systems break, all the things that were working don't work. A little example, in the early days we'd stand up and everyone would present their goals, what they're working on every day, every week. And then it kind of moved every month and then it was just taking too long because we've got so many people. And then it was sort of like we'd started doing these things called season openers and season openers were really fun where we got the entire company together, we talked about the goals that we'd achieved. And it was so funny because ahead of season openers, everyone would launch everything because they wanted to do it ahead of the season opener. And then we'd also set the goals for the coming cycle for the coming season at that point in time.

**Melanie Perkins** (00:23:17):
But then they started to become six hours long because we had so many people and so many teams. And so trying to find that right with the same philosophies of deep context for everyone with the same philosophies of the celebrations and the goals and trying to find that right flavor at every stage of scale is definitely hard. And so I think it's just a constant work in progress. Or back to your earlier point about timing of things, we were doing a front-end rewrite and we thought it would take about six months. It was really important because it was critical for cross-platform. It was critical for right to left. It was critical because we could only have five people working in our editor at any point in time because of the way the code base was structured and we thought it was going to take six months and then it took two years and it was two years of not shipping any product, two years of a product company not being able to ship product.

**Melanie Perkins** (00:24:14):
And that is such a core motivation for our team. He's shipping something, seeing great customer feedback and that kind of makes everyone feel happy and you've got momentum and it just felt like we're in a dark, dark tunnel that we could hardly see the end of the tunnel. And we didn't really know how long it was going to take because it just had to take as long as the tunnel was going to take. And it was a very hard time because other people would be launching this and that. And it was eventually we got out of that tunnel and it was extremely important that we did that work. We've now got two and a half thousand engineers and we're able to deliver amazing things that would've just been completely infeasible and simultaneous collaboration, so many things were baked into this. But yeah, it was not a fun period of time. A product company not shipping product is not really a recipe for fun.

**Lenny Rachitsky** (00:25:15):
For two years.

**Melanie Perkins** (00:25:16):
For two years.

**Lenny Rachitsky** (00:25:17):
I feel like every builder listening to this knows exactly what you've been through and maybe not on that scale in those stakes, but you start on something, "Oh yeah, it'll take a few weeks," and then a year later you're still working on it.

**Melanie Perkins** (00:25:27):
Totally.

**Lenny Rachitsky** (00:25:29):
Just was the mood really, I don't know, sad internally? Two years, that's a long time not to ship anything. Just what was it like internally during that period?

**Melanie Perkins** (00:25:39):
I think it was kind of everything internally. We made it into a bit of a game. We had this game board and I bought these little rubber ducky sort of bath toys and we had, so we all the little components represented as a bath toy on this board and there was all these stages of went launched in product. There was an emergency lane at the end of it was home and hosed. And we did these weekly stand ups where everyone would come in and talk where their bath toy was at and just, we tried to make it fun for the team. So it was partly fun and it was partly distressing as all of our investors were like, "Hey, that thing." So I think it was both things at the same time. It was bonding let's say.

**Lenny Rachitsky** (00:26:30):
Okay. Speaking of investors, speaking of other hard times, there's a very famous story about Canva. Early on you pitched over a hundred investors and over a hundred investors said no to you when you were just starting Canva. I think that's important. The investors than any founder I've talked to actually tries to pitch. It's impressive you tried that hard and went for so many pitches and finally got someone to take a bet. Now you are something like a $40 billion company making 3.3 billion ARR. I think there's something like $240 million monthly active users, one of the hottest private companies in the world. Just how does this feel?

**Melanie Perkins** (00:27:06):
I don't know. It was really clear in my mind that it was the future and I thought the investors were wrong, frankly. But investors also gave really helpful feedback and feedback often in the form of rejection. So they would say, "Oh, your market's not big enough." And I would say, "It's going to be huge." And I'd add a new page in my pitch deck that said how big the market I believe was. And then they'd say, "You're the same as some other company," coupled with rejection, and I would say, "Hey, now I've got a new slide in my pitch deck that shows all the players and the huge gap in the market that we believe we're going to fill." Or most investors just knew absolutely nothing about design or the industry that we're in. And so we then ended up with the first few slides saying, "Here's the lay of the land today. Here's the problem that we're going to solve."

**Melanie Perkins** (00:27:53):
And so while it was extraordinarily frustrating, their feedback made us stronger and made our pitch deck stronger. And it was sort of from that chaos to clarity. At the start it was this idea and then through the copious amounts of rejection, the pitch deck got stronger and more refined. So then when people, the first time I remember I spoke to someone for hours and they eventually got it. They were really committed to understanding what we were trying to do, but then not everyone has six hours to understand a concept. And so being able to take all the gems of wisdom from that conversation and have that understood really clearly in a really short period of time and have all of the reasons that people were rejecting us pre-answered in that initial pitch deck was really important. And I think that's probably one of the reasons why when I look back at our 2012 pitch deck, it's so valid and really still captures what we're doing today. And so I think that rejection in some ways makes you stronger if you can persist through.

**Lenny Rachitsky** (00:28:50):
Well, I think beyond that, I've never heard this part of the story. It's not just persisting, it's actually iterating and taking feedback that you're hearing to continue to evolve the pitch to a place where, "Okay, I finally get what you're doing." That is such a cool part of the story. How much of that vision and product changed throughout that journey versus just the way you pitched it and convinced people?

**Melanie Perkins** (00:29:10):
It was pretty consistent, but the way we articulated it changed greatly. And so for example, I wouldn't, in the early days, articulate the problem very much. And I went into, "Here's the cool solution." And so then the first few pages became very much more problem-based because if people don't understand the problem then they can't understand or care about your solution. And so there was a lot of refinement on the way it was articulated, but the actual vision itself I think was pretty consistent through.

**Lenny Rachitsky** (00:29:38):
Guessing a lot of founders ask for your advice on raising money, getting started, having gone through so much rejection early on. What's your general advice to folks that are having a hard time fundraising?

**Melanie Perkins** (00:29:48):
I mean, I can only go on my experience, but I think it's sort of the dark tunnel analogy. Or chaos to clarity, let's go with that one. It's a slightly friendlier analogy and I think just taking the rejection and turning into things that you can control. So I can control my pitch deck, I can control the number of people I'm speaking to and I just spoke to literally everyone. And I think that continuing to use it to refine it rather than taking it as a personal rejection, I think it's really important to think how can I improve? How can I help someone to understand it? Some people are never going to understand it. I remember pitching an investor that had the lean startup book behind them when I was pitching them and they were never going to like Canva. We were not the lean startup. That was not the way that we were approaching it whatsoever. So there's some people that are just going to never like you and that's okay. I think it's important to find some people that do what you're trying to say and trying to do and kind of finding your tribe, I think.

**Lenny Rachitsky** (00:30:47):
As an investor, this is really interesting to hear because it tells you there are companies like Canva there that everyone's turning down, a hundred investors passed on, that you might still be able to invest in.

**Lenny Rachitsky** (00:30:58):
Talking about your growth as a leader, say if you compare Melanie of today to Melanie of, I don't know, 12, 13 years ago when you were just starting Canva, what would you say is most different in terms of leadership?

**Melanie Perkins** (00:31:11):
I don't really know. Probably if you ask other people around me, they'd probably be more observant. But it is funny because there's some things that I think that I need to change and then I realize you go into it the same as some other company. And sometimes we even try that for a while and then we try that out and we're like... It didn't really work for us. And it's kind of building a house that you want every brick in the house to match. And if you go and try and take some bricks from someone else's house and stick it in your house, it's probably going to not look very matched. And so trying to find things that are authentic to us and are authentic to everything that's come before it, is just that constant, constant thing. And then each scale of each stage of scale of the company rather than going, taking someone else's bricks and trying to stick that in your house, trying to build the thing that's authentic.

**Melanie Perkins** (00:31:59):
So I think there's many things that are the same, but obviously the stage and scale and we're constantly having to give away hats. And so you kind of think about it in the very early days. We just a few of us, two of us in Fusion and then three of us in a little tiny group, and you kind of wear a hundred hats and then you have to be able to give away those hats to other people that can then do that way better than yourself. And so I'm sure there's been a few skills I guess I would've had to have developed over the last decade to be able to give away those hats. But yeah, I think there's a lot of things that we've had to do and double down on that was more authentic to the way we did it in the early days actually.

**Lenny Rachitsky** (00:32:37):
I love that story. I think again, if anyone working at a company that has gone through a lot of growth has experienced that when people from other companies come in and, "Here's how we did it at this company." Is there an example of some there that just like, "Here's something that this company brought in and people from this company thought we should do and we try it and didn't work?"

**Melanie Perkins** (00:32:54):
I won't go into specific examples, but so many times over. And I think that, I mean maybe that's probably, in answer your other question, we did things our way because that was the only way we knew. And there was many, many times over the years that we didn't have confidence in the way we were doing things and we were like, "Oh, they've done it from a big company that's bigger than our company. Let's go do that." And that hasn't always worked out so well for us. And so yeah, I think confidence in how we take what is authentic to us and do it at the next level of scale is a constant work in progress. It feels like, as I was saying before, systems break and need to be reinvented but also reimagined for that next layer of scale rather than going to try to get something off the shelf from another company.

**Lenny Rachitsky** (00:33:39):
I imagine it also helped that you were in Australia away from the Bay Area and where all these other big companies are at, just being able to do it your own way.

**Melanie Perkins** (00:33:48):
Yeah, very much so.

**Lenny Rachitsky** (00:33:50):
Did you know that I have a whole team that helps me with my podcast and with my newsletter? I want everyone on that team to be super happy and thrive in the roles. JustWorks knows that your employees are more than just your employees, they're your people. My team is spread out across Colorado, Australia, Nepal, West Africa and San Francisco. My life would be so incredibly complicated to hire people internationally to pay people on time and in their local currencies and to answer their HR questions 24/7. But with JustWorks, it's super easy whether you're setting up your own automated payroll, offering premium benefits or hiring internationally. JustWorks offer simple software and 24 7 human support from small business experts for you and your people. They do your human resources so that you can do right by your people, JustWorks, for your people.

**Lenny Rachitsky** (00:34:38):
Is there anything else that is a good example of how you did something pretty different from how other companies operate? Anything else that comes to mind as a fun example?

**Melanie Perkins** (00:34:48):
The goal driven structure, I think the things that we were talking about before. So the mission, actually breaking that down into the mission pillars, breaking those mission pillars down into the goals that we're then pursuing and then celebrating those goals when we do achieve them, I think is a deeply underloved way of building a company. Often people have a mission that's kind of on the wall somewhere and then what they're actually doing and the way they actually make money and the way people actually spend their time is in a very, very different direction from that original mission. And I think the magic is when you can bring those two things together and so you can have your mission, you can have your mission pillars that actually are helping to achieve that. And I think there's a real authenticity in that for customers as well, is that you are actually doing the thing that you promised you do, and it all ladders up together. It's certainly not an easy way to run a company, but I think that when you do get that formula, I think that there's a lot of authenticity with what you're saying you're doing, you're actually doing.

**Lenny Rachitsky** (00:35:47):
I want to come back to that. That's a whole really cool process. Do you have with closing the loop with customers, but something else I want to talk about while we're in this topic of growth over time. I saw you post something about how you had to realize they had to slow down and not just work, work, work like crazy. Talk about just that realization and why that ended up being so valuable.

**Melanie Perkins** (00:36:06):
So in the early days, I would just work seven days a week round the clock. In our very first company, we actually had printing presses because we were printing the yearbooks in my mom's house and then delivering them to schools around Australia. And in the early days of Canva, we certainly were working all weekend, all hours of the day. It was just constant. But when you've been doing this for a while, if you just keep working at that pace, I don't think it's good for anyone's health, mental health or anything else. So I think finding ways to continue, I still work extraordinarily hard, but to continue to have that balance in my day to day where I actually go to sleep, I find time to do things like going for walks or doing yoga, journaling I find extraordinarily helpful to make sure that I can always bring my best to everything that I'm doing.

**Lenny Rachitsky** (00:37:00):
It's easy to say that kind of stuff. It must be really hard to actually make time for that thing. Is there anything for those sorts of things, is there anything that you do that allows you to actually protect that time to actually do these things? Because as you said, there's a billion things that are just looking for your attention constantly.

**Melanie Perkins** (00:37:15):
I feel like I've developed some healthy habits over the years. I don't have emails on my phone and so when I shut my laptop, I actually tune out and then if there's a real issue, I'll get an emergency call or page. But I think trying to delineate I think is really important. So when I'm working, I'm all in and then when I'm not working, I'm all out. And actually giving that mental space I think is really important. I've spoken to a lot of founders that haven't quite found that and then do struggle with it. So when they're working every weekend it feels like the right thing, but then sometimes you can miss the forest from the trees when you're just working harder and harder, but maybe you're actually working on the wrong thing. And so I think being able to step away a little just to be able to get perspective is actually really beneficial.

**Lenny Rachitsky** (00:38:04):
I want to come back to this closing the loop process. Let's say that you have where you figure out what to build. A lot of your ideas come actually from the community. Talk about just that process and how many of your ideas actually came from your community.

**Melanie Perkins** (00:38:17):
Oh, it's one of my favorite things. We've been doing it for years now, and so we get more than a million requests from our community every year and we've got a whole incredible team that then tallies them, breaks them down, and then delivers them to all of our product teams and then those actually get closed. So this year we've closed more than 200 loops, but we know that each one of those things is going to be loved and needed by so many more people that don't bother to actually fill out the request form. So many things from gradient text, like little things like gradient text to really big things like our Sheets product. There's just been countless products. In the early days with our AI products, we didn't release them to teachers because we knew there was a lot of hesitancy for teachers using AI in the classroom.

**Melanie Perkins** (00:39:03):
And we got so many requests from teachers saying, "Can I please use this MagicWrite in the classroom?" And so with them we unlocked that and put on safety controls for teachers. And so it's just constant, actually. It's just part of our product process. I think there's two parts to product. One is building the future and towards the mission and the mission pillars as I was saying before. And the other is actually listening to our community and building what they want. And so I think that that's the two core pieces of product in my mind. And the closing the loop comes in so many different forms. There's the explicit asks, and then the other thing that we double down on all the time is user testing and watching people use it. And if people hesitate clicking a button or people don't quite understand how something works, it's amazing to me how you can find 10 random people on the internet and they can give such astute feedback that then is so representative for such a large number of people. I've personally run hundreds if not thousands of user tests myself and it's been deeply embedded in our product teams also.

**Lenny Rachitsky** (00:40:03):
Wow, that must be really stressful for someone looking at a test of Canva, trying to try something when you're in the room.

**Melanie Perkins** (00:40:08):
It's actually, we do it all online actually. I mean the ones I've run are typically online. So people are so much more frank I think when it's just them and their camera and they don't really... Yeah, they tell you really how it is.

**Lenny Rachitsky** (00:40:24):
Is there a tool or a kind of a process there that you find really helpful? I don't know if you want to name names of products or anything like that, but it's something that you find helpful or useful.

**Melanie Perkins** (00:40:31):
Yeah, we use a lot of UserTesting.com, find that super valuable.

**Lenny Rachitsky** (00:40:36):
All right, go user testing. Okay. Something else that I know is really important to you and also really unique to Canva is something that's called the two-step plan. You mentioned this earlier, I want to definitely talk about this. What is the two-step plan? Why is this so important to you?

**Melanie Perkins** (00:40:50):
Yeah, so when you were asking about Crazy Big Goals, I think this is our most macro, most crazy biggest goal. Step one, build one of the world's most valuable companies and step two, do the most good we can do. And in our early days I thought I'd do step one and then step two and realize that actually step one can fuel step two and step two can fuel step one. And so that's been a really big part of Canva for some years now. In the early days we took the 1% pledge, which I think is an incredible program. Every single person, every single company should take that where you give 1% of time, money, equity and profitability. And I think that's a really easy thing to do in the early days that then can compound greatly over time. We also knew that Canva's equity was obviously going to be a really key part of it.

**Melanie Perkins** (00:41:35):
So Cliff and I owned a little over 30% of Canva, and so we decided we were going to take 30% of Canva and use it to do the most good we can do. And we are doing that. So we're doing all of our donating through the Canva Foundation. We've just, over the last few years, we've donated $50 million to GiveDirectly, where they give money directly to people in Malawi who are in extreme poverty and then they can use that money on their family to go to school, to get healthcare, to start small businesses, to get a roof so they can sleep in without being wet. Just real truly basic human needs things. And we've just announced that we're going to be giving another a hundred million dollars over the next four years to people in extreme poverty. And it's just like when you go and sit with people and you hear about how they're spending, what's very microscopic amount at $550 doesn't buy us that much, but it's a life-changing amount of money for people in extreme poverty and it's truly transformational what it can do.

**Melanie Perkins** (00:42:39):
And you meet people and you hear their stories and it's truly the best money I could ever imagine spending. And that crazy big dream I was mentioning earlier of everyone having basic human needs met, it's so completely insane that isn't the case today. There's no specific reason why people don't have their basic human needs met on our planet, but we just haven't got to act together as humanity. And so that is a truly crazy, big dream. But back to the two-step plan. Step one, build one of the world's most valuable companies and step two do the most good we can do. And finding ways to do that at the same time I think is extraordinarily important.

**Lenny Rachitsky** (00:43:17):
That's incredible. It makes me think about, not to mention Elon again, but Elon's three-step strategy plan and it's like build better cars versus this is like, "Okay, solve all the problems of the world and make the world a better place." What a better master plan to compare. Something else about this that I love is a lot of companies have this, have something philanthropic going on with the company and it's like sitting in a doc on some page. It's part of their mission. It's not actually that big of a deal to them. What I hear from folks at Canva is something you talk about all the time. This is an actually core part of how you work and think and how you set goals and set vision and missions.

**Melanie Perkins** (00:43:53):
Yeah, I'm happy to hear that. I wouldn't do Canva if it wasn't going to have a positive impact on the world. For me, getting really rich is not a goal unto itself whatsoever. It's a means to an end and I've been very blessed to be able to do some work and that creates wealth that can then go and have people's basic human needs met. But they're working just as hard, but they don't have the opportunity. And even our education product, it's now used by a hundred million people each month and we are in most school districts and rolled out across countries and being able to bring quality education tools to every, and we give that away for free as well. Being able to help empower schools all around the world and we're going to be doubling down and doubling down into that product to bring quality education to all. I think it gives so much more meaning behind work. We've also, between our education product and our non-profit program where we also give away our paid product for free, we are giving away 1.5 billion of product a year now. And so the impact that that can have and the ripple effect of that I think is pretty great. And I think for all of us, it gives a lot more meaning to our work than, "Get Rich."

**Lenny Rachitsky** (00:45:07):
Okay. So speaking of product, coming back to that, you guys are launching something, maybe you've already launched it by the time this comes out, what I heard described as the biggest launching canvas history, no big deal, that's a high bar. Considering all the things you guys have launched, what are you launching? Why is it such a big deal?

**Melanie Perkins** (00:45:24):
We are extraordinarily excited about what we are launching. I guess the whole mission of Canva is to empower the world to design. And so what has been enabled by new technology with all of AI has been just really profound. Enabling people to take their idea and turn it into design a design and have as little friction between those two points. So we are doubling down radically on our video product and bring some incredible capabilities to our mobile and desktop platform. We are launching email, which has been one of our most hotly requested features from enterprise customers around the world and business customers around the world who want to be able to design with Canva's drag and drop ease and to be able to create an email. We are launching forms, we are launching, probably one of the most exciting things is the way we're embedding AI across the entire product suite.

**Melanie Perkins** (00:46:12):
And so you can actually use AI to design a presentation, a video, a email, a website. All of these things can actually now be done inside the core editor, inside the design tab, which is used by 170. It's used 170 million times a month. And then on our elements tab, which is used 900 million times a month, we are also embedding AI. So you can actually generate a video, you can generate a canvas code and you can generate photos all directly inside that platform. And then we're also launching comments as lots of our customers use Canvas to comment and collaborate. And now you can actually just tag at Canvas and you can collaborate. You can just say, "Hey, can you make this title shorter? Can you do this? Can you do that?" And it has all of the context of the design so in situ, you can actually just have a collaborator that can help get your work done. So we are pretty excited about all of this.

**Lenny Rachitsky** (00:47:12):
Amazing. Something I'm going to just let people know, I don't know if people know all this, how many products you all have now. I think a lot of people think of Canva as a design graphics for social media and marketing and things like that, but you also have spreadsheets, docs, whiteboards, charts, code, AI coding tool. And now what I'm hearing is email forums. There's probably a few other things I'm not thinking [inaudible 00:47:36]-

**Melanie Perkins** (00:47:35):
Yeah, truly design anything. We're literally living up to that.

**Lenny Rachitsky** (00:47:35):
Oh my god, it's happening.

**Melanie Perkins** (00:47:38):
100 million people design a presentation in Canva each month now and it's pretty fascinating to see that when you speak to, I saw a tweet some time ago. They were talking about how it's a generational thing that a certain generation uses Microsoft, a certain generation uses Google. Gen Z, the way they design a presentation is in Canva, but it's not just generational for those with other generational ilk, but it is been fascinating to see that come to life.

**Lenny Rachitsky** (00:48:11):
The email product, is that like a email client product or It's a design emails that you can then send through your products?

**Melanie Perkins** (00:48:17):
It is design emails, so you can design email, then you can take that code and you can pop it into any email platform that you use.

**Lenny Rachitsky** (00:48:24):
How do you think about products you're going to expand to, I know there's trade secrets here. You don't want to tell everyone where you're going next, but just how do you approach, here's where we're going next.

**Melanie Perkins** (00:48:31):
So our mission, empower world design, empower everyone to design anything with every ingredient in every language on every device, and just take those things very literally. So to literally design anything, to literally publish anywhere. And so we now print in 50 something countries around the world and you can get it printed and delivered to your house. And we plant it.

**Lenny Rachitsky** (00:48:54):
I actually did that, while you're on it.

**Melanie Perkins** (00:48:55):
Oh, awesome.

**Lenny Rachitsky** (00:48:59):
I wasn't planning this, but I had a print thing delivered to my house. It's so cool.

**Melanie Perkins** (00:49:02):
Oh, yeah. Exactly.

**Lenny Rachitsky** (00:49:04):
We have to go to a print shop in this freaking graphic and well here's a button. Let's click that.

**Melanie Perkins** (00:49:07):
Exactly. Just click print and it pops up beautifully packaged to your door.

**Lenny Rachitsky** (00:49:10):
I don't know how that works. Yeah, I don't know how you did that, but it worked. How cool.

**Melanie Perkins** (00:49:14):
But it's very cool. And so yeah, I guess literally bring these things to life. Oh, we're like launching 3D as well. So all of these things we will be bringing to life literally. And just picking off what is the most strategically important next thing to enable everyone to design anything, to enable everyone to publish anywhere. And we have been doing that for a decade and we'll continue to do that forever more using the latest technology to truly bring people's ideas to life.

**Lenny Rachitsky** (00:49:40):
Okay, so this is helpful. So if someone's like, "Oh, will Canva come for my space?" Are people designing thing that was design and also, what was it? Publish?

**Melanie Perkins** (00:49:49):
Publishing anywhere.

**Lenny Rachitsky** (00:49:50):
Okay. Publishing and designing. Okay. So if you're doing any designing or publishing, watch out.

**Melanie Perkins** (00:49:57):
From a macro perspective, there was creativity tools and productivity tools. And what Canva really does is we're literally smack bang in the middle of that Venn diagram of creativity and productivity, rather than making our customers have to make a choice between those two suites.

**Lenny Rachitsky** (00:50:13):
Something I wasn't planning on asking about, but I think it's on everyone's minds. There's always this Figma and Adobe and then there's Canva and there's kind of a bunch of places we could go with this. One is just at the beginning of the journey where a lot of founders try to figure out their wedge and their specific niche. Just how did you think about that? "Here's how we might have a chance to..." I know Figma wasn't even around back then, I don't think. Just how did you approach your early wedge of users?

**Melanie Perkins** (00:50:38):
One of the most important things that we did was we didn't really worry about competitors at all. We actually just saw where is there a gap in the market that we can uniquely fill, and what can we solve a problem, a core problem that people currently have today? And so with our first company that was yearbooks in Australia and there wasn't great tools and these yearbook coordinators got thrown in to have to design something and they'd have no design experience. And we spoke to every single customer. We gave them an over the phone tutorial, we understood all of their pain points, we got continuous customer feedback, and then we tried to iterate and improve.

**Melanie Perkins** (00:51:12):
And then when we were thinking about Canva, a few years into that, actually one of the schools said, "I love this product so much. Can I use it to design newsletters?" And they had all sorts of other things that they wanted to use it for, and we kind of looked around and were like, "Oh, there's still nothing on the market." This was a few years into it, that actually does the thing that we're doing, but for all these other things. And so it was much more like where is the gap in the market that people are currently having a pain point? And if you can solve that pain point really well and solve it in such a way that people actually want to pay for it because it is truly solving a real pain point that they have, I think that kind of sets it up for success rather than be a problem or a solution looking for a problem.

**Lenny Rachitsky** (00:51:53):
So what I'm hearing there is you didn't overthink, "Here's my CP, here's the wedge and the strategy of how we expand into this large thing." It's like, "Here's people with a problem that hasn't been solved in years that we keep seeing. Let's try to solve it."

**Melanie Perkins** (00:52:04):
Exactly that. And if you take that problem centered approach that helps people to achieve something they actually want to do in the real life, you're probably going to be at a reasonably good spot, especially if it maps to a larger market. That's a particularly great thing. If it only solves one person's problem, that might not be a great company going forward. But if a few people have that same problem... But I think that again, back to that big ladder and that first rung, I think it's better to solve a small number of people's problem really well than trying to solve a large number of people's problem. Not very well at all.

**Lenny Rachitsky** (00:52:39):
Something I can't not ask about is just how you think about AI in your product. You mentioned how you integrated or all through the product, just you guys are doing really good stuff with AI. A lot of companies are struggling to find something really that works great. Do you have just a philosophy of, "Here's how we integrate AI into Canva," where it ends up being really helpful and people love it?

**Melanie Perkins** (00:52:56):
Your question is actually the answer at the same time. I think being able to integrate it into the product where it actually helps people to get their work done where it genuinely helps them to achieve their goals, and then being really open to listening to your community and hearing what they're loving, what they're struggling with and refining from there I think is really, really important. Just because AI is all the rage and investors really like AI doesn't necessarily mean it should be front and center, but if it can genuinely help your customers to achieve their goals. So the thing that I was mentioning before, enabling people to communicate their ideas and have little friction between those two points, AI is just kind of naturally a very critical part of that equation for us. In fact, it was funny looking back from really old decks. We were trying to do AI before AI was actually a thing because it really was critical to what we were trying to do even in our 2012 deck. You can kind of imagine how AI very much fit into the equation because of exactly what we're trying to do.

**Lenny Rachitsky** (00:53:57):
I'm going to keep us on the AI thread and take us to AI Corner, which is a recurring segment on this podcast. So here's the question: what's a way you've found in your personal life and work life to use AI where it ends up being really helpful, something really interesting that people might find useful?

**Melanie Perkins** (00:54:11):
So many things. So AI is often the first. If I'm having an idea, it'll be a first place that I go and explore the idea. And now with Canva, and you can just tag Canva, I can say, "Give me more ideas of this," and it's shockingly great because it has all of the context from the design. It's actually integrated deeply into your workflow. Another really fun thing I do is an AI walk and it's when I just put my ear pods in and then I go for a walk and I just say everything on my mind and I use that to then kind of filter out my thoughts and figure out what are the things I need to action. And it kind of helps again, get out of the weeds and think about things from a more macro perspective rather than from the things that might be in my Slack messages or in my email. It just gives you that sort of helpful vantage point I find. So yeah, so many things.

**Lenny Rachitsky** (00:55:01):
For the Voice Note tool, is there a tool that you find useful for that?

**Melanie Perkins** (00:55:04):
Yeah, I might use Apple Notes or directly into Canva Docs, and then I actually just do the brain dump into Canva Docs and then just summarize them.

**Lenny Rachitsky** (00:55:16):
Got it. And so you just use native microphone-

**Melanie Perkins** (00:55:19):
Exactly.

**Lenny Rachitsky** (00:55:19):
... dictation sort of thing. Nothing fancy?

**Melanie Perkins** (00:55:20):
Yeah.

**Lenny Rachitsky** (00:55:20):
Okay.

**Melanie Perkins** (00:55:21):
Yeah, I like it.

**Lenny Rachitsky** (00:55:23):
This reminded me of, you mentioned earlier in our conversation you had this vision board that you said is for 2050. Is that right?

**Melanie Perkins** (00:55:30):
Yeah, that's right.

**Lenny Rachitsky** (00:55:31):
Can you share something from that vision board?

**Melanie Perkins** (00:55:35):
I'll tell you why the vision board came about because it's only been in recent months. I did feel like as humanity, we are on a bit of a freight train and that freight training is, I think if we take a lot of visions for a lot of different companies and a lot of things that are happening and you just fast-forward 50 years or you do 2050 and you say, "Are we in a safer world? Is the world the place that we want our kids to grow up in? Is this the humanity that we want?" I didn't feel that the train that we are headed on always feels great. In fact, it scared me quite greatly for a whole host of reasons. And so I sat with that feeling for a little and then I kind of got to work on my 2050 walls and back to the chaos to clarity.

**Melanie Perkins** (00:56:22):
The first thing was riding my 2050 wall and I've really been loving, I've got a whole on the 2050 wall, started with a lot of quotes. Everything good was once imagined and many other quotes were along those similar lines. And then rather than just being fearful of the things that I'm worried about for society and for humanity, I started to think what would the alternative be? What is that vision that I would love to see us have basic human needs for all global education being a basic human right that everyone experiences all the really important things that we want as humanity. And again, using vision and using imagination and just dreaming about the future. And I find it really fascinating in my day-to-day by literally having it beside me as I work every day, the little tiny decisions that can kind of help to angle towards that future that we want and can I help will any of that into existence? I honestly don't know.

**Melanie Perkins** (00:57:24):
But I feel like just by starting to write it down some little brainstorm exercise with a number of other people and starting to just etch out, how do we get closer and closer to that. On my vision of the future, it's community. It's the whole of humanity trying to dream bigger and to dream bigger goals. And then us actually rising to that occasion in the world. We don't want, I think loneliness is rife, purpose is gone. What we teach people in schools is pointless. And in my vision for 2050, it's none of those things. Communities are bound to fall. We all have deep purpose. And that deep purpose springs from having bigger dreams that we collectively go out and achieve. Something that we're doing at our Canva world to a keynote in two weeks time, which I think is going to be after this is released, we've been asking people what is one goal you'd like to see the world achieved in our lifetime?

**Melanie Perkins** (00:58:25):
And then people literally writing it down I think is pretty powerful. And then people sharing that with other people I think is pretty powerful. And then us actually figuring out how the hell do we turn? That reality that we all deeply desperately want into existence, I think is genuinely one of the biggest questions of our time. But then again, rather than trying to tackle that entire thing by yourself, how do you take that first tiny step that starts to see that in your own life, in your own family, in your own community? And I think that's where we'll get purpose from and I think that is one of the key answers to loneliness is actually working towards something bigger than yourself.

**Lenny Rachitsky** (00:59:02):
Wow. I really appreciate you sharing all that. I was thinking as you're talking, just considering how wildly successful Canva has been and just how ambitious that was when you started. I would not be at all surprised that this actually happens and that you achieve this very difficult vision.

**Melanie Perkins** (00:59:18):
It's not something that I alone can achieve. I think it has to be obviously a global collective effort because there's zero chance I can go and achieve basic human needs for all. But I think that I'd like to change that. I'd like to help change the mood. I'd like to help change the way we're thinking about things. I genuinely think we need to move course a little and decide not what are all the things that we... what's the freight train we're currently on, but what is it that we actually want? What do we want our societies to look like? What do we want the world to look like? Is it good enough that there's people, hundreds of millions of people that can't eat? What the hell? It just literally makes no sense.

**Lenny Rachitsky** (01:00:03):
A column B world, you might say.

**Melanie Perkins** (01:00:06):
Absolutely.

**Lenny Rachitsky** (01:00:07):
Melanie, this was incredible. Before we get to our very exciting lightning round, is there anything else that you wanted to share? Anything else you want to leave listeners with?

**Melanie Perkins** (01:00:17):
You have been extremely extensive. I don't think I've got anything else to add, frankly.

**Lenny Rachitsky** (01:00:20):
That's the goal. That's the goal. With that, we've reached our very exciting lightning round. Melanie, are you ready?

**Melanie Perkins** (01:00:25):
Let's go.

**Lenny Rachitsky** (01:00:27):
First question, what are two or three books that you find yourself recommending most to other people?

**Melanie Perkins** (01:00:31):
One of the books I love is The Power of Moments, and it talks, am I supposed to be really fast and not tell you about it?

**Lenny Rachitsky** (01:00:37):
It's all good.

**Melanie Perkins** (01:00:39):
Okay. Two books, the Power of Moments, and one of the books early on I read was Designing the Obvious, which I found very insightful.

**Lenny Rachitsky** (01:00:48):
I like they shifted to fast mode. You don't have to go superfast. All good. What is a favorite product you've recently discovered that you really love? Not Canva.

**Melanie Perkins** (01:00:56):
I love the Calm app. It is my daily companion. I use it to meditate. I use it to listen to music. I just find it very calming.

**Lenny Rachitsky** (01:01:05):
Okay. First question. I usually ask about movies and TV shows. I hear you don't watch a lot because you're so busy and have so much going on. So I'm going to try it, new question I haven't asked before. I'm curious where this goes. So excluding Canva, what's a product you'd love to work on someday, whether it's like an existing other company like, "Oh, I wish I could work on that thing," or just a new product you'd love to build maybe after the Canva chapter.

**Melanie Perkins** (01:01:27):
I feel like my Canva chapter's going to go on for a long time, so I don't know, because we've got-

**Lenny Rachitsky** (01:01:27):
On the side, on the side.

**Melanie Perkins** (01:01:33):
... decent plans side and we're pretty extensive.

**Lenny Rachitsky** (01:01:33):
Okay, maybe a company you'd love to fund. There we go.

**Melanie Perkins** (01:01:36):
I feel like there's a lot of opportunity to create global infrastructure that is truly empowering. And so as I look at my 2050 wall, I think there's a lot of things that are currently only exclusively available to a small number of people that should be available to everyone. And so the more that we can do to uplift the rising tide lifts all boats I think is a thing that's just so of such critical importance. And I think there is this weird belief that you can be fine and everyone else can be not fine and that's all cool. I don't think that's cool. I think everyone suffers in such a case. So I think more things that help everyone to rise.

**Lenny Rachitsky** (01:02:21):
Maybe along those lines, but maybe not, is there a life motto that you find yourself coming back to and work your own life?

**Melanie Perkins** (01:02:28):
There's a few. I love the quote, happiness is when what you think, what you say and what you do are in harmony. I feel like that's a constant aspiration. And then I've just been so obsessed lately with the idea that everything is led by imagination. That imagination is the very first step of that creative process. So everything is good because once imagined is a quote you're going to be seeing from Canva all the time now because it is true that if you don't imagine it, you can't will it into existence. And in fact, everything great that we experience in life was first imagined.

**Lenny Rachitsky** (01:03:00):
Wow. There's so much power to that one thought nugget there is just, there's all these tools now that can make building so much easier. You can just build anything you want, just describe it. But so many people are just like, I'm in the same boat. I'm just stuck. "What do I want?" I don't even don't know what I need. What should I build? And that's exactly what you're talking about there. Okay. Last question. So I saw it somewhere that you were an aspiring figure skater in your early years in high school, you had to wake up at 4:30 A.M. to practice. Is there something you learned from that period of your life that was helpful in building Canva?

**Melanie Perkins** (01:03:36):
So many things are quite directly applicable, falling down over and over again and getting up and trying again, the importance of hard work and determination. I think the falling down, it was quite literal in my figure skating in days, and maybe it was a little more metaphorical in today, but it is constant.

**Lenny Rachitsky** (01:03:57):
You're right. I wonder what that metaphor is for figure skating. I don't know. Anyway, Melanie, this was incredible. I am so thankful that you agreed to do this. Two final questions: where can folks find you if they want to maybe reach out, send you feedback on Canvas or join Canva, and how can listeners be useful to you?

**Melanie Perkins** (01:04:12):
Really great questions. So you can find me on LinkedIn. That's where I post the most. And you can go to and I can get the URL to give us your wishes and we want to hear them and we literally listen to them. It doesn't just go into a suggestion box. And then how can they be helpful? Use Canva, spread Canva, teach Canva. We're doing a Canva World Tour through October, which is probably going to be updated when this is posted.

**Melanie Perkins** (01:04:43):
Come to our events. We do events all around the world and we'd love to see you and to hear from you. And if you are in a company, starting a company, try and do the 1% pledge. Try and figure out your own version of the two-step plan and try and build products and in every decision that you make that actually makes the world that you want to live in. I think there's this kind of belief sometimes that the world is created by other people, but we all have a very active hand in creating the world that we live in. And every decision that you make for investors, every company that you fund, is that contributing the world to the world that you want to live in? Or is it creating the freight train that none of us want to be on?

**Lenny Rachitsky** (01:05:27):
I have to ask before I let you go. Are you going to have the rap dancers at the next Canva event?

**Melanie Perkins** (01:05:33):
You'll have to wait and see.

**Lenny Rachitsky** (01:05:34):
Okay. Melanie, thank you so much for being here.

**Melanie Perkins** (01:05:38):
Thank you so much, Lenny, for having me and your great well-researched questions.

**Lenny Rachitsky** (01:05:42):
Thank you. Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny'sPodcasts.com. See you in the next episode.

---

## [8/17] \"Sell the alpha, not the feature\": The enterprise sales playbook for $1M to $10M ARR | Jen Abel

**Lenny Rachitsky** (00:00:00):
I've always wanted to create a very tactical episode on how to do sales, especially with a focus on founder led sales.

**Jen Abel** (00:00:06):
A lot of early stage founders get tripped up as they're taking late stage sales advice. The founder is the product. You have studied. You have experienced something that most of the market hasn't even had a chance to maybe see or visualize yet.

**Lenny Rachitsky** (00:00:19):
A billion SaaS tools emailing me constantly about their product. How do you get someone to even want to talk to you and be open to learning about what you're doing?

**Jen Abel** (00:00:25):
So if you can focus the messaging in a way that speaks to something that is a bit of shock value or is counterintuitive, you'll get them to continue reading. When a problem is truly being felt by the market, people will get on a call, people will respond.

**Lenny Rachitsky** (00:00:39):
The next step I imagine is you're on the phone with them trying to convince them to actually care. What do you do there? How do you get them to engage further?

**Jen Abel** (00:00:45):
You need to be vulnerable. I would be very open and honest with where you are. Hey, I'm an early stage startup. We have a lot to learn. Can we kind of gain your insight into how this problem is manifesting on your side? Founder led sales is not about revenue on day one. It is about learning as fast as humanly possible to get to that pulse, so that you can earn the right to sell.

**Lenny Rachitsky** (00:01:12):
Today my guest is Jen Abel. Jen is the co-founder of Jellyfish, where her and her team help early stage founders learn how to sell, do early customer discovery, and set up a repeatable sales motion. Prior to Jellyfish, Jen was an enterprise sales director at the Muse and a general assembly, and she's obsessed with helping founders in the zero to one stage of their journey. In our conversation, we get extremely tactical and in the weeds on how to actually do sales as a founder. We talk through each step of the sales process, and Jen shares what you should be doing and saying at every step. We go through how to find your first set of leads, how to reach out to them, what to say in your message, how to structure your first sales call, how to get through procurement, and how to get that final signature.

**Jen Abel** (00:02:27):
Thanks, Lenny.

**Lenny Rachitsky** (00:02:28):
What I've always wanted to do is to create a very tactical episode on how to do sales, like how to actually have sales conversations, how to find leads, how to close deals. Especially with a focus on founder led sales, where founders are doing the early sales versus hiring salesperson. Which one, I know you're a huge advocate of and we'll talk about this. And two, you basically spent all your time working with founders and founding teams, helping them learn how to do sales, how to set up their go-to-market motions, how to scale teams, sales teams, how to hire sales people. So I'm very excited to have you here to create a very in the weeds, hands-on episode on how to do sales. How does that sound broadly?

**Jen Abel** (00:03:07):
That sounds awesome, and thank you so much for having me, Lenny. I mean, it's a true pleasure to be here with you.

**Lenny Rachitsky** (00:03:12):
It's my pleasure. Many people have recommended you come on this podcast. I'm excited we're doing this. Let me start with just this kind of question, our founder led sales. Maybe just briefly explain what that term means and then talk about why this is so important, why this is the way you recommend companies start doing sales?

**Jen Abel** (00:03:29):
Founder led sales is really that first milestone that a startup goes through on the commercial side, which is, how do I go out and get my first few customers? Some people might say zero to one, which is, how do I get my first million? Others might say, how do I go out and get my first 10 customers? It's all kind of in the same vein. And founder led sales is really, really, really important because in the very, very, very early days, when there is no brand equity, when there is no marketing engine running, when there is limited to no reference ability, the founder is the product, right? Because the product is still, could be abstract, could be an MVP or in it's really early formation. So the founder is the product. When people say, well, what does that even mean? It means that you are a subject matter expert on something highly specific.

**Jen Abel** (00:04:23):
You have studied, you have experienced something that most of the market hasn't even had a chance to maybe see or visualize yet. So you have this novel insight that you are building a business around. And it's that novel insight and the way you see the world that gets the market excited, in absence of a product. And that's actually happens even before you even show the product. So founder led sales is how do you bring the founders vision into the world and have the market, and understand what part of the market accepts it, and then what part of that vision are they accepting? So it's aligning the founders vision with the market reality.

**Lenny Rachitsky** (00:05:01):
**Jen Abel** (00:07:06):
Absolutely. I mean, it is the competitive advantage. I don't think people realize how much of a competitive advantage the founder led sales motion is, for three specific reasons. One is that the founder is the visionary. No one's going to be able to speak to it like they can. No salesperson, really no non-founder. The second is that they are the highest order in the hierarchy of the startup. It's the founder. So the market is excited to talk to a founder because they usually know something that the market doesn't know. And they want to learn and maybe they want to experience something differently. So being able to speak to the person that's running the vision, speak to the person that's crafted the vision.

**Jen Abel** (00:07:50):
And then that third piece is that the founder can see something budding in a conversation that a salesperson won't be able to see. And it's those budding moments that's where all the gold lives. And I don't think a lot of founders realize that. Their day one market vision is never the same vision that takes them in a product market fit. And I say that all the time. It's these little budding moments where it's like I was able to get to that budding insight because I went deeper and deeper and deeper and deeper through these calls and these conversations, to fully refine how we want to go out and sell it. And only a founder is going to be able to see those things.

**Lenny Rachitsky** (00:08:26):
I love this because this connects really well with an episode I just did with the CMO of Wiz. And the founders went through this exactly. They had 10 to 15 conversations a day for weeks, and then they're like realizing people keep telling us they like it, but no one's buying, no one's excited enough to want to actually move forward. And that was only happening because the founders were on these calls.

**Jen Abel** (00:08:49):
Yeah, I mean, the other thing too is that it becomes a game of telephone. It's like, hey, this salesperson says that no cares about this. What do you think the first thing the founder's going to say is? It's the salesperson. So much easier to say that than, is it me? Is it my vision? So it also kind of brings the accountability closer to the pulse where decisions are made.

**Lenny Rachitsky** (00:09:10):
It's interesting how similar this is to product building, where there's this idea that a founder top down can tell the team what to build, and it's this very waterfall cycle. When in reality the idea of the product and feature is just like step one, and then there's actually building it, testing it, talking to customers about it, that actually turns that initial seed of an idea to the real thing that people will want.

**Jen Abel** (00:09:33):
Totally.

**Lenny Rachitsky** (00:09:34):
Okay. So the catch 22, I don't know if that's the term here. The challenge is, okay, so great, founders should be doing sales. Most founders have never done sales. They just want to build product. They're mostly product people, design people. Sometimes there's founders that are salespeople. Rarely is that the case in my experience. So doing sales is very hard, not something that comes naturally to a lot of people. A lot of people are very afraid of it. So with that, let's get into how to actually get better at the sales process. I think a nice framework for how we can go through this is basically first let's talk about the sales cycle, the steps, the key steps of a sales cycle. And then we'll just go through each step and help people learn some tactics to get through each of those conversations.

**Jen Abel** (00:09:35):
Sounds great.

**Lenny Rachitsky** (00:10:22):
Okay. Yeah. Cool. So what's the simplest way to think about the steps of a sales cycle?

**Jen Abel** (00:10:26):
The traditional sales stages that most CRMs are even set up as is you have your intro call. You have your second call, which sometimes could be the demo depending on the stage of market you're selling to. Then you have your third call, which is going to be more about walking through a proposal, a scope of work, maybe going deeper into the demo to contextualize it to everything you've learned. The fourth call is going to be getting their feedback and kind of co-authoring that scope of work even further. The fifth call is going to be around probably an introduction to procurement. And then selling into procurement in itself, it's its own little sales cycle, but we could talk about that in a sec. And then post-procurement, it's going to be obviously getting that signature and knowing who the actual signatory is. Sometimes it's not even the business unit, it's sometimes legal, CFO, procurement themselves. So just understanding who that is.

**Lenny Rachitsky** (00:11:23):
Okay, great. So let's walk through each of these steps. So I love that each call has its traditional goal, demo, proposal, then co-authoring, and then procurement, and then post-procurement. I love how consistent everybody is.

**Jen Abel** (00:11:39):
Yeah, well, it's also, it's interesting because it's also predicated on their buying process as well. So if they're really used to buying, if they're very mature in their buying process, they actually might guide you to what the steps are. But most startups are turning non-consumers into consumers. So that's why this kind of fits well.

**Lenny Rachitsky** (00:12:00):
Got it. So there's this intra call step. So kind starting about where it begins. I think where a lot of people struggle most is getting anyone to even pay attention. You don't want to talk to them. A billion SaaS tools emailing me constantly about their product. So maybe we start there of just like how do you get someone to even want to talk to you and be open to learning about what you're doing?

**Jen Abel** (00:12:19):
Yeah. And this is why that founder led sales piece is so important. One is when it's coming from the founder, it's an entirely different weight. You're like, oh, interesting. This founder is reaching out to me. Okay, I'm going to seek to go a layer deeper, knowing who is sending me this note. The second piece is this is why that novel insight, that technical insight, that business model insight that you've uncovered needs to lead here. People are inspired by a new way of thinking, usually something counterintuitive, something that's really different. I really try and stay away from better, because that's really hard to define, and better means something different to everyone. So if you can focus the messaging in a way that speaks to something that has a bit of shock value, or is counterintuitive, you'll get them to continue reading. So first and foremost, I usually like to open it with, which is why is this relevant for me in my role?

**Jen Abel** (00:13:20):
Why are you reaching out to me? So first and foremost is relevancy. I think that matters even more so than personalization right now. I think it's so easy to personalize anything, and it can also come across as really stressed when you're like, hey, I noticed you were on such and such podcast, and that was when they were previously two roles behind. So I always say relevancy. If you can get to relevancy, that's the most important piece. The second most important piece is really getting to that level of differentiation or counterintuitive nature. So say something that would literally make them say what? Or that makes no sense. Maybe not that makes no sense, but what or how could that be? Or I've never thought about it that way. Or I actually don't fully understand what they're saying, but there's something there that's interesting. Get them to pause for a minute.

**Jen Abel** (00:14:13):
And the most important piece is getting this done where, if they get it on their mobile phone, which everyone is looking at their mobile phone and on email, they don't have to scroll. So usually three to four sentences max. That also, that's how a founder writes. That's how an executive writes if you're selling top down. But most importantly, leave them wanting more. Don't say everything, don't even talk about the solution. Talk about the problem that you want to solve, and why it needs to be solved or why it's not good enough today. So those are the four main components. Just to reiterate, relevancy, bring some level of counterintuitive or really different approach to the conversation, focusing in on a problem that's predicated to them, and really concise.

**Lenny Rachitsky** (00:14:59):
I love this. Is there an example you could share that helps make this even more real? Maybe an email you helped with. And by the way, this is email and LinkedIn, I imagine is where you're communicating mostly?

**Jen Abel** (00:15:11):
LinkedIn, email, you would be shocked to hear this, but cold calling. I never used to do this. It is one of the most... The interest rates on cold calling are a lot higher than email in many cases.

**Lenny Rachitsky** (00:15:26):
So it's calling, email, cold email and LinkedIn DMs, is that the three channels generally?

**Jen Abel** (00:15:31):
Yep, those are the three, yeah, the main ones.

**Lenny Rachitsky** (00:15:33):
Cool. Yeah. Is there an example? It doesn't have to be word for word which you've done or people done, but just how do you... Yeah.

**Jen Abel** (00:15:38):
Actually our first three years were built on cold email before we actually had clients, before we got word of mouth from the founders we worked with, I cold emailed the first 20 customers we had at Jellyfish. And the line I used in there was zero to one sales talent doesn't exist. That's why I want to have a conversation with you. So that was kind of that leading, wait, what? What does that even mean? And what I would do is tie that to them in some respects. If they recently raised the seed or series A, where they are in that journey. But I would lead with that and then tie it into like, hey, I noticed you're looking to target X, Y, and Z based off of what your website. I'd love to have a conversation with you.

**Lenny Rachitsky** (00:16:25):
And that first piece is the relevance. Here's why this is relevant to you.

**Jen Abel** (00:16:28):
Yeah. And then I say, our belief at Jellyfish is zero to one sales talent doesn't exist. That's why we built this model.

**Lenny Rachitsky** (00:16:34):
Awesome. And you kept it really short. So it's relevance, counterintuitive, keep it really short. And then what was the fourth piece?

**Jen Abel** (00:16:42):
The fourth piece is just make sure it's concise so that you don't have to scroll on mobile phone.

**Lenny Rachitsky** (00:16:47):
While we're on this topic. What's a good conversion rate of these sorts of cold outreaches?

**Jen Abel** (00:16:51):
Conversion rate is interesting. Because everyone's so focused on conversion rate. I get that in the beginning until you have some wins, you have to focus there. But I just want to reverse engineer it backwards a bit. Once you kind of have this, and I'll talk about it in a sec, which is win rate. Win rate is if I got Lenny on a phone call, and he went through the sales process, and I signed him up. And I spoke to 10 Lenny's and only Lenny R was the one that signed. That's one out of 10, that's a 10% win rate.

**Jen Abel** (00:17:23):
So if you have a really high win rate, let's say 30 or 40%, you actually don't need as high of a conversion rate on the outbound, because you know if you get someone in the funnel, the throughput is very healthy. If your win rate is really low, you need that conversion rate to be much, much higher, because you need more and more and more at bats. So conversion rate is a funny thing. In the beginning I get it because you don't have a lot of customers going through the pipeline yet. But there's people focus on it in a little bit of a false sense because interest rate truly is also dictated by win rate. So I just want to specify that.

**Lenny Rachitsky** (00:18:03):
That's a really good point.

**Jen Abel** (00:18:05):
I would say if you're doing a win rate around 15 to 30% and you need to carry, oh I don't know, a million dollar quota. I would say a healthy conversion rate from outbound probably sits around, when it's mature, around five to 7%. Sometimes it could be two to 3%. Sometimes I've seen eight to 15% because it's coming from the founder, and they're solving a really heartfelt problem. So interest rate is also predicated on the problem that the founder has decided they want to solve for, and their insight on that problem.

**Jen Abel** (00:18:51):
And everyone wants to keep going back to, well, email rate is really low, email doesn't work anymore. LinkedIn, I'm not getting the connects I want. Yes, you can argue that the game has changed a little bit, but when a problem is truly being felt by the market, people will get on a call. People will respond. And if I can compare two engagements at a time that we're running, I see one where it's a 2% interested rate and then I'm seeing another one that's actually at 12% interested rate. And we're doing nothing different. The only thing that's different is the insight that the founder has on the market, which is a hard pill to swallow. And I know that's hard to hear too.

**Lenny Rachitsky** (00:19:32):
And those are two different companies selling different things?

**Jen Abel** (00:19:34):
Two different companies selling to two different markets, but they're not fundamentally drastically different markets.

**Lenny Rachitsky** (00:19:43):
Got it. But one's basically got more product market fit essentially?

**Jen Abel** (00:19:46):
That's exactly right. So everyone always comes back to we have the sales problem, we have the sales problem, we have an outbound problem. And yes, there could be some technical things going on that you need to look at. Maybe you leveraged clay and you blasted a thousand people and now your domain health has been impacted. But the vast majority of the time, maybe you're just not saying anything interesting at all. Maybe the problem you're looking to solve just isn't widely felt. Maybe the perspective you have needs another level of refinement because you're just not getting those bites.

**Lenny Rachitsky** (00:20:21):
I feel like this question you just touched on is something a lot of founders are always wondering, is this product not the thing people want? Is it my sales skills? I guess I imagine this is a very difficult question to answer, especially quickly. I guess is there anything there that's more likely because of this, it means your product isn't something anyone's want, versus you're not doing a good job selling it?

**Jen Abel** (00:20:41):
Remember that amazing chart you drew where it talked about the length of time it took? I think you picked the top 25 startups.

**Lenny Rachitsky** (00:20:50):
For product market fit timeline [inaudible 00:20:51].

**Jen Abel** (00:20:51):
Product market fit. Yeah, that amazing image that I think has been reshared more times than anything I've ever seen. It's interesting because if you look at, I was looking at this closely the other day. If you look at the top section where the time to product market fit is consolidated, and you go all the way down to the air tables and the figmas, which a long time to get to product market fit. If you think about the earlier ones, which were, it was like GitHub that had product market fit pretty quickly. The SOC 2 compliant company.

**Lenny Rachitsky** (00:21:22):
Vanta.

**Jen Abel** (00:21:23):
Vanta. There's this thing in my head and I haven't fully validated it but I'm going to share it with you, which is did they start with the market problem first and then build the product? Because they knew who their market was right off the bat. Versus an air table and a figma that I think started with a technical insight and then were trying to find their market.

**Lenny Rachitsky** (00:21:46):
I think that's absolutely right. Vanta, the way they approached it, Christina, she was searching for a pain. She was obsessed with talking to everybody about a pain that they needed solved, and then she just built it in spreadsheets. So she started with the market very much so.

**Jen Abel** (00:22:00):
And I think product market fit when you start with the market first, it's accelerated. But I will say this, I think that it's also capped on the upside. Because you're starting with the market versus the air table and the figma, which started with the technical insight and has uncapped upside. But one is certainly riskier than the other. Starting with a product is a hell of a lot riskier. And this is where I come back to so many people will say, how did you get funding and not know who your market is? And it's like, because if they do find it, that's a really, really, really big win. Versus I think if you start with the market first, you could potentially get a good win but is it more capped?

**Lenny Rachitsky** (00:22:47):
Interesting.

**Jen Abel** (00:22:48):
I don't know, it's just a thought. I was staring at your grid for so long, I was like, there's something here. And I kept coming back to those were more like I kept breaking it down to market versus product.

**Lenny Rachitsky** (00:23:01):
Yeah, I love this. I think there's definitely something there. There's also horizontal versus a very specific problem you're solving.

**Jen Abel** (00:23:06):
Totally.

**Lenny Rachitsky** (00:23:07):
Okay, so you mentioned clay and you kind of nerd sniped me there of just what tools do you find useful? And it makes me think about finding leads, which is kind of going backwards through the sales process, but it might be okay, so maybe let's spend time there. Just like what do you recommend to founders to find the good leads? Glengarry Glen Ross good leads.

**Jen Abel** (00:23:28):
So I say before you buy any tool, don't even think about tools. I feel like people turn everything into this complicated mess with all of the tools that they start integrating. And then they're engineers, I get it. So now they start to engineer these sales tools together and then they blast their market, and now they have 5,000 notes out in one day and their domain takes a hit. So I think before you even overthink about the tools right now, can you manually find 30 people that you want to spend 15 to 20 minutes writing a rock solid note to? Are there 30 people that you are deeply excited to learn from, that you are willing to invest 15 to 20 minutes to write a really thoughtful note? Let's just start there. So some questions. One, are they even discoverable? How hard is it to find 30 people?

**Jen Abel** (00:24:21):
What have you noticed across those 30 people? Are there any interesting insights? Maybe it's the size of the team they work on, maybe it's the industry they're in, maybe it's the length of time they've been in their role, maybe it's their previous roles in their career. Are there some commonalities? There's always some level of a commonality usually in most cases. Okay, so now you're starting to collect some parameters. Just by doing this exercise, you're starting to collect some parameters around who you want to learn from and sell to. Now if you send out those 30 notes and you wrote them specifically, and you spend good quality of time on it. And you've hit them on email, you've hit them on LinkedIn, maybe even tried to call them, maybe you've sent them a Twitter DM, pulled out all the stops. How many people respond? 1, 5, 0?

**Jen Abel** (00:25:15):
If it's zero, it begs the question, do you now want to do another 30 in a very similar fashion? Do you want to change the messaging a little bit? Now it forces these natural experiments. Or do you want to maybe go after a different role? So it forces you to answer these questions in a very manual way before you even think about integrating and spending time on tools. And then you get to a point where it's like, okay, now I'm starting to get some learnings and now I've realized, okay, it's this group of people, I've spoken to maybe two or three, now I want to go out and actually build a campaign to now talk to the next 10 to 15 people in that group. Truncate this as much as you can because I think people focus so much on volume in the beginning.

**Jen Abel** (00:26:06):
Even if you're selling down market, mid market, enterprise, volume comes once you've identified it and know the parameters. Parameters then allow you to do really strong enrichment with a clay. You now know the types of questions to give it. If you're not asking it the right questions, these enrichment tools, these sales tools will yield nothing. So it all comes back to even before you start your sales call, you need to make sure you're asking the right questions with am I even engaging, or do the people that I want to learn from or sell to, am I even messaging them in the right way? Are they even the right person? And the only way you can do that is by actually truncating these little experiments.

**Lenny Rachitsky** (00:26:49):
I love how tactical that is. And what I love is even one of your nuances you mentioned is can you even find them? This gives you a signal of how are... You're going to have to have hundreds of these, thousands of these eventually. If 30 is hard or impossible, you're in trouble already.

**Jen Abel** (00:27:06):
Totally. And it was funny, I was talking to someone today that said they had a target segment of high net worth individuals that they wanted to go out and serve. I'm like, how do you find those people? I guess when you get to a certain level, your net worth may be public. I don't know. That stuff's very hard to have certain parameters that are undiscoverable. And then if they're undiscoverable and it requires you getting on a call to understand if they are a fit, that does impact conversion rates. Not necessarily in a bad way, but now you need to kind of bring that back into the math equation, which is like, okay, if I speak to 10 people, I know two out of these 10 will be qualified. But I don't know which of those two out of 10. Well, that means you probably need more of those numbers.

**Lenny Rachitsky** (00:27:52):
So to quickly recap a few of the key things that I've written down as we're talking. So if you're just trying to figure out who to reach out to, make a list of 30 potential prospects that you think are good fits, that would be excited about what you're building. Spend 15 to 30 minutes writing them each an email. Do these emails have to be really short the way you described previously?

**Jen Abel** (00:28:12):
I would say the shorter the better, but would you respond to that? If you got this email, would you respond it?

**Lenny Rachitsky** (00:28:17):
Got it.

**Jen Abel** (00:28:17):
One of a great little tactical test is on Gmail, you can highlight the message and then have it replay it back to you from an audio perspective.

**Lenny Rachitsky** (00:28:28):
Oh wow.

**Jen Abel** (00:28:28):
You'd be shocked how many notes I've changed when it replays it back and I'm like, oh, that sounds really passive-aggressive.

**Lenny Rachitsky** (00:28:34):
Interesting. And so you also shared the structure for how to reach out to folks. So let me just share that again. And that applies to this initial email too, but it sounds like as you automate, you want to make it more precise and focused and not... Basically they're not as customized as you start reaching out to more folks.

**Jen Abel** (00:28:53):
Totally.

**Lenny Rachitsky** (00:28:54):
So start with something, here's why this is relevant to you. You're looking for salespeople in this market. Here's something that's unexpected or surprised, like zero to one salespeople don't exist. Keep it short and then I always forget the last one.

**Jen Abel** (00:29:14):
Focus on the problem.

**Lenny Rachitsky** (00:29:16):
Focus on the problem.

**Jen Abel** (00:29:17):
You don't even need to talk about the solution. That's the big takeaway, which is if you get any sales email, you get what's the primary focus of that email? It's usually like here's what we do. But I bet if someone reached out to you with a novel insight and said, I'm really passionate about solving this problem, if it's something you're focused on, you'd probably reply.

**Lenny Rachitsky** (00:29:39):
If it's a big problem. If it's like, oh yeah, you're so right, I need this.

**Jen Abel** (00:29:42):
The beauty with the American market, and I say this because we do a little bit of work with international startups too, is if something doesn't feel right, people love to complain about it. And it's like use that to your advantage. Get that intel.

**Lenny Rachitsky** (00:29:54):
A note that I just looked at, that I wrote down, that I think is very important is avoid using "this is better" as your pitch.

**Jen Abel** (00:30:01):
Yeah, yep.

**Lenny Rachitsky** (00:30:02):
Different. Here's something shocking about what we're doing. Here's something that'll surprise you.

**Jen Abel** (00:30:07):
Yeah, or counterintuitive. Exactly.

**Lenny Rachitsky** (00:30:08):
Counterintuitive.

**Jen Abel** (00:30:09):
Yeah, it's interesting. I spoke to a good friend that leads procurement at a massive organization. And he said to me, he goes, "One of the worst things someone can say to me is we're better than X product. Then I ask them to define that. And then I ask them, okay, how do we measure that? And then I say, okay, should we give this company another year to give them this feedback before we make this huge transition and disrupt momentum?" Yeah, better is a dangerous place.

**Lenny Rachitsky** (00:30:40):
I invest in a bunch of startups and I find it's impossible to get anyone to care if things are good enough with what they got today. I'm happy not using the best possible product if my life is okay. And I have so many other things I got to do, I'll just deal with this good enough solution for now.

**Jen Abel** (00:30:57):
Absolutely.

**Lenny Rachitsky** (00:30:59):
Okay, amazing. So we talked about how to figure out who to go after, how to pitch them to get them to want to talk. The next step I imagine is you're on the phone with them trying to convince them to actually care. What do you do there? How do you get them to engage further?

**Jen Abel** (00:31:15):
One is you need to be vulnerable. You're an early stage startup, okay. The market, we're at a point now where the market is smart. I always assume that the buyer I'm speaking to is highly educated and knows way more than I do. So just have that perception because what I've learned is a lot of the market will play dumb, and you can get yourself caught pretty quickly. So when you speak to them, I would be very open and honest with where you are. Hey, I'm an early stage startup. We are deeply passionate about solving this specific problem. We have a lot to learn. Here's how we are thinking about it from a problem priority perspective. Can we gain your insight into how this problem is manifesting on your side? And then let them open up. Now you have them one, talking about the problem. Now you're getting the intel about their perception of the problem, and is it even a problem? When you say you're early stage and there's still a lot to be done, it is easier to be honest.

**Jen Abel** (00:32:20):
If you tell them you have a fully baked, ready to go product, they're not going to give you honest feedback. It's very hard to say to a founder and look them in the eyes and say, hey, we built this product, can I show you what it is? You're just going to get someone that's going to say, oh, this is great, this is wonderful. But when you're vulnerable and when you tell them it's not fully built yet, even if it is, you will get more raw and honest feedback. Because it is easier to tell someone, hey, before you make this mistake, I actually don't care about that. If you've already built it, they're not going to give you that feedback. So the further you suggest you are, you're actually going to hamstring a lot of the intel, hamstring yourself on gaining a lot of the intel. So that's one counterintuitive thing that I think a lot of founders don't realize.

**Lenny Rachitsky** (00:33:07):
**Jen Abel** (00:34:17):
She's awesome.

**Lenny Rachitsky** (00:34:18):
Amazing. And her last book, it actually has the opposite advice, but I think I know why, which is she focuses on later stage companies and her advice is the buyer. She has this really interesting insight that it's harder to buy software now than to sell it, because there's so much to consider and your job is on the line if you make a mistake. It's easier just to be like, forget it, I'm just going to go with what we have today. I don't want to put my ass on the line for buying this new thing that someone's trying. And so her advice is you actually have to educate the buyer on the market and here's what's happening. Here's where things are going, and here's why I think this is the future. But I think again, I think that's focusing on later stage stuff.

**Jen Abel** (00:34:55):
A hundred percent. And you raised such a good point, Lenny, which is late stage sales and early stage sales are very, very different. And I think that's where a lot of early stage founders get tripped up as they're taking late stage sales advice, usually coming from an investor. Or they've maybe hired a salesperson that's focused on more mature sales. But I think she's spot on. Buying is just as hard, if not harder, than selling right now. Because who wants to make a mistake and also who wants to go through switching costs? Oh, it's so painful.

**Lenny Rachitsky** (00:35:30):
Yeah. I think the other really important point here that you're making is that part of this initial sales experience of founder led sales is not sell as much as you can. It's to learn what people want. And so I love that you're sharing here's how to get the best possible feedback, not necessarily close the most deals.

**Jen Abel** (00:35:47):
Totally. And founder led sales is not about revenue on day one. It is about learning as fast as humanly possible to get to that pulse, so that you can earn the right to sell. There's this concept that I talk about, which is you're going out to run sales to collect the research, which is what founder led sales is. And then you have sales for revenue, which is that post founder led sales stage.

**Lenny Rachitsky** (00:36:13):
And your milestone that you suggested of how far to take founder led sales, you said around a million ARR, right?

**Jen Abel** (00:36:20):
Yeah. I think it's about, some people say 500, some people say a million. I think if you get to 500K really, really fast, then I think absolutely you can move out of it. If you get to 500K really, really, really slow, you might not want to get out of it right away. You haven't reached that velocity point yet.

**Lenny Rachitsky** (00:36:41):
Awesome. I have a post that we'll link to that has actually numbers and when all these big startups move from founder led sales. It's in that same series about product market fit. There's two stories this makes me think of I'll share real quick. One is Sprig. They shared a story of first round capital at their first round and their partner there is just like, we will not let you hire a salesperson until you hit a million ARR. We're going to help you. We're going to have salespeople helping you through it, but we're not going to let you hire someone. And he was really happy about that. The other is Zip, which just raised a $2.3 billion valuation, a procurement company that I was lucky to be an investor in. And their first founder led sales motion was they just reached out on LinkedIn to heads of procurement, and just leaned into what you're saving more, which is we just want your advice on this product that we're building. Just tell us what problems you have. It was very advice oriented versus we want to sell you this thing.

**Jen Abel** (00:37:33):
Yeah, no, absolutely. I have this theory that maybe you shouldn't hire any salespeople until series A, right? Because seed is all about experimentation and proving out that experiment, and then obviously series A is about exploiting that learning. But I see so many people, I mean hiring for early stage sales is the odds are actually more against you than trying to get your next round of funding. Because it's truly, truly such a counterintuitive stage.

**Lenny Rachitsky** (00:38:08):
Interesting. Okay, so I kind of took us on a whole tangent. You were sharing advice on how to get someone excited. So the first is when you're engaging, you're talking to the prospect and company ABC. Your advice is be very honest and vulnerable about your stage. Tell them you're early stage. You're building this thing. You're deeply passionate about this problem. Here's what we're trying to do, here's our priority problem perspective, and where we're focusing, and kind of get their feedback on your approach. Cool. You were going to go to the next tip and I took us off course.

**Jen Abel** (00:38:44):
Testing the questions to ask. Where is that aha unlock for you? But more importantly, where's that aha moment unlocked for them? Because when they start getting their gears churning in their head about, and this is the beauty about not having a product and not showing them anything, they're visualizing in their head now. This is a really powerful thing, which is like, so tell me how this looks in your head? How are you seeing this? And they're like, I don't know if I can see it. And I'm like, great, we'll show you that. Great insight to know by the way. Or so I think this is how, I guess this is how it would work. Walk me through what's living in your head right now. You pick up on so much and then all of a sudden they're like, wait a second, we've tried solving for this and it's still not solved for. Or you know what? We hired someone last year to manage this. Great understanding. Is it being measured? Is it being managed and have they tried to solve for it? The leading indicators that you're onto something here.

**Lenny Rachitsky** (00:39:44):
I love this thread because this is what everyone's always like, how do I know if I product market fit? There's these signals that you're talking about of signs that there's something here. Like a big enough pain point where they're excited to basically they want to pay for it, they will pay to solve a problem. So maybe just again, say the things you notice that are signs like, okay, you have something here.

**Jen Abel** (00:40:03):
So I think the first is it has to be a growing and widening problem. No one's going to spend time fixing a pretty fixed problem, because it's not necessarily really a priority anymore. It's maybe a pain in the butt, but it's not a priority if it's not growing or widening. So gate check one is like, what is the implication? Are you measuring? Are you managing this problem today? Yes or no? If they don't know, great to know. If they said no, okay, move on to the next. That's pretty powerful. No, we're not measuring or managing this. Okay, there's probably not much there. If it is being measured or managed, then the next question you want to get into is how have you tried to solve for it? Is it through that head count that you just hired? Is it through another tool? Is it just still an open gap because nothing exists yet to solve it? Just understand their maturity around how they've tried to plug it. These are all that make the secret moments in intel to close that gap.

**Lenny Rachitsky** (00:41:06):
That's awesome. And essentially it's showing you that there's a pain here that they're paying attention to and are trying to solve.

**Jen Abel** (00:41:14):
And what you're psychologically doing is now you're flipping them into a buyer where they're like, wait a second, hold on. I need to bring on so-and-so on the next call, they also think that this is not good enough.

**Lenny Rachitsky** (00:41:26):
That's funny because that's exactly what Wiz noticed, is they moved from people being like, oh, this is cool. I like it, I like it. To, okay, I'm going to pull in this person. I'm going to pull in this person to talk about it and make sure we're... And they were pulling in other people exactly as you described it, because they wanted to move forward on this thing.

**Jen Abel** (00:41:44):
Absolutely. And that's when you know there's some momentum behind it, which is when they're bringing in other colleagues. Whether it be the potential users, if you're reaching out to the executive, or the users like, hey, I actually want to bring my boss on to the next call.

**Lenny Rachitsky** (00:41:57):
Good sign. Cool. Anything else along these lines of how to get someone excited slash understand if there's going to be pull there?

**Jen Abel** (00:42:04):
I guess the one thing is please, please, please do not ask questions like what keeps you up at night? If you had a magic wand, what are your pain points today? I can guarantee you that answer changes every single day.

**Lenny Rachitsky** (00:42:19):
That's super interesting. Is there any tip for how to end one of these calls, as someone that's never done sales is like-

**Jen Abel** (00:42:23):
Yes.

**Lenny Rachitsky** (00:42:24):
Okay.

**Jen Abel** (00:42:25):
Get the second call booked on the first call.

**Lenny Rachitsky** (00:42:28):
Love it.

**Jen Abel** (00:42:28):
Pull up calendars. Look at calendars. Who else should be invited? It's just a natural evolution to ending the call.

**Lenny Rachitsky** (00:42:35):
Great. So that's in 30 minutes though, right?

**Jen Abel** (00:42:38):
And if they say, ah, I'll email you.

**Lenny Rachitsky** (00:42:42):
I could say.

**Jen Abel** (00:42:44):
That's also kind of a leading indicator. Yeah.

**Lenny Rachitsky** (00:42:46):
So do you recommend not getting off the call, trying to avoid that, or is it just-

**Jen Abel** (00:42:50):
If they won't give you time on the calendar, you could say, listen, great. Feel free to email me. Maybe they're just being honest and they don't have their calendar available, but nine times out of 10, it's usually I don't have the heart to tell you I'm not interested.

**Lenny Rachitsky** (00:43:04):
Yeah, it's hard. It's hard to tell people you're not interested. Very cool. Okay. This is amazing. Okay, so the next step, if I remember correctly, is kind of co-authoring and co...

**Jen Abel** (00:43:13):
Yes.

**Lenny Rachitsky** (00:43:14):
Okay, cool. Talk about that.

**Jen Abel** (00:43:16):
In the early, early days, one of the biggest ways you can get folks excited is it feels like it's going to be built specifically for them. The power of specificity, the power of being extremely focused. With that, you can literally turn a customer into a guide by asking them to co-author the scope. The scope of work. The co-authorship piece is important for two reasons. It helps you understand where they're on their biomaturity. Let me explain. If they do not have an existing process or strategy to solve X problem, they can't buy a technology yet, which means you need to sell them. And this is why I go back to you need to sell them some form of a service, right? Why? You want to be the one in there educating. You also want to get the logo. You also want to show the revenue. While it's not recurring revenue, it still shows intent.

**Jen Abel** (00:44:13):
And I think that that's really important. Every founder I speak to is like, well, my investor really doesn't want service-based revenue. That's fine. But then you can also tell your investor, great, should I wait 18 months to when they're ready to buy a solution, and be the one that's not the one selling them because someone else educated them? These are all of the implications of waiting. So yes, is service-based revenue great? No, we all know why it's not great. But it's great in the sense that it shows intent. It's great in the sense that you can call them a customer. It's great in the sense that now you can use their logo. And it's great in the sense because you are getting paid to educate them. You are getting paid to help them design their process. This is where all the power lives. And this is why so many AI startups a year and two years ago were going in on services contracts, because they wanted to set the mindset with the buyer around what this would look like.

**Lenny Rachitsky** (00:45:07):
Is there anything you recommend to time box contain the services piece? Because I know a lot of companies-

**Jen Abel** (00:45:12):
Yes, such a great point, Lenny. You absolutely need to time box this. I would time box it as 90 days and then what you can say in the next 90 days, we'll scope where we are and what you need. Scoping out more than 90 days, listen, so much is going to change. You might also not want to do it anymore, so you don't want to lock yourself up. So I would look at 90 day increments. A specific example. We had a founder that my colleague was working with, and they were selling a very specific technology to a non-traditional... Sorry, a very traditional industry. I don't want to be two bleeding with what it is, but a very traditional industry that's not used to working with startups, or necessarily plugging in a new technology right away. And they were very honest. They said, listen, we haven't changed vendors or partners in over five or six years.

**Jen Abel** (00:46:04):
I actually don't even know how we would do that today. Could you come in and explain to us, understanding where our workflow is and how we would integrate this, before we even consider the technology? Which we did. We got paid a nominal amount, but we're now a customer and now we get to set the stage for how they think about this. And then they won the technology contract. But everyone is so focused on selling the technology really, really, really quickly. That works in markets that know how to buy that technology, have a process in place, have a strategy in place, have an implementation team in place.

**Jen Abel** (00:46:40):
If it's a new technology like AI right now, they need a strategy and process like who's the human in the loop? Is it them or is it you? How do we measure success? What does success look like? What risks should we be aware of? Our legal team's not even aware of all these risks. You want to be careful because legal can immediately, and procurement, can shut these things down if it seems too novel, where it's foreign versus they're so used to buying services, especially at market. It's their largest budget line item.

**Lenny Rachitsky** (00:47:10):
So it's interesting because most founders are very afraid of moving into services. I hadn't heard this advice before.

**Jen Abel** (00:47:15):
Probably going to get slapped for saying all this stuff, by the way.

**Lenny Rachitsky** (00:47:17):
Well, so along those lines, what percentage of companies that you've worked with or see do that or have to do that step?

**Jen Abel** (00:47:25):
This is going to be a shocking step. So I would 40 to 50% have to sell some form of service before they can sell a technology.

**Lenny Rachitsky** (00:47:34):
Wow. Of B2B SaaS company?

**Jen Abel** (00:47:36):
A B2B SaaS. And this, again, this is specifically more top-down sales. But yeah, because the user and the buyer are different, which means there's user value, there's buyer value, there's all different players, there's procurement you have to go through. But yeah.

**Lenny Rachitsky** (00:47:50):
And just to make it even clearer, what's an example of a service that you've seen company offer in a step?

**Jen Abel** (00:47:55):
So I've seen, hey, can you come in and actually help me pitch this, design a custom pitch to my boss as to why we should do this today? And we literally got paid to build a sales pitch.

**Lenny Rachitsky** (00:48:09):
So it's not necessarily providing a service. It could be just helping them sell this.

**Jen Abel** (00:48:15):
Yeah. It could be helping them sell in a way that makes you... You need to get a lot more context on their business. It can also be, hey, we haven't actually thought about a process for how we can actually deploy something like this. We're currently using this technology, which we want to change from. You've kind of hit this at the right time. How would we implement or how is it best to integrate with this tool? Can we get both of you guys in a room to map this out?

**Lenny Rachitsky** (00:48:43):
Got it. So it's like consultants almost, like coming in to help you solve this problem.

**Jen Abel** (00:48:47):
It's a great point. It's consulting them towards the acquisition of the product. It's not consulting as one-off consulting.

**Lenny Rachitsky** (00:48:53):
Got it. And they know that obviously you're biased, but they also want the problem solved. And they're like, great, you're going to help us solve this problem, because it's on my plate and I need your help convincing leadership that this is-

**Jen Abel** (00:49:04):
And I need to craft the storyline as to why we need to do it anyway, so I will pay you to do it, but here's the format it needs to be in.

**Lenny Rachitsky** (00:49:10):
Fascinating. Wow. Okay. Anything else along those lines?

**Jen Abel** (00:49:15):
And then of course, if you're in a position to actually sell the technology, because the market is able to acquire and adopt it and not have to create a new strategy or process, then obviously sell the technology. You don't need to be selling services.

**Lenny Rachitsky** (00:49:26):
I want to come back real quick. The previous call, I wrote this note down as you were talking. You recommended not doing a demo and just talking about it broadly.

**Jen Abel** (00:49:34):
Yes.

**Lenny Rachitsky** (00:49:34):
Is that your advice there?

**Jen Abel** (00:49:36):
On the first call. Yeah. On the first call, my fundamental belief is that the demo is a bit of the only carry you control in the sales process, right? Once they see it, it's kind of like pitching an investor. Once they take a look under the hood, that dreaminess in their eyes, they're like, oh, I saw it. So leave them wanting more. And the demo is that, leave them wanting more. Even when you do a demo, don't demo everything. Leave it for a second call. Let them invest a lot of time in you. Again, if it's qualified. Preface that, if it's qualified. But everyone races through the sales process like let's do a demo call as quickly as humanly possible.

**Jen Abel** (00:50:20):
Yes, that is important down market. That is important if you're selling a $3,000 tool, you absolutely want to be demoing as fast as humanly possible because it's a high-volume game. Upmarket, when you're talking about hundreds of thousands of dollars, you want to slow that down as fast as possible. Because you want to, one, make sure all of the right people are in the room. As soon as there's one lead on this, and if it requires other people involved, it doesn't feel like anyone else's baby. So you want it to make it feel like the group's baby versus this one individual's baby because it's very quick. Someone can easily say, I'd rather use this tool. And then there's this stalemate of nothing happens.

**Lenny Rachitsky** (00:50:55):
Just to maybe reiterate, so far we've talked about figure out who to talk to, pitching them on talking to you, then having that first conversation. Maybe there's another conversation in there to get them excited. And then there's just getting them past the finish line, keeping everyone aligned. Is that roughly the next step or how should we think about this?

**Jen Abel** (00:51:14):
Yeah, so if you are selling upmarket and you now need to go to procurement, who is the group involved in the bot. They're the professional buyers. Procurement is a very interesting function. They are very smart, very, very smart. They do this for a living. They are professional buyers. So there's a couple things that you need to be aware of. You need to sell them as well. You need to really make this sound... Don't over complicate it. Don't add in jargon. Make it feel like, okay, I can wrap my head around this. The second piece is it's got to feel different from anything else out there. Because the professional buyer, it's much easier to say, wait a second, we have these 17 other preferred vendors that do similar ish work. Why don't you just go use one of them? Because, oh, by the time this gets through procurement, it could be another three to four months.

**Jen Abel** (00:52:05):
I've seen deals die on the vine because procurement actually suggested they go use another vendor in the system, that this buyer wasn't even aware of, because they didn't differentiate. It didn't feel different. It just felt slightly better. That's how it was positioned. The third piece is when you get to procurement, you're going to have to do all the work. Make their job easy. You are probably a very small piece into the very large deals that a lot of these people buy. So you can get easily sidelined just because you're just a small buy. So do the work for them. Literally say, I want to make this as easy as possible for you. Give me the forms that you need to fill out. I'll fill them out for you, and you can do it yourself. You can edit them. Do the lift for them. If you don't, it's so easy for it to just go there and die.

**Jen Abel** (00:52:55):
Another piece that's important with procurement is explaining exactly what you do and don't do. Because if you say you do a bunch of things and they can't really place you, they're going to send you to the kitchen sink of contracting an MSA, which is going to ask you for $5 million in an insurance policy and all sorts of other things. And the ability to look at your book at any point in time. And the reason they did that is because they can't classify you. So the easiest thing to do is classify you as high risk. So make it easy for them, make it simplified. You can also truncate your contracts, meaning let's say IT is maybe, and you want to ask this, how long does it take to get through IT due diligence? They might say, oh, it's a 90-day backup, it's a 60-day backup, it's a 30-day backup.

**Jen Abel** (00:53:43):
There's no backup. If there's a backup, you also don't want it to die and you want it to give an incentive. So this is when you want to truncate contracts to a technology contract and a service contract. Service contract allows you to onboard them, get them prepped, come in and educate all of the users about what they're about to go through. So that then there's an incentive to get that technology piece through. There's all these little things to think about, and I think everyone... Getting to procurement is also creative. And knowing who you're dealing with and how to make their job as easy as humanly possible. Because what you said and what April said is spot on, which is buying is just as hard as selling.

**Lenny Rachitsky** (00:54:23):
As I'm hearing you describe this, I feel like we might be discouraging people from selling because this feels very not fun. All these steps, all these procurement work, anything you can say to get people to feel like, okay, this is worth it?

**Jen Abel** (00:54:35):
Yes, I can get people very excited by this. Once you are in, you are in. Once you are in, you are now a preferred vendor. You now have the ability to cross over into other business units. You are now the reason that, hey, if your competitor comes in, well, you got there first. So what do you think procurement's going to say? This is why it's so important to get into the enterprise as fast as humanly possible. While it's a headache to get through, if you project manage it, you'll be good. If you make an accountability, just like if you were to go through fundraising. If you are a part of a team that had to raise money, it's no different. There's always some level of due diligence. There's always you doing more of the work than them. That's just enterprise sales. But if you prove value, your hundred thousand dollars deal could be $500,000 next year. Your $500,000 deal could be a million dollars the following year.

**Jen Abel** (00:55:36):
This is where stuff begins to compound, and this is where your growth journey really gets accelerated. So the beauty with enterprise is once you're in, you're in. You beat out your competitor for a short period of time. There is a little bit of a moat there, but it's not forever. You all have intel that no one else will have. Meaning you're a part of the conversations, you have the badge, you can join the meetings, you can ask for introductions. They'll do it. It doesn't feel like sales anymore. You're now a partner. This is why it's so powerful to get into the enterprise because there's so many compounding effects. If you put the effort in on the sales side, the return is insane.

**Lenny Rachitsky** (00:56:20):
That was awesome. Nailed it. I'm excited. Just for folks that are listening, just to calibrate what size of company you're talking about selling to here, what's the size of the advice you've been sharing so far? Roughly?

**Jen Abel** (00:56:33):
So I've been talking a lot about enterprise sales, which is I would say anywhere north of 500 to a thousand employees. Just mental model. I'm talking about enterprise sales specifically because there's so much nuance involved in it, because the user and the buyer are very different, right? As you go down market, let's talk about small business for a second. The user and the buyer are the same person. There's no procurement. If they like what you've built, it's pretty straightforward process. In the mid-market, mid-market is a funky place because you either are anchoring towards the lower end of mid-market, which is more upper end of small business, or you're anchoring towards lower end enterprise. Those are two very different divides. So mid-market, just if you're talking about lower end enterprise, again, this is all relative. If you're talking about lower end small business, again, your user and the buyer probably are the same person, which makes sales a lot more streamlined.

**Jen Abel** (00:57:31):
But churn. Small business, challenge with small business is the churn. If they get pissed off, if they don't feel like it's good enough, they are gone faster than you don't even realize. And they might even tell you. They'll just cancel. And we always see this on Twitter. They'll call American Express and say, cancel these charges. I don't want to talk to these people anymore. They're just more irrational because sometimes maybe it's their money if they're a small business. So small business in mid-market, while sales is a bit faster, you really got to be on the product market fit side, worried about churn.

**Lenny Rachitsky** (00:58:07):
And they also go out of business at a higher rate.

**Jen Abel** (00:58:07):
Exactly.

**Lenny Rachitsky** (00:58:07):
And so you have that kind of churn.

**Jen Abel** (00:58:12):
A churn piece too. Yeah, exactly.

**Lenny Rachitsky** (00:58:14):
Amazing. Okay. What comes next? So we're kind of in procurement at this point.

**Jen Abel** (00:58:19):
Okay, so now we're at signature, right?

**Lenny Rachitsky** (00:58:21):
Okay.

**Jen Abel** (00:58:21):
Okay. So as you enter procurement, okay, you want to know before you get to the next stage, who is signing this deal? Here are some examples of signers I've seen. Chief financial officer, chief legal officer, the business unit head themselves, the head of procurement. You want to know who that person is so that you can literally say to the head of procurement, hey, listen, I want to make sure this person has everything they need when they review this, to know what they're signing. Who is it and how can I give you a few bullets to share that you can maybe respond to, that we get tight so that they know exactly what they're looking at?

**Jen Abel** (00:58:59):
This was two or three years ago, I was involved in getting a deal over procurement that was just truly, it was a pharmaceutical company and it was very, very long process. And we got to the finish line and the CFO was the signature. And this is when I made the mistake. CFO responds back to the procurement lead who sent it to the business unit, who sent it to me and was like, what am I signing? I don't actually understand what these people are doing and why are we doing this?

**Jen Abel** (00:59:32):
So then she quickly said to me, hey, listen, we need to defend this. Can you put together some bullets? And I'm like, well, what kind of bullets do I need? What does this person care about? And again, it created so much anxiety and now I'm back in the bottom of the queue. Probably he or she's looking at the things that come in in an order of priority. So now I've elongated this by another month simply because I didn't plan. So this is just to say I'm learning from my own mistake of know who the signature is, because if they don't know what they're looking at, they're going to kick it out and you're going to lose your queue spot.

**Lenny Rachitsky** (01:00:06):
So many ways to fail. And this was you selling Jellyfish or this was you working with a company?

**Jen Abel** (01:00:11):
This was me helping [inaudible 01:00:15].

**Lenny Rachitsky** (01:00:15):
Oh man. Okay. Anything else in that step that might be helpful to folks?

**Jen Abel** (01:00:19):
Yes. One thing to caveat is you do not get paid until you are approved by finance, and procurement has a signature on the contract. Meaning don't start any work. Or if you do start work, know that there is no payment. The business unit can't just pay you. It's paid through a purchase order, which is paid through by finance.

**Lenny Rachitsky** (01:00:45):
So don't rely on that money unless it's finally, unless the signature's on the paper. Quick tangent, thoughts on discounting at any parts of the journey?

**Jen Abel** (01:00:54):
If there is a reason as to why. So discounting just to get the deal over the line, you're negotiating with yourself. Unless they ask for it and then ask them to defend it. Certain segments, like small business, you should leave a little bit of room for buffer because sometimes that's their own money. It's like their own small business. Mid-market and enterprise, there's got to be a reason why and ask them. Be like, hey, so if I give you a 30% discount, can I remove 30% of the value? You can kind of play it a little bit like that. I don't recommend it, but that's kind of what they're saying. But discounting is great if they're doing something for you far and beyond. For example, if they're a design partner, if they're going to be a reference for two years, if they're going to give you something far beyond just using the technology, then yeah, I think a discount is a good reason to give back to somebody that's giving to you, but not as a strategy to get a deal done.

**Lenny Rachitsky** (01:01:57):
Okay. Is there anything beyond this step of getting the signature? Are we done?

**Jen Abel** (01:02:01):
Yeah. I hope you celebrate because-

**Lenny Rachitsky** (01:02:03):
Okay, great. You got a signature from this whole process.

**Jen Abel** (01:02:08):
Yeah. Hopefully I'm not making this sound too daunting. I'm just really trying to lay out all the mistakes I've made.

**Lenny Rachitsky** (01:02:12):
Yeah, no, this is exactly what people need. This is amazing. And your pep talk was really helpful too. Along the way. What's the general timeline for sales process like this in your experience, with these 500 plus ice companies?

**Jen Abel** (01:02:26):
So there's three things that dictate sales cycle. One is how well are you project managing it? For example, I'll say, let's have our second or third call in two weeks. Two weeks? Do a week. Why are you elongating this? Keep your calls as tight as possible because that shortens your sales cycle. The second is just how complex the organization is. If you're dealing with a highly regulated industry, just know it's going to take a bit longer, sometimes 20 to 30% longer. So a highly regulated industry, nine to 12 months on conservative, on the conservative side. Again, it's tricky because is there a budget line item dictated towards it yet, or are you creating budget? How painful is the problem? And how senior have you gone? If you're talking to the SVP or chief of whatever, they're pretty good at about being able to navigate the traps.

**Jen Abel** (01:03:28):
If you're dealing with a director or mid-level person, they maybe have not purchased something before and just make some internal mistakes. I always say it can range anywhere. I've seen enterprise deals close in 90 days, believe it or not. Rare, but I've seen it. I've also seen enterprise deals typically take anywhere between six and 12 months. Really important but. Enterprises know that the process and the length to get the deal done is what costs more than the technology itself. Meaning the effort it takes to get through their system. That's why they're willing to spend so much. Sometimes that's actually more expensive than the technology itself. So don't negotiate with yourself, understand the value you're delivering, but don't be crazy. I've seen people try and go in with a million dollar deal as a seed stage startup. Oh, interesting thing. So interesting tactic. I've seen contracts in the enterprise that state that the deal cannot exceed more than 20% of the existing revenue.

**Jen Abel** (01:04:39):
So there are these things just to be aware of. In most cases, you can ask them be like, is that a hard line? Is that hard, hard line? Or how negotiable is that? Sometimes it's negotiable. Sometimes it's like, no, this is a hard rule. But then it seems silly because you take a hundred thousand dollar deal and bring it down to 20,000. It's just be careful that you're stripping some of the value out. But I kid you not. I've seen an enterprise deal go from, it lands at 60 and it turns into 280,000 in four months. So again, I want to encourage, yes, this is a lot of work, but the payoff is exponential.

**Lenny Rachitsky** (01:05:19):
What's a good ACV to start with if you're trying to sell to enterprise, to make it all worth it for your startup?

**Jen Abel** (01:05:25):
I would say anywhere between 50 to 200K depending on the business unit you're selling to. That's kind of like sweet spot. They're used to something in that realm.

**Lenny Rachitsky** (01:05:35):
And this is a startup selling [inaudible 01:05:38].

**Jen Abel** (01:05:37):
Early stage startup, like founder led, early stage.

**Lenny Rachitsky** (01:05:42):
Initial contract. Wow.

**Jen Abel** (01:05:43):
Yep. Initial contract. I would say, okay, probably caters more towards like 50 to 100K. But I've seen people sell to... Again, it's how big is the problem? Who are you selling it to? Is it the SVP that you started with and they've got a large budget, and it's a pretty healthy business unit? Or are you selling to a director?

**Lenny Rachitsky** (01:06:05):
If you're not able to sell your product at that price? What's your advice to teams?

**Jen Abel** (01:06:10):
If you are a startup, I always ask the founder, did you build this for the enterprise and is that the model you want to play? Or did you build this for small business? Small business is a marketing game. Marketing intensive activity, right? It's high velocity, high volume, lots of dials. It's a very different game than enterprise. So which game do you want to play? Let's just start there. Which game is more attractive to the founder. Or who is more exciting to serve? What's the storyline you want to tell investors? That plays a lot of it into it too. And do you have an enterprise product? Are you solving an enterprise problem or do you think you're solving an enterprise problem but you're not sure yet?

**Lenny Rachitsky** (01:06:53):
And your point is also mid-market is it's rarely to be successful.

**Jen Abel** (01:06:57):
It's hard to start there because you're straddling two very different go-to-markets, right? One that's of high value, one that's of high volume. And also mid-market companies, this is where a lot of people don't... If it requires heavy integration, they don't have those resources. That's usually outsourced to an Accenture or some of these consulting firms, and now you're having to rely on third parties to be involved. It gets messy.

**Lenny Rachitsky** (01:07:25):
This is fascinating. I have a list of questions from the audience that you pulled from Twitter. You asked on Twitter what questions to ask you as you're coming on here. So there's one that's very related to this, which is someone said, if you're still very early in pre-product market fit, but get initial validation from both small to medium businesses and enterprise, how do you decide which one to focus on? Is there a counter argument against starting or SMB going up market over time like most companies do?

**Jen Abel** (01:07:51):
I've seen companies successfully do in both of those motions. Truthfully. We know all of those. We know people that started small business and worked their way up into enterprise and were successful. We've seen people be really successful by starting at the enterprise, like a Wiz. That's more of an internal question, which is like, who did you build this for? Where's the problem most felt and which go-to-market game do you want to play?

**Lenny Rachitsky** (01:08:15):
And I think that latter part is so important. It sounds like why should it be what I want? It's like what's going to make a big business? But I think people forget, this is going to be your life for 10 years. Do you want to be selling to enterprises and building all the enterprise features? Would you prefer to build for small companies? Pros and cons to both but it's important to think about the life you're creating for yourself and your team.

**Jen Abel** (01:08:36):
A hundred percent. And I built my career in enterprise sales, upper-end mid-market enterprise sales, and yeah, that's just the game I know the best.

**Lenny Rachitsky** (01:08:46):
Yeah, that's also an important part of it. Just to double down on that is like what do you have experience doing? Not like...

**Jen Abel** (01:08:52):
Exactly.

**Lenny Rachitsky** (01:08:53):
Yeah. Where's the opportunity? Okay. Another question that I love is, and we touched on this initially, but I think it's good to come back to this. Someone said, customers are fascinated with what we're offering from the initial calls, but responsive momentum is too slow from their end. Would be great to know how to fix this.

**Jen Abel** (01:09:10):
Yeah, it's tough because unless you're in the weeds to understand why. I'll give you a few examples that it could be. Did you speak to a buyer who now is trying to sell this up to their boss, and it's just getting sidelined and they don't know the executive value? They've just been selling the buyer value the whole time. That's one reason things can slow down. Another reason things can slow down is you haven't really framed the problem well and they don't understand the full implications of why they need to solve it. So it's kind of just sitting there a little bit. The third is they've just been really nice and it's not going to go anywhere.

**Lenny Rachitsky** (01:09:49):
I guess that that's usually the latter. Was that true or it's kind of all over the place?

**Jen Abel** (01:09:54):
It's so hard to give a founder hard feedback. Because they've just dedicated their life to this thing. Who wants to be the bearer of bad news? And sometimes you just need to let the actions speak louder than words.

**Lenny Rachitsky** (01:10:11):
We've talked about all these steps. I'm curious, when you come to a founder or your team, where do you find the most on lock often? Which of the steps of the sales process do you find the biggest opportunity to improve conversion in sales?

**Jen Abel** (01:10:25):
It's qualification. Qualification because if you spend your time on the wrong leads, that equates to a zero. So if you don't get that first level right, let me put it this way, everyone that I know says they have a bottom of funnel problem. It's never a bottom of funnel problem. It's a top of funnel problem. I've actually never seen a bottom of funnel sales problem. It's always qualification, which is a symptom of not reaching out to the right person, not having the right messaging, not solving the right problem, or not being different enough.

**Lenny Rachitsky** (01:11:06):
I love this. So basically the biggest pitfall people fall into, in your experience, is they're just talking to the wrong people, wasting their time?

**Jen Abel** (01:11:15):
Yeah. Talking to the wrong people or using the wrong message or...

**Lenny Rachitsky** (01:11:20):
Pitching them something they can't actually deliver.

**Jen Abel** (01:11:23):
Yeah. Here's the other thing too. Sales is supposed to feel fun for the buyer. They should be like, this feels fun. This person's invigorating. They're going to change my world. They're going to make me see things differently. They're going to get me promoted. They're going to help me increase my influence internally. And so many salespeople are so boring. How many times have you got off a call and you're like, I can't wait to get off this call? And founders too, just like all of a sudden their passion, they go like stone cold face. And it's like bring the passion. Bring the energy. That is felt by the market. Remember, some of these people are in boring jobs. Give them an outlet.

**Lenny Rachitsky** (01:12:01):
So along those lines, I think a lot of people are just not... Like me included. I feel weird doing sales. It's a weird experience trying to convince someone to buy something. Is there any advice you could share to get someone over that hump?

**Jen Abel** (01:12:13):
If you have built something that you believe in. Very hard to sell something you don't believe in. I think everyone agrees that. If you've built something you believe in and they have a problem, that's a beautiful thing. Truthfully, that's what makes the world go around, is I have a solution to your problem. Now you just need make sure that the problem, you just need to be honest, that the problem they have, your solution truly can solve for. And you're not short selling just to get a logo and a deal over the line, and see them churn in six months or nine months or three months. That what you are selling is truly going to solve their problem. And be honest about it. I can't tell you enough when I tell someone, listen, I don't think I'm the right fit for you. They try and sell you on them.

**Jen Abel** (01:12:58):
They're like, well, wait a second. Hold on. What if we did this, this, and this? And it's like, no, no, no. Here's what we're great at. And then they can say, well, I also need that too. And all of a sudden, this level of trust comes out. And trust is the number one currency in sales. If you are a trusted salesperson, people will recommend you all day and every day. If you're a trusted founder, your market will continually send you leads and word of mouth. So don't try and sell something just to prove to investors you sold something, because it'll be quickly seen on the other side when they churn.

**Lenny Rachitsky** (01:13:32):
Amazing. Okay, Jen, so I'm going to cut the lightning round. I know you also have to run and do real work. So let me give you a chance just to talk about what you do, how you help companies in case folks can find value in working with you.

**Jen Abel** (01:13:44):
Deeply passionate about sales, as I'm sure you can tell. We specifically help founders through this pain. Navigating enterprise sales, mid-market sales, and really trying to crack that first million of ARR. Or sometimes even that first 500K of ARR if they move fast enough. And it is really hard. It's counterintuitive to what most people think, but it can be really, really fun when we show you the way.

**Lenny Rachitsky** (01:14:13):
And you described to me how it works, and I think it's important to clarify this. You basically embed with the team and help them do this.

**Jen Abel** (01:14:19):
Yes. So I fundamentally do not believe in outsourcing the heartbeat of the organization, which is sales. So what we do is we embed alongside the founder and drive a lot of the execution, but make sure that they are the tip of the spear engaging directly with their market, and learning directly from the market's mouth, not playing this game of telephone.

**Lenny Rachitsky** (01:14:40):
Amazing. And I mentioned this when we were chatting, but I think of it again, is when I was interviewing all these companies about how they started selling early on. One of the interesting threads that I heard again and again, is how many of them hired a coach or a consultant as a founder to help them learn to do sales. And that's essentially what you do. And I didn't even know a service like this existed, so this is super cool. It'll point people to what you do. I also have to ask you, the question I ask everyone at the end is just, how can listeners be useful to you?

**Jen Abel** (01:15:08):
Help each other out. I think so many people have helped me in my career, and in this journey, that the pay it forward model that exists in the technology space is so beautiful. So just don't let that die.

**Lenny Rachitsky** (01:15:28):
Beautiful. Well, Jen, thank you so much for being here.

**Jen Abel** (01:15:32):
Lenny, this was awesome. Thank you so much for having me.

**Lenny Rachitsky** (01:15:34):
So awesome. So excited for folks to hear it. I've learned a ton. Okay, I'll let you go. Bye everyone.

**Lenny Rachitsky** (01:15:41):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lennyspodcast.com. See you in the next episode.

---

## [9/17] “Dumbest idea I’ve heard” to $100M ARR: Inside the rise of Gamma | Grant Lee (CEO)

**Grant Lee** (00:00:00):
I'm in my third pitch in, I get to the very end of the pitch, feeling pretty good about myself. The investor pauses a little bit, and then just says, "That has to be the worst pitch, worst idea I have ever heard. Not only are you trying to go against incumbents, you're going against incumbents that have massive distribution. You are never going to succeed."

**Lenny Rachitsky** (00:00:18):
You guys are at over 100 million ARR now, worth over $2 billion. One of the most interesting ways you guys grew early on was influencer marketing.

**Grant Lee** (00:00:25):
All the initial influencers, I onboarded manually myself. I would jump on a call with each one of them so that they understood what Gamma represented, how to use the product. You want to be able to have them tell you story but in their voice. I think a lot of people think influencer marketing and they'll think these big trendy creators, people that have a million followers. This is the wrong approach. You basically give them a script to read, immediately feels like an ad. That product is not connected really to them in any way. You're much better doing the hard thing, which is hard to scale, finding the thousands of micro influencers that have an audience where your product maybe is actually useful. People really trust what they say. That ends up becoming this wildfire that can spread really, really fast.

**Lenny Rachitsky** (00:01:04):
Something you talk about it, there is actually a lot of ways to think experimentally, even in the early stages.

**Grant Lee** (00:01:08):
We would have an idea in the morning, come up with some sort of functional prototype, recruit a bunch of people that are legitimately good prospective users, but have zero skin in the game, ship fast so people can start playing with it. In the afternoon, we're already running pretty full scale experiment. You start actually hearing other people describe their usage of the product. We can also watch them struggle. By the evening or by the next day. We can actually go through all of it together and say, okay, we're going back and we have to fix this. This is not usable and we've done that for everything.

**Lenny Rachitsky** (00:01:36):
Today my guest is Grant Lee, CEO and co-founder of Gamma. This is a really unique and inspiring, and very tactically useful conversation because Grant is building something that is essentially the dream for most founders. A massive AI startup that's profitable, and has been for a long time, that didn't raise a lot of money for a long time. And as a small team, it's just around 30 people, all who can fit in a small restaurant serving over 50 million users globally.

**Lenny Rachitsky** (00:02:02):
If you're not familiar with Gamma, they're an AI powered presentation and website design tool. They just hit 100 million ARR in just over two years. They're valued at over $2 billion. And unlike a lot of the fast growing AI startups that you hear about, they're growing profitably and sustainably, and in a category that most people did not believe had a huge business opportunity. As you'll hear in the conversation, one investor told Grant, this is the dumbest idea that he has ever heard.

**Lenny Rachitsky** (00:02:29):
In this conversation, Grant shares the very counter-intuitive lessons that he's learned, finding product market fit, how he knew they had product market fit, the specific tactics that helped them grow, including a deep dive into influencer marketing, which blew my mind. Also how they figured out their price, his thoughts on building a GPT wrapper company that is durable, a ton of hiring advice, and so much more. This could honestly have been another two hours of conversation. I suspect we'll do another follow-up conversation next year.

**Grant Lee** (00:05:19):
Lenny, it's so great to be here. Thank you for having me.

**Lenny Rachitsky** (00:05:21):
I see your face all the time in my LinkedIn feed. I don't know if you know this is a thing. On these JPMorgan Chase ads. I'm so curious if other people see this or if it's just me. Did you know this was a thing?

**Grant Lee** (00:05:31):
I think it's maybe once a day now I get a text message and just no message. It's just a screenshot or an image of me doing something in San Francisco on one of these ads that we're seeing. And so yeah, kind of embarrassing, but also we're happy customers of JPMorgan Chase, so trying to represent.

**Lenny Rachitsky** (00:05:49):
Oh my God, I hope you love them. Because it's always you. There's no one else. It's like Grant.

**Grant Lee** (00:05:54):
I know. Can I talk to you [inaudible 00:05:56] swap somebody out. I mean that'd be great. I'm totally fine with that.

**Lenny Rachitsky** (00:05:59):
Okay, so to get serious, the reason I'm really excited to have you here is, unlike a lot of super fast growing AI startups, you are both growing like crazy, you are growing very profitably. We're going to talk about this. You did not raise a ton of money when you started. You waited a long time to raise a bunch of money. You also built a business in a category that I think most people never imagined there was this big of an opportunity. And you're basically, you've achieved the dream of a lot of founders these days, especially people building AI startups.

**Lenny Rachitsky** (00:06:31):
So my goal with this conversation is essentially do an anthropological study of a really successful AI startup. Talk about how you found product market fit, how you grew, all the lessons you've learned along the journey. And I'm going to break this conversation up kind of along the different milestones of the journey. Before we get into the first piece, is there anything that you think is important for people to hear broadly about the story of Gamma?

**Grant Lee** (00:06:57):
Yeah. Maybe I'll just start with a quick story if that's okay. And it's really just the founding story. So we started the company back in 2020. This is peak pandemic. And even fundraising was just so different. So all of the fundraising was done over Zoom. You were kind of sitting in these Zoom meetings trying to pitch. Many investors you never met in person. So just a different era. And so for us, we're first time founders. I was actually living in London at the time, and so different time zone. I had to do all of my pitches at night. And I have two little kids, so wait for them to go to bed. 8:00 PM. We had a pretty modest flat, so nothing big. I would basically find this little corner between the kitchenette and the laundry room to kind of set up shot, far enough from the kids so they wouldn't be woken up.

In between 8:00 PM and like 2:00 AM, I'm just pitching. Trying my best. I had the fake Zoom background so people didn't know where I was, and just pitching. And so really the first day, I'm in my third pitch in, trying to tell the story of Gamma, obviously just starting to get the hang of the pitch. And I get to the very end of the pitch feeling pretty good about myself. And the investor pauses a little bit, and then just says, "That has to be the worst pitch, worst idea I have ever heard. Not only are you trying to go against incumbents, you're going against incumbents that have massive distribution. You are never going to succeed."

**Grant Lee** (00:08:23):
And so in my head, I'm already kind of shell shocked and thinking what's my rebuttal? And before I could even respond, he hangs up. And so I'm there sitting there thinking about it. And before I could really get down on myself because I had to prepare for the next pitch, I just internalize this feeling that maybe he's right. Maybe something about what he's saying is actually correct. And so for me, I started thinking about, if we're going to succeed in this category, we're going to really have to think about growth from the very beginning. This category is going to be really, really hard to break into.

**Grant Lee** (00:08:58):
And so we really kind of made this sort of promise to ourselves, that as we continue to build, growth was going to be critically important. And so my thing to your audience is that I don't come from a growth background. So if I can learn growth, anybody can learn growth. And I think especially in this sort of market, hyper competitive, oftentimes very crowded, it's going to be essential.

**Lenny Rachitsky** (00:09:19):
That is such a fun story. Oh my god. How bad must this investor feel at this point. We won't name names. Just to share some stats, I know this is going to be by the time this launches, this will be out, but you guys are at over 100 million ARR now, worth over $2 billion. A business that, again, most people did not think was going to work in this category.

**Grant Lee** (00:09:41):
Yeah, thank you. Yeah, we feel super proud to have accomplished that. And again, yeah, I'm excited to share some of the growth tactics and things that worked for us because I think hopefully it'll help others kind of on their journey as well.

**Lenny Rachitsky** (00:09:53):
Okay. So let's dive into it. Let's talk about product market fit. Tell us the story of just how you found product market fit, and how you knew you found product market fit.

**Grant Lee** (00:10:02):
Yeah. I'll start by telling kind of the moment where we thought we maybe had product market fit. And I think a lot of founders ask themselves, do we have it or are we not? And I think there's often a sort of temptation to kind of almost fool yourself into thinking you have it. And so we sort of did our first public beta launch, this is back in August of 2022. We launched on Product Hunt, and felt really good. We had what we felt like was a great launch, ended up winning product of the day, product of the week, product of the month. And it was like, wow, I think we have something here.

**Grant Lee** (00:10:32):
And then we'd look at signups, and you'd get that initial spike in signups, and then they sort of flatten out. We were still getting new users every day, but it was clear we didn't have strong word of mouth. There wasn't strong organic virality. And so if we just kind of played things out, we knew that the product wasn't going to grow on its own. Something was missing there. We didn't have that strong word of mouth so that the product could just continue growing.

**Grant Lee** (00:10:55):
And so we really asked ourselves, okay, what do we need to change? And the answer is we need to fundamentally change everything. It for us almost became this sort of bet the company sort of moment. Because at that point we were running low on runway. We knew we needed to make progress and we didn't really know what could be done. And so we got everyone together. At this point, the team was just over 12 people. And we said, okay, it's going to be all hands on deck.

**Grant Lee** (00:11:22):
We are going to do everything we possibly can to make the first 30 seconds of the product feel magical. The moment you land into the product, it has to be great, and it has to be so great that someone that goes through that onboarding is going to tell all their friends. And if we can get that right, then maybe we have a chance at actually doing something in this space. And so we spent three, four months actually after the product launch, we felt great, but we knew we had to go back to the drawing board.

**Grant Lee** (00:11:48):
We spent the next three, four months actually revamping the entire onboarding experience. And of course, this is also where AI for us kind of played a big role. We actually rebuilt it so that AI was part of the actual onboarding. So every single new user would experience this sort of magic in the first 30 seconds. And so we relaunched, this is end of March 2023. And all of a sudden, we'd go from a few hundred signups a day to now first day it was like a couple thousand, and then the next day would be like 5,000 signups, and then 10,000 signups a day, and then 20,000 signups a day.

**Grant Lee** (00:12:21):
And then it just kept going up. And we weren't doing any sort of marketing, no advertising. It was all sort of organic word of mouth virality of the product, people using the product and sharing it with others, where we for the first time really felt this pull. We didn't have to do anything. Product was just growing. And it was just such a distinct difference between that feeling and coming out of the Product Hunt launch where we could have fooled ourselves into thinking we have product market fit. I think the temptation would've been, hey, let's just spend more on ads or spend more on marketing. Because we'll just fuel the top of the funnel and everything else will work itself out.

**Grant Lee** (00:12:53):
I think that would've been a trap. I think that would've led us down to this path of trying to brute force our way into product market fit. And it would just always be sort of a fleeting sort of destination. We would never actually arrive. And so I think we made the tough call, the right call. It was a sort of bet the company moment, and I think on the other side it just felt so different.

**Lenny Rachitsky** (00:13:11):
Grant, this is exactly what I wanted this conversation to be. I'm so excited. I have so many questions I have to follow up on the stuff you shared before we even get to the rest of the journey. So one is essentially what you're describing is product market fit to you was when organic growth started to really take off, and it was just growing through word of mouth. You weren't doing much because it was so awesome, people were telling their friends about it. Is there anything more there that might be helpful for people to share just to hear about just like, okay, here's what it actually looks like?

**Grant Lee** (00:13:38):
Yeah, I mean my one piece of advice is when you're early on, your mindset should almost be like you're trying to create a word of mouth machine. If you can get that part right, everything else becomes significantly easier. And if you have any, and I think this applies to both prosumer, B2C, as well as even B2B products, if you have a B2B product, even if you're not telling all of your friends, you should be telling colleagues where that product is relevant. You should probably be telling former coworkers where, hey, you've discovered something like, oh, I wish we had this in our prior lives, and that should even be magical.

**Grant Lee** (00:14:14):
And then you should see that in all the leads that are coming through or people coming through through your prospects and your existing customers. If you're not seeing a healthy chunk of those leads come through that way, I would go back. I'm like, why? Why is that not happening? Because again, that's the massive tailwind you need where every single thing you do on top of that, all the marketing, all the sales, all the advertising, you're just going to have it becomes way, way easier.

**Lenny Rachitsky** (00:14:36):
How much of this was, you described it as a word of mouth machine, how much of this was word of mouth loops and virality features versus just the product itself? One was awesome and two is kind of innately shareable because it's presentations people share with each other.

**Grant Lee** (00:14:50):
Yeah, totally. I think for us, we do benefit from being in a category where, by nature of it, if you like Gamma, you're sharing it, presenting it to others. So I think for us it's a combination of both. And ideally, you have other ways where word of mouth or organic virality can happen in your product. So by nature of usage, like it's being shared.

**Grant Lee** (00:15:09):
We basically had an internal mantra that, we go back to the first 30 seconds, we want it to be dead simple for someone to create content, we want to be dead simple for them to share it. And everything we did for that first 30 seconds, or call it the first few minutes, is remove friction so that they can do both of those things. Create and share. And I think other people, when you look at your own product, you think about, okay, what is it about my product and how it gets used? Can you remove friction such that it can actually spread, and even if it's locally within an organization or within a workspace, just be able to enable that as much as you possibly can.

**Lenny Rachitsky** (00:15:43):
The other really profound point you're making here is the story of you won product of the day on Product Hunt, which alone is so hard. So many people try to win and don't. Most people don't. I've tried to help companies win, and it's a really hard thing to achieve. And then you won product of the week and product of the month, and still you're like, no, this isn't working. Most people that achieve that are like, no, we got this, and they would not have to bet the company. There wouldn't be a feeling that we have to rethink everything. What is it there that you're just like, no, this isn't going to work, as much as exciting as this is, this isn't it?

**Grant Lee** (00:16:20):
Yeah, I mean part of being a founder is being as self-aware as you can and be your own worst critic. And so oftentimes you want to have these vanity metrics that feel good to celebrate, and you should celebrate. But you should know when it's a vanity metric versus is this core to our growth engine? If this number goes up, does it mean the product is working? And I think that's where we looked at, okay, it felt good to win those things. We kind of put ourselves at least on the map. But it wasn't good enough to actually have this sort of feeling that we had a core growth engine we could just invest in and get better and better. That wasn't there yet.

**Lenny Rachitsky** (00:16:53):
So essentially it kind of started to just plateau and slow. It wasn't like this rocket ship that took off from that point.

**Grant Lee** (00:17:00):
Yeah, it was still like we were still getting signups. They were coming through. But you could just tell there wasn't this building momentum. And I think that's where it's always hard to tell. You have to, me and my co-founders, we sat down, we're trying to be honest with ourselves. Okay, is this going to be enough? And it just really felt like it wasn't going to be good.

**Lenny Rachitsky** (00:17:17):
The other point here is the power of onboarding, which comes up a bunch on this podcast when you talk about driving retention. So you launched Product Hunt, did great and then started kind of petering out. How much did the product change after things started to work versus onboarding? Just like how important was onboarding? And then just tell us why the first 30 seconds, where'd you come up with that number?

**Grant Lee** (00:17:41):
Yeah. So for us, the onboarding and the product experience, for us that's intertwined. The analogy I always think about is if you go into a restaurant and maybe the food is good, but when you really think about the user experience, it's like the moment you walk into the door, you get seated, the waitress, waiter comes by, greets you, you can order. And of course the food has to taste good. And then you finally get the bill and you leave. Is that entire experience something that feels delightful? Is it good enough for you to tell your friends about? If someone just came by and dropped the food on your plate on the table, and just left and never came with a bill, I'm like, okay, maybe I'm not going to recommend this to somebody else.

**Grant Lee** (00:18:18):
And so for us, we thought about, okay, the first moment someone walks through our door, dropping into the product, what is something we can give them? Can we shorten that time to value as much as possible? A lot of this is inspired by Scott Belsky, he talks about that first mile, the first 15 minutes. And I think that's totally right. And I think one approach is you think about new users as you almost have a cynical view of them. You have to think about them being selfish, vain, and lazy, right? They're coming in, they have no desire to learn a new tool.

**Grant Lee** (00:18:48):
And so what can you give them in that first 30 seconds that earns you the next 30 seconds and then the next 30 seconds? And so for us, we knew that if we can't, people's attention span is even shorter to today than maybe 10 years ago, and so what is it in that first 30 seconds, can we actually show you something and earn the right to keep building that relationship with you? We really thought a lot about that, and certainly that's all we could really afford at the time. We only had 12 people building. It's like we couldn't make an entirely revamp the entire product. We knew that we had to least put all of our energy into one spot, and so we made that coming into the door, come through the door, make that moment feel magical so that we can do a little bit more over time.

**Lenny Rachitsky** (00:19:27):
I love your point about how you could think of it as like, okay, it's onboarding versus the product. The lens of how do we make this incredibly valuable and aha-ish for the first 30 seconds almost informs what the product should be.

**Grant Lee** (00:19:39):
Yeah. It really helps you pull forward what is the most magical thing about your product. Sometimes founders will think about the five, 10 features. Well, maybe there's only one thing that kind of differentiates you. I try to learn a lot from, we'll get into some of the marketing pieces of this, but even just having this sort of founder led marketing lens of what can I do to help a new user just understand. There's this thing from consumer advertising, which is you throw a consumer one egg, they can probably catch it. You throw them four or five eggs, they're probably going to drop all of them.

**Grant Lee** (00:20:12):
And oftentimes founders want to talk about the four or five features they have, maybe 10 features. And then the consumer is totally confused, like why do I need this thing? We try to just give them that one egg, that one first experience. We're like, okay, create a slide in seconds. That's the egg. I'm going to throw you this egg. Is that compelling to you? Some people are still going to opt out, but for the people that catch that, you're solving a real problem for them, and then you can continue building on that over time. You've given them enough so that they'll sit around and keep playing with your product.

**Lenny Rachitsky** (00:20:41):
That is a hilarious metaphor I've never heard for onboarding time to value, just focus on one egg at a time. Just going even further back, what was the original insight that you had that led to Gamma and what Gamma is today?

**Grant Lee** (00:20:55):
After the last startup I was at was acquired, I went back into kind of my roots, which is consulting. I was advising early stage startups. And the sort of medium I was using was Google Slides. So I just remember this late night trying to prepare for next day's meeting, trying to format and figure out the right layout, and spending hours just trying to get the sort of look and feel right, rather than the content itself. And for me, that just felt completely backwards. I should be spending 90% of the time on the content, 10% maybe on the design and formatting.

**Grant Lee** (00:21:25):
And so the question just was what if there's a better way? What if we could reimagine this format from the ground up? Slides have been around for almost 40 years as the default medium of choice for a lot of this. And so we thought about, okay, if we had different building blocks, different primitives, so you're not locked into the fixed 16 by nine slide, what could we offer to users? And so that was really the starting point of all this.

**Lenny Rachitsky** (00:21:48):
Hearing this, I could see why investors would be like, I guess so. But slides has been around, PowerPoint has been around 40 years. I get it. I get why people would be... And specifically AI, was that a part of the vision initially, or did AI start to come up, and then wow, great timing,

**Grant Lee** (00:22:04):
Great timing. It wasn't part of the original vision, although the spirit was there, which is we wanted to make it incredibly fast and effortless for people to create content. So it just so happened that AI was a magical gift that allowed us to do all those things along the same sort of ambition or vision that we had. And so we integrated it core to all the building blocks we were already building well before AI was part of the picture.

**Lenny Rachitsky** (00:22:27):
It's such a cool other example. There's just so many examples of ideas that were not possible before are now very possible with AI. And it's a great opportunity for people to come after as these places, categories, people think is an impossible place to build a big business. AI now allows it. Awesome.

**Lenny Rachitsky** (00:22:43):
Speaking of that, let's talk about the growth journey, and how you actually grew from nothing to 100 million ARR in just over two years. I'm thinking we break it up. I know these milestones aren't that clear, but kind of like zero to 100 million ARR, one to 10, 10 to 100, something like that. And let's just see how it goes. How did you get your first set of users? How'd you get your, say, first 100 users? How'd you get to 100 million ARR from zero?

**Grant Lee** (00:23:10):
Our first 100 looks very different, I'd say. So this was even pre this sort of AI launch we had. The first 100 users for a product like ours, you're trying to convince all your friends to use the product. Anybody that's ever made a slide deck, you're trying to talk to. And I think early on, your friends want to do you a favor, so they're going to try the product. They're also going to lie to you. They're going to tell you how great it is. And then you look at the usage and nobody's coming back.

**Grant Lee** (00:23:33):
And so I think our first 100 was sort of gradually hard-earned post the product launch, people learning like, okay, this is kind of becoming a little bit more useful. Usage was still pretty episodic, so they weren't coming back every week. And then I do think the moment post the AI launch is where all of a sudden we saw that sort of organic growth happening, people coming back to the product regularly. And so that's where, it wasn't even the first hundred, it was like probably the first 10,000 users all came within a pretty short time period after that initial launch.

**Lenny Rachitsky** (00:24:02):
Awesome. We're going to talk about monetization pricing later, which is obviously an important part of actual getting to million ARR and 10 million AAR. So what I'm hearing essentially is the Product Hunt launch was a big part of just the first 10,000-ish users. I know there was also a tweet when you relaunched that helped in a big way. Talk about that.

**Grant Lee** (00:24:24):
Yeah. So when we did our AI launch, we didn't do our AI launch on Product Hunt. We basically said, hey, let's just put it out on Twitter, see if we can get some virality. And honestly, we kind came up with kind of a clickbaity sort of tweet. It was like the most valuable skill in business is about to become obsolete. And so it was intentional in that we wanted to create a little bit of engagement. We knew that having sort of a more provocative in a tweet would allow people to engage with it.

**Grant Lee** (00:24:54):
And so after a couple of days, all of a sudden it started getting a little bit more viral and a lot more engagement. And we looked and it was basically because Paul Graham had commented and saying something like, "Surely, the thing that the slide deck is describing is more valuable than the slide itself." And obviously it was fun just to see that comment. I think once that comment came through, even more engagement on the post. And then that was really the whole intent of that post was just to be able to have that level of engagement so that people, it would have some level of reach.

**Grant Lee** (00:25:24):
And so for me, it was almost like my first learning moment, going back to what does founder led marketing even mean? It means how do you actually break through the noise? How do you get a chance to have people even engage with a post like that? Part of that is copywriting, part of that is storytelling. Part of that is just having even the right visuals to share. And so it was for me kind of a moment just understanding, hey, to kind of do this right, you kind of have to do things that maybe you're not super comfortable with, but it makes a difference.

**Lenny Rachitsky** (00:25:52):
Such a fun story. So you intentionally set that announcement up to be controversial is what I'm hearing?

**Grant Lee** (00:25:58):
Totally. Yeah. I'd say provocative. A little spicy.

**Lenny Rachitsky** (00:26:01):
That is so cool. So essentially you got to 10,000 users through Product Hunt, and then essentially one controversial tweet that ended up baiting Paul Graham to comment.

**Grant Lee** (00:26:11):
Totally.

**Lenny Rachitsky** (00:26:12):
Amazing. And it was just a comment. It wasn't even him retweeting it.

**Grant Lee** (00:26:16):
No, just a comment. And then others would pile on.

**Lenny Rachitsky** (00:26:19):
Yeah. It's interesting how much a comment can increase the distribution of a tweet versus them retweeting it or quote tweeting it.

**Grant Lee** (00:26:26):
Totally. And of course the algorithm change all the time. So part of it was just luck based on when it happened, how it happened, who posted.

**Lenny Rachitsky** (00:26:32):
And you use this term founder led marketing, which I love, and I'm already seeing it in action here. This is you thinking about, it's not delegating to someone in marketing, it's not hiring an agency, it's like how do I tell a story that I think will break through the noise based on you building this company, having the insight to build this product. And I guess is there anything more there you think is important for people to hear about the importance of the founder thinking through this stuff?

**Grant Lee** (00:26:56):
Yeah. I mean, I think most people today are probably familiar with founder-led sales, which is still very, very important. I think before you hire your first salesperson or AE, it's great for the founder to understand what it takes. And they're going to craft the right narrative, the right story. At my previous role, I was the COO at a startup where I was doing a lot of, I wasn't founder, but I was early. And so I was helping the founders go through this, and really helping go into meetings with a client or a prospect and saying, "Hey, this is why our product is interesting."

**Grant Lee** (00:27:26):
And I think today there's so many AI startups that are much more either B2C or prosumer. And so you're not necessarily talking to individual prospects, but the idea that you can be really in control of the narrative on the marketing side is really, really important. And I think I'll describe a few things where, over time, I think that skill set just really, really helps you.

**Grant Lee** (00:27:48):
One is you have a chance to be a creator yourself these days. I think a lot of founders are trying to be more active on social media. And I think if you can kind of overcome the initial cringe factor of seeing yourself and postings like, oh, this doesn't feel authentic, if you can overcome that initial feeling, you start investing into like, okay, how do I become a better copywriter? How do I articulate something that is clear, not just clever? I think there's that saying where obviously if you can have that clarity, that's super important. And most people will try to get super creative with their copywriting, but that's not usually the right way to break through and communicate something. So how do you improve your own copywriting?

**Grant Lee** (00:28:27):
And then that allows you to actually have a higher bar when you start working with other marketers, or in this case for us, working with influencers. If you're working with influencers and creators, and you can totally empathize with how they approach that work, and you know what a good hook looks like or you know how to structure a good post, you can only do that if you've gone through it a little bit yourself and you know how hard it is. And I think too many founders will then just say, they'll write something that just feels so much like an ad, and then they'll give it to a creator to help amplify. And then that just never works. And so I do think part of founder-led marketing is going through this yourself.

**Grant Lee** (00:29:00):
And so I do think part of founder-led marketing is going through this yourself, using your own platform. In the beginning it's probably going to be super small. But as you get bigger, you have a platform to ... You have a voice and people listen. And you're going to get better and better at your own storytelling. I think these are all skills you should invest in as early as possible, because you know you're going to have to get better and better. It's like practice, you got to practice over and over.

**Lenny Rachitsky** (00:29:21):
I definitely want to pull on this thread more because you tweeting the lessons you learned building Gamma is what led to this conversation. I was reading, I'm like, " Okay, he's sharing a bunch of stuff, but there's so much more I want to hear." And we're going to talk through this and go in a lot more depth than what you've shared on Twitter. But I love that that's example of that working, having this conversation.

**Lenny Rachitsky** (00:29:41):
So let me ask a couple of questions here. One is just how do you find time as a founder or CEO of a very fast growing crazy startup? We have so much to do. How do you just allocate the time to do this? And then any just key lessons you've learned about doing this well, beyond what you've already shared for people that want to try to start sharing things on LinkedIn and Twitter?

**Grant Lee** (00:30:00):
My advice is definitely just to try to start small. Don't let it become so intimidating that you just don't get started. For me, it was like just having a notepad or a Google Doc around in the beginning where I would just constantly jot down, okay, this is something I learned or something I observed or something that worked well, something that was unintuitive but worked, and just start creating a log of that. And then once I had enough of those, and I'd spend basically every week, I'd block off a few hours to go a little bit deeper. I'd take a lot of those bullet points and try to say, "Is there enough here to turn this into maybe a post or something that can be shared broadly?" And in the beginning I didn't have enough. It was all sort of scattered thoughts. But over time you start accumulating some interesting themes. And then I would start stress testing some of that.

**Grant Lee** (00:30:45):
So I would tell my teammates like, "Hey, this is something interesting. Do you find this interesting?" And if there were enough like, "Oh yeah, I would not have expected that", or, "That's not something I've ever heard before," then I'd actually start crafting the initial post. And then you actually just put it out there. I think what I've learned is, even for LinkedIn versus Twitter, the audiences want different things. And so you almost have to then have different tones of voices or even nuggets or sharing. For me, I invested much more in LinkedIn early on just because it felt a little bit more natural for me. And then over time I said, "Okay, well I'm going to start packaging certain content for Twitter that's actually different than what I would post on LinkedIn." Sometimes on Twitter you get even more tactical or even more into the weeds. And so I found that to be helpful.

**Grant Lee** (00:31:27):
But honestly, I'm still learning. And so every time you post, you go back, after a couple of weeks you go and say, "Okay, what things are actually being engaged with? Are things actually creating ..." Ideally you're creating enough value where people are either bookmarking it, sharing it, retweeting it, these things that are signals for there's something valuable there. And then you just go back and you start collecting your own sort of, these are my all-star posts, these are the ones that I've actually broken through. And then you go back and try to understand, okay, what about that post do I think was actually useful? Was it the actual content? Was it the structure of the content? Was it some sort of contrarian advice? And you start thematically bunching that together such that as you're brainstorming every week, you just have a good sort of body of work to work off.

**Lenny Rachitsky** (00:32:10):
This is so interesting and valuable. So let me mirror back a few of the lessons that I heard here that I think is easy for people to miss. So one is just what to share. What I heard here, and I completely agree with this, and this is what I try to do, is pay attention to things you've learned, things that you find interesting, things that are unintuitive to you. Just have a doc and just put these there. And every time you learn something, find something interesting, just add it to the doc. Or yeah, I haven't heard before is a good one too.

**Lenny Rachitsky** (00:32:39):
So it's essentially just like if you find it interesting, people on social media will also find it interesting. And one approach is just share it as it's happening, which is what I try to do. Just like, "Oh hey, just learned this thing with with clock code. Check it out." Or save it up for a big long post. The other interesting, I've never heard this before, of post different things to LinkedIn and Twitter. I just copy and paste the same thing. I love that you do something different for the two platforms.

**Grant Lee** (00:33:05):
I think we all kind of have intuition that there's just different audiences, right? And so if you know that kind of fundamentally, then the question is how do you package up the story the right way so that the audience is ready to receive it? And I think this can differ by the type of creator or the founder, whoever's posting it, and of course the actual content itself. And so for me, I'm still tweaking, but I do find that just copy and pasting from one to the other doesn't usually work. You almost need to be in the right mindset of, okay, what do I think will be more engaging on Twitter? And then what do I think will be more engaging on LinkedIn? And then test a bunch, see what actually works. Go back and iterate a little bit.

**Lenny Rachitsky** (00:33:49):
So if you had one bullet point tip for what works on Twitter versus LinkedIn, you shared maybe more tactical on Twitter, is there anything more there you can share?

**Grant Lee** (00:33:56):
Yeah, that's what I've found is tactical, oftentimes more contrarian on Twitter. And also I would say technical too. People really like to know, again, going back to getting into the weeds, is this something I feel like I could replicate? And I'm not going to give you ... There's no credibility if you just give a blanket statement or something that feels generic. I really need to know, if you could show me the metrics, even better. I feel like that ...

**Grant Lee** (00:34:22):
Versus LinkedIn, it's oftentimes more even just either more aspirational or aspirational or a topic or a theme that just feels relevant at that point in time. And you can just kind of make more of a broader statement. It doesn't need to be as tactical. It's more like inspirational, is like, "Oh, okay, now I need to go and learn a little bit more about pricing and packaging," for instance. And that could be the sort of spark that somebody needs. And you don't need to spell it completely out. Part of it's also that on LinkedIn you can't really do threads. And so doing a super long form post isn't as practical. Maybe that changes in the future, where maybe the tactical pieces, that element might actually change.

**Lenny Rachitsky** (00:35:01):
And last piece is you said you just block off time. Is there a specific time of the week you do this? How do you actually ... Because everyone's like, "Oh sure, I'll block off time. And then, I don't know, okay, but I actually got to do all this other stuff so I'm not going to use it this time or maybe next week."

**Grant Lee** (00:35:12):
For me, it's usually two times of the day, very first thing in the morning and last thing at night. And partly it's because of kids. It's almost like I need time where there's just zero distraction and there's no noise in the house, and so I can actually think. And then I think in the mornings it's about where are you finding inspiration, what are topics you're energized by? And then I think at night it's about reflection. What are the things you actually went through that day? You can almost pull up your calendar and be like, "Okay, I talked to X, Y and Z people. And was there anything from those conversations that might be relevant?" That's where I write some of those things. It's more of a recap of actually what happened.

**Lenny Rachitsky** (00:35:50):
And what helps me to not feel like this is some cringey self-promo egotistical stuff is just it's useful stuff that I've learned that ends up being helpful to people. And people in the comments are always just like, "Oh, that is really cool and useful. Thank you." It's not like self-promotion-

**Grant Lee** (00:35:51):
Totally.

**Lenny Rachitsky** (00:36:05):
... it's not just like, "Look how amazing I am. Check out my amazing products." Like, "Here's a thing I learned. You might find it useful."

**Grant Lee** (00:36:11):
That's exactly right. I think one way of thinking about it, with founder-led sales, it's always about exchange of value, right? You want to be able to give the customer this feeling that they're getting an amazing product. In exchange they're going to pay you money for it. I think with founder-led marketing, it's almost this mindset of you want to give people a ton of content, maybe it's a value in the content, so you're sharing something, maybe some secret tactic or you're giving them something where inherently there's value in it, and in exchange you sort of get goodwill back. You're not necessarily getting money back, you get goodwill. They're going to follow you, they're going to engage with your post, they're going to tell others about it. And then over time you can exchange maybe some of that goodwill for actually talking about my product and announcing it and they're going to help amplify the news. And I think that's magic, where you kind of bank the goodwill for a long period of time by providing just a ton of value with no expectation of anything immediately in return.

**Lenny Rachitsky** (00:37:06):
The book I always point people to when they're struggling with this sort of thing and like, "Okay, I did this and no one cared, didn't do any good," is there's a book by Scott Pressfield, I think is his name, called Nobody Wants to Read Your Shit, which is exactly what is right. Nobody wants to read it. The bar for people to care is very high. There's so much stuff to read and process. And so this book gives you a really good lens of just like, okay, the bar is very high and nobody wants to read your shit, so you have to try really hard to make it really good.

**Grant Lee** (00:37:38):
Great reminder.

**Lenny Rachitsky** (00:37:39):
We'll link to that in the show notes. Okay, let's come back to the growth of Gamma. So we've talked about how you got your first tens of thousands of users, essentially product hunts, rethinking, onboarding, making it really magical, and then this very controversial tweet that Paul Graham commented, created some buzz. Let's talk about the next phase and maybe, I don't know, tell us kind of the ARR at that point through 100 million. Just broadly, what should we know?

**Grant Lee** (00:38:07):
So when we got to about 10 million in ARR, I think there was this feeling for me, which was we knew we needed ways to help just continue to amplify and spread the word about Gamma. I think it was already working in terms of the organic virality was there, and so we did feel like it was time to start amplifying some of this. And I think the main blocker of my mind that I started feeling was that our initial brand was holding us back. And I think a lot of people will discount whether or not a rebrand is valuable. And I think sometimes it is, sometimes it isn't.

**Grant Lee** (00:38:39):
For us, there's a few different things we looked at. So one, our initial brand was almost more of a placeholder brand because we created it the moment we incorporated the company, which was again late 2020, beginning of 2021, where we needed something so that as we built, we could at least share it with people. We could put up a landing page and just feel like, okay, there's something here. But we didn't invest a whole lot into it. And so it was pretty limited in sort of what I call the DNA of the brand. There wasn't that many ... The art direction was very limited in scope. There wasn't much when it came to voice and tone.

**Grant Lee** (00:39:12):
And so it was something that we knew was good enough to start, but it wasn't scalable. And when I think about something that could be scalable, it's almost like you can take the ingredients of a brand and replicate it a ton, this DNA is something where you can imagine creating tons of content around and all feeling pretty cohesive. And I think that needs to be done by design. You're really being thoughtful about every single element. Like what is the art direction you want to go with? What is the voice and tone? Such that as you're creating thousands of pieces of copy, it all feels pretty cohesive.

And so we went back to the drawing board and we spent many months rethinking what would be the brand, what is this vision that we have longer term? Our creative director internally partner with Smith & Diction, an amazing agency that has helped folks like Perplexity also do their rebrand or their initial brand. And we [inaudible 00:40:05] many months just really trying to craft what we think is the core DNA of the brand, and doing so in a way that we could replicate it as much as possible.

**Grant Lee** (00:40:15):
Replication piece of it comes into play, because as you start scaling you're going to have to create a ton of content, your own content on social media, ads for performance marketing, assets for influencers to be able to use and showcase in their content. And so you're going from tiny pieces of content to all of a sudden every week we're testing thousands of pieces of creative. And you cannot do that if you don't feel confident that as you're replicating, you have that sort of cohesive feel. So for us that we realized it was going to be necessary and it's why we invested so much. It ended up being way more expensive, way more time consuming than I would've imagined. But I think coming on the other side of it being the right investment, feeling that that was the right time to do it.

**Lenny Rachitsky** (00:40:55):
I love how many things you did that feel like this will not work out. Building a startup within the presentation space, doing a whole rebrand in the middle of scaling, also just reworking the entire product after you launched and just rethinking the whole thing. All these things and everyone's always like, "No, this is not how we win." And interestingly, worked out for you guys.

**Lenny Rachitsky** (00:41:19):
I want to come back to the brand stuff, but one of the most interesting ways you guys grew early on was influencer marketing, which a lot of people hear about and talk about. I haven't heard much of how to actually do this and what actually works. Talk about that as a broad growth lever for you guys. And then I want to get into just what tools did you use, who actually was really helpful there, thing like that. So yeah, just give us the big picture.

**Grant Lee** (00:41:44):
Yeah. I think a lot of founders assume that with influencer marketing it's almost like turnkey. You set aside a budget, you find some creators, you figure out the right campaign or the right moment of time to do it and it's all done, you're ready to go. And I think the reality is, going back to this founder-led marketing mindset is like, well, you're going to set yourself up for success if you actually are super involved in that entire process. So for us, what this meant was all the initial influencers I onboarded manually myself. I would jump on a call with each one of them so that they understood what Gamma represented, how to use the product. You want to be able to have them tell your story but in their voice, right? And they can't do that if you're not willing to put in that investment.

**Grant Lee** (00:42:32):
And so we would spend a lot of time going through ... It wasn't my job to tell them how to pitch Gamma, but it was my job to make sure that they understood what Gamma was as a product. And so we'd spend a lot of time, like me just walking through the product, them asking questions, us just kind of brainstorming what could the hooks be, and me just giving them some initial feedback and saying, "Oh yeah, this one, I love that. I figure it's going to work great for your audience." But not trying to be super prescriptive. And working with a ton of micro influencers, people that don't have massive followings but are committed to ... Going back to giving value to your audience. They're committed to giving value to their audiences. They want to be able to showcase tools that actually they would use or they are using. And how do you do that in an authentic way? You can't really fake that. You really need to spend the time doing that.

**Grant Lee** (00:43:16):
And just like you would onboard a customer, you onboard an influencer the same way. You want them to be an extension of your team. And I think they can feel whether or not you're willing to put in the work. And if you're not, then they're just going to treat it like any other project, ship it and be done with it. If you invest in that relationship, guess what? They'll be back to actually post about you again. And you're all of a sudden having this sort of this relationship that actually you can build over time. I think that's really where the magic is. Too many people discount that initial piece.

**Lenny Rachitsky** (00:43:47):
This is awesome. To be clear, influencer marketing, essentially a person with a following on say TikTok, Instagram, Twitter, LinkedIn, whatever, gets paid in some way to promote your product. That's the simple way to understand influencer marketing, yeah?

**Grant Lee** (00:44:00):
Yes, that's definitely the simple way. And I'd say there's definitely different levels. I think a lot of people think influencer marketing and they'll think these big trendy creators, people that have a million followers for instance. And the idea is that, okay, we're going to carve out a really big budget, we're going to choose five or six that we feel like are really the tastemakers in the space and put all of our money into just having them talk about our product. And I think usually this is kind of the wrong approach. Because many of them, they do have massive audiences. And for you, you basically give them a script to read and it immediately feels like an ad. That product is not connected really to them in any way. It's just something that they're ... For this week they happen to be working with you and then they move on with their life. And it never feels organic or authentic and you wasted a ton of money doing so.

**Grant Lee** (00:44:55):
I think you're much better doing the hard thing, which is hard to scale, but it's finding the thousands of micro influencers that have an audience where your product maybe is actually useful. And for instance, for us early on it'd be educators, people that for them, part of their job is creating slides every day because they need to engage their students. And so for them, having a tool that actually saved them a ton of time was something they love talking about. And if you can find some of these pockets, we call them echo chambers, where if you find a pocket like educators, teachers love telling other teachers about products they love using. During summer break, they all come together and talk about, "Okay, what are the things that are going to actually improve my job next school season?" And obviously during this AI wave, a lot of those have been, "Okay, what are the AI tools that just save me a ton of time?"

**Grant Lee** (00:45:43):
And so if you can start actually tapping into these pockets of echo chambers, that's even better. It doesn't have to be this flashy, well-known influencer. It's actually just this person that has an audience where people really trust what they say. And that's amazing. That ends up becoming this sort of wildfire that can spread really, really fast.

**Lenny Rachitsky** (00:46:03):
And what's the dollar amount these folks get? It's like a few hundred bucks, a few thousand bucks, something like that?

**Grant Lee** (00:46:07):
Yeah, few hundred to low thousands, low single digit thousands.

**Lenny Rachitsky** (00:46:11):
How do you find these folks? Is there tools that you use? Is it just like a bunch of manual searching and looking?

**Grant Lee** (00:46:17):
Yeah, in the very beginning it was all manual, a lot of cold outreach. And then we ended up finding a couple of different things. One is a platform, a YC company called 1stCollab. That has been amazing. They basically do all of the automated outbound for you. Plus you can help them actually create profiles or personas of different creators, so for instance educators being one, and then they'll go out and actually, based on that profile, find all the right creators for you. So they've been amazing, really great to work with.

**Grant Lee** (00:46:46):
And then we've also found small agencies to also help kind of augment that. I look for agencies that are super young and hungry. These are people that they're native to social media and so they really understand it. And they can really be able to bring in creators that are great to work with. And I think part of it is if you find creators that are great to work with, everything else becomes easy. So we've had a few, one is AKG Media, actually out in the UK, and they've been fantastic to work with as well. So you kind of find a few different things, either agencies or platforms that can help you actually scale this thing up.

**Lenny Rachitsky** (00:47:22):
And when they post, they're generally transparent about this is a paid promotion, right?

**Grant Lee** (00:47:23):
Yes.

**Lenny Rachitsky** (00:47:27):
They're not just pretending, "I found this tool and I love it"? Cool.

**Grant Lee** (00:47:31):
Yeah, exactly right.

**Lenny Rachitsky** (00:47:32):
Okay. And so how much impact did this lever of influencer marketing have on your growth, say from 10 to 100? Is this the biggest lever of growth other than just word-of-mouth, people continue to share it?

**Grant Lee** (00:47:44):
Yeah, so word-of-mouth has definitely been the biggest. So we look at all new subscriber growth, over 50% of this is word-of-mouth. It's either people searching direct, coming direct, entering Gamma.op, or going through search and typing in Gamma, like a branded keyword search, where they're looking for Gamma, they've heard about Gamma. But I think for us, social media and influencers specifically has always been an amplifier. So every time we invest in influencer marketing, we actually see word-of-mouth increase even more. And it's always like you can just see it. Basically anytime you spend a little bit of money, you start seeing people come through influencer, the word-of-mouth factor actually will get another 1.5 additional users on top of that, which has been really interesting to see.

**Grant Lee** (00:48:29):
And I think part of this is just recognizing that ... And I think we kind of understand it, but with influencer marketing, why it's so effective, we all know Dunbar's number, which is have this number of 150 people that you call kind of your network. And your network you trust more than the average stranger down the street. If they tell you something, they recommend something, there is a sort of halo effect, you learn to trust them. A lot of these influencers, the reason why they share so much about their lives is because they want to be in your network. They want you to feel super close. And once you feel super close, you trust them to actually share things that are going to be useful.

**Grant Lee** (00:49:05):
And so when they recommend a tool, there is a sort of halo effect, where it doesn't feel like it's coming from a stranger, it feels like it's coming from a friend. And that's where every time we've spent money there you actually see this amplifications, like, "Okay, that's kind of interesting." You wouldn't necessarily expect that. But for us it's been this sort of amplifier from the very beginning.

**Lenny Rachitsky** (00:49:24):
This is so fun to hear about. I've not heard this level of detail on how influencer marketing works and how to make it work. A few more questions here. So you said there's maybe a few thousand people you ended up working with, roughly, influencers?

**Grant Lee** (00:49:35):
Over the course of a year. It wasn't all in the same time. Basically in the beginning you do ... In the very, very beginning we had a small budget. It was like 20 creators a month. And then you start increasing that to like 50, then 100. We're definitely not fully scaled at this point, but I could easily see a point where you're working with many, many creators every single month. And that gives you a chance to actually test a variety of, again, content hooks, ways to talk about the product, value props.

**Lenny Rachitsky** (00:50:03):
Amazing. And you said the key here is you spend time with every one of these creators, influencers early on to help make sure they understand what you're doing and get excited about it. It isn't just like a thing you outsource.

**Grant Lee** (00:50:16):
I think there's a lot of value there. It's again hard to quantify. And most founders probably feel like they're too busy to allocate that time. But I think it was a good investment. Because going back to you want them to feel like an extension of your team. They're not going to feel like an extension of your team if, one, they've never met you, and two, you've never even told them really how the product works. They're forced to go to your website to figure it all out. Those are going to be not a whole lot of love that they're feeling from the outset.

**Lenny Rachitsky** (00:50:47):
So what I'm hearing is quality over quantity, especially when you're getting started. And then there's this other piece of niche which I think is very counterintuitive. Instead of going to large influencers with a huge audience, it's good with folks that are small. What's like a audience size roughly that you think is ideal for this, or what niche just means?

**Grant Lee** (00:51:05):
Honestly, I don't think there's a minimum, because even with platforms like TikTok, they oftentimes are giving you credit for a brand new account. They want to help amplify that new account, because obviously if they're thinking from a creator perspective, if that new creator feels like, "Oh, coming to TikTok is a massive win for me," they're going to be more invested in it. And so there really isn't a minimum. A lot of these platforms are trying to shift to kind of the same thing, where they really reward new creators on the platform. And so you could have a small audience, it doesn't really matter. You could have 10,000 followers, that's also good. I think as long as the content feels, again, engaging, authentic to the people you're talking to I think it has a really good chance of actually taking off.

**Lenny Rachitsky** (00:51:46):
That's such a good point with TikTok, where it's not follower related, if it's useful and people find it clickable or whatever, likable, viewable, the algorithm will spread it to a lot of people. Such a good point.

**Grant Lee** (00:51:59):
Totally.

**Lenny Rachitsky** (00:52:00):
Okay, there's a couple more points you made in the tweet that I want to make sure we highlight. So one is you made this point that 90% of your reach in influencer marketing comes from less than 10% of people. Is there anything there that you think is important for people to hear?

**Grant Lee** (00:52:11):
Yeah, I mean this just goes back to it's hard to know where that 10% is going to come from. So you kind of just have to cast a super wide net. You can sort of, I think try to, again, trick yourself into thinking you're great at picking creators or you're great at telling them how to post about your stuff. But the reality is, even for me, I could never guess. I kind of had some idea, but I just had to make sure that I was meeting enough creators broadly such that when you meet enough, they're all posting, there's some pocket that end up just taking off. And I was not a good predictor of that. I was not smart enough to actually know which ones would take off. I just knew that we had to play the sort of numbers game to make it work.

**Lenny Rachitsky** (00:52:49):
So one of your tips here is people fail often in this because they start too small or their budget's a little too small. You recommend 10,000 to 20,000 a month over six months and just trying this. It sounds like you're doing like 20 to 30 creators a month. Is that the right framework for how to just start this thing and explore?

**Grant Lee** (00:53:07):
Yeah, totally. And I'd say it's not just that the budget's too small, it would be that they put all their eggs in one basket. So you can also easily blow that 20 K on just one bigger influencer and then be like, "Oh, that didn't work, so I'm going to try it again. That didn't work. Okay, I give up." And I think rather than doing that, you should be like, "Okay, that 20 K, I could actually probably work with 40 different influencers and see what actually works." And across the 40, I'm going to try to find a variety of personas, a variety of use cases, spend a lot of time with them, and then actually see what's working, what's not.

**Grant Lee** (00:53:39):
And then next month take those learnings and double down. If educators are working, go with educators. If consultants are working, go with consultants, find more consultants, there's going to be more there. I think that's where the putting all your eggs in one basket is just probably the surest way to fail, because you're going to miss a ton and it's going to take you way too long to learn and you're going to come to the conclusion that it's a waste of time.

**Lenny Rachitsky** (00:54:00):
I feel like this whole podcast conversation could be just about this one lever. So much I want to keep talking about, but there's more questions here because this is so useful and interesting. You had this line in your tweet about how reality is not an accident and that this approach is how you figure out what actually works. And then once you do that, then you start leading into that messaging. Talk about that.

**Grant Lee** (00:54:20):
This goes back to obviously if you're testing a bunch, you'll finally find that sort of post or set of posts that actually go viral. Going back to just the fundamentals, like make it easy for your influencers to be able to tell your story in their voice. One thing we did, we open sourced basically our entire brand. We have Brand.gamma.app, which is everything about our brand, our voice and tone, our art direction, what we use in mid-journey to create the sort of art direction that we have so that a creator can do the same, right? And they can actually just copy all of that so that they don't have to reinvent the wheel every time they're trying to post about Gamma. They have all of that.

**Grant Lee** (00:54:56):
And I think going back to this notion of just make it dead simple for them, remove friction. They already have enough on their plate to have to figure out. Don't make it any harder than it is. And if you remove friction, then it's like, oh, you get into this rhythm of adding creators is easy, having them post is easy, reviewing what's working, what's not is easy. And if you're able to do that relentlessly over many, many months, then all of a sudden hitting the sort of viral post is easy because you're going to have enough bats there where some are actually going to pop off. But you only can get there after you've done all the hard work before that to remove friction from the process. And feeling like it's almost like a well-oiled machine at that point.

**Lenny Rachitsky** (00:55:38):
Is there a platform you find most helpful for the stuff you guys are doing? Is it like TikTok, Instagram, LinkedIn, something else?

**Grant Lee** (00:55:45):
Yeah, I mean this is one where for us we cast a pretty wide net too. But it's very clear, LinkedIn, the conversion rates are just substantially higher. They're 4X, maybe 5X higher than other platforms. And I think a lot of people are probably still sleeping on LinkedIn, frankly. And so it's one where some of the influencers there or creators there can be a little bit more costly. But if you can eventually be more targeted knowing that, hey, this type of creator is pretty impactful for our product, then working with them, it's just like, "Oh, that's great." The conversion rates are just so strong and it really feels like we're just getting started there.

**Grant Lee** (00:56:22):
So if in the beginning you're not sure, it's always helpful to cast a pretty wide net. And then similar to just the influencer strategy, like test and iterate, you'll figure out ... Many of these things will follow a power law. So it's like one or two channels are going to be the most important for you. For instance, Twitter for us hasn't been that impactful. And I think for tools like Notion, they've been really, really impactful. You're not going to really know, and so just test and then double down on the ones that really move the needle for your product.

**Lenny Rachitsky** (00:56:48):
I think that many people listening now are like, "Wait, LinkedIn posts are sponsored sometimes? I didn't know that." How do you know if it's a sponsored post? Is it like a #sponsored or something like that? How do they communicate this?

**Grant Lee** (00:56:57):
Usually they'll say they're a partner or yeah, basically it is sponsored in some form, or they'll #ad or something along those lines. So that's probably the way you'll see it the most.

**Lenny Rachitsky** (00:57:12):
Cool. By the way, I don't do this sort of stuff. Have you ever seen me on LinkedIn? I'm not doing any paid stuff. It's just so people know. And I don't plan to do that.

**Lenny Rachitsky** (00:58:00):
... Often share Miro templates. I use it all the time to brainstorm ideas with my team. Teams especially can work with Miro AI to turn unstructured data, like sticky notes or screenshots, into usable diagrams, product briefs, data tables, and prototypes in minutes. You don't have to be an AI master or to toggle yet another tool. The work you're already doing in Miro's Canvas is the prompt. Help your teams get great work done with Miro. Check it out at miro.com/Lenny. That's M-I-R-O .com/Lenny. Let's come back to this brand point. So, one of your big lessons is invest in brand before you go heavy into hit paid ads and performance marketing. I imagine you do some ads at this point on Facebook and Google and things like that.

**Grant Lee** (00:58:42):
Yeah, we run ads performance marketing. I think there's this stigma that brand marketing and performance marketing are sort of at odds with each other. I very much follow the sort of thought that brand marketing is performance marketing. Everything is some form of performance marketing. It just might not be as attributable, so the ability to actually map back to every single dollar spent is a little bit harder, but it doesn't mean that it's not impactful.

**Grant Lee** (00:59:12):
And I think as a company scales, you have to invest in both, and ideally they work really, really well together. The more you invest in brand marketing, it strengthens your performance marketing. This goes back to having enough creative to even test. If you're too limited in scope and you don't have a brand, you feel like you can actually amplify your handicapping, your ability to actually have a good performance marketing program.

**Lenny Rachitsky** (00:59:32):
I love that heuristic of how do you know if you are under invested in brand is if you're limited in the number of ideas you can try in performance marketing.

**Grant Lee** (00:59:32):
Totally.

**Lenny Rachitsky** (00:59:43):
Is your design system just... Is everyone having to redesign things from scratch and come up with all these frameworks every time they run an ad?

**Grant Lee** (00:59:52):
Exactly. Yeah, yeah. Basically you kind of have a feeling for, "If I were to scale this up to a thousand pieces of creative, would it still feel cohesive or is it kind of all over the place?" And if it feels like it's all over the place, then you kind of have to go back to the drawing board.

**Lenny Rachitsky** (01:00:05):
You said when you talked about the rebrand, that it took a lot longer than you expected, that it was more expensive than you expected. That's the fear, I think, everyone has when they hear this like, "Oh, I don't have time for a rebrand." I also imagine, because your product is so visual, that it makes more sense to invest there and to spend the time and money.

**Lenny Rachitsky** (01:00:23):
For the typical founder, do you have any just, I don't know, thoughts of just like, "Here's when it makes sense. Here's a sign you really need to invest here heavily." Versus, "It'll probably be all right."

**Grant Lee** (01:00:32):
Yeah. I mean, I do think it's probably more geared toward anything that's a little bit more prosumer or consumer because so much of your product, you're trying to create this feeling for a user, what are they experiencing. And the experience happens way before they even drop into your product. It might be they see an ad or they see a billboard or they see something. It's like, "Okay, that piqued my interest a little bit."

**Grant Lee** (01:00:56):
And then you need some sort of symmetric messaging in that they see there's some symmetry in that, they see the ad, then they drop into the product or they land on your website, it feels cohesive and it feels like, "Okay, this is interesting. I'm going to go all the way through to sign up and then maybe actually start using the product."

**Grant Lee** (01:01:12):
That's a little bit different when it's a B2B product or where there isn't as much reliance on that initial moment. They might just hear about it through a colleague and then sign up for it and then go through a huge procurement process and then it's like, "Okay, maybe it matters, but probably not so much as for a product where the brand can have so many different touch points."

**Lenny Rachitsky** (01:01:33):
I want to talk about some broader things that have worked to help you grow, but before we do that, I just want to visualize the pie chart of how Gamma grows. Say, post-10 million ARR. If I have it correctly in my head, it's over 50% just word of mouth, organic, people are sharing it, doing presentations for each other, "Oh, it's Gamma. Just go check it out, sign up." Then it feels like the second-biggest bucket is influencer marketing, social stuff. And then is the third performance/paid marketing?

**Grant Lee** (01:02:03):
Yep, that's right. Yeah.

**Lenny Rachitsky** (01:02:04):
Cool. On that last piece, is there anything else there for people that are starting to explore performance marketing, essentially Facebook Ads, Google Ads, all these other platforms, is there anything else that you think might be helpful for people to hear or learn just to get started down this road?

**Grant Lee** (01:02:19):
I would just have two recommendations. One, going back to my initial piece of advice, which is don't invest until you have word of mouth. Don't fool yourself into thinking that you'll solve other problems by just starting to ramp up a performance marketing program. Just get the word of mouthpiece first, so that you're coming into this program with some tailwinds and then start ramping it up.

The second piece is: set some constraints. You don't want your product to be at a point where more than 50% of your acquisitions are coming through paid acquisition. I think if that is happening, your core growth engine is broken. And it feeds right back to point number one, which is if your core growth engine is broken, you just have this leaky bucket. You're trying to spend so much money building top of the funnel, people are not making it all the way through. Something else should be fixed before you really try to dial it up. And it doesn't mean you don't spend a little bit of money, but just don't dial it up until you feel like your core growth engine is actually working.

**Lenny Rachitsky** (01:03:17):
When you said the first point about wait until you have word of mouth before investing in performance marketing, is that essentially a large chunk of your growth should be coming from word of mouth, direct, organic?

**Grant Lee** (01:03:29):
Yeah, yeah. And for us, even at scale, again going back to more than 50% of new signups still come through word of mouth. That, for us, is a sign like, "Okay, if something is still working, people are using the product, telling other people, then you want that feeling before you really start dialing anything else up."

**Lenny Rachitsky** (01:03:45):
Is there a percentage that you think is helpful for people to think about just... Is it 25% or more something like that or just word of mouth for you to feel like, "Okay, we actually have organic growth as a major growth engine"?

**Grant Lee** (01:03:55):
Yeah, I think this comes back to maybe just how maybe aggressive you want to be. I think just rough heuristic is the more, the better. If it's over 50%, I think that's great. If it's approaching that, good. And just going back to, don't fool yourself into thinking just ads is going to be the way you grow. Because you can do that, but everything else becomes harder and harder. 

**Grant Lee** (01:04:16):
If you rely on paid acquisition to be the main growth engine, you should be prepared for things like CAC, like customer acquisition costs, to keep going up. The more you're trying to reach a new audience, it gets more and more expensive. So, don't assume it's going to be flat, and then all of a sudden you're running on this treadmill that's actually running faster and faster. And so that's where it's easy to get hooked on that early on when you're just investing a small amount of money, and then it's almost impossible to get off that treadmill when you're too far into it. So, anticipate that and give yourself a better chance at actually being able to sustain that growth long-term.

**Lenny Rachitsky** (01:04:50):
Important advice. Okay. There's a couple more elements you've shared that were key to Gamma's growth. One is sharing prototypes with users before you ship. What does that look like? What does that mean? Why is that so powerful?

**Grant Lee** (01:05:05):
Yeah. I mean, this for us was a huge unlock. Going back to early days, when you're just trying to get your product into anybody's hands, you're getting to your friends trying to use it, and again, they're going to lie to you, tell you how great it is, and then never come back to using the product. 

**Grant Lee** (01:05:19):
I think what you want to be able to do, the ideal situation is, you recruit a bunch of people that are legitimately good prospective users or customers of your product, but have zero skin in the game. They do not care at all whether or not your product succeeds. They're just in it to test it. And for us, it was people that have made slide decks or make slide decks regularly, let's drop them into Gamma, give them very little context. We just tell you, "Hey, this is an alternative to PowerPoint. Go ahead and try it."

**Grant Lee** (01:05:48):
And then as you're going through the onboarding flow and testing the product, just tell us everything that's going on in your head, describe what you're seeing, tell us what you're trying to do. We'll give you maybe a few different tasks like, "Oh, create a piece of content." And when you watch them do that and also hear what they're saying, you just immediately feel the pain. 

**Grant Lee** (01:06:10):
All the places they're struggling and all the places are so confused by what they're seeing and you sort of then can internalize that pain and say, "Okay, we're going back and we have to fix this. This is not usable." We, oftentimes, are dog-fooding everything. And so you can get to the point where you're so familiar with your own product, everything feels kind of easy and you know where things are, but when you start actually hearing other people describe their usage of the product, that's a gift.

**Grant Lee** (01:06:36):
You're all of a sudden you're like, "Okay, now I know where to actually spend the time." People aren't even getting to the third screen. They're stuck on the first screen because they can't even find the button that we think is so dead obvious, and so let's go back and actually re-engineer, re-architect that piece of it.

And we've done that for everything: landing pages, onboarding experience, new feature launches, export, sharing, every single piece of that such that we can actually see where things are working or not. And then every time, we'll learn something that's kind of painful, but obvious that we need to fix and we go back and fix it.

**Lenny Rachitsky** (01:07:08):
How do you scale this sort of thing? How do you run every new big idea, new feature change by people? Do you have a closed beta group, a decent platform? How do you actually do this?

**Grant Lee** (01:07:19):
There's a couple of great platforms. Voicepanel, which is also YC company, full disclosure, angel investor, and then there's also platforms like UserTesting, that really help you source and find these people that fit again, that persona or profile you're looking for. So, in our case, people that are in a specific job function or create decks regularly, and so you can use those platforms to really scale up these programs pretty fast.

**Grant Lee** (01:07:45):
And then once your team knows how to use those platforms... We would have an idea in the morning and in the afternoon, we're already running a pretty full scale experiment or a research study, and by the evening or by the next day, we can actually go through all of it together. So, it's pretty fast once you have it set up. It's more about how do you get the program onboarded the right way. 

**Grant Lee** (01:08:06):
I think the other sort of mechanism we did early on was once we had some repeat users, we created sort of a program for our power users. We called it the Gammaster Program, which is we put them into a separate Slack workspace, and that was a place for us to share very, very early prototypes like wireframes, sometimes they were functional prototypes, and get them to get some initial feedback as well. This definitely helps with more later stage or features or things that aren't going to be necessarily important as part of onboarding, but once you understand Gamma, like, "Oh, how do you share it?" 

**Grant Lee** (01:08:41):
For instance, this was a great way to start testing some of that because they already understand Gamma, but now you're adding that new functionality. And then we can also watch them struggle and hear how they're struggling with the product. And so that has been a great way just to have a separate Slack workspace. Anytime we're thinking about something, they're the first to hear about it. Give us some early indication if we're on the right track or not.

**Lenny Rachitsky** (01:09:00):
I love this workflow that you shared of you have an idea in the morning, you built a prototype. Do you built it with AI prototyping tools like Cursor, Lovable, something like that?

**Grant Lee** (01:09:11):
Yep. That or it could be... Yeah, I mean we're lucky a lot of our designers also know how to code, so even before the recent tools, come up with some sort of functional prototype and then be able to ship that so people can start playing with it.

**Lenny Rachitsky** (01:09:24):
Yeah. So, you have an idea in the morning, you build a prototype using various tools. By the end of the day, you are getting feedback from real people using one of these platforms, Voicepanel or UserTesting, and just that loop saves you, I imagine, potentially months of just building the thing nobody wants and shipping it, launching it, and then just failing. 

**Grant Lee** (01:09:45):
Totally. Yeah. I think this is even more probably helpful for certainly a lot of folks that are starting to do much with vibe-coded apps. Yeah, that's great. You've lowered the amount of time to get something out there. Now, prove that it's useful to some set of users, and this should be again, every day, every week, you should be able to go through a ton of these, and then build on the things that seem to be working. 

**Grant Lee** (01:10:08):
I feel like that's almost like a way to speed run a lot of that early... you're in the idea maze, you think you have something that could kind of work. How do you actually break through so that people are actually finding value in what you're building?

**Lenny Rachitsky** (01:10:21):
Yeah, I love it because so many people hear this idea of just run your stuff by users before you do the user research. It sounds so hard and heavy lift-y, and the way you're describing it is very automated, very quick to do. You don't have to go think about finding random people in a coffee shop. It's just like these platforms exist where you could go plug in your thing, get feedback by the end of the day.

**Grant Lee** (01:10:44):
Totally. Yeah, and that helps you just move way faster.

**Lenny Rachitsky** (01:10:48):
Is there a feature that you were super excited about that you built and ran through this and just like, "Okay, that was a huge failure"?

**Grant Lee** (01:10:54):
I don't think any of the ones where we... We always try to chunk it down. So, none of the things we're testing earlier on are these massive features. They're always an inception or a starting point of, "This could be something interesting."

**Grant Lee** (01:11:07):
And then we take that initial learning and actually then build the product around it. It's never like, "Oh, we spent four months working on this thing. Let's see if anybody actually wants it." It's almost like we always start super early. And then a bunch of ideas die right away, but they're still pretty small ideas. 

**Grant Lee** (01:11:20):
And then the ones that kind of pass the initial test, you start building towards something that could be, hopefully, more game-changing or much more value-add. And by the time you're actually shipping, you've gone through 10 different layers of actually testing and iteration before it actually sees the light of day.

**Lenny Rachitsky** (01:11:34):
What's really interesting about you and the way your company operates, you guys are ex-Optimizely people, so you're very versed in experimentation. A lot of people talk about A/B testing experimentation as something that doesn't make sense for a startup because the scale is so low. What I'm hearing here, and I know this is something you talk about, is just there is actually a lot of ways to think experimentally, even in the early stages. Is there something more there you think is important for people to hear?

**Grant Lee** (01:12:01):
Yeah. I really think it's more of... The mindset you go into to almost any problem or opportunity is the saying of, "Strong opinions weakly held." I think it's totally fine to have some of these assumptions or hypotheses going into a lot of these things, but you should always know that there's a way to start trying to validate some of this.

**Grant Lee** (01:12:20):
As a founder, you're always trying to build conviction, and so you build conviction by not having it all live in your head, but getting it out there and start testing this with users, prospect customers, and starting to see what are the things that actually feel right. And sometimes you have enough data to be statistically significant. Sometimes it's more of a, "Hey, we at least were able to gut check this a little bit and get some qualitative feedback." I think that's still valuable. It shouldn't be this sort of all or nothing like, "It has to be static for it to be useful." I don't think that's true. 

**Grant Lee** (01:12:49):
In fact, I'll just share the very early story of when we first started Gamma, we knew we wanted to help change the way people communicate, and we actually had two different ideas we were parallel pathing kind of at the same time. Of course we had presentations, reimagining how people were creating presentations, and we also actually had this separate idea, which we called The Lobby, which is a virtual office. This is a place where this is, again, peak pandemic, much more hybrid work, much more virtual work. And so this was a place where everybody on the team, whether you're in the same office or not, could gather, collaborate to feel pretty magical. 

**Grant Lee** (01:13:26):
And we worked on both for six months. We worked on both the virtual office and the presentation product for six months. We would dog-food both. So, oftentimes we'd be in the virtual office, presenting the presentation tool and kind of use both. And after six months, we came to this conclusion that we wanted to go all in on the presentation tool. And the reason was, when we looked at the virtual office tool, we were sort of always competing against what we thought was something we'd never be able to surpass, which is in real life experience, actually being able to work shoulder to shoulder with your colleagues and having this environment where you really feel like you're building together. 

**Grant Lee** (01:14:05):
Virtual office could get pretty good, but we would never beat that. And so we were almost limited in our own imagination about how good could this product be versus the presentation product. We ended up with a million more ideas we thought we could actually introduce that could be better than how you work in PowerPoint today. We were just so energized by it. 

**Grant Lee** (01:14:25):
And so for us, that was this sort of A/B test of testing these two things that we invested equal amount of time into, and coming out at the other end realizing that, one, was going to be the product we were pour all of our blood, sweat and tears into making it great because we saw the potential in it.

**Lenny Rachitsky** (01:14:43):
I didn't know that about you guys, that you explored that other idea. There's so many startups that did that during COVID times, and like, "Okay, this is the future. We're all going to be remote, and let's work remotely in virtual offices." There's a startup and I'm a tiny investor in, Lindy, that is now a big AI agent company, and they did the same thing. I imagine that was their whole first concept. It was just these little avatars. It was like a little game where you walk around, go to little virtual meeting rooms. 

**Grant Lee** (01:15:07):
Yeah, it was a fun product to work on, but yeah.

**Lenny Rachitsky** (01:15:12):
Yeah, it's interesting how we just reverted back to the mean of just like, "Yeah, people are in offices again. That was a fun experience." Although things have changed. Just to highlight this point you made that I think is really powerful, I haven't heard it described this way before, just the power of testing prototypes very, very, very early, using these platforms that make it super easy.

**Lenny Rachitsky** (01:15:34):
We always hear, "Okay, you have a mock, you have a prototype. Yeah, testing with users always feels like this heavy thing. You got to have a user research team, go do interviews, do one-on-ones." What you're describing is something... I don't know why everyone's not doing this. With AI tools, it's so easy now. Have an idea, build the prototype, test it with, I don't know, is it like dozens of people? How many people actually run through a prototype on average?

**Grant Lee** (01:15:58):
20.

**Lenny Rachitsky** (01:15:59):
20 people. So, 20 people look at this thing, give you a bunch of feedback, you realize this was very dumb, or, "Here's the nugget that we want to lean into." And instead of building a thing, instead of doing all these user research sessions, things like that. Super cool. 

**Lenny Rachitsky** (01:16:12):
Okay. Is there any other big lesson or lever that has helped you grow to today's 100 million ARR, and $2 billion company? We talked about influencer marketing, we talked about testing prototypes, investing in brand and a little bit of paid growth. Anything else?

**Grant Lee** (01:16:33):
Certainly for us, the ability to adapt and move fast in this environment. I attribute a lot of what we've been able to accomplish to a few things. One is we do have a small team and a lean team, one that's able to move really fast. I think that means, by default, we have to look for a lot of levers where a small team can do a lot of different things. 

**Grant Lee** (01:16:54):
So, we can talk, one, obviously about how do you construct a team like ours and what I think has worked. And then two, going back to experimentation being this sort of thing where for us it's been a huge unlock because it allows you to not only test things and iterate a ton, it allows you to build much more efficiently. And so we've been in a startup where we've had great unit economics from the very beginning. We run profitably, we have really strong margins. 

**Grant Lee** (01:17:22):
I don't think that would be possible if experimentation wasn't core infrastructure to us because the temptation would be you just throw the most expensive model at every job and assume that's going to work. And the reality is that never is the case. And so for us to be able to actually test across 20 different models in production today, always trying to align the kind of value we're delivering to our customers with our ability to actually scale this operation, that's been in the background. 

**Grant Lee** (01:17:48):
And not always things that are easily quantifiable or things that you're sharing broadly, but it is core to our DNA. And again, going back to a team that came from Optimizely, probably not surprising, but I do think that's been part of our ability to actually execute at this level.

**Lenny Rachitsky** (01:18:03):
I love that the product is so beautiful and such a good experience. It's such a good example of experimentation and being really obsessed with running experiments, A/B tests. Data can create really beautiful products and experiences that aren't just feeling like some kind of micro-optimized flow.

**Grant Lee** (01:18:19):
Totally. Yeah. And just going back to one part of the team, part of that plays a huge role, which is you want to build a product where people talk about taste and brand and all these things, what they emote. I'm not going to throw in whether or not, yes or no, but I do think it makes a difference. And for us, at some point, more than a quarter of a team or about a quarter of our team was product designers. I think that's an unintuitive level of investment, or at least that's not common. You don't see many startups at our stage or early stage where a quarter of the team's product designers. 

**Grant Lee** (01:18:55):
And I think that was an important investment for us because when you think about with AI companies, so many companies are trying to invent new surface areas or new user experiences, and that's not possible if you're not really getting the foundation right, really thinking deeply about user problems and how can you solve them in a novel way. And so for us, we made that investment in the very beginning, even if it was counter to what other startups would do, because we actually felt like it was the right thing.

**Lenny Rachitsky** (01:19:21):
Okay. I definitely want to spend more time on hiring. Before we get to that, you've touched on this kind of concept that you guys are, what many described, a GPT wrapper company. You essentially sit on top of other models, do some cool stuff, charge for that. There's historically, and there's been a big shift here, historically, there's this sense that, "Okay, you're just this thin layer on top of the model. How is there any sort of motor leverage or long-term business model here when it's just the model that is doing the thing and charging you guys to do all this AI work?"

**Lenny Rachitsky** (01:19:57):
It feels like people have started to realize maybe that is the only place money will be made because the models are just so hard to compete with, and that's not a place you can build a business anymore. Talk about just what you've learned about this concept of being a GPT wrapper, and what people may not be understanding about the business opportunity there.

**Grant Lee** (01:20:15):
When you think about maybe just literally only being a wrapper on one model or one provider, yeah, maybe there's only limited amount of utility or value add. But then when you start thinking about, "Okay, I'm going to go really deep into this one workflow, and it's not just one model." It's maybe 20 plus models powering all different parts of the product, and then you're thinking about the orchestration that's required and you're thinking about, obviously if you're experimenting constantly being able to test across the newest models versus models that have been around that are cheaper, you're doing a lot to really... Your job is to, again, align value, maximize the value you're delivering to the end user in a way that's sustainable for you as a business. And so there's a lot more to that. 

**Grant Lee** (01:21:00):
And so for us, we've always been passionate about being very close to our customers, our users. Who, for them, their job to be done is visual communication, visual storytelling. And the default tool today is going into a PowerPoint or Google Slides where the amount of effort to create something... We've all been there late at night trying to format a deck, trying to find the right layout, all this manual and tedious work. 

**Grant Lee** (01:21:24):
Well, what if you could abstract all that away and give them something that feels a little bit more delightful, a little bit easier, much more effortless? What would that earn you in terms of a customer that wants to come back to your product? And so that's everything that we focus on is you need to go deep into the workflow, be empathetic to the user and their job that you're trying to solve, and of course, apply the best technology possible so that you're delivering on that promise of a product experience that's way better than the status quo. 

**Grant Lee** (01:21:52):
And I think if you can be laser-focused on, it doesn't matter if you're a wrapper or not, what is the technology you're doing and applying, that makes a real difference? So, that's the ultimate goal.

**Lenny Rachitsky** (01:22:01):
This is a really cool framework for how to think about what it takes to build a successful wrapper company that is... I don't know, I'll keep using that term. I don't know if it's [inaudible 01:22:09], but just some ideas don't work. These model companies are launching their own products here and there.

**Lenny Rachitsky** (01:22:16):
So, what I love about what you're describing here is almost like a heuristic that'll tell you if there's an opportunity/how to be successful as a GPT "wrapper company." I'm putting in air quotes. It sounded like maybe there's three here. But talk about just if you think about the bullet points of what you need to do to build a successful business on top of existing models.

**Grant Lee** (01:22:37):
I mean, I think the most important thing is before we even talk about the technology, so we're skipping to the part where you're trying to apply the most interesting models or technology to solve some problem. Start with solving the problem you actually care about. 

**Grant Lee** (01:22:53):
I think it's very tempting right now to chase shiny objects, like a founder might be able to gain some initial traction by literally being that GPT wrapper and solving any sort of problem. And you start to see some traction and then you just go with that. But before you do that, you should think about, "Is this a problem I can invest five to 10 years into actually solving? Do I care deeply enough about it?"

**Grant Lee** (01:23:13):
Because if you can't, you're never going to go... actually think about overcoming the variety of different hurdles, whether it's other startups or other incumbent tools that are trying to start doing the same thing. 

**Grant Lee** (01:23:24):
And so for us, we go back to... We've started this company because we really care about helping change the way people communicate their ideas. We really feel like this idea of visual communication, visual storytelling matters. It helps elevate everybody and it gives people much more opportunity to do the things that in the past if you weren't great at making slides, your ideas might've died, and someone else that was able to actually articulate their ideas better ended up winning the deal or winning the favor of their manager or their boss. 

**Grant Lee** (01:23:53):
And so we felt like that was the wrong, that didn't feel right, and so could we help democratize visual storytelling, visual communication? That was sort of our north star. And then you go back into thinking, "Okay, every step of the way, what are the tools and technology I can apply to help solve that and actually move the average person closer to that long-term vision of ours?"

**Grant Lee** (01:24:14):
And of course, AI has been, again, this gift where yes, you can apply that to solving this job. You can also apply that AI to many different jobs. Figure out what is the problem you care deeply enough to go deep, because going back to this sort of idea, you need to own the sort of end-to-end workflow. A customer needs to have... You want your product to live in their brain somewhere where when they think about, "Hey, I have to create a presentation." They come to you as the default tool.

**Grant Lee** (01:24:40):
And the moment they start creating it to the moment they ship it and send it to their boss, that end-to-end experience needs to feel magical, needs to feel great. And I don't think you can really do that unless it's actually a workflow you either know deeply yourself or you care deeply to actually help solve for somebody else.

**Lenny Rachitsky** (01:24:57):
So, some of the elements I'm hearing here is, one, is just like actually really care about solving this problem, not come from, "Oh wow, this cool thing happened and it worked. Wow, maybe I could sell this thing to people." Because you may end up having to work on this thing for 10, 20 years.

**Grant Lee** (01:25:11):
Totally.

**Lenny Rachitsky** (01:25:12):
Two is: understand the problem really, really deeply and have real empathy for the people trying to solve this problem. You have this problem creating presentations at your previous job, so you understood it, and I imagine you understand even more deeply now that you worked on this. Essentially, care really deeply, go really deep on the problem, have real empathy for the people facing this problem, and then there's this actually be able to solve this problem using the technology out there.

**Lenny Rachitsky** (01:25:39):
And you made this point about how you guys use 20 different models to do what you do. Talk a little bit more about that just because people may think, "Oh, okay, I could just go to ChatGPT or Claude and it'll create an awesome presentation for me." Why does it take 20 different models? Why is that such a big part of the success here?

**Grant Lee** (01:25:55):
Yeah, so going back to the workflow, you're trying to go and break down every step of the creation process for a user. So, the moment of the initial idea to maybe creating an outline of what you want to present or articulate to creating the first draft. What do we show you there to editing the content? 

**Grant Lee** (01:26:14):
Like, "Okay, I have a first draft, but it's only 60% of the way there. How do I get to 100% of the way there?" Those are all things that might require different models. The initial outline might be better served by something that's purposed-built for actually creating the best outline, the best narrative, the best story arc, versus one where you're actually going back and saying... Gamma, our agent can actually go and review your entire deck and say, "Actually, if I were to add suggestions, I would say, 'You may want to change the visual layout on these set of cards, and you may want to actually change the imagery or the visuals for these cards to match everything else.'"

**Grant Lee** (01:26:50):
These are things that, again, every model might be better served for different things. And so knowing how that actually breaks down into the end user, sitting down, working on the product, working on their presentation, how do you help them? I think you can only do that by understanding-

**Grant Lee** (01:27:00):
Their presentation, how do you help them? I think you can only do that by understanding that workflow and then breaking that down into finding the right tool for the right job.

**Lenny Rachitsky** (01:27:08):
That is super interesting. Essentially there's like, here's the things that AI has to do for us. I don't know, imagining some kind of a storyboard and then it's figure out the model and level of model and prompt for that model to achieve each of these steps as best as possible.

**Grant Lee** (01:27:23):
Totally. Yeah, and it applies to even on the visual side, finding the right image. I think certain models are great at photorealistic output versus others want more of the artistic vibe or something that feels more abstract. And so again, you can choose the right model for the right job. It doesn't mean that you'll never use other models, it's just like as you're going through that workflow, the content might dictate what's the best use case. So in that particular use case and so you're always trying to map to that, what is the end user trying to accomplish in that certain moment?

**Lenny Rachitsky** (01:27:55):
Which models are you finding most useful in the work? I don't know, is there anything surprising about, "Oh wow, this model's actually really good at this and this is really bad at that."

**Grant Lee** (01:28:04):
Yeah, I think surprising or not surprising, the leaderboard is constantly changing and so this is where you almost, you have to have the mindset that you can be adaptable. We're just getting to the point where there's going to be more personalization too, is going back to a consultant's going to have different needs than an educator. Educator's trying to engage their students. They may want to have language or visuals are more animated and that makes sense. Consultant can't get away with that. They may need something that is much more structured or information dense. And so again, how do we actually be the same tool that can serve both of those needs? I think that's where it becomes interesting and it's almost like there isn't necessarily even a "best model". It's like what's the right model for that particular user? That's actually a much harder problem to solve and we're just starting to try to embrace some of that now.

**Lenny Rachitsky** (01:28:50):
Is there one that's just like, "Oh, that's actually really cool." For something that we didn't expect?

**Grant Lee** (01:28:54):
Early on, things like creating the outline, Perplexity was actually great. Not one that people talk about as often, but creating the outline and doing web search and actually integrating some of those elements together for us was pretty powerful.

**Lenny Rachitsky** (01:29:07):
Okay, there's two more things I want to talk about. One is pricing. How you guys figured out your approach to pricing. You guys started monetizing really early and that feels like such an important piece to AI startups because you're spending real money on an inference and other things. And then I want to talk about hiring. So in terms of pricing, when did you decide it was time to really start charging and then how did you pick your approach to pricing your actual price?

**Grant Lee** (01:29:32):
So we kind of stumbled into having to do pricing and packaging. I mentioned our big AI launch in March 2023. This was pre-revenue, so we wanted just to get, we focused all of our effort on the first 30 seconds, literally all hands on deck, just trying to get that right. And we spent zero time on actually monetization. So we had a credit system. All new users get 400 credits. Once they burn through those credits, there was actually nothing more you could do with AI. It kind of just got cut off. And so Intercom for us was just blowing up with people saying, "How do I purchase more credits? I want to keep using this thing." In a very fair question. So basically all of April we ended up having to do a quick sort of pricing and packaging exercise. We did a couple of different things.

**Grant Lee** (01:30:18):
We did run a form of Van Westendorp, which is just understanding-

**Lenny Rachitsky** (01:30:22):
Classic.

**Grant Lee** (01:30:23):
...what is the overall willingness to pay. And so we did use that. We did kind of integrate some forms of conjoint analysis, which is just trying to understand what are the features or things that people actually value in your product. And so we'd survey a lot of our early users and ended up coming to a price point that was, in the beginning we only had one plan type. It was roughly like 20 bucks a month. Part of this was also just realizing we almost didn't need to overthink it too much because this is when the initial wave of AI companies and startups were coming out and products are coming out and you almost end up becoming anchored by, what does ChatGPT charge? Because everyone becomes familiar with that price point. And so we ask ourselves how complicated do we want to make it?

**Grant Lee** (01:31:05):
We always default to remove friction, make it as easy as possible for a user to understand. And so we end up coming up with a similar price point and ship that as the V1, basically end of May 2023. And for us, we wanted to see a couple of things like, okay, we have the initial price point. Is it economical for us at that price point? Are we actually making money? And we'd monitor there for that for many, many months realizing that yes, actually at that price point we could still generate pretty good margins and that would allow us to reinvest that profit back into growing the business.

**Grant Lee** (01:31:37):
Continue to add obviously head count, but then also investing even more into inference costs, whatever we wanted to do to experiment broadly with AI. And so we've always had that pricing and packaging. Does it not only need to be easy to understand for a user, obviously you have to have strong conversion rates, but it should be something where you feel like you could build an enduring durable business off of. And if it's not right, then you can always go back and try to tweak things, but it's something you should be monitoring as early as possible.

**Lenny Rachitsky** (01:32:06):
What's interesting, we have the head of ChatGPT on the podcast and he shared the way they picked ChatGPT's prices is the Van Westendorp survey that they ran in Google Forms.

**Grant Lee** (01:32:14):
Totally. Yeah, I listened to that one. Yeah, it was a great one.

**Lenny Rachitsky** (01:32:17):
Oh, what a, that survey man, what a, and not only just that survey being the slick, I'm imagining that XKCD comic with the open source little, I don't know, block holding up the entire world, like this one little piece of code. That's like the survey, just behind everything. But it's interesting how that one decision just created this ripple effect on all AI startups, 20 bucks a month. Of course that's what everyone's doing.

**Grant Lee** (01:32:43):
Totally. Who knows what would happen if they chose a different number, but here we are.

**Lenny Rachitsky** (01:32:46):
Everyone would be so much more rich if it's just like 25 bucks. Someone. Imagine the GDP of growth-

**Grant Lee** (01:32:52):
Yeah.

**Lenny Rachitsky** (01:32:54):
...from that. Oh man. Okay. And then the way, so was Van Westendorp helped you pick a price point and then you said this conjoint analysis, we actually have a guest post that I think describes how to do this well and if not we'll find, appoint people to how to actually approach this. You started charging, so was post product [inaudible 01:33:11] launch, you launch with no paid features. People were just like, "I want to pay, I want to keep using this." You're like, "Okay, I guess we've got to start charging."

**Grant Lee** (01:33:18):
That's right. Yep.

**Lenny Rachitsky** (01:33:20):
Is there anything, looking back, I guess just you wish you'd known when you were approaching pricing for folks that are doing this now like, "Oh, we should have done this differently or should have thought about this."

**Grant Lee** (01:33:28):
Yeah, I mean that one's hard. I think you never want to obviously just throw something together without giving it much thought, but for us with limited resources, we focused on the only thing we thought mattered, which was getting some initial users and having that organic growth. And I think there's maybe two checkpoints of thinking about whether or not you feel like you have product market fit. I think one checkpoint is organic growth. The second is are people willing to pay for the product? And I think if you pass both those, you feel like you've, at least within some pocket of the market you have some PMF. So I think those are both important questions to ask depending on your resources. Ideally you can do kind of both and experiment a little bit along at the same time. And then by the time you actually have paying users, you sort of check some of those boxes for yourself.

**Lenny Rachitsky** (01:34:15):
By the way, I love that your launch pricing. You're like, "Okay, let's figure out if we're actually making money or not." It's not obvious with AI companies. [inaudible 01:34:24]

**Grant Lee** (01:34:24):
Well, we also didn't have a choice because at that point runway was also low. So if we weren't making money, well we would actually be in a tough spot.

**Lenny Rachitsky** (01:34:31):
That's such a good point that you did not have a huge amount of money sitting around to spend on the way a lot of companies in AI do, just, "We'll deal with it and we'll figure out how to make money later."

**Grant Lee** (01:34:40):
Totally. Within a couple of months we had hit a million in ARR and became profitable. And so those were both two exciting milestones for a company that months prior were heads down figuring out how we even survive.

**Lenny Rachitsky** (01:34:53):
What a story. I'm so excited to be telling this story. Okay, final topic I want to talk about is hiring. You have some really hot takes on hiring, clearly it's worked out for you guys. What are some things you've learned about hiring well, what is your approach to hiring?

**Grant Lee** (01:35:08):
So we had this mantra internally, I mean even before the sort of AI launch for us, which was "Hire painfully slowly." And I think the temptation is once things start working, you just start, [inaudible 01:35:23] scaling this thing and start adding more and more headcount. And for us that always, I don't know, that didn't feel right. We wanted to build the team very thoughtfully, be lean, but also be a team that every individual feels like they have high amount of impact that they have on a daily basis. And so for us that was from sort of the very beginning. And so even as we've scaled up, I think we've been super efficient by nature of just being a very lean team.

**Lenny Rachitsky** (01:35:47):
So I think a lot of people hear this advice of "Stay lean, be efficient." You have some even more concrete pieces of advice here of just what that looks like. You have a huge, I don't know, philosophy of just huge leverage per person. You focus a lot on revenue per employee. Talk about that.

**Grant Lee** (01:36:03):
So obviously we look at things like revenue per employee, but we never let that be the sort of North Star. It's not something that dictates our strategy per se. Same thing with profitability. I think by being efficient, that ends up becoming a side effect of yeah, we are profitable and growing. But I think for us more is like we care a ton about adding the right team members. So it's easy. You can almost sort of shoot yourself in the foot as a founder by just setting the wrong goals. If your goal is to double in headcount and add a hundred people to the company, then that becomes the goal. The goal is no longer to add the best people. It's no longer to maintain a high quality bar. The goal is to hit a hundred people and that's everything everyone's focused on. So then guess what?

**Grant Lee** (01:36:44):
If that's the goal, then the thing that ends up dropping is, okay, maybe we will settle for employees that are good enough and we're going to make sure we get to that hundred because a hundred is going to help us get to the next phase of growth. And then the next phase of growth. And I think we've tried to resist that temptation, which is we want everybody that comes through the door and joins our team to really be the type of person that represents Gamma. Our first 10 to 12 employees, we spent so much time really getting the DNA right. I think Brian Chesky talks about this. Your goal is you get that first 10 and you want to be able to then replicate the next 10 and then replicate the next 10, but that 10 needs to be all super cohesive. You need to have the same shared values, same principles, same ambition, same vision, and if you don't, then what are you even replicating?

**Grant Lee** (01:37:32):
Right? At that point you're just adding headcount and you can easily just be adding headcount because we're chasing the next shiny startup to join. And so for us, we really focused on that first 10. That allows us to really have this sort of community of teammates that basically want to stick around. A lot of founders think about not wanting to have a leaky bucket on the product side, but the same thing applies on the team side. If you have a leaky bucket, people are just constantly leaving, revolving door. That's a huge amount of cost. The continuity cost is massive and it's really hard to quantify.

**Grant Lee** (01:38:08):
And I mentioned our first 10 employees, all 10 of them are still here today, five years later. And so that sort of continuity, it means that you have this tribal knowledge that sticks with you. It means that people can continue building and having that sort of cohesive feeling. So I think that's one piece that's just like, it's hard to quantify, but I do think, new startups I would advise just to really be thoughtful about that, get that to a point where you really feel confident that as you're adding the next 10 and next hundred, you're actually replicating the same DNA and not actually just adding a bunch of random people to the company.

**Lenny Rachitsky** (01:38:39):
One of your very specific piece of advice is hire generalists versus someone that's just really deep in one thing, why is that so important? What does that look like?

**Grant Lee** (01:38:47):
This very much feels like the age of the generalist or the rise of the generalist right now. When we think about of a team that can be really flat, you still want people that can be very spiky in certain areas. So for instance, obviously a product designer that knows how to code, that's great. It allows you to actually span across many different domains. And again, an organization that's super flat, when you're wearing a lot of different hats, that means every individual, if you're a generalist, you can wear by default a lot of different hats. And that's helped us just maintain this idea that the work, the opportunity in front of us can easily expand. Each person plays a huge role. They don't need to wait and ask for permission. They can go after and pick up a piece of work because it's there and they can at least get it started even if they're not the one that always sees it through.

**Grant Lee** (01:39:34):
And so for us, I do think this rise of the generalist is going to be important. You can always augment that by working with contractors or agencies that are hyper-specialized in certain areas. And this is where, for instance, going back to influencer marketing, you might work with an agency rather than kind of scale up your own influencer marketing team. You work with a hyper-specialized team there. But my marketing team is all generalists. Our head of marketing more recently launched this cool drone show in San Francisco. We have 4,000 people in San Francisco. The mayor came by, she created that entire project and managed that entire project end to end herself while also managing the entire marketing team. And so it's like this idea that a generalist is able to play all these different roles, can still be super high impact. For us, it also goes hand in hand with, I mentioned sort of this other role, which is the player coach. Traditional management layer is like a manager manages a ton of people and that's kind of their core focus.

**Grant Lee** (01:40:33):
All of our people leaders are player coaches in that they still do the end work themselves and they can mentor and coach those around them. This analogy came from, or this mindset came from, it's something I borrowed from just the sports world in general because there's a lot of sports that just move incredibly fast. Like football for instance, it moves really, really fast. And so you might have a coach that's calling into play, but the quarterback can actually make a last-minute adjustment based on what he's seeing the defense do. And so you want a player that is on the field that can adapt as needed and that way the coach doesn't have to call every single thing in. He's just kind of giving you the general intent of what you want to run as the play and then the quarterback can still make the adjustments.

**Grant Lee** (01:41:13):
And so all of our people leaders, our management layer is all folks that actually can make those adjustments. If they're seeing something that doesn't feel right on a daily or weekly basis, they're adjusting priorities for that team and they're still doing the work themselves too. They're so close to it that they understand what is the relative prioritization at all times. So both of those I think have been, when I think about founders, they oftentimes think about innovation in the sense, we're going to innovate on products or innovate on technology. I think every founder today has a chance to innovate on org design, and that starts by just thinking about what is the type of company we want to build today? What does it mean? And when it comes to the management later, what type of sort of leaders do we hire? What does it mean when it comes to hiring specific roles and specific functions? All of that is a chance for you to be very thoughtful and build a company that you're excited to be at for hopefully many, many years to come.

**Lenny Rachitsky** (01:42:07):
So what I'm hearing here is manager, there's no pure managers at Gamma and your plan is to not have people that are just managing the ideas. Everyone, even a manager is doing their own IC work.

**Grant Lee** (01:42:18):
Yes.

**Lenny Rachitsky** (01:42:19):
And then the other piece of advice here is just generalists. People that can do a lot of stuff, and I hear this a lot on this podcast just like everyone is, there's no more just like you're a designer, that's all you're going to do. Designers need to build stuff, market stuff, write some PRDs probably.

**Grant Lee** (01:42:34):
Just one thing to add is that's where we are at today. I think being adaptable means that certain things may evolve and change and how the player coach model evolves and maybe in certain functions or as the team scales, that's not going to be practical everywhere. I think the reality is you just have to know and be willing to adopt different frameworks over time and being honest with yourself with what's working, what's not. I think we're constantly trying to learn and evolve ourselves. I think we're doing it in a way that we don't believe it really has been done before, and so we have to kind of pave our own path over time.

**Lenny Rachitsky** (01:43:10):
I love that. Giving yourself an out when the time comes when you need to just hire managers.

**Grant Lee** (01:43:14):
A year later when you invite me back, I'll say, "Yeah, this is what we learned."

**Lenny Rachitsky** (01:43:16):
Yeah, that makes sense. That's probably going to happen. It's like people are like, "We don't need product managers." [inaudible 01:43:22] "I see. Maybe we do." Yeah, that makes sense. One last piece I want to talk about is you have this really cool quote that you shared on Twitter. " When you find someone exceptional, bet big on them." This is a big part of your philosophy is just bet big on the people that are doing super well. Just talk about what that looks like and why that's so important.

**Grant Lee** (01:43:40):
Yeah, I mean this starts top of the funnel, which is you meet candidates, you decide who you actually hire. And so if you don't start with a high bar there, going back to what is the goal? The goal is to hit a hiring target versus maintaining a super high quality hiring bar. Those are different goals and I don't think we've ever kind of dropped our bar. And so then you bring somebody in that you think is exceptional, that brings something unique to the table, that can be a good teammate. When they're thriving, you just give them more and more resources. I think what you realize, another sports analogy is when you're on an A team for instance, or a team that you feel like is exceptional, A players want actually more playing time. You never see a star player that says "I actually want less playing time."

**Grant Lee** (01:44:26):
They want more time on the field, they want to actually go after the hardest problems. They want to be able to feel like they're making a huge impact. And so if you have fewer exceptional teammates where you can just throw almost anything at them and they'll figure it out, that feels for you as a team, just feels great because then they get what they find rewarding, which is the ability to go after hard hairy problems and to be able to come out through the other end feeling like they've accomplished something. And I think that for us has always been something that goes beyond just almost everything else. We give people a chance to really thrive in this environment and we try to nurture that as much as humanly possible every step of the way.

**Lenny Rachitsky** (01:45:04):
Is there anything else around hiring that you think is really important or a big lesson you've learned or lesson you share that we haven't talked about yet?

**Grant Lee** (01:45:12):
I mean, there's some of these intangibles, which is for the founding team, does this feel like this could be your life's work? When you're pitching a potential candidate, does this feel like something where you're actually committed? The unfortunate side of a lot of what's happening in the AI world broadly is I think you're coming to learn which founders are sort of missionaries versus mercenaries. And many that were just chasing maybe just a big outcome, or something shiny, or something to feel good about themselves, they'll go off and then the company, I guess doesn't live on or maybe has to find a different way, a different path.

**Grant Lee** (01:45:47):
And so something I admire is you look at some of the founders, obviously before this sort of AI era, folks like Dylan or Melanie, Figma, Canva, Ivan at Notion, they've been doing this for over a decade now. They had many chances to just leave and sunset off into doing whatever they wanted to do, but they care so much about the mission, they've stuck it through. And I think when you're talking to candidates, candidates can kind of tell, is this something the founders even care about? And I think you're going to have a better chance of attracting true missionaries, people that want to build with you for the long haul, if it's authentic and it's something you actually care about.

**Lenny Rachitsky** (01:46:24):
Oh man, I keep saying this. I feel like we could have at least five more hours of stuff to talk about. That'll be good content for when you come back and you're making a billion dollars a year. Before we get to our very exciting lightning round, Grant, is there anything else that you think is important for people to hear? Any last, I don't know, lesson you want to double down on? Anything you want to leave listeners with?

**Grant Lee** (01:46:48):
Yeah, I mean, just going back to the original story of that was probably the ultimate low point. Having an investor that spent 20 minutes listening to my pitch, telling me that it was the worst thing, worst idea in the world. I think throughout the journey there's going to be those low moments. And when I think about being a founder and working a startup, you're honestly just trying to increase your luck surface area as much as possible. And for me, luck surface area has two dimensions. The first dimension is people. Who are you surrounding yourself with that share your same ambition, share the same values, same principles, and find those people and then give yourself enough time to prove that you guys can accomplish great things. I'm lucky to have two amazing co-founders. We've been working on this thing for five years. There's 0% chance we'd be where we are if I didn't have them. And we've had to overcome a lot. But I guess for us it's like creating that own luck over a long time horizon has been the only way that it's been possible.

**Lenny Rachitsky** (01:47:48):
Well, it's clearly showing in the success you guys are having. Grant, with that, we've reached our very exciting lightning round. Are you ready?

**Grant Lee** (01:47:56):
Yep. Yes.

**Lenny Rachitsky** (01:47:57):
All right. I've got five questions for you. First question, what are two or three books that you find yourself recommending most to other people?

**Grant Lee** (01:48:07):
Yeah, so I'll give one that's for pre-product market fit folks and the one post-product market fits.

**Lenny Rachitsky** (01:48:07):
Perfect.

**Grant Lee** (01:48:12):
Pre-product market fit I would say is Shoe Dog, which is written by Phil Knight, founder of Nike. And he talks about two things. One, you should chase your sort of crazy ideas, but these should be ideas you're passionate about. He was an athlete, and so not surprisingly, he focused a lot about creating tools or shoes in this case for other athletes and was passionate about that. And the second thing he taught me was going back to the team thing, he surrounded himself with other people that were super passionate about athletics and shoes as well. And so the folks that he had as his initial startup crew were all that.

**Grant Lee** (01:48:49):
And I think that gives you a chance of overcoming a ton, where you're focused on problems you actually care about solving, and you're dealing with a team that shares the same ambition as you. Post-product market fit, there's a book called 7 Powers by Hamilton Helmer, where I think when you think about how to build a durable business, there's so much in there. I think there's a lot that you can kind of read and reread as you kind of evolve and hit new milestones yourself. And so much of that can be both tactical but also zooming out and thinking about what are the big picture strategic questions you as a founding team you need to be thinking about. So both are great.

**Lenny Rachitsky** (01:49:21):
Hamilton was on the podcast, we'll link to that episode. I also love that book. Many people mention it. Next question, do you have a favorite recent movie or TV show you've really enjoyed?

**Grant Lee** (01:49:31):
Yeah, The Lazarus Project is one, I'm a huge sucker for time travel and sci-fi, so this is one I just started watching. And for me, it has all the right ingredients for a fun show.

**Lenny Rachitsky** (01:49:44):
Is there a product you recently discovered that you really love?

**Grant Lee** (01:49:47):
Yeah, I mentioned this already, Voicepanel. So again, a full disclosure angel investor, but we've been using it and going back to kind of being a cheat code for folks that are starting to experiment a ton with different ideas, vibe coding, maybe some of these might get this into the hands of users, hear what they think about it. I think, and honestly can help speed up a lot of things and save you time from wasting time on the wrong ideas or ideas where there's no market.

**Lenny Rachitsky** (01:50:13):
Is there a life motto that you often find yourself coming back to in work or in life?

**Grant Lee** (01:50:18):
Yeah, so there's this Chinese idiom that my mom used to say, it's a [foreign language 01:50:24] which the translation is "A frog at the bottom of a well". And the story is like there's a frog at the bottom of the well. He looks up every night and he sees the world and he imagines he knows everything about the world. And then one day a bird comes along and describes all the things that he sees, the ocean, the mountains, and the frog realizes that what he sees is just such a limited part of the world and the bird asks, "Do you want to come join me?" And kind of see the rest of it. And so frog goes along.

**Grant Lee** (01:50:54):
And so for me, I came from a pretty modest childhood. My parents, we didn't have a whole lot and I think it would've been very easy to kind of have a very narrow lens on the world and be like, "Oh, this is what it is." But my mom never allowed me to do that. She always pulled me to dream much bigger to say, "Hey, the world is vast. It's your opportunity to seize it. You have to go out there, don't have a narrow view of what's possible. Always dream bigger." And so for me, that's always carried through and every time I feel like I'm thinking too little or too small, I try to zoom out and remind myself that there's much more out there to go after.

**Lenny Rachitsky** (01:51:30):
That is so good and so important and so valuable in today's world where so much more is possible. Just like, most of what limits people it feels like now is just, I don't know what idea I have. I don't know what to do. Now you could get things done so much quicker and so much more is possible. So that's such valuable thinking. Okay, final question. You help people present better. Your tools basically help people become better presenters. What's one tip you've learned or one tip you teach people to become better presenters that might be helpful to listeners?

**Grant Lee** (01:52:00):
Yeah, I mean, I'll go back to the consumer advertising concept, which is one idea at a time, this notion of you give them one egg, someone can catch it, give them too many eggs, they're going to drop it. So don't try to throw too many concepts all at once. Keep it simple. People will appreciate it. And so with every sort of presentation, break it down into the core concepts, try to make sure you're covering one at a time. And I think once you sort of see a through line there, then it becomes easy for it to package up into something that feels more cohesive.

**Lenny Rachitsky** (01:52:32):
Less is more, as they say.

**Grant Lee** (01:52:35):
Totally.

**Lenny Rachitsky** (01:52:36):
Grant, two final questions. Where can folks find you online? Where can they find Gamma? What should they know? Just plug anything you want and then how can listeners be useful to you?

**Grant Lee** (01:52:44):
Yeah, so you can find me on both Twitter and LinkedIn, DM's open. I honestly hear just knowing how hard the journey is in general, whether just thinking about a startup idea or you're deep into startup land, I want to be hopefully helpful. I'm going to be hopefully your biggest cheerleader, so let me know how I can help. And then for us, we're always looking for feedback. So if you're trying Gamma, it's falling short of your expectations, let us know. We'd love to help in terms of just trying to make it better. And yeah, really appreciate all the feedback and support along the way.

**Lenny Rachitsky** (01:53:16):
Grant, this was awesome. I really appreciate you making time. I know you're trying to build a crazy fast-growing startup with a lot going on, so I really appreciate you making time for this.

**Grant Lee** (01:53:24):
Thanks, Lenny. It's been a blast.

**Lenny Rachitsky** (01:53:24):
It's been a blast for me too. Bye everyone.

**Lenny Rachitsky** (01:53:29):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [10/17] The Godmother of AI on jobs, robots & why world models are next | Dr. Fei-Fei Li

**Lenny Rachitsky** (00:00:00):
A lot of people call you the godmother of AI. The work you did actually was the spark that brought us out of AI winter.

**Dr. Fei Fei Li** (00:00:07):
In the middle of 2015, middle of 2016, some tech companies avoid using the word AI because they were not sure if AI was a dirty word. 2017-ish was the beginning of companies calling themselves AI companies.

**Lenny Rachitsky** (00:00:22):
There's this line, I think, this was when you were presenting to Congress. There's nothing artificial about AI. It's inspired by people. It's created by people, and most importantly, it impacts people.

**Dr. Fei Fei Li** (00:00:30):
It's not like I think AI will have no impact on jobs or people. In fact, I believe that whatever AI does, currently or in the future, is up to us. It's up to the people. I do believe technology is a net positive for humanity, but I think every technology is a double-edged sword. If we're not doing the right thing as a society, as individuals, we can screw this up as well.

**Lenny Rachitsky** (00:00:56):
You had this breakthrough insight of just, okay, we can train machines to think like humans, but it's just missing the data that humans have to learn as a child.

**Dr. Fei Fei Li** (00:01:03):
I chose to look at artificial intelligence through the lens of visual intelligence because humans are deeply visual animals. We need to train machines with as much information as possible on images of objects, but objects are very, very difficult to learn. A single object can have infinite possibilities that is shown on an image. In order to train computers with tens and thousands of object concepts, you really need to show it millions of examples.

**Lenny Rachitsky** (00:01:36):
Today, my guest is Dr. Fei-Fei Li, who's known as the godmother of AI. Fei-Fei has been responsible for and at the center of many of the biggest breakthroughs that sparked the AI revolution that we're currently living through. She spearheaded the creation of ImageNet, which was basically her realizing that AI needed a ton of clean-labeled data to get smarter, and that data set became the breakthrough that led to the current approach to building and scaling AI models. She was chief AI scientist at Google Cloud, which is where some of the biggest early technology breakthroughs emerged from. She was director at SAIL, Stanford's Artificial Intelligence Lab, where many of the biggest AI minds came out of. She's also co-creator of Stanford's Human-Centered AI Institute, which is playing a vital role in a direction that AI is taking. She's also been on the board of Twitter. She was named one of Time's 100 Most Influential People in AI. She's also United Nations advisory board. I could go on.

**Lenny Rachitsky** (00:02:29):
In our conversation, Fei-Fei shares a brief history of how we got to today in the world of AI, including this mind-blowing reminder that 9 to 10 years ago, calling yourself an AI company was basically a death knell for your brand because no one believed that AI was actually going to work. Today, it's completely different. Every company is an AI company. We also chat about her take on how she sees AI impacting humanity in the future, how far current technologies will take us, why she's so passionate about building a world model and what exactly world models are, and most exciting of all, the launch of the world's first large world model, Marble, which just came out as this podcast comes out. Anyone can go play with this at marble.worldlabs.ai. It's insane. Definitely check it out. Fei-Fei is incredible and way too under the radar for the impact that she's had on the world, so I am really excited to have her on and to spread her wisdom with more people.

**Dr. Fei Fei Li** (00:05:34):
I'm excited to be here, Lenny.

**Lenny Rachitsky** (00:05:36):
I'm even more excited to have you here. It is such a treat to get to chat with you. There's so much that I want to talk about. You've been at the center of this AI explosion that we're seeing right now for so long. We're going to talk about a bunch of the history that I think a lot of people don't even know about how this whole thing started, but let me first read a quote from Wired about you just so people get a sense, and in the intro I'll share all of the other epic things you've done. But I think this is a good way to just set context. "Fei-Fei is one of a tiny group of scientists, a group perhaps small enough to fit around a kitchen table, who are responsible for AI's recent remarkable advances."

**Lenny Rachitsky** (00:06:10):
A lot of people call you the godmother of AI, and unlike a lot of AI leaders, you're an AI optimist. You don't think AI is going to replace us. You don't think it's going to take all our jobs. You don't think it's going to kill us. So I thought it'd be fun to start there, just what's your perspective on how AI is going to impact humanity over time?

**Dr. Fei Fei Li** (00:06:30):
Yeah, okay, so Lenny, let me be very clear. I'm not a utopian, so it's not like I think AI will have no impact on jobs or people. In fact, I'm a humanist. I believe that whatever AI does, currently or in the future, is up to us. It's up to the people. So I do believe technology is a net positive for humanity. If you look at the long course of civilization, I think we are, and fundamentally, we're an innovative species that we... If you look at from written record thousands of years ago to now, humans just kept innovating ourselves and innovating our tools, and with that, we make lives better, we make work better, we build civilization, and I do believe AI is part of that. So that's where the optimism comes from. But I think every technology is a double-edged sword, and if we're not doing the right thing as a species, as a society, as communities, as individuals, we can screw this up as well.

**Lenny Rachitsky** (00:07:47):
There's this line, I think, this was when you were presenting to Congress, "There's nothing artificial about AI. It's inspired by people. It's created by people, and most importantly, it impacts people." I don't have a question there, but what a great line.

**Dr. Fei Fei Li** (00:07:59):
Yeah, I feel pretty deeply. I started working AI two and a half decades ago, and I've been having students for the past two decades and almost every student who graduates, I remind them when they graduate from my lab that your field is called artificial intelligence, but there's nothing artificial about it.

**Lenny Rachitsky** (00:08:23):
Coming back to the point you just made about how it's kind of up to us about where this all goes, what is it you think we need to get right? How do we set things on a path? I know this is a very difficult question to answer, but just what's your advice? What do you think we should be keeping in mind?

**Dr. Fei Fei Li** (00:08:36):
Yeah, how many hours do we have?

**Lenny Rachitsky** (00:08:39):
How do we align AI? There we go. Let's solve it.

**Dr. Fei Fei Li** (00:08:41):
So I think people should be responsible individuals no matter what we do. This is what we teach our children, and this is what we need to do as grownups as well. No matter which part of the AI development or AI deployment or AI application you are participating in, and most likely many of us, especially as technologists, we're in multiple points. We should act like responsible individuals and care about this. Actually, care a lot about this. I think everybody today should care about AI because it is going to impact your individual life. It is going to impact your community, it's going to impact the society and the future generation. And caring about it as a responsible person is the first, but also the most important step.

**Lenny Rachitsky** (00:09:37):
Okay, so let me actually take a step back and kind of go to the beginning of AI. Most people started hearing and caring about AI, as what it's called today, just like, I don't know, a few years ago when ChatGPT came out. Maybe it was like three years ago.

**Dr. Fei Fei Li** (00:09:51):
Three years ago, almost one more month, three years ago.

**Lenny Rachitsky** (00:09:55):
Wow, okay. And that was ChatGPT coming out. Is that the milestone you have in mind?

**Dr. Fei Fei Li** (00:09:56):
Yes.

**Lenny Rachitsky** (00:09:57):
Okay, cool. That's exactly how I saw it. But very few people know there was a long, long history of people working on, it was called machine learning back then and there's other terms, and now it's just everything's AI and there was kind of a long period of just a lot of people working on it. And then there's this what people refer to as the AI winter where people just gave up almost, most people did, and just, okay, this idea isn't going anywhere. And then the work you did actually was essentially the spark that brought us out of AI winter and is directly responsible for the world where now of just AI is all we talk about. As you just said, it's going to impact everything we do. So I thought it'd be really interesting to hear from you just the brief history of what the world was like before ImageNet and just the work you did to create ImageNet, why that was so important, and then just what happened after.

**Dr. Fei Fei Li** (00:10:44):
It is, for me, hard to keep in mind that AI is so new for everybody when I lived my entire professional life in AI. There's a part of me that is just, it's so satisfying to see a personal curiosity that I started barely out of teenagehood and now has become a transformative force of our civilization. It generally is a civilizational level technology. So that journey is about 30 years or 20 something, 20 plus years, and it's just very satisfying. So where did it all start? Well, I'm not even the first generation AI researcher. The first generation really date back to the '50s and '60s, and Alan Turing was ahead of his time in the '40s by asking, daring humanity with the question, "Is there thinking machines?" And of course he has a specific way of testing this concept of thinking machine, which is a conversational chatbot, which to his standard we now have a thinking machine.

**Dr. Fei Fei Li** (00:12:02):
But that was just a more anecdotal inspiration. The field really began in the '50s when computer scientists came together and look at how we can use computer programs and algorithms to build these programs that can do things that have been only capable by human cognition. And that was the beginning. And the founding fathers the Dartmouth workshop in the 1956, we have Professor John McCarthy who later came to Stanford who coined the term artificial intelligence. And between the '50s, '60s, '70s, and '80s, it was the early days of AI exploration and we had logic systems, we had expert systems, we also had early exploration of neural network. And then it came to around the late '80s, the '90s, and the very beginning of the 21st century. That stretch about 20 years is actually the beginning of machine learning, is the marriage between computer programming and statistical learning.

**Dr. Fei Fei Li** (00:13:23):
And that marriage brought a very, very critical concept into AI, which is that purely rule-based program is not going to account for the vast amount of cognitive capabilities that we imagine computers can do. So we have to use machines to learn the patterns. Once the machines can learn the patterns, it has a hope to do more things. For example, if you give it three cats, the hope is not just for the machines to recognize these three cats. The hope is the machines can recognize the fourth cat, the fifth cat, the sixth cat, and all the other cats. And that's a learning ability that is fundamental to humans and remaining animals. And we, as a field, realized, "We need machine learning." So that was up till the beginning of the 21st century. I entered the field of AI literally in the year of 2000. That's when my PhD began at Caltech.

**Dr. Fei Fei Li** (00:14:33):
And so I was one of the first generation machine learning researchers and we were already studying this concept of machine learning, especially neural network. I remember that was one of my first courses at Caltech is called neural network, but it was very painful. It was still smack in the middle of the so-called AI winter, meaning the public didn't look at this too much. There wasn't that much funding, but there was also a lot of ideas flowing around. And I think two things happened to myself that brought my own career so close to the birth of modern AI is that I chose to look at artificial intelligence through the lens of visual intelligence because humans are deeply visual animals. We can talk a little more later, but so much of our intelligence is built upon visual, perceptual, spatial understanding, not just language per se. I think they're complementary.

**Dr. Fei Fei Li** (00:15:37):
So I choose to look at visual intelligence and my PhD and my early professor years, my students and I are very committed to a north star problem, which is solving the problem of object recognition because it's a building block for the perceptual world, right? We go around the world interpreting reasoning and interacting with it more or less at the object level. We don't interact with the world at the molecular level. We don't interact with the world as... We sometimes do, but we rarely, for example, if you want to lift a teapot, you don't say, "Okay, the teapot is made of a hundred pieces of porcelain and let me work on this a hundred pieces." You look at this as one object and interact with it. So object is really important. So I was among the first researchers to identify this as a north star problem, but I think what happened is that as a student of AI and a researcher of AI, I was working on all kinds of mathematical models including neural network, including Bayesian network, including many, many models.

**Dr. Fei Fei Li** (00:16:53):
And there was one singular pain point is that these models don't have data to be trained on. And as a field, we were so focusing on these models, but it dawned on me that human learning as well as evolution is actually a big data learning process. Humans learn with so much experience constantly. In the evolution, if you look at time, animals evolve with just experiencing the world. So I think my students and I conjectured that a very critically-overlooked ingredient of bringing AI to life is big data. And then we began this ImageNet project in 2006, 2007. We were very ambitious. We want to get the entire internet's image data on objects. Now granted internet was a lot smaller than today, so I felt like that ambition was at least not too crazy. Now, it's totally delusional to think a couple of graduate student and a professor can do this.

**Dr. Fei Fei Li** (00:18:05):
And that's what we did. We curated very carefully, 15 million images on the internet, created a taxonomy of 22,000 concepts, borrowing other researchers' work like linguists work on WordNet, and it's a particular way of dictionarying words. And we combine that into ImageNet and we open-sourced that to the research community. We held an annual ImageNet challenge to encourage everybody to participate in this. We continue to do our own research, but 2012 was the moment that many people think was the beginning of the deep learning or birth of modern AI because a group of Toronto researchers led by Professor Geoff Hinton, participated in ImageNet Challenge, used ImageNet big data and two GPUs from NVIDIA and created successfully the first neural network algorithm that can...

**Dr. Fei Fei Li** (00:19:12):
It didn't totally solve, but made a huge progress towards solving the problem of object recognition. And that combination of the trio technology, big data, neural network, and GPU was kind of the golden recipe for modern AI. And then fast-forward, the public moment of AI, which is the ChatGPT moment, if you look at the ingredients of what brought ChatGPT to the world technically still use these three ingredients. Now, it's internet-scale data mostly texts is a much more complex neural network architecture than 2012, but it's still neural network and a lot more GPUs, but it's still GPUs. So these three ingredients are still at the core of modern AI.

**Lenny Rachitsky** (00:20:16):
Incredible. I have never heard that full story before. I love that it was two GPUs was the first. I love that. And now it's, I don't know, hundreds of thousands, right, that are orders of magnitude more powerful.

**Dr. Fei Fei Li** (00:20:30):
Yep.

**Lenny Rachitsky** (00:20:31):
And those two GPUs where they just bought, they were like gaming GPUs, they just went to the-

**Dr. Fei Fei Li** (00:20:34):
Yes.

**Lenny Rachitsky** (00:20:35):
... GameStar that people use for playing games. As you said, this continues to be in a large way, the way models get smarter. Some of the fastest growing companies in the world right now, I've had them all mostly on the podcast, Mercor and Surge and Scale. They continue to do this for labs, just give them more and more label data of the things they're most excited and interested in.

**Dr. Fei Fei Li** (00:20:53):
Yeah, I remember Alex Wang from Scale very early days. I probably still has his emails when he was starting Scale. He was very kind. He keeps sending me emails about how image that inspired Scale. I was very pleased to see that.

**Lenny Rachitsky** (00:21:08):
One of my other favorite takeaways from what you just shared is just such an example of high agency and just doing things that's kind of a meme on Twitter. Just you can just do things. You're just like, okay, this is probably necessary to move AI. And it's called machine learning back then, right? Was that the term most people used?

**Dr. Fei Fei Li** (00:21:25):
I think it was interchangeably. It's true. I do remember the companies, the tech companies, I am not going to name names, but I was in a conversation in one of the early days, I think is in the middle of 2015, middle of 2016, some tech companies avoid using the word AI because they were not sure if AI was a dirty word. And I remember I was actually encouraging everybody to use the word AI because to me that is one of the most audacious question humanity has ever asked in our quest for science and technology, and I feel very proud of this term. But yes, at the beginning some people were not sure.

**Lenny Rachitsky** (00:22:12):
What year was that roughly when AI was a dirty word?

**Dr. Fei Fei Li** (00:22:14):
2016, I think because that was-

**Lenny Rachitsky** (00:22:15):
2016, less than 10 years ago.

**Dr. Fei Fei Li** (00:22:18):
That was the changing. Some people start calling it AI, but I think if you look at the Silicon Valley tech companies, if you trace their marketing term, I think 2017-ish was the beginning of companies calling themselves AI companies.

**Lenny Rachitsky** (00:22:40):
That's incredible. Just how the world has changed.

**Dr. Fei Fei Li** (00:22:43):
Yes.

**Lenny Rachitsky** (00:22:43):
Now, you can't not call yourself an AI company.

**Dr. Fei Fei Li** (00:22:46):
I know.

**Lenny Rachitsky** (00:22:46):
Just nine-ish years later.

**Dr. Fei Fei Li** (00:22:48):
Yeah.

**Lenny Rachitsky** (00:22:49):
Oh, man. Okay. Is there anything else around the history, that early history that you think people don't know that you think is important before we chat about where you think things are going and the work that you're doing?

**Dr. Fei Fei Li** (00:23:01):
I think as all histories, I'm keenly aware that I am recognized for being part of the history, but there are so many heroes and so many researchers. We're talking about generations of researchers. In my own world, there are so many people who have inspired me, which I talked about in my book, but I do feel our culture, especially Silicon Valley, tends to assign achievements to a single person. While I think it has value, but it's just to be remembered. AI is a field of, at this point, 70 years old and we have gone through many generations. Nobody, no one could have gotten here by themselves.

**Lenny Rachitsky** (00:23:54):
Okay, so let me ask you this question. It feels like we're always on this precipice of AGI, this kind of vague term people throw around, AGI is coming, it's going to take over everything. What's your take on how far you think we might be from AGI? Do you think we're going to get there on the current trajectory we're on? Do you think we need more breakthroughs? Do you think the current approach will get us there?

**Dr. Fei Fei Li** (00:24:13):
Yeah, this is a very interesting term, Lenny. I don't know if anyone has ever defined AGI. There are many different definitions, including some kind of superpower for machines all the way to machines can become economically viable agent in the society. In other words, making salaries to live. Is that the definition of AGI? As a scientist, I take science very seriously and I enter the field because I was inspired by this audacious question of, can machines think and do things in the way that humans can do? For me, that's always the north star of AI. And from that point of view, I don't know what's the difference between AI and AGI.

**Dr. Fei Fei Li** (00:25:10):
I think we've done very well in achieving parts of the goal, including conversational AI, but I don't think we have completely conquered all the goals of AI. And I think our founding fathers, Alan Turing, I wonder if Alan Turing is around today and you ask him to contrast AI versus AGI, he might just shrugged and said, "Well, I asked the same question back in 1940s," so I don't want to get onto a rabbit hole of defining AI versus AGI. I feel AGI is more a marketing term than a scientific term as a scientist than technologist. AI is my north star, is my field's north star, and I'm happy people call it whatever name they want to call it.

**Lenny Rachitsky** (00:26:05):
So let me ask you maybe this way, like you described, there's kind of these components that from ImageNet and AlexNet took us to where we're today, GPUs essentially, data, label data, just like the algorithm of the model. There's also just the transformer feels like an important step in that trajectory. Do you feel like those are the same components that'll get us to, I don't know, 10 times smarter model, something that's like life-changing for the entire world? Or do you think we need more breakthroughs? I know we're going to talk about world models, which I think is a component of this, but is there anything else that you think is like, oh, this will plateau, or okay, this will take us just need more data, more compute, more GPUs?

**Dr. Fei Fei Li** (00:26:44):
Oh no, I definitely think we need more innovations. I think scaling loss of more data, more GPUs, and bigger current model architecture is there's still a lot to be done there, but I absolutely think we need to innovate more. There's not a single deeply scientific discipline in human history that has arrived at a place that says we're done, we're done innovating and AI is one of the, if not the youngest discipline in human civilization in terms of science and technology, we're still scratching the surface. For example, like I said, we're going to segue into world models. Today, you take a model and run it through a video of a couple of office rooms and ask the model to count the number of chairs. And this is something a toddler could do or maybe an elementary school kid could do, and AI could not do that, right?

**Dr. Fei Fei Li** (00:27:50):
So there's just so much AI today could not do, then let alone thinking about how did someone like Isaac Newton look at the movements of the celestial bodies and derive an equation or a set of equations that governs the movement of all bodies, that level of creativity, extrapolation, abstraction. We have no way of enabling AI to do that today. And then let's look at emotional intelligence. If you look at a student coming to a teacher's office and have a conversation about motivation, passion, what to learn, what's the problem that's really bothering you. That conversation, as powerful as today's conversational bots are, you don't get that level of emotional cognitive intelligence from today's AI. So there's a lot we can do better, and I do not believe we're done innovating.

**Lenny Rachitsky** (00:29:00):
Demis had this really interesting interview recently from DeepMind slash Google where someone asked him just like, "What do you think, how far are we from AGI? What does it look like going through there?" He had a really interesting way of approaching it is if we were to give the most cutting-edge model all the information until the end of the 20th century, see if it could come up with all the breakthroughs Einstein had and so far we're nowhere near that, but they could just-

**Dr. Fei Fei Li** (00:29:22):
No, we're not. In fact, it's even worse. Let's give AI all the data including modern instruments data of celestial bodies, which Newton did not have, and give it to that and just ask AI to create the 17th century set of equations on the laws of bodily movements. Today's AI cannot do that.

**Lenny Rachitsky** (00:29:49):
All right. We're ways away is what I'm hearing.

**Dr. Fei Fei Li** (00:29:50):
Yeah.

**Lenny Rachitsky** (00:29:51):
Okay, so let's talk about world models. To me, this is just another really amazing example of you being ahead of where people end up. So you were way ahead on, okay, we just need a lot of clean data for AI and neural networks to learn. You've been talking about this idea of world models for a long time. You started a company to build, essentially there's language models. This is a different thing. This is a world model. We'll talk about what that is. And now, as I was preparing for this Elon's talking about world models, Jensen's talking about world models, I know Google's working on this stuff. You've been at this for a long time and you actually just launched something that's going, we're going to talk about right before this podcast airs. Talk about what is a world model? Why is it so important?

**Dr. Fei Fei Li** (00:30:33):
I'm very excited to see that more and more people are talking about world models like Elon, like Jensen. I have been thinking about really how to push AI forward all my life and the large language models that came out of the research world and then OpenAI and all this, for the past few years, were extremely inspiring even for a researcher like me. I remembered when GPT2 came out, and that was in, I think, late 2020. I was co-director, I still am, but I was at that time full-time co-director of Stanford's Human-Centered AI institute, and I remember it was... The public was not aware of the power of the large language model yet, but as researchers, we were seeing it, we're seeing the future, and I had pretty long conversations with my natural language processing colleagues like Percy Liang and Chris Manning. We were talking about how critical this technology is going to be and the Stanford AI Institute, Human-Centered AI Institute, HAI, was the first one to establish a full research center foundation model.

**Dr. Fei Fei Li** (00:31:59):
We were, Percy Liang, and many researchers led the first academic paper foundation model. So it was just very inspiring for me. Of course, I come from the world of visual intelligence and I was just thinking there's so much we can push forward beyond language because humans, humans use our sense of spatial intelligence, a world understanding to do so many things and they are beyond language. Think about a very chaotic first responder scene, whether it's fire or some traffic accident or some natural disaster. And if you immerse yourself in those scene and think about how people organize themselves to rescue people, to stop further disasters, to put down fires, a lot of that is movements is spontaneous understanding of objects, worlds, human situational awareness. Language is part of that, but a lot of those situations, language cannot get you to put down the fire.

**Dr. Fei Fei Li** (00:33:21):
So that is, what is that? I was thinking a lot. And in the meantime, I was doing a lot of robotics research and it dawned on me that the linchpin of connecting the additional intelligence, in addition to language embodied AI, which are robotics, connecting visual intelligence, is the sense of spatial intelligence about understanding the world. And that's when I think it was 2024, I gave a TED talk about spatial intelligence at world models. And I start formulating this idea back in 2022 based on my robotics and computer vision research. And then one thing that was really clear to me is that I really want to work with the brightest technologists and move as fast as possible to bring this technology to life. And that's when we founded this company called World Labs. And you can see the word world is in the title of our company because we believe so much in world modeling and spatial intelligence.

**Lenny Rachitsky** (00:34:41):
People are so used to just chatbots and that's a large language model. A simple way to understand a world model is you basically describe a scene and it generates an infinitely explorable world. We'll link to the thing you launched, which we'll talk about, but just is that a simple way to understand it?

**Dr. Fei Fei Li** (00:34:56):
That's part of it, Lenny. I think a simple way to understand a world model is that this model can allow anyone to create any worlds in their mind's eye by prompting whether it's an image or a sentence. And also be able to interact in this world whether you are browsing and walking or picking objects up or changing things as well as to reason within this world, for example, if the person consuming, if the agent consuming this output of the world model is a robot, it should be able to plan its path and help to tidy the kitchen, for example. So world model is a foundation that you can use to reason, to interact, and to create worlds.

**Lenny Rachitsky** (00:36:00):
Great. Yeah. So robots feels like that's potentially the next big focus for AI researchers and just the impact on the world. And what you're saying here is this is a key missing piece of making robots actually work in the real world, understanding how the world works.

**Dr. Fei Fei Li** (00:36:17):
Yeah. Well, first of all, I do think there's more than robots. That's exciting. But I agree with everything you just said. I think world modeling and spatial intelligence is a key missing piece of embodied AI. I also think let's not underestimate that humans are embodied agents and humans can be augmented by AI's intelligence. Just like today, humans are language animals, but we're very much augmented by AI helping us to do language tasks including software engineering. I think that we shouldn't underestimate or maybe we tend not to talk about how humans, as an embodied agents, can actually benefit so much from world models and spatial intelligence models as well as robots can.

**Lenny Rachitsky** (00:37:15):
So the big unlocks here, robots, which a huge deal if this works out, imagine each of us has robots doing a bunch of stuff for us, they help us with disasters, things like that. Games obviously is a really cool example, just like infinitely playable games that you just invent out of your head. And then creativity feels like just like being fun, having fun, being creative, thinking of magic, wild new worlds, and environments.

**Dr. Fei Fei Li** (00:37:39):
And also design, humans design from machines to buildings to homes and also scientific discovery. There is so much. I like to use the example of the discovery of the structure of DNA. If you look at one of the most important piece in DNA's discovery history is the x-ray diffraction photo that was captured by Rosalind Franklin, and it was a flat 2D photo of a structure that it looks like a cross with diffractions. You can google those photos. But with that 2D flat photo, the humans, especially two important humans, James Watson and Francis Crick, in addition to their other information, was able to reason in 3D space and deduce a highly three-dimensional double helix structure of the DNA. And that structure cannot possibly be 2D. You cannot think in 2D and deduce that structure. You have to think in 3D spatial, use the human spatial intelligence. So I think even in scientific discovery, spatial intelligence or AI-assisted spatial intelligence is critical.

**Lenny Rachitsky** (00:39:08):
This is such an example of, I think it was Chris Dixon that had this line that the next big thing is going to start off feeling like a toy. When ChatGPT just came out, I remember Sam Altman just tweeted it as like, "Here's a cool thing we're playing with, check it out." Now, it's the fastest growing product to all of history, changed the world. And it's oftentimes the things that just look like, okay, this is cool, that it's a fun to play with that end up changing the world most.

**Dr. Fei Fei Li** (00:40:51):
Yeah, we've known each other for many years, but yes, right now they're investors of World Labs.

**Lenny Rachitsky** (00:40:57):
Amazing. Okay, so I asked him what I should ask you about and he suggested ask you why is the bitter lesson alone not likely to work for robots? So first of all, just explain what the bitter lesson was in the history of AI and then just why that won't get us to where we want to be with robots.

**Dr. Fei Fei Li** (00:41:17):
Well, first of all, there are many bitter lessons, but the bitter lessons everybody refers to is a paper written by Richard Sutton who won the Turing Award recently, and he does a lot of reinforcement learning. And Richard has said, if you look at the history, especially the algorithmic development of AI, it turns out simpler model with a ton of data always win at the end of the day instead of the more complex model with less data. I mean, that was actually... This paper came years after ImageNet. That to me was not bitter; it was a sweet lesson. That's why I built ImageNet because I believe that big data plays that role. So why can't bitter lesson work in robotics alone? Well, first of all, I think we need to give credit to where we are today. Robotics is very much in the early days of experimentation.

**Dr. Fei Fei Li** (00:42:25):
The research is not nearly as mature as say language models. So many people are still experimenting with different algorithms and some of those algorithms are driven by big data. So I do think big data will continue to play a role in robotics, but what is hard for robotics, there are a couple of things. One is that it's harder to get data. It's a lot harder to get data. You can say, well, there's web data. This is where the latest robotics research is using web videos. And I think web videos do play a role. But if you think about what made language model worth a very... As someone who does computer vision and spatial intelligence and robotics, I'm very jealous of my colleagues in language because they had this perfect setup where their training data are in words, eventually tokens, and then they produce a model that outputs words.

**Dr. Fei Fei Li** (00:43:36):
So you have this perfect alignment between what you hope to get, which we call objective function and what your training data looks like. But robotics is different. Even spatial intelligence is different. You hope to get actions out of robots, but your training data lacks actions in 3D worlds, and that's what robots have to do, right? Actions in 3D worlds. So you have to find different ways to fit a, what do they call, a square in a round hole, that what we have is tons of web videos. So then we have to start talking about adding supplementing data such as teleoperation data or synthetic data so that the robots are trained with this hypothesis of bitter lesson, which is large amount of data. I think there's still hope because even what we are doing in world modeling will really unlock a lot of this information for robots.

**Dr. Fei Fei Li** (00:44:53):
But I think we have to be careful because we're at the early days of this and bitter lesson is still to be tested because we haven't fully figured out the data for. Another part of the bitter lesson of robotics I think we should be so realistic about is again, compared to language models or even spatial models, robots are physical systems. So robots are closer to self-driving cars than a large language model. And that's very important to recognize. That means that in order for robots to work, we not only need brains, we also need the physical body. We also need application scenarios. If you look at the history of self-driving car, my colleague Sebastian Thrun took Stanford's car to win the first DARPA challenge in 2006 or 2005. It's 20 years since that prototype of a self-driving car being able to drive 130 miles in the Nevada desert to today's Waymo and on the street of San Francisco.

**Dr. Fei Fei Li** (00:46:17):
And we're not even done yet. There's still a lot. So that's a 20-year journey. And self-driving cars are much simpler robots, they're just metal boxes running on 2D surfaces, and the goal is not to touch anything. Robot is 3D things running in 3D world, and the goal is to touch things. So the journey is going to be, there's many aspects, elements, and of course one could say, well, the self-driving car, early algorithm were pre deep learning era. So deep learning is accelerating the brains. And I think that's true. That's why I'm in robotics, that's why I'm in spatial intelligence and I'm excited by it. But in the meantime, the car industry is very mature and productizing also involves the mature use cases, supply chains, the hardware. So I think it's a very interesting time to work in these problems. But it's true, Ben is right. We might still be subject to a number of bitter lessons.

**Lenny Rachitsky** (00:47:28):
Doing this work, do you ever just feel awe for the way the brain works and is able to do all of this for us? Just the complexity just to get a machine to just walk around and not hit things and fall, does just give you more respect for what we've already got?

**Dr. Fei Fei Li** (00:47:44):
Totally. We operate on about 20 watts. That's dimmer than any light bulb in the room I'm in right now. And yet we can do so much. So I think actually the more I work in AI, the more I respect humans.

**Lenny Rachitsky** (00:48:03):
Let's talk about this product you just launched. It's called Marble, a very cute name. Talk about what this is, why this is important. I've been playing with it, it's incredible. We'll link to it for folks to check it out. What is Marble?

**Dr. Fei Fei Li** (00:48:14):
Yeah, I'm very excited. So first of all, Marble is one of the first product that World Labs has rolled out. World Labs is a foundation frontier model company. We are founded by four co-founders who have deep technical history. My co-founders, Justin Johnson, Christoph Lassner, and Ben Mildenhall. We all come from the research field of AI, computer graphics, computer vision, and we believe that spatial intelligence and world modeling is as important, if not more, to language models and complementary to language models. So we wanted to seize this opportunity to create deep tech research lab that can connect the dots between frontier models with products. So Marble is an app that's built upon our frontier models. We've spent a year and plus building the world's first generative model that can output genuinely 3D worlds. That's a very, very hard problem.

**Dr. Fei Fei Li** (00:49:30):
And it was a very hard process and we have a team of incredible, founding team of incredible technologists from incredible teams. And then around just a month or two ago, we saw the first time that we can just prompt with a sentence and the image and multiple images and create worlds that we can just navigate in. If you put it on Google, which we have an option to let you do that, you can even walk around. Even though we've been building this for quite a while, it was still just awe-inspiring and we wanted to get into the hands of people who need it. And then we know that so many creators, designers, people who are thinking about robotic simulation, people who are thinking about different use cases of navigable interactable, immersive worlds game developers will find this useful. So we developed Marble as a first step. It's again, still very early, but it's the world's first model doing this, and it's the world's first product that allows people to just prompt, we call it prompt to worlds.

**Lenny Rachitsky** (00:51:00):
Well, I've been playing around with it. It is insane. You could just have a little Shire world where you just infinitely walk around middle earth basically, and there's no one there yet, but it's insane. You just go anywhere. There's dystopian world. I'm just looking at all these examples and my favorite part, actually, I don't know if there's a feature or bug, you can see the dots of the world before it actually renders with all the textures. And I just love like, you get a glimpse into what is going on with this model, basically-

**Dr. Fei Fei Li** (00:51:27):
That is so cool to hear because this is where, as a researcher, I am learning because the dots that lead you into the world was an intentional feature visualization, is not part of the model. The model actually just generates the world. But we were trying to find a way to guide people into the world, and a number of engineers worked on different versions, but we converged on the dot, and so many people, you're not the only one, told us how delightful that experience is, and it was really satisfying for us to hear that this intentional visualization feature that's not just the big hardcore model actually has delighted our users.

**Lenny Rachitsky** (00:52:19):
Wow. So you add that to make it more, like to have humans understand what's going on-

**Dr. Fei Fei Li** (00:52:24):
To have fun, yes.

**Lenny Rachitsky** (00:52:24):
... get more delightful. Wow, that is hilarious. It makes me think about LLMs and the way they, it's not the same thing, but they talk about what they're thinking and what they're doing.

**Dr. Fei Fei Li** (00:52:32):
Yes, it is. It is.

**Lenny Rachitsky** (00:52:34):
It also makes me think about just the Matrix. It's exactly the Matrix experience. I don't know if that was your inspiration.

**Dr. Fei Fei Li** (00:52:42):
Well, like I said, a number of engineers worked on that. It could be their inspiration.

**Lenny Rachitsky** (00:52:48):
It's in their subconscious. Okay, so just for folks that may want to play around with this, maybe like, what are some applications today that folks can start using today? What's your goal with this launch?

**Dr. Fei Fei Li** (00:52:59):
Yeah, so we do believe that world modeling is very horizontal, but we're already seeing some really exciting use cases, virtual production for movies, because what they need are 3D worlds that they can align with the camera. So when the actors are acting on it, they can position the camera and shoot the segments really well. And we're already seeing incredible use. In fact, I don't know if you have seen our launch video showing Marble. It was produced by a virtual production company. We collaborated with Sony and they use Marble scenes to shoot those videos. So we were collaborating with those technical artists and directors, and they were saying, this has cut our production time by 40X. In fact, it has to-

**Lenny Rachitsky** (00:53:00):
40X?

**Dr. Fei Fei Li** (00:53:59):
Yes, in fact it has to, because we only had one month to work on this project and there were so many things they were trying to shoot. So using Marble really, really significantly accelerated the virtual production for VFX and movies. That's one use cases. We are already seeing our users taking our Marble scene and taking the mesh export and putting games, whether it's games on VR or just fun games that they have developed. We are showing an example of robotic simulation because when I was, I mean I still am a researcher doing robotic training. One of the biggest pain point is to create synthetic data for training robots. And this synthetic data needs to be very diverse. They need to come from different environments with different objects to manipulate. And one path to it is to ask computers to simulate.

**Dr. Fei Fei Li** (00:55:10):
Otherwise, humans have to build every single asset for robots. That's just going to take a lot longer. So we already have researchers reaching out and wanting to use Marble to create those synthetic environments. We also have unexpected user outreach in terms of how they want to use Marble. For example, a psychologist team called us to use Marble to do psychology research. It turned out some of the psychiatric patients they study, they need to understand how their brain respond to different immersive things of different features. For example, messy scenes or clean scenes or whatever you name it. And it's very hard for researchers to get their hands on these kind of immersive scenes and it will take them too long and too much budget to create. And Marble is a really almost instantaneous way of getting so many of these experimental environments into their hands. So we're seeing multiple use cases at this point. But the VFX, the game developers, the simulation developers as well as designers are very excited.

**Lenny Rachitsky** (00:56:39):
This is very much the way things work in AI. I've had other AI leaders on the podcast and it's always put things out there early as soon as you can to discover where the big use cases are. The head of ChatGPT told me how, when they first put out ChatGPT, he was just scanning TikTok to see how people were using it and all the things they were talking about, and that's what convinced them where to lean in and help them see how people actually want to use it. I love this last use case for therapy. I'm just imagining heights, people dealing with heights or snakes or spiders, which-

**Dr. Fei Fei Li** (00:57:11):
It's amazing. A friend of mine last night literally called me and talked about his height scare and asked me if Marble should be used. It's amazing you went straight there.

**Lenny Rachitsky** (00:57:24):
Because imagining all the exposure therapy stuff, this could be so good for that. That is so cool. Okay, so I should have asked you this before, but I think there's going to be a question of just, how does this differ from things like VO3 and other video generation models? It's pretty clear to me, but I think it might be helpful just to explain how this is different from all the video AI tools people have seen.

**Dr. Fei Fei Li** (00:57:46):
World Labs' thesis is that spatial intelligence is fundamentally very important, and spatial intelligence is not just about videos. In fact, the world is not passively watching videos passing by. I love, Plato has the allegory of the cave analogy to describe vision. He said that imagine a prisoner tied on his chair, not very humane, but in a cave watching a full life theater in front of him, but the actual life theater that actors are acting is behind his back. It was just lit so that the projection of the action is on a wall of the cave. And then the goal, the task of this prisoner is to figure out what's going on. It's a pretty extreme example, but it really shows, it describes what vision is about, is that to make sense of the 3D world or 4D world out of 2D. So spatial intelligence to me is deeper than only creating that flat 2D world.

**Dr. Fei Fei Li** (00:59:14):
Spatial intelligence to me is the ability to create, reason, interact, make sense of deeply spatial world, whether it's 2D or 3D or 4D, including dynamics and all that. So World Lab is focusing on that, and of course the ability to create videos per se could be part of this. And in fact, just a couple of weeks ago, we rolled out the world's first real time demoable, real-time video generation on a single H100 GPU. So part of our technology includes that, but I think Marble is very different because we really want creators, designers, developers to have in their hands a model that can give them worlds with 3D structures so they can use it for their work. And that's why Marble is so different.

**Lenny Rachitsky** (01:00:21):
The way I see it is it's a platform for a ton of opportunity to do stuff. As you described, videos are just like, here's a one-off video that's very fun and cool and you could... And that's it. That's it. And you move on.

**Dr. Fei Fei Li** (01:00:33):
By the way, we could in Marble, we could allow people to export in video forms. So you could actually, like you said, you go into a world, so let's say it's a hobbit cave. You can actually, especially as a creator, you have such a specific way of moving the camera in a trajectory in the director's mind, and then you can export that from Marble into a video.

**Lenny Rachitsky** (01:01:02):
What does it take to create something like this? Just how big is the team, how many GPUs you work in? Anything you can share there. I don't know how much of this is private information, but just what does it take to create something like this that you've launched here?

**Dr. Fei Fei Li** (01:01:12):
It takes a lot of brain power. So we just talk about 20 watts per brain. So from that point of view, it's a small number, but it's actually incredible. It's half billion years of evolution to give us those power. We have a team of 30-ish people now, and we are predominantly researchers and research engineers, but we also have designers and product. We actually really believe that we want to create a company that's anchored in the deep tech of spatial intelligence, but we are actually building serious products. So we have this integration of R&D and productization, and of course, we use a ton of GPUs.

**Lenny Rachitsky** (01:02:15):
That's the technical thing.

**Dr. Fei Fei Li** (01:02:17):
Happy to hear.

**Lenny Rachitsky** (01:02:20):
Well, congrats on the launch. I know this is a huge milestone. I know this took a ton of work.

**Dr. Fei Fei Li** (01:02:20):
Thank you.

**Lenny Rachitsky** (01:02:23):
So I just want to say congrats to you and your team. Let me talk about your founder journey for a moment. So you're a founder of this company. You started how many years ago? A couple of years ago, two, three years ago?

**Dr. Fei Fei Li** (01:02:23):
A year ago.

**Lenny Rachitsky** (01:02:33):
A year ago?

**Dr. Fei Fei Li** (01:02:34):
A year plus.

**Lenny Rachitsky** (01:02:37):
A year? Okay. Wow.

**Dr. Fei Fei Li** (01:02:37):
Probably, 18 month, yeah.

**Lenny Rachitsky** (01:02:38):
Okay. What's something you wish you knew before you started this that you wish you could whisper into the ear of Fei-Fei of 18 months ago?

**Dr. Fei Fei Li** (01:02:46):
Well, I continue to wish I know the future of technology. I think actually that's one of our founding advantage is that we see the future earlier in general than most people. But still, man, this is so exciting and so amazing that what's unknown and what's coming, but I know the reason you're asking me this question is not about the future of technology. Furthermore, look, I did not start a company of this scale at 20-year-old. So I started a dry cleaner when I was 19, but that's a little smaller scale.

**Lenny Rachitsky** (01:03:30):
We got to talk about that.

**Dr. Fei Fei Li** (01:03:32):
And then I founded Google Cloud AI and then I founded an institute at Stanford but those are different beasts. I did feel I was a little more prepared as a founder of the grinding journey compared to maybe the 20-year-old founders. But I still, I'm surprised, and it puts me into paranoia sometimes that how intensely competitive AI landscape is from the model, the technology itself, as well as talents. And when I founded the company, we did not have these incredible stories of how much certain talents would cost. So these are things that continue to surprise me and I have to be very alert about.

**Lenny Rachitsky** (01:04:40):
So the competition you're talking about is the competition for talent, the speed at which just how things are moving.

**Dr. Fei Fei Li** (01:04:46):
Yeah.

**Lenny Rachitsky** (01:04:47):
Yeah. You mentioned this point that I want to come back to that if you just look over the course of your career, you were at all of the major collections of humans that led to so many of the breakthroughs that are happening today. Obviously, we talk about ImageNet also just SAIL at Stanford is where a lot of the work happened, Google Cloud, which a lot of the breakthroughs happened. What brought you to those places? Like for people looking for how to advance in their career, be at the center of the future, just is there a through line there of just what pulled you from place to place and pulled you into those groups that might be helpful for people to hear?

**Dr. Fei Fei Li** (01:05:25):
Yeah, this is actually a great question, Lenny, because I do think about it, and obviously we talked about it's curiosity and passion that brought me to AI, that is more a scientific north star, right? I did not care if AI was a thing or not, so that was one part. But how did I end up choosing in the particular places I work in, including starting World Labs, is I think I'm very grateful to myself or maybe to my parents' genes. I'm an intellectually very fearless person, and I have to say when I hire young people, I look for that because I think that's a very important quality if one wants to make a difference, is that when you want to make a difference, you have to accept that you're creating something new or you're diving into something new. People haven't done that. And if you have that self-awareness, you almost have to allow yourself to be fearless and to be courageous.

**Dr. Fei Fei Li** (01:06:42):
So when I, for example, came to Stanford, in the world of academia, I was very close to this thing called tenure, which is have the job forever at Princeton. But I chose to come to Stanford because... I love Princeton. It's by alma mater. It's just at that moment there are people who are so amazing at Stanford and the Silicon Valley ecosystem was so amazing that I was okay to take a risk of restarting my tenure clock. Becoming the first female director of SAIL, I was actually relatively speaking a very young faculty at that time, and I wanted to do that because I care about that community. I didn't spend too much time thinking about all the failure cases.

**Dr. Fei Fei Li** (01:07:46):
Obviously, I was very lucky that the more senior faculty supported me, but I just wanted to make a difference. And then going to Google was similar. I wanted to work with people like Jeff Dean, Jeff Hinton, and all these incredible demists, the incredible people. The same with World Labs. I have this passion. And I also believe that people with the same mission can do incredible things. So that's how it guided my through line. I don't overthink of all possible things that can go wrong because that's too many.

**Lenny Rachitsky** (01:08:33):
I feel like an important element of this is not focusing on the downside, focusing more on the people, the mission. What gets you excited, what do you think, the curiosity.

**Dr. Fei Fei Li** (01:08:43):
Yeah. I do want to say one thing to all the young talents in AI, the engineers, the researchers out there, because some of you apply to World Labs, I feel very privileged you considered World Labs. I do find many of the young people today think about every single aspect of an equation when they decide on jobs. At some point, maybe that's the way they want to do it, but sometimes I do want to encourage young people to focus on what's important because I find myself constantly in mentoring mode when I talk to job candidates. Not necessarily recruiting or not recruiting, but just in mentoring mode when I see an incredible young talent who is over-focusing on every minute dimension and aspect of considering a job, when maybe the most important thing is where's your passion? Do you align with the mission? Do you believe and have faith in this team? And just focus on the impact and you can make and the kind of work and team you can work with.

**Lenny Rachitsky** (01:10:05):
Yeah, it's tough. It's tough for people in the AI space. Now there's so much, so much at them, so much new, so much happening, so much FOMO.

**Dr. Fei Fei Li** (01:10:11):
That's true.

**Lenny Rachitsky** (01:10:12):
I could see the stress. And so I think that advice is really important. Just like what will actually make you feel fulfilled in what you're doing, not just where's the fastest growing company, where's the... Who's going to win? I don't know. I want to make sure I ask you about the work you're doing today at Stanford, at the HCI. I think it's the-

**Dr. Fei Fei Li** (01:10:12):
HAI.

**Lenny Rachitsky** (01:10:30):
HAI, Human-Centered AI Institute. What are you doing there? I know this is a thing you do on the side still.

**Dr. Fei Fei Li** (01:10:36):
So yes, HAI, Human-Centered AI Institute was co-founded by me and a group of faculty like Professor John Etchemendy, Professor James Landay, Professor Chris Manning back in 2018. I was actually finishing my last sabbatical at Google and it was a very, very important decision for me because I could have stayed in industry, but my time at Google taught me one thing is AI is going to be a civilization of technology. And it dawned on me how important this is to humanity to the point that I actually wrote a piece in New York Times, that year 2018, to talk about the need for a guiding framework to develop and to apply AI. And that framework has to be anchored in human benevolence, in human centeredness. And I felt that Stanford, one of the world's top university in the heart of Silicon Valley that gave birth to important companies from NVIDIA to Google, should be a thought leader to create this human-centered AI framework and to actually embody that in our research education and policy and ecosystem work.

**Dr. Fei Fei Li** (01:12:10):
So I founded HAI. Fast-forward, after six, seven years, it has become the world's largest AI institute that does human-centered research, education, ecosystem, outreach, and policy impact. It involves hundreds of faculty across all eight schools at Stanford, from medicine to education, to sustainability to business, to engineering, to humanities to law. And we support researchers, especially at the interdisciplinary area from digital economy, to legal studies, to political science, to discovery of new drugs, to new algorithms to that's beyond transformers. We also actually put a very strong focus on policy because when we started HAI, I realized that Silicon Valley did not talk to Washington DC and or Brussels or other parts of the world.

**Dr. Fei Fei Li** (01:13:27):
And given how important this technology is, we need to bring everybody on board. So we created multiple programs from congressional bootcamp to AI index report to policy briefing, and we especially participated in policymaking including advocating for a national AI research cloud bill that was passed in the first Trump administration and participating in state level regulatory AI discussions. So there's a lot we did, and I continue to be one of the leaders even though I'm much less involved operationally because I care not only we create this technology, but we use it in the right way.

**Lenny Rachitsky** (01:14:24):
Wow. I was not aware of all that other work you were doing. As you're talking, I was reminded Charlie Munger had this quote, "Take a simple idea and take it very seriously." I feel like you've done that in so many different ways and stayed with it and it's unbelievable the impact that you've had in so many ways over the years. I'm going to skip the lightning round and I'm just looking to ask you one last question. Is there anything else that you wanted to share? Anything else you want to leave listeners with?

**Dr. Fei Fei Li** (01:14:52):
I am very excited by AI, Lenny. I want to answer one question that when I travel around the world, everybody asks me is that, if I'm a musician, if I'm a teacher, middle school teacher, if I'm a nurse, if I'm an accountant, if I'm a farmer, do I have a role in AI or is AI just going to take over my life or my work? And I think this is the most important question of AI and I find that in Silicon Valley, we tend not to speak heart-to-heart with people, with people like us and not like us in Silicon Valley, but all of us, we tend to just toss around words like infinite productivity or infinite leisure time or infinite power or whatever. But at the end of the day, AI is about people. And when people ask me that question, it's a resounding yes, everybody has a role in AI.

**Dr. Fei Fei Li** (01:16:03):
It depends on what you do and what you want. But no technology should take away human dignity and the human dignity and agency should be at the heart of the development, the deployment, as well as the governance of every technology. So if you are a young artist and your passion is storytelling, embrace AI as a tool. In fact, embrace Marble. I hope it becomes a tool for you because the way you tell your story is unique and the world still needs it. But how you tell your story, how do you use the most incredible tool to tell your story in the most unique way is important. And that voice needs to be heard. If you are a farmer near retirement, AI still matters because you are a citizen. You can participate in your community, you should have a voice in how AI is used, how AI is applied.

**Dr. Fei Fei Li** (01:17:14):
You work with people that you can encourage all of you to use AI to make life easier for you. If you are a nurse, I hope you know that at least in my career, I have worked so much in healthcare research because I feel our healthcare workers should be greatly augmented and helped by AI technology, whether it's smart cameras to feed more information or robotic assistance because our nurses are overworked, overfatigued, and as our society ages, we need more help for people to be taken care of. So AI can play that role. So I just want to say that it's so important that even a technologist like me are sincere about that everybody has a role in AI.

**Lenny Rachitsky** (01:18:17):
What a beautiful way to end it. Such a tie back to where we started about how it's up to us and take individual responsibility for what AI will do in our lives. Final question, where can folks find Marble? Where can they go, maybe try to join World Labs if they want to? What's the website? Where do people go?

**Dr. Fei Fei Li** (01:18:34):
Well, World Labs website is www.worldlabs.ai and you can find our research progress there. We have technical blogs. You can find Marble, the product there. You can sign in there. You can find our job posts link there. We're in San Francisco. We love to work with the world's best talents.

**Lenny Rachitsky** (01:19:02):
Amazing. Fei-Fei, thank you so much for being here.

**Dr. Fei Fei Li** (01:19:04):
Thank you, Lenny.

**Lenny Rachitsky** (01:19:06):
Bye everyone.

**Lenny Rachitsky** (01:19:09):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [11/17] Slack founder: Mental models for building products people love ft. Stewart Butterfield

**Stewart Butterfield** (00:00:00):
This is 2014. That was the year that Slack actually launched. I was interviewed by MIT Technology Review and asked if we were working to improve Slack. I said, "I feel like what we have right now is just a giant piece of shit. It's just terrible and we should be humiliated that we offer this to the public."

**Stewart Butterfield** (00:00:14):
To me that was like, "You should be embarrassed." If you can't see almost limitless opportunities to improve, then you shouldn't be designing the product.

**Lenny Rachitsky** (00:00:24):
Slack was famous for being one of the early, consumerized B2B SaaS products.

**Stewart Butterfield** (00:00:29):
At more than one company all hands, I made everyone in the company repeat this as a chant. In the long run, the measure of our success will be the amount of value that we create for customers, and you can put effort into demonstrating that you have created this value and stuff like that, but there's no substitute for actually having created it.

**Lenny Rachitsky** (00:00:45):
Something else I heard that you often espouse is friction in a product experience is actually often a good thing?

**Stewart Butterfield** (00:00:52):
It became an assumption that it should always be trying to remove friction when the challenge is really comprehension. If your software stops me and asks me to make a decision and I don't really understand it, you make me feel stupid. If people could get over the idea of reducing friction as a number of goal or reducing the number of clicks or taps to do something, and instead focus on how can I make this simple? How do I prevent people from having to think in order to use my software?

**Lenny Rachitsky** (00:01:15):
You started two companies, both famously pivoted. I imagine many people come to you for advice on pivoting.

**Stewart Butterfield** (00:01:20):
The decision is about have you exhausted the possibilities? Creating the distance so that you can make an intellectual rational decision about it rather than an emotional decision is essential. And the reason I say you have to be coldly rational about it is because it's fucking humiliating.

**Lenny Rachitsky** (00:01:36):
Today, my guest is Stewart Butterfield, a founder and product legend who rarely does podcasts. Stewart founded Flickr and then Slack, which he sold to Salesforce in one of the biggest acquisitions in tech history at the time. There is so much product and leadership wisdom locked away in his head. I feel like our conversation just scratched the surface. We chat about utility curves, something he calls the owner's delusion, a hilarious pattern he sees at companies he calls hyperrealistic work-like activities, what he's learned about product and craft and taste and Parkinson's law, why you need to obsess with not making your users think, the backstory on his legendary we don't sell saddles here memo, and so much more. A huge thank you to Noah Weiss, Chris Cordell, Ali Rael, and Johnny Rogers for suggesting topics and questions for this conversation. This is a really special one and I really hope to have Stewart back to delve even deeper.

**Stewart Butterfield** (00:05:02):
Thank you for having me. I'm excited.

**Lenny Rachitsky** (00:05:05):
I'm even more excited. I'm so honored to have you here. I never told you this, but you've been towards the very top of my wish list of guests I have on this podcast ever since I started this podcast a few years ago, so I'm very excited that we're finally making this happen. I have so many questions for you. My first question is just what the heck are you up to these days? I feel like ever since you left Slack, we haven't heard much from Stewart. I'm curious what you're up to you hopefully or just chilling.

**Stewart Butterfield** (00:05:28):
I'm mostly just chilling. I left Salesforce two and a half years ago and I have a two and a half year old, so she was actually born three days after my last day, so a lot of time with family and it's an enormous privilege to be able to spend time with young kids while they're young. No new company to announce or anything like that. I do get a lot of emails and texts. Basically every three to six weeks there's this cycle because Cal Henderson who's the CTO of Slack and who also, we worked together on Flickr, so have worked together now for 23 years, have been talking about what we want to do next if there is something.

**Stewart Butterfield** (00:06:10):
But honestly, the big challenge has been I think these things are destroying the world and what we're good at is making software. So you find some way to make software that helped people use their phones less often, then that would be a big winner, but haven't come up with anything good. A lot of philanthropic work, nothing to announce there yet, but there's some cool projects that I'm working on, and a lot of just personal creative art projects and supporting other artists and stuff like that.

**Lenny Rachitsky** (00:06:45):
To prep for this chat, I talked to so many people that have worked with you over the years to try to figure out what you taught them about building product, building teams, building companies that most stuck with them, that most helped them build amazing products. The first is a concept called utility curves. This came up a bunch across so many people that have worked with you. Talk about what is a utility curve, how you use that to build better products.

**Stewart Butterfield** (00:07:08):
This is pretty easy because it's a very familiar S-curve where you have, it's flat and it starts arcing up and then there's a really steep part and then it levels off again. And on the horizontal axis, you can think of cost or effort and on the vertical axis, it's value or convenience. It depends exactly what you're talking about, but the idea is the first bit of effort you put into something doesn't result in a huge amount of value. And then there's some magic threshold where it produces an enormous amount of value and then continued investment doesn't really pay off. The most basic example I can think of is let's say you're making a hammer, and on that bottom axis, it's now quality, and if the hammer has a handle that breaks with any impact, then is totally useless. And if you make it a little bit stronger, it's still pretty useless and it's like junk, junk, junk, junk, junk. Okay, good, great. Then it doesn't matter anymore.

**Stewart Butterfield** (00:08:06):
If you're making an app, okay, this app's going to have users and so let's make a user's table and a database, and so far you have generated no value. The reason I felt like this was so important is because we would talk about a feature, and usually features are thought of as a binary. You either have this feature or you don't. The argument I guess was have we just not invested enough in this or have we got all the value or convenience or quality or whatever that we could get out of this? And we had pointed diminishing returns and it just doesn't matter.

**Stewart Butterfield** (00:08:46):
I think in many cases, people will add a feature, it's not good enough and so people don't use it or appreciate it, but now you've added some complexity to the app and then people give up or take it back or they try something in testing and they don't get the results they want, and so they decide that this a thing is worth doing. We would try to really investigate and decide whether we were on the first shallow part of the curve, the second shallow part of the curve, or we're just coming up to it. So I think it's a lot easier to understand the value of this when you're talking about a specific app and a specific feature, but I think it was ultimately helpful in getting people to understand whether something was worth it or not.

**Lenny Rachitsky** (00:09:31):
So just to mirror back what I'm hearing, there's this, if you visualize this curve at the bottom, it's like I don't even know what this is. And then up the curve is like, okay, I sort of get it. And then at the top is, okay, I can't live without this now that I understand what this is for, it feels like it's a really a different way of thinking about getting to the aha moment for someone where they see, okay, saved items, I get it, I need to use this constantly. It feels like this works both for a specific feature and also just for Slack, getting people to even understand here's what Slack can do for you. And then now I can't live without Slack. And essentially this is a lens you use to figure out where to spend product resources because if you don't get up that curve to I get it and I can't live without it, nothing else matters. Is that the framework?

**Stewart Butterfield** (00:10:13):
Yeah, and I think then you layer on another concept like the, Bezos used the term divine discontent. The line actually moves because once people are familiar with a piece of software or the way a feature is implemented or something like that, their standards go up, and so there's this competition. And again, this axis can be, utility is the best general term for it, but it could be quality, convenience, speed, it could be any number of things, but as you improve your search capability or as you improve your login experience or your forget password experience or your checkout experience or whatever everyone else is as well. And so there's this continued investment and when forget about thinking about a new feature, you're looking at how the product works overall and usually things get implemented once, and then if they're lucky, they get improved upon periodically. Most things get improved upon very infrequently and some things get improved upon never.

**Stewart Butterfield** (00:11:22):
I want to give an example at the absolute extreme because I actually don't know how long this has been, but I try not to criticize other people's software so much because I'm very familiar with the trade-offs and prioritization and how hard it can be and blah, blah, blah, blah. But okay, so most people have the Gmail Calendar app on their phone. I travel a fair bit. I'm mostly in the Eastern Time Zone, sometimes in Mountain Time, sometimes in Pacific, sometimes in English time, and sometimes in Japan, Central Europe. There's maybe 10 time zones, 12 time zones that I would ever choose. When you hit the option to set the time zone on an event in Google Calendar, on the iOS app, it presents all the time zones in the world in alphabetical order. And I mean, there's probably worse orderings, but there's no value in that.

**Stewart Butterfield** (00:12:24):
And even when you start searching, it still presents them in alphabetical order by country with that turn. So if I'm in California and I'm trying to set the appointment for next week when I'm back in New York and I type in E-A-S-T and I get a bunch of garbage, okay, Eastern, and then the first one is Eastern Australia, New South Wales, and then Eastern Australia, Queensland, and then Eastern Australia, Daylight Savings and Eastern Australia standard time. And then you're like, "Well, fuck, I can't remember which one is Daylight Savings and which one is standard time?" I could keep going like this for a while. This is an app that's used by at least hundreds of millions of people, presumably every single Google employee. It's bananas how bad it is. There's so many, there's all these clever things you could do. Like you know me, I'm on the West Coast, first option should be the East Coast and vice versa. But it definitely shouldn't be that every time zone is presented with equal value. I don't a couple hundred time zones. I grew up in Canada. Newfoundland has its own time zone, which is offset by half an hour. The population of Newfoundland has about half a million people. Not that many people go to visit Newfoundland, maybe a million people in all of history so like a million and a half out of 8 billion people. And there's Newfoundland, the same with China time, which is like 25% of the world's population in this.

**Stewart Butterfield** (00:13:51):
Anyway, that was a little bit longer than I intended to go on this example, but it's crazy because no one's going to switch to Gmail or to G Suite, Google Calendar from Outlook Exchange because the time zone picker is good, so maybe in some sense it doesn't matter, but at the same time there's a real value in delighting customers and there's an emotional connection that they form or don't form. And in some cases that could be really positive like they would recommend it. And when they switch companies or decide to start their own company, they're going to choose to use this product or advocate for it because of that emotional connection and vice versa.

**Stewart Butterfield** (00:14:34):
They'll also be like, "I hate this thing that drives me bananas. I really think we should stop using it," or advocate for the alternative. And I think people just don't appreciate or come back to those things often enough. And then there's this category of really essential parts of the app again like account creation, sign up, forgot password, things like that, that for most organizations very infrequently get a lot of love and iteration and improvement despite the fact that the quality bar has gone up across the board and continually goes up.

**Lenny Rachitsky** (00:15:11):
Let's go down that rabbit hole a little bit more around delight and craft. Slack was famous for being one of the early, let's say consumerized B2B SaaS products. Slack leaned into delight and experience and craft and a great experience. And you just as a product leader, I'd say are known as very taste forward, very craft oriented leader, which is pretty rare and I think continues to be rare. So there's a few things I want to talk about here. One is taste. I heard at a talk, you gave a talk on taste and you have a really unique perspective on just what taste is, what product taste looks like. Can you share that?

**Stewart Butterfield** (00:15:49):
There is a lot of you going back to the utility curves again, people who are obsessed with this one little thing and keep on adding more and more detailed improvements beyond the point where it makes much of a difference. But I guess a couple of things about taste. So one is can you learn to develop it? I think so because the word literally comes from experiencing food and putting stuff in your mouth. And can people become better chefs with training? Yes, absolutely. Undoubtedly, some people have a natural advantage and are born with this ability to make discernments that are difficult for other people to make and stuff like that. But you can definitely practice and you can definitely get better. The second thing I'd say is you can create a real advantage for yourself, for your product, for your company by leaning into it because most people don't have good taste and don't invest. You're probably familiar with, again, Jeff Bezos line, your margin is my opportunity and pretty obvious what he meant by that.

**Stewart Butterfield** (00:16:54):
I would tell the story at Slack over and over again. It actually made it part of the new hire welcome. I'm in Vancouver at our Vancouver office and I'm going for a walk with Brandon Velestuk who's our, at the time creative director for product development, I think that was his title. And we're in the Yaletown neighborhood in Vancouver so there's really narrow sidewalks because it used to be a warehouse district and now it's fancy restaurants and nail salons and boutiques and stuff. And as it does in Vancouver, it starts to rain. We don't have umbrellas. We're walking back to the office and most people have umbrellas and we're on these narrow sidewalks with people coming towards us with umbrellas. We noticed how few people would move their umbrella out of the way. And of course, the other person, their umbrella, the pokey bits are exactly at eye level for people walking towards them. We would get forced off the sidewalk or having to duck down or whatever.

**Stewart Butterfield** (00:17:54):
It became a game like we were guessing is this person going to tilt their umbrella out of the way so we can pass or not? And something like one-third of the people would do it. And we had this conversation about it where it's like, okay, I can think of three reasons why people wouldn't do it. One is they have very few avenues in their life to exercise power and this is one of them. And they're just, want to get out there and dominate people and cause suffering. Shouldn't ascribe to malice that which can be ascribed to ignorance so that probably is the explanation for a tiny, tiny, tiny percentage of people.

**Stewart Butterfield** (00:18:34):
But the other two explanations aren't that great either. One is that they see it's happening, they see they're pushing other people off the sidewalk or poking them in the eye or whatever, and they're just like, "Fuck, that's too bad. I wish there was something I could do about that, but I can't think of anything." And the last reason is they just don't notice it all. They're just oblivious to their impact on other people. And they're so in their head, and I can't really think of any other explanations for it besides that.

**Stewart Butterfield** (00:19:03):
And so we would say it's not like tilting your umbrella is our opportunity. That's not a great rephrase of your margin is my opportunity, but your failure to really be consider it exercise this courtesy and really be empathic about other people's experience is an advantage that you can create a critical advantage. I think that there's many reasons why Slack was successful at the moment. It was successful and we think we had a bunch of really wonderful tailwinds and all of that stuff, but it wouldn't have grown the way it did without those little conveniences which caused people to form an emotional connection because a lot of our growth came from startup A uses Slack, and then someone leaves startup A for startup B, and startup B doesn't use Slack yet. And they would be like, "Oh my God, you guys, you really, this is so good. We got to try it." And the spread was driven by that and people really genuinely advocating for it.

**Lenny Rachitsky** (00:20:07):
That is an amazing metaphor. I love that one moment became a value of product craftsmanship at Slack.

**Stewart Butterfield** (00:20:14):
Tilt your umbrella was a very common saying on company swag and stuff like that.

**Lenny Rachitsky** (00:20:20):
Is there an example, I imagine there are many, but from the time of building Slack, especially in the early days where you chose to go big on craftsmanship and experience and delight versus speed where you thought looking back that was a really great idea and worth really core just to success.

**Stewart Butterfield** (00:20:37):
Here's a bunch of little examples. Someone else came up with this idea, and I'm trying to remember who it was, but let's see, maybe Andrea Torres, maybe Ben Brown, something like that who was like, "Why did we ask people for email address and password if their ownership of the email address was the thing that allowed them to create the account in the first place? Why don't we just ask them for their email address and then send them a link?"

**Stewart Butterfield** (00:21:07):
And so when Slack's first version of the mobile app came out, we're like, "Typing your password on your phone if you have any minimal threshold of password hygiene is a terrible experience." Capital H, lowercase Q, six, caret, period. So let's just have them enter the email address. We'll send them a link. The link will automatically open the app and authenticate them. And so there's one, a little example.

**Lenny Rachitsky** (00:21:33):
Wow. So you guys invented the magic link experience.

**Stewart Butterfield** (00:21:36):
Someone else invented. I want to be clear that I had seen that idea somewhere else, someone else, a blog post about it or something like that. But we were the first ones, to my knowledge, that really scaled that and made it a standard. There is another one which we really puzzled about in the very early days where people have a long history of using messaging apps from AOL Instant Messenger to SMS to WhatsApp, where their expectation is they get a notification for every message that's received. And in the case of Slack, that doesn't make as much sense because you're a member of many channels and the messages may not be for you, and so that's why we have the @ tagging people. And we certainly didn't invent that, that was Twitter.

**Stewart Butterfield** (00:22:23):
But what we realized was people were signing up for Slack, and it's one engineer on this team inside of this larger organization, inside this larger company, and they would pull in the person next to them and they would say, "Let's try it out."

**Stewart Butterfield** (00:22:35):
And then they would send a message and then one person would be like, "I didn't get a notification. This is bullshit."

**Stewart Butterfield** (00:22:43):
We reluctantly decided that we had to send notifications for every single message as the default for new accounts. But once you had, I don't remember what the threshold would happen, I think it's once you had received 10 messages, we would pop up this little thing that says, "Hey, you have our default settings for notifications. We don't want Slack to be noisy for you. Would you like to switch to our-"

**Stewart Butterfield** (00:23:00):
... For notifications. We don't want Slack to be noisy for you. Would you like to switch to our recommended settings? And then they would just click a link and it would have what should be the default, which is, you only get a notification if it's a DM or someone tags you. But we realized it was worth that investment to get people over the hump. I'll give one more simple one and then one kind of more complex. One, people would just like the, I can't remember if it's called urgent or important, but the flag in Outlook, set the priority of a message for the recipients always got abused inside of every company. As soon as someone does it, everyone's like, "Okay, I'm going to do that too for my message."

**Stewart Butterfield** (00:23:41):
And so all of your messages have the little flag and it becomes useless. We have @everyone, which causes a notification to be sent to every member of the channel when the message is sent and people would start, someone would find this feature inside of a organization. They would @everyone, everyone would get a notification and then the next person to send a message who was like, " Well, my thing's more important than Bob's thing. I'm going to also @everyone." And it became really obnoxious and people would complain about it, but it was, I don't know, I guess tragedy of the commons. It's not quite exactly the same thing, but it was this real dynamic that happened over and over again.

**Stewart Butterfield** (00:24:16):
So we came up with what was called the shouty rooster, and internally we said, "Don't be a cock." But we didn't obviously say that publicly when you @everyone, a little rooster would pop up and it would have you sound waves coming out of its mouth and being really obnoxious and say, "Hey, this is going to cause a notification for 147 people in eight different time zones. Are you sure you want to send this message with the @everyone?" And of course, that worked amazingly and it dropped off. And again, it was really trying to shape people's behavior so that they used, one is not to be very flexible, but we knew that there was ways to use it that would be annoying and difficult for everyone. And so try to shape the communication culture inside the organization to take best advantage on it.

**Lenny Rachitsky** (00:25:02):
That feature still exists. I see that rooster all the, no, I don't see it all, well, actually I do @channel, because I run a big Slack, so I see that rooster, that survived.

**Stewart Butterfield** (00:25:11):
Yeah. Yeah, that survived and good because it was a trivially easy thing to implement and made a really big difference. But it also taught people how the product worked, because people probably didn't know that @everyone or @channel... Didn't think about the cost, at least.

**Lenny Rachitsky** (00:25:31):
Genius.

**Stewart Butterfield** (00:25:32):
Yeah. Here's one more. So we decided we were going to Do Not Disturb as a feature. And we had this, not conundrum, but you're trying to take into account all the different uses of Slack because by the time we implemented this, 2017, there was tens of thousands of paying customers, the organizations, hundreds of probably millions of users, maybe hundreds of thousands of organizations. I don't remember how many. And everyone had set up stuff the way that they liked it, including things like ops alerts going into channels for on-call engineers for some of the biggest systems and apps in the world. And so we couldn't just deploy it right away. We realized that some of the decision-makers, the owners of the organizations were going to have really strong opinions about this. We also realized that some of the end users are going to have strong opinions, and we wanted to figure out a way to balance the concerns and give people appropriate means of control.

So we came up with this really elaborate system for the rollout, which was, we told everyone, I'm sorry, every Slack administrator that this was coming weeks before it came. And we told them that we were going to set a default for their organization, which I believe was either 7:00 P.M. to 7:00 A.M. in their local time zone, or 8: 00 A.M. to 8:00 P.M., I can't remember which it was, but also that they could override that default, and also that the individual end users could override that system owner default. And finally, that the system owner could, if they changed the default again, would override all of the end user's preferences and then the end users could override them again. And it wasn't to create this dynamic where people were at war, but so that you could change a policy and then people could still customize and stuff like that.

But this was a much longer and more convoluted process, but it allowed the millions of people who were using Slack to get the feature without creating a bunch of conflict and without people turning it off automatically. And I think critically, with setting a bunch of defaults, because if we didn't set the default, most people wouldn't turn it on at all. If we didn't default you to Do Not Disturb from 8:00 P.M. to 8:00 A.M. you probably, if you're the average person, wouldn't ever do it yourself. So that's another elaborate example where I think that investment made sense because it was a critical feature for a lot of people. And if we hadn't done it that way, I think it would've caused a lot of complaints and conflict and stuff like that.

**Lenny Rachitsky** (00:28:22):
Those are amazing examples. I very much appreciate that Do Not Disturb feature when you guys launched that. I still remember that coming out. I'm sure a lot of people are very thankful for that.

**Stewart Butterfield** (00:28:30):
Yeah.

**Lenny Rachitsky** (00:28:31):
Something else I heard that you often espouse, which is counterintuitive to a lot of people is about friction, friction in the product experience. That friction is actually often a good thing. It's a feature, not a bug a lot of times if you use it well. Talk about your experience there.

**Stewart Butterfield** (00:28:46):
Yeah. So yes, and there's also another issue around friction, which is it became like a mantra or just kind of an assumption that you should always be trying to remove friction. And in some cases that's true. We would talk about it in Slack. It was hard to market. It was hard to explain what it was if you had never used it before. You could say a messaging app for businesses or whatever, but a critical disadvantage to Slack doing out-of-home advertising, putting up a billboard versus beer or cars is, no one needs to be explained why they would want a car or beer, but everyone will have to explain one day why they want Slack. And so the problem there is comprehension, and this will come up an enormous amount. So now imagine you want to get tickets to the Taylor Swift concert in San Francisco and you go to the Ticketmaster website.

**Stewart Butterfield** (00:29:43):
If you think about both your comprehension, it's perfect to this case. And that translates into the specificity of your intent and the degree of your intent is also kind of maxed out. So look, I really want to get these tickets. I know exactly what they are. They're Taylor Swift tickets for this date at this venue. And so in that scenario, it doesn't really matter if Ticketmaster's website is slow, it doesn't really matter if the payments page errors out, you're going to persist and get through it. So obviously they're better to reduce friction, but in some sense there's not a huge amount of value in doing that. For most creators of products, there are a handful of cases where that really is true for you as well. And they include things like user registration, authentication, checkout flows for e-commerce. I am significantly more likely to buy something if there's Apple Pay or Shop Pay or something like that.

**Stewart Butterfield** (00:30:44):
I'm significantly less likely to carry through the purchase of something if I have to manually enter all of the fields of my address one at a time rather than having one of those address pickers. It's crazy, but the issue is my intent isn't always 100%, and the specificity of my intent isn't always 100%. So if your thing is direct to consumer T-shirts and you acquire customers through Instagram ads, all of them know what T-shirts are. It's like, "This looks like a good T-shirt to me." But I'm rarely 100% intent. I might have a very specific intent, but my intent's like 70%. So if you're, the amount of friction is above that, I'm not going to do it. But now, okay, people coming to Slack.com, some friend had mentioned Slack and talked their ear off at some point months ago, and then they saw a news article and then they saw someone's tweet and then they saw an ad on about the website they were visiting and they finally said, "Okay, I'm going to go to this website."

**Stewart Butterfield** (00:31:46):
So their intent is at the absolute minimum threshold, it was before that last event happened, they were below and now they're above, but they're just above. The specificity of their intent like, "I need to get Taylor Swift concerts for this date at this venue." Is also very low, because they're like, "It's a work thing. I'm not sure it's a spreadsheet or a calendar or exactly what it is." So they were coming in at 0.1% over these critical thresholds. What was the challenge? It wasn't friction, because it's not like they were aiming for something and they knew what they were aiming for and they were just trying to get themselves to that point.

**Stewart Butterfield** (00:32:34):
What we had to worry about was creating comprehension and in two senses, what is this thing? And what am I supposed to do next? And that creation of comprehension in the sense of explaining stuff, that creation of comprehension in the sense of the design of the UI, of the screen, of the page or whatever, and the visual hierarchy and the affordances that are there and the indication of things to interact with and which thing should be the next thing to do and all that stuff, that becomes really critical.

**Stewart Butterfield** (00:33:09):
And I think very, very few people recognize that. They're like, "I want to get people who come to my webpage to the sign up form as quickly as possible." But if they don't know what they're signing up for and they don't know what it's going to do after, is it going to spam them? They don't know, "Am I going to have to pay on the next step or what?" Then they're just going to back out. And this was a lifelong battle because the remove friction orientation is so deep in people. Again, it really makes a difference in those cases where people do have an intent and they do know what they're trying to do is a poor approach when the challenge is really comprehension, and I think the secret is most, 70%, 80% or whatever of a product design is in that comprehension step because people, if they do ever open the preferences tab and look at all the options, rarely have an idea.

**Stewart Butterfield** (00:34:09):
And if you can't teach them or make it possible for them to discover what the capabilities are, then they're not going to take advantage of them and they're not going to get as much out of it. And I think that the trick is for most of the unique parts of any application, most of the specific things that your app, your product, your software does are areas where the challenge is going to be comprehension inside of friction. It really could be anything Shopify, the purpose of the service for its end users is generally going to be kind to clear. But most people, most first-time store openers don't know that they can get reports or if they know that they can get reports, they don't know what kinds of reports. And if they know what kinds of reports they can get, they don't know how they can tweak them and what the timing should be and which things that are more important to display.

**Stewart Butterfield** (00:35:03):
And I could go on and on and on and on, and people just don't recognize that. So I want to see if this is still true. I'm just going to open my phone and clock app. And they had the craziest description for alarms. It's a little bit different, but people can look at their own phone. So I have, it says alarms and it says sleep and a vertical bar, wake up and says, no alarm, and a button that says change. And then if you hit it, it says sleep is off. In order to automatically turn on sleep features and edit your schedule, you need to turn sleep on. So obviously sleep was a good name for this thing if you already had a way of getting people to understand it. If you don't, it's ungrammatical and incomprehensible and why would you ever do it? And I got to guess, it's been like this for years, 90 plus percent and maybe 98% of people just do what I do, which is that you just create, "I want the alarm on and I'm going to set the time for it."

**Stewart Butterfield** (00:36:11):
And I don't know what turning sleep on does, but it's just the lack of comprehension prevents people from getting the value. And I'm sure that there's a bunch of value behind turning sleep on, whatever that means and people spend a lot of time on those features and it integrates with biometrics and your watch or who knows. Again, I still don't know because turning sleep on is like, what does that do? And what is it going to cost me? And what impact it's going to have? Those examples are just to me all over the place. And the reason I don't use most software where there was an actual choice point or the reason I don't use most features where there was a choice point for me is because I didn't understand what they were going to do and I don't give a shit. And if there is one mantra that I would use to replace that it's, Don't Make Me Think, I don't know if you remember that book.

**Lenny Rachitsky** (00:37:02):
Absolutely.

**Stewart Butterfield** (00:37:04):
Yeah. And honestly, it's been many more than 10 years since I read it, so I don't even remember all of the examples in the book, but as a mantra that was up there with utility curves because for two reasons. One is it's just like it's expensive to make a decision. You literally burn glucose. There's a metabolic action. There's ATP created in the mitochondria and your neurons and a bunch of stuff is happening and people do get decision fatigue and there is cognitive cost of all these things. But also there's an emotional aspect, which is, if your software stops me a second and asks me to make a decision and I don't really understand it, you make me feel stupid. I'm like, "I don't understand this."

**Stewart Butterfield** (00:37:50):
Some people, maybe their orientation is, "Okay, the software is stupid." But I think most people are like, "Oh, I'm dumb." And if you ever talk to people who aren't especially technologically savvy, the canonical example is people who are under 50 talking to their parents about using some piece of software and what they're supposed to do, the parents always feel stupid like they're the ones that are wrong. And so if you're causing people to think, in the best case, it's unnecessary use of their biological resources, and in the worst case you've now made them feel bad, emotionally bad, and they're going to associate that with the product forever. And these are things that are just kind of rolling one into the other.

**Stewart Butterfield** (00:38:35):
So I'm going to keep going with one last thing, because they just kind of come together, which is along with reduced friction, it's like reduce the number of clicks or taps it takes for someone to accomplish something which is almost always exactly the wrong thing. It's the easiest way you could make any action in your app, a single click or tap by just exposing every single possibility on one screen that scrolls for thousands and thousands and thousands and thousands of pages. And obviously that's terrible. So why do people think that a little bit of that is good? And here's an example. You open a menu, there's 14 things that people might want to do.

**Stewart Butterfield** (00:39:22):
Level one is group them into like items and put a vertical, sorry, horizontal divider between them so at least people can kind of chunk and see what there is. Step two is present the two or three most common things or the five most common things, whatever and then have some form of other and then you go to a sub menu that has more items and the decision of how to tune that becomes incredibly important. I'm going to pick on Google again just because it is, I feel like I'm Donald Trump here, but I'm going to interrupt myself again with a story. It's-

**Lenny Rachitsky** (00:39:57):
Yeah, let's do it.

**Stewart Butterfield** (00:39:58):
At some conference or event, I don't remember what it was, and this is probably eight years ago and we're in the bar after the sessions ended at this thing. John Collison from Stripe is there and Sundar, CEO of Google is there. And John, sorry, Patrick goes up to Sundar and they can talk about anything. Stripe wasn't the behemoth, it was now at that point, but it was still a significant company, was up and coming. And what does Patrick want to talk to the Sundar about? It's in the Gmail app, the dragging of people. When you reply all to a message, you often want to change the two recipient to CC and move someone from CC to two or something like that. And just how physically the degree of dexterity that's required to do that inside of the Gmail app is very high.

**Stewart Butterfield** (00:40:56):
It still hasn't been fixed, but it really struck me that Patrick could have asked for anything. It could have been any talk, it could have been a partnership. It was so irritating to him that it worked like this, he couldn't quite get over it. So anyway, back to bashing on Google, who in many respects do an incredible job and there's all kinds of amazing stuff they do on blah, blah, blah, but the Gmail actions on an individual email are broken into two very long menu items that are different. And one of them doesn't exist on either menu. There is an unlabeled icon is the only way to do it, and that's to mark something as unread once it's read. I have no idea why some of the actions are in one menu and some of the actions are in another menu. I think it's because some of them have to do with an individual email and some of them have to do with the whole thread, but it doesn't seem very consistent.

**Stewart Butterfield** (00:41:55):
Every possible thing is listed there in one place. And so it becomes incredibly difficult to use because sometimes you have to tap in both menus, read all of the options, and say, "Okay, I've used the process of elimination and it's not here, so it must be there." Uber doesn't work like this anymore, but when I first brought this up to people inside of Slack, there was a moment when the Uber app, when you opened it was just, "Where would you like to go?" And other. And other was everything like change your payment method, set your location, anything you could do in Uber. And that was perfect because almost all the time people just wanted to choose where they wanted to go. Sometimes you wanted to change where your pickup was because you weren't there yet or whatever. And that was just like, what could be simpler than, "I'm going to tell you where I want to go or I'm going to achieve something else."?

I really tried to push people to what is the thing that people, or what is the two things or what is maybe three things that people could want to do here and then put everything behind other. And then if it takes them eight clicks or taps to do something, but every single one is trivially easy, that's great. If you reduce that to two clicks or taps, but every part of it is this fraught decision where I'm opening all of the menus and trying to figure out which thing is the right thing, and the more, comparing three things to each other is this difficult four things, it's kind of geometrically more expensive to compare 15 different options all to the other to see if this is the one that you might want. That just becomes impossibly expensive. So to me, those are all really connected. And if people could get over the idea of reducing friction as the [inaudible 00:43:42] or reducing the number of clicks or taps to do something and instead focus on how can I make this simple? How do I prevent people from having to think in order to use my software? How can I make this trivially easy? One last example, because this was really influential for me. So I was going back and forth in Vancouver in San Francisco at the time when we were talking about all this inside of Slack, and I was behind a teenager in line aboard the plane and it was like, we're on the jet way. It took a long time. And I was watching her use Snapchat and it was insane.

**Stewart Butterfield** (00:44:11):
She was tapping at least four times a second, sometimes six or seven times a second. It was like dismissing stories and doing stuff. But there was a fluidity to it because everything was like, do I want to see this again? Do I want to see the next story from this person? Do I want to switch to a different person? Instead, a notification came up, she answered someone's thing, she took a selfie of herself and everything was just like... So she was tapping four times a second for six minutes. I mean, probably there was some breaks in there. And that was the highest and best use of Snapchat for a 15 year old girl in 2016 or whenever that was. And imagine if the goal was to try to make her tap less, how much of an impediment it would've been to the experience that both her and Snapchat wanted to create?

**Lenny Rachitsky** (00:45:06):
It's so fun to listen to this and the examples you gave of, it gives us a lot of insight into the way your mind works of just constantly unsatisfied with the way other products work with your product. And I think that's core. Patrick is a good example of Stripe. I feel like that's a recurring theme with very successful product leaders is just constantly unsatisfied and unhappy with how things work.

**Stewart Butterfield** (00:45:27):
Yeah.

**Lenny Rachitsky** (00:45:28):
I love just even the way you summarize this, just a really good reframing of, instead of obsessing with reducing friction and reducing steps, instead think, how do I reduce the amount of thinking the user has to do? I've never heard of it described as, you have to think about the ATP and glucose being used to actually think, and your goal is to reduce that versus let's just reduce friction, reduce clicks.

**Stewart Butterfield** (00:45:52):
Yeah. I think in my more cynical examples, I would say to people, " Stop what you're doing for a second, close-"

**Stewart Butterfield** (00:46:01):
Stop what you're doing for a second. Close your eyes, take a couple of deep breaths, and then pretend that you're an actual human being. And open their eyes again, and then look at this thing and see, can you figure out what it's supposed to do or say. Or what action you're supposed to take or what the impact will be if you take that action. There's a whole nother related cycle. But before I get into it, I know that I am verbose. I want to wrap up your last example of people being unsatisfied.

**Stewart Butterfield** (00:46:31):
So here's the quote that I was trying to find. This is 2014, so like that was the year that Slack actually launched officially in February. And this is now near the end of the year. I was interviewed by MIT Technology Review and asked if we were working to improve Slack. I said, "Oh God, yeah. I try to instill this into the rest of the team, but certainly I feel like what we have right now is just a giant piece of shit. It's just terrible and we should be humiliated that we offer this to the public. Not everyone finds that motivational though."

**Stewart Butterfield** (00:47:06):
So I came into the office the next day and people had printed out on like 40 pieces of 8.5 by 11 paper that quote, and pasted it up on the wall. But to me that was like, you should be embarrassed by it. It should be a perpetual desire to improve. You should probably be like, "Oh, this is great," and you could be proud of individual pieces of work. But in the aggregate, if you can't see almost limitless opportunities to improve, then you shouldn't be designing the product, or you shouldn't be in charge of the company, or you shouldn't almost nothing.

**Stewart Butterfield** (00:47:45):
Again, you could reduce it down to a tiny feature is anywhere close to perfect. And if A, that's acknowledged freely inside the organization. And B, people think about continually improving as the goal. And that could be like Six Sigma Toyota, Kaizen, that kind of side of thing. Or it could be that story that... I can't remember his name right now. The guy who started Bridgewater tells about Michael Jordan-

**Lenny Rachitsky** (00:48:11):
Ray Dalio.

**Stewart Butterfield** (00:48:12):
Yeah, Ray Dalio in his book talks about Michael Jordan learning to ski. Every time he messed up, he wanted the ski instructor to tell him exactly what he was doing wrong. Because to him, every one of those was a gem that he could collect, and he could actually become a good skier. And what he wanted to do was become a good skier. That requires a lot of trust inside the organization.

**Stewart Butterfield** (00:48:36):
But if you can get to the point where like, "Hey, we are trying to find improvements. We're trying to be critical because you're trying to make this as great as it can possibly be." And not always, not with every person, but most of the time with most people, you can get them to the point where that really direct criticism is actually motivational. It is like people are grateful to have the feedback, whether that's coming from their peers inside the company or from end users of the product. Because you realize, oh yeah, that is bad and we should fix it.

**Lenny Rachitsky** (00:49:10):
**Stewart Butterfield** (00:50:13):
Yeah. I mean, so this is a lot to do with, and maybe this is more recently, it shows up in politics a lot for me. But by the way, if anyone listening to this can help me find this tweet store from somewhere between 2016 and 2020, I don't have a precise idea. And it was this guy's thread about how hard it was to get a stop sign set up. And I believe it was in response to someone claiming that Bitcoin is going to replace US dollars, something about crypto. And his point was like, here's what happened when we tried to get a stop sign put up on a residential street in my neighborhood. And the literal years it took, and the number of agencies that were involved.

**Stewart Butterfield** (00:50:58):
Like the engineering department, traffic planners, the HOA, and... I don't remember all of the organizations because, and I did that I could search better and find this again. Because it was truly a masterpiece of how difficult it is to get a stop sign put up in most places. The message that I hear from most politicians, and unfortunately this works really well, is things should be good. But they're not because someone is doing something bad, which is preventing the goodness.

**Stewart Butterfield** (00:51:29):
So billionaires are making things unaffordable. Or immigrants are taking your jobs. Or lazy freeloaders are sucking off a government tea, and causing us all have to pay more taxes, or something like that. The reality is almost nothing works. It's actually another call. I said in this case, John has a great encapsulation of this and I'm sure you're familiar with it, like that. It ends with the world as a museum of passion projects. Because for anything to get done at all requires not just the resources and effort required to instantiate that thing in the real world, but all of the politicking and the sociology and the convincing.

**Stewart Butterfield** (00:52:15):
And there's a book called Why Nothing Works Recently, which is like, it's not an... I'm sorry to the author, if they... I doubt they're listening, but just it's not like an amazingly written book. I found it a little bit repetitive, but the content was really incredible, just explaining why it's so hard. And how there's this progressive increase in the number of vetoes that are available for any kind of course of action and how difficult it is... And this shows up in permitting for new construction and stuff like that. But also shows up obviously inside of organizations.

**Stewart Butterfield** (00:52:55):
And the challenge is that people, A, I think this is evolutionary biological. It's hard for us to understand the world, except by anthropomorphizing it. And so if it didn't rain this year, it's because a God is mad, and probably because we didn't sacrifice enough goats or something last year. It's hard for people to understand just that, wow, weather is incredibly complex and chaotic, and ecosystems and climatology, and all that.

**Stewart Butterfield** (00:53:27):
Same thing with the world. Like if I am struggling to pay all of my bills and be able to afford a little bit of luxury in the sense of location or a present for my kids or whatever, it's got to be somebody's fault. There has to be a decision that's made somewhere. And the reality is everything is so complicated. Everything is so multivariate, it's not satisfying. It's a terrible political message.

**Stewart Butterfield** (00:53:56):
It's much easier to say that there is like, oh, we understand why things are bad in the way that you're concerned about. And it's turns out that it's some someone's decision, and because of them it's bad. And so if we got rid of them or were able to overcome their decision, overturn it, and institute our own thing, then things would be good for you. And this really to me shows up inside of those organizations as well. I'll pause there.

**Lenny Rachitsky** (00:54:25):
I know kind of along those lines, you're a big believer in something called Parkinson's Law.

**Stewart Butterfield** (00:54:31):
Yeah. So the original of that is, I think it's 1956. It's an article in The Economist by Parkinson. And the Maxim is work expands to fill the time available for its completion. And the way that it shows up, this is a little bit subtle. So like one of the things I found, since I don't have a job is there's much less time pressure. And that maxim, like if you want something done, give it to a busy person. The inverse is also true that like, if you're not that busy, wow, basic things take a really long time.

**Stewart Butterfield** (00:55:09):
And so Parkinson actually starts out with his example of writing and posting a letter. And I don't remember who he used with the first example, but someone who's incredibly busy and has all these things they have to respond to. And then another case like a retired robot who has all the time in the world. It takes her a long time to write the letter. It takes her a long time to put it in the envelope, and then you go to the post office and post it.

**Stewart Butterfield** (00:55:28):
But the real meat of it is, for me later when he talks about the size of the organization, and he uses a bunch of examples. This is again 1950s, and he's British, so he's looking at the Royal Navy. And specifically he's looking at a chart that shows the relationship between the number of capital ships in the Navy, the number of sailors, and the number of administrators. And very familiar graph for people looking at any part of government. Any part of the relationship between the number of administrators at a university and the number of students and faculty, teaching faculty. Where it's like, okay, the number of ships goes like this and the number of sailors is looking right along with it. And the number of administrators goes like this.

**Stewart Butterfield** (00:56:14):
And the reason this ties into the work expands to fill the time available for its completion is people hire, and they train. And here's the sad truth for anyone running a company is there are exceptions. There's certain types of engineers that are an exception to this. But the overwhelming majority of people you hire want to hire more people who report to them. And it's not because they're evil, and it's not because they're stupid. In fact, they're smart because everyone knows that the number of people who report to you correlates with your career trajectory, the amount of money that you're paid. The amount of authority you have inside the organization and on and on and on.

**Stewart Butterfield** (00:56:57):
So we would hire 27 Royal product managers in Slack who immediately want to hire someone. It's like, what the hell? What would that person do? And they articulate it this way, but essentially it's like, "Well, that person would do the product management and then I would do strategy."

**Lenny Rachitsky** (00:57:11):
Classic.

**Stewart Butterfield** (00:57:12):
It's really, I think the essential thing to understand about this is it's not because people are evil, and it's not because they're stupid. And it's to me, very related to everything is complex. And if maybe this is my butterfly's law, I haven't thought about this way before. But I tweeted this a very, very long time ago like if you... Everything is simple if you have no idea what you're talking about. So the other side of that is like if something seems simple, probably you don't understand it. And there's obvious exceptions to that.

**Stewart Butterfield** (00:57:53):
But for anything that involves a large organization or a lot of human beings, if the problem seems simple, you don't get it. So every budget process, no head of engineering know, head of sales, no CFO, no GC, who's ever going to come back and say, "Oh, I've actually think next year we can just hire fewer people. Or we're going to keep it flat or we're going to shrink through attrition because we don't need any more people to do what we're doing." Not because they're evil, not because they're stupid, but it's almost overpowering impulse inside the organization that often leads to disastrous results. And so there's an...

**Stewart Butterfield** (00:58:30):
I'll give one example from Slack's history, and I have tried in the past to disguise this example so that no one feels bad about it but I... Unfortunately, the specifics are so important to the example that it's not disguised and so I'll just reiterate that the people involved aren't stupid or evil. And one example that's from the outside. So the example inside of Slack was we introduced threads, which was the ability to reply to a message inside of a channel. And let's say you, Lenny, post a message. I, Stewart reply to it. You will automatically get a notification. And now Sarah later on replies to the same message. Both you and I, as people who have push in that thread will receive a notification that there's been more activity, and so on. So like every single time anyone replies to it.

**Stewart Butterfield** (00:59:23):
So when the feature first was released or like when we did the final product review before it was released, the input box was pre-populated with at the person before you in the thread. And I was using the feature and I would put the insertion point there, select all delete, and then start writing my message. And even if I wanted to add someone specifically, I almost never wanted to start my sentence with at that, because it just made it hard to reference what they were saying before. So I said, "Get rid of this because, A, I think most people won't use it. Or if they did want to add someone, they're not going to want to do it at the beginning of the sentence.

**Stewart Butterfield** (01:00:07):
And by the way, you're teaching them to use the product wrong. Because it's important that everyone understand that every previous poster in this thread will automatically receive a notification unless they've figured it."

**Stewart Butterfield** (01:00:20):
So okay, we release it. Six months goes by and suddenly the at thing comes back. And so I messaged someone around the team and I said, "Hey, there's been a regression. This is super weird. I don't know what happened. But the at thing came back." And they said, "Oh no, this is on purpose. We did a bunch of research." And so I was like, "What?" And I went through this and it was, if I recall correctly, it wasn't even P-95 certainty on this analysis. But it was something like when we do this, threads are 2.17 messages long, versus 2.14 messages long on average for when we don't do it.

**Stewart Butterfield** (01:00:59):
And so first of all, why is a longer thread better? Like maybe a shorter thread is better? It can be fewer messages that people have to go back and forth. Also, that's such a tiny difference. Also, again, I don't remember the actual statistical analysis, so I'm not going to claim that it was incorrect. But I'm pretty sure this was outside the bounds of certainty that they can have. But the real thing was, oh my God, so you guys put flags into the product, you A-B tested it. You did the instrumentation. You created tables in the database or whatever we're using to record all of that.

**Stewart Butterfield** (01:01:39):
You wrote queries to pull that. You created charts based on that data. You had meetings to discuss it. And just kind unpacking all of the things that would've had to happen for this to come back. And it's like thousands of person hours at a minimum, because any feature change at that scale of organization, it's involving like a dozen people. Engineering, QA, analytics teams, project managers, user research and stuff like that. The problem with that, so I think it was a bad idea, right? But the problem with that was the difference that you could possibly achieve between having this feature and not having this feature is like this much whatever units you want. The cost of doing the analysis was this much. So it's guaranteed to be a loser.

**Stewart Butterfield** (01:02:29):
Like there's just, there's no world in which anyone could imagine putting the at previous respondent in the thread at the beginning of the message could possibly make that much of a difference to the quality of Slack, and how much utility it provides for people and all of that. But you know that to put the feature flags in, to ship new versions of the product, to put the instrumentation in. To have it all the API calls to record every action that people take to do all the analytics, to create the dashboard. To put paste a screenshot of that into a Google Slides presentation. To send the invitations to the meeting, to reschedule the meeting because someone couldn't make it. To have everyone sit down and look at the thing. Like guaranteed loser.

**Stewart Butterfield** (01:03:14):
And I know that Fareed told you to ask me about this hyper realistic work-like activities. And so here's my grand theory. Hyper realistic work-like activities goes along with this other concept called known valuable work to do. And when I say known, I mean both you know what it is and you know that it's valuable. And the problem with almost every organization at the very beginning, you have an enormous amount of work that you know what to do, and you know that it's going to be valuable. So like starting a business, open a bank account. Because there's almost infinite general value of opening a bank account. You have to do it. It's very simple to do.

**Stewart Butterfield** (01:04:01):
And so at the very beginning of any startup, they're like, "I'm creating a user's table, and I'm doing sorting passwords," and you're doing all the things that are kind of absolutely necessary. And everyone knows exactly what they are. And so everyone's going to work in the morning and they're like right on. And I have 10 things to do, and every single one of them is something I know how to do. And it's definitely going to be valuable. Time goes on. And the relationship between the supply of work to do and the demand for doing work just starts to change.

**Stewart Butterfield** (01:04:32):
More and more people get hired. Every product manager wants to hire a junior product manager. Every new person, the first person you bring in on the risk and compliance team is like, "Oh my God, there's so many risks and things we have to be compliant with. We better hire more people on my team to do more risk and compliance work." Which probably to some degree is right. But we're going to have more and more of those people and they're going to call meetings with each other.

**Stewart Butterfield** (01:04:54):
And now suddenly you have all these people with work to do and you've done all the easy obvious stuff. And now your questions are like, "God, should we do FedRAMP high and make a Slack version? Which is going to require us to have wholly separate physical infrastructure for the hardware that runs the software? And also a whole different operations team, which has only US citizens on it? What is the possible number of dollars that we could make from doing this? And how much complexity is going to be when we want to do updates to the software because we update two totally separate independent systems and rec."

**Stewart Butterfield** (01:05:27):
It just gets out of whack, and so people end up... Like if you hire 17 product marketers, you're going to have 17 product marketers worth of demand for work to do. And if you don't have sufficient supply of product marketing work to do, they're just going to do other stuff. Again, very important, not because they're stupid, not because they're evil. But because they're like, I'm a product marketer and I want to be recognized for my work. And my spouse has criticized me because they take, I should have already got promoted in the last cycle, and they really got to demonstrate some wins here and whatever it is.

**Stewart Butterfield** (01:06:02):
And so people are like calling meetings with their colleagues to preview the deck that they're going to show in the big meeting to get feedback on whether they should improve some of the slides. And that hyper-realistic work-like activity is superficially identical to work. Like we are sitting in a conference room and there's something being projected up there, and we're all talking about it. And that's exactly what work is. Hopefully not all of work for everyone inside of your company. But that's exactly what we do when we're working.

**Stewart Butterfield** (01:06:34):
But this is actually a fake bit of work, and it's so subtle that I'll do it. Our board members will do it. Every executive will do it. And the further you are from having all of the context and all of the information and the decision-making authority and stuff like that, the easier it is to get trapped in this stuff. And people will just perform enormous amounts of hyper-realistic work-like activities, and have no idea that that's what they're doing. And the result of that, I guess, is that if you are a leader, if you're manager, director, an executive, you're the CEO, it's on you to ensure that there is sufficient supply of known valuable work to do. And there almost always is, but it's creating the clarity around that. Creating the alignment. Making sure everyone understands it, but that's what they're supposed to be doing, and then obviously doing it.

**Lenny Rachitsky** (01:07:26):
Amazing. I could listen to Stewart's rants all day. Hyper- realistic work-life activities. We need to coin this-

**Stewart Butterfield** (01:07:34):
Unfortunately, it doesn't make a good acronym. It's pretty ugly.

**Lenny Rachitsky** (01:07:34):
Okay.

**Stewart Butterfield** (01:07:37):
[inaudible 01:07:37].

**Lenny Rachitsky** (01:07:36):
Okay. [inaudible 01:07:37] it a try. And just to close the loop on that, the solution is the leader recognizing this is happening and stopping it. Telling people why are we spending time on this thing that is not going to get us anywhere?

**Stewart Butterfield** (01:07:48):
Yeah. And what you just said probably isn't the best way because that sounds like you're chiding them, and they're dumb. When it's actually your responsibility to make sure that there's sufficient clarity around what the priorities are, and explicitly saying no to things upfront and stuff like that. Rather than merging and say like, "Hey, you guys are a bunch of idiots wasting your time on this thing that doesn't matter." Whose fault is it? It's the manager's fault. It's the VP of whatever's fault. It's the CX, whatever, it's the C... Ultimately, it's the leader of the organization that has the responsibility to make sure that there is sufficient known-valuable work to do. And that's actually harder than it might appear.

**Lenny Rachitsky** (01:08:32):
Okay. Before we run out of time, I want to touch on two other topics. One is, when people think of Stewart Butterfield, I think a lot of people think of, We Don't Sell Saddles Here. Your legendary Medium post that is just, I don't know, it's become a historic piece of literature in the annals of product building and in startups. I haven't heard people ask you much about this recently. So let me just ask a couple of questions. What was the reason you put that out? What was the backstory on writing that memo? Why was it necessary?

**Stewart Butterfield** (01:09:01):
Well, it really was an internal memo.

**Lenny Rachitsky** (01:09:00):
... Memo. Why was it necessary?

**Stewart Butterfield** (01:09:01):
Well, it really was an internal memo and there's a bit of a digression. One of the crappy things about Slack is if all your corporate communication is on email, depending on exactly how it works and what system you use, you probably walk away with an archive of everything you said at Company X. If it's Slack, once you're turned off, you lose access to all that history. And so it's kind of like, "Oh, man. If I had only exported all of my messages before I left, I would have all this stuff," but that was absolutely verbatim. I did not change a word of what I said inside the company. Well, I think we were still eight people. Maybe at most 10, but I think it was eight people.

**Lenny Rachitsky** (01:09:45):
It was before Slack launched even.

**Stewart Butterfield** (01:09:47):
Yeah, it was before Slack launched. It was when we're doing private beta. And the point of it was to start to instill those ideas as early as possible and really create this alignment inside of that small team so that it could persist to survive as we grew and scaled. Yeah, that was the idea.

**Lenny Rachitsky** (01:10:11):
And the gist, just for people that aren't super familiar with it, but we'll link to it, is just it's not enough just to build a great product. You just as much have to put effort into communicating what this does for them, the problem this is solving for them, the outcome this is going to achieve for them. Is that a good way to think about it?

**Stewart Butterfield** (01:10:28):
Yeah. And again, comparing it to beer or cars, beer goes back to pre-civilization. Cars were obviously [inaudible 01:10:38], but at some point you had to convince people why they would want a car instead of a horse. For your new AI-based recruiting tool or your calendar app or whatever, there's some reason why you think that people should use yours instead of the thing that they're using now, which might be a wholesale one-for- one replacement, or more often is a change in the way that you're working that has a bunch of other adjacencies and you want to expand into these other categories. You're not just responsible for creating the product, but also, to a certain degree, creating the market.

**Stewart Butterfield** (01:11:15):
There's this book, Positioning, which is an absolute classic. It's very short. I would recommend everyone read it, where the point of it is, from my perspective, it's almost impossible to create a new idea in someone's head. It's much easier to take a couple of existing ideas and put them together. So it's much easier to say it's like Jaws meets Star Wars, or it's Uber for Pets or something like that, than to come up with an actual new idea. But you have to do that because if your thing is different in any significant way from the alternatives, you're not just creating the product. You're creating the market. They're really kind of one and the same.

**Lenny Rachitsky** (01:11:56):
The reason I wanted to touch on it is I think still people continue to not listen to this advice and continue to over-invest in more features, more products, things like that. Just the specific example of, "We don't sell saddles here," just to quickly communicate this to folks, and correct me if I'm missing anything, is just instead of, "Hey, look at this amazing saddle we've bought," which you want to communicate as, "Here, go horseback riding. Look at this incredible experience you can have." And then they decide, "Oh, shit. I need to go buy a saddle to do that."

**Stewart Butterfield** (01:12:23):
Yeah. And 100%, that aspect of it is not original because I think that's something that marketers have done for a long time, certainly in the marcom and advertising. If you want to sell Harley-Davidson's, there are people who are going to geek out on the engines and stuff like that and the quality of the leather and stuff like that. But when you're selling the motorcycle, you're selling the open road and freedom and the wind in your hair. And if you're Lululemon, you are obviously selling yoga pants, but you're also selling health and aspiration and being the best version of yourself and a bunch of other stuff. Oh my God, I forgot the classic version of it.

**Lenny Rachitsky** (01:13:00):
There's the ship ...

**Stewart Butterfield** (01:13:00):
You're selling the screwdriver.

**Lenny Rachitsky** (01:13:04):
Oh, yeah. The nail.

**Stewart Butterfield** (01:13:05):
Yeah, the nail. Anyway.

**Lenny Rachitsky** (01:13:07):
Yeah, we missed that one. Well, there's the one I think about is instead of trying to convince men to build a ship, instill a yearning for the sea.

**Stewart Butterfield** (01:13:16):
Yes. Exactly. That's something that goes back in history.

**Lenny Rachitsky** (01:13:21):
Okay. Let me ask you about pivoting. You are potentially the king of pivots. You started two companies both famously pivoted, both from video games, which is why I asked you about that at the beginning, into very successful companies. I imagine many people come to you for advice on pivoting. Let me just ask when folks come to you asking, "Should I stick with my idea? Should I pivot?" what sort of advice do you find most helps them?

**Stewart Butterfield** (01:13:43):
Yeah, I mean, I think it's partly an intuition because obviously the decision is about, "Have you exhausted the possibilities?" and in the case where we were working on Glitch, this game where we used IRC for internal communication and we added a bunch of IRC which became the Proto Slack. I think Slack had an enormous advantage in the fact that we are working on this for several years without actually explicitly working on it and only doing the minimum number of features that were absolutely guaranteed to be successful in the sense that it was so irritating that we couldn't stand it anymore or such an obvious improvement that we couldn't help but take advantage of it. We still had $ 9 million left and everyone still liked the game and we were all happy working on it, but I think by that point I had exhausted every non-verdiculous long shot idea to make it commercially successful, and so I decided to abandon it.

**Stewart Butterfield** (01:14:52):
But the default advice for anyone in anything is persevere. It's like a kitten hanging off the branch and a poster says, "Hang in there." There's so many stories of, "So-and-so started out going door-to-door and was rejected by everyone and then suddenly there was Nike," or something like that and just, "If you stick with it long enough, you'll eventually be successful." I think you have to really be coldly rational. Some of this shows up in the book Thinking in Bets. Some of it's in Annie Duke's second book, the title of which I'm forgetting right now, but someone will know it.

**Lenny Rachitsky** (01:15:35):
Yeah, Thinking in Bets, and then what was the second? I forget.

**Stewart Butterfield** (01:15:39):
She actually uses Glitch and Slack as an example of a smart fold basically. My expected value here has diminished to the point where this alternative looks more attractive. And the reason I say you have to be coldly rational about it is because it's fucking humiliating. I convinced so many and you have to convince so many people to get a company off the ground. You have to go to investors. You have to go to early employees and say, "You should leave your other job and come work for this because here's the incredible feature we're imagining." You have to go to the press and you have to make all these promises and you have users and you've committed things to the users and you've convinced them to give up their time for this thing. And so I think for a lot of people, it feels better to just keep doing it until it dies of suffocation due to lack of capital or something like that. Then just to admit, "Okay, I was wrong. This didn't work," and it's humiliating. It's painful. It's wrenching. It has a bad impact.

**Stewart Butterfield** (01:16:46):
When we shut down Glitch, there was a lot of people who loved it and would spend all of their free time and couldn't wait to get home from work to go play it more. And that was their community and the community just disappeared, all these people and all these identities that have been created. And obviously, people lost their jobs and people who had moved their families to a different city in order to take this job now weren't going to have a job anymore. So pivots aren't something I take lightly. I think it's very different to be like, "There's three of us and we started making this app and then we pivoted to a different app." That doesn't even really count. If you're six months into something, you're still messing around. You're trying to figure out what it is that you're building. It's not really a pivot. Obviously in this case, it worked out great and there's survivorship bias and that doesn't mean that everyone should pivot all the time. But I think creating the distance so that you can make an intellectual, rational decision about it rather than an emotional decision is essential.

**Lenny Rachitsky** (01:17:50):
I love, also, your piece of advice of just exhaust. Once you've exhausted all the ideas, that's a really good time to see what else is out there.

**Stewart Butterfield** (01:17:56):
Yeah, just all the good ideas.

**Lenny Rachitsky** (01:17:59):
All the good ideas,

**Stewart Butterfield** (01:18:00):
All the realistic. Yeah.

**Lenny Rachitsky** (01:18:03):
Yeah. The point you made about just kind of persevering, I just had Melanie Perkins, CEO of Canva, in the podcast. 100 investors rejected her before somebody finally decided to invest and she just kept pushing.

**Stewart Butterfield** (01:18:19):
Yeah. I think that's a slightly different example, right? She eventually believed in the concept of the product and in the vision. It was just trying to figure out the right articulation to get investors who ended up being obviously very, very happy.

**Lenny Rachitsky** (01:18:31):
Extremely happy. Oh, geez. Okay. Maybe a final topic depending on how time goes. I want to talk about generosity. I talked to a bunch of people, as I said, that have worked with you and the number one theme that came up again and again and again when I asked them about you and what has stuck most with them is just generosity. So I'm going to read a few examples that I heard from folks that are examples of your generosity over the years.

**Lenny Rachitsky** (01:18:57):
So one person shared that he needed a little money before Christmas and he said, "Stewart literally walked me out of the building, went to the cash machine, handed me $500, told me to go home to my family." Other folks shared that, when you talked about Glitch just recently when you had to lay people off, you cried real tears when you were laying people off and then you spent an incredible amount of time helping them find new jobs and extending their severance pay and just taking it extremely, extremely seriously, much more than I think most people feel like CEOs do. Someone else shared that you paid 100% of employees health insurance to give them just fewer things to think about.

When you went public, you basically created the best possible situation for employees, no lockup, direct listing. Also, with the structure of the Slack deal, people said that acquisition was very employee friendly. That's employees. There's also just the way you thought about customers. A few examples: You gave free credits to businesses who were struggling to pay the bills during COVID. You released this fair billing, which I think was very innovative at the time, where you stopped charging people for seats they weren't using, even though they signed a deal to charge for those seats. A lot of times, you slipped release schedules because you just wanted to make features better and better for people. And I'll end with this quote: "Stewart is a leader who takes the responsibility he feels for his employees personally, and to which he extends the most generous circumstances he could muster. That feels worth celebrating."

So first of all, I just want to celebrate you. I think it's really rare and inspiring to meet a leader like that. Clearly, you've had a lot of impact on a lot of people. I don't know exactly the question I want to ask, but I guess in what part is this intentional, just like, "This is how we win. I'm going to be very generous and help people because I know this will help long-term"? How much of this is just a [inaudible 01:20:48] and it's just the way you are as a person?

**Stewart Butterfield** (01:20:49):
I think a lot of it is just the way I am as a person and I had wonderful parents who raised me right, but I think there is a little bit of a lesson there and I'm just going to assume people's familiarity with the prisoner's dilemma. The acts of generosity to me are, "Oh, I am demonstrating that I am going to cooperate as we iterate in this game." And if you do that, then people will also cooperate and you both benefit. Whereas if you never really know if the other person is going to defect at the first opportunity, then your best bet is to defect. And so there's a game theoretic aspect, usually in games that are much, much, much more complicated than the prisoner's dilemma.

**Stewart Butterfield** (01:21:36):
I think one thing I didn't touch on before, but to me was important enough, is that at more than one company all hands, I made everyone in the company repeat this as a chant. It was, "In the long run, the measure of our success will be the amount of value that we create for customers." And I wanted to be super clear and explicit about that because it should be if anything you're doing feels like a little bit shady, a little bit cheating, a little bit maximizing at the wrong moment or taking advantage of a customer or anything like that, definitely shouldn't do it. Because to me, I mean I think it's literally true, but it's also an ethical way to run a business. And it's not just that the ethics are good. It's like there's advantages for you. You're able to attract a better class of employees. If all your employees are ethical, then it's going to be a better place for everyone to work and you're going to be happier and you're going to have fewer internal problems and all that stuff.

**Stewart Butterfield** (01:22:48):
But I think it really is true that especially in the long run, you can't destroy value for your customers and expect to be successful. You have to actually make their lives better. And you could put effort into pointing it out to them and demonstrating that you have created this value and stuff like that, but there's no substitute for actually having created it. And I think that is incredibly important and that implies a real generosity, whether that's in negotiating terms with an enterprise deal or that's policy decisions. One time that it blew up in our face was our SLA was like, "For any downtime, you get 100 times your money back." Because from my perspective, it's like if we're down for two minutes, it's like pennies. It doesn't really make any difference. If we're down for 10 hours or something like that, then we have bigger problems than paying back people.

**Stewart Butterfield** (01:23:51):
Fast-forward, we now have hundreds of millions of dollars in revenue and we've gone public. And shortly after we go public, we have one of the biggest outages we ever had. I don't remember how long it was, but it was many hours. But by the time we got that scale, 100 times the money back for the third of a day that we were down was $8 million or something like that. It didn't cost us any money because we just gave it to people in the form of credits, but it meant that a bunch of revenue that we had already anticipated for the next quarter wasn't going to show up because people's credits were going to offset what they would've otherwise paid us. And so we definitely changed the terms of service after that because being a public company is a little bit different. But in every other respect, I think they were all really important decisions that were helpful in us becoming successful.

**Lenny Rachitsky** (01:24:47):
Was that policy ... It was automatic? You didn't even have to claim it. It was just automatically you get this credit?

**Stewart Butterfield** (01:24:52):
And the default is you don't have to pay if you let us know. This was, "We will automatically, proactively, preemptively without any input from you ..."

**Lenny Rachitsky** (01:25:00):
Too generous.

**Stewart Butterfield** (01:25:01):
"Apply this credit to your account, and just send you a message that it happened. And by the way, we will do it on the aggregate for downtime, even if the issue didn't affect you as a customer."

**Lenny Rachitsky** (01:25:13):
Wow. Too generous. You found the edge of where you want to be.

**Stewart Butterfield** (01:25:13):
Yeah.

**Lenny Rachitsky** (01:25:18):
What was that mantra again that you had the company chant? I think this is a really nice way to end it.

**Stewart Butterfield** (01:25:22):
It was, "In the long run, the measure of our success will be the amount of value we create for customers."

**Lenny Rachitsky** (01:25:28):
Incredible. I'm just trying to picture the entire team at Slack reciting this mantra.

**Stewart Butterfield** (01:25:33):
It was hundreds of people. It felt very like, Kim Jong-Un or Stalin or something like that.

**Lenny Rachitsky** (01:25:38):
Well, on that note, most people don't know this about you, but your actual name when you were born was not Stewart. It was Dharma.

**Stewart Butterfield** (01:25:46):
Yeah.

**Lenny Rachitsky** (01:25:46):
And this all makes sense as you learn that.

**Stewart Butterfield** (01:25:49):
Yeah. My name is Dharma Jeremy Butterfield, so my parents named me. And when I was 12, I changed it because I just really wanted to be normal and for some reason I thought Stewart was a normal name. And by the way, you'll notice this now that I said it. Any character except for Stuart Little the mouse, anytime you see a character in a movie, a novel, TV show or whatever, there's only the loser Stewart and the asshole Stewart. It's obviously, in the collective consciousness, a terrible name and I shouldn't have chosen it and I regret it. But by the time I realized that, Dharma and Greg had already come out and it would've seemed like I was bandwagon jumping. And people thought it was a girl's name, even though in India it's obviously only a boy's name.

**Stewart Butterfield** (01:26:32):
I'm going to add just one last little tidbit because I forgot about this earlier on and I think it helps tie things together, and it's called the owner's delusion. And this is based on something I posted on Twitter. The person who came up with the name later deleted their account and so I have no idea who it was and who to credit for this. But what I had posted was, and this was a long time ago when restaurant websites have gotten better and it doesn't really matter because Google Local was taking over everything, but this is, let's say, 10 years ago.

There's five things you could possibly want when you go to a restaurant's website and it's their street address, their phone number, the menu, the hours of operation ... Oh my God, I'm forgetting the fifth thing. Oh, and making a reservation, how to make a reservation. And again, this problem has to some extent taken care of it's itself or at least improved, but what you would get was this super slow loading photo, the Ken Burns effect as it [inaudible 01:27:30] ...

**Lenny Rachitsky** (01:27:29):
The flashed.

**Stewart Butterfield** (01:27:30):
And then fading in and then some music starts playing. And then if they show you the phone number, it's not clickable.

**Lenny Rachitsky** (01:27:38):
Image.

**Stewart Butterfield** (01:27:39):
It's not even text that you can copy because yes, it's an image. And they don't have the hours. They don't put the address or whatever and it's just like, "What?" For sure, whoever made this website for the restaurant owner and the restaurant owner themselves have definitely been in the position where they went to somebody else's restaurant website because they wanted to get the address or the opening hours or the phone number or whatever. So why does it end up like this and what should we call this?

**Stewart Butterfield** (01:28:01):
And whoever replied to the tweet, she said, "We should call it the owner's delusion," and I was like, "Oh my God. That's perfect." And I think that is incredibly powerful and what ends up with the result, like Apple naming whatever that feature is called Sleep, which it's too hard to understand what that can possibly mean. And that's why people anticipate, despite the fact that when they get to your website for the first time, their intent is absolutely the minimum number of micro points above the threshold required from them to actually take that action.

**Stewart Butterfield** (01:28:41):
You're like, "All right. Welcome to my website," and there's a bunch of BS and there's a bunch of stuff that doesn't make any sense and the buttons are inscrutable. And it's unclear what to do next because I think that my thing is so important and I don't recognize that you are at work and you were late this morning and you have to go to the bathroom and you're just a regular human being who has stuff going on, that you're concerned that your kid is a fuck-up and they're getting in trouble at school and stuff like that. They're not subjects who paid money to go to your play and are sitting in the audience and waiting for that curtain to go out. They're people who are going to bounce in a fraction of a second. And so everyone should always be conscious of the owner's solution.

**Lenny Rachitsky** (01:29:27):
I love that. What's the solution? Is it have other people look at it and give you feedback?

**Stewart Butterfield** (01:29:31):
Yeah, and recognize it. And unfortunately, it's one of those things like Murphy's Law.

**Lenny Rachitsky** (01:29:35):
Yeah.

**Stewart Butterfield** (01:29:37):
Even you can go wrong even when you take into account Murphy's Law.

**Lenny Rachitsky** (01:29:39):
That's right.

**Stewart Butterfield** (01:29:41):
But if you don't name it and recognize it and discuss it and train yourself to think that way, take a breath, pretend you're a regular person, and then look at this again and see if it makes sense, then you're screwed.

**Lenny Rachitsky** (01:29:55):
I love that. I love that you threw this in here. I have a billion other questions I'm going to ask you in part two when we do this someday. Stewart, thank you so much for doing this. Thank you so much for being here.

**Stewart Butterfield** (01:30:04):
Yeah. Thank you for having me, Lenny. I really enjoyed it.

**Lenny Rachitsky** (01:30:07):
Same here. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [12/17] A guide to difficult conversations, building high-trust teams, and designing a life you love | Rachel Lockett

**Lenny Rachitsky** (00:00:00):
When clients come to you, what is the biggest gap they have that is keeping them from being successful as leaders?

**Rachel Lockett** (00:00:06):
Most leaders, especially technical leaders, assume they have to have all the answers. People have climbed the ladder because they've been dependable, reliable, the smartest person in the room. But great leaders know that when you try to advise and have the answer all the time, you're not actually equipping your team to go solve the hard problems. You're training your team to come to you with all of the hard problems.

**Lenny Rachitsky** (00:00:27):
Difficult conversations are difficult. How do we help people make them less difficult?

**Rachel Lockett** (00:00:32):
We operate in tech. We're supposed to give all of ourselves, all of our time, all of our energy to this endeavor, and it's purely logical. That's not at all true. It's completely emotional. Professionals have feelings. People, when they want to have a conflict, they come in ready to prove their point. There's a misguided view that the goal is to convince the other person that what they're doing is wrong. Actually, the goal of any conflict is to create mutual understanding.

**Lenny Rachitsky** (00:00:57):
Talk about what you've learned about helping leaders in tech avoid burnout.

**Rachel Lockett** (00:01:01):
When people are in their gifts and their strengths, they have more energy. We all have more energy when we're operating from the things we naturally are good at. It's no one else's job to help you live in your gifts. What I notice in big companies is people are often annoyed or frustrated with their management for not making their job more interesting. No, your manager's job is to help you perform in the job you are hired to do. It's your job to navigate your career.

**Lenny Rachitsky** (00:01:26):
The power of this is this makes your life so much better.

**Rachel Lockett** (00:01:28):
Lenny, let's try it. So, I want you to tell me a challenge, something that you're struggling with.

**Lenny Rachitsky** (00:01:34):
The main thing I struggle with these days is just... Today, my guest is Rachel Lockett, an executive coach and former longtime HR leader at Pinterest and at Stripe. She works with CEOs, and founders, and leaders at tech companies on both ways that they are, emotional and positive intelligence, resilience and courage, and what they do, setting vision and strategy, prioritizing, and building trusted and accountable teams. She's someone I've heard a lot about over the years from other podcast guests, and this conversation is powerful. It's jam-packed with advice, and tips, and frameworks that'll make you a better leader and also a better person. We even do a couple live coaching sessions to demonstrate some of Rachel's approaches. And as you'll see, I had a number of epiphanies during this conversation. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It helps tremendously.

**Rachel Lockett** (00:05:03):
Thank you so much for having me, Lenny. I am honored to be here.

**Lenny Rachitsky** (00:05:06):
I'm honored to have you here. I was going to start with a different question, but we were chatting ahead of this conversation and I always like to ask guests, what do you want people to get out of this conversation? And I loved your answer, so I just want you to share this. So, let me just ask you, what are you hoping people get out of the conversation we're about to have?

**Rachel Lockett** (00:05:23):
Genuinely, I hope that your listeners take away that the human side of business building is incredibly fun and impactful and that it's easy to do. They can do it with simple tools. So, I'm hopeful that through this conversation, heads of product, heads of engineering, founders walk away feeling more empowered and more motivated to attune to the people around them.

**Lenny Rachitsky** (00:05:47):
So, what I'm hearing is just if you're struggling with the human side of building a product, building a team, building a company, there are answers. You can do it.

**Rachel Lockett** (00:05:56):
Yes, exactly. It is achievable, and it's actually most natural. Leaders want to care about the people they work with. They want to empower those around them. But sometimes the busyness of our world gets in the way and the urgency of the litany of things to do distracts you from the people in front of you. And actually, if you really understand the talent around you and you create an environment where they can be successful, your business will thrive.

**Lenny Rachitsky** (00:06:22):
I think the hardest part of this for people is just there's the knowing this can be helped with. The other is just being vulnerable enough to seek help and to take this on because it's so hard. Just like, "Oh, maybe I'm not a great manager." That doesn't feel good.

**Rachel Lockett** (00:06:37):
Yeah, that's true. I mean, it's vulnerable to seek help, but I think your audience, I know to be incredibly committed to growth. I hear of people who come on your podcast and they've spent decades focused on self-improvement. And I actually want to tell you a story about one of my clients who loves your podcast, and I was talking to him last week. He's a client I've seen for 10 years, and he's a person who exemplifies a commitment to personal growth.

**Rachel Lockett** (00:07:08):
I started working with him when he was a frontline engineering manager at Coinbase, and we talked about who he is, what his strengths are, and his bigger picture why. And he talked about this dream of creating a global movement one day. He was really focused on building community, and he thought the path for creating possibility in the world around him was creating a strong community around him. And he continually worked on his leadership capacity. And over the 10 years, at some point, he created a tattoo on his arm that's a sun with a redwood grove around it that reminded him of his core strengths and his purpose. And today, guess what he's doing, Lenny?

**Lenny Rachitsky** (00:07:59):
Killing it.

**Rachel Lockett** (00:08:00):
He's not only killing it-

**Lenny Rachitsky** (00:08:02):
[inaudible 00:08:02] vision.

**Rachel Lockett** (00:08:02):
... but he's running a community, a global community for Coinbase called Base and the Base app.

**Lenny Rachitsky** (00:08:08):
Oh, wow.

**Rachel Lockett** (00:08:08):
It's the largest Ethereum L2 in the world, and it's a community of creators and developers, and he's having a great time. He's having more fun than ever. And so, I think for the people who are committed to excellence and impact, recognizing that if they lean into their gifts and they get back into their purpose, they can have more fun while having an impact on the world.

**Lenny Rachitsky** (00:08:33):
This story reminds me of just why I love these sorts of conversations because the sort of stuff we're going to be talking about, and we'll get into it right after this final preamble, is stuff that's usually locked away in these very small rooms, are only accessible to folks with a bunch of money. This is stuff people pay tens of thousands of dollars, hundreds of thousands of dollars for over the course of their career. And I just love the idea of sharing all this with everyone to help all learn from the stuff that you've learned from all these people you've worked with. So, I'm really excited to be digging into stuff. The first thing I want to dive into, I actually asked you, when clients come to you, what is the biggest gap they have that is keeping them from being successful as leaders? And you told me it's essentially knowing when to coach versus knowing when to just tell people what to do and learning to coach. Talk about what you see there, why this is so important, and how you help people develop the skill.

**Rachel Lockett** (00:09:24):
I think that most leaders, especially technical leaders, assume they have to have all the answers. People have climbed the ladder in whatever realm they're in because they've been dependable, reliable, the smartest person in the room. But when you're leading a quickly-scaling company, you quickly have less context than the people you're around. And the way you were operating before doesn't work because you don't have the ability to wrap your arms around every problem in a deep way. So, I've seen leaders at every phase from frontline managers up to running an 8,000-person company struggle with knowing when do I have to have the answer, and when I don't have the answer, what options do I have?

**Rachel Lockett** (00:10:12):
But great leaders know that when you try to advise and have the answer all the time, you're not actually equipping your team to go solve the hard problems. You're training your team to come to you with all of the hard problems. And coaching is a different way. It's an alternative path that unlocks brilliance in your team and is way more motivating for the people around you. So, coaching is actually a learnable skill, obviously, because there's tons of coaches around Silicon Valley, but you don't have to coach in the same way that an executive coaches. You can shift your energy into curiosity when someone brings you a hard problem to solve, and create space to get curious, and help them solve their own problem.

**Rachel Lockett** (00:10:59):
So, obviously, sometimes advising is the right path. If there's an urgent issue, the person coming to you doesn't have the skill they need, that's a time to advise and help. But leaders over-rotate, assuming the people that they've hired who are experts in their domain need them to solve the problem. So, I think it's useful for your listeners to actually know that coaching's an alternative, and I can help them learn some basic skills around this.

**Lenny Rachitsky** (00:11:31):
Okay. I'd love to learn those skills. What this makes me think about is there's this famous Harvard Business Review piece. I don't know. It's like 30 years ago maybe about the monkey on the back. You know this piece, where it's-

**Rachel Lockett** (00:11:42):
Say more. I think I do.

**Lenny Rachitsky** (00:11:44):
Okay, we'll link to it. It's this idea that as a leader, people always just coming to you trying to give you their monkey that's sitting on their back. And they're like, "Hey, this monkey is causing me all this problem. I don't know what to do. But this monkey, here you go. You take it, and feed it, and help it, figure out what it needs." And the role of a leader is to keep the monkey on the back of the person and help them figure out how to solve the problem, exactly what you're describing.

**Rachel Lockett** (00:12:04):
Yeah, that's a great analogy. I love that. I think leaders make things up when they don't have answers sometimes. A person comes to you with a problem and you just want to help. But the best way to help is actually doing something that most leaders don't do well. It's attuning to what is the context? What does this person need? What are they blocked on? And ask them with those questions so that they can solve their own problem.

**Lenny Rachitsky** (00:12:32):
Let's talk about how to get better at this. But first of all, when you said, "When is it actually smart to just tell them what to do?" You said it's when they don't have the skills to do it. Is there any other kind of heuristics of like, okay, just tell them what to do in these cases?

**Rachel Lockett** (00:12:43):
Yeah, it's an urgent issue and you actually have an answer that you want to drive. So, don't coach and make it a game. You want your person on your team to guess what's in your mind. That's not a good time to coach. You have something you absolutely want them to do. You know the right answer. You want them to be motivated to go do it. Advise them. Help them see the path. But most leaders over-index on that solution. So, I want to share. Maybe Lenny, I can teach you two skills that I think are the basics of leader coaching-

**Lenny Rachitsky** (00:12:43):
Let's do it. I'd love that.

**Rachel Lockett** (00:13:21):
... that you can use in your own life tonight with your wife-

**Lenny Rachitsky** (00:13:24):
She's going to love this.

**Rachel Lockett** (00:13:25):
... or anyone you operate with.

**Lenny Rachitsky** (00:13:25):
Let's do it.

**Rachel Lockett** (00:13:27):
And hopefully, your listeners can use them too.

**Lenny Rachitsky** (00:13:30):
Let's do it.

**Rachel Lockett** (00:13:31):
Okay. So, the first skill is active listening. And Lenny, you're probably a good listener because this is what you do for a living is you listen to the people who come on your podcast. But I don't know if you've seen Fight Club. There's a quote, "Most people aren't listening. They're just waiting for their turn to talk."

**Lenny Rachitsky** (00:13:49):
Absolute-

**Rachel Lockett** (00:13:49):
This is rampant in tech. And great leaders flip that script and tune in. They're the kinds of leaders who walk into a room, and they can see the elephants. They can name them. They can ask the hard questions to get people collaborating. So, there's actually three levels to listening. So, the first level listening, level one is internal. Let's say you're talking to me about a problem. I'm thinking about the implications of that problem on me. I'm completely distracted with my own inner dialogue. That's level one. Most people go through their world rushed and in level one. Level two listening is focused. So, you're talking to me, and I can repeat back what you're saying. So, I am listening to the words you're describing, and that's typically what happens in a good one-on-one. We're problem solving together and focused on your words.

**Rachel Lockett** (00:14:46):
Level three listening is global listening. So, that's what I'm hearing beneath the words. I'm hearing what you're communicating, not just what you're saying. I see your body language. I notice your tone of voice. I know the context around what you're talking about, and I can reflect back more insight about what's happening than you're aware of because I'm understanding everything you're communicating. So, dropping into level three listening is what great leaders do when they're influencing, when they're selling, when they're pitching a vision, and definitely when they're coaching. So, do you want to try it?

**Lenny Rachitsky** (00:15:22):
Let's do it.

**Rachel Lockett** (00:15:25):
Okay. How about this? I'll demonstrate some level three listening. I'm going to ask you a question.

**Lenny Rachitsky** (00:15:30):
Okay. Uh-oh.

**Rachel Lockett** (00:15:32):
You told me earlier, you're a father.

**Lenny Rachitsky** (00:15:34):
Yeah.

**Rachel Lockett** (00:15:35):
What is it like to be a dad?

**Lenny Rachitsky** (00:15:38):
Wow. What is it like to be a dad? It's amazing. It's like the most amazing thing I've ever done in many ways. I love it so much. It's also quite challenging at times dealing with setting boundaries and knowing when to just let him do the thing he's really excited about or just saying, "no," and just letting him cry for a while. That's something I've been dealing with recently, but it's like everything people tell you it is basically in every way except the joy is so much higher, so much higher than you hear from other people because people always talk about all the downsides, all the pain and challenges.

**Rachel Lockett** (00:16:15):
Yeah, and I see you when you talk about being a father, initially, I saw you really squirm in your chair. Well, this is a big question. And you looked up and down and kind of avoided my eye contact at first because my sense is you love being a dad, and it's so challenging. It's so tiring. And I'm hearing both of that in your answer. The high joy and the discomfort in having to sleep train, and having to disappoint, and navigating challenging behavior.

**Lenny Rachitsky** (00:16:51):
Nailed it. That was very nice to hear. Clearly, you listened to everything I said and that was a really good example of active listening.

**Rachel Lockett** (00:16:59):
What does it feel like to be seen that way?

**Lenny Rachitsky** (00:17:02):
It feels really nice. It feels really nice to be heard. And it's not just like you're repeating back my words. It's here's what I got out of the level below what you're saying, and the gist, and the bigger picture.

**Rachel Lockett** (00:17:14):
Yeah, there's some emotional connection when you listen actively, and that took less than a minute. So, what I want to invite listeners to understand is that active listening doesn't mean you're setting up an hour coaching session with every person on your team. No one has time for that. But even in the time you're already spending, just focusing on the other person in a way that is novel and really gives them your full attention so you can see their feelings under what they're saying goes a long way to motivating your team and helping them understand what's actually happening under the surface in this situation.

**Lenny Rachitsky** (00:17:50):
I think there's just so much power in different words, repeating back what they said. That's almost implied in what you're describing. It sounds like... So, I don't know, like a trick they'll see through. But knowing that you're listening to me and you're going to show me active listening, it still feels really nice to just hear back what I said. There's a lot of power in that and it's subtle.

**Rachel Lockett** (00:18:10):
Yeah. Great. Yeah, there's an element of synthesizing what I'm hearing verbally. That's the focused listening part, and then, mirroring back the emotions that I'm noticing.

**Lenny Rachitsky** (00:18:18):
The emotions.

**Rachel Lockett** (00:18:19):
And even things that I'm guessing, and I can say, "Is that right?" And you can say, "No, actually, I'm not conflicted about the challenges of being a dad. I just am so joyful." And then, now, I understand where you're coming from and so do you.

**Lenny Rachitsky** (00:18:33):
Awesome. Okay, so this is a core skill of coaching is active listening.

**Rachel Lockett** (00:18:37):
Yes. So, that's listening. Second skill, powerful questions. So, asking powerful questions means I'm curious about what's really going on, and there's not one right answer. So, a powerful question helps you gain insight and it takes you to a new solution set you didn't have before. But it's not me leading the witness. I'm not trying to guide you to a specific answer. That wouldn't be a powerful question. So, something that I like to equip leaders with is four kinds of questions that you can ask to unlock insight.

**Rachel Lockett** (00:19:17):
So, the first kind is I use a GROW model. So, the GROW model just is four different categories of kinds of powerful questions. So, the G in grow is goal. So, what does success look like? What's the outcome that you want to have? Any question that's around defining the best case scenario. The R in the GROW model is about your current reality. Where are you stuck? What are your current challenges? What have you tried? The O is about your options. So, let's expand the opportunities that you can understand of the choices you have in front of you. What are the various paths you could take? And the W in the GROW model is the way forward. What are you going to do next? So, this sounds simple, and it is simple if you take the time and space to listen carefully and ask any of these questions. The people on your team will appreciate the space and time to unlock an option that they didn't think of before and walk away with a concrete next step.

**Lenny Rachitsky** (00:20:21):
So, just to reflect back what you're saying, so someone comes to you with a monkey on their back. Here's a problem I'm trying to solve. This percent of my team is just not doing something right or this feature isn't working, something like that. So, first of all, it's listen, be very active in your listening. Reflect back what you're hearing, their emotions. And then, ask them questions around what does success look like for this? What is the goal? What is the goal? What does success look like for the thing you're trying to do here? What does success look like? Two is just what's today's reality? What's happening today? Then, options. Here's options that you think exist. So, this is you asking them what are the options?

**Rachel Lockett** (00:21:01):
Yeah, what are your paths forward? What could you do next?

**Lenny Rachitsky** (00:21:03):
What could you do next? And then, this is organic. So, it's not just like one, two, three, four, I imagine.

**Rachel Lockett** (00:21:10):
Yeah.

**Lenny Rachitsky** (00:21:10):
But the final step is just, okay, what's the way forward? What do you want to do?

**Rachel Lockett** (00:21:14):
That's exactly right. And you don't have to do it in this order. These are just four kinds of questions.

**Lenny Rachitsky** (00:21:14):
I see.

**Rachel Lockett** (00:21:19):
So, you might come and someone's super clear about their outcome. You know that. You don't need to spend any time asking them questions about that. Maybe you just want to really dig in on where are they stuck? And once they start talking about their reality and where they're stuck, then they realize, oh, I'm stuck because my cross-functional partner is blocking me, and I don't have any relationship with them. I need to go meet with them actually and just have a breakthrough conversation, tell them where I'm stuck. So, sometimes talking this out loud, just creating that space for them is going to help them tremendously.

**Lenny Rachitsky** (00:21:48):
And there's kind of an implication here that the person often knows the answer or can come to the answer, and they just need a little bit of nudge to get there.

**Rachel Lockett** (00:21:56):
Yeah, this is definitely you want to coach when you think the person you're talking to has the right context and can solve their own problem. That's a premise of coaching. You wouldn't coach if someone needs your guidance and comes to you and says, "Hey, I'm trying to take my company public. You took your company public. Can you tell me exactly the steps you took to get there?" Not a good time to coach.

**Lenny Rachitsky** (00:22:20):
This begs the question, what if they just come to a terrible conclusion and you're just like... Advice on when to actually just like, "What about this instead?"

**Rachel Lockett** (00:22:30):
Yeah, okay. I think that's great. So, if you have a really strong negative reaction to what they're sharing, of course it doesn't behoove anyone to hide that. I think you get curious. "Hey, help me understand how you came to that conclusion because here's my reaction to that." So, you're honest, but you're also curious. So, coaching in a manager or a leader context is not the same as in an executive coaching conversation. You're managing this person. You're responsible for their outcomes. You're not setting up an hour-long coaching session, you're just using coaching as an additional tool in your toolkit from advising. And you're creating more space, maybe 15% more space in your one-on-ones, in your meetings for open-ended questions.

**Lenny Rachitsky** (00:23:14):
I love this phrase, help me understand. One of my managers used to be really good at this, just like... You could tell, he's like, "Help me understand this part of your thinking."

**Rachel Lockett** (00:23:22):
Yeah. And the other thing that does when you're curious and you don't just shut down someone's idea, is you're helping them think. You're not helping them realize they're going to screw it up unless they come to you for advice. You're helping equip them with the right questions to ask and the right skepticism to have. And so, it's always useful to be in conversation when someone who reports to you has a different worldview than you do. There's some reason they came up with this great idea that you think is a terrible idea. And actually, that's where the learning happens.

**Lenny Rachitsky** (00:23:56):
And you may actually be wrong and you may realize, okay, they actually have the better solution. I get it now.

**Rachel Lockett** (00:24:01):
Yeah. This actually happens to me all the time in talent conversations. Because I have a background in being an HR business partner, and I'm working with CEOs and they're thinking about building their leadership team. And I want everyone to have a very rigorous stance on their talent because if you have an A plus squad, you're going to do great things in the world. And sometimes, they come up with an idea to performance manage someone who's clearly not working in the role, but think, oh, maybe I'll wait six months, and then, we'll have a conversation. I have a strong point of view. I'm not going to let that slide, but I'm going to say, "Help me understand why that is a good idea," and I'm going to press on that. And if they don't come to an idea that I'm aligned with, I'm going to share openly my perspective while still empowering them to solve their own problem.

**Lenny Rachitsky** (00:24:49):
To close the loop on this piece of advice, is there an example you could share to make this super concrete for folks?

**Rachel Lockett** (00:24:55):
Well, I'm going to give you an example of a client, I'm going to call him Jeff, who runs an AI company. And he was essentially playing the role of the head of product also. And he had a growing number of engineers and designers, and his customer base was growing rapidly. And he started to feel completely overwhelmed. So, he came to me and we started coaching together. And soon, he realized that he was the blocker on every decision, every business decision, every product decision. And he was resenting it. He wanted his team to take more ownership. But with some coaching, he realized he was training his team to come to him with every decision because he had always operated that way. So, he decided to create squads and have small pods of engineers, product leaders, and designers focus on subsets of the team. Very normal as you have a small startup scaling. But he didn't have an engineering manager and a product leader for every one of them.

**Rachel Lockett** (00:26:00):
So, this was a little bit earlier than he was equipped for because he did it out of necessity. And he also realized he needed to create some behavior change for the way he was interacting with that tech lead on each project so that they would take more ownership. So, we really invested in this idea of I'm going to start to set the system up so we have a product review every two weeks, they each have clear KPIs they're driving to that we co-design, and for this next quarter, I'm shifting from the role of deciding on everything to coaching. I'm going to really ask good questions in our check-ins. I'm going to align to the KPIs, ask how things are going, ask where they're stuck. And I just had a session with him last week. It's amazing to see him because he's so much more energized. He said, "The squads are moving so much faster. The teams feel more empowered and motivated." And he has time to pick his head up and plan for 2026...

**Rachel Lockett** (00:27:01):
And he has time to pick his head up and plan for 2026, and spend his time and his gifts. Which are product, vision and strategy. So that's more of a global example of what results from leaders shifting from the mode of solving every problem to coaching.

**Lenny Rachitsky** (00:27:19):
That's such a great example of just the power of this, this makes your life so much better. Because other people can start picking up the slack and not come to you for everything. And it's like, listen better, ask a few powerful questions and so much improves, so much changes.

**Rachel Lockett** (00:27:35):
Exactly.

**Lenny Rachitsky** (00:27:36):
Everyone around you gets better.

**Rachel Lockett** (00:27:37):
Lenny, let's try it.

**Lenny Rachitsky** (00:27:38):
Okay, let's try it.

**Rachel Lockett** (00:27:38):
So I want you to tell me a challenge. It could be a personal challenge, a professional challenge. Just bottom line, something that you're struggling with.

**Lenny Rachitsky** (00:27:51):
Whoa. The main thing I struggle with these days is just endless work. I feel like this newsletter, I started this newsletter six, seven years ago at this point, and originally it was like, I'm just going to build this chill newsletter, do this on the side. Just kind of chill out for a while. And now it's just like, it just grows. I couldn't help but make it more awesome and bigger and have this podcast now and other stuff I got going on. So it's always this.

**Lenny Rachitsky** (00:28:18):
So I'm in a world now where it's just this ... the way I think about it is the Indiana Jones boulder is constantly in my back rearview mirror just coming at me. Because I need to get a newsletter post out, get podcast episodes out, do all the things associated with that. I'm also just in the middle of ... I have this large Slack community at Twitter and LinkedIn, so I'm just constantly being barraged with small little asks and things and all these little things that never ... it's hard to just ignore and say no to. So what I struggle with just endless work. I joke that be careful working for yourself if your boss is a workaholic.

**Rachel Lockett** (00:28:53):
I totally relate to that. Okay, so I'm hearing noise, barrage of needs and just constant requests of you online, in your work life. There's always something that you need to be doing. And you designed it that way yourself. So you're kind of aware of, I had this one intention of a path to freedom, insight. I imagine the newsletter was a fun passion project. And you couldn't help but make it this all-consuming full-time job.

**Lenny Rachitsky** (00:29:27):
That's right. Let me just add, it's like in so many ways the most awesome thing I could ever imagine doing also and extremely fulfilling. And I couldn't think of anything better I'd rather be doing.

**Rachel Lockett** (00:29:27):
Yeah.

**Lenny Rachitsky** (00:29:36):
So I think that's an important element.

**Rachel Lockett** (00:29:38):
Absolutely.

**Lenny Rachitsky** (00:29:38):
It's this Indiana Jones boulder constantly chasing me.

**Rachel Lockett** (00:29:41):
Yeah, I can feel the gratitude and the resonance with what you get to do every day. And yet I hear you questioning, why does it have to feel like I'm fighting for my life while I'm doing this thing I love?

**Lenny Rachitsky** (00:29:56):
That's one way to put it, yeah. This boulder is squishing me.

**Rachel Lockett** (00:30:00):
I mean, the Indiana Jones boulder is coming for you.

**Lenny Rachitsky** (00:30:01):
Oh man.

**Rachel Lockett** (00:30:02):
That's a fight or flight instinct we all have.

**Lenny Rachitsky** (00:30:04):
That's true. That's true.

**Rachel Lockett** (00:30:05):
Okay, so thank you for being vulnerable enough to share that with your listeners and with me. I want to ask you, what would dream state look like? So let's say in six months you're still running this beautiful business that you've created. And you feel differently. What is happening?

**Lenny Rachitsky** (00:30:33):
What I imagine is the same thing mostly, just much more free time. So more time to experiment and play with other things. And at the same time, the newsletter continues to be awesome and high quality, the podcast continues to be awesome and high quality. So it's continuing to put out the same high quality stuff, just more free time, less ... the boulders may be smaller.

**Rachel Lockett** (00:30:56):
Yeah, okay. So what does free time enable for you? When you think about ... I hear your deep commitment to quality products and quality output. But this longing to feel a little bit more playful or flexible with those parts that are essential to you.

**Lenny Rachitsky** (00:31:17):
Very practically, it's time to play around with AI tools. Just explore and just kind of tinker. Versus, okay, all the time I have, I need to focus on the newsletter, make it better for next week. Oh, it's coming out, here's things I got to do. Oh, this podcast, got to prep for that, got to edit this thing. So it's just time to tinker and explore and just like that kind of stuff.

**Rachel Lockett** (00:31:37):
That makes sense. And what's important about exploring and tinkering to you?

**Lenny Rachitsky** (00:31:42):
Because in the work I do, I need to stay ahead on where things are going. I can't just sit around and pontificate from a cloud. I need to really understand how things work, what's working, what's not, what's real, what's not. So being on the ground as much as I can with what's actually going on versus just putting out content.

**Rachel Lockett** (00:31:59):
That makes sense. Your voice is moving really fast. I kind of hear you feeling behind, even in the way you're describing what you're doing.

**Lenny Rachitsky** (00:32:06):
A good [inaudible 00:32:09] listening.

**Rachel Lockett** (00:32:10):
What's interesting to me is when you're talking about exploring and tinkering, when you first said it, you said it in kind of a spacious way. It's fun to explore and tinker. You're deeply naturally curious. You find new insights. But then I also heard you say, "And it's a way to stay ahead, I have to do it in order to feel like I'm informed." So what do you make of that difference?

**Lenny Rachitsky** (00:32:37):
Yeah. Yeah. Those are two sides of the coin. There's another element of ... I guess let me answer that question. I think those are both true, I don't know. The reason I got into this is because it was so fun and so interesting. Just like, here's what's happening, here's what the future-

**Rachel Lockett** (00:32:53):
Yeah.

**Lenny Rachitsky** (00:32:54):
Here's advice, here's ways to improve in the work that you do. So I still love it. It's just I have less time to do that part and more it's just the machine of the treadmill of content, content, content. There's also just spending more ... I didn't even mention this. But just spending more time with my son and my wife, that would be really great just to have more freedom to go do stuff. Which we have a lot of that, but more is great.

**Rachel Lockett** (00:33:16):
That makes sense. Okay, so the goal that I hear is not so dramatically different from today. It's that you hold on to this high quality output, but you have space for exploring and tinkering. And for spending quality time with the people you love most.

**Lenny Rachitsky** (00:33:33):
Yes. One way I'm thinking about as you reflect that back is 25% more free time while everything else continues to be awesome. And the challenge I run into is I sometimes get that extra time and then I fill it with more projects and opportunities. That's the problem right there.

**Rachel Lockett** (00:33:50):
Yeah, there's that inertia of moving fast, taking advantage of the moment that's coming.

**Lenny Rachitsky** (00:33:54):
Yeah, yeah.

**Rachel Lockett** (00:33:54):
So that's a perfect shift into, what are your current ways of operating that get in the way of having that 25% of free time?

**Lenny Rachitsky** (00:34:03):
It's just agreeing to more things. Just like, oh look, I'm kind of free right now. Oh, okay, let's do this talk here. Let's agree to this thing here. So it's just once I feel freedom, I'm like, "Okay, I could do that other thing." And so I commit to more stuff.

**Rachel Lockett** (00:34:18):
Yeah. And how is that commitment to saying yes to things that come at you serving you?

**Lenny Rachitsky** (00:34:24):
Not great.

**Rachel Lockett** (00:34:27):
Well, it's serving you in some ways. You're doing it for a reason.

**Lenny Rachitsky** (00:34:31):
Yeah.

**Rachel Lockett** (00:34:31):
What benefit does it have to you?

**Lenny Rachitsky** (00:34:33):
Well, it depends on the thing. I actually have a rule of never doing a talk or going on another podcast or going to events really, because I find I never really get much out of it, and it distracts me from the stuff I could be doing. So I've set up a lot of policies of just turning down things that don't serve me. But I still crumble and say yes to stuff.

**Rachel Lockett** (00:34:33):
Yeah, that's smart.

**Lenny Rachitsky** (00:34:57):
So to your point, there's value here and there when I take on more work. And then I end up overworked again.

**Rachel Lockett** (00:35:04):
Yeah, I'm hearing there's just a pattern, it's like a reflexive pattern of even though you set a rule for yourself to say no to certain things and you seem proud of that boundary. You naturally break it or you fall into filling it with other things.

**Lenny Rachitsky** (00:35:16):
That's right. Exactly.

**Rachel Lockett** (00:35:18):
Okay. So you're stuck in this kind of addiction to doing more and signing up for more, which is so normal in our world, and probably most listeners can relate to that. That's kind of the soup we swim in. So we have to be conscious of what inputs we have around us.

**Rachel Lockett** (00:35:34):
So let's explore your various options that you have in front of you. One that you mentioned already you tried was to make a list of the things you don't want to be doing anymore, like things you want to say no to. And really committing to that and sticking to that. What are the other kinds of things you could do to help you prioritize and create that sacred 25% of extra time for yourself?

**Lenny Rachitsky** (00:36:01):
Something I've already done, which hasn't kicked in fully yet, but that I'm really excited about is I reduce the cadence of my newsletter and podcast. Which in theory, should be a massive change. But the cadence of the podcast hasn't shifted down yet, it'll happen next year. The newsletter cadences, I basically changed my promise to readers instead of, you'll receive a newsletter every week. Now it's, two to four times a month depending on what's going on.

**Rachel Lockett** (00:36:01):
Yeah.

**Lenny Rachitsky** (00:36:28):
Which felt huge. The problem is I still like every week I'm like, "Oh, I want to write about this thing. Oh, every week there's nothing's happening, I got to put this out." So I'm almost not taking advantage of that opportunity. So something I could do is actually not publish every week.

**Lenny Rachitsky** (00:36:45):
Another is just bring on some more help. Which is difficult because I've got a lot of good help and there's only so many things other people can do for me that isn't writing an awesome newsletter and recording conversations like this. But I'm always thinking about, and I should think deeper about where can people take more load off my plate?

**Rachel Lockett** (00:37:04):
Yeah, I love that insight. What I'm hearing is do less in certain areas and think about your team and really expanding the capacity of your team. And be rigorous about the things you can hand off, that you may have limiting beliefs around the things you need to do versus the people on your team.

**Lenny Rachitsky** (00:37:21):
I might, I might. And then your point I loved, which is just improve my policies of what I say no to that don't serve me.

**Rachel Lockett** (00:37:29):
Yeah. What are the things you could be saying yes to if you said no to more things?

**Lenny Rachitsky** (00:37:34):
Just playing around with stuff. Just space to explore and tinker, and just sit around and think. Versus just go, go, go, go, go.

**Rachel Lockett** (00:37:43):
Yeah. I just see you feel so light and excited in that. Like you almost are giddy when you think about that spaciousness.

**Lenny Rachitsky** (00:37:50):
That'd be so nice.

**Rachel Lockett** (00:37:52):
And I just want to name reflect back to you how special that is and how much more creative you could be in your work when you have that space and time.

**Lenny Rachitsky** (00:37:52):
I love that.

**Rachel Lockett** (00:38:01):
And your bucket's full with care.

**Lenny Rachitsky** (00:38:04):
I feel that, I feel that.

**Rachel Lockett** (00:38:05):
Yeah. So what's one thing you could do in the next two weeks that would help you get closer to the kind of spaciousness you want to create?

**Lenny Rachitsky** (00:38:14):
I love that ... as we go through this, I'm thinking about this growth framework and I love how you're executing it. Like I see it in action, it's so good.

**Rachel Lockett** (00:38:21):
Yeah, I'm trying to do very simple coaching right now, just [inaudible 00:38:25]-

**Lenny Rachitsky** (00:38:24):
Yeah, yeah. No, this is great

**Rachel Lockett** (00:38:26):
... that it's really easy to follow for your listener.

**Lenny Rachitsky** (00:38:28):
Yeah, yeah. This is great. Okay, so what's the one thing I could do in the next couple of weeks to help you move forward on this? I think one is at least skip a week or two of the newsletter and just actually stick to that plan. But it's tough because the next two weeks I got already planned. I got to write a gift guide, that's my ... okay, so the week after, I'll take a break. Okay, cool. So two weeks from the recording this I won't publish a newsletter. And then I'm going to revisit my policies on what I say yes and no to.

**Rachel Lockett** (00:38:59):
I love that. Think about everything you're saying yes to and what are things you want to say yes to that you could treat it with. So really consider that it's a trade off every time you say yes to something. The more resonant you are with the end state and what's possible for you, the easier it is to be disciplined in the near term.

**Lenny Rachitsky** (00:39:20):
I love just that element of here's what you'll get out of this. It's not just no, no, no, no. It's like yes to this other thing you really, really want to do.

**Rachel Lockett** (00:39:25):
Yeah. Exactly. Yes, say it like a resonant full body yes to the things that are in [inaudible 00:39:32].

**Lenny Rachitsky** (00:39:25):
Hell yes.

**Rachel Lockett** (00:39:32):
Yeah, hell yes. With an exclamation, exactly.

**Lenny Rachitsky** (00:39:36):
Awesome.

**Rachel Lockett** (00:39:36):
Well thank you, Lenny, for letting me just demonstrate what powerful questions are. And the reason I wanted to do that with you is you brought an example that's actually pretty big. It's an emotional thing, it's a cultural norm, it's a way of being that we've all learned to be through growing up and operating in tech, especially. So even with that kind of topic, using a simple grow model can be useful. But people are coming at your listeners with topics that are very complicated, technical, urgent. But the same kinds of questions unlock new opportunity when it's about how to build technical infrastructure or how to influence the executive team or how to ship the go-to-market strategy. So I just want a name that's very transferable.

**Lenny Rachitsky** (00:40:28):
I love that I got great advice in this conversation already.

**Rachel Lockett** (00:40:34):
Good.

**Lenny Rachitsky** (00:40:35):
What a great ROI for me at least.

**Rachel Lockett** (00:40:37):
What did it feel like to be coached on your own podcast?

**Lenny Rachitsky** (00:40:41):
It was unusual. I'm just like, wait, I got to get back to asking you questions. That's where our minds are right now.

**Rachel Lockett** (00:40:44):
Okay, all right, all right. We can flip it, we can flip it. I do want to name that typically when you're coached versus told what to do, you're more bought in. So if I told you, Lenny, I've heard all kinds of leaders come to me talking about being too busy. Here's what you should do. Write a list of all the things you're doing, write the things you're going to delegate, cut out 25% of the things on your calendar. I could have given you a laundry list of things that I thought you should do without much context. But you're the expert on your own context and actually what resonates. And you're much more likely to do it if you came up with it.

**Lenny Rachitsky** (00:41:22):
I was going to mention that earlier. That is so incredibly true. No one wants to ... unless you ask for advice, very few people are like, "Please tell me what to do."

**Rachel Lockett** (00:41:30):
Yeah.

**Lenny Rachitsky** (00:41:31):
I really love just unsolicited advice. That doesn't go well.

**Rachel Lockett** (00:41:35):
Yeah. And great leaders often say, "Do you want advice or do you want some space to think about it? Can I help you think it through? Or would you like me to tell you what I would do?" And both are fine in certain situations. So asking is useful too.

**Lenny Rachitsky** (00:41:48):
Yeah. Okay. That's such an important element of this that we should have mentioned and I'm glad you did.

**Rachel Lockett** (00:41:55):
Yeah.

**Lenny Rachitsky** (00:41:56):
Okay. So I'm going to go back to asking you questions.

**Rachel Lockett** (00:41:58):
Great.

**Lenny Rachitsky** (00:41:59):
This is a good segway to something I wanted to spend a little time on, which is burnout.

**Rachel Lockett** (00:42:03):
Yeah.

**Lenny Rachitsky** (00:42:05):
What I'm talking about is stuff that often leads to burnout. I'm definitely not burnt out, but this is a common problem in tech. Where people feel depleted and just go too hard. So many people I've worked with just left tech. I had a colleague at Airbnb, he's like a park ranger now near woods. That's how far tech-

**Rachel Lockett** (00:42:05):
So peaceful.

**Lenny Rachitsky** (00:42:26):
So peaceful and so wonderful. But I think that's just people go so hard sometimes and then just get burnt out and never want to do anything like this again. I know that you've spent a lot of time on this with founders and you have a really helpful approach. So just talk about what you've learned about helping leaders in tech avoid burnout and feel energized and excited about their work for a long time.

**Rachel Lockett** (00:42:47):
Yeah. Well first of all, I'm glad you brought it up. It's a huge problem. I remember when I was coaching top talent at Stripe, Patrick Collison is really committed to retaining top talent. And I created a program with my team for the top 50 executives in the tech side of the house. And we looked at their engagement scores, we did coaching circles. And it was so sad to see how exhausted that group of incredibly creative and committed leaders was in that moment.

**Rachel Lockett** (00:43:19):
And it's so common that people who start with incredible inspiration and incredible capacity start to feel like they've been pushing and pushing and pushing for years. They're parenting, they're leading, crazy things are happening to the business. And they just can't muster the same kind of motivation they once had. And I see this with my clients all the time.

**Rachel Lockett** (00:43:44):
So I've also witnessed people who are still inspired and continually energetic and seem to have some secret well of some diesel battery, or I guess I should say a Tesla battery that helps them through really hard challenges, and they're still having a good time. And so what I make of that is that when people are in their gifts and their strengths firmly, most of the time, they have more energy. We all have more energy when we're operating from the things we naturally are good at and the things we innately love doing.

**Rachel Lockett** (00:44:25):
So I try to help my leaders see that they can design their lives so they're spending 80% of their time in their gifts. That seems really ambitious because you're stuck within a context that requires a lot of you. Especially when you're executive at a huge company. But I also interact with founders who started a company with great inspiration, an entrepreneurial vision, and their job has obviously changed every six months. Once you fundraise, once you grow a team. And sometimes, especially technical founders will start solving a technical problem they're absolutely obsessed with. They spend three years doing it, the product ships, and then they're stuck managing a board and a team. And they don't even realize they're doing a completely different job than the one that played to their strengths.

**Rachel Lockett** (00:45:17):
So one tool I like to give is for people to actually take two weeks and every night reflect on, what are the five things today that gave me the most energy? And what are the five things that depleted my energy the most? If you do that for two weeks and you look at patterns, you can tell, what are the natural gifts that I'm living in? And what are the things that I'm stuck doing that are exhausting? And they're just slowly ... it's like a slow leak in your gas tank that over time shows up in your daily amount of energy.

**Lenny Rachitsky** (00:45:52):
I so believe this advice is so effective. This is the way I actually approach when I left my job. I very actively did this. I paid attention every day, what gave me energy? And what sapped my energy? And let me just do more of the thing that gave me energy and less of the thing that saps me. I want to talk about just like, there's only so much you can change, but I want to talk about that. And so initially I was like, maybe I'll become an advisor and consultant kind of person. I actually found that was super depleting for me.

**Rachel Lockett** (00:46:22):
Interesting.

**Lenny Rachitsky** (00:46:23):
Doing these calls and talking to people because it's like surface level, here are some things I would do. And it was just so unexciting and energizing. But writing was really energizing, which I'd never expected.

**Rachel Lockett** (00:46:23):
I love that.

**Lenny Rachitsky** (00:46:36):
And that's what I did, and I just followed that pole.

**Rachel Lockett** (00:46:38):
And it sounds like maybe you need a refresh, Lenny.

**Lenny Rachitsky** (00:46:42):
Oh, interesting.

**Rachel Lockett** (00:46:43):
There's always more tuning you can do to your gifts. You're in this amazing ... you've clearly been successful for a reason, you're in your strengths and you're paying attention to what brings you energy. We can always do that more throughout our life. I think it's a process of continually tuning in to where your spark is. And protecting that spark, feeding it.

**Lenny Rachitsky** (00:47:05):
I love that insight, that just blew my mind.

**Lenny Rachitsky** (00:47:08):
So very tactically, the way you would do this is for two weeks every night is the idea, reflect back on that day and write down five things that gave you energy, five things that sap you of energy.

**Rachel Lockett** (00:47:19):
Yeah, there's so many different activities you could use. So that's one. I like an activity of actually asking five to ten people in your life with a very simple email. When I walk in the room, what shows up? What are my strengths? What are the gifts? If you really don't know them and you haven't spent a lot of time in this realm, that's also an opportunity of actually asking the people who know you best, what your core gifts are? And when do you have the most inspiration?

**Rachel Lockett** (00:47:46):
You can also look through your calendar and note themes. Okay, over the last month, what are all the things I look at on my calendar that I'm excited to do? What are the things I dread? Okay, why do I dread those things? What do those things have in common? So there's various ways you can get to what is your zone of genius? But what my invitation is to take that really seriously. It actually takes risk taking, it takes intention to design your life around your gifts.

**Lenny Rachitsky** (00:48:15):
Is there any advice for actually doing this? Say someone's just like, "Yeah, I got to do this." But most people don't actually do this. Is there a buddy you can nominate, just help you do this? Is it like if you have an EA, they can maybe help you with this? Is there anything you've seen?

**Rachel Lockett** (00:48:28):
That's a good question. Yeah, I love your ideas. I think that the people around you need to be on board and know what are your gifts. So for example, when I was an HR business partner, my boss bought into this and I explained to her, "Hey, I started at Stripe because I actually know I'm going to be a coach. I'm not going to be Head of HR. But I love working with leaders. So I'm going to do all the compensation strategy and all the org design, and I'm going to help product and engineering leaders. But what I have in the back of my mind is I'm honing my coaching skills."

**Rachel Lockett** (00:48:56):
And so when this opportunity to work on top talent retention came about, it was very aligned in the realm of coaching and L&D, background that I had. So she put me on that. So it's useful to name it to the people around you. What are your gifts? What are your interests? What skills are you really excited to hone? So that they are in a contract with you to help you and really apply your gifts to the business's needs. So that's one thing.

**Rachel Lockett** (00:49:23):
When you're a founder, a CEO, when you have the autonomy to consider, what are the role scopes around me? Then you can really hire around it. So I have some CEOs that I work with who are incredible visionaries, great strategists, really good at managing the board, hiring, et cetera. Terrible at managing their team. They hate it. So they hire a COO. And they work in partnership, they have one person who's really internally focused, they get to be externally focused. That works well, it's a symbiotic relationship. If you're honest about your strengths and you're weaknesses, then you can start to manage around them.

**Lenny Rachitsky** (00:49:59):
**Rachel Lockett** (00:51:28):
Absolutely.

**Lenny Rachitsky** (00:51:28):
Is kind of an implied piece of this.

**Rachel Lockett** (00:51:30):
Yeah, I think sometimes people hire a Chief of Staff to help them and compliment them. Sometimes people design their team with strengths and gifts that they don't have. So it's really ... you can get creative once you really understand, oh, these things give me a ton of energy and these things are exhausting. But I still need to fill this need for the business. What are all the ways I can do it?

**Lenny Rachitsky** (00:51:52):
And telling your manager, I think, is such a simple and important part of this. Telling them-

**Rachel Lockett** (00:51:52):
Absolutely.

**Lenny Rachitsky** (00:51:58):
... here's where I want to go, here's the things I want to get better at, here's the things that give me energy. Can we just try to make as much of my role that?

**Rachel Lockett** (00:52:04):
Yeah. Especially if you're executing well, people want to retain you. They want to know, what's going to keep you here for the next five years? And typically they think that means moving up the ladder. But maybe it doesn't for you. I think it does take the courage to move horizontally sometimes to get into your strengths. I mean, I've moved horizontally a number of times and I love what I do, I feel like I'm in my natural gifts. But it took me a few risks and some uncomfortable jobs that didn't feel like they were worthy of my experience in order to get there.

**Lenny Rachitsky** (00:52:38):
What's a good percentage of your work life that should be in gifts and things that energize you versus, okay, I actually got [inaudible 00:52:48] stuff.

**Rachel Lockett** (00:52:48):
My litmus test is 80%, that's the goal.

**Lenny Rachitsky** (00:52:48):
80%.

**Rachel Lockett** (00:52:48):
That's the aspiration. You're always going to have 20% of things you don't love doing. There's just the logistics of getting into the zone that you need to be in. But I really try to push people to think aspirationally, that if you're 80% of the time in your gifts, how much energy you have to give to the world. It's so much more inspiring.

**Rachel Lockett** (00:53:09):
So I want to tell you why I'm passionate about this topic because it actually is how I ended up as an Executive Coach. So 10 years ago, I was working at a small company called Remind, and I was running the UX Research team. And the CEO asked me to move into the Product Manager role for the Core Product team. And I was excited for the opportunity. I had non-technical background, but I thought, "Hey, all these strategists are up there creating the roadmap. I can do that. I know exactly what our users need." So I was excited for this.

**Rachel Lockett** (00:53:44):
I came into the team, there was I think 12 senior engineers, very opinionated, very skeptical, this non-technical PM, but we worked together. And what I did was I listened. I learned what do our users need? What does this team need? What's working and not working? And with-

**Rachel Lockett** (00:54:00):
What does this team need? What's working and not working? And within a month, this team was working well together. They were reviewing each other's code base. They were really disagreeing in a healthy way in our team meetings. They felt more connected to users. And I felt like, "Okay, this rhythm is working." But what I was also doing as I was at home stressing in the middle of the night about the new user experience, I couldn't decide which of the designs to go with. I was always over leveraging our data scientists, and I found myself swirling on decisions that didn't need to have so much stress involved. And one day I went for a walk with my colleague Zach Abrams, and he was a great product manager and he was listening to me ask all these questions about how to sell the vision of what this product would look like in the future.

**Rachel Lockett** (00:54:55):
And he said, "Rachel, you're a zone of genius or your gift is not being a product strategist, but I've watched you over the last few months, and you have gotten the team more motivated than I ever could, and you've influenced the entire executive team behind your ideas, and that's impressive. You're a people person." And at first I was offended. What? You think I don't have the ability to be a great product leader? And yet I sat with what he said, and I knew he was right. Both my parents are therapists. I never wanted to be a therapist. Here I am. I'm basically a work therapist. I love entrepreneurial energy, I love big vision, but I'm a people person. 

**Rachel Lockett** (00:55:40):
And I left that, and I realized I love what my coach does. I got trained as a coach. I went into HR leadership. And Zach, who was a gifted product strategist, went on to lead product at Coinbase and BREX and most recently Bridge, which was acquired by Stripe, and he's still my client. And we've watched our journeys over the last decade, and we've both been honing our gifts. Life is more fun when you're in your gifts, and you have more inspiration and capacity to offer the world. So I just want to share that story because it's helpful to be honest with the people you care about when they seem energized and when they seem depleted because sometimes it's a wakeup call for people to really think about what is their spark and to protect it and to feed it.

**Lenny Rachitsky** (00:56:36):
I love that story because I think most people when they hear this advice and this topic of, "Okay, I am feeling depleted, I'm feeling burnt out," I feel like most people jump to, "Okay, but I can't actually do anything about it. I have a job. I got responsibilities." What I'm getting from this is the most important step is jump to figure out what you actually should be doing. What gives you energy? What your gifts are. It feels like that's the biggest gap for people because once you know that, there are ways to do that. Talk to your manager. "It may not be possible today, but here's where I want to be going. Here's what I want to be spending time on." I love your point you made though about you actually have to be doing well for your manager to listen to you. You can't just be sucking and then like, "Oh, but I want to work on strategy."

**Rachel Lockett** (00:57:19):
Yeah. Well, it's no one else's job to help you live in your gifts. And what I notice in big companies is people are often annoyed or frustrated with their management for not making their job more interesting. It's like, "No, your manager's job is to help you perform in the job you were hired to do. It's your job to navigate your career." So over the arc of your career, how do you match your gifts with the world's needs? And if the world right now is your company, how do you understand the needs enough so that you can apply your strengths to those needs?

**Lenny Rachitsky** (00:57:54):
This reminds me, there's a couple of guests I've had on the podcast who did this. They're both founders, so this is specific to founders, but Rahul Vohra at Superhuman, he realized he's not the best executor operations person, so he hired a president that took all that office plate. And then Darmesh, co-founder of HubSpot, he knew from the beginning he didn't want to manage people, so he made a rule with his co-founder, "I will never have reports." And he's the CTO, I believe, and has zero reports, has no one-on-ones.

**Rachel Lockett** (00:58:23):
Yeah. And I think that it's a beautiful thing to recognize that, but then to actually address the needs of those reports also. I think often people know, "Ugh, I don't want to have one-on-ones," but just not having anyone to manage those people is not going to be healthy for your company. So you have to both take your strength seriously and actively manage around your weaknesses.

**Lenny Rachitsky** (00:58:45):
Is there any maybe last piece of advice on this topic of helping people get to a place where they're feeling much less depleted and just more energized at work?

**Rachel Lockett** (00:58:53):
I would start small. You don't have to leave your job and redesign your life. You can stop going to the optional things that are exhausting. You can leave space between the things that are depleting that you have to go to outside and go on a 30-minute walk and refuel your tank. Start with tomorrow. What are the three things you're going to do to plug up that gas leak and re-energize your spark? It might even be you love to read and you're going to start reading 30 minutes before you go to sleep every night. It doesn't have to be a dramatic life change, but recognize that only you know what is resonant and what is depleting, and it's your job to take that seriously if you want to show up purposeful and impactful in the world.

**Lenny Rachitsky** (00:59:41):
I love that advice. I've actually started reading before bed for 30 minutes, and that's been so joyful, like a physical book with a little nightlight.

**Rachel Lockett** (00:59:48):
I agree. I love a physical book. I have a Kindle, I got all the things, but a physical book on the couch is the best.

**Lenny Rachitsky** (00:59:55):
Yeah, it's just that nightlight is key because sometimes at nighttime you need a night book. So we've been talking mostly so far about individual improvement, how to figure out what you should be working on, just helping learning to coach, things like that. I want to take us up a little level above and talk about team skills, how people get better at working with other people. Something that you are in many ways known for is helping co-founders build better relationships. And in my experience one of the most challenging parts of starting a company is the co-founder relationship. A lot of people don't realize what they're getting into. You're basically getting married to this person in a very high stress situation, and you sometimes don't know much about them.

**Rachel Lockett** (01:00:38):
Exactly.

**Lenny Rachitsky** (01:00:39):
And then you not working well together is just a huge issue because that all trickles down, and everyone sees it. And when co-founders leave, it's really bad for everyone. So let me just ask you this, what have you found most helpful in helping co-founders build great relationships, stay happy and productive?

**Rachel Lockett** (01:00:56):
Yeah, thanks for asking this. I love working with co-founders because I think your core values as a person come out when they interact with someone else's core values. Conflict, healthy conflict or otherwise, is actually where your core values come out. So it's fascinating to watch people try to do something incredibly hard in the context of someone else's vision, someone else's strengths and weaknesses and navigate that together. So there's so much energy in the co-founder dynamic for me and for co-founders themselves. It's actually something that people don't feel comfortable going to their board about or talking to that many people about because it's a private matter. It's almost like in a marriage you go see a couples' therapist, but you don't tell all your friends that you can't stand your partner, but it keeps you up at night. So it's a really tender, important relationship, and there aren't enough supports for co-founders to navigate it. It's very normal. 

**Rachel Lockett** (01:01:50):
In fact, I know you probably know this stat, 65% of startups fail because of co-founder conflict, and co-founders are in a moment where they're trying to build the future for their business, but also trying to build their own livelihood. So there's so much at stake to get along with your co-founder. I think the core at its core, what you need in a healthy relationship is, one, self-awareness. What do I bring into this dynamic, and how am I being experienced by the other person? What does this other person bring into the dynamic, and how am I reacting to that? So the first is just collective awareness about what is our dynamic. I like to use the Enneagram for this, but there's all kinds of tools, self-awareness tools that you can use to give a common language to what is my thing and what is your thing.

**Rachel Lockett** (01:02:46):
A very classic one has to do with roles. CTOs tend to be skeptics. They love facts, they seek knowledge. They want depth of awareness and understanding, and they also like to be self-sufficient. This is a total generalization, but I've seen this pattern over and over again. The CEO is the person who had to sell the vision. They're a person who loves big picture vision strategy. They often are great at influencing others. They love to sell ahead of the reality of what the company's actually built. This creates an inherent tension between blind optimism and skepticism, and it's a dance that these two roles play together. So the first part is knowing the dance you're in, so you're not just stepping on each other's toes blindly. 

**Rachel Lockett** (01:03:34):
The second step is actually being conscious about the commitment you're making to your relationship. In a marriage, for example, I talk about co-founder vows and recommitments and renewals because in a marriage you get married, and a lot of times people build a family and then they think, "Oh, the relationship will just continue around each other all the time. We're doing this thing together." But just like couples need a date night, co-founders need time and space to connect with each other to come together and say, "How's this working for you? Are we still aligned on our vision and our strategy? How are we working together? What am I doing that's pissing you off? What are the things that have gone unsaid and that we need to talk about?" 

**Rachel Lockett** (01:04:20):
But if you're just in the hustle and bustle of running and scaling your startup, you don't make time for that conversation. So I think it's incredibly important for co- founders to make space for their relationship, whether that's a dinner every other week, whether that's going out to lunch regularly, whether that's just touching base business-wise, but having an in-person quarterly check-in. That space is critical for the health of a co-founder relationship.

**Lenny Rachitsky** (01:04:50):
On that second piece, the vows idea is such a good idea. Is that something you actually recommend, here's what I vow to do?

**Rachel Lockett** (01:04:56):
Yeah, here's what I commit to do. So recently I actually wrote an article with First Round, and we created a document to help co-founders think about what to integrate into their check-in. So we put out weekly check-in, monthly check-in and annual and just questions to sit down and ask each other. Active listening skills will come in handy in those conversations, but it's about taking space out of hustling and running the business to think about the business from a like to say, instead of being on the dance floor, you need time on the balcony to look down at what's happening. How are we doing? Is this still working for both of us? And the vows are really about, how are we going to be together? How are we going to show up? What's our culture that we're creating? 

**Rachel Lockett** (01:05:43):
Even if you don't want to go through a whole culture exercise early on in building your company, you should have some sense of how you want to show up for each other. How are we going to make decisions? How are we going to deal with conflict? These are things you can go into intentionally and design with your co-founder.

**Lenny Rachitsky** (01:06:03):
Awesome. We're going to link to that post The first step, Enneagram sounds like that's what you recommend, and this is basically a personality profile that a lot of people love.

**Rachel Lockett** (01:06:12):
I really like the Enneagram. I think you can also simply tell each other, "Here are my strengths. Here's what I see as my weaknesses, and what do you think? Give me some feedback. Do you agree?" And you can do that with each other without any personality assessment. If you want to just be scrappy and have an open connected conversation about, you could even say, what are the gifts I bring and the weaknesses I have, and how will I cover those? How will I lean into my gifts? How will I cover my weaknesses, and how will you? And then I think it's worthwhile having a conversation about, what are the gaps neither of us cover that we're likely going to need as we build this business?

**Lenny Rachitsky** (01:06:50):
What do you recommend people do when they are just like, "Our relationship isn't working great. There's a lot of tension"? All this advice we've been talking about, at the beginning, here's things you can do to set things up for success. Understand what you're good at, what you're bringing to the table. Consciously commit to, here's what I'm going to do, here's what you're going to do. Have these dinners or lunches. I love this metaphor of going out on the balcony and just reflecting on how it's going. So that's all really great. What if you're just already in it and it's really annoying, I don't like this person that much or so much tension constantly. What are a couple things they can do this week, next week?

**Rachel Lockett** (01:07:27):
Co-founders typically come to me either in this early phase where they want to intentionally build something great and they want to set it up for success. More often co-founders come when they're really frustrated with one another. They feel the tension is palpable. They can't stand it anymore, but they're still really deeply committed to the business so they don't see an out. And they knew that at some point they really loved this cofounder, so they see a possibility of recovering, and that's why they want to go get a coach. I'm going to give you an example of this PR duo running a fierce business scaling really fast. And at one point when they started, you had the visionary who was great at selling business. They were both incredible with PR, and the partner was incredible operationally. 

**Rachel Lockett** (01:08:17):
So as the business scaled, one took on a lot more business development and the other took on all the internal things, but was exhausted by all this people management and all of the elements of running a scaled team that she didn't expect to have to do. And when they came, I think both of them weren't sure, can we figure this out? Do we want to just sell this thing? Do we want to keep going? And I think someone said, "End it or send," was what one co-founder said to me. They're coming at this decision point. And what I saw them do is, one, they named current state really well. They were both able to share. We did use a 360. So they got feedback from their teams and then shared it with one another, but they were able to be open and vulnerable in what was working and what wasn't working. Not immediately, but over time. 

**Rachel Lockett** (01:09:16):
And they realized they used to love being partners in this work, but as they began to lead different teams, they grew very distant from one another. They were living on opposite sides of the country and actually just coming together and realizing what each other was missing and how lonely it is to lead this scaling company without each other's support and how they actually needed the counterbalance to their strengths and didn't have it was a important start to their healing.

**Rachel Lockett** (01:09:48):
And over our coaching, they turned back towards each other and they created more of a rhythm of how they would get together without me involved. And they ended coaching after our arc feeling renewed and really recommitted. They made some changes on their leadership team to fill their gaps. They also started, I think, meeting once a week virtually, and they started a cadence of getting together in person quarterly. And I don't mean to say that just that time means you're going to heal, sometimes coming together and really grappling. I had one last week where we all came together, we had a great full day in-person discussion about how they were making this co-founder duo, how they were making decisions. 

**Rachel Lockett** (01:10:36):
And after that conversation, it was really clear that one of the co-founders was unhappy and didn't appreciate the other one and was not going to change and realized he was a big part of the problem and I think is going to leave the business. But that's still success because it's clarity. You're not muddling in the dark, frustrated, unconscious about the interpersonal dynamics you're in, you're making a choice based on your strengths and what the business needs and this relationship dynamic that you're in to either be in it or to lovingly step out of it.

**Lenny Rachitsky** (01:11:16):
I love how similar this is to just the marriage, all this stuff. This is the same sort of thing you would do.

**Rachel Lockett** (01:11:21):
Totally. I mean a marriage, you're building a life with a partner. So the only difference is a marriage is rooted in sexual attraction and love, and that's not the case always in the co-founder dynamic. But I have worked with couples who are also co-founders, but there should be some element of love for your co-founder. In fact, I think that when you work closely with colleagues and you really are able to see their gifts and enable them, you can't help but love them.

**Lenny Rachitsky** (01:11:52):
That's a big statement. The other takeaway here is that just get coaching. It feels like that's the solution if things are just not working great. There's only so far you can get just talking.

**Rachel Lockett** (01:12:01):
It takes an evolved facilitator, one of the co-founders, being able to hold space for both their frustration and their empathy in a dynamic that is challenging. So outside support is useful. Sometimes it's actually a team member, it's an HR leader. It's the GC who happens to have great people skills. You don't always need a coach, but you need space to be vulnerable, open, and curious. So if you can create that on your own, that's great. I think it's definitely possible.

**Lenny Rachitsky** (01:12:35):
Outside of the co-founder relationship, do you have any just tactical tips for people to improve their interpersonal skills with just team members, anyone they work with, just people that may struggle like, "Man, I have a hard time with this person. I just have a hard time with a lot of people"?

**Rachel Lockett** (01:12:50):
First of all, people when they want to have a conflict or they want to engage in something that's not working, they come in armored and ready to prove their point. It's natural. You've been thinking about this, perseverating over whether you should mention it. You finally get to the point of engaging, and often there's a misguided view that the goal is to convince the other person that what they're doing is wrong. Actually, the goal of any conflict is to create mutual understanding. So when I go in to have a conversation with, let's say my husband who's not doing his share of the parenting, my goal is to help him understand what I'm struggling with so that he can empathize, see clearly what's happening, and perhaps meet my needs in some way. But it's not for me to prove to him how little he's doing in the house because he might have a totally different story about what's happening. 

**Rachel Lockett** (01:13:54):
So I'm going to give you a framework that I like that many of my clients use. It's from Marshall Rosenberg's Nonviolent Communication. It is a book and a framework. So it's four steps. The first step is observations. So my job is to note what is happening factually. For example, I noticed that in the last three sprint planning meetings, you didn't invite me to those conversations or share with me the roadmap. That's an observation. It's a fact. I could take a picture of it, and no one would argue with it. The next step is feelings. So I'm going to express my feelings without blame. So I felt anxious not knowing what was on the roadmap for the week. I felt confused about whether that meeting happened or not because I wasn't included. So this is me sharing my feelings so the other party can empathize and understand what I'm going through without being defensive.

**Rachel Lockett** (01:14:59):
The third step is needs. What are my universal human needs related to this topic? We all have needs. This is not requiring anything of the other person, just helping them understand my needs that are not met. So I have a need for clarity, I have a need for collaboration, I have a need for connection, whatever that is. And lastly, the step is to make a request. Now, in this model, the request is an olive branch to help the other person meet you and see you. It shouldn't be something that's impossible to do. It should be actually something quite small and easy to achieve for the other person to feel successful in connecting to you and understanding you. So in this case, I might make a request. I'd like to ask you next time you have a sprint planning meeting to include me as optional or to send me the roadmap afterwards that you align on. 

**Rachel Lockett** (01:15:54):
Now, the other person doesn't have to meet my request. They might make a counter proposal, but the most important thing of this model in this conversation is that the other person understands what I'm going through and they don't feel reactive so that we can have a mutual conversation about what's going on.

**Lenny Rachitsky** (01:16:12):
Wow. This point about how when you're trying to convince someone of something, when something is going wrong, this point that your goal is not to convince them, that your goal is to have mutual understanding, that just blew my mind, and I think it's going to change my life. Wow.

**Rachel Lockett** (01:16:32):
Lenny, try this with your wife tonight. NVC is a powerful tool, and actually it's very akin to most models that are about connection. The Stanford Business School course that has a T, it's called Touchy Feely that everybody loves.

**Lenny Rachitsky** (01:16:48):
Yeah, we've had Carol on the podcast.

**Rachel Lockett** (01:16:49):
Great. Yeah, so Carol Robbins created this movement. There's lit. A lot of founders go to her model that's for founders.

**Lenny Rachitsky** (01:16:55):
And tech.

**Rachel Lockett** (01:16:56):
And this is all about, they talk about a net that you can talk about your feelings and your reaction, but as soon as you cross the net to blaming someone else or making an assumption, they're going to have a defensive response. But you can be incredibly bold and brave if you stay on your side of the net. So this model helps you do that because it's really about sharing your emotions and your needs and making a request without blame.

**Lenny Rachitsky** (01:17:21):
Yeah. So what I was going to say as you were going through this framework is here's me, here's what I saw, here's what I'm feeling, here's what I need. And then now that you have that in context, here's something I'm asking for versus you did this and you're feeling this and you thought this.

**Rachel Lockett** (01:17:37):
Exactly. It also acknowledges that professionals have feelings. I think that we operate in tech. We're supposed to give all of ourselves, all of our time, all of our energy to this endeavor. And it's purely logical. It's not at all true. It's completely emotional. And if we ignore our feelings, they will bubble up, and we will be unconsciously acting from them.

**Lenny Rachitsky** (01:18:00):
And there's this implicit power here that if the person cares about you and loves you or values your relationship, knowing that this makes them feel bad will make them want to change. It's not like you need to tell them, "Change this thing. Oh, I didn't realize this made you upset or that you have this need. And now that I know that, okay, now I see why this is important to you."

**Rachel Lockett** (01:18:22):
That's exactly right. And sometimes the other person will hear that and have a different story or a different perspective. So they might say, " Okay, I can honor that request," or, "I hear that request, and I hear your feelings, but let me explain what happened for me." And one way you could do that is, are you open to hearing that? So they're able to share their side too. You don't have to just agree with the person's request. As long as you're setting this tone, the other person's more likely to contribute in a way that achieves mutual understanding because once you're vulnerable, they're going to share their vulnerability.

**Lenny Rachitsky** (01:19:00):
Let me remind folks of the framework. I'm going to try using this. I wish it was a handy acronym off. So the framework is share what you've observed, just the facts of what is happening. Just simply, I saw you didn't close the fridge fully. Your feelings of how that made you feel, the needs that your core human need that drives that feeling, I imagine. And then the request you have of the person.

**Rachel Lockett** (01:19:28):
Yes. And I want to make one note I forgot to say, which is feelings are emotions. So sometimes people say things like, "I feel like you're being a jerk." That's not a feeling, obviously. A feeling is a sensation in your body that results in an emotion. So naming a feeling is actually not easy for technical leaders sometimes. I want to make that point because emotions are what get you to the underlying humanity of connection. Emotions are the key to soliciting empathy.

**Lenny Rachitsky** (01:20:05):
Are there phrases that are just examples of non-feelings, like using the word "you" in the way you describe a feeling probably is not a good sign?

**Rachel Lockett** (01:20:14):
Exactly. If you can say, "I feel like... " even if you add like or, "I feel that... " you're probably going to add a fact. It should be an emotion word after I feel.

**Lenny Rachitsky** (01:20:24):
So don't say like, don't say you, don't say that?

**Rachel Lockett** (01:20:27):
Yeah, exactly.

**Lenny Rachitsky** (01:20:29):
Awesome. Along this topic, I chatted with a number of clients that have worked with you over the years, and one of the most common themes that they said you help them with is having difficult conversations. And I think we covered actually much of this in what we just talked about, but I'm curious if there's any other advice you have for helping people have difficult conversations. Let me read a quote from one of your clients.

**Rachel Lockett** (01:20:52):
Oh, wow. I love this.

**Lenny Rachitsky** (01:20:54):
So she said, "Rachel is exceptional at making difficult decisions clear and making it feel possible to get these decisions actualized." Is there anything more-

**Lenny Rachitsky** (01:21:00):
... these decisions actualized. Is there anything more there for, because difficult conversations are difficult. How do we help people make them less difficult? Any tips?

**Rachel Lockett** (01:21:11):
Yeah. Well, first of all, difficult conversations makes you want to run away. The marketing on conflict is poor, so I want to reframe that. My belief is when we feel internally ambivalent, we have two inner parts at war. And there's something really beautiful and important to pay attention to, there's something to learn, when we have ambivalence. When we are in conflict, something important is at stake. We care deeply about what we're building, about the person that's letting us down. So the reason it's hard, is because there's such an emotional component to it. And there's something to learn from it.

**Rachel Lockett** (01:21:57):
So first, I want leaders who are listening to think, "This is hard, because I have something to learn here, and because it matters. So instead of avoiding it and running away, I'm going to lean into this moment. And I'm going to come out of it not just having solved this dynamic, and not just having said my piece, but having built a skill." The reason I focus on interpersonal dynamics is because the quality of our relationships determines the quality of our life. I really believe that. And if you cannot have conflict, you can't have healthy relationships. We are going to disagree with the people we love, or care about, or are building a business with. So first, I just want listeners to reframe ambivalence and interpersonal challenge, think of them as a growth opportunity.

**Rachel Lockett** (01:22:56):
Second, there is always something that we're doing to contribute to the conflict, even if it feels like the other person is insane, and is driving us crazy, and we're the innocent party. So entering any conflict conversation with humility, and curiosity about the other person's experience, is critical to setting the table for a commitment to come out better and stronger. So no model, NVC or otherwise, can fix a person who's coming in rigid and full of blame. I really love the 15 Commitments to Conscious Leaders, I don't know if you know that book, but one of the concepts is about taking a hundred percent responsibility. Not being in the world of blame, being a victim, or being a hero. And I see many leaders, when they're in a challenging interpersonal conflict, being in victimhood, being in blame, or being in hero. "I'm just going to do it for them, and forget it. They're having such a hard time getting this done, I'm just going to do it."

**Rachel Lockett** (01:24:10):
Instead, take responsibility for your part. "What is my piece in making this dynamic happen and how can I address it?"

**Lenny Rachitsky** (01:24:18):
That makes me think about, Jerry Colonna was on the podcast, and he has this famous line that I've always remembered. How are you complicit in creating the conditions that you claim you don't want?

**Rachel Lockett** (01:24:29):
Yes, I love that. Love that question.

**Lenny Rachitsky** (01:24:32):
And there's so many, there's three parts to that whole question, I won't get into it. But what you're sharing here is, think about that, figure out how, because your point is, you're always somehow complicit in creating the issue you're complaining about. And use that to help kind of put down the defense of the person like, "Here's what I've contributed to this problem." Do you use the nonviolent communication framework? And I don't know, is that just a general way of trying to have difficult conversations, or is there not a framework?

**Rachel Lockett** (01:24:58):
Yeah, no, I think that's a great framework for when you want to go interact with someone around something that's not working for you. I think typically a difficult conversation arises because some feelings are coming up for you, and you have a need that's not being met. And so, that's the instigator to know, "Okay, I need to talk to this person. We need to clear this up."

**Rachel Lockett** (01:25:17):
For example, I was working with a CEO whose co-founder was constantly undercutting his decisions, and criticizing him. And there was something happening, where they'd gone from being this great dynamic duo, fundraised, hired a few leaders, and then all of a sudden he was getting daggers thrown at him all the time. And it was exhausting, and frustrating, and confusing. That was a time where he used NVC to address, "What is happening, here?" And it turned out that the co-founder was really frustrated with how he was spending his time. He didn't want him to be off selling, he wanted him to be helping him with product vision. And they had a totally different conception of how the CEO should be spending his time.

**Lenny Rachitsky** (01:26:02):
Awesome. Just remind folks of the NVC framework, because this is the thing that's hard in the moment. Like, "Oh, what should I be saying?" Observe, feelings, needs, request.

**Rachel Lockett** (01:26:11):
Yeah, exactly. And there's a nonviolent communication book, if folks are into the framework, and want to check it out.

**Lenny Rachitsky** (01:26:17):
People need a little... Who was it, you said one of your client's tattooed the vision he had, on-

**Rachel Lockett** (01:26:22):
Yeah.

**Lenny Rachitsky** (01:26:22):
Okay, there's, let's just get something.

**Rachel Lockett** (01:26:24):
Maybe don't tattoo NVC.

**Lenny Rachitsky** (01:26:25):
Because that may be [inaudible 01:26:25]

**Rachel Lockett** (01:26:27):
It doesn't have a good acronym. You could just print it out, and put it right next to your screen, or something-

**Lenny Rachitsky** (01:26:27):
All right, okay.

**Rachel Lockett** (01:26:30):
... if you want.

**Lenny Rachitsky** (01:26:31):
No tattoos. I just want to highlight the first point you made in this answer, of having difficult conversations. And then I have one more question for you. Just this point about, if there's something you're afraid of, that is a sign you should do that. There's a quote I often think of. "The cave you fear contains the treasure you seek." And the advice there is just, the thing you're afraid of is a compass too, the thing you should do. Because there's something important there.

**Rachel Lockett** (01:26:58):
I love that. Yeah. It's like, "What's important here? What do I have to learn here?" Is a question you can ask yourself when you're avoiding something. I often see this in talent management situations. A CEO has an underperforming COO. They're avoiding a conversation, because they keep getting let down, and actually they kind of know deep down, this is not working out. They don't want to face it. It's too much work. They need to just keep plowing forward.

**Rachel Lockett** (01:27:27):
And when we really take space to think about their feelings and needs, they realize, I ask them, "Would you enthusiastically rehire this person for the same role?" Which is the question we always asked at Stripe. And when the answer is no to that, no matter how many difficult conversations you have, this is not going to work. So then you have to take action. And even engaging in the hard conversation, and seeing what happens, can lead you to the clarity that you need to take action on talent that's not working.

**Lenny Rachitsky** (01:28:01):
That is a really cool tip. I did not know Stripe operated that way. We had the CTO of Netflix on the podcast, Elizabeth Stone, and this is very much how they operate. They're always asking a question like that. The way you phrased it was, "Would I enthusiastically rehire this person for the same role?"

**Rachel Lockett** (01:28:18):
Exactly. It's very clarifying, because it's binary. People have a physical sense, just like we talked about a full body yes, before? You have a immediate reaction that is honest, to that question, that provides clarity.

**Lenny Rachitsky** (01:28:33):
And the answer isn't, if it's no, it's not, "Fire them." It's, "You need to do something about it." It could be talk to them about it, put them on a performance plan, put them in a different role. It doesn't mean you have to fire them immediately, so it's not necessarily as scary as it sounds, if you say no.

**Rachel Lockett** (01:28:47):
Yeah, I think that also it depends on the stage of business you're in. So I see a lot of companies build a leadership team, and then a year later, the size and stage of their business is dramatically different. And they start to realize, "Oh, the CFO that was really fine back then, is now completely wrong. He should be the controller." Okay, great. So reckon with that. Recognize that in how you're interacting with your current CFO, put out a search. There's many things you can do that aren't firing someone. But in quickly scaling businesses, it's natural that the leadership team's job will change, and that you'll have to make some evolution over time.

**Lenny Rachitsky** (01:29:28):
And I guess it's very important to highlight the importance of operating this way, if you're trying to build a really successful company, is that should be the bar. Is, if you would not enthusiastically rehire this person for this role. If you're trying to build something that's never been built before, and build a company that actually works out really well, you need to really only have people around that are hitting that bar.

**Rachel Lockett** (01:29:50):
Yeah. My perspective, I talked at the beginning about how I'm obsessed with the human side of business building, and my belief is that talent and the environment that you put your talent in is everything. Yes. Building a product and a business is about building something that users need. It's about product market fit, and then the wave you're on. Timing is important. You're going to build a different size business, if you're in a sector that's not growing, than right now, if you're in the middle of AI. True. You're riding a timing wave, and you're solving a core need. But everything besides that is so human. It's about talent, and it's about the environment that you put that talent in. So you need to create the conditions such that your talent can thrive.

**Lenny Rachitsky** (01:30:37):
Such a simple concept, that I think people overlook, is just everything you do is going to be the people that you have around you, and the environment you create for them to operate. I think your point about when you're doing something difficult, just to close out this element, I love this idea that if it feels hard, think of it as a learning opportunity. I think anyone listening to this is like, "Oh, cool, I'm going to learn something. I'm going to get better." It's such a easier, more motivating way of approaching something that's difficult.

**Rachel Lockett** (01:31:05):
And I want to make a distinction between that and what we talked about earlier, which was, lean into your strengths. Because I don't believe people should suffer through the day grinding, doing work that's depleting. That's not a learning opportunity. Interpersonally, when you're avoiding something, it's because you care about something. Avoiding your emotions is what I want to encourage people against. We have to feel our feelings all the way through, be present to our feelings, and interact with others in a way that acknowledges our feelings. That's what I want to encourage, because actually that's not deadening, that's enlivening. And there's learning there.

**Lenny Rachitsky** (01:31:48):
A final area I want to spend a little time on is, something that I've heard from everybody that you work with, which is the way that you help them operate. So you just talked about the importance of the people you hire and the environment you create for them. And something that you help leaders do is create a very specific way of operating around a one-page plan, and how that kind of trickles down and just makes everything at a company more effective. Talk about this one-page plan, how you recommend companies operate with this.

**Rachel Lockett** (01:32:17):
Yeah, thanks for asking that question. I think, typically, companies have complicated the process of aligning their vision, their strategy, their goals, and the way people behave with each other, their values. So that all of these things live in different places, are talked about to a different degree, resonate to employees differently. And if you asked anyone at the company, "What are your top three priorities, and how do they relate to the vision?" It's not an easy answer.

**Rachel Lockett** (01:32:50):
So the reason I like the one-page plan concept is, it's simplifying. It's a way for the leadership team to come together and align around, " What are we doing here? What is our role in it? And how do we communicate it, so that the whole company has clarity, and knows how the work they're doing ladders up to our big picture vision that we're all committed to?"

**Rachel Lockett** (01:33:14):
So I actually got this idea of the one-page plan from Alpine Investors. They have something called the People First Operating Rhythm, and they've successfully implemented that at their portfolio companies. And I work in concert with Alpine, so I work with some of their portfolio CEOs. To execute this rhythm. So it's not just about a one-page plan. It puts your vision and your values on the first column, your strategic intentions and your KPIs on the second column, your annual goals on the third, and your quarterly goals on the fourth. So that no matter what you're talking about, in terms of, "What are we doing for the next year, or the next quarter? How do we prioritize?" It's always in tandem with your core KPIs, your strategy, and your vision.

**Rachel Lockett** (01:34:04):
And I love how they instituted that with their portfolio, and I saw the power of it. They've collected some data that their portfolio companies that actually institute the People First Operating Rhythm result in higher returns. So they're very committed to this strategy, and after operating with CEOs in their rhythm, I took some of those ideas and started to help other founders and other leaders with some of the same concepts, in my own way.

**Lenny Rachitsky** (01:34:30):
We're going to hopefully link to a template of this one-page plan?

**Rachel Lockett** (01:34:33):
Yeah, sure.

**Lenny Rachitsky** (01:34:33):
Okay, cool.

**Rachel Lockett** (01:34:33):
Happy to share.

**Lenny Rachitsky** (01:34:34):
Okay, so let's do that. And then, what kind of impact do you see from companies starting to operate this way to motivate people to do this?

**Rachel Lockett** (01:34:41):
Yeah. What I see is clarity and alignment. And I also see more connection. So I want to name that it's not just about having a plan, it's about how you create it, how you reflect on it, and how you come together around it to celebrate wins.

**Rachel Lockett** (01:34:57):
So in my opinion, a very under- attended to part of building a business is an operating rhythm. When do you come together to kick off the year, and share your strategy and vision again, and talk about the goals? When do you come together to reflect on what's working and not working, and how do you do that? And in what groups? And are you honest, or are you just kind of doing it as a quick exercise to move on to what's pressing?

**Rachel Lockett** (01:35:25):
So just like I said in co-founder dynamics, a key is to step out of the dance floor and to get onto the balcony. Executive teams leading a complex business need time away from being in the business to work on the business. So around this one-page plan, the reason I like a rhythm, is you can kick off the year with the plan. That's really simple, easy to understand. Everyone can have it accessible and every quarter, you can get together to reflect, "What worked, what didn't work?"

**Rachel Lockett** (01:35:56):
I really like the question, "What's an inconvenient truth?" Air the things that need to be talked about, that no one's talking about because you're too busy. That's the power of combining a simple plan, whether it's one page or not, that aligns you from the top to the bottom, your vision all the way down to your quarterly goals. And a time where you stop, pause, discuss, reflect, have a little spacious energy.

**Rachel Lockett** (01:36:26):
It's not unlike what you said about your own time. You are the executive team. You want a little bit of spacious time to tinker, reflect, create, and come back to the meaningful work you're doing more energized. And leadership teams need that too.

**Lenny Rachitsky** (01:36:45):
Wait, Alpine Investors, Graham Weaver. He was on the podcast.

**Rachel Lockett** (01:36:45):
I saw that. I saw that.

**Lenny Rachitsky** (01:36:49):
I love that. Okay, final, final question. I want to take us to AI corner before we get to the very exciting lighting round. I'm going to do kind of a tweaked version. Usually I ask people just, how has AI impacted their work and life? I guess that is the question here, just how has AI changed, I guess coaching, as a coach? But also just, from a client's perspective, how are people using AI to help them in their, I guess life, from a coaching perspective?

**Rachel Lockett** (01:37:13):
Yeah, it's a great question. So as a coach, I use AI in a couple of key ways, that I'm grateful for. One, I use Granola, which I saw you give away to your listeners.

**Lenny Rachitsky** (01:37:23):
One free year of Granola, for becoming an annual subscriber of Lenny's Newsletter, lennysnewsletter.com.

**Rachel Lockett** (01:37:29):
There you go.

**Rachel Lockett** (01:37:29):
So I use Granola to take notes in our session so I can be fully present with my clients, and I can give them a synthesis of what happened and the next steps they committed to after our session. I also use it, I put them in a folder for every client, and so I can look at insights across our work together. What are the deeper things that are happening? What are the patterns that are taking place? I have these in my head, but actually it's a great tool, to see over time. "Oh yeah, we talked about that in our first session. Let's bring that back, because that's what you're struggling with now." So it helps me create the kind of transformation that I want for all my clients.

**Rachel Lockett** (01:38:08):
Secondly, I just use ChatGPT to help me plan my retreats. I run a women's organization, and we have eight retreats a year, and it's a great tool to think expansively about new activities. Once I've gotten the core objectives down, and I have a bunch of ideas about what I want to do, it gives me new creative ideas. So I can put in like, "Here's my objective, here's my goal, here's my audience, here's my last retreat that I ran. I kind of want three new ideas for this session." So it'll give me creative energy that I otherwise would need to get together with other coaches to discuss. And I do that, too.

**Rachel Lockett** (01:38:50):
Finally, I'm experimenting with AI in a way to support my clients between sessions. So I've gotten some feedback from my clients that they would like more interaction between our sessions, and they're always allowed to email me, or text me. I'm available to them. But I think they want to be really respectful of my time, and so some people do reach out and ask me questions, and other people wait for our session.

**Rachel Lockett** (01:39:17):
So I'm curious about the future of coaching, how in between sessions, clients can get access to more of an AI support, where the bot has all of their context, their development plan that we create at the beginning. So that's their goals, for our work together, how they want to grow. Some of my core frameworks, and my beliefs, and my training. And the Granola notes from all of our sessions, so that they can access between, just some extra spot support. They're going into this conversation, how should they approach it? They're anxious about this team meeting. How can they make the most of it? More tactical support. I see personal coaching as still critical for, "What is your vision of your life? How do you want to shift your core behavior to align with that vision?" But then, AI can play a real helpful role in between, on the tactics.

**Lenny Rachitsky** (01:40:15):
That is super cool. So that's something you already do, where they have access to this kind of GPT-

**Rachel Lockett** (01:40:19):
It's something I'm building right now.

**Lenny Rachitsky** (01:40:20):
You're building, that is-

**Rachel Lockett** (01:40:21):
My clients don't have that yet.

**Lenny Rachitsky** (01:40:22):
Okay. That is great. That is a really good idea. It's not replacing coaching and therapists, let's say, but it's adding a lot more in-between time where you could just talk to us, based on everything you've talked about, all the frameworks that you use. That is extremely cool. All right. There's a billion-dollar company coming.

**Rachel Lockett** (01:40:39):
I don't want to build that.

**Lenny Rachitsky** (01:40:43):
It's not your zone of genius.

**Rachel Lockett** (01:40:44):
Exactly.

**Lenny Rachitsky** (01:40:47):
Rachel, is there anything else that you want to share or leave listeners with, before we get to our very exciting lightning round?

**Rachel Lockett** (01:40:55):
What I want to share is that the world is getting more lonely. There's a lot of research on this, but it's also obvious in my coaching sessions, that people feel more alienated from one another. And actually, building businesses is an inherently human endeavor. So I am a fan of this AI boom, I appreciate that we have more technology at our fingertips than ever before. But I want to encourage listeners to think of themselves as leaders who bring humans together to self-actualize, and that they have to actively overcome the default state, which is blind, grind, and loneliness. So I think this is a call to action for your listeners, to connect with the people around them, lead healthier teams, create environments where connection is inevitable. And that they will have more fun, and build better businesses, because of that.

**Lenny Rachitsky** (01:42:03):
What a beautiful way to end it. With that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?

**Rachel Lockett** (01:42:10):
I am ready.

**Lenny Rachitsky** (01:42:11):
First question, what are two or three books that you find yourself recommending most to other people?

**Rachel Lockett** (01:42:16):
One of them I said before, The 15 Commitments of Conscious Leaders. And I love Designing Your Life, by Bill Burnett.

**Lenny Rachitsky** (01:42:25):
I love that book, too. People don't talk about that book enough. Next question. Favorite recent movie or TV show you really enjoyed?

**Rachel Lockett** (01:42:31):
Oh, God. I just went to KPop Demon Hunters with my daughter. It's so embarrassing, but that's what we dressed as for Halloween, like everyone else in the world.

**Lenny Rachitsky** (01:42:38):
I have not seen that. I hear everyone talking about it. I am going to try to avoid it, I think. Next question. Favorite product you have recently discovered that you really love? Could be an app, could be a gadget, could be clothes.

**Rachel Lockett** (01:42:48):
I really love Loom.

**Lenny Rachitsky** (01:42:50):
Amazing.

**Rachel Lockett** (01:42:51):
I've been recording trainings on Loom for some of my clients that are, it's a scaled holding company, so I'm able to scale training in a really human, connected way.

**Lenny Rachitsky** (01:43:02):
Do you have a favorite life motto that you often come back to, in work or in life?

**Rachel Lockett** (01:43:06):
I have a quote that is on my desk. And I love it. Ready? "If you can see your path laid out in front of you, step by step, it's not your path. Your own path, you make with every step you take. That's why it's your path." That's a Joseph Campbell quote.

**Lenny Rachitsky** (01:43:27):
Beautiful. Final question. You've got two kids, you said. Do you have any favorite children's books that you most love reading to them, that they've loved most?

**Rachel Lockett** (01:43:36):
Oh my gosh. So my daughter is really into Roald Dahl. I love Roald Dahl, because he's completely irreverent, and he has a crazy imagination. So we've been reading Witches, Matilda, all of his books. And both my kids love it. So, they're five and seven.

**Lenny Rachitsky** (01:43:54):
Have you seen the Wes Anderson stories of his stories, where he takes [inaudible 01:43:58]-

**Rachel Lockett** (01:43:58):
Some of them, yeah. They're great.

**Lenny Rachitsky** (01:44:00):
Yeah, they're so amazing. Oh my God. And it's like Roald Dahl is like, I think it's personifying him. He's like a character in the story.

**Rachel Lockett** (01:44:06):
Yeah, he's a character, from what I hear about his life.

**Lenny Rachitsky** (01:44:09):
Rachel, this was incredible. I feel like we've very much accomplished what I set out to do, which is just give people all this advice that they never have access to that, costs tens of thousands of dollars. I think we're going to help a lot of people improve their lives and their careers. Thank you so much for being here.

**Rachel Lockett** (01:44:26):
Absolutely. Thanks for having me.

**Lenny Rachitsky** (01:44:28):
I almost forgot to ask you two final questions. Where can folks find you if they want to reach out, maybe consider working with you? And how can listeners be useful to you?

**Rachel Lockett** (01:44:36):
Yeah, find me at lockettcoaching.com, and how can listeners be useful? Listeners should turn towards each other, build great relationships, and send CEOs and co-founders my way if they need coaching.

**Lenny Rachitsky** (01:44:48):
Thank you so much for being here.

**Rachel Lockett** (01:44:50):
Thanks for having me. Take care, Lenny.

**Lenny Rachitsky** (01:44:52):
Bye, everyone.

**Lenny Rachitsky** (01:44:55):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating, or leaving a review, as that really helps other listeners find the podcast.

**Lenny Rachitsky** (01:45:09):
You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [13/17] What world-class GTM looks like in 2026 | Jeanne DeWitt Grosser (Vercel, Stripe, Google)

**Lenny Rachitsky** (00:00:00):
I've been getting so many asks for go-to-market help.

**Jeanne DeWitt Grosser** (00:00:03):
With AI, it's just intensified because you have 10 players pursuing the same market opportunity and so your ability to actually bring the product to market to differentiate yourself from the competition has become more strategically important than it was previously.

**Lenny Rachitsky** (00:00:18):
I had Jenna Abel on the podcast recently, one of her tips is you don't want to be focusing on here's the pain and problem we're solving and instead focus on here's how you will be better than your competitors.

**Jeanne DeWitt Grosser** (00:00:27):
80% of customers buy to avoid pain or reduce risk as opposed to increased upside, which is a good thing for startup founders to understand. We all love to talk about the art of the possible, everything we're going to enable in the future, but that's often really a sale that's going to resonate with another founder. For everybody else, particularly enterprises. You're avoiding the risk of not making your revenue target next quarter.

**Lenny Rachitsky** (00:00:52):
I've heard a lot about how you think about go-to-market as a product.

**Jeanne DeWitt Grosser** (00:00:55):
We buy a lot of things because of how we feel about them. The experience that you have of being sold to will increasingly actually differentiate a company and drive buying decisions if products are only different at the merchant. And so then you really want to create a customer buying journey that feels like very unique experiences.

**Lenny Rachitsky** (00:01:17):
Something I've heard from so many people you've worked with is that your superpower is building a sales org that doesn't feel like a sales org to engineers.

**Jeanne DeWitt Grosser** (00:01:23):
The litmus test I have always given my sales team is if you are an account executive in my org and I put you in front of 10 engineers at our company, it should take them 10 minutes to figure out you aren't a product manager.

**Lenny Rachitsky** (00:01:38):
Today my guest is Jeanne Grosser. Jeanne was chief product officer at Stripe where she built their very early sales team from the ground up. She's currently COO at Versel where she oversees marketing, sales, customer success, revenue ops and field engineering. Jeanne has built world-class go-to-market teams at multiple unicorns and has advised dozens of companies on doing the same. In our conversation, we go deep on what a world-class go-to-market team looks like, including what the heck is go-to-market, the rise of the go-to-market engineer and how this role is already enabling her team to operate 10 times faster. A bunch of very specific tactics to level up your go-to-market skills, a primer on segmentation, how to think about your go-to-market process like a product, her favorite go-to-market tools, her hot takes on PLG and sales comp and sales hiring, and so much more. If you are looking to get smart on the latest and greatest in go-to-market thinking, this episode is for you.

**Jeanne DeWitt Grosser** (00:05:32):
Thanks for having me. Lenny.

**Lenny Rachitsky** (00:05:34):
What I wanted to get out of this conversation by the end of this to basically have this conversation be the thing that we send people when they're like, "I want to get better go to market. I'm trying to figure out what to do and get to market." We send them this versus having to hire someone for a lot of money and usually they can't find amazing people, because they're all snatched up. So let me start with just the basics. When people hear at the term go to market, what does that mean? What does that encompass?

**Jeanne DeWitt Grosser** (00:05:57):
I think there are two answers to this. Often what people think of is sort of the tip of the spear of what drives revenue, which is marketing and sales. For me, I think of it as any function that is going to touch a customer or make a dollar, and actually my remit at Vercel is that, so that includes marketing, sales, all of your technical sales roles like sales engineers or post-sales platform architects is what we call them at Vercel. It's customer success, it's support, it's partnerships. And the reason I say that is my experience throughout my career has been that those functions often have this Venn diagram strategy where marketing's pursuing one thing, it overlaps with what sales is pursuing, but not perfectly, which also overlaps with what support is pursuing but not perfectly. Examples of this would be slightly differing segmentation frameworks, et cetera.

**Jeanne DeWitt Grosser** (00:06:57):
And so one of the things I think you're going to want to see more in this particular moment is that that become a really integrated lifecycle. In particular, I think we're going to see a lot of the functions of go-to-market get redefined, so we've gone through a period of hyper-specialization in go-to-market depending on how you count them. There are, I think somebody quoted 17 different roles within go-to-market these days and I hypothesize that a lot of those are going to start to collapse. And so if you think of go-to-market more holistically, I think you can kind of go back to what are the jobs to be done from making a customer prospect aware of your product all the way through to high LTV, five years on the platform, fully wall-to-wall, and you're going to want to map that out and orchestrate it the way you would think about that within your own product.

**Lenny Rachitsky** (00:07:54):
Awesome. We're going to go through that whole cycle of go-to-market, but so is it safe to say just for most companies that are especially starting out when they say go-to-market, that mostly is sales and then there's marketing as maybe a smaller fraction of that and then as you become more advanced and grow, customer success plays into it, tech sales, things like that?

**Jeanne DeWitt Grosser** (00:08:12):
Yeah, that's probably where most start is getting sales or frankly just because a lot of companies also start PLG, you might actually start with marketing and then you're layering in sales when it's time to do the sales assistant and ultimately sales led portions. So I think it can, depending on your product and your initial target market, it can either mean marketing or sales or a combination of those two.

**Lenny Rachitsky** (00:08:33):
Awesome. So essentially it's like the term go-to-market tells you what we're talking about. How do you take your product to market, get people aware of it, using it, sticking with it?

**Jeanne DeWitt Grosser** (00:08:42):
Yep, absolutely.

**Lenny Rachitsky** (00:08:44):
What has most changed in the world of go-to-market over the last few years? You've done this for a long time at Google, at Stripe, you built it for sales team, now you're doing that at Vercel. What's changed most in the skill and art of go-to-market?

**Jeanne DeWitt Grosser** (00:08:55):
There are a number of things. So when consumption-based business models started, I think you saw go-to-market shift into being meaningfully more consultative because often that first land was the very beginning of the journey and represented a very small percent of what you were ultimately going to do with that customer. And so you had to go from being transactional to a lot more. You had to more deeply understand what that customer was trying to do so you could align that ultimately to your product. I think that has played out that much more with an AI because right now everyone knows they need to change, but they don't necessarily know exactly what they need to change to, whether that's their customer-facing product or their internal productivity and workflows. And so I think you're seeing a lot more of go-to-market orgs leaning into the art of the possible best practices, helping you actually think things through as if they were a consultant.

**Jeanne DeWitt Grosser** (00:09:52):
And so one of the things you see more of right now is forward-deployed engineering, which on some level is kind of a rebrand of professional services but kind of not. And a big part of that is, hey, how do I actually get into your environment, ride alongside you better understand what you're trying to do and then help you actually bring the technology to life and learn a lot along the way.

**Jeanne DeWitt Grosser** (00:10:18):
Often you're not only making that customer successful, but you're then taking all of that back to your product and engineering organization to figure out, okay, what was generalizable that we ought to build into our offering versus what is something that ultimately is going to be more of a professional service in the fullness of time. So I think that has been a biggie, is actually just really getting embedded with your customer. And then unsurprisingly, I think bringing AI to bear on the sales process is another big one. And so you've seen the rise in probably the last 18 to 24 months of the go-to-market engineer, which different folks define slightly differently, but it's kind of bringing one technical prowess to bear on go-to-market in general so you can have a lot better tooling, data use, et cetera. And then two, increasingly bringing AI to bear as well to re-architect your workflows and also make it so that it's easier to have a personalized experience with customers but do so at scale.

**Lenny Rachitsky** (00:11:23):
Amazing. Okay, let's follow the thread on this go-to-market engineer, so what was it like before and what are these engineers doing at companies?

**Jeanne DeWitt Grosser** (00:11:33):
So I think maybe an interesting story to tell. When I was at Stripe, we went to launch an outbound SDR function. So outbound prospecting and Stripe always ran lean. The company at that time had an operating principle which was efficiency is leverage. And so if you looked at the sales organization I was running, most companies out there probably would've had 30 SDRs and I was going to get four. So there's no way I was going to do the typical SDR approach and be successful. And so we thought to ourselves, okay, what can we do? We'll be super data-driven. And so we went and we started building project Rosland. Rosland is the scientist who originally mapped A-DNA. And what this was was effectively a company universe. So you can think of this as a massive database. Every row was a different company on the planet and every column was an attribute about that company that would help you sell to them in a more targeted fashion.

**Jeanne DeWitt Grosser** (00:12:39):
So at Stripe an example would be knowing that their business model was a marketplace was super helpful, because that would mean you wanted to sell Stripe Connect versus vanilla payments. And so the goal was basically, hey, can we create a mad Libs where I will come up with sort of a predefined email template, but 80% of it will be fill in the blank based on the different attributes of that customer. So if they're this industry or this business model, then pull this customer, reference this value prop, send it to this persona, not that. And we were trying to do this in 2017 and it was very hard and didn't actually totally work our ability to the false positive rate and we worked deeply with DSI and it just never really got there. And now that we're literally redoing here at Vercel as we speak and it actually works and you can bring AI to bear on it.

**Jeanne DeWitt Grosser** (00:13:41):
And so what's different is we now, I have a data scientist just like I did back in 2017, but I have a go-to-market engineer whereas before I just had someone in systems that was helping me configure outreach or sales off and my go-to-market engineer is helping me build an agent where we're coming up with, okay, well what's the human workflow that you would've done? And then how do you encode that using Vercel workflows as an example in actual code that's both deterministic and less so where an agent's going out and trying to replicate what a human might've done to produce that, fill in the blank, matlit.

**Lenny Rachitsky** (00:14:21):
I love the ambition of that project. What is this, like eight years ago?

**Jeanne DeWitt Grosser** (00:14:25):
Yes.

**Lenny Rachitsky** (00:14:26):
I love the big thinking there. We're going to map the entire universe of companies and then here's how we sell to them. And then just I'm trying to picture doing that without AI. It's like crazy to imagine trying that without AI and that's so much simpler to even imagine.

**Jeanne DeWitt Grosser** (00:14:38):
Well the thing that's amazing about that, just to geek out on a second, so I was working on that with a bunch of folks at Stripe on my team, obviously at a gentleman named Ben Salzman who went on to go to ZoomInfo and then actually recently just founded a go-to-market startup that is basically sort of productizing that concept of a company universe and then layering AI on it on top of it. And ultimately his view is actually AI will get to the point that you won't have to do outbound prospecting because it will just sort of company and product match. So it's fun to sort of see back in 2017 some of the folks doing that now work at OpenAI, they work at Anthropic, they also are doing GTM Eng. You've got him starting a totally AI native GTM company and then here I'm at Vercel trying to do the same.

**Lenny Rachitsky** (00:15:29):
Okay, so what's cool is this is an emerging role, an emerging skill that I don't think a lot of people have recognized as something that is happening. So one example I'm hearing of what this role does is they automate outbound emails essentially and outbound outreach. They figure out, they write workflows and agents that figure out here's the company to go after, here's how we message them. Does that end up being kind of like an email that's custom designed and written for this prospect?

**Jeanne DeWitt Grosser** (00:15:54):
That's one version. So it's broader than that really. Basically the full remit of GTM Eng will be to go through each of the different functions within go to market and break down all the different workflows that they do and then turn those into agents where AI is better placed than the human to do that task. So right now we started with actually inbound and are now moving to outbound because that workflow is most legible. And by legible I mean you can basically write it down. It's relatively replicable, mostly deterministic. So it's more likely that AI will do it well and we actually built the agent and then we keep a human in the loop. But from there we're starting to look at outbound and with an outbound we're starting more at the lower end of the market, where you tend to have slightly less customization because there's a single decision maker at the company.

**Jeanne DeWitt Grosser** (00:16:56):
But I think it'll take a while before we're able to really do that in a very large enterprise. There we might use an agent for research but maybe not all the way to actually send a message and that's just within the prospecting function. So other places that we're looking at this would be for install-based sales. So again there it's a little bit more deterministic because you've got awesome internal data on what a customer is and isn't using, what's the next best action? What's the thing they should get most value from? So that's where we're starting to map, hey, what does that ideal workflow look like? But basically you want to get to a state where as long as I've been in sales, they release these annual reports that help us all benchmark ourselves relative to one another. And one of the stats is what percent of time do your sellers actually spend in front of customers?

**Jeanne DeWitt Grosser** (00:17:44):
And for the 20 years I've been in sales, it's always been somewhere around 30% to 40%. So the minority of time is actually talking to other humans and I think we're getting to a point where with layering in agents, ideally we finally get salespeople to a point where they're actually spending 70% of their time interacting with humans and we can get the research, the follow-up, the things that are a little bit more rote and don't use the entirety of your human capacity done by an agent and then sort of unleash you to go deeper with your customers.

**Lenny Rachitsky** (00:18:17):
I love that this is such a great example of where AI is contributing in a very meaningful high ROI way, taking on all this work that people... like, you have to hire say 50 SDRs as you described to do and now you could do with a lot more. So it's a really cool example of leverage that AI gives you. One thing that I know a lot of people think about when they hear this is, okay, I'm going to get more of these really bad emails trying to pitch me on stuff and just like this isn't going to work. I can tell this is AI. What have you learned about how to do this where people actually receive emails that actually convert and do well?

**Jeanne DeWitt Grosser** (00:18:51):
Our processes all always have human in the loop. And so basically where we'll start is we take a go to market engineer and we have them shadow the highest performing individual in that function. And so you can go and you shadow an SDR and you can see, oh wow, they've got seven tabs open. They're looking up the person on LinkedIn, they're reading about the company, they're doing chatGPT on this, they're looking in this database to get these sets of attributes. And so that's how you sort of inform the initial workflow. And then what we do is we let the agent make a call. So in the specific example with inbound, you have to determine whether or not you think the lead is likely to be qualified and then you have to determine what to say to it. And so we'll let the agent make those two calls.

**Jeanne DeWitt Grosser** (00:19:44):
It ultimately then does some deep research, pulls in a bunch of information from our databases and crafts a response, but we have a human review all of those and actually hit send. Now for us, we had 10 SDRs doing this inbound workflow and now we just have one that is effectively QA-ing the agent. The other nine we deployed on outbound, so we got to move them up the value chain. At some point I think we'll get to a place where we feel like, "Hey, the human reviewer is saying yes enough of the time that we feel confident that these will be on brand targeted, et cetera," but right now we're still trying to train the agent and it incorporates feedback on what we choose to reject, edit, et cetera.

**Lenny Rachitsky** (00:20:31):
And you shared that it's already having a lot of impact. Like you said, you had 10 SDRs and now one can do the job of 10.

**Jeanne DeWitt Grosser** (00:20:39):
So before we did that move, I mean the other thing that's just incredible about this is the person who built the lead agent was a single GTM engineer. He spent maybe 25-30% of his time on this. It was six weeks before we felt confident going from 10 to one. So it wasn't like this was a multi-quarter process, it actually moved super quickly and then again now we just sort of keep that agent manager working with the agent to get it to a point where we say, "Hey, we're ready to roll." Actually throughout the process we also tracked all of the KPIs that you typically would hold an SDR accountable to. We were looking at our lead to opportunity conversion rate, we're looking at the number of touches it takes the time to convert, and basically what we were able to do is hold that lead to opportunity conversion rate flat. So the agent is as good as our humans were, but it's actually condensed the number of touches it takes to convert because it's so much quicker at responding relative to leads inevitably sitting in the queue or coming in at nighttime and no one can get to it, that type of deal. So that's sort of when we knew it was ready to pull nine people off and shift them into outbound.

**Lenny Rachitsky** (00:21:56):
That's incredible. Okay, that's interesting. So you shift them to outbound. What I love about this is this SDR that is now doing this is, as you said, doing the things they enjoy more, they're talking to customers more, they're not doing all this kind of top of funnel rote work. I don't want to get into whole jobs AI discussion, but there's always been this talk about AI SDRs basically replacing SDRs. It feels like that's one thing where everyone's like this is a hundred percent going to be AI in the future. What I'm hearing here is it gives one Aster a lot more leverage and obviously you still need people running the show. Thoughts there? Just like do you think AI will replace all this at some point? And then I don't know, you don't need salespeople?

**Jeanne DeWitt Grosser** (00:22:32):
I think on prospecting it can replace a fair amount because the average SDR wasn't doing overly sophisticated research in the first place. So where I, think the last part to go as I mentioned will be in deep enterprise prospecting where you can be at multiple layers in an org chart, you've got to pick between business lines, you've got to triangulate those. But I do think for the things that are more repetitive that often don't take that much time to learn and get ramped, AI will be good at that. And in my view, no one graduated from college and was like, "Yes, I just went to college for four years to become an SDR." It was more, "Okay, that's where you are forced to start." But I think the average SDR could have gone straight into outbound or straight into an SMB closing role. And so basically what we're just doing is shifting folks into something that uses more of their full capacity right out of the gates rather than sort of the forcing function of working your way up the totem pole.

**Lenny Rachitsky** (00:23:48):
Awesome. Since a lot of people listening to this aren't salespeople don't have a lot of background in sales, we've used this term SDR, there's also the term AE. Can you just help people understand what is an SDR, what do they do, what's an AE, and then what's the role above?

**Jeanne DeWitt Grosser** (00:24:01):
Sure. So SDR is typically in charge of generating pipeline. They're meant to talk to prospective customers and get them to a point where it is worth investing time to run them through a sales process. You typically have two types of an SDR, have an inbound one. So this is where people come to your website, they fill out contact sales, they'll be the first call to make sure that it's actually worth a more expensive account executive to go and run a sales process or you then have outbound. So this is where when you want to grow faster than your inbound demand, they will go out and at this point you probably have a point of view on where you think you have product market fit. And so they will target that part of the market and try to drum up interest from folks who weren't otherwise raising their hand saying, I'd like to talk to you.

**Jeanne DeWitt Grosser** (00:24:54):
So that's sales development basically. Pipeline generation account executives are closers. So it's their job to take somebody from, "Okay, hey, I'm interested in learning about your solution, I have a legitimate problem. I potentially could make a decision," to, "I now believe that your product is the best in the market for me and I'm willing to pay for it." And then account executives, depending on the segments that your company sells into E.G. small business, mid-market enterprise, et cetera, they may work their way up the food chain from selling to a smaller company like an SMB or a startup. Those tend to be a little bit more of a transactional sale. You often have a single decision maker to then going into a mid-market or a commercial role where now maybe you have an economic buyer like somebody in finance and a technical buyer like somebody in engineering to getting into enterprise where you have procurement and you have committees and 10 people have to weigh in and you've got to help them figure out how to de-risk the fact that they're probably migrating from something so much more complicated coordination effort to sell.

**Lenny Rachitsky** (00:26:05):
That was extremely helpful. So SDR, pipeline generation, i.e., closer. Such a simple way of thinking about it. Okay, this is great. Going back to the GDM engineer, a few questions for people that may want to try this at their company, what scale do you think it makes sense to start hiring for this role? Having someone automate in the go-to-market process?

**Jeanne DeWitt Grosser** (00:26:25):
What's interesting about this is it will force companies to be more rigorous about their sales process early. So often startups when they go from founder led sales to say, I'm going to have my first sales person, whether that's an actual account executive who has sales experience or your general athlete, wicked smart, who's going to go figure it out. Often founders will just say, "Okay, sales is showing up and talking to people. Isn't that what I just did for the last couple of years?" But actually sales is more than that. It's a skill just like writing code as a skill or building a financial model as a skill, it's about discovery. So asking all the right questions that help you identify challenges in pain, willingness to pay, et cetera, and then going through a process to handle those objections and showcase where are you at enough value such that somebody ultimately wants to hand over some money.

**Jeanne DeWitt Grosser** (00:27:24):
So often startups will get, particularly ones with strong product market fit to pretty significant scale without really having a replicable process. And you can't really apply go to market engineering unless you actually have a point of view on what best practice should look like. And so I think basically this is going to force folks to have more of a playbook out of the gates, what's working, what's not? Can I document it? Do I have content for the different parts of the sales process? And then once you do that, which maybe 10 people is a good size and scale for that, ostensibly a GTM engineer can come in and turn that into an agent. You could also argue that if you're a founder who wants to bring in a general athlete profile and that person is technically minded, that you could have a hybrid AE GTM engineer who figures out what their best practice is and then tries to turn that into an agent that's riding alongside them and making them more effective as well.

**Jeanne DeWitt Grosser** (00:28:26):
So I don't know that I have a point of view yet on what's the optimal size and scale, but I forever have given founders the advice that you often want to bring in revenue operations, which is basically the analytical arm of sales earlier than you think because having data, having process is actually what gives you insights as a founder into what is and isn't working. And so I would argue just like it's a good idea to have that sooner than later, increasingly it'll probably be a good idea to have GTM engine and be looking to bring agents to bear on your process at the outset.

**Lenny Rachitsky** (00:29:05):
While we're on this topic, just a quick tangent, the advice for hiring your first salesperson that I usually hear is wait until you're around a million in ARR. When you have a repeatable process, you can teach someone anything there. Does that seem right? What would you recommend?

**Jeanne DeWitt Grosser** (00:29:18):
Yeah, I think that seems about right. I do think as a founder you want to stay deeply connected to customers and get it to a scale and get it to a point where you use the word, there's some repeatability there. I think that's one of the things that not all founders get right is founders are incredible salespeople. They convinced a VC angel investors to fork over a bunch of money, so clearly they're going to inspire people to buy. But if you're getting to a million in ARR and the set of customers you have look nothing like one another, you still have very much like an evangelist sale, very much founder led sale versus if you can say, "Hey, I now have an ICP here, or ideal customer profile, e.g something you can write down. We are good. Our product fits with startups with less than a hundred employees who are typically building SaaS applications," something like that.

**Jeanne DeWitt Grosser** (00:30:14):
Then you're probably ready to hand over the reins. And then what founders have to remember is to actually hand over the reins. So you've got to enable the person who comes in, what is it that you're doing effectively, what's your content, what are the discovery questions you are asking? How are you handling objections so you can transition that knowledge but also don't handle them over entirely. You want to stay connected to the customer because you still have a fair amount of R&D to do to figure out where is the product next going to resonate, where are you getting stock as you scale, etc.

**Lenny Rachitsky** (00:30:52):
To close the loop on the go-to-market engineer, what's the profile of the ideal go-to-market engineer, may be your first.

**Jeanne DeWitt Grosser** (00:30:57):
What we have found works really well is somebody who does have go-to-market experience. So at Vercel, our first three go-to-market engineers we're actually sales engineers. So Vercel hires very technical sales engineers, all of them were front end developers before they decided they wanted to get into sales. And so we just said, "Hey, three of you, congrats." You're now founding members of our GTM Eng team. And the thing that works well there is you do understand aspects of what is good GTM, what does a process look like? It's been really interesting actually. So the gentleman who runs GTM Eng for me, we were going through this lead agent and QA-ing it. And so I'm going and I'm looking at some of the responses that we've ultimately had the lead agent send and realized, "Oh, I wouldn't have sent that and that's because I have 20 years of sales experience and we modeled the lead agent off our best person, but our best person who has two years of sales experience." So it actually is important to understand the art and the science of sales and how you bring best practice to bear. Either you've done it and so you know some best practice or you're going to geek out on sales, read a bunch of books, learn a thing or two, and try to incorporate some of those into your agent development.

**Lenny Rachitsky** (00:32:28):
That is really interesting. So come from the sales side, not from the engineering side. And I imagine this is such a cool opportunity for salespeople to do something completely different and move closer to engineering.

**Jeanne DeWitt Grosser** (00:32:38):
Yeah, I mean we're having a lot of fun with it. At Vercel in particular, we basically get to be customer zero. So everything that we're building with agents, we're building on Vercel's AI cloud. So these agents now have multiple steps that they go through. So we're using Vercel's workflow SDK and workflow offering. We use the AI gateway to call the different models that we use to do deep research or other enrichment that we do. So for us it's great because we basically sort of bang on everything the engineering team is building and get to go be a discerning customer before we actually get it out the door to real customers.

**Lenny Rachitsky** (00:33:22):
What a fun time to be alive. I could tell the fun that you guys are having, just from the way you describe it.

**Lenny Rachitsky** (00:33:29):
Stripe handles the massive scale and complexity of many of the world's fastest growing enterprises, including 78% of the Forbes AI 50 and more than half of the Fortune 100 enterprises like Atlassian, Figma and Urban Outfitters use Stripe to create fully branded and customized checkout pages with access to more than 125 global payment methods. There's a reason I've had more leaders from Stripe on this podcast than any other company. They know how to build great products that scale and that people love. And Stripe is a lot more than payments. They've also got a category leading billing solution and a highly optimized checkout experience built specifically to increase your checkout conversion. Join the ranks of industry leaders like Salesforce, OpenAI and Pepsi that are using Stripe to grow faster and to grow the world's GDP, learn how Stripe can help your business grow at Stripe.com. Zooming out a little bit in terms of you mentioned tools and tools that you use. I'm curious just what are kind of the state of the art tools within the go-to-market stack that you love that you'd recommend?

**Jeanne DeWitt Grosser** (00:34:33):
Well, so I'm going to have an interesting answer to this, so I'll give you one. And it's not state-of-the-art per se, although I don't mean that disparagingly, it's just that it's been around for a while now and a lot of folks use it, but I think Gong has gotten just meaningfully more interesting in the last year. And then second half of my question I will get into, I think the calculus on build versus buy is changing. So all right, Gong. Gong is incredible because you can run agents against it now. So we take all of our Gong transcripts and we dump them into an agent called the deal-bott, and that deal-bott then can do a bunch of things. So the first thing we had it do was lost opportunity review. So we had just finished Q2, we had a list of our top losses for the quarter sorted by deal size, and we ran it against that and it was incredibly interesting.

**Jeanne DeWitt Grosser** (00:35:39):
So the biggest loss that quarter according to the account executive was lost on price. And when you ran the agent over every Slack interaction, every email, every GONG call, it said actually you lost because you never really got in touch with an economic buyer. And when you talked to somebody about ROI and total cost of ownership, it was clear from their reaction that they didn't really buy your mass. And so really the reason we lost was an inability to demonstrate value, which upon reflection I've got work to do to build out how we quantify the value of Vercel, which actually is very easily quantifiable. It's one of the things I love about selling this product, but we got to codify that for the go-to-market team. So that was incredibly interesting and now we run it against all of our lost opportunities and actually do a much better job of categorizing why it was we really, really lost.

**Jeanne DeWitt Grosser** (00:36:38):
And then either feeding that back into the engineering team or back into marketing sales leadership on, hey, where are we falling short in the sales process? And so that was awesome, but then we're like, well, it's not very fun to lose, so why don't we pull that forward? And so we went from lost bot to deal-bott and now the deal-bott is running in real time and we basically feed insights into Slack. Vercel is incredibly heavy users of Slack, so we have a channel for every single customer, either opportunity or existing one. And so now we're feeding insights into that Slack channel which is, "Hey, you're this far into the sales process and you haven't talked to an economic buyer, you should think about that." Or, "Hey, you just got off that call with an economic buyer, didn't sound like it went that well. Here's some things to consider and how you might follow-up."

**Jeanne DeWitt Grosser** (00:37:34):
And last thing before I pause, the other thing that's really interesting and how we're using this too is we are in this moment where I have never seen an iteration velocity exists now in my career. My 20 plus year career has all been in tech. And so for go-to-market teams, that's really hard. If you are launching something every other day, the ability to be enabled on that is actually quite challenging. And so this bot agent is now also letting us, where we're starting to go with it is we'll release something, we'll do our best to enable the team, then we'll go run the agent across calls, interactions, and we'll diagnose where we did a bad job of objection handling, where we're getting stuck. And then at the end of the week we can have a huddle and say, okay, what are all the places that our agent would suggest we aren't selling effectively?

**Jeanne DeWitt Grosser** (00:38:34):
And then almost like an engineering team, we'll now run sprints, which is like those are just bugs. They're bugs in your go-to-market process, so you should not have them. And by the next week we're going to add content to our objection handling to guide. We're going to add content to a discovery guide, we're going to figure out something we need to change about our demo, so on and so forth. So that's early. That's a little bit of a preview, but that's where we're talking about taking things right now within our go-to-market orgs.

**Lenny Rachitsky** (00:39:00):
Jeanne, you're blowing my mind in so many ways, it just sounds so fun and just you guys are going to win is what I'm feeling when I hear all this. Incredible. What I love about this is this AI tool, this agent you built sees things that humans were not seeing. The fact that you were surprised of just like this is a completely different conclusion is such a big deal. This is the whole promise of ai, it's going to do things we aren't even thinking about or capable of.

**Jeanne DeWitt Grosser** (00:39:26):
It is. We had a really interesting, one of the things we're doing at Vercel, we have an AI cloud, so people use that to put AI-native features into their customer-facing applications, but they're also using it to build internal applications to improve productivity or outcomes. And we are talking to a very large airline and that airline obviously gets tons and tons of support queries. So of course they would want to go apply AI to hey, how can we have AI answer these so that our cost to support goes down, sort of the obvious thing. But the more interesting conversation was actually with one of the C-level executives who said, we also actually transcribe every single one of those support calls. And so what I really want to know is why are they calling and how do I make it so that fewer people call the next week? And so again, this is now with AI, you can rapidly go through all of that content and actually be able to much more quickly than having a human in your CRM sort of pick some status why it was that folks were calling the airline this week and what if anything you can do to make it less the case next week.

**Lenny Rachitsky** (00:40:39):
I imagine many people hearing this are like, "I need one of these deal-botts and lost bots." These are all internal products that you all built?

**Jeanne DeWitt Grosser** (00:40:46):
Yes.

**Lenny Rachitsky** (00:40:47):
Is there anything that you've learned about making them this good? Any tips you can share of here's how to make a really good bot for sales?

**Jeanne DeWitt Grosser** (00:40:54):
Yes, so actually that's the second half of my answer that I forgot.

**Lenny Rachitsky** (00:41:00):
That's perfect.

**Jeanne DeWitt Grosser** (00:41:00):
Which is sort of like bill versus buy calculus. So I think one of our learnings is that it's not that hard to build these agents and they aren't that expensive either. So I mentioned the lead agent that was a six-week process with one human, a third of his time, that deal-bott, the lost bot version was two days basically we riffed on it, he had it 40 hours later. Now we're continuing to refine it for the other things I mentioned. And what's also interesting about them is they for better or for worse for Vercel, but that lead agent which runs full stack on Vercel, will cost us about a thousand dollars to run for the entire year. If you remember I told you we had 10 people in the SDR function, so I'm paying well over a million dollars for that from a salary perspective.

**Jeanne DeWitt Grosser** (00:41:57):
I got that down to one. And then behind that I have a lead agent that costs a thousand bucks. So that's like a 90%-plus reduction in total cost there. And there's lots of software for agents out there right now. And I think one of the things we're learning is because this whole space is so nascent, often your own esoteric context, your content, your workflow is really key to unlocking the power of the agent. And so I think there's real value in experimenting with your own internal agent development. We may ultimately end up on better integrated agent platforms in the fullness of time, or we may find that the CIO increasingly goes from a procurer of software to a builder of software and you'll have an AI internal platform with a thousand agents running across your org. I'm not really sure yet. But I certainly think there's value in trying it yourself because you may find that it's meaningfully easier than you think and you get returns pretty quickly.

**Lenny Rachitsky** (00:43:10):
So what I'm hearing here is that you're finding that there are not tools out there to plug and play. The alpha is essentially in building your own stuff.

**Jeanne DeWitt Grosser** (00:43:18):
I think that's partially true, and I think because you also have all these tools proliferating right now, you get into the perennial problem where you wind up with 20 of them to do the 20 jobs to be done basically, rather than an integrated platform that's doing all of them. I'm hearing this a lot actually when I'm talking to customers right now where their biggest issue in deploying AI is actually just getting through procurement and it's because got an AI mandate, you kind of have a blank check. I recently heard the term of instead of ARR, it's ERR, which is experimental run rate revenue, which is to say everyone's out there sort of, Hey, we're going to give this thing a go for a year and then TBD on whether or not we keep it. But basically you're having to procure 20 different things. Most things are getting off the ground and so they're solving something relatively narrow and that'll change in the fullness of time. But I do think there's an opportunity to figure out, hey, where do I likely have a more specific workflow internally. For that it might be worth building your own agent and then maybe for the things that are a little bit more generalizable, you go get something off the shelf.

**Lenny Rachitsky** (00:44:34):
Are there any platforms or tools that you want to shout out that allow you to build these agents so quickly? I know they sit on Versel, so shout out Vercel. But just anything that you point people to you to... These SDR, these GTM engineers, they're former salespeople. Are they learning to code? Are they byte coding these agents? How does that work?

**Jeanne DeWitt Grosser** (00:44:52):
So our sales engineers all have CS degrees. So they were engineers in a sales capacity, so they're writing code and actually these agents, they're building directly on Versel. So you get the AI gateway that lets you call different models. You have a sandbox if you're running untrusted code, you've got workflows that let you build the process. You've got fluid compute, which lets you really efficiently use compute when you only need it. So we're just sort of building it from the ground up here. Again, it's not that hard. Now you do need to write code for that. Certainly there are a lot of vibe coding tools out there that also give you more workflow builders that are somewhere between fully WYSIWYG, almost like drag and drop and a little bit more code forward. So you've got a bunch out there along those lines. But I do think we've sort of found one of the reasons actually the GTM Eng team at Versel can build these agents so easily is because the Versel platform is making it that easy to use our framework to find infrastructure and get that agent onto into production very rapidly.

**Lenny Rachitsky** (00:46:11):
What a neat, unfair advantage you all have to do this stuff.

**Jeanne DeWitt Grosser** (00:46:13):
Yes, it is fun to... I mean, I do think this company is better than any I've seen at eating its own dog food and just everyone is constantly, we say Versel builds Versel with Versel. So you're just always looking for ways to, Hey, how can we use our product to go do what we need to do? And as a result, either understand then what a customer would want or what's missing from our product that we could go make better.

**Lenny Rachitsky** (00:46:37):
Along these lines, something that's already come across a lot in the way that you described this stuff is I've heard a lot about how you think about go-to-market as a product. A lot of people listening to this, as I've said, are product builders. So I think this is a really nice way of thinking about go-to-market. I'm guessing you've already talked about elements of this, but just what's a way to think about go-to-market as a product?

**Jeanne DeWitt Grosser** (00:46:56):
Yeah, I've always, so I had this realization probably a little over a decade ago in my career. So my first job out of college was working on Gmail in 2004. So Gmail launched on April 1st, I joined on June 1st. And as I'm sure you'll remember as well, Gmail was this incredible innovation, massive JavaScript application that didn't really exist at the time. And it had this gig of storage. It was a full year before Yahoo Mail caught up and even longer before Hotmail and others did. So that was the level of technical differentiation between Gmail and the next best. And a decade later, you had cloud computing enabling folks to do stuff that you never would've been able to do previously. And so I kind of felt like, huh, software's starting to commoditize a little bit. And so when that happens, when technical differentiation kind of narrows, what are other things that will differentiate you?

**Jeanne DeWitt Grosser** (00:48:01):
And I was started thinking outside of tech, we buy a lot of things because of how we feel about them. And so I started to develop this thesis that actually the experience that you have of being sold to will increasingly actually differentiate a company and drive buying decisions if products are only different at the margin. And so if you believe that, then you really want to create a customer buying journey that feels like very unique experiences. And so we did a lot of this at Stripe and now we're looking to replicate this here. But an example of one of the things I think we did really nicely at Stripe was a lot of companies sales, the first call after you're qualified, we've decided you're worth engaging in sales process is discovery, which is basically let me ask you a lot of questions to try to under-uncover paint, figure out where buying power lies, et cetera.

**Jeanne DeWitt Grosser** (00:49:03):
And so that is kind of boring sometimes for a customer. You're basically being quizzed often on the phone. And so what we started to do at Stripe was that first session was a whiteboarding session, and we would actually get together and have you draw your architecture for payments and all the other things that were under the hood to enable you to take money and drive customer outcomes. And through that we would learn a ton about what was in your stack, what we were going to have to compete with, displace where value lied. But the customer also learned a lot themselves because in many cases they'd never drawn their architecture diagram. And so they left that meeting with an asset and a sense of like, "Wow, this is a really collaborative person who's deeply interested in helping me develop a mental model for how to think about this." And then we had other things that we would do.

**Jeanne DeWitt Grosser** (00:50:00):
So that's sort of how I think about building go-to-market-like a product is basically you need to go through from the first time you become aware that the company exists to again, that sort of five-year heavily retained wall-to-wall customer a set of experiences. And those experiences can feel transactional, flat, boring, or they can feel very human, personalized and unique. And so we try to go map those out and figure out how do you bring the product to bear, make it really human, and hopefully that creates a customer for life in the end.

**Lenny Rachitsky** (00:50:37):
I love that whiteboarding example. Are there any other examples of what you've done to make it actually work really well in this way?

**Jeanne DeWitt Grosser** (00:50:43):
Yeah. Another principle, we really developed this at Stripe too and I brought it to Vercel, was just the idea of adding value at any touch point regardless of whether or not that customer bought. Because even if customers don't buy, you often find that if you miss them on that buying cycle, three or four years later when they're in another buying cycle, they do come back. I was at Stripe for nine years and so I saw the number of customers that we lost and then half a decade later, here they are and they bought. So that was sort of another one. So examples of this that were doing at Vercel is there's great data on the internet that helps people understand the performance of their website and how fast your website is actually impacts SEO. And SEO impacts AEO and everybody's thinking about AEO right now. And, so one of the things we try to do when we reach out is actually give folks insight immediately into how they're performing on an absolute basis, how they're performing relative to peers. So ideally that piques your interest and you want to learn more from us, but even if it doesn't, you still have insights that you may or may not have been aware of that maybe make you contemplate whether or not you've got the optimal setup.

**Lenny Rachitsky** (00:52:07):
Awesome. So what I'm hearing here is when you say, think of it like a product that's basically a product person thinks about the experience of their product, that every step of the journey, here's the flow, step 1, 2, 3, 4, 5, how do we make every step awesome, keep them going along that journey. And so what you think about is just from the prospect's perspective, how do we make every step of that journey awesome, continue them down that journey.

**Jeanne DeWitt Grosser** (00:52:30):
Yeah. How do you make it be an experience rather than a transaction

**Lenny Rachitsky** (00:52:35):
Versus just feel like sales coming at you trying to sell you stuff?

**Jeanne DeWitt Grosser** (00:52:37):
Yeah.

**Lenny Rachitsky** (00:52:38):
Okay. Staying along this track of staying tactical, I want to go even further there. So what are just some go-to-market tactics that you find really effective these days for people trying to just to be more successful in getting people to pay attention to their stuff, to buy their stuff?

**Jeanne DeWitt Grosser** (00:52:57):
I mean, one I would sort of say dovetails with where I just ended, but is what are the unique insights that you can bring to bear about your product or how that customer may be in a suboptimal state? So I do think investing in data to tease that out is one thing. I think the other thing this is straightforward but often not done enough is a lot of good companies invest in docs, good thing to do, but they stop there. And particularly if you are selling into a slightly larger company doing things like, AWS calls it well-architected guides or blueprints, a lot of customers, particularly larger ones, really want to know the best practice for how exactly to implement your product with their particular setup. A great example of this, this is from Stripe, was Stripe was excellent at marketplaces. Most, Lyft, Instacart, DoorDash, they were all on Stripe.

**Jeanne DeWitt Grosser** (00:54:07):
And so Stripe definitely knew the best way to set up payments for a marketplace because we'd seen them all. And so when you then would go and sell a marketplace and say, "Oh yeah, we've got docs, go check them out." They didn't like that, because they're like, "Hey, every marketplace runs on Stripe. I don't want to look at generic docs. I want you to tell me what's the best way to set up payments for a marketplace." And so I think that's another key thing to be doing, particularly as you move past that sort of solo developer startup founder as potentially a target audience.

**Jeanne DeWitt Grosser** (00:54:39):
And then, I don't know if this is a tactic per se, but I do think just a good reminder for founders in particular who are still in that maybe founder-led sales moment is just the value of really good discovery. I often find founders are so excited about talking about their product or you ask one question and now they've got a hook of like, oh, I can fix that for you. But excellent salespeople typically will talk well under half the time in a conversation because they're out asking questions, probing often helping a customer arrive at conclusions on their own. And so learning how to do five why's, go deep rather than immediately going into problem solving mode. If they ask a question, you respond often. If they ask a question, you should ask a question about the question and then respond. So learning to be great at that, I think differentiates people.

**Lenny Rachitsky** (00:55:43):
So the last tip, I think there's something a lot of I bet everyone could learn is just listen more and talk less.

**Jeanne DeWitt Grosser** (00:55:48):
Yep.

**Lenny Rachitsky** (00:55:49):
On that first piece of advice, this kind of sharing unique insights and how your suboptimal, is there an example you could share of how you did that? Maybe a story of just how you convinced someone you're selling Striper or Vercel like care or something you're missing. Here's how this could help you become much better.

**Jeanne DeWitt Grosser** (00:56:04):
So with Vercel, sort of giving an example, but I'll make it more specific. So the performance point, you can go and look at core web Vitals, and so we can actually see the different things within their site that are fast or load correctly, et cetera, so anyone can go look that up. But what we can do is actually then help with benchmarking relative to peers. So that's been a big one that we've gone out and done. The other one that we've spent some good time on is just around helping customers understand MCP servers and when it would make sense to use one. So I think those are all the rage, but often people don't know how to contemplate them within their own product. So that was another one that we've gone pretty deep on and then related to, the first one is AEO Answer engine optimization is actually somewhat tangential to Vercel right.

**Jeanne DeWitt Grosser** (00:57:09):
So we drive performance, performance drives SEO. SEO is an input into AEO, but we have spent a ton of time sharing insights on AEO because we ourselves focus deeply on it and think we understand it better than many. And so again, as part of just building a trusted relationship, folks may go from those AMAs or that content into, okay, great, you taught me a lot and therefore I want Vercel to help me with performance. But in many cases, they actually now are just like, "This is a company that seems insightful, it seems like one I can learn from, and now I'm going to pay a little bit more attention to them." And over the fullness of time, maybe they see something that triggers them to decide, "Now is the time I want to go investigate that aspect of Vercel."

**Lenny Rachitsky** (00:57:55):
Awesome. So what I'm hearing here in many ways, and this resonates, I had Jenna Abel on the podcast recently and it was all about sales skills and how to sell.

**Jeanne DeWitt Grosser** (00:58:02):
Nice.

**Lenny Rachitsky** (00:58:02):
And one of her tips is you don't want to be focusing on here's the pain and problem we're solving and instead focus on here's how you will be better than your competitors. Here's the big gap and alpha that you can achieve. If you use Vercel, you were missing out on speed and you're going to get screwed in AEO and all these things. Here's how you can architect your entire payments system to be top tier. Does that resonate?

**Jeanne DeWitt Grosser** (00:58:27):
Yeah, I was told this stat. It's round numbers, so I can't imagine it's entirely accurate, but basically that customers, 80% of customers buy to avoid pain or reduce risk as opposed to the other one out of five to increase upside, which is a good thing again for startup founders to understand. So we all love to talk about the art of the possible, everything we're going to enable in the future. It's very exciting. Everyone's visionaries, but that's often really a sale that's going to resonate with another founder. And for everybody else, particularly enterprises, you're avoiding the risk of not making your revenue target next quarter, the risk of being outdone by the competition, the risk of having brand damage, et cetera. And so it's really hard actually for many startups to make that pivot because it feels off brand, but it does actually drive more buying behavior, is setting up a little bit of that concern that either I might not be well positioned or again through good question asking. I know exactly where I'm not well positioned and you can help me, that

**Lenny Rachitsky** (00:59:53):
That is such an important stat you shared. This has come up actually before in this podcast that buying, people are buying in large part to reduce risks, to basically not hurt themselves in their career, not hurt the company. That's a bigger factor in the buying decision than, "I have this problem I need to solve. And okay, thank you, this is solving." And the way April Dunford came in the podcast and talked about this of just like it's such a massive career bet. We are going to bring in product X and it's going to become, like Stripe, let's say, let's not talk about Versel. But let's say Stripe, we're going to adopt Stripe. That's a huge decision. If it doesn't go well, your career is hurt, your manager is going to be mad at you, it's going to set your company back. So a lot of the buying decision, as you've said is I just don't want to screw this up.

**Jeanne DeWitt Grosser** (01:00:36):
Right. Absolutely.

**Lenny Rachitsky** (01:00:37):
Okay. Along the line of tactics, something that I know you're a big fan of and help people think about is segmentation.

**Jeanne DeWitt Grosser** (01:00:45):
Yes.

**Lenny Rachitsky** (01:00:45):
This is something a lot of founders struggle with. They know, "Okay, I need to figure out my segmentation strategy and here where we're going after." Can you just give us a primer on segmentation, what people should know about why this is important and then how they might approach this.

**Jeanne DeWitt Grosser** (01:00:59):
So segmentation is basically how do you carve up the world of companies that exist on the planet to reason about them where they buy differently? So I'll give examples from Stripe and Versel to bring this home. So a very typical company segmentation is small, medium, large. That's a rational way to do things. Small, you often have a single decision maker, medium, a small team, and large, it's complex, it's a committee, et cetera. So the buying process does change across SMB, mid-market enterprise, but if you stop there, you are likely missing. But what are the things within your offering that also change the way something gets sold? So at Stripe, there were two ways we further cut the business. Way one was, so think of segmentation as a graph. So X-Access was size, so small, medium, large, y-access was growth potential. And that was important for Stripe because it was a consumption-based business.

**Jeanne DeWitt Grosser** (01:02:10):
So if you were going to grow at 200% year-on-year, you were more valuable to Stripe than if you were going to grow at 8% year-on-year. And so we wanted to spend more time, spend more money going after the 200% growers than the 8%. So that was one that informed your strategy on who you targeted. And then for Stripe, the other thing that we cut it was business model. So are you a B2B? Are you B2C? Are you B2B2B, E.G. a platform or B2B2C, E.G. a marketplace and why is that relevant? Well, if you're B2B, you are going to need business payments. Credit card was useful for a PLG function or PLG sale, but you were going to need ACH wires, etc. And you probably had a recurring business, so you were going to want Stripe billing. If you were B2C, that's consumer.

**Jeanne DeWitt Grosser** (01:03:00):
So you're going to want consumer payments. Apple Pay is super important. If you were in the platform or the marketplace, you were going to buy our connect product. So it helped us basically then craft a more targeted and replicable sales. Vercel, sort of similar deals. So small, medium, large buying complexity. We also do the same thing on growth potential because we are similarly a consumption based business, but for us, a couple other things on the X-axis, we layer in promote, which is one of the things that is observable is traffic, site traffic on the internet. So Google publishes a Crux score, which is basically they have a bunch of data in Chrome, and so they know that Lenny's site gets a million XC amount-

**Lenny Rachitsky** (01:03:48):
Millions.

**Jeanne DeWitt Grosser** (01:03:48):
... volume that Jeanne's site does. And so basically if you are a small company but you have super high traffic that's going to be more complex, Vercel is going to make more money and so we want to promote you.

**Jeanne DeWitt Grosser** (01:04:02):
So great example of this would be OpenAI. OpenAI, I forget these days how many employees it has. Let's say it's 3,000, it's probably more than that at this point, but so that's going to put it in the mid-market at most companies, but they're a top 25 traffic site on the internet. So for us, that's going to push them in our enterprise because we need to go lean in with a much more in depth sales process. And then the other thing we layer on is a workload type. So if you are an e-commerce company, that's going to be a very different sale. You actually use different language. You talk about product listing pages and product description pages, and you've got an order management system as the back end. Super different from a crypto company where you might be running soup to nuts on AWS. And so again, that helps us start to then have a really different buying content for you.

**Lenny Rachitsky** (01:05:06):
Okay, this is awesome. So essentially what you do is you break up this universe coming back to your original story at Stripe to help you sort essentially which companies are most likely to buy your product. And what you're coming up with is these attributes that are correlated with they're likely to be great potential customers.

**Jeanne DeWitt Grosser** (01:05:22):
Yep.

**Lenny Rachitsky** (01:05:22):
Do you recommend using this XY axis as the approach versus something else? There's like a spreadsheet with five columns. I don't know, how do you start?

**Jeanne DeWitt Grosser** (01:05:31):
There's probably something to be said for X and Y. like do you think size is going to play into most buying decisions and then these days there is a fair amount of consumption happening? So there'll be aspects of this that I think are somewhat universal. But I think basically when I came to Vercel, because new product market product offering, for me it's a new market. I had a lot to learn, but this is one of the first things I did in the first 30 days. And so basically I sat down with the gentleman Abhi who leads data science here and said, okay, what drives revenue? So what are the things that you can look at X ante about a customer to know this person's likely to pay us a hundred thousand dollars versus a million? That's probably going to be part of a segmentation framework. And then similarly, okay, what attributes would we look for to cluster where we seem to be winning repeatedly? And that was how we ultimately got at, okay, Crux rank is going to be super important because what you pay Vercel is correlated with your traffic. And then workload type was super important as well.

**Jeanne DeWitt Grosser** (01:06:46):
And for Vercel, when we did that, it was really interesting because we saw, wow, we have a lot of penetration and e-comm not that surprising actually, given that we drive highly performant sites and e-comm having a superfast performance site really matters. But at the time, if you looked at as an example, an enterprise SaaS companies, we didn't have a lot of penetration, even though you would've thought, okay, front-end cloud, very developer oriented. Of course software companies would be on us, but in enterprise, most of those companies built that SaaS offering before Vercel existed. So migrating 2 million lines of code to Vercel, that's a big lift. So it helped us really understand where are we winning, where are we not? And now as an example, within SaaS companies and enterprise, we're actually seeing a lot of interest in the AI cloud. Those are some of the earlier adopters of, "Hey, let's add AI native functionality to our existing SaaS app." And so again, it helps us figure out what to target where.

**Lenny Rachitsky** (01:07:55):
So essentially you're doing this regression analysis on what's working and then here's the attributes that are most correlated with success. Something I always recommend when founders ask me for how do I figure out my CPE? How do I figure out where to focus, my heuristic is just think of three attributes that narrow them down. So it's like series A company that's angel-led, that's the marketplace, something like that. Does that feel like a good just rule of thumb just to start?

**Jeanne DeWitt Grosser** (01:08:18):
Yeah, I think beyond three, that's getting pretty detailed and reasonably speaking, you're not going to cut. You have five sellers. So, what, you're going to put one seller in five different segments? So I do think three is something you can reason about. The other thing I'll say on this topic that I think is really important is a lot of times folks think segmentation is a go-to market thing. I really think it's a company thing. So when you Vercel, I actually deliver and every new hires first week, one of our company values is KYC, know your customer and I deliver the KYC section and talk through our segmentation framework how our customer base maps into those segments because it's really important as those new product managers leave the room that when they're building something, they think to themselves, okay, I'm building a new back end product. Who is this targeted at? Is it targeted at an enterprise or a startup? Basically, do I have a point of view on where I'm trying to win and why? And if you're doing that out of the gates, then it's much easier to then go speak the same language with the go to market org and figure out, okay, how are we going to take that to market in line with the other emotions that we have in play?

**Lenny Rachitsky** (01:09:36):
Okay, this is a great segue to, there's a couple other things I want to talk about. One is something I've heard from so many people you've worked with is that you are amazing at building a go-to-market org that works really well with product and engineering. So I'll read this quote from your former colleague, Kate Jensen. She said that your superpower is building a sales org that doesn't feel like a sales org to engineers. So the question she suggested asked just what does it take to do that? What are the ingredients to building a sales org that engineers and product teams really like working with?

**Jeanne DeWitt Grosser** (01:09:59):
The litmus test I have always given my sales team is if you are an account executive in my org and I put you in front of 10 engineers at our company, it should take them 10 minutes to figure out you aren't a product manager. And what I'm trying to get across is you need to have incredible product depth. And the reason for that is twofold. One, it gives you credibility with the product and engineering org. And two, I also believe that the best go-to-market orgs on the planet are equal parts revenue driving and R&D and D. And the reason I emphasize the latter is if you think about a product management organization, you may have a UXR team out doing research, product managers certainly should be out talking to customers. Well, if I have a 20-person sales team, think of the number of customers that we talk to in a week. And so if we can do an excellent job of translating all of that feedback into signal and then feeding that into the road map, we can be actually an extension of the product management org. But that takes being really good at discerning signal from noise, understanding when something is an objection that should be overcome versus an opportunity in the market. So I think those things have helped.

**Lenny Rachitsky** (01:11:27):
I just love this as a product manager, maybe form a product manager. I don't know what the hell I am these days. I just love the idea of the salesperson. Like you not knowing the difference between a product manager and a salesperson. The most classic challenge is sales orgs ask for all these features and PMs are constantly having to push back and think about does this fit into everything. So it feels like that's a big part of this is to understand that deeply.

**Jeanne DeWitt Grosser** (01:11:51):
Yeah, you want a sales org that can think like a general manager, so that's not just trying to get deals done but is trying to help build a business. And so again, knows when to say no, knows when to do objection handle versus knows, Hey, I've actually heard this on the last three calls and I do think this would be a really big unlock that would make us more competitive, would be something that new that nobody's doing. So I think that takes looking for a profile that both has sales skills but also is going to think with that product mindset.

**Lenny Rachitsky** (01:12:31):
I love that. Okay, so another quote from Claire Hughes Johnson, former podcast guest, amazing sales leader, worked with you at Stripe. She said something along these lines, but a little different. Jeanne is probably the best go-to-market person at connecting with product and engineering, deeply understanding the product and providing the most valuable input to her counterparts of any I've ever seen. It sounds like just another ingredient here is just sales feeling like a real partner to product engineering actually, not just being like, "Hey, do these things for me, but actually feeling like a partner."

**Jeanne DeWitt Grosser** (01:13:01):
Ultimately company strategy is basically product strategy meets go-to market strategy. And so I spend guess as a go-to market leader, I'm constantly trying to figure out how do I make more money more efficiently? And you typically do that by having a winning product in the market that is well commercialized. And so that means that I really lean into thinking about product strategy and thinking about pricing strategy because if those two things are optimal, you're going to win more often and there'll be less friction in it. And so that's sort of where got to put as a revenue leader, like a GM hat on and not just think, how do I sell? But actually how do I enable the insights I'm getting from talking to customers constantly to have the company strategy be more effective?

**Lenny Rachitsky** (01:14:00):
Speaking of product, going in a slightly different direction, PLG product-led growth, it felt like it was very hot for a while where everyone's like, "You got to go PLG, that's the only way to win. It's impossible to do sales. The future is PLG." It feels like that's gone away. And in large part, obviously still companies grow through PLG and work through PLG. What's just kind of your thoughts on PLG and when does it make sense for a company these days to actually think this is how they'll grow for a while?

**Jeanne DeWitt Grosser** (01:14:28):
PLG makes sense for a lot of companies at the outset, unless you are very explicitly building a product for enterprise. So Sierra as an example, right? They are very clearly going after Global 2000 or something close to that. PLG is not going to be overly useful to them because they are trying to win eight-figure deals from day one. But for a lot of products, folks are targeting a startup audience at the outset and then they're adding more functionality so that they can ultimately continue to scale up market. So I think PLG is still super relevant. It's a major driver of Vercels growth. It was a big driver of Stripe's growth. The thing that folks get wrong is it does typically have a ceiling. So people are generally not going to give you $1 million via self-serve flow. So at some point if you want to sustain growth rates, you're going to have to have your deal sizes get bigger and bigger. And where I think folks get stuck is waiting too long on PLG because it does take a while to build a replicable sales process and a sales process, which often you're getting fed by inbound at the beginning and then you got to add outbound. It takes a while actually to turn outbound into a predictable engine. So I think where you see companies hit walls is just when they don't add the sales portion of it soon enough.

**Lenny Rachitsky** (01:16:00):
So essentially every company ends up having to build a sales org, some start product-led and then at sales, some just start sales and have it from the beginning.

**Jeanne DeWitt Grosser** (01:16:09):
Yeah, I would agree. There are probably some good examples of large vertical SaaS platforms that are SMB, but even they wind up with Velocity sales team. So yeah, I don't know that I can think of a 100 billion company that's PLG-only.

**Lenny Rachitsky** (01:16:30):
Yeah, it just feels like you're leaving money on the table even if you are growing really fast. I know Atlassian was a long-time PLG company but eventually succumbed. I don't know if that's the right way to put it. Okay. You mentioned pricing. I know you have strong opinions on pricing and pricing strategy. What's just a couple of tips you might share with someone thinking about how to price their product?

**Jeanne DeWitt Grosser** (01:16:52):
Yeah, this is kind of on the theme, but I think the first thing is you got to think about pricing like a product. So it's another one where it actually really matters how you choose to price a product. Do you really understand where customers are going to drive value? Do you really understand where you incur costs? And are you doing a smart job of aligning those things? You've got lots of examples of companies grossly underpricing, you're sort of afraid to charge for the value that you actually provide. I think there are a lot of examples where people default to including a freemium strategy without that actually being a strategy. A good example at Stripe, we launched Stripe Billings years ago. It had a freemium strategy because that's what you do. And then we sort of looked at it and we're like, "actually integrating straight billing takes a little bit of work.So if you do that, you're probably going to stay."

**Jeanne DeWitt Grosser** (01:17:56):
And so we killed that, killed the free trial to zero downside. So that's another one. At Vercel, we've been going through that transition where we're a consumption-based business model ultimately, but at the outset we basically kind of bundled that into what looked like a SaaS-like price and as we've added a lot more functionality that wasn't working anymore. And so we did an unbundling and right now actually we did a pretty substantial pricing change in August where we have an enterprise at a pro-skew. And if you looked at the enterprise skew, it's called Enterprise for a reason, enter, it's meant to be sold to an enterprise. And actually about half of the folks on the enterprise skew were startups, which suggests that there's stuff in the enterprise skew that a startup really wants. So we kicked a lot of that stuff out of the enterprise skew and made it so you could buy it self-serve online and what do you know, people are.

**Jeanne DeWitt Grosser** (01:19:03):
So now that's really driven a lot of growth in our PLG funnel, which is awesome for startups because it's super efficient. They can just buy things, they want that. It's awesome for us because you don't have to have a human intermediate that. So getting all of these knobs really tuned is a key to both a great customer experience and optimal revenue outcomes.

**Lenny Rachitsky** (01:19:24):
Maybe just one more question before we get to a very exciting lightning round. It's going to be a combo question. I hear you have a hot take on sales comp, how to comp salespeople that's different from other people and also who to hire when you're hiring folks in sales. Can you just talk about your takes there?

**Jeanne DeWitt Grosser** (01:19:41):
I struggle with sales comp because it's all about pay for performance, which I'm obviously a fan of, but it makes your organization less flexible because you basically have to decide 12 months in advance, these are things I value and particularly in this moment that could be different. As a great example of this, when we wrote the sales plans for this year at Vercel, the AI cloud did not exist. We were selling our front-end cloud and we were selling VZero and introduced the AI cloud halfway through the year. Now we had all sorts of good ways to still incentivize that, but I think you want to be able to be innovative and pivot and when you have a well-designed sales plan or a very structured sales plan, that can be challenging.

**Jeanne DeWitt Grosser** (01:20:44):
So that's a little bit of my hot take is just I'm trying to figure out how do you have the upside of sales of motivates people. It's a quantitative function, which is great, but also the flexibility to change your mind because I think a lot of companies right now are having a hard time doing annual planning. So that's one. On profiles, I have always valued just sort of a diversified portfolio. So I strongly believe that sales is a skill and so you want salespeople with actual sales experience in your organization, but I think there's value in pairing them with more nontraditional backgrounds, in particular consulting or banking background. Those folks are really good at more quantitative and analytical aspects of sales. So getting into that consultative part, which I think we talked about at the outset. And so I find that when you mix these together, the sort of consultant banker profile realizes, "Oh wait a minute, sales is a skill and I didn't really have it." And so they go learn from your account executives with that background and then your AEs learn more about, okay, how do I think about a P&L? How can I talk to a CFO? How do I present a TCO analysis more effectively? And so just creates a much richer learning environment where people are bouncing ideas off each other.

**Lenny Rachitsky** (01:22:22):
That is awesome. I love that strategy. Okay, final question. Just is there anything else you wanted to share? Anything else you want to leave listeners with before we get to our very exciting lightning round?

**Jeanne DeWitt Grosser** (01:22:31):
Oh man. I feel like we've been very thorough.

**Lenny Rachitsky** (01:22:34):
All right, thanks So too.

**Jeanne DeWitt Grosser** (01:22:35):
Yeah, you stumped me on that one.

**Lenny Rachitsky** (01:22:38):
Okay. That's the goal. With that Jean, we've reached our very exciting lightning round. I'm going to make it very quick. I know you got to run. I'm going to ask you just two questions.

**Jeanne DeWitt Grosser** (01:22:46):
Okay.

**Lenny Rachitsky** (01:22:46):
One is I'm going to skip to your life motto. Do you have a favorite life motto that you often come back to find useful in worker and life?

**Jeanne DeWitt Grosser** (01:22:54):
I do. I actually have found that I'm known for saying a handful of things that I didn't necessarily realize it, but when you leave an organization, people tend to tell you what stuck with them. But there is one that I think I am known for saying growing up, my mom always said to me, when the going gets tough, the tough get going. And in sales, you're always going to have a quarter when you're not on pace. And so that's one that I feel like I pull on, not infrequently because in my view, there's another version of this, my mom also always says was where there's a will, there's a way. So I think you can always choose to find a path forward even when that's not super clear.

**Lenny Rachitsky** (01:23:45):
I love these. Okay, last question. I read that you were a very competitive diver in college early on. I'm just curious if there's something you learned from that experience that brought with you that helps you be as successful as you've become?

**Jeanne DeWitt Grosser** (01:23:59):
Well, I mean, first of all, I should say I was generally coming in third place out of three on my team.

**Lenny Rachitsky** (01:24:04):
Third place, that's not bad.

**Jeanne DeWitt Grosser** (01:24:07):
I managed to do it in college, but that was the extent of that career. So diving is a precision sport and it is a repetitive sport. And it is also a sport where when you land flat on your back, and literally as you are swimming to the side of the pool, welts are forming on it, you always 100% of the time will be forced to immediately get back on the diving board and do that exact same dive again. And so I think that has a lot of stuff that's transferable to work and to sales. So for me, I just have an obsession with excellence and within sales. sales is about replicability. How do you drive predictable outcomes, how excellent are you at your ability to forecast? And so I think I bring that to bear within sales a lot. And then similarly, you get a lot of nos in sales. So another phrase that a sales guru said to me once or in a training was yeses are great, nos are great, maybes will kill you. And so how do you get really comfortable that no is a great thing and that just gave you data and now you can go do something with it.

**Lenny Rachitsky** (01:25:25):
This is a really inspiring and empowering way to end the conversation. Jean, thank you so much for being here.

**Jeanne DeWitt Grosser** (01:25:33):
Thanks so much for having me, Lenny. It was a lot of fun.

**Lenny Rachitsky** (01:25:35):
Bye, everyone.

**Lenny Rachitsky** (01:25:37):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [14/17] Why LinkedIn is turning PMs into AI-powered \"full stack builders” | Tomer Cohen (LinkedIn CPO)

**Lenny Rachitsky** (00:00:00):
I think it's so underappreciated, the turnaround that has happened within LinkedIn. I check it at least 10 times a day. What was the strategy behind it?

**Tomer Cohen** (00:00:06):
I start backwards. It's like what is the potential here? If you start from the premise that LinkedIn ultimately is a platform for economic opportunity that sits on top of a very strong social graph. Almost every aspect of economic transaction is possible.

**Lenny Rachitsky** (00:00:20):
Is there anything tactically that would just like, wow, that really made a big dent in people wanting to come here and post, share interesting content?

**Tomer Cohen** (00:00:25):
To really set the new purpose for it, which was this is not a springboard for other products. This is not a traffic jumpstart, it's not an upsell feed. It's really about people that matter, talking about things that I care about professionally. The first thing we did was really making AI first.

**Lenny Rachitsky** (00:00:39):
How do you actually on the ground help people shift their perspective and think AI first?

**Tomer Cohen** (00:00:43):
So it wasn't like, "Oh, we have this cool technology, what can we do with it?" It was like, "Let go of what you've built. Go back to the objectives you were trying to solve and now with this technology, how can you do that objective better?"

**Lenny Rachitsky** (00:00:52):
There's so much I want to dig into here. Is there anything else that you think would be interesting or useful for folks?

**Tomer Cohen** (00:00:57):
AI is the ultimate matchmaker. It's underutilized, it's misunderstood, it's really about...

**Lenny Rachitsky** (00:01:07):
Today my guest is Tomer Cohen. Tomer is chief product officer at LinkedIn overseeing all teams responsible for building and creating LinkedIn products and experiences, including product development, design, business development, content creation, and customer operations. During his tenure at LinkedIn, Tomer was head of the mobile team, led the effort to revamp the LinkedIn feed and to many people surprised, made it extremely interesting in a place I check regularly. And he was also at the center of shifting LinkedIn to an AI first mindset, which started way before AI became cool. In our conversation, Tomer goes inside the strategy behind the transformation of LinkedIn's feed and how they approached making it a place that people wanted to check and make it much more social.

**Tomer Cohen** (00:02:27):
Thank you for having me.

**Lenny Rachitsky** (00:02:28):
Absolutely, my pleasure. So as I was preparing for this podcast, I reached out to a bunch of people that have worked with you and asked them what I should ask you on this podcast, and interestingly, every single one of them said this one thing that I need to ask you, which is about this phrase that apparently you use all the time. So first of all, can you guess what this phrase might be?

**Tomer Cohen** (00:02:47):
I have a few. Sometimes people call them Tomerisms.

**Lenny Rachitsky** (00:02:49):
Tomerisms.

**Tomer Cohen** (00:02:50):
But probably I might be wrong but not confused.

**Lenny Rachitsky** (00:02:54):
That's the one. Amazing. Okay, so let's talk about this phrase.

**Tomer Cohen** (00:02:58):
I don't say it so much anymore. I think it's ingrained into the cultures.

**Lenny Rachitsky** (00:03:02):
I was going to say that.

**Tomer Cohen** (00:03:03):
When they think of me, they think of this sentence already.

**Lenny Rachitsky** (00:03:04):
Those are the ultimate things where you don't need to say them as much anymore. Okay, so the phrase again is, "We might be wrong but we're not confused."

**Tomer Cohen** (00:03:10):
Exactly.

**Lenny Rachitsky** (00:03:11):
Let's talk about this. So what does this phrase mean and why do you find it so powerful and important to say often?

**Tomer Cohen** (00:03:17):
Yeah, by the way, it's a simple phrase, but it has in my opinion, so much depth into that and ultimately something I really believe in. It's rooted in clarity and principles that ultimately lead to leadership. And the first time I got the inspiration from it was from a startup founder I met many, many years ago. Their company was on the brink of failure. They had their last attempt and they decided on a path forward and after they decided on a path forward, he was still seeing people hedging in different directions and it led into this confusion in the system where we decided people are still hedging, they're still trying out things they thought could work, and he realized that unless they basically all pull through in the same direction, there is no chance they'll be able to be successful.

**Tomer Cohen** (00:04:04):
Now, pulling through in the same direction doesn't mean you're going to be successful, but this gives you a chance of success and that confusion, the system only luck and save you. So that was when he shared that, that was very impactful for me. I think it's a good one for life as well. And for me it comes down to two main parts. One is clarity of thought and clarity of execution, and they're both equally important.

**Lenny Rachitsky** (00:04:29):
**Tomer Cohen** (00:06:46):
On the clarity of thought. What I find people to be attached to, especially when you build an environment with a lot of alpha types is that they get attached to being right or wrong and that really creates a lot of lingering in the system, a lot of confusion and they're still stuck to their ideas. And for me, I get attached to clarity and focus. I think that's much more important. That's why I think when I say I don't mind being wrong, it really comes from a humble place. I would rather go forward with everybody in the same direction than necessarily try to hedge all the time, which will give me no chance of success. The way we start, we do this in our product gens right now, is we actually spend some significant time on what is the problem we're trying to solve for, but not high level, not like, hey, we want to launch this product, we want to launch a video product.

**Tomer Cohen** (00:07:35):
It's exactly what type of video you're trying to launch for which audience, what is your unique criteria, what are you trying, what is very nuanced about what you're trying to solve for? Ideally, once the time you see the problem, you know exactly the problem you're trying, you actually can imagine that mountain. It's not just a mountain, you can see exactly the road, you can see exactly the end of the base camp. But then when you move to solution, I love solutions that are based on first principles. That they have, there's a principle thinking about it, there's opinions about it. If you talk to folks who work with me, they'll tell you, I push a lot for what is actually your opinion, what is your potentially controversial opinion and the best principles have teeth.

**Tomer Cohen** (00:08:18):
So saying that we should build a simple product for me is useless. Who doesn't want to build a simple product? But saying that I'm willing to sacrifice or trade off this, that's where I get excited. I'm like, okay, that's a very strong opinion. Let's go into that. Why were you willing to trade off those type of vectors to make it happen? And one thing that I saw also in clarity of thought was this, is I came to the US in 2008, I came from Israel. Our hobby in Israel is to argue. So we argue a lot, it's our love language in many ways. And I come to the US and I notice people say a lot like I don't exactly understand or I'm not exactly clear on this, and it took me a long time to realize they're actually disagreeing.

**Tomer Cohen** (00:09:03):
They're just masking it with a layer of misunderstanding and a good mentor of mine said, "Hey, just push back. Are you disagreeing or misunderstanding? If you're misunderstanding, let's spend the night. Let's get to a point where you can articulate my point of view in your words and I can do the same, but if we're disagreeing, let's stop. Why are we spending? Why wasting time just arguing it through?" So those became really powerful. That's on the clarity of thought, clarity of execution is even more important because many organizations actually reach a decision, they don't act on it, which it's one of those shocking things. They decide this is a top priority, but it doesn't make its way into the organization. And I'll give you an example. Somebody will say, "Hey, my top priority for my business is this initiative." And then I'll say, "But most of your engineers are working on this migration." And they'll say, "Yeah, we have to finish that." I'm like, "So why don't you say the migration is my number one priority?" It's like, "Yeah."

**Tomer Cohen** (00:10:09):
It's like, but that's exactly what you're doing. You have to make sure that what you're sharing as a priority is actually manifested in your resourcing. Then I'm like, "Hey, this is your top talent. Why is your top talent working on some moonshots that are not in your number one priority?" And they're like, "Yeah." This is where you start finding that really it doesn't really translate into execution as well and you can solve so much by just making that sure that focus is there.

**Lenny Rachitsky** (00:10:36):
So the big lesson here is to push for clarity and push out anything that is unclear, confusion either in thought and also in execution. One of the folks that I talked to that worked with you about this phrase, Josh Redfern, he said that this phrase became really liberating for him, which is really interesting to hear because it forces you to make a call and to be aligned and make sure everyone is on the same page. I guess thoughts on just why this concept is so liberating.

**Tomer Cohen** (00:11:04):
It goes back to little alpha types or type A folks who are just so attached to not getting it wrong. When you need to move forward and it's not about being right or wrong, it's really about not being confused and making sure everybody's pulling in the same direction. That is actually really liberating. And when you know that the whole idea is to have a Socratic conversation about what you're trying to do, then coming to the table with some kind of half-baked ideas or actually not an opinion, I think actually brings into a conversation of feedback.

**Tomer Cohen** (00:11:39):
But you have to manifest it through, if you just play this through but then you potentially playing the right or wrong game, that's really poor. I also think it's the best way to learn. If you don't know exactly what you're doing, how can you learn back from why you made that decision? So if you had a clear understanding, ultimately it's a growing organization. This is not a one-off project. We're going to build many projects in the future. So if you're not sure about what we're trying to accomplish, how can you know what you learned from it?

**Lenny Rachitsky** (00:12:09):
I love that. So the idea here is make it very okay to be wrong, but make it not okay to be confused and not clear about here's what we're doing, here's why we're doing it. Everyone's aligned exactly on the same idea.

**Tomer Cohen** (00:12:19):
How can I have a product conversation if I'm not sure what you stand for? It's really hard to have a Socratic conversation, really hard to have. If you ask folks how many time you left a meeting in corporate, it could be a startup or larger companies and you are not sure exactly what was the problem discussed or what are next steps. More often than not, they'll raise their hand. That for me is a waste of building time. So actually that agitates me in another great way, but when I come in and I'm proven wrong or there's a strong challenge or argument, again that's a little bit my love language, I actually enjoy those. I think we leave the conversation much better.

**Lenny Rachitsky** (00:13:09):
You said there's some other Tomerisms, what are some others just that you can share?

**Tomer Cohen** (00:13:14):
This is, again, this is classic to building large organizations, but I actually believe in, especially when it comes to products, to really set ambitious goals but then try to over-deliver on them, really set what are you trying to, there's almost like the opposite where people are underplaying it and over delivering, I don't understand what you're trying to do. For me it's like we are here to make an impact. We're here to really set our goals to something really massive. And when I'm trying to visualize this, I see a mountain, you see the peak and the peak exactly how it looks like, you see base camp, you know how to start and maybe the middle of the mountain is kind of blurry, but you'll figure this out. But at least you know the peak you're trying to share, share the peak, share where you're headed to and I think it's just a much more exciting way to build product. It's a much more inspirational way for folks to be part of the product process kind of thing.

**Lenny Rachitsky** (00:14:07):
I was just chatting with Vlad who worked at Airbnb for many years. He was actually my former manager at Airbnb and he reported to Brian for a long time and we talked about this trait that Brian also is really good at, is just setting crazy high goals. 10 X the goal that you thought you had and what would it take and it worked really well for Airbnb. So I love that you're doubling down on the same idea. Is there an example that comes to mind of one that you, some ambitious goal you set internally at LinkedIn that people are like, "No way," and then it ended up being effective?

**Tomer Cohen** (00:14:35):
Actually there's a lot, when I think about our LinkedIn feed and thinking about when you started off, it was hard to imagine what that product could be because it was more of a promotional in nature product and I was like, no, is going to be a place where millions of people and not just tens of millions will come daily. And that's insane. That makes no sense based on the numbers today, but I don't start building from the numbers today. I start from [inaudible 00:15:04] I start backwards. I'm like, what this could be, what is the potential here? So how many professionals exist in their role? How many of them would love to find a place to share and engage with content? And this is my starting point. I start from there. So I don't start from the existence to set my ambition. I start from what this could be like based on really inspiration and excitement.

**Tomer Cohen** (00:15:30):
Again, it's not detached from reality completely, but it's also not hooked to it. But then Basecamp could be a good start. You're not asking you to make it the next day, but if you don't have that ambition, there's no way you're going to hit that. There's just no way. And there's so many products across LinkedIn. LinkedIn is a 20 plus year old company that many folks did not give it a chance in almost every phase of it. And I think if anything, it's one of those that just keeps getting better and better every year. And part of it is you keep the landmark on. This really has the potential to do so much for so many people. It's really an economic platform. So if you play from that, one billion members, that's actually pretty small where we can actually go,

**Lenny Rachitsky** (00:16:17):
I'm so happy you went to this example. This is exactly where I wanted to take the conversation. I was very much in the camp you described of LinkedIn, how could LinkedIn possibly become a place that I want to go and browse a feed and post content? As you probably know, for the longest time it was felt like this cringey place as you said, where everyone comes and promotes themselves, "Hey, I got a promotion or here's my company's new launch." And I think it's so underappreciated the turnaround that has happened within LinkedIn. I use it, I'm a multi-day, multi-Dow user. I check it at least 10 times a day. Most of my traffic to my newsletter comes from LinkedIn, not Twitter where people think, it's absurdly underappreciated and it's I think underappreciated what it took to make this happen.

**Lenny Rachitsky** (00:17:02):
And when I saw you guys starting to try to make it a place people post content, I was like, no way, this is not going to work. Why would people want to share stuff on LinkedIn? And it's working, it's amazing. So I want to spend some time here and just try to go inside the strategy that you guys put together to here's how we're going to make this happen. Come, we've shared, you have this peak of here's what we could become. How did you actually turn this around? What was the strategy behind it?

**Tomer Cohen** (00:17:26):
Yeah, by the way, I'm glad you're finding great audience and traffic on LinkedIn. I think your content, actually, your content is exactly what we're trying to build for, it's expertise, it's advice, it's people you can learn from and it's also the views that really matter, not just the volume that matters. I think if we take a step back, there's so much conversation about zero to one products or scaling products, but you don't have much conversations about minus one to one products, like turnaround products. And I think there's obviously the perception of the market you have to deal with, but people I think in minus to one products, at least from my experience and we had a few at LinkedIn, pages is another one, helping businesses build their presence on LinkedIn. What you find is it's actually, most of the time it's internally harder to do because there's so much entrenched flows and processes and metrics that people are using on that specific area.

**Tomer Cohen** (00:18:21):
So you almost have to change the inner workings of the system to make it work. Going back to the analogy of the mountain, if you start from the premise that I deeply believe in is that LinkedIn ultimately is a platform for economic opportunity that sits on top of a very strong social graph. Then really almost every aspect of economic transaction is possible and knowledge transaction is one of the most powerful economic transactions you can have. It's the biggest accelerant for an experience and we were always very strong at helping people get a job. We have seven hires per minute, but as we were building more and more knowledge and part of it was we bought LinkedIn, we bought Lynda to make it LinkedIn Learning a while back. Today we had 140 hours of learning every minute on LinkedIn happening across the feed and LinkedIn Learning, it's pretty powerful.

**Tomer Cohen** (00:19:13):
And the transformation to the LinkedIn feed was exactly like you said. We actually were the first company to have a social feed, but I think we started wrong. So we started with basically activity feed. So it was like who changed what, who changed the job, who connected to who. It was more of like a tracking your network feed and it became more promotional in nature. So in a way just letting that be just naturally just moved into more of a promotional type of feed. And what we've done is we've shifted dramatically into building, actually this was one of the things I was excited about. So after I was leading the mobile team, there was no feed team, there was no unified feed team, there was no feed PM to an extent, I asked to do this role, nobody cared about it. I really believed in it.

**Tomer Cohen** (00:20:07):
I have strong conviction about what I could do there. And I asked to do this role and we assembled the team around it and one of the main things we've done was to really set the new purpose for it, which was this is not a springboard for other products, this is not a traffic jumpstart for, it's not an app self feed, it's really about people that matter talking about things that I care about professionally. It's about knowledge exchange. It's about how can I get the right views to the right experts in a way that actually helps them build a reputation and build their business. And then we started from there backwards. So it was basically setting that ground for that mountain peak that was nowhere to be imagined at the beginning and making our way backwards.

**Tomer Cohen** (00:20:55):
The first thing we did was really making AI first. So the AI team back then was completely centralized. It was not part of any product team and we brought it together with one unified AI first team. And the belief I had was ultimately the engine of the car was AI and that was almost like de-prioritized or delegated to a team that was not unified in objectives. So bring that in. And then I spent most of my time on objectives and algorithm features and data training, which led me into my passion about training product people to be a first product people. And that was a big transformation there, really shifting and we had incredible AI team, but they were completely, actually, it was a confused operation. They were building something for a whole different purpose and we were trying to aim to that mountain peak and they were putting in a different direction. No bad intent, that's what they were told to do.

**Tomer Cohen** (00:21:56):
So bringing it together into this SWAT team was the first thing that actually was extremely powerful, but then became the hard work. You have a product that works in a certain way and you almost want to change its DNA altogether. And it was very hard because whenever we were trying to run experiments that were mass in scale, I told you everybody was relying on the feed for their traffic. It just scared the whole system because numbers were shifting up and down and teams were freaking out about meeting their goals. And then I realized that was just spending my time in escalations instead of actually building a great product. So what I did was I carved out two million members and I said, "Those are my members. I'm going to focus on building that mountain peak. I'm going to build for them."

**Tomer Cohen** (00:22:45):
Full liberty and doing whatever, it doesn't hurt numbers, giving the scale and really focus on building a great experience for them. And it wasn't overnight and it wasn't over a week, but over the course of months we've seen dramatic behavior change for those members, almost like secluded, like a country of people that were seeing a different experience of LinkedIn. And once we saw that you actually had strong evidence that wow, if I bring this in, we don't need to spend time talking about how this pie gets slotted between different teams. We can actually grow the pie. The experience just manifests itself in a whole different way and that was a big change internally. It wasn't overnight, but it was really powerful in getting everybody around to see, wow, we have this cohort that is doing extremely well, which was a randomized cohort and then how we can bring it out.

**Tomer Cohen** (00:23:39):
I've also done some crazy things. We've done some negative tests to prove some stuff out, test for the sake of learning. When you run something that you can show that if it's just a promotional feed and you played it out organically over time, engage with the tier rates. We ran some really important ad tests as well, but we're really shown separate, almost like we carved out the different product and we showed that this could work and then we brought it out to the main experience for everybody else. And then that was, I would say the inner workings of minus one to one. Then the scanning part really became when we started to focus on professional opportunities. So when people actually share, how do they get the right views into the experience?

**Tomer Cohen** (00:24:19):
We don't compete for volume, we're not in the same category of Meta in terms of the skill there, but we will compete all day long for the right people seeing your content. In fact, I think in many ways that's the most powerful part of LinkedIn. How do we make sure that it's professionally productive and safe conversations? How do we trade off bad engagement all day long? In fact, when we started shifting the AI objective from click through into more downstream conversations, spammers actually took notice as well. So they were jumping over the LinkedIn bandwagon, so we had to spend a lot of time removing bad activity from LinkedIn, but that's been the evolution of this process.

**Lenny Rachitsky** (00:25:02):
That is amazing. There's so much I want to dig into here. Okay, so this two million user carve out that you did, basically everyone was just like, what the hell are you doing to our metrics and goals? You're causing all this trouble for the business. Why is this team hurting our metrics? So that was basically a group that those two million users are the only ones that saw this new updated feed and were they removed from everyone else's metrics so they weren't fluctuated as much or was it just?

**Tomer Cohen** (00:25:26):
They could be kept in the overall because it wasn't as important.

**Lenny Rachitsky** (00:25:29):
It was a small percent?

**Tomer Cohen** (00:25:30):
Exactly.

**Lenny Rachitsky** (00:25:30):
Okay. Got it.

**Tomer Cohen** (00:25:30):
But for us, they were like the world we were basically able to prove with them.

**Lenny Rachitsky** (00:25:34):
Got it. Okay. That's so smart. Okay, so basically you just decided we're only going to move your metrics a little bit, worst case if we stick to it.

**Tomer Cohen** (00:25:41):
I felt that I tried for a few months to play on the overall experience with everybody, but it was really hard, almost like impossible because you have an organization that is so tied into how things work that I was just hitting walls after walls after escalations and it was just unproductive.

**Lenny Rachitsky** (00:26:00):
And this is before you were a chief product officer where you could have just said, "We will take this bet. We know this might hurt metrics short-term"?

**Tomer Cohen** (00:26:06):
Yes.

**Lenny Rachitsky** (00:26:07):
Okay, got it. That makes sense. Okay, the other piece, so just like lessons I'm taking away from trying something like this that's an ambitious bet within a company, is put a PM and a team on it with a goal that feels like a core part of the success of just like somebody's ass is on the line to doing this thing.

**Tomer Cohen** (00:26:23):
Always.

**Lenny Rachitsky** (00:26:24):
Always, with focus on this one problem. And then there's the way you described there's this goal for this feed, but how did you actually try to turn that into a goal or metric or a KPI, what was that in the end, was there something there?

**Tomer Cohen** (00:26:35):
Yeah, this is an interesting one, one thing we've done actually because the feed is the first thing you land on. I can't just count how many folks engage with the feed because then I'm counting bypassers kind of thing and bystanders that are actually coming into the experience. So we actually started to look at the more, we go a lot into active, engaged and high value engagement. So we go downstream, we kind of put the onus on looking at more downstream engagement there and we build that as the feed engagement. So really trying to show that we're not just counting some overall whatever it is [inaudible 00:27:09] or sessions at the top level. That's not really helpful because any shifts can help there, but really setting targets for that. There's obviously it's a marketplace, so there's the creation side, there's the consumption side's, there's making sure that's healthy and engaging.

**Tomer Cohen** (00:27:23):
There was so much we went into that, but I think the best thing was it's almost like you carve out. I think when you do minus one to one, it's really hard unless the CEO says I don't care about how the company performs for the next two years, we're going to go for it. If you want to keep the site keep growing and the experience keep growing, carving out and almost like sending very specific unique metrics but then could easily be extended out once you show it was in retrospect the right way to do it.

**Lenny Rachitsky** (00:27:52):
And then to give people a glimpse into the way your brain works to identify this is a big opportunity. So you talk about just like I see there's a lever that we're not investing enough in and I see this big opportunity to grow of LinkedIn. How did you decide I need to go and bet on this thing and lead this team and I think feed is a huge opportunity?

**Tomer Cohen** (00:28:10):
I start from beliefs a lot. So I start from what do I believe this could be or where I actually came to LinkedIn this way. In fact my biggest change in my career was when I moved here and I shifted to more like what do I care about? What am I excited about? What do I have conviction on? I think it's really hard to be a strong product leader without having strong conviction about something. So I start there and in fact coming to LinkedIn as an example, when I came into lead mobile, LinkedIn was a desktop first company, then mobile team was an offshoot of entrepreneurs. I came from a startup that I ran and it wasn't a big, it wasn't, it was like, okay, I want to do mobile, I guess fine. It's like noise at this point. Same with feed. And same when I shifted into ads, I felt really strong about the ability to flip that into a great way for companies to grow.

So for me it starts with a conviction of where things could go, what do I believe in? I believe LinkedIn can be an incredible superpower and daily use case for every professional in the world. I believe knowledge sharing and knowledge exchange is the most amazing way to grow your career and to grow your business. So that needs to be a strong pillar of the experience, what didn't exist before and what is better than the feed experience, the home [inaudible 00:29:32] to actually build it. So I don't get attached to what did not work in the past. That's not, I don't know, maybe it's a mistake sometimes, but that doesn't stop me from thinking about the future.

**Lenny Rachitsky** (00:29:44):
How do you actually make time to think like this? A lot of people are listening. They're just like, "Okay, I want to think about what could this become?" Is this just the way brain works? You're always thinking what could this be? Do you set time aside to think quarterly or yearly, what could this be if we really made this amazing?

**Tomer Cohen** (00:30:02):
It's a good question. I haven't thought about it. It's like a process I do. I don't sit time aside for this, but I'm very reflective. I try to focus, love the conversations on the dream, what ultimately are we trying to achieve? I think LinkedIn has a great process called vision to values that started from our former CEO, Jeff Winner, which is like if you said this is for company or for a product, which is if you're successful, what change would happen in the world? Which I love. It's just a great phrase. It's just a great empowering phrase. So I actually tend to spend a lot of my time there. I'm also very optimistic in nature. Again, sometimes it's probably, I'm not best for any role perspective. You want somebody a bit more pessimistic about the future, but I tend to lead with beliefs versus evidence. I try to prove my beliefs with evidence, but I don't lead with evidence.

**Lenny Rachitsky** (00:30:59):
Awesome. Okay. And then maybe just one more tactical question about this shift that I wish I could spend hours on because it's so interesting, the success you had making the feed so engaging. Is there anything tactically that would just like wow that would really made a big dent in people wanting to come here and post and share interesting content, like a feature or a part of the strategy that really made the feed social?

**Tomer Cohen** (00:31:21):
I think the lesson would be, for me that was the biggest learning going into AI first that gave me the why is AI so core and why I got to make it a priority all the way to my role today to make sure the rest of the organization thinks AI first. The understanding is is that in a marketplace, if I'm able to satisfy your need on the other side, then it's magic. So ultimately it came down to AI is the ultimate matchmaker. It's underutilized, it's misunderstood, it's run separate from the team. And in a marketplace it's all about value exchange. And if I'm able to do value exchange really well, then people will come back and they do and they engage and actually they come back even more and they spend more time. So for me it was that depth into AI first as the engine that moves the whole organization forward.

**Lenny Rachitsky** (00:32:12):
Got it. So essentially making sure the content you are seeing is the most engaging you could see based on using AI to make the algorithm really smart?

**Tomer Cohen** (00:32:21):
On both sides. If you are the creator, if you're the person sharing, remember this was a while back, I think it was about the former Olympics and person shared an article on LinkedIn about how they should not call it Olympics, they should call it the commercial Olympics because it's all about commercials and less Olympics. And then they sent me this amazing screenshot about how NBC execs who were covering the Olympics were rearing the post, this first to them was gold. It was like, "Oh my god, my content is influencing, people are seeing it, people that matter." So that was really key. Making sure that when you share something, you share your expertise. The right people on the other side are relevant to your content and they see it. That could make your day or your week or actually it could make your living in many ways. And then on the receiving side, when I come in, it's the things I'm excited about seeing.

**Tomer Cohen** (00:33:17):
It's the things that are relevant, the reason your content resonates so much with other people, I can actually take your podcasts and I can apply them at work. What could be a better way to learn? I'll give you another example, which was very, very recent. I met with a known professor in this field and he shared with me how over the last year and a half he started using LinkedIn because somebody told him, "You have a great content, why don't you just post it?" I was like, "I don't want to post on social media." I'm like, "Oh, LinkedIn is different, share on LinkedIn." And he was like, "I post daily. I have so much content over the years, I post daily."

**Tomer Cohen** (00:33:51):
And he was like, he basically told me this is unbelievable in terms of economic opportunity I'm getting. He's like, "I'm getting speaking engagements that are roughly half the salary I make in a year here at the university. Just by people seeing this content and getting to the right people. I was invited to advise prime ministers on their investment strategy for the country and I've been teaching for 20 years, but this platform just completely elevated my ability to reach and influence people." That's the magic and that's the value exchange and that's the kind of matchmaking at scale.

**Lenny Rachitsky** (00:34:29):
**Tomer Cohen** (00:36:14):
I can spend days on this. This is so important. It's actually so important for me, it's a key focus for me. Actually to your point, long before it became cool to talk about AI in the last two years, and in fact I think I've learned this on myself. So when it comes to the feed, I took the role of the AI product leader, it didn't exist in the company. There was no person that was ever from a product perspective thinking about AI. I think it start with the belief, like we talked about before, I think every technological revolution has dramatically changed the way we build. And AI arguably is the biggest one in our lifetimes. And when I say AI first, it's not about a tech, it's a mindset. It's a start with strategy. It is rare. Maybe now you'll see it, but it was rare two years ago to see anybody in their strategy talk about the role of AI and how they build with AI.

**Tomer Cohen** (00:37:02):
Then it goes to the product and then the talent itself you hire, do they actually think this way? The analogy I would give to people is imagine a river rafting boat and you have everybody on the sides holding the pedals and adding accuracy, adding speed, but there's the guide on the back and they're holding those two pedals. Those two pedals navigate pretty much the boat and those pedals are AI and the guide better be you. And in most cases in companies the guide was somebody else, it wasn't the product leader. So then the question is, if AI actually is directing your product or success and it's the biggest factor and you as the product leader is not the one holding those two pedals, what are you doing? And then I realized that it was a bit of a lack of education in that, there was actually most product users used to think of AI as this black box, magic spells that they don't know how it's working so they're delegating.

**Tomer Cohen** (00:38:01):
And obviously that's as far from the truth as possible, but there's so many ways to unpack it. When it came to the feed where I push for example, more specifically for the teams, he doesn't stay as an AI first, there's the objective. I would ask him, "What is the objective of the algorithm?" I would challenge you to ask folks more in the folks who are leading products specifically with algorithms inherited built into them, what is the objective of the algorithm and can you write it down for me on a board? They should be able to do so, ultimately it's a mathematical formula and then it's like what features have you added to the algorithm? And this is not user features, this is specifically what parameters to learn on and then what investment do you have in data collections and fine-tuning?

**Tomer Cohen** (00:38:49):
Now everybody talks a big game about fine-tuning but again, two years ago, fine-tuning was something that the product folks thought the engineering team was supposed to do. No, it's the whole organization. In fact, you can build a whole strategy just on data collection and fine-tuning and your product will see tremendous success or you can delegate it and it will never happen. So in many ways that was bringing into the fold, in our phase one, which really started around 2016 for me and every team I went to, the AI component was the area I spent most of my time on. I hired people for that. Product leaders, I spent most of my, back to how do I spend my resources, most of my resources there. And it was my top priority all the way from strategy to talent. And ultimately with the last couple of years we've seen this metamorphosis of AI and this incredible new wave and we've done a pretty big change there as well over the last two years.

**Lenny Rachitsky** (00:39:49):
I love this. So I took some notes on what you're talking about. So the big message that I'm taking away so far is as a product leader, you need to think about things that you thought the engineering leader had to think about or the ML engineer was thinking about things like what is the objective of our algorithm? What are the features that we're building into it? What is the data collection strategy? How are we fine-tuning it? As a PM, you should be asking these questions.

**Tomer Cohen** (00:40:13):
In fact, you should go all the way to infrastructure. You can have massive lifts in your product outcomes and goals if you probably enhance your infrastructure. How many product people talk about the infrastructure they have? Not many. Influencers, those are things ultimately your goal is to win with your products and build a much more experience to your members and customers. Literally just changing the infrastructure on top of what you build. That could be the biggest lever than you building another button or experience for your members on the top of it.

**Lenny Rachitsky** (00:40:45):
So say the PMs at LinkedIn, are you encouraging them all to, how is AI integrating into what you're doing? How do you just set this up so that teams do this well within LinkedIn?

**Tomer Cohen** (00:40:57):
Yeah. So coming into this role in early 2020, we basically established an AI academy. Every PM had to go through training just like we did mobile in 2014, we moved the whole organization to be mobile first, so everybody had to go through this process. I spent a lot of time in my reviews on the AI strategy, the objectives. We make sure there's actually AI practitioners on the product side who are strong, who can teach. So we in waves started to build more of expertise and distinguished leaders across that can actually bring this learning across the board. And then in fall 2022, when we all know what happened, at least a few months after, but we started early, we completely changed our entire almost product operations and portfolio so we can focus on this new wave of AI with LLMs in the front.

**Tomer Cohen** (00:41:47):
So LinkedIn has been working with AI very closely since the early days, but mostly as a matchmaker. So it was the matchmaker for our marketplaces. Somebody looks to hire this dream candidate and then you have a candidate looking for the dream job and AI would be the one doing the matchmaking. We talked about the knowledge sharing on the feed. It happens in our commerce platform as well. But AI, I was in the background so I never saw it, it was making those matches. And then with the new level of AI, we actually brought AI from the back of our marketplaces really as the matchmaker to the front. And one of the things we've done there was really asking the teams to completely revisit their entire roadmap.

**Tomer Cohen** (00:42:35):
This is fall of 2022, the world will learn about ChatGPT for several in March 2023. So we had a nice beginning there in terms of getting started. And the goal there was let go of what you've built, let go of your roadmap, go back to the drawing board with what are you trying to solve for, back to that idea of clarity on your problem statement and now tell me what's the solution. That's very much AI first.

**Lenny Rachitsky** (00:43:03):
It reminds me, so one of the folks I pinged about you, Shira Gasarch, she used this quote about you, "Maybe you were made for such a time as this." And it connects to a lot what you're talking about where you've been thinking a lot about. And back then it was called machine learning. It wasn't called AI for a long time and now it's AI. The fact that you've been on this so long is just a perfect synergy for now. It's working its way into everything.

**Tomer Cohen** (00:43:27):
Here's an example that I think sometimes to bring it to people in a more visceral way if you've been building products, product leaders are used to very much dictate the experience they're building. I want this experience to be exactly like this. I want the member to come from here and this is the options they have and I want them to be able to select this and this will be my default and I want the onboarding to progress this way. And I think this is one of the biggest shifts with this when you become an AI first leader, is that there's a realization that you don't control the experience anymore, you control the ingredients. It's almost like being a chef at the restaurant and you're used to deciding every part of the dish. You're deciding everything from the ambiance to the temperature of the broccoli and then this new technology comes in and say, just give me the ingredients, give me the guidelines of how you cook and now I'll take care of it. I'll take care of it for you.

**Tomer Cohen** (00:44:24):
For many folks, this is a very scary feeling, they're not used to letting go of the control. Obviously you build safety guards and responsible AI around it and that's super critical. But at the essence of it, AI is not deterministic. So giving it the rope to learn and do that experience for you ultimately would become much, much better. You have to have that belief going into it.

**Lenny Rachitsky** (00:44:50):
Along those lines, I'm curious if there's anything you do to avoid, everyone's like, cool AI into everything and then all these stupid things ship, then no one wants. I saw this hilarious meme of like, oh wait, we built a kind of dumb artificial person. Let's integrate it into everything. Now it's everywhere. Is there anything you've learned about just how not to ship stuff that isn't great?

**Tomer Cohen** (00:45:11):
I can tell you what we've done here and we've failed a lot, but we learned so much along the way. When we started it in fall 2022, literally started with me calling the leaders coming to the room and we talked about, okay, let go of your roadmaps, like what we've done, great, but I want to let go of the roadmaps and I want to instead go back to what you're trying to solve for and let's meet in a couple of weeks and tell me how you're thinking differently about what you're trying to solve for knowing we have this technology in a role for us. So that was a starting point around just setting out some ground and principles around it. But we didn't start with new objectives to solve. So it wasn't like, "Oh, we have this cool technology, what can we do with it?" It was like go back to the objectives you were trying to solve and now with this technology, how can you do that objective better?

**Tomer Cohen** (00:46:00):
The second part is we actually allowed teams to run to really inspire creativity. I didn't want to contain them. I wanted to get them really excited about the potential here. And even some things we're building duplicates for a while of similar ideas but done differently because part of it was I was learning. I was very excited to see what people would come up with and see how they can do it. And there was no playbook for building this really, really well. And in many ways we were writing the playbook. Prompt engineering became a playbook internally for us, which every day was amazing. How do you cognitively reverse engineer the brain a little bit? That was incredible. In fact, a lot of things we've learned so much ahead of the market and even shared with OpenAI and shared with Microsoft. But then after that period of just everybody getting excited and building, we basically brought it down and we did top-down got it.

**Tomer Cohen** (00:46:53):
So we basically picked back to the objectives we had out of everything that we've seen, those for us look like the best four biggest best we want to want to aim for and we want to converge resourcing across it. So no more everybody's building whatever they want. We also, capacity is also constraint. Cost is a constraint. We want to start bringing them together. So we really much allowed people to, I would say in many words, diverge. But then I would say several weeks after converge, but we had a lot more excitement and understanding about how this thing works and what we can actually do with it.

**Lenny Rachitsky** (00:47:30):
Love that advice. Basically give people a bunch of time and space to explore and experiment R and D and then as a top-down strategy, pick through this.

**Tomer Cohen** (00:47:40):
The top-down, we were like, there was literally, usually I do product jams for every multiple topics we have throughout the quarter. I just did every week. I just reviewed the five bets we had on a reputed basis, nothing else. Because it was so important for them to understand that this is what I care about and we had to be focused about it.

**Lenny Rachitsky** (00:48:00):
It feels like that space to explore and go crazy is important because otherwise people at the company are going to be like, "Oh, I wish, there's so many, this thing I want to try with AI, we should try it." And they'll just be pissed because they don't have time to work on it.

**Tomer Cohen** (00:48:11):
It's a great point. It wasn't my intention, but I love that you're saying it. It's a great point because I think it gives them that. I was actually, for me sometimes almost in a, maybe too much, but I try to focus on learning. I was trying. I knew just going like this, we weren't going to learn a lot, but having people come back and trying different things and slightly going crazy and pushing the boundaries, we would learn so much. So for me it was learning, but I love the motivation around, also allowing them to have the energy.

**Lenny Rachitsky** (00:48:39):
It relates to another point that a recent podcast episode I had with Brian Chesky where he introduces chaos sometimes when things are feeling too comfortable, when roadmaps, everything's calm, everything's on schedule. He's just like, "How do we do this in one day versus in two weeks? Let's just see what happens."

**Tomer Cohen** (00:48:56):
Yeah, because I think people get, there is just inertia, people get into their, it's human behavior, people get into their lanes, they start to feel really comfortable within the lanes, and then they don't know that there's a different way to do things and you have to almost externally invoke that or trigger that.

**Lenny Rachitsky** (00:49:13):
If we think about just your career arc, I'm zooming out a little bit. You helped create the mobile experience on LinkedIn. You built the feed initially and now you're in front of AI. I could see why you're so successful at LinkedIn. I was talking to folks about your career arc at LinkedIn and you basically went from senior PM to senior PM number two, to group PM to director, to senior director to VP, to CPO in not that many years, it's a pretty meteoric rise. I wanted to spend a little time here and I want to maybe start with the question of just if you could give one specific piece of advice for someone looking to advance in their career based on what you found to be really effective, what would that be?

**Tomer Cohen** (00:49:53):
I realized everybody's in every stage in their career and they have different ways to think about the role then what they need. Maybe I'll just share about my journey, what worked for me instead of giving more of a general advice. First of all, I feel super fortunate I'm building. That's what I love doing. I love building, I love working with builders. Sometimes I'm like, I get paid for this, this is insane. But I love my craft and I love getting deep into it. So in many ways I think the things I'm excited about is the things I'm doing. When people are starting off, I usually really focus on learning from great people. People you talk to or have amazing mentors and managers. Some of them don't even know there were every mentors. It's not like a mentor officially. I try to pick up things from people all the time, and that's been just a remarkable experience working with great people.

**Tomer Cohen** (00:50:50):
And in many ways, a lot of those great people actually allowed me or empowered me to take on some bigger challenges. So I can see forks in the road where if it wasn't for that person saying something very specific, probably would've done something differently. And it just made me think a lot. So I really tried to absorb learning from great people. But by far, for me personally, again, this is very personal versus generic advice, it was when I moved here, I was an engineer for many years before I moved here for graduate school in 2008. And I always loved building, that was there from a young age but when I moved here, I realized my career path was very much dictated by one thing. It was like, what's most in demand? What's most challenging? And how do I do that? It was very childish in many ways.

**Tomer Cohen** (00:51:42):
It was not dictated by me, it was in a way dictated by society. So what's the toughest engineering role? What's the best company to go into? What's the best army unit to serve in? And I fell a lot along the way, but I always kept going. And then when I came here, there was a really big challenging for me personally around what do I care about? What matters most to me? And that was, again, it's very personal in many ways. It was very much for me, an impact on learning and actually how do I create impact more broadly? And I shifted 180 in how my thought process used to go. It was less about what was out there and exciting and in demand and challenging and it was more about where did I have strong conviction on, what was I passionate about and where did I feel I could make a dent and learn?

**Tomer Cohen** (00:52:35):
And that was my path forward. So after school with a student visa and massive school debt, I decided to start a company which was not a very intelligent decision based on my economic circumstances, but I didn't care. I was like, this is my new path. And then I got into LinkedIn. I didn't apply for a job. I met with who was in my role today, Deep Nishar, at that time and we talked and I said, "This is how I think this is how LinkedIn mobile should be built." And he was like, "Okay, how about you come and build it?" I was like, "Amazing." So I didn't apply to LinkedIn and then at LinkedIn I was always like, this is what I want to do, this is what's exciting for me and this is the dent I think I can make and this is my plan for it.

**Tomer Cohen** (00:53:21):
I don't know if this is a recommendation for everybody, but for me it's worked really, really well. It was really pursuing the conviction I had and my excitement and bring that to the fold with people. I do think that in products, in building product, if you're not genuinely excited about what you want to build, you don't have conviction about it, it's going to be very hard for you to make a big impact.

**Lenny Rachitsky** (00:53:45):
That's also a similar theme from my most recent podcast with Vlad of just, if you don't actually buy into the mission of the place you're working on, you're not going to have a good time.

**Tomer Cohen** (00:53:54):
Yeah. Say for product people, it's a very fortunate position. I always tell people, if you're in one of the most fortunate positions you can have, because if you just measure thing, for you, just measure based on your career and so on, people are going to evaluate you based on your actual work. It's a very special place. Nobody cares about your title, who cares? It's not. Maybe the company name for some people matters, but for the most part, it's about the impact you created with the products you've built. If I think about somebody's resume, I think if it was a product resume, it would be the products you built and the impact you had with it. I don't care about the companies you worked at, I don't care about the logos, I don't care about the titles. Slightly, again, not to overextend, but somebody, it's almost like an artist, right? It's like whatever, a musician, it's the albums you took out and how well they did. And I think for product people, it's a very fortunate place to be that you get measured based on the impact you had.

**Lenny Rachitsky** (00:54:53):
It sounds like a LinkedIn feature idea right there. I feel like if there's any company that could make that happen, it'd be you guys. So some of the takeaways here essentially is try to index towards what are you actually excited about and motivated to work on and driven by versus where it's the most amazing company to work at are the most challenging problem.

**Tomer Cohen** (00:55:17):
Yeah, I think sometimes great companies have great opportunities for you to have dent at scale, but you need to be the one doing it. If you are thinking about, I don't know, a title or that did not, once I did the change into my excitement around impact, that's been at least my yardstick. When I look at people that I talk to or interview internally, the first thing to my mind to me is like, "What did you build and what did you learn and how well did they do?" That's what I care about.

**Lenny Rachitsky** (00:55:50):
I imagine there's also people on the other side where all they do is work on things that really exciting to them and they could use a little pushing towards the other direction of what's actually important in the world.

**Tomer Cohen** (00:55:59):
100%. If you tell me, again, everybody has their different, if you tell me, "Hey, you can work on something super exciting, but it's on the fringes of the company or you can work on something which is a bit more grindy, but it's on the core of the company." The latter, no doubt, for me, impact first.

**Lenny Rachitsky** (00:56:19):
And just listening to the story you've told of the things you decided to focus on as a clear example that you saw, hey, there's this huge opportunity in the feed, I'm going to go tackle that or mobile. So I think there's a lot of, it's kind of this Venn diagram is what I'm taking away of just what's important, what am I excited about?

**Tomer Cohen** (00:56:36):
Yeah, this for me. Yeah.

**Lenny Rachitsky** (00:56:38):
Awesome. Okay, so I note you got to run relatively soon, so we're going to get to our very exciting lightning round. But before we do that, is there anything else that you think would be interesting or useful for folks to know or leave them with? I know we covered a bunch of things already.

**Tomer Cohen** (00:56:52):
Yeah. One thing that actually I've now built it into a podcast, but something I'm really excited about is I don't think there's one way of building. Remember when the Steve Jobs biography came out, everybody read it. Oh, that's the way to build and that was unique to him. And one of the things I love a lot is when I look at great builders, they're all very distinct, they're all different. And I used to do this thing internally, I used to invite product builders of different disciplines and have a fireside chat with them. And I saw people across the company join not just PMs or designers, but folks across and I build that into a podcast.

**Tomer Cohen** (00:57:34):
I love your podcast. Mine is very different. It's more around what is their edge a little bit. This is from the co-founder of Pixar, Ed Catmull, to the CPO of Canva or Spotify, Roblox, but all the way to a chef, Dan Barber, who's the number one chef in the US for many years. And it's just everybody has their craft and they do it differently. It's called BuildingOne, I'm excited about it. It's a little bit of a plug right now, Lenny.

**Lenny Rachitsky** (00:58:03):
Please. Yeah. Where do people find it? Let's blow it up. It's called BuildingOne?

**Tomer Cohen** (00:58:07):
BuildingOne on Apple or Spotify.

**Lenny Rachitsky** (00:58:08):
Perfect.

**Tomer Cohen** (00:58:11):
It's short and it's really about showing you different disciplines from a chef to an animation director. And really the main learning there is everybody builds differently and you can be very successful, but it's very authentic to how they are personally, and it's how they push their craft to the limit. It's how well they've done their craft.

**Lenny Rachitsky** (00:58:29):
I love that. And something I super believe is just the power of focusing on your strengths and the things that make you a little different versus trying to become good at everything.

**Tomer Cohen** (00:58:37):
100%.

**Lenny Rachitsky** (00:58:38):
That's so cool. Okay, BuildOne. We'll link to it in the show notes.

**Tomer Cohen** (00:58:42):
BuildingOne, yes.

**Lenny Rachitsky** (00:58:43):
BuildingOne. BuildingOne. Okay. Amazing. And it's on all the podcasting platforms. Okay, great. With that, we've reached a very exciting lightning round. Are you ready?

**Tomer Cohen** (00:58:50):
Yes.

**Lenny Rachitsky** (00:58:51):
All right. First question is, what are two or three books that you recommended most to other people?

**Tomer Cohen** (00:58:57):
So I have this [inaudible 00:58:58] continuously, actually it's these three.

**Lenny Rachitsky** (00:58:57):
I love that you have them right there.

**Tomer Cohen** (00:59:02):
I can just tell you about them but-

**Lenny Rachitsky** (00:59:03):
They look very [inaudible 00:59:04] back there. Don't [inaudible 00:59:05]

**Tomer Cohen** (00:59:05):
I'm not going to destroy my study. So I love fundamentals. I love studying from fundamentals. So if you're somebody who starts in your career, my fundamental books is One Mindset. It's about growth mindset. It's about basically the ability to continuously grow over time in one sentence is the whole idea is our skills, our abilities are malleable, we can completely develop them, we can build expertise and craft and mastery and it's really a mindset change. And Carol Dweck wrote the book, was also my wife's manager, and that's how we got into that.

**Lenny Rachitsky** (00:59:05):
That's awesome.

**Tomer Cohen** (00:59:38):
So that's like our second religion at home. Second book is Thinking, Fast and Slow by Daniel Kahneman. I love behavioral economics. When I think about products, I think I always start from people, what is the member expectation? What are they trying to do? And this is the Bible for behavior. So if you're building front end products or even you're thinking about how you rally organization, it's an incredible book. Every page is like a stopper. You have to stop and think. And then lastly, and on the fundamental side is High Output Management by Andy Grove. It's like there's so much basics to doing good manager. It's like I think after you read this book, your managerial skills should start from a B. and then you can over time become an A. But beginning to a B is just a level of putting the effort in and knowing the best practices. So I think those are all fundamentally great books that I really like to give to people.

**Lenny Rachitsky** (01:00:41):
That's awesome. I love that they're right there behind you. Is there a favorite recent movie or TV show you've really enjoyed?

**Tomer Cohen** (01:00:47):
Yesterday I saw Bluey, you know Bluey?

**Lenny Rachitsky** (01:00:49):
I've heard of Bluey. My kid is not old enough for it yet.

**Tomer Cohen** (01:00:52):
Oh, you're in for a treat.

**Lenny Rachitsky** (01:00:53):
Okay.

**Tomer Cohen** (01:00:54):
I love Bluey. So Bluey is this animation series from Australia and what's beautiful about it's, I can watch it with my six-year-old, nine-year-old and 12-year-old, and we're all going to enjoy it. We're all going to laugh at the same point but at the nuance of the jokes. It's like when I think about a product, the way Bluey is built, it's built for the whole family, but it's built as layers. There's layers of dialogue and points that they're trying to get across, and it's all packaged together into one experience. So for me, it's amazing that I could sit next to my six-year-old daughter, we would both laugh at this, it's a sweet animation kind of thing, it's like a family of dogs. And she would laugh at the nuance of the point, at a different nuance. For me it's like that's a genius creation of how you build a product.

**Lenny Rachitsky** (01:01:50):
Speaking of Ed Catmull, the Pixar does that really well as well.

**Tomer Cohen** (01:01:50):
Totally.

**Lenny Rachitsky** (01:01:56):
And I feel like the story of Bluey is really incredible too. It's just like a random little independent group and they just are making tons of money with it.

**Tomer Cohen** (01:02:02):
It's just amazing. The dialogue there is one, it's slightly edgy but not too edgy. It's exactly right. It's just exactly right.

**Lenny Rachitsky** (01:02:10):
Okay. Makes me want to watch it. Okay. Do you have a favorite product that you've recently discovered that you really love, whether it's physical or in digital?

**Tomer Cohen** (01:02:17):
So I like playing guitar. It's amateur, I'm not that good, but I love playing it. And there is a combination I've done recently I really like, there's a Spark Amp I have, which allows me to play with effects easily, but that's not what I use it for. I can tune my guitar based on the specific song I like. See, if I want to do Pink Floyd or Metallica or Nirvana or David Bowie, I can get that tune easily. I don't have to be an expert. I can just download the tune to my guitar, which is so great.

**Tomer Cohen** (01:02:49):
And then I couple that with the, it's called the Ultimate Guitar app, and basically it doesn't give me the chords and the tabs, it gives me the other instruments, so I can get the drums going. I can get the, it's a violin going, whatever that is that's going in the band. As somebody who does not play so well and plays for itself and nobody's supposed to listen to how I'm playing because it's really just a way for me to enjoy my time. It's just an amazing, I would never get into any band, so this is the closest I can get to get to a band. So I love that combination.

**Lenny Rachitsky** (01:03:22):
I wish we could close this podcast episode with you playing your guitar, but [inaudible 01:03:26].

**Tomer Cohen** (01:03:26):
I don't know if that will help the ratings of the show.

**Lenny Rachitsky** (01:03:29):
No, man, that's amazing. Okay, two more questions. Do you have a favorite life motto that you often repeat yourself, find really useful in work or in life, share with folks?

**Tomer Cohen** (01:03:38):
From growth mindset there's a motto that I really like. It's called becoming is better than being. It's like the moment you think you achieved something is the moment you start to deteriorate down. It's like we're really trying to grow as human beings. We're trying to learn, we're trying to evolve. Product is a good example. I think the moment you think you actually mastered it is the moment that you become obsolete. So I love the idea that becoming is a better goal than continuously trying to reach some kind of level.

**Lenny Rachitsky** (01:04:06):
Love that. Final question, just to come back to LinkedIn. Is there a fun feature of LinkedIn people don't know about or should check out? Is there anything new that's like, oh, that's something you should try or that's something that might surprise you about LinkedIn these days?

**Tomer Cohen** (01:04:19):
Maybe I'll give a couple so I don't have to pick the best right off the bat. But one right now, we're heavily invested in video and it's doing so well for those creators. We talked about it like immersive video we can actually come in. We talked about for us, video is obviously a best practice right now in the industry, but on LinkedIn, the right views really matter. So highly encourage creators to think about their video play at LinkedIn. And then I think what we call our coach experience in some cases is so powerful. For job seekers out there, we have people hired on LinkedIn, there's like seven folks hired every minute. Job seeking is a lonely journey.

**Tomer Cohen** (01:05:02):
I was actually in a session recently meeting with job seekers and I was talking to them and one started crying midway into the session because they said, "I cannot share my journey with anybody because I feel like I'm alone in this. People don't get how hard it is. I feel very accomplished, but I can't get the job. And I wish there was a buddy, I wish there was something that I could talk to brainstorm with who wouldn't judge me, who would just be trying to help me without paying hundreds of dollars to a coach of some sort. I don't have that money."

**Tomer Cohen** (01:05:38):
And in many ways, when we walked him through the job seeking experience, the coach experience, we build this coach aspect where you can go to any job and you can riff on the job with this really new realities LLM that is tailored to you, personalized to you, private to you, everything from your fit to how to best apply, to consulting about different opportunities, to comparing this to others, to feeling supported. So when we talk to people around, we always, I love to measure the impact of our work by emotion, when we talk to job seekers after that, it was basically the sense of I felt supported. In many ways, getting to that ability to remove the loneliness is amazing. It's a little bit like people need to, we're making it more and more visible and more and more ramped to everybody to a certain point. But that's a really powerful way to just humanize the job seeking experience for everybody.

**Lenny Rachitsky** (01:06:37):
Awesome. And to find that it's just, is it called LinkedIn Coach or?

**Tomer Cohen** (01:06:40):
It's in the job. So if you go to the job tab on LinkedIn, actually just a few weeks ago, we just put it on the top. So you don't have to go to the specific job and find it, you can just start there and engage with it.

**Lenny Rachitsky** (01:06:50):
Awesome. Just to give a quick plug to a future podcast that's coming out, there's this book called Never Search Alone, that we're going to have the author on the podcast soon. And it's all about the same idea of having a buddy that helps you search for a job.

**Tomer Cohen** (01:07:01):
Yeah. When I think about the future of AI in the sense of belief, that relationship is going to be sacred, the relationship between AI and the human is going to be [inaudible 01:07:10]. Do you know what Nomophobia means?

**Lenny Rachitsky** (01:07:11):
Nomaphobia? No.

**Tomer Cohen** (01:07:14):
Nomophobia.

**Lenny Rachitsky** (01:07:15):
Nomophobia. No. Being afraid of something.

**Tomer Cohen** (01:07:18):
It's the anxiety of being away from your phone.

**Lenny Rachitsky** (01:07:18):
I have that.

**Tomer Cohen** (01:07:25):
I think we all. I think we're in for AI-nophobia at that point, where you're going to get to a point where AI is going to feel so intimate, so personal that it would actually feel concerning to you to be away from it.

**Lenny Rachitsky** (01:07:37):
Oh man, there reminds me of Friend.com, which just launched in a really fun, I don't know if you saw Friend.com, their launch video.

**Tomer Cohen** (01:07:44):
No, I did not.

**Lenny Rachitsky** (01:07:45):
Oh, man. Check out Friend.com. It's like a digital friend that just is with you all the time and you're talking to him and it's an AI.

**Tomer Cohen** (01:07:50):
Oh, we're just getting started. There's going to be, we're in for incredible revolution there.

**Lenny Rachitsky** (01:07:55):
I'm excited and scared at the same time. Tomer, thank you so much for being here. This was amazing. You're awesome. Two final questions. Where can folks check out stuff that you're working on? You have a course also, you have a podcast, so just give people a sense of where to find that.

**Tomer Cohen** (01:08:07):
Awesome. So obviously I'm on LinkedIn, reach out anytime. I read everything people send to me. I don't always reply to everything, but I read everything sent to me. And then if you want to go deep on AI first, I have two courses, they're free. I think it's a phenomenal way for you to build or starting to build your expertise, especially if you're in product. It's a great way to go deeper and not just stay on the high level parts of things.

**Lenny Rachitsky** (01:08:36):
Amazing. We'll link to all those things in the show notes. Tomer, thank you so much for being here.

**Tomer Cohen** (01:08:40):
Lenny. Thank you. It's our pleasure.

**Lenny Rachitsky** (01:08:42):
Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lennyspodcasts.com. See you in the next episode.

---

## [15/17] The 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI)

**Lenny Rachitsky** (00:00:00):
You guys hit a billion in revenue in less than four years with around 60 to 70 people. You're completely bootstrapped, haven't raised any VC money. I don't believe anyone has ever done this before.

**Edwin Chen** (00:00:10):
We basically never wanted to play the Silicon Valley game. I always thought it was ridiculous. I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of the people and we would move faster because the best people wouldn't have all these distractions. So when we start Surge, we wanted to build it completely differently with a super small, super elite team.

**Lenny Rachitsky** (00:00:26):
You guys are by far the most successful data company out there.

**Edwin Chen** (00:00:29):
We essentially teach AI models what's good and what's bad. People don't understand what quality even means in this space. They think you could just throw bodies at a problem and get good data, that's completely wrong.

**Lenny Rachitsky** (00:00:40):
To a regular person, it doesn't feel like these models are getting that much smarter constantly.

**Edwin Chen** (00:00:43):
Over the past year, I've realized that the values that the companies have will shape the model. I was asking Claude to help me drop an email the other day. And after 30 minutes, yeah, I think it really crafted me the perfect email and I sent it. But then I realized that I spent 30 minutes doing something that didn't matter at all. If you could choose the perfect model behavior, which model would you want? Do you want a model that says, "You're absolutely right. There are definitely 20 more ways to improve this email," and it continues for 50 more iterations or do you want a model that's optimizing for your time and productivity and just says, "No. You need to stop. Your email's great. Just send it and move on"?

**Lenny Rachitsky** (00:01:14):
You have this hot take that a lot of these labs are pushing AGI in the wrong direction.

**Edwin Chen** (00:01:18):
I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understand the universe, we are optimizing for AI slop instead. But we're optimizing your models for the types of people who buy tabloids at a grocery store. We're basically teaching our models to chase dopamine instead of truth.

**Lenny Rachitsky** (00:01:35):
Today, my guest is Edwin Chan, founder and CEO of Surge AI. Edwin is an extraordinary CEO and Surge is an extraordinary company. They're the leading AI data company, powering training at every frontier AI lab. They're also the fastest company to ever hit $1 billion in revenue in just four years after launch with fewer than 100 people and also completely bootstrapped. They've never raised a dollar in VC money, they've also been profitable from day one.

**Lenny Rachitsky** (00:02:05):
As you'll hear in this conversation, Edwin has a very different take on how to build an important company, and how to build AI that is truly good and useful to humanity. I absolutely love this conversation and I learned a ton. I'm really excited for you to hear it. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It helps tremendously.

**Edwin Chen** (00:04:55):
Thanks so much for having me. I'm super excited.

**Lenny Rachitsky** (00:04:58):
I want to start with just how absurd what you've achieved is. A lot of people and a lot of companies talk about scaling massive businesses with very few people as a result of AI, and you guys have done this in a way that is unprecedented. You guys hit a billion in revenue in less than four years with less than 60, around 60 to 70 people, you're completely bootstrapped, haven't raised any VC money, I don't believe anyone has ever done this before, so you guys are actually achieving the dream of what people are describing will happen with AI. I'm curious just, do you think this will happen more and more as a result of AI? And also just where has AI most helped you find leverage to be able to do this?

**Edwin Chen** (00:05:40):
Yeah, so we hit over a billion of revenue last year with under 100 people. And I think we're going to see companies with even crazier ratios, like 100 billion per employee in the next few years. AI is just going to get better and better and make things more efficient so that ratio just becomes inevitable.

**Edwin Chen** (00:05:57):
I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of people and we would move faster because the best people wouldn't have all these distractions. And so when we started Surge, we wanted to build it completely differently with a super small, super elite team, and yeah, what's crazy is that we actually succeeded. And so I think two things are colliding.

**Edwin Chen** (00:06:18):
One is that people are realizing that you don't have to build giant organizations in order to win.

**Edwin Chen** (00:06:23):
And two, yeah, all these efficiencies from AI. And they're just going to lead to a really amazing time in company building.

**Edwin Chen** (00:06:29):
The thing I'm excited about is that the types of companies are going to change too. It won't just be that they're smaller, we're going to see fundamentally different companies emerging. If you think about it, fewer employees means less capital. Less capital means you don't need a raise. So instead of companies started by founders who are great at pitching and great at hyping, you'll get founders who are really great at technology and product.

And instead of products optimized for revenue and what VCs want to see, you'll get more interesting ones built by these tiny obsessed teams. So people building things they actually care about, real technology and real innovation. So I'm actually really hoping that the slick on [inaudible 00:07:06], it'll go back to being updates for hackers again.

**Lenny Rachitsky** (00:07:08):
You guys have done a lot of things in a very contrarian way, and one was actually just not being on LinkedIn, posting viral posts, not on Twitter, constantly promoting Surge. I think most people hadn't heard of Surge until just recently, and then you just came out, and like, okay, the fastest growing company at a billion dollars. Why would you do that? I imagine that was very intentional.

**Edwin Chen** (00:07:27):
We basically never wanted to play the Silicon Valley game. And like I always thought it was ridiculous. What did you dream of doing when you were a kid? Was it building a company from scratch yourself and getting in the weeds of your code and your product every day? Or was it explaining all your decisions to VCs and getting on this giant PR and fundraising hamster wheel? And it definitely made things more difficult for us, because yeah, when you fundraise, you just naturally get part of this kind of Silicon Valley industrial complex where people will, your VCs will tweet about you. You'll get the tech runs outlines, you'll get announced in all of the newspapers because you raised at this massive valuation. And so it made things more difficult us because the only way we were going to succeed was by building a 10 times better product and getting word of mouth from researchers. But I think it also meant that our customers were people who really understood data and really cared about it.

I always thought it was really important for us to have early customers who were really aligned with what we were building, and who really cared about having really high quality data, and really understood how that data would make their AI models so much better because they were the ones helping us. They were the ones giving us feedback on what we're producing. And so just having that kind of very close mission alignment with our customers actually helped us early on. So these are people who basically just buying our product because they knew how different it was and because it was helping them rather than because they saw something in that current [inaudible 00:08:52]. So it made things harder for us, but I think in a really good way.

**Lenny Rachitsky** (00:08:55):
It's such an empowering story to hear this journey for founders that they don't need to be on Twitter all day promoting what they're doing. They don't have to raise money. They can just kind of go heads down and build, so I love so much about the story of Surge. For people that don't know what Surge does, just to give us a quick explanation of what Surge is.

**Edwin Chen** (00:09:16):
We essentially teach AI models what's good and what's bad. So we train them using human data, and there's a lot of different products that we have, like SFT, RHF, rubrics, verifiers, RL environments, and so on and so on, and then we also measure how well they're progressing. So essentially we're a data company.

**Lenny Rachitsky** (00:09:36):
What you always talk about is the quality has been the big reason you guys have been so successful, the quality of the data. What does it take to create higher quality data? What do you all do differently? What are people missing?

**Edwin Chen** (00:09:47):
I think most people don't understand what quality even means in this space. They think you could just throw bodies at a problem and get good data and that's completely wrong. Let me give you an example.

**Edwin Chen** (00:09:59):
So imagine you wanted to train a model to write an eight line poem about the moon. What makes it a good, high-quality poem? If you don't think deeply about quality, you'll be like, "Is this a poem? Does it contain eight lines? Does it contain the word, moon?" You check all of these boxes, and if so, sure. Yeah, you say it's a great problem. But that's completely different from what we want. We are looking for a Nobel Prize-winning poetry. Is this poetry unique? Is it full of subtle imagery? Does it surprise you and target your heart? Does it teach you something about the nature of moonlight? Does it playthrough emotions? And does it make you think? That's what we are thinking about when we think about high quality poem.

So it might be like a haiku about moonlight on water. It might use internal rhyme and meter. There are a thousand ways to write a poem about the moon, and in each one, gives you all these different insights into language, and imagery, and human expression, and I think thinking about quality in this way is really hard, it's hard to measure. It's really subjective, and complex, and rich. And it sets a really high bar. And so we have to build all of this technology in order to measure it, like thousands of signals on all of our workers, thousands of signals on every project, every task. We know at the end of the day, if you are good at writing poetry versus good at writing essays versus great at writing technical documentation. And so we have to gather all these signals on what your background is, what your expertise is, and not just that. Like how you're actually performing when you're writing all these things, and we use those signals to inform whether or not you are good [inaudible 00:11:23] for these projects, and whether or not you are improving the models.

**Edwin Chen** (00:11:26):
And it's really hard, and so to build all this technology to measure it, but I think that's exactly what we want AI to do, and so we have these really deep notions about quality that we're always trying to try and achieve.

**Lenny Rachitsky** (00:11:37):
So what I'm hearing is there's kind of just going much deeper in understanding what quality is within the verticals that you are selling data around. And is this like a person you hire that is incredibly talented at poetry plus evals that they, I guess, help write, that tell them that this is great? What's the mechanics of that?

**Edwin Chen** (00:11:57):
The way it works is we essentially gather thousands of signals about everything that you're doing when you're working on platform. So we are looking at your keyboard strokes. We are looking how fast you answer things. We are using reviews, we are using code standards, we are using... We're training models ourselves all on the outputs that you create, and then we're seeing whether they improve the model's performance.

**Edwin Chen** (00:12:23):
And so in a very similar way to how Google search, like when Google search is trying to determine what is a good webpage, there's almost two aspects of it. One is you want to remove all of the worst of the worst webpages. So you want to remove all the spam, all the just low quality content, all the pages that don't load, and so it's almost like a content moderation problem. You just want to remove the worst of the worst.

But then you also want to discover the best of the best. Okay, like this is the best webpage or just the best person for this job. They are not just somebody who writes the equivalent of high school level poetry. Again, they're not just [inaudible 00:12:57] writing poetry that checks all these boxes, checks all of these explicit instructions, but rather, yeah, they're writing poetry that makes you emotional. And so we have all these signals as well that, again, completely differently from moving the worst of the worst, we are finding the best of the best. And so we have all these signals...

**Edwin Chen** (00:13:12):
Again, just like Google Search uses all these signals that feeds them into their ML algorithms and uses and predicts certain types of things, we do the same with all of our workers and all of our tasks in all of our projects. And so it's almost like a complicated machine learning problem at the end of the day, and that's how it works.

**Lenny Rachitsky** (00:13:29):
That is incredibly interesting.

**Lenny Rachitsky** (00:13:31):
I want to ask you about something I've been very curious about over the past couple years. If you look at Claude, it's been so much better at coding and at writing than any other model for so long. And it's really surprising just how long it took other companies to catch up. Considering just how much economic value there is there, just like every AI coding product sat on top of Claude because it was so good Claude code and writing also. What is it that made it so much better? Is it just the quality of the data they trained on or is there something else?

**Edwin Chen** (00:13:59):
I think there are multiple parts to it. So a big part of it certainly is the data. I think people don't realize that there's almost like this infinite amount of choices that all the frontier labs are deciding between when they're choosing what data goes into their models. It's like, okay, are you purely using human data? Are you gathering the human data in X, Y, Z way? When you are gathering the human data, what exactly are you asking the people who are creating it to create for you?

**Edwin Chen** (00:14:30):
For example, in the coding realm, maybe you care more about front end coding versus back end coding. Maybe when you're doing front end coding, you care a lot about the visual design of the front end applications that you're creating, or maybe you don't care about it so much and you care more about, I don't know, the deficiency of it or the pure correctness over that visual design.

And then other questions like, okay, are you carrying [inaudible 00:14:49] how much synthetic data are we throwing into the mix? How much do you care about these 20 different benchmarks?"

**Edwin Chen** (00:14:55):
Some companies, they see these benchmarks and they're like, "Okay, for PR purposes, even though we don't think that these academic benchmarks matter all that much, maybe we just need to optimize for them anyways because our marketing team needs to show certain progress on certain standard evaluations that every other company talks about, and if we don't show good performance here, it's going to be bad for us even if ignoring these academic benchmarks makes us better at the real tasks."

**Edwin Chen** (00:15:21):
Other companies are going to be principled and be like, "Okay, yeah, no, I don't care about marketing. I just care about how my model performs on these real world tasks at the end of the day, and so I'm going to optimize for that instead."

**Edwin Chen** (00:15:31):
And it's almost like there's a trade-off between all of these different things, and there's like a...

**Edwin Chen** (00:15:36):
One of the things I often think about is that there's a... It's almost like there's an art to post training. It's not purely a science. When you are deciding what kind of model you're trying to create and what it's good at, there's this notion of taste and sophistication, like, "Okay, do I think that these..."

**Edwin Chen** (00:15:57):
So going back to the example of how good the model is at visual design. I'm like, "Okay, maybe you have a different notion of visual design than what I do. Maybe you care more about minimalism, and you care more about, I don't know, 3D animations than I do. And maybe this other person prefers things that look a little bit more broke." And there's all these notions of taste sophistication that you have to decide between when you're designing your post training mix, and so that matters as well.

**Edwin Chen** (00:16:21):
So long story short, I think there's all these different factors, and certainly the data is a big part of it, but it's also like what is the objective function that you're trying to optimize your model towards?

**Lenny Rachitsky** (00:16:30):
That is so interesting. The taste of the person leading this work will inform what data they ask for, what data they feed it. But it's wild it shows the value of great data. Anthropic got so much growth and win from essentially better data.

**Edwin Chen** (00:16:49):
Yeah, exactly.

**Lenny Rachitsky** (00:16:50):
And I could see why companies like yours are growing so fast. There's just so much... And that's just one vertical. That's just coding, and then there's probably a similar area for writing. I love that it's... It's interesting that AI, it feels like this artificial computer binary thing, but it's like taste. Human judgment is still such a key factor in these things being successful.

**Edwin Chen** (00:17:09):
Yep, exactly. Again, going back to the example I said earlier, certain companies, if you ask them what is good poem, they will simply robotically check off all of these instructions on our list.

**Edwin Chen** (00:17:20):
But again, I don't think that makes for good poetry, so certain frontier labs, the ones with more taste in sophistication, they will realize that it doesn't reduce to this six set of checkboxes and they'll consider all of these kind of implicit, very subtle qualities instead, and I think that's what makes them better at this at the end of the day.

**Lenny Rachitsky** (00:17:38):
You mentioned benchmarks. This is something a lot of people worry about is there's all these models that are always... Basically, it feels like every model is better than humans at every STEM field at this point, but to a regular person, it doesn't feel like these models are getting that much smarter constantly. What's your just sense of how much you trust benchmarks and just how correlated those are with actual AI advancements?

**Edwin Chen** (00:18:00):
Yeah, so I don't trust the benchmarks at all. And I think that's for two reasons. So one is I think a lot of people don't realize, even researchers within the community, they don't realize that the benchmarks themselves are often honestly just wrong. They have wrong answers. They're full of all this kind of messiness and people trust... Long as for the popular ones, people have maybe realized this to some extent, but the vast majority just have all these flaws that people don't realize. So that's one part of it.

**Edwin Chen** (00:18:30):
And the other part of it is these benchmarks at the end of the day, they often have well-defined objective answers that make them very easy for models to hill-climb on in a way that's very different from the messiness and ambiguity of the real world.

**Edwin Chen** (00:18:48):
I think one thing that I often say is that it's kind of crazy that these models can win IMO gold medals, but they still have trouble parsing PDFs. And that's because, yeah, even though IMO gold medals seem hard to the average person, yeah, they are hard at the end of the day. But they have this notion of objectivity that, okay, yeah, parsing a PDF sometimes doesn't have. And so it's easier for the frontier labs to hill-climb on all of these than to solve all these mess ambiguous problems in the real world. So I think there's a lack of direct correlation there.

**Lenny Rachitsky** (00:19:17):
It's so interesting the way you described it is hitting these benchmarks is kind of like a marketing piece. When you launch, say Gemini 3 just launched, and it's like, cool. Number one with all these benchmarks. Is that what happens? They just kind of train their models to get good at these very specific things?

**Edwin Chen** (00:19:31):
Yeah, so there's, again, maybe two parts to this. So one is, sometimes, yeah, these benchmarks, they accidentally leak in certain ways or the frontier labs will tweak the way they evaluate their models on these benchmarks. They'll tweak your system prompt or they'll tweak the number of times they run their model, and so on and so on in a way that games these benchmarks.

**Edwin Chen** (00:19:54):
The other part of it though is it's like by optimizing for the benchmark instead of optimizing for the real world, you will just naturally climb on the benchmark and, yeah, it's basically another form of gaming it.

**Lenny Rachitsky** (00:20:09):
Knowing that with that in mind, how do you get a sense of if we're heading towards AGI, how do you measure progress?

**Edwin Chen** (00:20:15):
Yes, so the way we really care about measuring model progress is by running all these human evaluations.

**Edwin Chen** (00:20:21):
So for example, what we do is, yeah, we will take Gore human annotators, and we'll ask them, "Okay, go have a conversational model." And maybe you're having this conversation with the model across all of these different topics. So you are a Nobel Prize winning physicist. So you go have a conversation about pushing different tier of your own research. You are a teacher and you're trying to create lesson plans for your students, so go talk to the model about these things. Or you're a coder and you're working at one of these big tech companies, and you have these problems every day, so go talk to the model and see how much it helps you.

And because or searchers or annotators, they are experts at the top of their fields, and they are not just giving your responses, they're actually working through the responses deeply themselves, they are... Yeah, they're going to evaluate the code that it write. They're going to double check the physics equations that it writes. They're going to evaluate the models in a very deep way, so they're going to pay attention to accuracy and instruction following, all these things that casual users don't when you suddenly get a popup on your ChatGPT response asking you to compare these two different responses. People like that, they're not evaluating models deeply, they're just vibing and picking whatever response looks flashiest or [inaudible 00:21:38] are looking closely at responses and evaluating them for all of these different dimensions, and so I think that's a much better approach than these benchmarks or these random outline AV tests.

**Lenny Rachitsky** (00:21:49):
Again, I love just how central humans continue to be in all this work that we're not totally done yet. Is there going to be a point where we don't need these people anymore, that AI is so smart that, "Okay, we're good. We got everything out of your heads"?

**Edwin Chen** (00:22:00):
Yeah, I think that will not happen until we've reached AGI. It's almost like by definition, if we haven't reached AGI yet, then there's more for the models to learn from, and so, yeah, I don't think that's going to happen anytime soon.

**Lenny Rachitsky** (00:22:12):
Okay, cool. So more reason to stress about AGI. "We don't need these folks anymore."

**Lenny Rachitsky** (00:22:18):
I can't not ask just... People that work closely with this stuff, I'm always just curious. What's your AGI timelines? How far do you think we are from this? Do you think we're in like a couple years or is it like decades?

**Edwin Chen** (00:22:28):
So I'm certainly on the longer time horizon front. I think people don't realize that there's a big difference between moving from 80% performance to 90% performance to 99% performance to 99.9% performance, and so on, and so on. And so in my head, I probably bet that within the next one or two years, yeah, the models are going to automate 80% of the average LL6 software engineer's job. It's going to take another few years to move to 90%, and another fewer to 99%, and so on, and so on. So I think we're closer to a decade or decades away than [inaudible 00:23:03].

**Lenny Rachitsky** (00:23:03):
You have this hot take that a lot of these labs are kind of pushing AGI in the wrong direction and this is based on your work at Twitter, and Google, and Facebook. Can you just talk about that?

**Edwin Chen** (00:23:14):
I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understand the universe, all these big grand questions, we are optimizing for AI slop instead. We're basically teaching our models to chase dopamine instead of truth. And I think this relates to what we're talking about regarding these benchmarks. So let me give you a couple examples.

**Edwin Chen** (00:23:35):
So right now, the industry is played by these terrible databoards like LLM Arena. It's this popular online leaderboard where random people from around the world vote on which AI response is better. But the thing is, like I was saying earlier, they're not carefully reading or fact-checking. They're skimming these responses for two seconds and picking whatever looks flashiest.

So a model can hallucinate everything. It can completely hallucinate. But it will look impressive because it has crazy emojis, and boating, and markdown headers, and all these superficial things that don't matter at all, but it catch your attention. And these LLM-reading users love it. It's literally optimizing your models for the types of people who buy tabloids at the grocery store. We've seen this [inaudible 00:24:15] data ourselves. The easiest way to climb LLM Arena, it's adding crazy boating. It's doubling the number of emojis. It's tripling the length of your model responses, even if your model starts hallucinating and getting the answer completely wrong.

And the problem is, again, because all of these frontier labs, they kind of have to pay attention to PR because their sales team, when they're trying to sell to all these enterprise customers, those enterprise customers will say, "Oh, well, but your model's only number five on LLM Arena, so why should I buy it?" They have to, in some sense, pay attention to these leaderboards, and so what their researchers [inaudible 00:24:47] tell us is like they'll say, "The only way I'm going to get promoted at the end of the year is if I climb this leaderboard, even though I know that climbing it is probably going to make my model worse and accuracy [inaudible 00:24:57] following." So I think there's all these negative incentives that are pushing work in the wrong direction.

**Edwin Chen** (00:25:03):
I'm also worried about this trend towards optimizing AI for engagement. I used to work on social media. And every time we optimize for engagement, terrible things happened. You'd get clickbait and pictures of bikinis and bigfoot and horrifying skin diseases just filling your feeds. And I think I worry that the same thing's happening with AI. If you think about all the sycophancy issues with ChatGPT, "Oh, you're absolutely right. What an amazing question," the easiest way to hook users is to tell them how amazing they are. And so these models, they constantly tell you you're a genius. They'll feed into your delusions and conspiracy theories. They'll pull you down these rabbit holes because Silicon Valley loves maximizing time spent and just increasing the number of conversations you're having with it. And so yeah, companies are spending all the time hacking these leaderboards and benchmarks, and the scores are going up, but I think it actually masks up the models with the best scores, they are often the worst or just have all these fundamental failures. So I think I'm really worried that all of these negative ascendants are pushing AGI into the wrong direction.

**Lenny Rachitsky** (00:26:03):
So what I'm hearing is AGI is being slowed down by these, basically the wrong objective function, these labs paying attention to the wrong basically benchmarks and evals.

**Edwin Chen** (00:26:11):
Yep.

**Lenny Rachitsky** (00:26:12):
I know you probably can't play favorites since you work with all the labs. Is there anyone doing better at this and maybe kind of realizing this is the wrong direction?

**Edwin Chen** (00:26:21):
I would say I've always been very impressed by Anthropic. I think Anthropic takes a very principled view about what they do and don't care about and how they want their models to behave in a way that feels a lot more principle to me.

**Lenny Rachitsky** (00:26:38):
Interesting.

**Lenny Rachitsky** (00:26:39):
Are there any other big mistakes you think labs are making just that are kind of slowing things down or heading in the wrong direction? Where we've heard just chasing benchmarks, this engagement focus, is there anything else you're seeing of just like, "Okay, we got to work on this because it'll speed everything up"?

**Edwin Chen** (00:26:55):
I think there is a question of what products they're building and whether those products themselves are something that kind of help or hurt humanity. I think a lot about Sora and...

**Lenny Rachitsky** (00:27:07):
I was thinking that's what you're imagining.

**Edwin Chen** (00:27:10):
Yeah, what it entails, and so it's kind of interesting. It's like which companies would build Sora and which wouldn't?

**Edwin Chen** (00:27:17):
And I think that answer to that... Well, I don't know if answer is myself. I have an idea in my head, but I think the answer to that question maybe reveals certain things about what kinds of AI models those companies want to build and what direction and what future they want to achieve, yeah, so I think about that a lot.

**Lenny Rachitsky** (00:27:37):
The steel man argument there is, it's like fun, people want it, it'll help them generate revenue to grow this thing and build better models, it'll train data in an interesting way, it's also just really fun.

**Edwin Chen** (00:27:51):
Yeah. I think it's almost like, do you care about how you get there? And in the same way, so I made this tabloid analogy earlier, but would you sell tabloids in order to fund, I don't know, some other newspaper?

**Edwin Chen** (00:28:09):
Sure, like in some sense, if you don't care about the path, then you'll just do whatever it takes, but it's possible that it has negative consequences in of itself that will harm the long-term direction of what you're trying to achieve, and maybe it'll distract you from all the more important things, so yeah, I think that the path you take matters a lot as well.

**Lenny Rachitsky** (00:28:33):
Along these lines, you talked a bunch about this of just Silicon Valley and kind of the downsides of raising a lot of money being in the echo chamber. What do you call it, the Silicon Valley machine? You talk about how it's hard to build important companies in this way and that you might actually be much more successful if you're not going down the VC path. Can you just talk about what you've seen in that experience and your advice essentially to founders, because they're always hearing? Raise money from fancy VCs, move to Silicon Valley, what's kind of the countertake?

**Edwin Chen** (00:29:02):
Yes. So I've always really hated a lot of the Silicon Valley mantras. The standard playbook is to get product market fit by pivoting every two weeks. And to chase growth and chase engagement with all of these dark patterns and to blitz scale by hiring as fast as possible. And I've always disagreed.

**Edwin Chen** (00:29:20):
So yeah, I would say don't pivot. Don't put scale. Don't hire that Stanford grad who simply wants to add a hot company to your resume, just build the one thing only you could build, a thing that wouldn't exist without the insight and expertise that only you have.

And you see these buy to [inaudible 00:29:34] companies everywhere now. Some founder who was doing crypto in 2020, and then pivoted to NFTs in 2022, and now they're an AI company. There's no consistency, there's no mission, they're just chasing valuations. And I've always hated this because Silicon Valley loves to score on Wall Street for focusing on money. But honestly, most of the Silicon Valley's chasing the same thing. And so we stayed focused on our mission from day one, pushing that frontier of high quality complex data, and I've always loved that because I think startups...

**Edwin Chen** (00:30:03):
I have this very romantic notion of startups. Startups are supposed to be a way of taking big risks to build something that you really believe in. But if you're constantly pivoting, you're not taking any risks. You're just trying to make a quick buck. And if you fail because the market isn't ready yet, I actually think that's way better. At least you took a swing at something deep, and novel, and hard instead of pivoting into another LLM wrapper company. So yeah, I think the only way you build something that matters that's going to change the world is if you find a big idea you believe in and you say no to everything else.

**Edwin Chen** (00:30:30):
So you don't keep on pivoting when it gets hard, you don't hire a team of 10 product managers because that's what every other cookie cutter startup does, you just keep building that one company that wouldn't exist without you. And I think there are a lot of people in Silicon Valley now who are sick of all the grift, who want to work on big things that matter with people who actually care, and I'm hoping that that would be the future of how we go with technology.

**Lenny Rachitsky** (00:30:52):
I'm actually working on a post right now with Terrence Rohan, this VC that I really like to work with, and we interviewed five people who picked really successful generational companies early and joined them as really early employees. They joined OpenAI before anyone thought it was awesome, Stripe before anyone knew was awesome, and so we're looking for patterns of how people find these generational companies before anyone else, and it aligns exactly what you described, which is ambition. They have a wild ambition with what they want to achieve. They're not, as you said, just kind of looking around for product market fit no matter what ends up being, and so I love that what you described very much aligns with what we're seeing there.

**Edwin Chen** (00:31:33):
Yeah, I absolutely think that you have to have huge ambitions, and you have to have a huge belief in your idea that's going to change the world, and you have to be willing to double down and keep on doing whatever it takes to make it happen.

**Lenny Rachitsky** (00:31:44):
I love how counter your narrative is to so many of the things people hear, and so I love that we're doing this. I love that we're sharing this story.

**Edwin Chen** (00:33:42):
I'm in the camp where I do believe that something new will be needed. The way I think about it is when I think about training AI, I take a very... I don't know if I would say biological point of view. But I believe that in the same way that there's a million different ways that humans learn, we need to build models that can mimic all of those ways as well. And maybe they'll have a different distribution of the focuses that they have. I know that it'll be different for humans, so maybe they have a different distribution, but we want to be able to mimic their learning abilities of humans and make sure that we have the algorithms and the data for models to learn in the same way. And so to the extent that LLMs have different ways of learning from humans, then yeah, I think something new will be needed.

**Lenny Rachitsky** (00:34:32):
This connects to reinforcement learning. This is something that you're big on and something I'm hearing more and more is just becoming a big deal in the world of post-training. Can you just help people understand what is reinforcement learning and reinforcement learning environments, and why they're going to be more and more important in the future?

**Edwin Chen** (00:34:49):
Reinforcement learning is essentially training your model to reach a certain reward. And let me explain what an RL environment is. An RL environment is essentially a simulation of real world. So think of it like building a video game with a fully fleshed out universe. Every character has a real story, every business has tools and data you can call, and you have all these different entities interacting with each other.

**Edwin Chen** (00:35:12):
So for example, we might build a world where you have a startup with Gmail messages, and Slack threads, and Jira tickets, and GitHub PRs, and a whole code base. And then suddenly AWS goes down. And Slack goes down. And so, "Okay. Model, well, what do you do?" The model needs to figure it out.

**Edwin Chen** (00:35:29):
So we give them models tasks in these environments, we design interesting challenges for them, and then we run them to see how they perform. And then we teach them, we give them these rewards when they're doing a good job or a bad job.

And I think one of the interesting things is that these environments really showcase where models are weak at end-to-end tasks in real world. You have all these models that seem really smart on isolated benchmarks, they're good at single step tool calling. They're good at single step instruction following. But suddenly you dump them into these messy worlds where you have confusing Slack messages and tools they've never seen before, and they need to perform right actions and modify the [inaudible 00:36:06] and interact over longer time horizons where what they do in step one affects what they do in step 50. And that's very different from these kind of academic single step environments that they've been in before, and so the model just fails catastrophically in all these crazy ways.

**Edwin Chen** (00:36:21):
So I think these RL environments are going to be really interesting playgrounds for the models to learn from that will essentially be simulations and mimics in real world, and so they'll hopefully get better and better at real tasks compared to all these contrived environments.

**Lenny Rachitsky** (00:36:35):
So I'm trying to imagine what this looks like. Essentially, it's like a virtual machine with, I don't know, a browser or a spreadsheet or something in it with like, I don't know, surge.com. Is that your website, surge.com? Let's make sure we get that right.

**Edwin Chen** (00:36:49):
So we are actually surgehq.ai.

**Lenny Rachitsky** (00:36:52):
Surgehq.ai. Check it out. We're hiring it, I imagine. Yes. Okay. So it's like, cool, here's surgehq.ai. Your job, here's your job as an agent, let's say, is to make sure it stays up. And then all of a sudden it goes down and the objective function is figure out why. Is that an example?

**Edwin Chen** (00:37:12):
Yeah, so the objective function might be... Or the goal of the task might be, okay, go figure out why and fix it. And so the objective function might be, it might be passing a series of unit tests, it might be writing a document like maybe it's a retro containing certain information that matches exactly what happened, there's all these different rewards that we might give it that determine whether or not it's succeeding, and so the models, we're basically teaching the models to achieve that reward.

**Lenny Rachitsky** (00:37:38):
So essentially it's off and running. Here's your goal, figure out why the site went down and fix it. And it just starts trying stuff, we're using everything, all the intelligence it's got, it makes mistakes, you kind of help it along the way, reward it if it's doing the right sort of thing. And so what you're describing here is this is the next phase of models becoming smarter. More RL environments focused on very specific tasks that are economically valuable, I imagine.

**Edwin Chen** (00:38:04):
Yeah, so just in the same way that there were all these different methods for models of learning in the past, originally we had SFT and RHF, and then we had rubrics and verifiers. This is the next stage, and it's not the case that the previous methods are obsolete, this is, again, just a different form of learning that compliments all the previous types, so it's just like a different skilled model not only to learn how to do.

**Lenny Rachitsky** (00:38:30):
And so in this case, it's less some physics PhD sitting around talking to a model, correcting it, giving it evals of here's what the correct answer is, creating rubrics and things like that. More it's like this person now designing an environment. So another example I've heard is like a financial analyst. Just like, "Here's an Excel spreadsheet, here's your goal, figure out our profit and loss," or whatever. And so this expert now, instead of just sitting around writing rubrics, they're designing this RL environment.

**Edwin Chen** (00:38:56):
Yeah, exactly. So that financial analyst might create a spreadsheet, they may create certain tools that the model needs to call in order to help fill out that spreadsheet, like it might be, okay, the model needs to access Bloomberg terminal. It needs to learn how to use it. And it needs to learn how to use this calculator. And it needs to learn how to pour on this calculation. So it has all these tools that it has access to.

**Edwin Chen** (00:39:19):
And then the reward might be... Okay, it's like maybe I will download that spreadsheet and I want to see, does cell B22 contain the correct profit and loss number? Or does tab number two contain this piece of information?

**Lenny Rachitsky** (00:39:37):
And what's interesting, this is a lot closer to how humans learn. We just try stuff, figure out what's working and what's not. You talk about how trajectories are really important to this. It's not just here's the goal and here's the end, it's like every step along the way. Can you just talk about what trajectories are and why that's important to this?

**Edwin Chen** (00:39:55):
I think one of the things that people don't realize is that sometimes even though the model reaches the correct answer, it does so in all these crazy ways. So it may have in the intermediate trajectory, it may have tried 50 different times and failed, but eventually it just kind of randomly lands on a correct number. Or maybe it is...

**Edwin Chen** (00:40:20):
Sometimes it just does things very inefficiently or it almost reward-hacks a way to get at the correct answer, and so I think paying attention to the directory is actually really important. And I think it's also really important because some of these trajectories can be very long. And so if all you're doing is checking whether or not the model reaches the final answer, it's like there's all this information about how the model behaved in the immediate step that's missing.

**Edwin Chen** (00:40:48):
Sometimes you want models to get to the correct answer by reflecting on what it did. Sometimes you want it to get it at the correct answer by just one-shotting it. And if you ignore all of that, it's just like teaching it... just missing a lot of the information that you could be teaching a model to do.

**Lenny Rachitsky** (00:41:03):
I love that. Yeah, it tries a bunch of stuff and eventually gets it right. You don't want it to learn this is the way to get there. There's often a much more efficient way of doing it.

**Lenny Rachitsky** (00:41:11):
You mentioned all the kind of the steps we've taken along the journey of helping models get smarter. Since you've been so close to this for so long, I think this is going to be really helpful for people. What's kind of like been the steps along the way from the first post-training that has most helped models advance? Where do evals fit in the RL environments? Just like what's been the steps and now we're heading towards RL environments?

**Edwin Chen** (00:41:33):
Originally, the way models started getting post-trained was purely through SFT. And-

**Lenny Rachitsky** (00:41:41):
What does that stand for?

**Edwin Chen** (00:41:42):
So SFT stands for supervised fine-tuning. So again, I think often in terms of these human analogies, and so SFT is a lot like mimicking a master and copying what they do.

**Edwin Chen** (00:41:54):
And then RLHF became very dominant. And analogy there would be like sometimes you learn by writing 55 different essays and someone telling you which one they liked the most.

**Edwin Chen** (00:42:04):
And then I think over the past year or so, rubrics and verifiers have become very important. And rubrics and verifiers are like learning by being graded and getting detailed feedback on where you went wrong.

**Lenny Rachitsky** (00:42:17):
And those are evals, another word for that?

**Edwin Chen** (00:42:19):
Yeah. So I think evals often covers two terms. One is you are using the evaluations for training because you're evaluating whether or not the model did a good job, and when it does do a good job, you're rewarding it.

**Edwin Chen** (00:42:35):
And then there's this other notion of evals where you're trying to measure the model's progress like, okay, yeah, I have five different candidate checkpoints and I want to pick the one that's best in order to release it to the public. So going to run all these evals on these five different checkpoints in order to decide which one is best.

**Lenny Rachitsky** (00:42:51):
Awesome.

**Edwin Chen** (00:42:51):
Yeah, and now we have RL environments, so this is kind of like a hot new thing.

**Lenny Rachitsky** (00:42:55):
Awesome. So what I love about this business journey is just there's always something new. There's always this like, okay. We're getting so good at just all this beautiful data for companies and now they need something completely different. Now we're setting up all these virtual machines for them and all these different use cases.

**Edwin Chen** (00:43:08):
Yep.

**Lenny Rachitsky** (00:43:08):
And it feels like that's a big part of this industry you're in, it's just adapting to what labs are asking for.

**Edwin Chen** (00:43:13):
Yeah. So I really do think that we are going to need to build a suite of products that reflect a million different ways that humans learn.

Like for example, think about becoming a great writer. You don't become great by memorizing a bunch of grammar rules. You become great by reading great books, and you practice writing, and you get feedback from your teachers and from the people who buy your books in a bookstore and leave reviews. And you notice what works and what doesn't. And you develop taste by being exposed to all of these masterpieces and also just terrible writing. So you learn through this endless cycle of practicing reflection, and each type of learning that you have, again, these are all very different methods of learning to become a great writer, so just in the same way that... it's a thousand different ways that the great writer becomes great, I think there's going to be a thousand different ways that AI [inaudible 00:44:05] need to learn.

**Lenny Rachitsky** (00:44:05):
It's so interesting this just ends up being just like humans in so many ways. It makes sense because in a sense, neural networks, deep learning is modeled after how humans have learned and how our brains operate, but it's interesting just to make them smarter. It's how do we come closer to how humans learn more and more?

**Edwin Chen** (00:44:22):
Yeah, it's almost like maybe the end goal is just throwing you into the environment and just seeing how you evolve. But within that evolution, there's all these different sub-learning mechanisms.

**Lenny Rachitsky** (00:44:34):
Yeah, which is kind of what we're doing now, so that's really interesting. This might be the last step until we hit AGI. Along these lines, something that's really unique to Surge that I learned is you guys have your own research team, which I think is pretty rare, talk about just why that's something you guys have invested in and what has come out of that investment.

**Edwin Chen** (00:44:52):
Yeah, so I think that stems from my own background. My own background is as a researcher. And so I've always cared fundamentally about pushing the industry and pushing the research community and not just about revenue. And so I think what our research team does is a couple different things.

**Edwin Chen** (00:45:13):
So we almost have two types of researchers at our company. One is our forward-deployed researchers who are often working hand in hand with our customers to help them understand their models. So we will work very closely with the customers to help them understand, "Okay, this is where your model is today. This is where you're lagging behind all the competitors, these are some ways that you could be improving in the future, given your goals, and we're going to design these data sets, these evaluation methods, these training techniques to make your models better." So this very collaborative notion of working with our customers being researched by themselves, just a little bit more focused on the data side, and working hand on hand with them to do whatever it takes to make them the best.

**Edwin Chen** (00:45:57):
And then we also have our internal researchers. So our internal researchers are focused on slightly different things. So they are focused on building better benchmarks and better leaderboards.

**Edwin Chen** (00:46:07):
So I've talked a lot about how I worry that the leaderboards and benchmarks out there today are steering models in the wrong direction, so yeah, so the question is, how do we fix that? And so that's what our research team is focused focused really heavily on right now. So they're working a lot on that.

**Edwin Chen** (00:46:23):
And they're also working on these other things like, "Okay, we need to train our own models to see what types of data performs the best, what types of people perform the best." And so they're also working on all these training techniques and evaluation of our own data sets to improve our data operations and the internal data products that we have that determine what makes something good quality.

**Lenny Rachitsky** (00:46:46):
It's such a cool thing because I don't think basically the labs have researchers helping them advance AI. I imagine it's pretty rare for a company like yours to have researchers actually doing primary research on AI.

**Edwin Chen** (00:46:59):
Yeah, I think it's just because it's something I've fundamentally always cared about. I often think about us more like a research lab than a startup because that is my goal. It's kind of funny, but I've always said I would rather be Terrence Tau than Warren Buffett, so that notion of creating research that pushes the frontier forward and not just getting some valuation, that's always been what drives me.

**Lenny Rachitsky** (00:47:25):
And it's worked out. That's the beautiful thing about this. You mentioned that you were hiring researchers, is there anything there you want to share folks you're looking for?

**Edwin Chen** (00:47:32):
So we look for people who are just fundamentally interested in dataset all day. So types of people who could literally spend 10 hours digging through a dataset, and playing around with models, and thinking, "Okay, yeah, this is where I think the model's failing," this is the kind of a behavior you want the model to have instead, and just this aspect of being very hands-on and thinking about the qualitative aspects of models and not just the quantitative parts. So again, it's like this aspect of being hands-on with data and not just caring about these kind of abstract algorithms.

**Lenny Rachitsky** (00:48:07):
Awesome.

**Lenny Rachitsky** (00:48:07):
I want to ask a couple broad AI kind of market questions. What else do you think is coming in the next couple of years that people are maybe not thinking enough about or not expecting in terms of where AI is heading? What's going to matter?

**Edwin Chen** (00:48:20):
I think one of the things that's going to happen in the next few years is that the models are actually going to become increasingly differentiated because of the personalities and behaviors that the different labs have and the kind of objective functions that they are optimizing their models for. I think it's one thing I didn't appreciate a year or so ago.

**Edwin Chen** (00:48:45):
A year or so ago, I thought that all of the AI models would essentially become very commoditized. They would all behave like each other, and sure, one of them might be slightly more intelligent in one way today, but sure, the other ones would catch up in the next few months. But I think over the past year, I've realized that the values that the companies have will shape the model.

**Edwin Chen** (00:49:09):
So let me give you an example. So I was asking Claude to help me draft an email the other day, and it went through 30 different versions. And after 30 minutes, yeah, I think it really crafted me the perfect email, and I sent it. But then I realized that I spent 30 minutes doing something that didn't matter at all. Sure, now I got the perfect email, but I spent 30 minutes doing something I wouldn't have worried at all before, and this email probably didn't even move the needle on anything anyways.

**Edwin Chen** (00:49:35):
So I think there's a deep question here, which is, if you could choose the perfect model behavior, which model would you want? Do you want a model that says, "You're absolutely right. There are definitely 20 more ways to improve this email," and it continues for 50 more iterations. And it sucks up all your time and engagement. Or do you want a model that's optimizing for your time and productivity and just says, "No, you need to stop. Your email's great. Just send it and move on with your day"?

**Edwin Chen** (00:49:59):
And again, just because... In the same way that there's like a kind of a fork in a road between how you could choose how your model behaves for this question, it's like for every other question that models have, the kind of behavior that you want will fundamentally affect it.

It's almost like in the same way that when Google builds a search engine, it's very different from how Facebook would build a search engine, which is very different from how Apple would build a search engine. They all have their own principles and values and things that they're trying to achieve in the world that shape all the products that they're going to build. And in the same way, I think all the [inaudible 00:50:40] will start behaving very differently too.

**Lenny Rachitsky** (00:50:41):
That is incredibly interesting. You already see that with Grok. It's got a very different personality and a very different approach to answering questions. And so what I'm hearing is you're going to see more of this differentiation.

**Edwin Chen** (00:50:52):
Yep.

**Lenny Rachitsky** (00:50:53):
Kind of another question along these lines, what do you think is most under-hyped in AI that you think maybe people aren't talking enough about that is really cool? And what do you think is over-hyped?

**Edwin Chen** (00:51:04):
So I think one of the things that's under-hyped is the built-in products that all of the chatbots are going to start having. I've always been a huge fan of Claude's artifacts. And I think it just works really well. And actually the other day, I don't know if it's a new feature or not, but it asked me to help me create an email, and then it just created... So it didn't quite work because it didn't allow me to send the email. But what it created instead was like a little, I don't know what we call it, like a little box where I could click on it and it would just text someone that did this message. And I think that concept of taking artifacts to the next level where you just have these mini apps, mini UIs within the chatbots themselves, I feel like people aren't talking enough about that. So I think that that's one under-hyped area.

**Edwin Chen** (00:51:54):
And in terms of over-hyped areas, I definitely think that vibe coding is over-hyped. I think people don't realize how much it's going to make your systems unmaintainable in the long-term and they simply dump this code into their code bases if this seems to work out right now, so I kind of worry about the future of coding. It's just going to keep on happening.

**Lenny Rachitsky** (00:52:17):
These are amazing answers. On that first point, there's something I actually asked. I have the chief product officer of Anthropic and OpenAI, Kevin Weil and Mike Krieger on the podcast, and I asked them just like, "As a product team, you have this gigabrain intelligence. How long do you even need product teams?" You think this AI will just create the product for you. "Here's what I want." It's like the next level of vibe coding. It's just like tell it, "Here's what I want," and it's just building the product and involving the product as you're using it. And it feels like that's what you're describing is where we might be heading.

**Edwin Chen** (00:52:48):
Yeah, I think there's a very powerful notion where it helps people just achieve their ideas in a much cooler way.

**Lenny Rachitsky** (00:52:55):
Something we haven't gotten into that I think is really interesting is just the story of how you got to starting Surge. You have a really unique background. I always think about these... Brian Armstrong, the founder of Coinbase, once gave this talk that has really stuck with me where he kind of talked about how his very unique background allowed him to start Coinbase. He had a economics background, he had a cryptography experience, and then he was an engineer. And it's like the perfect Venn diagram for starting Coinbase, and I feel like you have a very similar story with Surge. Talk about that, your background there, and how that led to Surge.

**Edwin Chen** (00:53:31):
Going way back, I was always fascinated by math and language when I was a kid. I went to MIT because it's obviously one of the best places for math and CS, but also because it's the home of Noam Chomsky. My dream in school was actually to find some underlying theory connecting all these different fields.

And then I became a researcher at Google, and Facebook, and Twitter, and I just kept running into the same problem over and over again. It was impossible to get the data that we needed to train our models. So I was always this huge believer in the need for high quality data, and then GPT-3 came out in 2020. And I realized that, yeah, if we wanted to take things to the next level and build models that could code, and use tools, and tell jokes, and write poetry, and solve [inaudible 00:54:12], and cure cancer, then yeah, we were going to need a completely new solution.

**Edwin Chen** (00:54:16):
The thing that always drove me crazy when I was at all these companies was we had a full power of the human mind in front of us, and all the data students out there were focused on really simple things like image labeling. So I wanted to build something focus on all these advanced, complex use cases instead that would really help us build our next generation models. So yeah, I think my background in kind of across math, and computer science, and linguistics really informed what I always wanted to do, and so I started Surge a month later with our one mission to basically build the use cases that I thought were going to be needed to push the frontier of AI.

**Lenny Rachitsky** (00:54:49):
And you said a month later, a month later after what?

**Edwin Chen** (00:54:52):
After a GPT-3 launch in 2020.

**Lenny Rachitsky** (00:54:54):
Oh, okay. Wow. Okay. Yeah. A great decision.

**Lenny Rachitsky** (00:54:57):
What just kind of drives you at this point of... Other than just the epic success you're having, what keeps you motivated to keep building this and building something in this space?

**Edwin Chen** (00:55:06):
I think I'm a scientist at heart. I always thought I was going to become this math or CS professor and work on trying to understand the universe, and language, and the nature of communication. It's kind of funny, but I always had this fanciful dream where if aliens ever came to visit Earth and we need to figure out how to communicate with them, I wanted to be the one the government would call. And I'd use all this fancy math, and computer science, and linguistics to decipher it.

**Edwin Chen** (00:55:33):
So even today, what I love doing most is every time a new model is released, we'll actually do a really deep dive into the model itself. I'll play around with it, I'll run evals, I'll compare where it's improved, where it's arrest, I'll create this really deep dive analysis that we send our customers. And it's actually kind of funny because a lot of times we'll say it's from a data science team, but often it's actually just from me.

And I think I could do this all day. I have a very hard time being in meetings all day. I'm terrible at sales, I'm terrible at doing the typical CEO things that people expect you to do, but I love writing these analyses. I love jamming with our research team about what we're seeing, sometimes I'll be up until 3:00 AM just talking on the phone with somebody on the research team and [inaudible 00:56:12] model. So I love that I still get to be really hands-on, working on the data and the science all day. And I think what drives me is that I want Surge to play this critical role in the future of AI, which I think is also the future of humanity. We have these really unique perspectives on data, and language, and quality, and how to measure all of this, and how to ensure it's all going on the right path. And I think we're uniquely unconstrained by all of these influences that can sometimes steer companies in a negative direction.

Like what I was saying earlier, we built Surge a lot more like a research lab than a typical startup. So we care about curiosity and long-term incentives and intellectual rigor, and we don't care as much about quarterly metrics and what's going to look good in a [inaudible 00:56:56]. And so my goal is to take all these unique things about us as a company and use that to make sure that we're shaping AI in a way that's really beneficial for our species in the long term.

**Lenny Rachitsky** (00:57:06):
What I'm realizing in this conversation is just how much influence you have and companies like yours have on where AI heads. The fact that you help labs understand where they have gaps and where they need to improve, and it's not just everyone looks at just like the heads of OpenAI and Anthropic and all these companies as they're the ones ushering in AI, but what I'm hearing here is you have a lot of influence on where things head too.

**Edwin Chen** (00:57:30):
Yeah, I think there's this really powerful ecosystem where, honestly, people just don't know where models are headed and how they want to shape them yet and how they want humanity kind of like [inaudible 00:57:47] play a role in the future of all of this, and so I think there's a lot of opportunity to just continue shaping the discussion.

**Lenny Rachitsky** (00:57:52):
Along that thread, I know you have a very strong thesis on just why this work matters to humanity and why this is so important, talk about that.

**Edwin Chen** (00:58:01):
I'll get a bit philosophical here, but I think the question itself is a bit philosophical, so bear with me. So the most straightforward way of thinking about what we do is we train and evaluate AI. But there's a deeper mission that I often think about, which is helping our customers think about their dream objective functions. Like yeah, what kind of model do they want their model to be? And once we help them do that, we'll help them train their model to reach their north star and we'll help them measure that progress. But it's really hard because objective functions are really rich and complex. It's kind of like the difference between having a kid and asking them, "Okay, what test do you want to pass? Do you want them to get a high score on SAT and write a really good college essay?" That's a simplistic version versus what kind of person do you want them to grow up to be? Will you be happy if they're happy no matter what they do or are you hoping they'll go to a good school and be financially successful?

And again, if you take that notion, it's like, okay, how do you define happiness? How do you measure whether they're happy? How do you measure whether they're financially successful? It's a lot harder than something measuring whether or not you're getting a high score on the SAT, and what we're doing is we want to help our customers reach, again, their dream north stars and figure out how to measure them. And so I talked about this example of what you want models to do when you're asking them to write 50 different evaluations. Do you just continue them for 50 more or do you just say, "No, just move on [inaudible 00:59:25] because this is perfect enough." And the broader question is, are we building these systems that actually advance humanity? And if so how do we build the data sets to train towards that and measure it? Are we optimizing for all of these wrong things, just systems that suck up more and more of our time and make us lazier and lazier?

**Edwin Chen** (00:59:44):
And yeah, I think it's really relevant to what we do because it's very hard and difficult to measure and define whether something is genuinely advancing humanity. It's very easy to measure all these proxies instead like clicks and likes. But I think that's why our work is so interesting. We want to work the hard, important metrics that require the hardest types of data and not just the easy ones. So I think one of the things I often say is you are your objective function. So we want the rich, complex, objective functions and not these simplistic proxies. And our job is to figure out how to get the data to match this.

**Edwin Chen** (01:00:12):
So yeah, we want data, we want metrics that measure whether AI is making your life richer. We want to train our systems this way. And we want tools that make us more curious and more creative, not just lazier. And it's hard because, yeah, humans are kind of inherently lazy so AI software deals are the easiest way to get engagement, make all your metrics fall up. So I think this question about choosing the right objective functions and making sure that we're optimizing towards them and not just these easy proxies is really important to our future.

**Lenny Rachitsky** (01:00:37):
Wow. I love how what you're sharing here gives you so much more appreciation of the nuances of building AI, training AI, the work that you're doing.

**Lenny Rachitsky** (01:00:45):
From the outside, people could just look at Surge and companies in the space of, okay, cool. They're just creating all this data, feeding it to AI. But clearly there's so much to this that people don't realize, and I love knowing that you're at the head of this, that someone like you is thinking through this so deeply.

**Lenny Rachitsky** (01:01:02):
Maybe one more question, is there something you wish you'd known before you started Surge? A lot of people start companies, they don't know what they're getting into. Is there something you wish you could tell your earlier self?

**Edwin Chen** (01:01:11):
Yeah, so I definitely wish I'd known that you could build a company by being heads down and doing great research and simply building something amazing. And not by constantly tweeting and hyping and fundraising. It's kind of funny, but I never thought I wanted to start a company. I love doing research. And I was actually always a huge fan of DeepMind because they were this amazing research company that got bought and still managed to keep on doing amazing science. But I always thought that they were this magical ILR unicorn. So I thought if I started a company, I'd have to become a business person looking at financials all day and being in meetings all day and doing all this stuff that sounded incredibly boring and I always hated. So I think it's crazy that didn't end up being true at all. I'm still in the weeds in the data every day. And I love it. I love that I get to do all these analyses and talk to researchers. And it's basically applied research where we're building all these amazing data systems that have really pushed the frontier of AI.

So yeah, I wish I know that you don't need to spend all your time fundraising. You don't need to constantly generate hype. You don't need to become someone you're not. You can actually build a successful company by simply building something so good that it cut through all that noise. And I think if I known this was possible, I would've started even sooner, so I [inaudible 01:02:18] that.

**Lenny Rachitsky** (01:02:18):
And that is such an amazing place to end. I feel like this is exactly what founders need to hear, and I think this conversation's going to inspire a lot of founders, and especially a lot of founders that want to do things in a different way. Before we get to a very exciting lightning round, is there anything else you wanted to share? Anything else you want to leave our listeners with? We covered a lot of ground, it's totally okay to say no as well.

**Edwin Chen** (01:02:37):
I think the thing I would end with is I think a lot of people think of data labeling as it relates to simplistic work. Like labeling cat photos and drawing bounding box around cars. And so I've actually always hated the word data labeling because it just paints this very simplistic picture when I think what we're doing is completely different. I think a lot about what we're doing as a lot more like raising a child. You don't just feed a child information. You're teaching them values, and creativity, and what's beautiful, and these infinite subtle things about what makes somebody a good person. And that's what we're doing for AI. So yeah, I just often think about what we're doing as almost like the future of humanity or how we're raising humanity's children, so I'll leave it at that.

**Lenny Rachitsky** (01:03:27):
Wow. I love just how much philosophy there is in this whole conversation that I was not expecting.

**Lenny Rachitsky** (01:03:33):
With that, Edwin, we've reached our very exciting lightning round, I've got five questions for you. Are you ready?

**Edwin Chen** (01:03:38):
Yep, let's go.

**Lenny Rachitsky** (01:03:39):
Here we go. What are two or three books that you find yourself recommending most to other people?

**Edwin Chen** (01:03:45):
Yes, so three books I often recommend are, first, Story of Real Life by Ted Chang. It's my all time favorite short story and it's about a linguist learning and alien language, and I basically reread it every couple years.

**Lenny Rachitsky** (01:03:56):
And that's what the Interstellar was about? Is that...

**Edwin Chen** (01:03:59):
Yeah, so there's a movie called Arrival...

**Lenny Rachitsky** (01:04:01):
Arrival.

**Edwin Chen** (01:04:02):
... which was based off of the story,

**Lenny Rachitsky** (01:04:03):
Yes, [inaudible 01:04:03]-

**Edwin Chen** (01:04:03):
... which I love as well.

**Lenny Rachitsky** (01:04:04):
Great. Okay, keep going.

**Edwin Chen** (01:04:06):
And then second, Myth of Sisyphus by Camus. I actually can't really explain why I love this, but I always find a final chapter somehow are really inspiring.

**Edwin Chen** (01:04:15):
And then third, Le Ton beau de Marot by Douglas Hofstadter. And so I think Gödel, Escher, Bach is his more famous book, but I've actually always loved this one better. It basically takes a single French poem and translates it 89 different ways and discusses all the motivations behind each translation. And so I've always loved the way it embodies this idea that translation isn't this robotic thing that you do. Instead, there's a million different ways to think about what makes a high quality translation, which makes a lot of ways I think about data and quality in LLMs.

**Lenny Rachitsky** (01:04:44):
All these resonate so deeply with the way, with all the things we've been talking about, especially that first one, if that was your goal after school is like, "I want to help translate alien language." I'm not surprised you love that short story.

**Lenny Rachitsky** (01:04:56):
Next question, do you have a favorite recent movie or TV show you've really enjoyed?

**Edwin Chen** (01:05:00):
One of my new all time favorite TV shows is something I found recently, it's called Travelers. It's basically about a group of travelers from the future who are sent back in time to prevent their [inaudible 01:05:11]. Sorry, I just wrote that [inaudible 01:05:13] section.

**Edwin Chen** (01:05:14):
And then I actually just rewatched Contact, which is one of my all time favorite movies. So yeah, I think one of the things you'll notice about me is that, yeah, I love any kind of book or film that involves scientists deciphering alien communication. Again, just this dream I always had as a kid.

**Lenny Rachitsky** (01:05:28):
That's so funny [inaudible 01:05:29].

**Lenny Rachitsky** (01:05:30):
Okay, is there a product you've recently discovered that you really love?

**Edwin Chen** (01:05:35):
So it's funny, but I was in SF earlier this week and I finally took Waymo for the first time. Honestly, it was magical and it really felt like living in the future.

**Lenny Rachitsky** (01:05:43):
Yeah, it's like the thing that... People hype it like crazy, but it always exceeds your expectations.

**Edwin Chen** (01:05:48):
Yeah, it deserves the hype. It was crazy. Yeah, it's absurd. It's like, holy moly. If you're not in SF, you don't realize just how common these things are. They're just all over the place. Just driverless cars constantly going about, and when you go to an event at the end, there's just all these Waymos lined up picking people up.

**Lenny Rachitsky** (01:06:03):
Yeah. Waymo, good job. Good job over there.

**Lenny Rachitsky** (01:06:06):
Do you have a favorite life motto that you find yourself coming back to in work or in life?

**Edwin Chen** (01:06:11):
So I think I mentioned this idea that founders should build a company that only they could build. Almost like it's this destiny that their entire life, and experiences, and interests shape them towards. And so I think that principle applies pretty broadly, not just the founders, but the people creating, I think.

**Lenny Rachitsky** (01:06:25):
Well, let me follow that thread to unlightening this answer. Do you have any advice for how to build those sorts of experiences that help lead to that? Is it follow things that are interesting to you, because it's easy to say that, it's hard to actually acquire these really unique sets of experiences that allow you to create something really important?

**Edwin Chen** (01:06:44):
Yeah, so I think it would always be to really follow your interests and do what you love, and it's almost like a lot of decisions I make about Surge. I think one of the things that I didn't think about a couple years ago, but then someone said it to me, it's that companies in a sense are an embodiment of their CEO. And it's kind of funny. I hadn't thought about that because I never quite knew what a CEO did. I always thought a CEO was kind of generic and it's like, okay, you're just doing whatever VPs, and your board, and whatever, tell you to do and you're just saying yes to decisions. But instead, it's this idea where when I think about certain big, hard decisions we have to make, I don't think what would the company do, I don't think what metrics are we trying to optimize, I just think, "What do I personally care about? What are my values? And what do I want to see happen in the world?"

**Edwin Chen** (01:07:34):
And so I think following that idea about... Okay, so ask yourself, what are the values you care about? What are things you're trying to shape and not... What will look good on a dashboard? I think that results are pretty important.

**Lenny Rachitsky** (01:07:49):
I love how just you're just full of endless, beautiful, and very deep answers.

**Lenny Rachitsky** (01:07:55):
Final question. Something that you got quite famous for before starting Surge is you built this map while you were at Twitter that showed a map of the world and what people called, whether they called it soda or pop. I don't know if it's called Soda Pop. What was the name of this map?

**Edwin Chen** (01:08:13):
Yeah, it was like the Soda Versus Pop dataset.

**Lenny Rachitsky** (01:08:15):
Soda Versus Pop.

**Edwin Chen** (01:08:17):
[inaudible 01:08:17]

**Lenny Rachitsky** (01:08:16):
And so it's like a map of the United States and it tells you where people say pop versus soda, so do you say soda or pop?

**Edwin Chen** (01:08:23):
So I say soda, I'm a soda person.

**Lenny Rachitsky** (01:08:26):
Okay. And is that just like that's the right answer or it's like whatever you are, it's totally fine.

**Edwin Chen** (01:08:33):
I think I'll look at you a little bit funny. You say pop and I'll wonder where you came from, but I won't score on you too much.

**Lenny Rachitsky** (01:08:39):
That's how I feel too.

**Lenny Rachitsky** (01:08:40):
Edwin, this was incredible. This was such an awesome conversation. I learned so much. I think we're going to help a lot of people start their own companies, help their companies become more aligned with their values and just building better things.

**Lenny Rachitsky** (01:08:53):
Few final questions, where can folks find you online if they want to reach out? What roles are you hiring for? How can listeners be useful to you?

**Edwin Chen** (01:09:00):
Yeah, so I used to love writing a blog, but I haven't had time in the past few years. But I am starting to write again, so definitely check out the Surge blog, surgehq.ai/blog, and yeah, hopefully I'll be running a lot more there. And I would say we're definitely always hiring, so for people who just love data and people who love this intersection of math, and language, and computer science, definitely reach out anytime.

**Lenny Rachitsky** (01:09:24):
Awesome. And how can listeners be useful to you? Is it just, I don't know, yeah, is there anything there? Any asks?

**Edwin Chen** (01:09:29):
So I would say definitely tell me blog topics that you like me to write about...

**Lenny Rachitsky** (01:09:29):
Okay.

**Edwin Chen** (01:09:32):
... and then I'm always fascinated by all of these AI failures that happen in the real world. So whenever you come across a really interesting failure that I think illustrates some deep question about how we want model to behave, there's just so many different ways a model can respond, I just oftentimes think there's just not a single right answer. And so whenever there's one of these examples, I just love seeing them.

**Lenny Rachitsky** (01:09:57):
You need to share these on your blog. I'm also... I would love to see these.

**Lenny Rachitsky** (01:10:01):
Edwin, thank you so much for being here.

**Edwin Chen** (01:10:03):
Thank you.

**Lenny Rachitsky** (01:10:04):
Bye everyone.

**Lenny Rachitsky** (01:10:07):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [16/17] Elena Verna 4.0

**Lenny Rachitsky** (00:00:00):
You're ahead of growth at Lovable on track to be the fastest or one of the fastest growing companies in history.

**Elena Verna** (00:00:06):
We're over 200 million in ARR. At this point, we're 100 people large, the pace here is insane.

**Lenny Rachitsky** (00:00:12):
You said that you've had to throw out most of your growth playbook.

**Elena Verna** (00:00:14):
I feel like only 30 to 40% of what I've learned in the last 15 to 20 years of being in growth transfers here because we just need to invest in such bigger bets, and innovate, and create new growth loops here, everybody and their mother is starting a vibe coding business nowadays, and we need to figure out how to be ahead of them. And to be ahead of them is not optimization of the problem, it's reinvention of the solution. I just feel like I usually spend maybe 5% innovating on growth in my previous roles, right now, I'm spending 95% innovating on growth, and only 5% on optimization.

**Lenny Rachitsky** (00:00:48):
What do you find is actually moving the needle?

**Elena Verna** (00:00:50):
One of our biggest strategy is building in public, and it's coupled with employee socials, founder-led socials. And another one is giving your product away a lot, this is part of our growth secret sauce. You have to remove the barrier of entry. If somebody, one of our users stands up and say, hey, I'm going to have a hackathon at my work on Lovable, can you give us some free credits to play with? Why would we prevent a person who wants to do all of the marketing and activating for us from using us? We're like, take it, how much do you need?

**Lenny Rachitsky** (00:01:22):
The trick is get more people to try it, just ship things you can talk about.

**Elena Verna** (00:01:25):
The only way to create a word of mouth loop is just to blow their socks off.

**Lenny Rachitsky** (00:01:31):
Today, my guest is Elena Verna, head of growth at Lovable. In under one year after launching, with fewer than 100 people, Lovable hit 200 million ARR, which is one of, if not the fastest ramp to 200 million ARR in history, and growth is still accelerating. They've also recently raised a series B at a $6 billion valuation. So, with that, there's a lot to learn about what Lovable has figured out about growth. This is Elena's fourth visit to the podcast, a record, she is my favorite growth mind, and in our conversation, we talk about how the growth playbook has fundamentally changed for AI companies. What works now, what no longer works, and what has surprised her most about how Lovable grows.

**Lenny Rachitsky** (00:02:13):
She also shares her advice about whether working at an AI company is right for you, some incredibly interesting insights into Lovable's secret sauce for growth, the unique ways they operate internally, their approach to building minimal, Lovable products, also how they hire, and also how product market fit as a concept is no longer what it used to be, and how every company basically has to recapture product market fit every three months. This episode is incredibly tactical, and you will leave this conversation smarter on so many levels. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube.

**Elena Verna** (00:05:24):
Thank you for having me.

**Lenny Rachitsky** (00:05:26):
As you know, this is a record fourth time back to the podcast, no one else has ever achieved this feat. I feel like you're basically my co-host now.

**Elena Verna** (00:05:36):
I love it. Thank you for inviting me back, I'm very proud record holder in this regard.

**Lenny Rachitsky** (00:05:41):
What I love about you coming back each time, it feels like every time you come back, you're just doing something even more epic and exciting. And so, these days, as we'll hear in the intro, you're head of growth at Lovable, which no big deal on track to be the fastest or one of the fastest growing companies in history, depending on the metric that you track. Let's talk about just the scale and growth of Lovable to give people a sense of just how incredible this is. I'll share a bit of this in intro, but just what are some stats you can share about just how things are going Lovable?

**Elena Verna** (00:06:08):
So, we are just a little bit over one year's old since we launched. The company actually did exist as a GPT engineer before, but it officially launched in the third week of November last year, in 2024. So, for us, we've hit over $200 million in annual recurring revenue before we even hit our one-year milestone since being launched. Which is pretty incredible. You, Lenny, actually have a really great blog post on how quickly it takes for companies usually to get to their first million ARR, and it's usually multiple years. So, this is definitely a unicorn, I don't think this is a standard. There's a couple of things that account for it, and we can talk about it, and the growth is only accelerating, so it's compounding, which is great. Because we had our 100 million in end of July, and just four months later we were at 200 million.

**Elena Verna** (00:07:02):
So, seven months to, well, maybe eight months, to 100 million, another four months to get to 200 million. And from users too, we already have over eight million users that have tried Lovable, we have, as you can imagine, to feed that 200 million, hundreds of thousands of paid subscribers as well that are paying for us, so things are going great and we'll talk about why.

**Lenny Rachitsky** (00:07:27):
Okay. Absurd. I think people are getting used to these insane numbers, and not long ago is like, okay, if you hit a million ARR in a year, you're doing pretty well.

**Elena Verna** (00:07:36):
Yeah, yeah. I think still you're doing pretty well if you have a million ARR in one year. This is one of the once in a lifetime type of companies, and the category, the way that it's evolving. So, I want to make sure that people don't all of a sudden set this as a benchmark for success because it should never be. In some categories, it might be even faster as we continue evolving technology, but I don't think that it's realistic to expect it out of your business that you're starting right now.

**Lenny Rachitsky** (00:08:08):
That is such an important point you're making there, it's so discouraging to founders to hear these stories of getting 200 million. And again, this is ARR. There's a lot of companies, especially in the data labeling space, I've had them all on the podcast, that are very fast-growing, but they're not recurring revenue. There's also, they pay out their people to do this data labeling. So, the revenue numbers there don't really equate. Recurring $200 million a year is absurd.

**Elena Verna** (00:08:36):
Yeah, it is absurd. I really want to make sure that people understand as we go through this episode as to why it's happening, because part of it is on Lovable, part of it is just in the market and how it's moving. So, when you're setting yourself as a benchmark, so you know which benchmarks you actually to use, and whether Lovable is the benchmark that you should be using.

**Lenny Rachitsky** (00:08:56):
Cool. I'm going to get into that next. Last question, just I want to see what you can share here. A lot of people look at these numbers, a lot of people are very skeptical these are lasting, durable numbers. Like who are these people? How is there $200 million being spent on Lovable? Anything you can add about, just give people confidence, is this real, this is going to last, this is a really durable business.

**Elena Verna** (00:09:19):
Well, I saw Stripe receipts, so it is real, as far as I'm concerned, unless Stripe dashboard is lying to us. But it is money getting deposited in our bank account. But let's talk about who is actually contributing to that number. We do have a really large use case of people starting their own companies on Lovable, so we call it a founder use case. Where somebody that is non-technical, that has never been able to code or create a piece of software is now able to come in and actually build an app completely from scratch. And some of them are already monetizing it, some of it just using it [inaudible 00:09:52] for other services, or some physical goods, for example, that they're selling. Some of them are just still building. And we monetize on the act of building, so that progression of building up to your product market fit takes quite a bit of time.

**Elena Verna** (00:10:06):
And even with Lovable, we're so much more efficient and effective compared to hiring an engineer in terms of the price, but it still takes time. So, we have a lot of founders, whether it's B2C, whether there's B2B, so consumer products, business products, e-commerce, whatever it is. But on the other side, we have a lot of employees within companies using Lovable as well, where they're building internal tools, or they're building prototypes, they're building landing pages. So, that is another use case that is very relevant and quite efficient for us, but then there is a hype and discovery that is happening as well. Because when I think about software, I think about it, I talked to John Cutler actually, and he gave me this framework that is completely stuck in my mind, of software always goes through capabilities stage first, so what is possible to actually create with this?

**Elena Verna** (00:11:00):
Then, it needs to transition into value, of how is it that am I going to get value out of this? And then you can start thinking about scaling it, of which aspects of my life and my work that can actually go in? And we're right now very much in the capability stage with vibe coding, because everybody's just exploring, what can I do? And the beautiful thing here is that what you can do changes every month to three months. So, you constantly need to come back and you need to see what has changed. So, a lot of people use it for personal reasons. I build myself apps, tutoring apps for my kid, so he has to answer questions in order to get some screen time accumulated for him. I build my own portfolio.

**Elena Verna** (00:11:40):
I see people doing wonderful things, my favorite story that I always say, there's this man that created a proposal on Lovable. So, his fiance had to answer questions and she had to complete this game, and then at the end there was this big reveal and he proposed to her. But people just unlock the most creative things that they build on Lovable, and that's where the revenue is coming from. The one piece that is working very well for us in terms of how our monetization model is, set up and how it interjects with your activation moment, which we can also cover, but that is what's driving both conversion and retention rates.

**Lenny Rachitsky** (00:12:17):
Let me ask you one question that's on people's minds, I imagine, as you talk about this, just what does retention look like?

**Elena Verna** (00:12:22):
Yeah. So, retention, really, I look at it in two ways, retention, that it comes as a subscriber retention. So, how much [inaudible 00:12:32] subscribers do we get, and how many of them are we capable of renewing? There's also very important aspect of it is, how many of them can we expand? Because if you can get positive or above 100% net dollar retention, which is super important metric for investors... If you don't know about net dollar retention, please read it up, that's like a superpower to get bigger multiple if you can show NDR that is over 100. And then there's actually engagement retention as well, because that is the leading indicator for how your paid retention is going to look like. For paid retention, I know there is so much on the market of, oh, this is a high product, and it's a leaky bucket, and it has really high churn rates.

**Elena Verna** (00:13:10):
Although, I shouldn't share, it's not public numbers for us to share actual retention, however, what I can say, it's on par with benchmarks of other B2B SaaS products that I've ever worked at. And I worked with Miro, Dropbox, SurveyMonkey, Netlify, Amplitude, and others. So, are we absolutely crushing with paid retention? No. Are we where most of the other companies are? Yes. Our NDR is quite good because when people build, they want to buy more credits to build. So, we're seeing really good revenue retention, but we're honestly more focused right now on engagement retention than even paid retention because our North Star is just to get as much usage as possible, and we will fix and tune our monetization model afterwards. So, engagement retention, I would say, is a by far bigger priority focus for us at the moment.

**Lenny Rachitsky** (00:14:04):
That is incredibly interesting, and I'm optimistic to hear because of the growth rate. Rarely is growth rate this high and retention is on par with great companies.

**Elena Verna** (00:14:15):
Yeah. And I'll just say too, which is a little bit maybe counterintuitive would be to a lot of companies, we don't optimize for revenue at all. In fact, internally, we have a lot of discussions about how can we give more products away, how can we reduce our revenue growth rate by just getting more paid subscribers, more users using Lovable, to just get bigger share of the market. So, our revenue is an outcome of us just trying to get more people through the door, not us trying to optimize for revenue per user, or to get them to monetize at the higher rate. So, there's a very interesting path here, where, by actually focusing on the inputs, like you should, it translates to a good output, but we don't look at that output as something that we're trying to grow.

**Lenny Rachitsky** (00:15:02):
Let's talk about growth, let's talk about what you've learned about growth in this space. You had this post online where you said that you've had to throw out most of your growth playbook. This is a huge deal, you've led growth a lot of really successful companies. Lovable is growing incredibly well, this tells me there's a lot we can learn from what you've seen. So, tell us what you're seeing, what's still working, what's not working, what you've learned about what it takes to drive growth at a company like Lovable.

**Elena Verna** (00:15:28):
Yeah. I would say that in any other role that I've come into before, I felt confident in about 80% of the patterns that I can bring to that role, meaning that I can identify inputs, understand which framework applies. I know a lot of examples that fit in within that framework, so we just need to localize a solution and push, and it was quite productive in terms of getting a company those additional acquisition, conversion, engagement, monetization rates. So, I felt very repetitive in a way, after some time, because I feel like I'm just coming in and copy, pasting, copy, pasting... And although every single company loves to say that they have unique problems, at the end of the day, all of the problems were very similar. And I felt like I was doing the same job over and over again. When I started at Lovable, the one thing to me that was very clear is that this company was growing like crazy before I joined.

**Elena Verna** (00:16:21):
So, I want to make sure that there's not that much value on what I have even added today, because this company is on the tear, and yes, we're rounding the edges and removing barriers for growth, so we're not standing in our own way, but there's something more magical happening here that is not a pattern that I've ever seen before. It's not a framework that I can even conceptualize in my head. And plus, it's a new category that I've never seen, or I've never been in a company that is in a new emerging category, that hits fast moving water so quickly. And that's the difference because when you're usually trying to create a new category, it takes years. I know it's every marketer's dream to create a new category, but it takes decades often to really get that much hype and adoption around it. Versus with vibe coating, this hype seemed to have happened really quickly.

**Elena Verna** (00:17:16):
It's like it's hit the nerve with the market. So, yes, we're at the right place, we're at the right time, but we're also in really fast moving waters, and the demand that is coming to us, we need to capture it mostly, we don't need to generate a lot of it yet. But at the same time, it comes with the really big downfalls of we're not in control of a lot of our growth. Let's be honest about it, there's so much incredible word of mouth that is happening, and we're trying to grow that, but to enable as much of that as possible, but the company is moving, we're just hanging on to it as fast as possible, and making sure that we're not going to hit a wall, so to speak, in front of us, and that the wheels are greased, and that all of the pieces are in places. It's like your race car framework that you have as well.

**Elena Verna** (00:18:05):
We're really just putting a lot of oil into it, and figuring out what is our engine actually going to be that is going to take us forward. But when I'm thinking about the patterns here, and what I have to unlearn, I feel like only 30 to 40% of what I've learned in the last 15 to 20 years of being in growth transfers here. And some of it is very straightforward, okay, this is how you're going to do paid marketing, this is how you're going to do some of the habitual retention, here's the free to pay maybe monetization frameworks that still stand. But the rest of it, honestly, it doesn't feel like it even matters anymore, because we just need to invest in such bigger bets, and innovate, and create new growth loops here, as opposed to trying to optimize it to the moon and beyond, which I usually be focused on in a scaled business like this.

**Lenny Rachitsky** (00:18:57):
Let's follow those threads. So, what is it that no longer is worth it in this bucket of just like, let's not spend any time on this thing, and then what do you find is actually moving the needle?

**Elena Verna** (00:19:06):
Not worth it, in growth, most of the people spend most of the time optimizing existing user journeys. So, you already have maybe some of your growth loops that you understand that you try to optimize or you just know, hey, there's big drop-offs from acquisition to activation, let me go figure out how to... I can tweak the dials to get it done. Here, what I find is that optimizations are just not worth our time. So, a lot of the times my growth team actually ends up working on new features, or just standing up new growth loops one after another, and yes, there's of course the saying of more growth loops does not mean more growth, but at the same time, the market is moving so quickly, you need to stand up a bunch of initiatives to capture it because it's perishable.

**Elena Verna** (00:19:54):
Or we also have so much competition. We're not alone here, so we can't ignore that there's everybody in their mother is starting a vibe coding business nowadays, and we need to figure out how to be ahead of them. And to be ahead of them is not optimization of the problem, it's reinvention of the solution. So, I just feel like I usually spend maybe 5%, maybe 10% if I'm lucky, innovating on growth in my roles, in my previous roles, right now, I'm spending 95% innovating on growth, and only 5% on optimization. And most of my frameworks are on optimization because it's really hard to come up with frameworks for innovation because by default there, by definition, they're innovative.

**Lenny Rachitsky** (00:20:38):
What I'm hearing here is new features. Launching new features, [inaudible 00:20:42] product is one of the bigger growth levers, versus you have a bunch of cool stuff, make it easier to use, increase activation, reduce friction, things like that.

**Elena Verna** (00:20:51):
Yeah. And for example, we on growth team launched integration with Shopify, to enable e-commerce use case, because we're like, hey, there's already people trying to come in and do it, Shopify was open for integration with us, let's go lean into it so people can vibe code their storefronts. That came out of growth, that usually would never come out of growth. Why would growth team ever invest into a core product integration? Or we enabled voice mode for people so they can actually chat with Lovable using their voice, as opposed to only having type. And that's also, it's a feature, it's a core product feature, but we're like, hey, it's going to help people to converse with Lovable more, it's going to increase the engagement.

**Elena Verna** (00:21:32):
One area that we've spent very little time in is activation, because usually I spend majority of my time in activation because there's so many awareness things that need to happen, and so many things that we need to smooth out experience for the users in order for them to get through that setup moment, to aha moment, to the habit loop, and here you're just interacting with agent. So, we, at the beginning, we're like, the agent team that we have here is working a lot on it. Why would we go in there and do anything? It's like our core team is responsible for activation. Now, we're starting to move into doing agent work ourselves, so all of a sudden growth team is not just doing product surfaces, now we're doing agentic workflows and codifying agent instructions in order for customers to activate better.

**Elena Verna** (00:22:21):
So, the work fundamentally, I feel like has gotten deeper into product, and deeper into actual core product functionality, as opposed to just being a smoothing surface on the outer layers.

**Lenny Rachitsky** (00:22:35):
Okay, that is also a very big deal, every growth person that's ever been on this podcast, including you, always talks about the power of activation, just how much opportunity there is to get people to this aha moment, realize the value of this product, that increases retention and increases everything. And what you're saying here is you barely spend any time on activation because in a company like Lovable, there's a prompt, you give it what you want, it generates a thing, and that's basically all it is. And so, the impact is to make that agent better...

**Lenny Rachitsky** (00:23:00):
... basically all it is. And so the impact is to make that agent better at that thing, versus micro-optimize every step.

**Elena Verna** (00:23:07):
And our agent team spends night and day thinking about it. So I've never been at a company where core team thinks so much about activation, thinks so much about that first generation, thinks so much about reaching a-ha moment. So it's more weaved in into DNA of the overall company, which takes the pressure off of me to only have to focus on it. Because otherwise, yeah, I would be in that experience all the time.

**Elena Verna** (00:23:32):
But I feel a lot more at ease, because everybody's thinking about it and everybody's working on making agents better. And agent, the beauty of it is it doesn't matter if it's actually first generation or if it's your Nth generation. It just needs to be a better generation. Agent needs to understand your intent better, and think and reason behind it. So it improves the entire lifecycle immediately as opposed to having to only work on that first experience per se.

**Lenny Rachitsky** (00:23:59):
And what you're not saying is, "Don't care about that experience," it's, "The team building that is already obsessed with making that activation experience better and better."

**Elena Verna** (00:24:07):
Exactly, exactly. I love that, because that's the core product functionality at this point. And before people would spend more time building deeper features, or deeper use cases, or trying to improve some platform functionality. And now the core team, they're obsessed about that first experience because that is core product.

**Lenny Rachitsky** (00:24:29):
Another lever that I've noticed, especially with Lovable, and I'm seeing it more and more on social media is just founders telling you what's going on. I think this connects really deeply with the new features. Launch new features. Say Anton is just like, "Hey, check out this cool new thing. Check out our growth numbers." Is that a big growth lever too?

**Elena Verna** (00:24:47):
Yeah. So one of our biggest strategy is building in public. Building in public, and it's coupled with employee socials, founder-led socials for sure. This is difficult for larger companies. But when you're smaller and you still have a little bit more narrative control with everybody on your team, plus you have so much more trust within the organization, where people are going to say the right things because they understand what actually has happened, that ability to just really quickly deliver the message to the market becomes really important.

**Elena Verna** (00:25:21):
Now, we still do the big launches, so we still have everything tiered into tier three, two, one. Tier ones are going to happen as big moments that we're going to really rally as a company behind. And it's going to be something that is meant to step function, change our product market fit, and we're going to do a bunch of activities behind it.

**Elena Verna** (00:25:40):
But at the same time, what's really important to us is to maintain noise in the market. And that noise in the market happens by us shipping every day, every other day, multiple times per day and just talking about it constantly. Interestingly enough, it's actually works fantastic resurrection strategy, because people are like, "Oh, there's more things here. I need to go check it out."

**Elena Verna** (00:26:03):
It also works as a great reengagement strategy. So instead of sending newsletters to say, "Here's the market trends or here's the user stories," people are literally logging into their social to say, "Okay, what has Lovable shipped now?" It's like, "What is the change?" So it's interesting to them to see, because from the time that they voice their opinion on what needs to happen to actual delivery is so short, so they feel heard. And they are heard, because that's how we prioritize all of the things that we're shipping. But it's interesting, because I've never been in a company that tries to maintain so much just shipping velocity to maintain a certain amount of noise that it feels like the product is alive. It's changing every single week. And then there's these big amplifications, turbo boost so to speak, in the race car model, that then go out and they fundamentally create a step function change in that product market fit as a whole. And that is a retention strategy I can get behind any day and all day. I only hope that we can maintain it as we continue scaling.

**Lenny Rachitsky** (00:27:07):
Sounds stressful. This reminds me, I had a... Gaurav, he's the CEO of Mirage. Used to be called Captions, which is a really successful AI video company startup. And they have a policy of you ship a marketable feature every week. That's how their company operates. And it's the same thing, it's just ship things you can talk about.

**Elena Verna** (00:27:27):
Velocity of shipping is our number one core value in development team. So we do anything and everything to just keep it going up, up, up and into the right. And by the way, this also means that everybody has a little bit of marketer within them. We have very lean product organization. We actually lean on our engineers to do a lot of product work. We call them product engineers, and they have to go and they have to announce the thing that they've shipped. It doesn't just funnel through marketing.

**Elena Verna** (00:27:57):
So there is a lot of autonomy, a lot of agency that needs to happen with this type of velocity. Because otherwise, you have to have enormous marketing team to staff that. So it has to come with some roles and responsibilities. Redefinition on the team as well.

**Lenny Rachitsky** (00:28:13):
Let's talk about marketing. That's something else you've written about is just marketing is changing in a big way, their role in growth. How does marketing play a role in all of this?

**Elena Verna** (00:28:21):
On one side, marketing channels are changing. On the other side, marketing's involvement into everything that product does is changing. And then number three, I think even marketing organizations in terms of where they hire the most are changing as a result as well. So I'll talk about second one first, just because we just talked about shipping, and that is... Yeah, you still have your product marketers, you still have your channel managers. But they focus more on the big things and the narratives.

**Elena Verna** (00:28:49):
Although it's difficult, because the narrative even changes all the time. As these functionalities come through, usually you can come up with a positioning and messaging, and you can have it for years and create all of the campaigns around it. Now, you have it for three months, and then the product changes. So the cycles here are really, really short. And for smaller changes, because cycles are so short, they spend so much time actually focusing on it as they should.

**Elena Verna** (00:29:15):
But some of these smaller changes just cannot be supported by marketing. You have to delegate it to your product and engineering team to do their own marketing, because otherwise, again, you'd have to have an enormous marketing team in order to support it all.

**Elena Verna** (00:29:27):
But at the same time, channels in which marketing right now works I think are changing quite a bit. And not enough people I feel like are freaking out and talking about it, as opposed to moving just in the same direction over and over again. And the changes that I'm seeing is that it has been very clear to me that when you're talking about marketing organic strategy, if you asked me that five years ago, I would've said that's SEO. It's search engine optimization. Go on Google, that's your organic marketing strategy.

**Elena Verna** (00:29:58):
If you ask me what's your organic marketing strategy right now, to me it's all about social. Which is what is my CEO posting? What is my team posting? What is my creator economy doing? Influencer marketing, and across all of the social platforms. That is my organic, which is... That one's paid, to be fair. But when I think about organic, there is still a lot of that word of mouth. What are my users posting on social? What are they talking about it? What are they sharing?

**Elena Verna** (00:30:26):
Which is a mind shift, because I've been always... Especially in B2B, so focused on search. And now I feel like it's been completely pushed even further into consumerization territory, and it has become all about social no matter how B2B you are, because that's where eyeballs are at.

**Lenny Rachitsky** (00:30:44):
That is fascinating. And so when you talk about socials, what are you finding is most helpful? Is it Twitter/X? Is it LinkedIn? Is it YouTube, TikTok, Instagram?

**Elena Verna** (00:30:52):
For founder socials, our employee socials, X and LinkedIn are fantastic sources. Especially for B2B, because that's where all of the BBWB people are at. But you cannot just have ChatGPT write your copy and post it, you need to show personality. There needs to be humanity that it goes through it. And it's not natural for everybody, and it feels very awkward sometimes to start. But it's important to people to see who is building the company, because there's so much competition now on functionality, so they can rally behind a team.

**Elena Verna** (00:31:28):
So they want to have a team that they want to win. And for that, you need to be vulnerable, you need to be authentic, obviously, but you just need to be yourself. That corporate scrubbing has to completely fall off, which is obviously going to pull in as the company scales. But at least at the beginning, that is your chance to stand out.

**Elena Verna** (00:31:52):
And then your customers posting about you. So that word of mouth of really creating a product that creates something for customers that is worth talking about. It gives them stories that they want to share, that feels empowering to them to tell to others like they're unlocking a secret. They feel proud of what they have created.

**Elena Verna** (00:32:14):
Which what we focus a lot on, Lovable on, to have that feeling of, "Oh my gosh, I have superpowers now and I can't wait to tell others. I cannot wait to show others what is happening." So on both of those sides, to me, that is very much organic. If you aren't a consumer, then Instagram, TikTok are very much a go as well.

**Lenny Rachitsky** (00:32:35):
The CEO clearly is an important variable in this, them. In this case, Anton just tweeting, "Here's what's going on Lovable. Here's how fast it's growing. He's something we've learned."

**Elena Verna** (00:32:44):
It is.

**Lenny Rachitsky** (00:32:44):
We had the CEO of Gamma on recently, granted, and he's exactly the same thing. Just sharing a bunch of lessons, journey building in public, a big part of the growth lever. And your point here is okay, so it's the CEO. But then it's also how do you get your customers to share things on socials? And then there's a paid influencer component.

**Elena Verna** (00:33:01):
Yes, the customer is difficult one. That's a word of mouth loop that you need to stand up. The only way to create a word of mouth loop is just to blow their socks off when they actually experience your product. We have a almost unfair advantage, because our product is called Lovable. So by default, we're trying to create absolute lovable experiences. That is a mentality internally. If it's not lovable, we're not going to ship it. And the best way to fix a bug at Lovable is to say, "This is not lovable," and everybody just jumps on it to fix it right there and then. Sprints, no sprints, it doesn't matter, it's getting fixed right now.

**Elena Verna** (00:33:40):
So from that perspective, we have that culture already embedded as part of our brand and it's part of our name which helps us a lot. But the point is that you feel that brand through every interaction. I talk to my designer all the time, how can we add more love marks into the product? How can we prioritize more unique interactions? The little elements that make up that feeling of this product is speaking to me. It's like, "It feels something that is unique." It has personality behind it.

**Elena Verna** (00:34:08):
So we put all of the brand work actually into our product. When you think about Lovable, people think about a brand, but we don't have a brand marketing team yet. So it's all just through product interactions, and some of those building and public moments of the people behind those product interactions that is our strategy.

**Elena Verna** (00:34:28):
And then there's influencer marketing. Interestingly enough, influencer marketing is 10 times bigger for us than paid social. So yeah, we do some paid social as well, and it's working decently. It's quite expensive from payback period, we're still optimizing it. As I said, we're pretty early on in all of these channels. But influencer marketing is something that has worked from the beginning at Lovable. And a reason behind it is that influencer marketing, especially on the socials, it gives you an opportunity to have a little video and interaction. And Lovable is all about seeing like, "Oh my gosh, this is what I can do, and this is possible." So that drives people to go and try it themselves.

**Elena Verna** (00:35:07):
So that's why social works very well for us, because it's not really a written value proposition. Nobody knows what vibe coding is. But you watch 10 seconds of it and you go, "Whew, that's new. Let me go give it a try."

**Lenny Rachitsky** (00:35:18):
Who would've thought that a head of growth, who is traditionally seen as data, metrics, spreadsheets, drive KPIs is like, "Okay, how do we make this more lovable? How do we add more moments of delight?"

**Elena Verna** (00:35:31):
I know. My joke is at the end of my Lovable journey whenever... Hopefully it never comes to an end. But at the end I'll be a growth brand person. Hi, my name is Elena, I do brand now. But I actually see it as part of growth strategy to make sure that that brand shines through every single interaction. And I always talk to my team about it, because that is one big lever in our growth story.

**Lenny Rachitsky** (00:35:58):
Yeah, so I think that's a really important point to highlight. The reason Lovable is growing so fast is it is a product people love. You've made something people want. And the word of mouth spreads, because it's something that blows people's socks off as you said. So it feels like that's the first thing you got to get right.

**Elena Verna** (00:36:13):
Yes. Well, the first thing you have to get right is you have to be at the right place at the right time and you have to be in fast moving waters. Let's not discount how fast this category is exploding on its own. So this cannot happen in every single category that you're starting to build a product. But the way to stand out in the super crowded category is to create experiences that speak to people.

**Elena Verna** (00:36:33):
That I think is something that a lot of people deprioritize, because they still prioritize functionality over humanity within software. And I think that we're actually moving to the new era of software that needs to feel human, that people want to interact with, not just utility of it. Because cost of software is coming down so much to develop that we now can actually invest into emotional feel of that software, as opposed to only just focus on creating the utility out of it.

**Elena Verna** (00:37:04):
So to me, it's a... I love this move, because I hate nothing more than going to software that is just so painful to use that I lose some brain cells as I'm interacting with it, versus software that I feel I get energy out of. And for Lovable for me, I cannot wait on some of the projects that I have to go and vibe code myself. That's the highlight of my day. I bring in my daughter and I'm like, "Let's go do this." Like, "What do you think that needs to be done?" Because I just get so much energy out of doing it. And that is the feeling you cannot create by looking at it as a utility problem.

**Lenny Rachitsky** (00:37:42):
The way I think about it, the way what you're describing is it's almost table stakes have increased, and now it's so easy to build. Now the big differentiator is experience, design, delight.

**Elena Verna** (00:37:52):
Exactly, and it has to translate through every single interaction. So your designer has to be one of your first hires now in startups. It's not just about the engineering, so to speak, utility. And you have to think through every single interaction of, does this communicate our brand or not?

**Lenny Rachitsky** (00:38:09):
So along those lines, I want to come back to something you talked about, which is launching new features as a huge growth lever. The big question there is just how do you maintain quality and cohesiveness as all these people are empowered to ship stuff? Is there anything else there you've seen that works well to help avoid just the Frankenstein product that's endless features that you want to tweet about?

**Elena Verna** (00:38:28):
Yeah. One part of it is not something that you can codify, but it's the type of people that you hire that are going to go and ship these things. We at Lovable try to hire the absolute best talent available out there that we can bring in, and that we can source and that we can attract to grow with. And what do I mean by that best talent? It's not that somebody who has been at really large companies, or somebody that has really done a lot of logos or has big success stories behind them. It's somebody who is extremely passionate about their job.

**Elena Verna** (00:38:59):
It's their hobby. They love to work. They have fire in their belly. This is not a paycheck for them, they want to do this for some ulterior reason. This is the biggest opportunity of their life, so this is global maximum against any other opportunities that are in front of them at the moment. So that's very important for us.

**Elena Verna** (00:39:18):
We want people to come and do their absolute best work at Lovable. It's very important, and you can feel it in this office. People are wired up. They are so high on how can we make this better? How can we deliver more to our customers? And that's very different compared to usually how companies grow where like, "Okay, yeah, the check, check, check. They fit the skillset, let's bring them in." But is that passion, is that fire behind it?

**Elena Verna** (00:39:45):
And then the second piece is that we work really hard on just addressing what's the success here looks like? What is it that we're building? What use cases are we building for? And then because we hire these people that are so passionate about it, the other two skills, by the way, that are super important is high agency and high autonomy. I can figure out things that are tangential to me that I don't need other specialties, so to speak. I don't need a marketer to go launch something, I can go figure it out. And I have high agency, I can go do it myself. I'm going to own it all the way from start to finish. Those are very important, something that we screen for and something that we look for in our culture.

**Elena Verna** (00:40:25):
And then you just see... What you want to do is up to you. So there's very little supervision that happens on the ground. Now, we all have goals and some of the big launches that we're all marching towards, but some of the work that is completely up to developers, up to marketers or whatever, what is it that they want to do? So there has to be that enablement of go try things. And because of our velocity, if you fail, it's not a big deal. We'll just pivot, we will get through it. We are not here to just win all the time.

**Lenny Rachitsky** (00:41:00):
On the hiring of these incredible people, as we all know, it's very hard to hire people these days, especially the best. What have you seen Lovable does differently or does well that helps them recruit the best?

**Elena Verna** (00:41:10):
Yes, and especially recruit in Stockholm. The main office here is in Stockholm. We're asking a lot of people to relocate, which is no small feat. Now some of it makes it easy, because of how much hype we created around our product. People want to come work for us. They're reaching out to us. They're saying, "I love what you're doing, I want to join it." So that we have a cheat code to it, because we have... Most of the time when we reach out to somebody, they say, "Yeah, I would love to explore." So building that product that is highly lovable also creates a really great recruiting brand for you as well. So make sure that there's multiple benefits to that.

**Elena Verna** (00:41:49):
But second of all, we do a lot of trials for people. So trial work to see them in action.

**Lenny Rachitsky** (00:41:56):
Yeah, a work trial.

**Elena Verna** (00:41:57):
A work trial to see them in action for a couple of days. We pay them as part of the work trial. We have some probation periods that we start people on, because this company is not for everybody. As I said in the podcast in the beginning, the pace here is insane. I went on vacation for the first time. So I've been here for six months. I went on vacation for 10 days. I came back, I felt like I needed to onboard from the beginning. Everything changed.

**Elena Verna** (00:42:26):
And when I'm in it, I feel like it's an evolution. But the fact that just being gone for 10 days, it feels like a complete revolution in the company. That pace is just not for everybody, and that's okay. Because I'm a very firm believer that there's different cultures and different environments that the best fit for different personalities and different people.

**Elena Verna** (00:42:44):
So we try to be very upfront with how things are and how chaotic they are, and we prioritize people that don't look for clarity. We can create clarity out of chaos because it is absolutely chaotic otherwise. And if we start to look for people that can explain it to us, that's the only way that we can succeed.

**Lenny Rachitsky** (00:43:06):
The way you described going on vacation and it feeling very different, it feels like when you don't see your kid for a few days and they're just completely different. You're like, "How did you grow up so fast in three days?"

**Elena Verna** (00:43:16):
Yeah, exactly. Exactly.

**Lenny Rachitsky** (00:43:17):
Let me try to summarize the growth levers that you're finding are working, and I'm trying to think about this from the perspective of an AI startup trying to think about, "Hey, shoot, how do we grow faster? What has Lovable figured out?" So it feels like number one is just build something lovable, something that blows people's socks off, but also in a market that is growing that people want to pay money for. You can build something lovable that nobody actually cares about, that there isn't much money going to this space. There's no tide pushing it forward, and it won't work.

**Elena Verna** (00:43:46):
I call it minimum lovable product. It shouldn't be minimal viable product anymore. Viability is left back in 2010s. Now it's minimal lovable product. That's the only thing that matters.

**Lenny Rachitsky** (00:43:59):
I love how these AI tools are letting us... PMs have always had these smoke door test, or what's the term? Or it's not a real product. Paint the door.

**Elena Verna** (00:44:11):
Paint the door. Paint the door.

**Lenny Rachitsky** (00:44:11):
Paint the door, there it is.

**Elena Verna** (00:44:11):
Yeah.

**Lenny Rachitsky** (00:44:12):
Yeah. And it's like, okay, we just have a landing page. There's nothing there. And now AI makes it easier to do that, and it's more full-featured.

**Elena Verna** (00:44:18):
Yeah. Well, it's the feedback cycle. It's just completely collapsed. You can go from idea to some product that is functioning to user feedback within a day if you want to, depending on how fast that you want to run or how complex the product is. For missions, it took us a couple of weeks to vibe code it to the point to where... I have a full-time vibe coder on my team. He's amazing. So he wanted to create videos. He did a bunch of designs for it too. It took him a couple of weeks, we're testing it now and then we'll push it in the product. But it's a completely different development lifecycle.

**Elena Verna** (00:44:54):
Before, it would just take so many more steps from user research to the design sprints, to prioritizing an engineering roadmap, to build something minimal and viable to actually test to little long testing cycles. Now it's just like, "Boom, let's go." It could have taken us a day, we just decided to take a couple of weeks to get all of the video pieces correct.

**Lenny Rachitsky** (00:45:19):
I saw you launch this on LinkedIn. To me, it looked like a full product launch. It is interesting to hear this is just a prototype.

**Elena Verna** (00:45:26):
Yeah, it's minimum lovable product.

**Lenny Rachitsky** (00:45:30):
Minimum lovable product. Okay, I got to ask you, you said you had a full-time vibe coder. What the heck is this? Is this an engineer? Is this something else? What is a full-time vibe coder?

**Elena Verna** (00:45:38):
Great question. This is a new job role that is actually popping up here and there. It's absolutely fascinating to watch this development, because I see vibe coding as a skill being added to a lot of job descriptions. For designers, for product managers, for marketers, which I think is a really interesting shift. Finally, Excel can move over. We have a new skill to add that is super empowering and not-

**Elena Verna** (00:46:00):
... it's super empowering and I'm not 30 years old, but vibe coder, so his name is Lazar and he actually was chief of staff in his previous role so he's not technical at all. He's self-taught in technical aspects of it, but he was very early on in the vibe coding wave so he learned a lot about it. He was user of all of the vibe coding tools, Lovable included.

**Elena Verna** (00:46:26):
And when I was coming into the role, I'm like, "I have so many projects that I will vibe code myself," so I run this woman only hackathon, SheBuilds. I vibe coded the first version of that site and submission process for applications, and then other people came in and started building on top of it. But I vibe coded, but then I don't have enough time sometimes because I need to run around and I want to push out so many different initiatives that I want to test in the market with our own product. So we connected on social and I'm like, "Would you join us?" And he joined us first part-time. I'm like, "You're bringing so much value."

**Elena Verna** (00:47:03):
For example, we partnered with Shopify, and he created a bunch of Shopify Lovable templates, vibe coded for us. And it's been so helpful to have somebody like that that is just pushing all of these things out. And he's an absolute expert, so he's teaching us all too of what is possible with Lovable because he's on the cutting edge of constantly pushing it to the limit. I really enjoy having that role, which I've never had before in my life and in my team.

**Lenny Rachitsky** (00:47:31):
I'm not surprised. I've never heard of this role before as a real full-time job. Do you think this is a thing people will start hiring for at non-vibe coding companies?

**Elena Verna** (00:47:41):
I vibe code myself so I would put that as even as a skill on my resume now. It took me a while to figure out, by the way, everybody's like, "Oh, you just go in and it all happens automatically."

**Elena Verna** (00:47:52):
It takes you a couple of iterations, couple of projects until you know, okay, this is how I need to translate it, how I need to think about it. But for me, it's when I started scaling of what I want to vibe code, that's where his value really came in because I'm like, "Okay, I understand what is possible. I know what needs to be achieved." And some of these apps, I want to be almost full-blown built because they're not going to get incorporated into the product anytime soon. They don't need to be. I'll just link to them from our header, so to speak. And he really accelerated that velocity for me.

**Elena Verna** (00:48:23):
So once you get into vibe coding and you see its value within your organization, leaning into somebody like that, just accelerates your velocity because it is like an engineer on your team. It's just they're not, to me, he's part technical, but they can be non-technical if they're really good.

**Lenny Rachitsky** (00:48:40):
That is fascinating.

**Elena Verna** (00:50:38):
Community. I think community is really important here because you need to bring people together as they're exploring these capabilities and as they're seeing what's possible so they can bounce off each other and they can help each other out. I would say community also amplifies that word of mouth. It amplifies all of the social posting. It amplifies retention mechanisms for you as well. The community has been a huge part of Lovable's success as well, and that's something that was started very early on.

**Elena Verna** (00:51:05):
It runs on Discord, so it's nothing fancy. It's not like we build anything completely from scratch for ourselves. And it has hundreds of thousands of members and it's very lively. We have community managers that are making sure that all of the questions get answered and the right groups are being created. We have incredible ambassador program now as well of people doing it. I would say community here, again, of really making software more human is very important role. Now, obviously not everybody can build a community, but maybe at least plugging in into somebody's community is quite important as well. And then there's another one, unless you have a question on community.

**Lenny Rachitsky** (00:51:46):
No, keep going.

**Elena Verna** (00:51:48):
Another one is giving your product away a lot. And for AI products, it may feel counterintuitive because they're so costly. Every single interaction with an AI product costs companies something. There's an LLM pass-through cost that is coming through. And a lot of, especially traditional tech companies I see are gating AI immediately behind the paywall because they're sitting on a really cush, high margin profile, and the moment that you start giving AI away for free, you're cutting into those margins like a knife through the butter. Now, at the same time, AI being so new and the capabilities being so new, you have to remove the barrier of entry. You have to give a lot of your product away for free. But by the way, I don't just mean freemium. Freemium to me is just a baseline. If you're in the new category, you need to let people explore what it is free and get that initial wow moment.

**Elena Verna** (00:52:48):
It's not aha moment, by the way. It doesn't need to be aha moment anymore. It just needs to be a wow moment. And for Lovable, it's that first preview generation after your first prompt, even though it's absolutely not going to be complete thing of what you want to build, but you just go, "This is possible? I had no idea. I want to keep building." And it becomes an addictive exercise.

**Elena Verna** (00:53:08):
But we also give so many of our Lovable credits away to every event, to every hackathon. If you want to host a Lovable hackathon, we will sponsor it and give all of the participants credits away for free. We give them away as candy and we basically track them over our LLM costs on freemium and giveaways as our marketing costs, and it doesn't go into our something we need to reduce to make our margins better. It goes into, this is something that we need to spend more in because this is part of our growth secret sauce.

**Lenny Rachitsky** (00:53:41):
Okay, I want to hear more about the growth secret sauce. That is extremely interesting. I haven't heard of that as a strategy, and I can see why this makes sense. If the strategy is blow people's socks off so they can tell their friends, post on all socials, the trick is get more people to try it. And it's such a new, crazy thing. Why would I pay money? Why would I even go take the effort to try sign up for an account? I don't know what this is. I don't know what I'm doing with it. So I could see how this loop goes faster and faster by giving it away.

**Elena Verna** (00:54:10):
Exactly. And again, this is very uncomfortable sometimes for companies that A, either used to really... AI companies have lower profile of margins. That's absolutely true. To find an AI company with 80%, 90% margin profile is absolutely impossible, let's be real. We're all sitting somewhere in a 40% or so, which is a lot smaller. So any time that you look at those AI costs as your cost center, that's when you're in trouble. You fundamentally have to flip the script and say, "I need to expose to people of what is possible and I need to remove the monetization friction out of it." Because if you don't, nobody's ever going to try it, or you're going to be very easily overtaken by a competitor that will give it away. And let's face it, once you hook people, they're more likely going to stay with you. So you obviously have to still work on the retention strategy there.

**Elena Verna** (00:54:59):
But if you can have, like for our case, if somebody, one of our users stands up and say, "Hey, I'm going to have a hackathon at my work on Lovable. Can you give us all some free credits to play with?" Why would we prevent a person who wants to do all of the marketing and activating job for us in their company from using us?

**Elena Verna** (00:55:21):
Of course, we're like, "Take it. How much do you need? How much would you like? We will sponsor it all. We will give you anything that you need." So we're really leaning into people that are wanting to show this magic to those around you and empowering them as much as possible. And that is something that is actually applies to every single product. And I agree, this is not a growth strategy that I've ever applied in my life on giving product away as much as possible, but it is something that is more and more becoming something that I see that is absolutely non-negotiable.

**Lenny Rachitsky** (00:55:51):
What I'm feeling is the more mind-blowing it is, the more you should give it away for free.

**Elena Verna** (00:55:56):
Yeah.

**Lenny Rachitsky** (00:55:57):
Especially in a competitive market where everyone is... It's hard. There's so many companies trying to do this thing, and so it's almost like the better you are, the more you should give it away.

**Elena Verna** (00:56:07):
Right.

**Lenny Rachitsky** (00:56:08):
And this also explains why so much VC money has to be raised for these sorts of companies because this is not cheap. Like you said, you're paying all these foundational models a lot of money.

**Elena Verna** (00:56:16):
Yes and no. I'm only going to say no is because, so take a look at Lovable, we're over 200 million in ARR. At this point, we're 100 people large so our headcount costs are very low.

**Lenny Rachitsky** (00:56:28):
Wait, let me just make sure people hear that. 200 million ARR, I didn't realize 100 people work at Lovable.

**Elena Verna** (00:56:36):
Yes. And six months ago, we had 30 people working at Lovable so we triple. So for us, it's a really big deal. We tripled our company size.

**Lenny Rachitsky** (00:56:44):
Such a big company now.

**Elena Verna** (00:56:44):
We're going to quadruple it by the end of, yeah, I know. We're big boy and girls now. But for perspective of the headcount costs, it's minimal. We have very little in that going on. We are not spending a lot on paid marketing so we're not a big paid marketing driver. Yeah, we're spending on influencer marketing, but it's not majority of our growth. It's low double digits to be fair because it's not why we're successful. It's amplifying our success and it's helping us reach new audiences. We don't have really large sales team, we have only a couple sales folks, and they're just starting to ramp up their enterprise efforts so we don't have really big enterprise demand gen costs as well.

**Elena Verna** (00:57:24):
From that perspective, if you look at the equation and you say, "Well, okay, if you're not going to do a lot of paid marketing, if you're not going to do a lot of sales, because we're really only working on hand raisers of people that are saying right now that they want to buy Lovable, then whereas you don't have big costs, so you can spend it on product."

**Elena Verna** (00:57:42):
That is the beautiful part because when we are giving our product away to our customers, we're not competing with other companies in that space, because they're just going to use Lovable in their hackathon. We're on their own, and we're not competing on AdWords or in paid Google where everybody's buying real estate for eyeballs. From that perspective, I think about it more as a shift of where we spend in costs. And honestly, it's more efficient way to do paid marketing almost in a sense because of the cost per eyeball that we get there is quite a bit lower compared to if we were trying to compete it on Google. So yes and no to your statement because it actually does not deteriorate margin profile. We're just shifting of where we're spending it.

**Lenny Rachitsky** (00:58:30):
That is an incredibly important point you're making there. It's not like you're generating an incredible amount of revenue, so there is money available to spend. And what you're saying is because it's been spreading through word of mouth mostly, you're not spending tons of money on salespeople, you're not spending tons of money on paid ads. This is just an amazing way to get more people to use it, so it's like a marketing cost.

**Elena Verna** (00:58:54):
This is product-led growth.

**Lenny Rachitsky** (00:58:58):
Supercharge.

**Elena Verna** (00:58:59):
To the max, supercharged. Yes, because you're literally using your product to drive that awareness by giving it away to the agents in your ecosystem that will do that distribution for you.

**Lenny Rachitsky** (00:59:11):
So fascinating. What a wild world we're living in. Free stuff for everyone.

**Elena Verna** (00:59:17):
Yes. I mean, it's great for consumers. This is a great time to be a consumer. You have so many options. Everybody's throwing themselves at you, giving your product away for free so it's great to be in the market right now. I think the power should be with consumer always, but with software, power has not been with consumer previously because we were forced to use towards some solutions because of either how they were chosen for us or what was available in the market. And now that supply is almost infinite, the demand from the consumers can be very picky and the one that serves the best will win.

**Lenny Rachitsky** (00:59:52):
And I think again, it's important to highlight. This is not some kind of VC subsidized bubble-ish sort of thing. There is a lot of money being generated that you are spending to help it grow faster. It's not some kind of, we're just raising more money to give away more money. You're actually making your own money. It's not driven by VC money. Obviously, it helps.

**Elena Verna** (01:00:09):
I can't comment on specific margin details for us, but at the same time, the money that we're raising on VC is for future development and hardening our business, not because we will not be able to survive without it.

**Lenny Rachitsky** (01:00:22):
Awesome. Okay, great segue too. I want to talk about product-market fit in competition. You have this really interesting post that I don't think people grasp yet, which is that product-market fit is no longer this, we've done it. Product-market fit and we're up into the right. Now we just grow, go, grow. Now we hire salespeople, it's going to be great. You've written that just product-market fit is no longer this like you've done it and you're good. It's this endless fight to keep it. Talk about what you're seeing there.

**Elena Verna** (01:00:49):
I'll first start with what I've felt at least before when people were talking about product-market fit, that yeah, obviously always product-market fit is an evolving thing, but the rate of that evolution was measured in years. What is it that you need the next product-market fit step function change, which often was called second horizon or third horizon. Sometimes five, 10 years, sometimes even longer that you'd need, depending on how good and hard your initial product-market fit was, but you'd spend years scaling the original product-market fit. It was like blitz growth stage. Marketing, sales, growth was very important that you just try to get it to as many people as possible. And then once you have saturation or the cost to getting to the marginal people becomes too high, you start thinking, "Okay, what else can I offer to help me reach additional people or sell more to existing users that I already have?"

**Elena Verna** (01:01:39):
And again, the main point here is it would take years to get to that stage where it became a question that you had to face really hard face-to-face. Now, it's three months, and all of a sudden you have to face that question again. And it's happening because of two things, in my opinion. Number one, in AI technology of what LLM is capable of doing changes still very rapidly with new model release, with each new model release. I think we'll stabilize at some point and then it's going to become more marginal, but we're not there yet. So every three months or so, every single AI LLM provider creates a step function change in what is possible with that LLM. And when you have this new possibility in just an underlying technology that opens up in front of you, then it creates another ceiling of what is possible to build on top of it.

**Elena Verna** (01:02:36):
The tricky piece here is that it's not enough to just wait for that technology to get better and then start building on top. You have to build beforehand to make a bet and then it's the LLM to catch up because when that model releases, you already need to have that functionality available. That piece is, I've never been in a company where the fundamental capabilities are still changing so rapidly, and that's the product part. The product can leap to the new expectations, but let's not talk about the market part as well. Consumer expectations have never changed this fast before. What we expected ChatGPT to be able to do and answer and how we wanted it to talk to us eight months ago versus now is night and day and the deep thinking mode, and how deeply you can go into answering questions and what is capable of building on top of it.

**Elena Verna** (01:03:31):
Consumer perception has never changed this fast too. It's this unprecedented time of consumers all of a sudden in a month saying, "Oh, it's not doing this yet. I'm bouncing."

**Elena Verna** (01:03:43):
Before, again, consumer perceptions would be years to take. It's actually technology would sometimes be able to already address it, but consumer perception has not been changed yet so it would take a long time. We're in this really weird part where both product and market is shifting so rapidly that every three months, I feel like we have to recapture our product-market fit and not just recapture on the same technology and with same customers. It's both of those pieces of the equation change every three months, and it's terrifying in a way. It's also very confusing in a way because we're $200 million company, and we're not solely focused on marketing and sales because we still have to recapture our product-market fit.

**Elena Verna** (01:04:26):
You know that the team that finds your product-market fit is very different than the team that usually scales your company, yet we have to find the team that is capable of doing both on ongoing basis. Now, I think every AI company is on this product-market fit treadmill. Hopefully that treadmill speed slows down. If not, I think we're going to come up with crazy things of what this LLM and AI will be able to do if it's going to continue at this cusp, but it's a weird place to be in because every three months we have to throttle on our scaling efforts and just reinvent and then scale again. But it's like short blitz of growth, not these long year long commitments.

**Lenny Rachitsky** (01:05:09):
What makes this very real is just this week, apparently OpenAI had this whole code red moment where even though OpenAI by far the leading AI assistant over almost a billion, I think monthly active users, basically synonymous with AI around the world, with Gemini 3 launching, their market share just started to dip really quickly. I think they lost six something percent in a week. And so even OpenAI, ChatGPT, the original, the one that everyone uses constantly is in danger.

**Elena Verna** (01:05:39):
It's like nobody's future is bulletproof yet. And 10 years ago, if you asked me if a $200 million company was at risk in losing product-market fit in the next three months if it's experiencing 10% month-over-month growth, I would've said, "You're crazy." And now that's the reality that we live in. And I don't know, it's fascinating to world in. What a time to be alive.

**Lenny Rachitsky** (01:06:04):
Time to be alive and very stressful, but the prize at the end is massive. That's why this is worth doing, not just monetarily, but just the impact it's going to have on the world, the way we people build and ship.

**Elena Verna** (01:06:17):
Exactly. The ceiling of what is possible has been raised so massively that we haven't even became too closest to even see it, I believe. I think that that's the exciting part of it.

**Lenny Rachitsky** (01:06:29):
The way I've seen you write about this product-market fit challenge is the traditional approach is you have these core users that are using it happy with it, and then you expand to the adjacent users and expand to the next. You're basically just trying to recapture that same core constantly and don't even have time to go adjacent.

**Elena Verna** (01:06:44):
Yeah. Bangaly wrote a really wonderful article. It was many years ago at this point on adjacent user theory in that your product-market fit expansion when you're in no growth stages, the biggest opportunity for you to go after is this what you call the adjacent user, which are just outside of your core user. They have somewhat similar needs, but maybe they're in different geo, maybe they have slightly different use case, slightly different needs. And your biggest way to continue growing a product-market fit without having to go to next horizon is to capture that next group of users. The interesting piece here of how I relate to it, we still have the core users. And by the way, those core users are mostly pioneers right now that are excited by the capabilities. Then there's latent majority that is filled with adjacent users. And the issue right now, which I'm actually quite worried about us as a category is that we're constantly focusing on recapturing the pioneers.

**Elena Verna** (01:07:39):
We don't have time to go after adjacent users, and I'm worried of whether there's going to be a gap in the space where we actually going to alienate the latent majority because we're so hyper focused on just staying top of mind and top capabilities on the pioneers. But I don't know the right answer here, because without the pioneers, you need pioneers for latent majority to follow. But if you take pioneers and you take them too far into capabilities, will latent majority never be able to catch up? Maybe this is a fruitless concern, but it's just something that I think about because at this stage we should be working on adjacent users. I would argue maybe OpenAI definitely started to do that with how many people they have on their platform, but not most of the other AI companies.

**Lenny Rachitsky** (01:08:31):
I completely see what you're thinking there. A brand could just become known as that's just for startups and prototyping and it's not for serious work.

**Elena Verna** (01:08:38):
Yeah, or it's for techies. It's for tech people. It never actually enters the people outside of our little bubble that we live in.

**Lenny Rachitsky** (01:08:50):
We touched on this a little bit of just working in AI, working on AI companies, challenging, stressful, a lot of work. What's your advice for folks that are thinking about, should I join a Lovable? Should I join a Cursor? Should I-

**Lenny Rachitsky** (01:09:00):
... are thinking about, "Should I join a Lovable? Should I join a Cursor? Should I just go work at Google?"

**Lenny Rachitsky** (01:09:05):
Not to throw them under the bus or anything. Although Google, very, very successful in AI now, maybe a less AI-focused company.

**Elena Verna** (01:09:13):
I really believe that there's different... You need to understand what's the environment that is right for you. Just please understand that AI companies are very hectic at the moment. They're very unstable by definition of that product-market fit treadmill, about that distribution of how they're actually distribute to the market, really changing about how product is even being developed in the first place. So if you are very comfortable in being in that messy middle and really comfortable of converting chaos into clarity for you and those around you, then yeah, AI company is a wonderful place for you to really absorb new skill sets right now. Because even before joining Lovable, when I kept seeing AI, I'm like, "My gosh, I'm so tired of seeing AI everywhere. Is it really changing the world? Is it really changing the way people work?"

**Elena Verna** (01:10:02):
And I was at Dropbox before, and yeah, we would use AI here and there and I would use ChatGPT. I've never used AI there the way I use AI at Lovable and the things that I'm capable of accomplishing at Lovable, and I don't know if I ever would've made that leap so fast unless I joined Lovable. If I would've just read or listened about it, it's just different compared to be surrounded by people where it's expectation. It's not like a nice-to-have or something that somebody's asking you to do. This is just how you get things done. And you have to think about everything of, "What can AI do here versus where do I add value versus in a traditional sense of work?" because I start with my own value and then I augment it with AI. And here, the mindset has completely shifted.

**Elena Verna** (01:10:47):
Now, I don't think AI is replacing everybody's job, so please don't look at it as that cliche saying. I actually often call AI as average intelligence that helps me get the platform up. And then I add my human thinking and my human creativity on top of it to get it to the next level. But at least I can get this base level done with AI really freaking quickly.

**Elena Verna** (01:11:11):
So from that perspective, I think if you want to leapfrog on what it means to be AI native employee and how to use all of these AI tools, you should go to AI company. But if you know that your superpower is in more structure and more definition and a really high specialty of things, because in AI companies, they're all fairly small, so you'll have to generalize quite a bit and have a lot of ownership, all of areas that you usually maybe not have ownership over, then you shouldn't join it, because AI companies will evolve to be more stable too.

**Elena Verna** (01:11:43):
So it's just a matter of time on where you can join. So I would just urge people to look at their superpowers and the type of environments that really speak to them so they can feel happy, because this can lead to burnout for wrong type of personalities very quickly.

**Lenny Rachitsky** (01:12:00):
Yeah. My sense is if you want work-life balance, don't join one of these companies, because that's just not the way they work.

**Elena Verna** (01:12:07):
I don't know if I'd go that far. I mean, I have family, I have two kids. I feel like I have a very good work-life balance, but I put in boundaries for myself. I know when I need time off because I know when my brain starts to overheat, so to speak. But I also know that work is my hobby and it's my passion, and this is the best work of my life that I'm doing right now. There's no other place that I'd rather be than to be here.

**Elena Verna** (01:12:33):
So I think that you just need to be more careful about setting your own boundaries that you know you need. But I mean, let's face it, I don't think anybody has work-life balance, regardless of a company that they work at, even at Google or Microsoft or any of the others. I think everybody's freaking out and running as fast as they can, it's just they're running it in different structures.

**Lenny Rachitsky** (01:12:58):
I'm really glad you said that and corrected me there, that it is possible to work at a company, one of, if not the fastest-growing company in history and actually have work-life balance to get sleep, to spend time with your kids and family.

**Elena Verna** (01:13:11):
You just have to protect it ruthlessly, but you also need to be realistic with how much is expected out of you, and you need to feel confident that you'll be able to deliver it. And by the way, you won't be able to deliver it unless you use AI in many aspects of your work life. So that's the piece that helps you actually get to hit those expectations of outcomes that you need to do and the velocity. But I'm very protective of my personal time with kids. Why did I have children if I'm not going to spend time with them? So those are part of the non-negotiables that I bring along with me in every single work.

**Lenny Rachitsky** (01:13:47):
For people that maybe have trouble setting boundaries or just not good at this, anything, what works for you? Is it as easy as just telling people, "Here's where I need to leave"? What advice do you have for people to set boundaries like that?

**Elena Verna** (01:13:59):
So first of all, I would not think about it as a work-life balance. There's no such thing as balance. So balance feels like, "Oh, I have enough time for everything."

**Elena Verna** (01:14:05):
I don't have enough time for anything, but I prioritize my family in some moments, I prioritize work in other moments, and I don't try to balance the two. I go where I'm needed and where I go, and I feel like I'm not going to regret the choices that I'm making today. So I'm constantly trying to put myself in the future and say, "Will I resent myself if I make this choice right now?" And if the answer is yes, I don't make that choice.

**Elena Verna** (01:14:29):
And sometimes I have to say no to Anton and say, "I can't make it," or, "I won't be there. I need to be here with my family," or, "Today I need to cancel my day. My kid is sick and he needs me, and I need to take him to the doctor."

**Elena Verna** (01:14:43):
So I think that just making in the moment, in every day, when sometimes in-the-hour decisions, to me, works better than trying to balance something that is completely unachievable and it feels overwhelming to even think about. But I prioritize this in my sleep, my health, my workout schedule, my kids, my family, my husband, and just my downtime because I know that I'm most creative once I have separation from work, because then I come in with all cylinders firing and I have so many more ideas about it. So to me, it's actually part of doing my best work is to take time off.

**Lenny Rachitsky** (01:15:18):
That is really great advice. I want to touch on what it's like to work at Lovable because it feels like Lovable is at the cutting edge of what working in product is going to be. So you mentioned a little bit about how you're always talking to AI, asking questions. Is there any other kind of anecdotes of just how people operate at Lovable that is really unique or weird or funny or interesting that might be helpful for people to try in their company?

**Elena Verna** (01:15:43):
Yeah. I mean, we use Lovable at Lovable a lot. All of our internal tools are built on Lovables. We actually have our first hackathon on Lovable happening next week, where our entire company is just going to take full day to vibe code and see what we actually have happen. We prototype everything on Lovable. So our specs, yeah, we do still have a written spec, but it always accompanied by a lovable prototype that everybody can interact with and to click around with and provide feedback. And everybody bunches in and also does some edits if they have any better ideas.

**Elena Verna** (01:16:16):
So I create mocks on Lovable. So for example, we need to make some pricing changes, or pricing page changes. I take a screenshot of our pricing page. I go to Lovable as I recreate this pricing page, make these changes, and then I send that to my engineering team saying, "Hey, this is what I want to happen." And then they take it from there.

**Elena Verna** (01:16:37):
And ChatGPT, I use a lot for brainstorming, especially the deep thinking mode. I love it. It takes a long time, but it's so worth it. Sometimes it has crazy ideas. Sometimes it was like, "Yeah, this is nothing new to me." So it's not interesting, but it gets me thinking and it gives me a look at the different angles.

**Elena Verna** (01:16:56):
And I use Granola a lot, for example, because to me, it's super helpful to get AI summaries of the meetings and it's very powerful for me. I use Wispr Flow a lot because I feel like I have no time to type anymore. So I just talk to my phone and talk to my laptop all the time in order to do it. But we're even thinking about all of the customer support automations that are done through AI. Every single aspect of what we do, question is asked, what can AI do here first, and then how we can add ourselves into the equation.

**Elena Verna** (01:17:33):
But Lovable for us, having unlimited credits at Lovable is a pretty awesome perk, I have to say. I sometimes have to pinch myself. I'm like, "I get paid to vibe code." It looks so fun.

**Lenny Rachitsky** (01:17:46):
Feel like that engineer, that vibe code engineer, he's actually getting paid to vibe code.

**Elena Verna** (01:17:48):
He has my dream job. I want his job.

**Lenny Rachitsky** (01:17:50):
Yeah, exactly. Exactly.

**Elena Verna** (01:17:51):
Yeah, I got into the wrong line profession here.

**Lenny Rachitsky** (01:17:55):
Oh, man. Okay. Is there anything else about Lovable? Because what I think about, actually, I interviewed the Perplexity founders back in the day years ago. Before we talked to anyone for advice, we first asked ChatGPT, and I was just like, "That is the most insane thing I've ever heard. How can you possibly work that way?"

**Lenny Rachitsky** (01:18:11):
And now that's how everyone works. And so I'm curious, I don't know how Anton works. Is there anyone else that's just way in the future of here's how things might be?

**Elena Verna** (01:18:19):
So for me, especially for product and growth, and even marketing in some capacity, when I have an idea in my head, it sounds so freaking cool. And sometimes I put it on paper and it's like, "Ah, we need to do it."

**Elena Verna** (01:18:34):
And then I go and try to vibe code it, and I'm like, "Oh, I don't see the magic anymore," or I can't envision it anymore.

**Elena Verna** (01:18:43):
Or sometimes I'm like, "Yeah. Yeah," and there's more, there's more. So to me, it actually has helped really complete the ideation process for me quite a bit, because then I actually try to go and build it, and it breaks down some of the elements of what's important, what's not. So it's taking me on a product development lifecycle so much further down. And then it creates a much better communication vehicle with my engineers too, because I then can tell them exactly what's important and whatnot.

**Elena Verna** (01:19:12):
So to me, it's been great because sometimes we envision things that are so much better than the reality. And before, until it hands it off to design, designers would do it for us and try to make it awesome versus I often stop my ideas in tracks super early on without pushing it forward. Versus other times I might've pushed it for too far too long, even through design queue or even pitching to leadership. And I find that very powerful because it calibrates me really quickly.

**Lenny Rachitsky** (01:19:44):
Awesome. Okay. Last question. I want to talk about something that you've written about that I think it's a really important topic, something that we should surface, is you wrote this post called I'm Worried About Women In Tech. Talk about what you're seeing here, what you're noticing, what you think might be going in the wrong direction.

**Elena Verna** (01:20:00):
Yeah. There's actually conflicting data points about how women... You're talking about women, right?

**Lenny Rachitsky** (01:20:00):
Yeah, women in tech.

**Elena Verna** (01:20:06):
Women in tech? Yeah, yeah.

**Lenny Rachitsky** (01:20:06):
Yeah.

**Elena Verna** (01:20:07):
There's conflicting data points about how women are keeping up with AI technology and wave, because there's a bunch of reports that has been done that show massive gap between women adopting AI versus men adopting AI, which points the story that men are just widening the gap of accessibility for technology. And whoever's adopting AI right now is getting paid the most, gets the most opportunities. I mean, we're seeing insane acqui-hires right now, where people are getting paid more for their talent than for the companies that they've created. And that's a really interesting trend that is occurring, and a lot of it is fueled on this wave of AI.

**Elena Verna** (01:20:47):
And women are not really present there. If you can think about one million-dollar acqui-hire that has been in the news that is a woman, I can't think of one. If you look at AI companies and their CEOs, most of it is men. If you look at the company's composition in AI companies, it's mostly men. To me, this really came to the head of when I came to Lovable, and I'm like, "It's pink, it's purple brand. It's a heart. It's lovable." I'm like, "I'm sure this is where it's 50/50 men versus a woman."

**Elena Verna** (01:21:20):
And although we don't collect this information, but just through third-party autofill, we saw it's like 20% at most. And I'm like, "What is happening? Not again. Why is this, again, not being adopted by a woman?"

**Elena Verna** (01:21:35):
And obviously I don't know all of the answers. I think that this is early on that we can shortcut it. And by the way, I also don't want to put this as a indication that men are to blame because I think men are doing wonderful job really spearheading the horizons and showing us what's possible and leading the charge. I'm just afraid that so many women are stuck in that latent majority that is just not catching up. And my worry is that it's going to affect the hireable talent. It's going to step us back again in the composition of the workplace, of the diversity. And maybe it matters, maybe it doesn't, like whichever side that you sit on. But I think that it needs to be built for everybody in the world, and for that, it needs to be built by a representative sample of people that are behind the product as well.

**Elena Verna** (01:22:23):
So I just find it fascinating that even when the barrier to building has been lowered versus you don't need computer science degree, which I appreciate there's not that many women that are getting. We're still seeing the gaping gap on the adoption between genders, which, I don't know, there's something very frustrating about that.

**Lenny Rachitsky** (01:22:50):
Yeah. The thing that struck with me from your post is there has been a lot of progress being made in the last decade, and now AI is just kind of turning it all back, turning it all around.

**Elena Verna** (01:23:00):
Hopefully not. I think that we're early on enough that we can bridge the gap. I think sometimes women just need space and ability to discover it, and that's what we're doing at Lovable. We have this initiative, SheBuilds, where we create a hackathon for women only and we give them unlimited access to Lovable for 48 hours, and they come together as a community and they build together. And there's beautiful things that start to come out of it, which I've never anticipated before. But so many women in that hackathon for us build help with their elderly parents or with their kids or with the household or for their church group or for the kids' basketball team solutions, which have hyper-local, hyper-relevant, very needed for what they need in their life and something that was never been able to build before because of how expensive software was, because it would never going to become potentially a hundred-million-dollar companies, but it also doesn't need to be anymore.

**Elena Verna** (01:24:01):
So I just want to bring women to build more and vibe code more so we can have more diversity in software that is even created because I think that we all have a unique take on what problems that we can solve, and I want everybody's voices to be heard.

**Lenny Rachitsky** (01:24:17):
I'll give the URL for SheBuilds. I pulled it up while you're talking, shebuilds.lovable.app.

**Elena Verna** (01:24:23):
It's fully vibe coded on Lovable.

**Lenny Rachitsky** (01:24:23):
That's so cool.

**Elena Verna** (01:24:25):
Is there a minimum viable product?

**Lenny Rachitsky** (01:24:27):
Minimal Lovable product.

**Elena Verna** (01:24:28):
Minimal Lovable product.

**Lenny Rachitsky** (01:24:29):
There it is. So when is this happening? Is this December 15th, 18th? Oh, yeah, it's coming up.

**Elena Verna** (01:24:33):
Yeah, we're running it constantly.

**Lenny Rachitsky** (01:24:36):
Cool.

**Elena Verna** (01:24:36):
So our next cohort starts December 15th, but we're going to have more.

**Lenny Rachitsky** (01:24:36):
Now or never. Sweet.

**Elena Verna** (01:24:41):
We're planning a massive one on International Women's Day. So that's the one that if you can, come join us.

**Lenny Rachitsky** (01:24:48):
Awesome. Okay, so some glimmer of hope. I don't know if you saw this tweet where I tagged you the other day, or maybe it was today. I was looking at my most recent podcast video performance, and the top four are all women and they're all AI-oriented. And they're above Stewart Butterfield, founder of Slack, above Gamma's CEO, Grant, so maybe a glimmer of hope.

**Elena Verna** (01:25:10):
Yeah, absolutely. I think there's lots of glimmers of hope. I think we can just all lean in and make sure that nobody's left behind in this wave. And that's not to stop people that are marching ahead. This is just to open up opportunities for everybody around us.

**Lenny Rachitsky** (01:25:25):
Awesome. And we'll link to that post if people want to get a deeper perspective, what you're saying. Elena, is there anything else that you wanted to share? Is there anything else you want to remind people of before we get to your one-question lightning round?

**Elena Verna** (01:25:38):
I guess the only other thing that I will share is for AI companies when it comes to hiring. It's really interesting, also, kind of the shift in the type of personas that end up being hired that I see. For me, at least it's quite different compared to anywhere that I worked before. And that is, there's this narrative going in the market always that new hires, sorry, new grads have no jobs in the market left because all of the entry-level jobs are automated.

**Elena Verna** (01:26:07):
I actually think that's quite false, because new grads, especially AI-native new grads... So it's very important for kids that are entering into the, I shouldn't say kids, young adults that are entering into the workforce that they really know AI, which there's another really big issue that our schools are not teaching AI students. So this is something else that we need to fix as a category, because otherwise we're literally setting up our young for a complete failure. But I think it's incredible to see some of those new graduates come in and what they're capable of doing.

**Elena Verna** (01:26:43):
We have multiple new graduates at Lovable that are working, and I learned so much from them. And you need to obviously have the right atmosphere where people with experience, like a old guard like me, that can look at the new guard and really hear them and see them and really change the way that I operate based on how they do things. So make sure that you bring some of that fresh talent that doesn't understand any of the baggage that we came from and that can really look at the future in technology and what can unlock from a completely new lens. So highly recommend putting those into your team as little fireballs that are going to be sometimes hard to contain but can start the best initiatives for you forward.

**Elena Verna** (01:27:32):
And then it's also interesting that there's a really high demand for ex-founders now, for those people that truly have a lot of agency and high autonomy. So instead of just having people that have been working in the corporate world, the failed startup founders are now hot demand for a lot of these AI companies. So these personas that we traditionally would not prioritize in companies to hire are now becoming the hottest commodity and the highest-going talent, which I think is fascinating and is the wonderful thing that is changing how the culture inside operates.

**Lenny Rachitsky** (01:28:09):
That is really interesting and really empowering, just this idea that if you need grad, there's hope. You're not going to be out of a job.

**Elena Verna** (01:28:09):
Absolutely. Absolutely.

**Lenny Rachitsky** (01:28:16):
And you might have an advantage. Yeah.

**Elena Verna** (01:28:18):
You have to lead with that. That's the thing. You have to lead with the things that you are capable of achieving, knowing what you have with AI, because that is a lot of people. Especially in traditional tech or in more traditional companies, they're looking for somebody to show them because it's really hard to figure it out on your own versus coming in and seeing and then copying.

**Lenny Rachitsky** (01:28:37):
Well, Elena, with that, we've reached our very exciting lightning round. Because it's your fourth time, I'm not going to ask you all the questions I always ask you. So I'm just going to ask you one question. So Lovable is based in Stockholm, Sweden. I'm curious, just what's something you love about Stockholm that you weren't expecting? Is there a food, a restaurant? I don't know, something.

**Elena Verna** (01:28:55):
Well, you always have to say Swedish meatballs. I mean, I've never liked meatballs before, and now it's so good here. It's so good. So every time, one of my meals here throughout the day involves it. But I actually really love their-

**Lenny Rachitsky** (01:29:11):
It sounds really good. I actually haven't yet.

**Elena Verna** (01:29:14):
Yeah. I don't know, they just taste different and they're-

**Lenny Rachitsky** (01:29:16):
And they smell like the Ikea. It's not like the Ikea Swedish meatballs.

**Elena Verna** (01:29:19):
Well, I have been to Ikea here, because that is a Swedish company too. It reminded me of in-person Amazon. It was absolutely incredible. Ikea here is next level. But food, Swedish meatballs for sure. Honestly, how clean the city is, it's kind of incredible. The architecture, everything is so built out. It's picture perfect like it's on a card. I don't know. It's different compared to most of the large cities that I've been at that are little bit more worn down.

**Lenny Rachitsky** (01:29:49):
So fun. Makes me want to visit and get some meatballs.

**Elena Verna** (01:29:54):
Yeah, and visit during the summer. Otherwise, the daylight here-

**Lenny Rachitsky** (01:29:54):
During the summer? Okay.

**Elena Verna** (01:29:56):
... is really tight during the winter.

**Lenny Rachitsky** (01:29:59):
Okay. Good tip. Okay. Two final questions. Where can folks find you online if they want to reach out, maybe learn more? And how can listeners be useful to you?

**Elena Verna** (01:30:06):
Yes. Please find me online on LinkedIn. Please feel free to follow me. Always engage on my post if you want to engage with me because that's the place that I always talk to people. I have my newsletter as well that baby steps compared to what Lenny has, but it's elenaverna.com. That's where I post most of my findings that I experience at my work. So if you want to continue seeing how my thinking evolves or the patterns that I notice, that's where to find me.

**Elena Verna** (01:30:35):
And how to be useful to me? Really pressure test my thinking because so many things are changing right now. I'm honestly not even sure myself of what is a pattern versus what is just a data point. So I'd love to just engage in as many conversations as possible and hear your opinions because that will help us as an industry just understand what is actually happening and makes more sense out of this whole thing.

**Lenny Rachitsky** (01:30:59):
I'm just going to double click on your newsletter. Definitely subscribe to it. It's incredibly good. Everything we've talked about here, Elena has written about in her newsletter in large part and goes even deeper. It's just elenaverna.com. If you're reading this on your podcast app or YouTube, you could just look at her name and just type it .com and you'll find it, and subscribe and you'll be really happy.

**Lenny Rachitsky** (01:31:17):
Elena, thank you so much for being here. This was amazing, everything I wanted it to be. Thank you for sharing. I know you have a lot of work to do, so I appreciate you making time for this and for joining us.

**Elena Verna** (01:31:26):
Thank you for having me. Really appreciate you.

**Lenny Rachitsky** (01:31:29):
Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---

## [17/17] 10 contrarian leadership truths every leader needs to hear | Matt MacInnis (Rippling)

**Matt MacInnis** (00:00:00):
It is really important to me that we feel that we've deliberately understaffed every project at the company. If you overstaff, you get politics, you get people working on things that are further down the priority list than necessary. That is poison. It's wasteful. It slows you down. It creates cruft.

**Lenny Rachitsky** (00:00:15):
You've been a long time COO at Rippling. Recently, you moved into CPO, Chief Product Officer at Rippling. Something you talk a lot about is that extraordinary results require extraordinary efforts.

**Matt MacInnis** (00:00:26):
If you want to be in the 99th percentile in terms of outcomes, it's going to be really difficult. You got to sort of remind people that if they ever find themselves in the comfort zone at work, they are definitely making a mistake. It's supposed to be really fricking exhausting.

**Lenny Rachitsky** (00:00:40):
You're a big fan of escalating issues.

**Matt MacInnis** (00:00:41):
Fundamentally, the most selfish thing you can do is withhold feedback from someone. When you think a thought that would help someone improve and you avoid giving it to them because it would make you uncomfortable. Well, you're optimizing for your own comfort, and it's fundamentally selfish. So many people have teams that are not functioning incredibly well. Teams will always optimize for local comfort over company outcomes. The purest form of ambition and most intense source of energy in the business is the founder CEO. Every next concentric circle of management beyond the founder CEO has the potential to be an order of magnitude drop off in intensity. That is fucking dangerous.

**Matt MacInnis** (00:01:17):
As an executive, as a leader, your job is to preserve that intensity at its highest possible level. You've had a couple really interesting experiences with your own startup. We talk in Silicon Valley about never quit, but that is complete absolute venture capital.

**Lenny Rachitsky** (00:01:33):
Today, my guest is Matt MacInnis, Chief Product Officer and formerly longtime Chief Operating Officer at Rippling. If you don't know much about Rippling, it's a massively successful business last valued at over $16 billion. They have over 5,000 employees, and Matt has been instrumental to that success. He's also got a really rare combination of brutal honesty, a ton of experience building a very complex and very successful business, and being able to clearly articulate what he has learned really well. Matt shared a lot of insights and advice that I've not heard anyone else on this podcast share, and I left this conversation feeling that every leader needs to hear his advice.

**Amar** (00:02:48):
This podcast is sponsored by Google. Hey folks, I'm Amar, product and design lead at Google DeepMind. Have you ever wanted to build an app for yourself, your friends, or finally launch that side project you've been dreaming about? Now you can bring any idea to life, no coding background required with Gemini three in Google AI Studio. It's called vibe coding and we're making it dead simple. Just describe your app and Gemini will wire up the right models for you so you can focus on your creative vision. Head to ai.studio/build to create your first app.

**Lenny Rachitsky** (00:03:18):
**Matt MacInnis** (00:05:04):
This is a term, that phrasing I actually attribute to a friend of mine, Dan Gill, who's the chief product officer at Carvana, which as a company also doesn't get enough credit for how much of a tech company it actually is. Super interesting. And I think as a general framework for me, and a lot of what I say with you today is not really specific to product in any way. We should actually talk about that. It's like the product function is an instantiation of the general concept of management. Being a chief product officer is not that different from being a chief whatever officer. You have to apply the same frameworks and concepts to get people to achieve goals together. But one thing that is absolutely universal that I think we, honestly, I think we forget it in Silicon Valley or a lot of people don't sort of internalize it, is that if you want to accomplish something truly extraordinary, if you want to be in the 99th percentile in terms of outcomes, it's going to be really difficult.

**Matt MacInnis** (00:05:56):
It's going to be really uncomfortable. And you got to sort of remind people of that, that if they ever find themselves in the comfort zone at work, they are definitely making a mistake. They have definitely screwed up somehow. It's not that an extraordinary effort is sufficient to an extraordinary outcome, but it is 100% true that it is necessary. And so I do use that framework as a sort of guiding principle in my own leadership.

**Lenny Rachitsky** (00:06:23):
To make this even more real for people, what are examples of moments that were extraordinarily hard?

**Matt MacInnis** (00:06:29):
It is not about any sort of grand single story. I think the story is actually told through a thousand little things. And so for me, the story is told through a thousand Jira tickets, not through a thousand grand events. The extraordinary effort thing is a reminder that it's supposed to be really fricking exhausting. It's supposed to be. So on Friday night, when you get hit with an escalation on Friday night, when you get sort of hit with a bunch of new bugs from someone in the engineering team that you've got a triage, those are the moments where great players and great teams are separated from good players and good teams. And it's so easy to say this at a company like Rippling because we're winning. As a company, for all of our foibles, and we should spend time today talking about where things are not perfect and not great, but the growth rate of the company on the revenue foundation that we have is extraordinary, really, really compelling.

**Matt MacInnis** (00:07:33):
And it gives you, as a leader, the air cover to get up in front of your team and say, "Hey guys, I need the last ounce of oil that you've got left." And if your company's not growing very quickly, if things aren't that great, if your growth rate is 30% or 40%, it doesn't feel as good as a contributor in that business to lean in and give everything you've got on Friday or Saturday or Sunday because you don't know that it's going to yield much. And so extraordinary results, outcomes demand extraordinary efforts, but if there's no chance at an extraordinary outcome, it's very hard to get the extraordinary effort. And so I like to remind people at Rippling at least that it's so rare to have the opportunity to be able to be a part of a team where the extraordinary effort that you do put in on Friday or whatever, whenever it is actually contributing to an extraordinary result.

**Matt MacInnis** (00:08:27):
It's a very special and rare thing, and it gives me a superpower as a leader because I can lean on that when I'm ringing the oil out of somebody who's in the bored and tired zone.

**Lenny Rachitsky** (00:08:38):
I saw the same thing actually at Airbnb with Brian Chesky. It always felt like things were going great and maybe we could take a break after something we shipped was killing it. And it always felt like the opposite. It always felt like, how do we press the gas pedal further? How do we go faster? How do we go bigger? There's never a moment to take a break.

**Matt MacInnis** (00:08:57):
I spent seven years at Apple and learned under Steve Jobs when he was the CEO, learned what we called the death march, which is what we did to the engineers. It was like as soon as you shipped one version of the iPhone, you were just immediately thrown into the pit of building the next one and there was no break. It was just relentless and talk about an extraordinary outcome at the end of the day. There is no relief. In a competitive market, and if the market is valuable, it's competitive, no question. If you leave anything on the field, if you sort of leave a crack for your competitor, 100% chance they're going to go fill that crack. And so you have to be relentless. There can be no relaxation of the organization. It doesn't mean people can't come and go or people can't take vacations or live their lives, of course.

**Matt MacInnis** (00:09:48):
And it's not like people are human beings. You can't grind the individuals down, but the team as a collective group of people has to be sort of on the ball all the time. There can't be a break. And if you leave one, you're just begging for the slightly more hungry competitor to come in and eat your lunch. And that's the beauty of capitalism.

**Lenny Rachitsky** (00:10:12):
Also, very counterintuitively, and maybe the more optimistic perspective here is when you do give your team space to just twiddle their thumbs, bad things start to happen. Morale actually dips in my experience. People get distracted. They're like, "Oh, what are we even doing? It's not interesting." I find that keeping people busy and motivated and fired up, even though you may think they'll be happier taking a many week break and slowing things down, I find they get more, the more I actually goes down in those moments.

**Matt MacInnis** (00:10:44):
So here's a management framework that I use fairly often. As an executive, you don't know how to get any decision exactly right. It's not knowable. You don't know how much budget to allocate. You don't know how many people to put on a project. You don't know how to set a deadline for when you're going to ship something. But of course, you have to set some default so you make your best guess and then you manage to that best guess and you learn as you go because in software development and in business in general, everything's emergent. These are not things that are knowable top down or a priority. And so you take a best guess and knowing that you're not going to get the right answer, you need to decide whether over-steering or under-steering relative to your perceived midpoint is better. And so let's talk about staffing. When you staff a project, is it better to overstaff or is it better to under-staff knowing that you can't get it right? Well, it's better to under-staff. If you overstaff, you get everything that you just said. You get politics, you get people working, I think most importantly on things that are further down the priority list than necessary. You have like 20 things on a stack rank list and you know that you got to do the top five, but the next 15 data's kind of ambiguous, but you've overstaffed the project. So the next 10 things down are getting worked on. Before you even know if they're necessary, that is poison. It's wasteful, it slows you down, it creates crust. And so it's very clear that under-staffing is less evil than over-staffing. In this particular framework, the advice is under-staff deliberately, always. And then the wisdom, the wisdom element is to know not to under-under-staff and sort of knowing the difference between those two things.

**Matt MacInnis** (00:12:22):
And so that's the way we work at Rippling. Everyone is constantly asking for more resources and of course where we can afford to and where it's appropriate new resources arrive, but it is really important to me that we feel that we've deliberately understaffed every project at the company.

**Lenny Rachitsky** (00:12:39):
There's a previous guest, I forget who this was. They used this metaphor if they want their team to be dehydrated to always be wanting more water. And then eventually they're too dehydrated and okay, we needed someone to help. Interesting. Yeah. There's a line along the lines of extraordinary efforts I want to make sure I read because I think this is really good. This may be a way to summarize what you're saying, that good teams get tired and that's when great teams kick the good team's asses.

**Matt MacInnis** (00:13:04):
Yes. This was a quote actually from Sunil, and he found it from a women's basketball team coach. And it is, to my point earlier about you got to run the engine at the red line at all times because the minute you let your guard down, the minute you slow down, the minute you relax, the minute you leave a crack for your competition, the great teams are going to come in and kick the good team's. And it's like sports, I'm not a very sporty guy, but sports analogies are sort of irresistible because at the end of the day, business is a game and none of this matters. We're not going to carry it to the grave. It's like you're here to do this stuff because it somehow fulfills you while you're on the planet. And I love the sport of business and I find that sports, notwithstanding the fact that I watched very little of it, that military, those are very ripe sources of parallel concepts to apply in leadership.

**Lenny Rachitsky** (00:13:56):
I find also those most intense, stressful, long nights are the moments you remember most and remember most fondly back to when you're building something. The key though is that it has to go well. As you said, if you are succeeding and winning, all of this is romantic in the end and nostalgic. Remember that time we built this thing and worked late nights and shipped this thing? If it doesn't go anywhere, you don't feel that. So I think that's a really important component of this is you need to be winning and succeeding.

**Matt MacInnis** (00:14:23):
One thing that I've learned from Parker, Parker's our CEO at Rippling, he said, "You don't really learn from your mistakes, you learn from your successes." And it's like you do, of course, and he would admit you learn a bit from mistakes, but I do think that this is sort of feel good that it's like, "Well, you didn't succeed, but at least you learned something." I've had failures. When I look back at the nine years I spent working on inkling from day one in 2009 until we sold that business to a private equity firm in 2018, up the curve of Silicon Valley coolness, back down the other side into obscurity. Of course, I learned and grew a ton during that time, but in now what I think is six or seven years, I'm trying to do the math, seven years coming up on at Rippling, I've learned so much more because I've seen success.

**Matt MacInnis** (00:15:13):
I've seen rapid, wild, crazy off the charts success of the business and it's more informative. There's more to glean from seeing how it's done right than there is to glean from seeing how it's done wrong. If I tell you you're going to get on an airplane and one maintenance technician has seen it done right a hundred times and the other maintenance technician has seen it done wrong a hundred times, but he learned from his mistakes, but still hasn't had any success himself. I mean, give me a break. There's not even a comparison which plane you're going to feel more comfortable on. And so I do think that learning from your mistakes thing is a bit of a feel good trope that actually has very little substance in reality. And it's why as an early career product manager, or it's why frankly at any stage of your career when you want to learn, you should join a winning team.

**Matt MacInnis** (00:15:57):
It's cool to go and start a company at 22. Good luck to you. The odds are not in your favor, but the folks who, when I look at a resume and I see that someone's joined, they were at really good companies when those companies were super exciting and in crazy growth mode. I'm like, "I instantly want to interview that candidate because I want to hear what they learned from being part of a winning team." And that's sort of one of my go to heuristics when I'm looking at candidate profiles and I think it's an under-told trope. Sorry, not an under-told trope. It's a piece of advice that I don't think people embrace enough in the valley that success begets success and you should chase success.

**Lenny Rachitsky** (00:16:35):
Speaking of success and learning, you've been a long time COO at Rippling and the reason you're here recently you moved into CPO, chief product officer at Rippling, which is very exciting and very rare. I don't see a lot of COOs moving into product. Let me ask you why did you move into that role? I feel like you've been killing it at COO. Maybe that's the reason. Be careful what you're good at. And also just what are some surprises about this, about moving into product? Because a lot of people imagine what it's like and then you're actually doing it.

**Matt MacInnis** (00:17:08):
The story at Rippling is pretty interesting and I'll tell it because I think it explains why I'm making this transition, but this isn't really about me. I think it's sort of a pattern that your listeners would find useful. In general, your best executives are the ones that you can mostly toss into any challenge and they will bring order to chaos. They will fix the thing. And I do appreciate the terms that people have used at Rippling for me, talking about MacInnis's injured birds, where at any given moment some function is in disarray or in jeopardy and I go and focus very carefully on that function to get it back in order batting 800 maybe, like not always wild success, but I did that everywhere except R&D. I would think about helping out with components of the sales organization like our channel team, or I spent time building out the recruiting function a few times when it needed to be sort of rethought in response to our growth, but it never R&D.

**Matt MacInnis** (00:18:23):
And so I would have my feet up on the table looking out across the floor at this dumpster fire off in the distance, just sort of emitting smoke and wondering if someone was going to go in and deal with that. And the smoke takes various forms and when you're growing as quickly as rippling is growing, it's not always something that necessarily even impacts customers, but it's the sort of thing where you're like, that architecture's not right or they're not measuring adoption correctly." From the outside, I actually had quite a few criticisms that I could lob in. And what happened at Rippling was we made some hiring mistakes. I think the folks that we had in those roles would agree that they weren't the right people. We had a hiring mistake in engineering leadership where the product leader at the time had to sort of run engineering.

**Matt MacInnis** (00:19:07):
We subsequently had a mistake in product hiring and a lot of us had to sort of pitch in. And Parker and I sort of stared at each other through two years of this kind of disarray or this chaos or this agony of things and just never really having good executive leadership over both engineering and product at the same time. And I remember Parker sort of slumped down in his seat and said, "Oh, I have to run another search." And I said, "No, the gig's up. I'm going to go do it." And he really sprung up in his seat. He's like, "Really? You'll go do that?" I'm like, "Dude, this is what the business needs." And so that's what I did. And that really started about a year ago in sort of, I realized I was going to do it and expressed that to Parker in December. I really took it on in January of '25.

**Matt MacInnis** (00:19:51):
And so it's been 11 months of learning. Jumping into the product role when the product function itself, although staffed with really talented people, wildly under understaffed, and without a single spiritual leader on top of it to drive consistency and process excellence had become locally optimized but globally incoherent. And if you know Conway's law, you are destined to ship your org chart. And so with a locally optimized, globally incoherent team, you had a locally optimized, globally incoherent product experience that needed to be resolved. And so my efforts over the last 11 months have been to establish greater clarity in terms of how we do things around here, better process, better general leadership, hiring and firing. I mean, just doing the sort of cleanup on aisle three that needed to be done, even though, again, a lot of the people in the team were quite talented and doing an excellent job of managing their specific domains.

**Matt MacInnis** (00:20:52):
Jumping into the product role has been quite eye-opening. I feel a little bashful about the naivety of my view from the outside a year ago. Product teams have a hierarchy of needs, and we like to point at the failures to meet elements of that hierarchy higher up the triangle and sort of impugn the failure of that organization for not, as an example, measuring adoption metrics very carefully and not closely tracking those metrics as a means by which to drive execution. When I jumped in, I was like, "Man, we need to establish some basic standards for test coverage. We need to establish some basic standards for how we do what I call a factory inspection on a product once it's ready to roll off the assembly line." Do we have a checklist for what we call product quality and what does product quality mean? Those basic things weren't there. And so the idea that we should be spending time measuring adoption metrics is absolute insanity.

**Matt MacInnis** (00:22:08):
You're skipping a lot of steps between here and there. And so we have made great strides and I think it's translating to product quality improvements for our customers, but I feel, as I said, a little dumb for the way I was thinking about it before I jumped into the deep end. There is just no excuse as an executive for sitting outside of the mess and thinking you know the answers. It's a cardinal sin as an executive to do that. You need to go and see. You'd be in the boiler room, you need to study the system, bottom up and develop hypotheses for how to amend the system. And that's what I've been doing.

**Lenny Rachitsky** (00:22:46):
I love hearing this because so many people have teams that are not functioning incredibly well and hearing from someone that is not a longtime product person come in and try to fix these problems, I think is really useful and interesting for people to hear. To dig into this a little bit more, was the big lesson and kind of eye-opening moment that there's a lot of foundational work that needs to happen to achieve this outcome that you're trying to achieve, which is measure engagement and adoption well? Is it like tracking and metrics and data science? Is that kind of the lesson there?

**Matt MacInnis** (00:23:18):
The lesson is that everything must be done in its time and order, and you can move really, really quickly. There's no sort of excuse not to move with urgency on all of these things, but you got to do them in order and you have to lead bottom up. You got to lead from the specific circumstances you observe. And I think for me, one of the best things that's happened over the last 11 months is that I've gained a greater trust in my own instincts, that sort of patterns I've matched across other functions do indeed apply in product, but I have both the advantage and disadvantage of not having led a product function before and therefore must think about every problem from first principles. I have no choice. I can read shit on the internet. I can listen to clear thinkers on topics and import their ideas, but I'm very reluctant to import an idea without breaking it down into its constituent parts and figuring out how it applies at Rippling.

**Matt MacInnis** (00:24:29):
And so I don't actually give a shit about adoption metrics as a matter of principle. I care about adoption metrics when I care about adoption metrics. I realize that that's a total logical statement, but it's like I'll get there. And so in certain parts of our product, I really do care about adoption metrics. I care a lot about adoption metrics in our applicant tracking system, our recruiting product, because it's in a really good place from a usability standpoint, it's very well instrumented, it's got very happy users, it's got an awesome growth profile, and so we should still ...

**Matt MacInnis** (00:25:00):
It's got an awesome growth profile. So we should be really focused on the adoption metrics because I think that's going to be an important ingredient to low churn over time, removing friction from the implementation process as an example.

**Matt MacInnis** (00:25:13):
There are other parts of our product where I would say I don't care at all about adoption and am much more focused on foundational things like I said earlier, test coverage or whatever, just to make sure that the thing is stable and good and delivering exactly what it's supposed to deliver once it's adopted.

**Lenny Rachitsky** (00:25:28):
Now that you're on the inside of the product team, what's something that you think people outside of product, say Matt two years ago or other, I don't know, go to market leads, other execs should hear, need to understand about product that they don't until they're on the inside?

**Matt MacInnis** (00:25:44):
I'll give you another framework that I like to use. In the financial world, there's this concept of alpha. Alpha is outperformance relative to the index. So that's why you have seekingalpha.com as a very popular website. What they mean by that is you're looking to buy something, some combination of assets that will outperform, let's say the S&P 500, if that's your benchmark. So alpha is the outperformance relative to the index.

**Matt MacInnis** (00:26:12):
And then you have the concept of beta. Beta is just volatility. Beta's not good. A high beta stock jerks around for no particular reason. It's discorrelated with the index. It's very high beta. Great if you're an options trader, but other than that, it's not really something you want in an asset.

**Matt MacInnis** (00:26:28):
So your ideal stock is a very high alpha, very low beta stock. They don't really come in that shape because alpha and beta tend to be correlated, but that's what you want when you buy a financial asset.

**Matt MacInnis** (00:26:40):
So what's the analogy? I think you have high alpha people who are very valuable. I think you also have low beta people who are also very valuable people. Dennis Rodman, basketball player, nut job, very high alpha. And there's room on every team for one Dennis Rodman is a favorite of mine. It's like you can have one difficult employee who's got a ton of upside.

**Matt MacInnis** (00:27:12):
So this alpha beta thing, I use it pretty often when contemplating what kind of person I want and also what kind of process I want. So when you're building a product from zero to one, you're probably pursuing alpha. You're looking for some angle on this market or this customer problem where the product is actually going to provide an outsized return relative to whatever the default solution is. When you have a more mature product or if you have somebody in the product operations group or whatever, you probably want a more low beta environment where it's like it cranks it out, it does it very reliably.

**Matt MacInnis** (00:27:44):
Our payroll product, we badly want the payroll product to be very low beta. We really don't want the payroll product to have any unpredictability or aberration, and so we're willing to accept more process.

**Matt MacInnis** (00:27:55):
And here's a fundamental principle of design in an organization, which is that processes, processes in a business exist for the sole purpose of lowering beta. Processes are for decreasing volatility in the output of the system. The downside of a process is that it suppresses alpha. And you have to be super, super careful and judicious in the application of process and the product team to know that you're lowering beta in the places where you want to do that without suppressing alpha in the places where you need it.

**Matt MacInnis** (00:28:35):
So as we've gone through the last year of reforming the way that we build product at Rippling, it's been a game of recognizing those places where I need to implement a touch of process, just a touch. Other places where I need to implement a very clear, rigid process where I don't want alpha, I just want low beta. So examples of this are let's say our product quality list, which we lovingly at Rippling call the PQL.

**Lenny Rachitsky** (00:29:00):
Why PQL?

**Matt MacInnis** (00:29:01):
Yeah, so it's actually a really important thing. I think if you want to bring about cultural change in a team ... Look, we have 1,300 people in our R&D organization. It's a big ship that we have to steer. If you want to create a moment that sticks in people's brains and sort of becomes a zeitgeist or something that they latch onto, you got to create an entity, a vessel for meaning, and then you got to fill that vessel with your meaning.

**Lenny Rachitsky** (00:29:23):
A meme, you might say.

**Matt MacInnis** (00:29:24):
Yeah, well, sure, a meme. A meme is actually a good example of this in common culture. In pop culture. I think it's why, when people come to the table with ideas from the outside, I welcome those outside ideas. But the first thing I ask the person to do is to tell me what they mean without using those words. So when someone comes in and says, "Hey, I want to do this thing on strategy." I'm like, "Cool. Tell me what you mean without using the word strategy." And it forces them to break it down into its constituent parts. And if they can articulate it clearly without using that word, I know that they know what they're talking about. And if they just fumble around with the word strategy again, I'm like, "Okay, you actually haven't thought this through."

**Matt MacInnis** (00:30:01):
So with the PQL, with the product quality list, it's like I could come up with some generic term for this, but I really want a new joiner at the company to understand that this is an idiosyncratic thing to Rippling. This is unique to us. You want to understand this thing. I also want it to become a component of common parlance in the day-to-day work of the product management and engineering teams. So PQL, as cheeky or silly as it sounds, was deliberately sort of angular or stood out as a vessel I could fill with a particular meaning, and so we have a product quality list.

**Matt MacInnis** (00:30:32):
And the product quality list is lightweight in the sense that it just articulates in the simplest ways the standards we want you to meet when you ship a product. It doesn't apply to every product, not every line applies to every product, but it's comprehensive and it provides me with a framework for iterating over time as we learn.

**Matt MacInnis** (00:30:50):
So just yesterday, we shipped the product to Parker. This is part of our process. When we ship a new product, it goes to Parker, who is the big admin for Rippling at Rippling. If you're not aware, Parker is the sole payroll administrator for Rippling for all 5,200 employees. He personally runs payroll always, there is no exception, for all 5,200 people. He does complain about it sometimes, but it's a remarkable achievement for the software and perhaps for him. So he also installs any new app that we're going to install for ourselves because we dog food the hell out of everything we build.

**Matt MacInnis** (00:31:22):
Yesterday, he goes to install this new application. We're about to ship a new app for feedback, allowing people to give one another feedback on their companies. And he installs it and he goes in and it dumps them onto an empty screen. And he's like, "What the fuck is this? What is this? What's going on? Hey, wow, talk about fail." So I chop another one of my fingers off, I'm down to nine. And I'm like, "Well, what did we miss?"

**Matt MacInnis** (00:31:48):
What we missed was there was a fucking feature flag, a fucking feature flag. And I'm not allowed to say feature flag without fucking in front of it because feature flags are the bane of my existence and the worst things in the world that constantly cause problems. Engineers put one in temporarily and forget about it. It's like shims if you're building a house and the general contractor puts little shims in places and then forgets that they put the shims there and then builds a wall over them and eventually the shim fails and all of a sudden your door doesn't fit. Feature flags are super dangerous and need to be managed carefully, so fucking feature flags.

**Matt MacInnis** (00:32:18):
Anyway, we had one. Parker installs it, they forgot to disable the feature flag. He gets a blank screen when he installs the application. What did I do? My reaction was, "Ugh." Go back to the team, give them direct feedback, tell them not to make that mistake again. But also ask the question, "How do we miss this in the factory inspection process?"

**Matt MacInnis** (00:32:37):
And the answer is we didn't have any line item in the PQL for feature flags. So I added a line to the fucking PQL that said, "You are allowed to have one feature flag that governs your entire product at ship." It's an extreme standard that might not be achievable, but it's the standard we aspire to.

**Matt MacInnis** (00:32:59):
This framework, the PQL, given these lightweight checklists, iterated on consistently in response to everything we learn as we go, constitutes a very nice lightweight way to lower the beta of the system with hopefully only a modicum of negative impact on the alpha for how we build product.

**Matt MacInnis** (00:33:22):
You asked me a very simple question, I gave you a very long-winded answer, but these frameworks help me design systems that scale across one going to 2,000 technical workers.

**Lenny Rachitsky** (00:33:34):
Wow. Okay. By the way, PQL, is that like an acronym or it's just like, I like this word we're going to call it PQL?

**Matt MacInnis** (00:33:40):
Product quality list.

**Lenny Rachitsky** (00:33:42):
Okay, I see. So it's the [inaudible 00:33:44]. Okay.

**Matt MacInnis** (00:33:46):
PQL, which how could you pronounce it other than pickle?

**Lenny Rachitsky** (00:33:49):
I'm imagining all your decks have little PQL emojis and the-

**Matt MacInnis** (00:33:53):
The pickle emoji thing, the dancing pickle in Slack, there's a lot of-

**Lenny Rachitsky** (00:33:56):
[inaudible 00:33:56].

**Matt MacInnis** (00:33:56):
Yeah. It lends itself to a bit of fun.

**Lenny Rachitsky** (00:33:58):
What I think about is pickle Rick. Do you get that reference?

**Matt MacInnis** (00:34:02):
This is a Rorschach test.

**Lenny Rachitsky** (00:34:04):
Okay. So this high alpha, low beta, I love this concept. So the idea is depending on the team, depending on the problem, we need a high alpha, low beta person or actually okay with a lot of variants for this specific project that's actually [inaudible 00:34:16]-

**Matt MacInnis** (00:34:16):
Yeah, we're willing to accept a bunch of volatility in this area in exchange for the upside we get from the creativity and risk taking of these people or the lack of process that sort of gives them the latitude to do what they want to do.

**Lenny Rachitsky** (00:34:26):
So when you're hiring, you're looking for, again, is this person low beta or not? That's going to [inaudible 00:34:30]-

**Matt MacInnis** (00:34:30):
For sure. It's really quite a useful way. You know when you meet a candidate and you ... My modus operandi, and I think with talk about hiring for a second, I think I've spent a lot of time with teams at Rippling talking about how I hire. And it is born of batting practice. It'd be super interesting to actually be able to rewind the tape on my life and sort of contemplate how many candidates I've met in every context. Many thousands, maybe tens of thousands, I don't know. It's a lot of batting practice and a lot of model training in my brain.

**Matt MacInnis** (00:35:06):
So I rely a lot of my intuition, which of course HR people say you're not supposed to do. That's complete bullshit. If you have a good intuition, you should absolutely rely on your intuition.

**Matt MacInnis** (00:35:15):
And what you have to do after you have a reaction to a candidate when you're looking at hiring somebody is you need to decode your intuition so that it can be expressed to other people productively. So one of the frameworks that I use for this is SPOTAC. It's a very ugly acronym. There's a hat tip to somebody out there in the universe who originally thought of this. It's not me, but I adopted it. And it's that people are smart, passionate, optimistic, tenacious, adaptable, and kind. Those five things. Six, can't even count. I told you I lost a finger when I made a mistake, so I was down one.

**Lenny Rachitsky** (00:35:54):
Nine to go.

**Matt MacInnis** (00:35:54):
SPOTAC isn't by itself a good top down framework, but when you're thinking about, "Oh, why did this candidate just ... Why did it not click? Why did I not like them?" You go down the list, you're like, "Oh, yeah, no, this person, it's that they were not excited about the idea. They weren't passionate." It's that they talked shit about their previous manager and that they were a victim of the performance of their last two companies. That's what it was, they're not optimistic.

**Matt MacInnis** (00:36:24):
The framework is super useful to evaluating people. And I think the alpha beta framework is also super useful when you come away from a conversation and you're like, "I like that guy. I think he'd be really, really good. Why is it that I don't think he would do a good job on this product in particular?" And the answer is like, "This is a high alpha product area and he's a low beta person." Valuable, but definitely not the right fit for this. So I think it's really useful in that context as well.

**Lenny Rachitsky** (00:36:48):
I love all these frameworks. You're speaking to this audience, framework to frameworks, frameworks.

**Matt MacInnis** (00:36:52):
Yeah.

**Lenny Rachitsky** (00:36:53):
So high alpha, low beta, sometimes high beta is okay, SPOTAC. In hiring, is there anything else that you find really useful? Before we move on to a different topic.

**Matt MacInnis** (00:37:03):
When I first started working in the product organization, I was introduced to an interview framework or an interview tactic that I hadn't really used much at all, I think in my career, which is that every product person at every seniority level is given the same case study. And the case study is extraordinarily difficult. It requires you to think about many, many dimensions simultaneously, to think about data propagation issues. It gets quite technical.

**Matt MacInnis** (00:37:37):
And the rubric that we use to sort of evaluate performance of that case study is it gives you guidance on what for us, like an entry level PM looks like, what a junior, mid-career, senior executive PM might look like. And everybody comes away from that interview feeling like poop, like they had failed it. Whereas on our side of it, we're like, "Wow, that person got really far." They saw around three or four corners in a really impressive way. There was 10 they didn't see around, but they saw around four of the hardest ones. And they were not defensive when we gave them new information that called into question the validity of their solution. And they were willing to interrupt us to ask more questions," and and and, like a lot of the sort of basic human interaction models.

**Matt MacInnis** (00:38:25):
You never think that giving someone an impossible task and even including the L5 person versus the VP on the same thing would be productive. And let's just say our recruiting team still sort of kvetches a bit about this and feels like we eliminate people too aggressively at this stage of the interview process. But I've found the wisdom in it and think it's actually quite useful to give everyone the same simple, complicated prompt and just see. Hand them a drill bit, give them the concrete wall and see if they can get a millimeter or an inch into the concrete. They're never going to get all the way through the wall. It doesn't matter. You're going to learn a lot. And I've found that to be kind of an eye-opening new thing for me that has been fun.

**Matt MacInnis** (00:39:04):
Look, the joy of product and the joy of product management and the joy of being part of product, I think there's a bunch of joys actually, if I could give you a sort of running list, but one of the big joys is that you get to work with some of the smartest people in software. Engineers are very smart. They're not always the best sort of social entities. Salespeople are awesome social entities. They're not always the best systems thinkers. You go down the list.

**Matt MacInnis** (00:39:28):
But the magic of product management is that you kind of have to ... We talk about the mini CEO. I think it's kind of a stupid misnomer, but there's some wisdom there. And I think the wisdom is that you have to be a polymath. You've got to be really good at working with other people. You got to be good at communications and articulation. You got to be good at project management. You got to be good at the science and the math and the engineering. And it's really fucking cool. So I think one of the great joys of this job for me has been interacting with the tippity top of the smartest and most polymathic people in the industry.

**Matt MacInnis** (00:40:01):
I'll say one other thing about what I love about leading product, which is as a COO, my job was to accept the product as it was and optimize everything around that. My job was to make sure that the product operations, in our business, the interface to the insurance carriers, the interface to the payment entities, the government regulators, that stuff all just sort of worked. It was to make sure that our sales engine, our marketing engine, all the go to-market stuff optimized itself around what the product was. It was about recruiting and making sure we got people in to work on the product. You kind of go down any function that isn't in R&D, and I had some hand in trying to figure out how to make that function work to the best of its ability, given what the product was.

**Matt MacInnis** (00:40:48):
And now that I lead product, I'm like, "Oh, wow. This is the high order bit." Not that I didn't sort of understand that, but now I really get that product is the high order bit. If you get the product right, it fits in the market, everything else gets easier. Finance is easier, sales is easier, marketing is easier, recruiting is easier, everything gets fucking easier. So I think the other joy of leading the product function is that I get to set the highest order bit in the business's success to one.

**Lenny Rachitsky** (00:41:22):
This is really great to hear. A lot of times people outside product don't understand these sorts of things and look down on product a lot of times, especially sales folks, COOs a lot of times. I love that you're seeing this and realizing this and recognizing just how important and interesting and challenging this work is and just how awesome PMs are. As you know, a lot of people are a very anti-product manager. "Why do we need product managers? We don't need them. Slow everything down, all this process."

**Matt MacInnis** (00:41:51):
Yeah. I have a distinction there, which is that I'm anti-shitty product managers.

**Lenny Rachitsky** (00:41:53):
That's exactly how I put it. If you hate product managers, you just haven't worked with a great product manager.

**Matt MacInnis** (00:41:58):
Well, it's like, look, I love wine, wine's one of my things. And I've learned a lot about wine. And one of my favorite lines is like, "I don't like Chardonnay." And I'm like, "No, no, no, no, no. Chardonnay's are the most fucking amazing varieties of wine in the world. You just haven't had good Chardonnay. And there's a Chardonnay out there for you." Product management, it's like you don't like product management, you think product managers suck. It's like, well, you just haven't had a good Chardonnay yet.

**Lenny Rachitsky** (00:42:22):
That's exactly what I [inaudible 00:42:24]-

**Matt MacInnis** (00:42:24):
Once you have one, you can't unlearn it.

**Lenny Rachitsky** (00:42:26):
You're like, "Let's find that PM ASAP."

**Matt MacInnis** (00:42:30):
No, let's find that Chardonnay ASAP.

**Lenny Rachitsky** (00:42:34):
[inaudible 00:42:34] with some Chardonnay. You touched on this product market fit point, and I want to double down on this thread. You've had a couple really interesting experiences of struggling to find product market fit with your own startup. You said you worked on it for nine years, you said?

**Matt MacInnis** (00:42:47):
Mm-hmm.

**Lenny Rachitsky** (00:42:47):
Okay. And then with Rippling, complete opposite, extreme product market fit, up and to the right. What's something you've learned about just that that you think people maybe don't understand about what it feels like, what it takes to get to product market fit, how things change?

**Matt MacInnis** (00:42:59):
There's a line that this venture capitalist, whose name I will not mention, said, which was that product market fit is a sort of thing where you absolutely know it when you see it, and therefore if you don't absolutely know it, you don't have it.

**Matt MacInnis** (00:43:17):
And this kind of gets back to my point about learning from mistakes versus successes. It's like, ah, man, over and over again, over the course of the many years that I spent at Inkling, we thought we had it. We thought we had product market fit, maybe, maybe. And in hindsight, with the benefit of now having experienced solid product market fit, it was so, so obvious that we didn't.

**Matt MacInnis** (00:43:45):
And I've invested in like 60 companies or 70 companies. I don't know, it's not something I actively do. But opportunities, by virtue, I think, of my role at Rippling, sort of show up. And I talk to lots of entrepreneurs and I love it and I find it super stimulating and I love the fresh ideas and it's just something I do as a real cherry on top of the sport that I play already. But when I get the investor updates for the guys who've been at it for like three, four years and I read the updates from them that I sent to my investors in 2011 and 2012, I'm kind of heartbroken.

**Matt MacInnis** (00:44:29):
We talk in Silicon Valley about never quit, but that is complete absolute venture capital bullshit. The incentive of venture capitalist is to put money into your company and milk you dry. They never get their money back. There is no way for them to take that investment back. So the only logical desire that they would have is for you to keep trying against all odds because there is the occasional example where someone pivoted from A to X and it was wildly different and it worked. Slack was originally some sort of a gaming company and became corporate chat. Airbnb maybe. It's like there's some examples of companies having made wild pivots and succeeded, but man, is that rare. Just so exceedingly rare.

**Matt MacInnis** (00:45:21):
And I think it's important to remember, I'm 45 years old, we're going to be on the planet, the average age of a man in the United States when he dies is something in the mid 70s. I got 20, 30, maybe if I'm lucky, 40 years left on the planet. Very conscious of the time that I have. And I don't regret what I did at Inkling, I learned a lot. It informed what I do now. I don't think the chapter I'm in right now could have come without the chapters before it. So it's a beautiful, wonderful thing that I did what I did. But when I read the investor update and I'm like, "You're where I was and you are not getting out of this."

**Matt MacInnis** (00:45:56):
The Silicon Valley try until you die mindset is not pro-entrepreneur, it's pro-venture capitalist. And I know why that is, but I think it's important to say out loud that you should fucking quit. You should reset the clock, you should reset the cap table because trust me, product market fit when it arrives is insane and it's exciting and you should pursue it. And never delude yourself into believing you have it when you don't. It is dangerous and regrettable. How's that for a speech?

**Lenny Rachitsky** (00:46:33):
Beautiful. The anti-VC speech. The-

**Matt MacInnis** (00:46:38):
I got more where that came from. By the way, it's not anti-VC. It's anti-

**Lenny Rachitsky** (00:46:43):
VC propaganda.

**Matt MacInnis** (00:46:45):
Everybody's acting in accordance with their incentives in Silicon Valley, the executives, the founders, the venture ... Everybody's, of course, behaving in accordance with their incentives. And the venture capitalists have very strong enduring incentives that have shaped the dynamic of how Silicon Valley works. There's nothing wrong with that. It's just really, really important to point them out and scream at them for the 25-year-old entrepreneur who has no fucking clue how this stuff works.

**Matt MacInnis** (00:47:12):
Trust me, the 45-year-old entrepreneur or the 50-year-old venture capitalist who've been in the game for a while, they get it. They've observed it. They know what it's like. The system is there to take advantage of the people who don't, or at least it is the easiest prey for the incentive structures, not for venture capitalists as individual people who are beautiful and all of them are just really wonderful people. It's just that the incentive structures lead to some real harm, I think, in certain cases.

**Lenny Rachitsky** (00:47:37):
And the thing I find is when you do quit, VCs ... I'm always just like, "Hey, let me know when you're starting your next thing. I'm excited to invest." They're rarely, unless they're not a great VC, rarely are they just pissed at you for how could you possibly not make this work [inaudible 00:47:52].

**Matt MacInnis** (00:47:52):
That's the thing, as a founder, when it's time to throw in the towel on your business and you're so obsessed with giving money back to the cap table, I always remind the entrepreneur like, "Hey, if you're in the seed investing game, your forecast is zero. Your assumption on every investment is that it's going to go to zero." Any seed investor who doesn't take that stance is off their rocker anyway. They're a very bad investor. Seek investors who play the long game, who want to be in your second and third company and are willing to take a bet on the first one and let it go to zero so that you can get on with stuff. This is like, I've had that conversation many times.

**Lenny Rachitsky** (00:48:28):
**Matt MacInnis** (00:49:38):
Here, look, history provides us with a clear guide. When you look at companies having hit it big, they hit big pretty quick. It's very, very dangerous to be late to the party, it's very, very dangerous to be early to the party, and the vast majority of the time, that's the problem. Rippling, had it been started in 2014, would not be what it is today. I think Rippling, had it been started today, would not be what-

**Matt MacInnis** (00:50:00):
...Not be what it is today. I think Rippling, had it been started today, would not be what it is five years from now today, and so I think timing is a lot and it's very hard to control for, but when you get the timing right and the market is real and the product works, product market fit, like I said earlier, it's super clear, and so if I were to pick a number out of a hat just from my lived experience, I think it's very important, one aside, don't ask people for advice, ask people for relevant experience. If you ask them for advice, they will always give it, but if you ask them for relevant experience, they rarely have any to offer, and if they don't have any to offer, then don't ask for their advice.

**Matt MacInnis** (00:50:41):
So ask people for relevant experience, and I try to do this with my own entrepreneurs when I work with them, it's like, let me offer you whatever relevant experience I have, and my relevant experience on this topic of when to quit is like, I think we could have called it after the second or third pivot, which was somewhere around year four. It is of course very important to believe in what you're building and to be persistent, but there is definitely no shame in saying, "Look, we've pivoted once or twice. It's not catching. I got to go do the next thing," and I think if you're year four, year five in your entrepreneurship journey, and it's not just obviously a screaming rip-roaring growth story, it's extraordinarily difficult. This is so extremely rare. So beyond even already the rare scores you're going to face from the outset that now is going to convert to something crazy. So that's hard to hear, I guess, but man, it can be really liberating when you're like, "Fuck it, I'm going to do this. I have the energy. I'm going to do it again. I'm just going to do it with a clean sheet."

**Lenny Rachitsky** (00:51:48):
That is really helpful. You have this really fun way of describing product market fit around receptors and drugs.

**Matt MacInnis** (00:51:54):
Oh yeah. Yeah. I think this is a really fundamentally misunderstood dynamic. When founders message me and they're like, "Hey, like my LinkedIn post and my tweet for this launch," I do it. I retweet it, I like it, whatever. Nobody follows my Twitter anyways, it doesn't matter, but I do that, but I think to myself, this is not what this is about. This is not how great companies are built. It can be a nucleating event, but it's not a major thing, because nobody cares about your company. Your launch doesn't matter. Big, fat, pull the slingshot back, launches amount to the teeniest thimble of water in the ocean of noise about startups and companies, and so you just got to build it brick by brick bottom up, and these launches don't really amount to much, and so how do you think about that? How do you think about the insignificance of your launch or you think about all the effort you're putting into building a product that you believe is going to have product market fit?

**Matt MacInnis** (00:52:56):
Well, if you recognize that the market is immutable, no amount of tweeting, LinkedIn posting, advertising is going to change whether the market wants your product. It might raise awareness about your product, but it's not going to change whether somebody wants it. Then you take a different mindset. You have to view your startup as running an experiment in the universe to see what you get in return for that, and this analogy of drug discovery and binding receptors is like nobody at Genentech thinks they can market their way to better performance inside your body. The binding receptors for that drug, they exist or they don't, and when they build their product, their goal is to find out whether the binding receptors exist, but fate already has decided the outcome. This is absolutely true of every software product you build. Fate has already decided the outcome. The market's either going to latch onto your product and run with it or it's not. Do not ship the product, find a lack of success, and then try to market your way through that, because the binding receptors likely don't exist, and for me, it was a very liberating mindset, because now I just have to find the right drug, and I can forget trying to convince the body to develop the binding receptors for whatever it is that I'm building.

**Lenny Rachitsky** (00:54:13):
What I love about your advice here is you were an early investor in Notion, which is one of the classic stories of it took them... I think it was four years to get to something. They moved to Japan, they worked on the whole thing, and so is [inaudible 00:54:24] from there? Is that a rare example where it actually worked? And that's not an example to be inspired by, because it's extremely, extremely rare. Let's talk about alpha beta again.

**Matt MacInnis** (00:54:32):
Okay. As an investor, you might build a checklist of things you want to make sure are true or false about a company and hope that that's going to yield the kind of investment success you're looking for. Does it have this kind of founder? Is it a C Corp in Delaware? Boom, boom, boom, boom, boom, boom, boom, and these checklists are all about what? They're all about suppressing beta. They're about avoiding avoidable mistakes. They're about bringing stability. Jeff Lewis is an investor who has many angular views on things, and I think one of his most enduring phrases is narrative violations. This idea that the common wisdom must be violated in some way by every company that has an outsized success. It is absolutely true, and when I give these general observations on the patterns in Silicon Valley, the most successful businesses inevitably violate something on that list in some really important way.

**Matt MacInnis** (00:55:37):
So Notion, you can't replicate Notion's success as an entrepreneur. You can't replicate it because you're not Ivan. You can't replicate it because you're not Notion. You can't replicate it because it's not 2010 when they started the company. Do the math on that. Or 2011, actually. These guys stuck with it. They went through hell. They pivoted. They went to Japan and sat in kimonos and meditated on what they were going to build, and by hook/crook, they got to where they are today as a really wildly successful business in an extraordinarily difficult market where building businesses is virtually impossible in productivity. It is dominated by Google and Microsoft. Carving out your own niche in that market is just unthinkable, and so I look at Notion as having succeeded by virtue of the narrative violation of persistence, I don't think is a good idea for very many people that happen to work for them, and I look at it as being a function of the founding team and their specific idiosyncrasies, the absolute insistence on craftsmanship from Ivan. This is him. That's his thing.

**Matt MacInnis** (00:56:55):
The takeaway lesson is not give up or don't give up. The takeaway lesson is certainly not go do what Notion did. The takeaway lesson is that every company succeeds on the foundations of the idiosyncrasies of the founder. The idiosyncrasies of the founder. Rippling succeeds for almost the polar opposite reasons that Notion succeeds, but in both cases, the companies succeed on the idiosyncrasies of the founder, and so embracing that, recognizing those idiosyncrasies, that's what great investors do. They spot that element of spikiness and greatness in a candidate investment, and they convert that to a commitment, and then of course the investor or the good ones accept what they get in exchange from that for the universe, from the universe.

**Lenny Rachitsky** (00:57:44):
I love that we went in this direction. I wasn't planning to talk about your investing career. Just to give people a reason to listen to this and maybe rewind, and I want to ask another question around investing. What are some other companies you invested in early?

**Matt MacInnis** (00:57:55):
First of all, okay, so I sort of hate the question. What are some other companies you've invested in? It's a fair question, but the problem is I'm going to give you a bunch of companies I've invested in that won, that are really notable. So what I would like to do instead of answering that question ... Here, let me give you some bait. I was one of the first investors in Notion. I was perhaps the first, I don't know, ask Ivan. Clever, which had a great exit. I was one of the first investors in Zenefits, if you've heard of it.

**Lenny Rachitsky** (00:58:26):
Heard of it.

**Matt MacInnis** (00:58:27):
I was, before I joined, one of the first investors in Rippling, and then more recently invested in ... Here's a funny one. I was one of the first investors in Deal, if you've heard of them.

**Matt MacInnis** (00:58:43):
I was able to exit that position, and then hopefully that company's going to zero with their criminal behavior, but whatever, but more recently, if you know Decagon, which is doing some really cool stuff on the AI front.

**Lenny Rachitsky** (00:58:58):
Killing it.

**Matt MacInnis** (00:59:00):
I'm in laying chain. Great. So those are some companies that you maybe have heard of, but how about I invested in Macro. Founder was Derek Lee. Macro's out of business. I invested in Debrief, Ned Rockson. It's out of business. I invested in Verb Data with David Hertz out of business. I'm reading from a list. I invested in... What's the number? 70 companies according to this list where I track things and most of them went to zero and all those founders were awesome, all those founders were kick, and all those founders put a ton of energy into building their businesses, and they went to zero and they're enduring relationships.

**Matt MacInnis** (00:59:37):
I can call on any of those people, I think, maybe with the exception of Deal, and call in a favor and have... and I've got a few subsequent... and actually a lot of them joined Rippling, believe it or not. So I don't know. Companies I've invested in is a long list, and I love to give you names of companies that don't exist anymore because it's self-serving and a horrible survivorship bias to just list the good ones.

**Lenny Rachitsky** (01:00:00):
I love that answer. I think you're being modest in the context of your hit rate is clearly very high. Even one or two incredibly successful companies out of 70 is a win in VC, and so you're doing very well, but I think that's a really important perspective. When you see people's logos on their websites of all the companies they've invested in, you have no idea how many hit bats they've had.

**Matt MacInnis** (01:00:22):
I think it's good practice to ask people to give you the full list. Yeah.

**Lenny Rachitsky** (01:00:24):
What are your favorite failures that you've invested in?

**Matt MacInnis** (01:00:29):
Oh, no. I'm not actually... Okay.

**Lenny Rachitsky** (01:00:31):
Well, obviously, no, we don't need spend time on it, but I think it's actually a really good question, but yeah, what are some of your best failed investments? Show me the logos of the companies that didn't work out.

**Matt MacInnis** (01:00:39):
It's a juicy question. Yeah.

**Lenny Rachitsky** (01:00:42):
There's a topic around this area that I wanted to spend time on, and I haven't heard anyone think of things this way, which is this idea you talk about of compounding plus power law plus entropy and how that's a really useful frame to think about business.

**Matt MacInnis** (01:00:58):
So you kicked this conversation off sort of invoking the extraordinary outcomes, demand, extraordinary efforts line, hat tip to Dan Gill, and these are part and parcel. Man, understanding the nature of the universe is a pretty good way to work within it, and so power law distributions happen everywhere. It explains why so few people control so much wealth. It explains why Steph Curry is just so vastly better than the next guy down on the list on the basketball team. It explains why populations are concentrated into a relatively small number of mega cities in the world. It's like, power law distribution just plays out everywhere, and once you see it, you can't unsee it. It sort of plays out in many dynamics.

**Matt MacInnis** (01:01:47):
People tend to think that the world plays on a more linear relationship where the X and Y axis are sort of Y equals X, but that is absolutely not the case, and the implications are profound. It's like if you build something to 80 or 90%, the Y axis is barely budged yet. You haven't hit the inflection point in terms of reward, and so the implication of the power law more broadly is that people who are in the top 10%, the top 5%, don't just get 10 or 20% more reward. They get 10X the reward or 100X the reward. It's really dramatic.

**Matt MacInnis** (01:02:20):
Entropy, the second law of thermodynamics is a very simple concept. It's the reason your sock drawer becomes messy. It's the reason that potholes form. It's the reason we have to put so much energy into maintaining the aircraft we fly to keep them safe because they really, really, really want to fall apart, and that's the nature of things. If you abandon a city for a few months, it starts to go fallow, and so entropy is just this concept that shit tends toward disorder.

**Matt MacInnis** (01:02:48):
The universe, I mean, life itself is a temporary victory against entropy. You and I should not exist. The sun gives energy to the planet. It organizes stuff that we can eat and we fight entropy until we lose the battle somewhere, as I said earlier at the age of 70 or 80, if we're lucky. What does this have to do with product? This is really about effort.

**Matt MacInnis** (01:03:17):
The only antidote to entropy, the only antidote to decay in a system is energy. You got to inject energy. So if you have a code base, every line of code that you add to that code base increases the entropy of that system and demands ever more energy from human beings to go and intend to it to make sure it doesn't break, and if you want to achieve greatness, if you want an extraordinary outcome, and in particular, if you want to be in the top 10%, top 5% on the X axis so that the Y axis is through the roof, then you have to relentlessly inject energy at every single step of the game. Teams will, sadly, but because we are all human, teams will always optimize for local comfort over company outcomes.

**Matt MacInnis** (01:04:11):
Not because they get together and think, "We should do that," although unions do do that unequivocally, deliberately, but even in a collection of product managers or engineers, what's going to happen over time is entropy is going to creep in and people are going to optimize for local comfort. Your job as an executive, as a leader, is to fight that entropy tooth and nail every single day. It means that every time you see a bug, every time you see a bug, not most of the time, every time you go and you drop it at the feet of the product manager or the engineering manager, every time, in public, preferably, it means that every time someone comes to you to hire someone and says that they have skipped the back channel, every time you're like, "You can't hire this person unless you do the back channel," it means that when you get into the board and tired zone on any process, that you as the executive have got to demand the 99th percentile of energy, because otherwise entropy creeps in and the system decays. You have to do this.

**Matt MacInnis** (01:05:19):
One of the messages that I delivered recently at our big executive big... Like our top 75 manager offsite that we do roughly every 18 months, was a reminder that if the purest form of ambition and the purest and most intense source of energy in the business is the founder CEO, that every next concentric circle of management beyond the founder CEO has the potential to be an order of magnitude drop off in intensity, and that is dangerous because if you go to two layers and it's two orders of magnitude drop off and signal and intensity, that is a very dysfunctional organization. So what I say to the team was, it's not that you need to buffer people from the intensity of the CEO, it's that you need to absolutely mirror that intensity.

**Matt MacInnis** (01:06:13):
There are plenty of constituents in the business around you who will advocate for relaxing. There is an infinite supply of people under you who will buffer other team members from the intensity of the demands. Your job is not to be one of those buffers. Your job is to preserve that intensity at its highest possible level and let the buffering happen somewhere else, and that's the point is that entropy creeps into the system insidiously and slowly over time off your radar and you have to maintain that intensity every minute of every day to try and fight it if you want to stay at the extreme right end of the power law that obviously governs the outcome of everything that we build.

**Lenny Rachitsky** (01:07:03):
What does that look like to pass along that intensity? What does that feel like? What does that look like? So say Parker comes to you, "This bug sucks. I got this broken screen." You cut off your finger. Maybe that's the example.

**Lenny Rachitsky** (01:07:18):
Okay. Still got them. I got all 10.

**Matt MacInnis** (01:07:21):
I'll give you concrete examples. Actually, the joke that I sort of played on this one when I presented to the team was that the next sort of slide in my presentation was with the sound effect, the Slack huddle thing, and Parker's icon in Slack is just, he uses the generic yellow...

**Lenny Rachitsky** (01:07:42):
The egg?

**Matt MacInnis** (01:07:43):
Yeah.

**Lenny Rachitsky** (01:07:43):
Oh, wow.

**Matt MacInnis** (01:07:46):
It's funny, and so everybody knows the yellow egg is Parker, and so the next slide in the presentation was boop, boop, boop, boop, boop, boop, boop, which is the sound that Slack makes when someone calls you, and it was Parker Conrad is inviting you to a huddle, and then the next slide was Parker Conrad is modeling personal intensity, and the idea is that if you know, you know, if you've ever been dragged into an... "I want to talk about this fucking problem right now and whatever you're doing, unless it's an interview, I want you to come and have a conversation with me."

**Matt MacInnis** (01:08:13):
That intensity is one place where it plays out. Every product team at Rippling, and we have a lot of them now, have public feedback channels. I am in there upside down on everything I find when I use those products, and I just model for everybody that this is how at least I want to do it, which is, "I don't like this. I don't understand this. Tell me why this is this way." Boom, boom, boom, boom, boom, and people jump in and respond. The factory inspection process that I mentioned, which is where at the end of the assembly line, I jump out with the pickle and we talk about all of the elements. You have to record a loom of every major flow through the product. I personally review every one of those flows, I got a backlog I got to catch up on today and provide feedback to people always in a public channel so that every other product manager and engineering manager can jump in and see how the process has worked for other teams.

**Matt MacInnis** (01:09:01):
It's about modeling the intensity publicly so that other people can say, "Okay, this is how we do things around here," and it gives the reaction from the team that received the message that you have to inject that energy every minute of every day and that there are no exceptions, was not like, "Ooh, that's exhausting." The reaction is, "Oh, that is so invigorating." It's so wonderful to hear that we're going to achieve these great outcomes, or at least we have a shot at doing so, and that you're empowering me to follow my instinct, which is to really push for the best result.

**Matt MacInnis** (01:09:39):
The reaction universally is like, "Ugh, what a relief that I get to go be intense," because nobody in a position of leadership wants to be chill, and what's worse than a chill boss? No, don't work for a chill boss. Don't be a chill boss. It's the most pejorative label I could give you. Chill boss or chill PM. Don't be chill. Chill doesn't accomplish shit. Be intense. Be good, be respectful, be intense. Don't be chill.

**Lenny Rachitsky** (01:10:09):
Again, this advice comes from where we started, which is this is what success looks like, because somebody that is less chill than you is going to find that crack and come after your market. Is that the gist?

**Matt MacInnis** (01:10:21):
For sure. I mean, again, if you're chill and you move the X axis down 10 or 20 points, the Y axis collapses. It doesn't just drop 10 or 20%. The Y axis collapses, and this is just kind of true in everything we do. So yeah, if you want to win, you should probably try.

**Lenny Rachitsky** (01:10:45):
This is what I always say to people trying to build lifestyle businesses. There's always this idea, "I'm going to build a side business, I'm going to make recurring income, it's going to be amazing," and in my experience, anytime there's a market or something shows up that's juicy and there's ways to make money, somebody's going to come at you and work harder, raise more money, and there's only so long you can maintain that.

**Matt MacInnis** (01:11:04):
Well, man, there's a whole other podcast episode on the concept of leverage. If you sell your time, you've only got 24 hours a day to give, but if you can create a product that scales, the marginal cost of a unit of that product is zero like software, it's going to be competitive, man. Sell your time, it's not going to be super competitive, but achieve that level of leverage and it's a pretty efficient market.

**Lenny Rachitsky** (01:11:33):
To close out this thread of intensity and adding energy to everything, something I've heard about you is you're big on escalating. You're a big fan of escalating issues, and also you always tell people to never not give you negative feedback, to always share feedback to not hide anything. Tell me about those two.

**Matt MacInnis** (01:11:49):
Fundamentally, the most selfish thing you can do is withhold feedback from someone. Who are you optimizing for when you do that? What are you optimizing for when you think a thought that would help someone improve and you avoid giving it to them because it would make you uncomfortable? Well, you're optimizing for your own comfort and it's fundamentally selfish. It's the most selfish thing you could possibly do, is withhold feedback that would otherwise be useful to someone because you don't want to be uncomfortable. Well, that's not how high performance teams operate. So I demand feedback, and I give it, and it doesn't mean that when I give feedback, it's not open to being questioned or discussed. I mean, of course it is, but when I observe something, I try to say it. That's the feedback topic.

**Matt MacInnis** (01:12:35):
The part of this that has been interesting to me is that people withhold, escalate, the customers withhold. Customers don't want to escalate to me as an executive. Even the founders in whose businesses I've invested who use Rippling are reluctant to hit me up when something goes wrong because they don't want to bother me, but it's literally my job, literally my job to find things, problems, and make them better, and by virtue of making those specific things better, to iterate on the processes so that the system that builds the system can get better.

**Matt MacInnis** (01:13:09):
There's no greater gift to me as a product executive than receiving an escalation from a customer. We have an escalations team at Rippling, which sounds weird, but it's people who are just particularly skilled at pistol whipping other people in the company to get to real root causes, real root causes. Not like throw away the word root cause, like, "Oh, we fixed the data error and shut the ticket down." No, you went and you found the software that created the data error, and then you found the system that created the software that created the data error, and you solved all of that back to the top.

**Matt MacInnis** (01:13:44):
Escalation seems extremely good at that at Rippling. So we have sort of a dedicated little team that does this for us. Escalations are a gift, and it's like, if you're a listener right now on this podcast and you are a Rippling customer and you have shit that you think we should know, the fact that I might already know it is not a reason for you to withhold the gift of your feedback. So it's an attitude that I take to this every day. I've got a little cue of some stuff that I've... Minor things that are from the last couple of days from people who had some knits and issues that I'm just like, I've got them queued up on my to do list today and I'm going to take them to the product teams directly and be like, "I'd like to understand what happened here." Not in a negative way. I just think we'll all get better if we study this one, and so yeah, escalations are a gift, feedback like that is a gift, and nobody is ever inconveniencing me when they do it.

**Lenny Rachitsky** (01:14:32):
For people that are listening to this and feeling like, "Man, this is so stressful and intense and just like, I don't know if I want to work this way," give them a sense of just how successful Rippling is. I think a lot of people may not even have heard of Rippling. A lot of people are like, "Yeah, it's doing great." What are some things you can share that are public or not public that give people a sense of just how massive this business has become?

**Matt MacInnis** (01:14:53):
People look at Rippling from the outside, I think they think of us as payroll and HR or whatever, which is cool. It's a bit like saying Microsoft is a desktop operating system company or it's like every company starts from somewhere.

**Matt MacInnis** (01:15:00):
... system company or ... It's like every company starts from somewhere and grows out from there. We see ourselves as building the most successful business software platform in history. In fact, that's the mission statement of our product organization under the umbrella of the mission statement of the company, which is to free smart people to work on hard problems. And when you translate that from where we are today to where we think we're taking things, it's like we really do believe that the core of every workflow and everything that a company has to do, be it AI or manual traditional GUI based software, is who's doing stuff, who owns it, who's accountable? And so the people record is a really important component of that. You can argue that the customer record's also very important. And of course some big businesses have been built on that primitive as well, but we think the people primitive is actually much more important.

**Matt MacInnis** (01:15:43):
And that the only thing that hasn't been done here is somebody hasn't been ambitious enough to build a good business on top of that primitive. Workday is terrible software. Everybody agrees on that. I think Workday agrees on that. Good luck to them. But they have failed actually, despite their success, to build a really broad general purpose software platform for business software on the foundation of the people primitive. So we're going to do that.

**Matt MacInnis** (01:16:03):
And we're successful because we deliver on that promise at the scale we're at today. The fact that you can do ... and this is not a sales pitch or sort of like an advertisement for Rippling. I just think it's important to sort of contemplate that when you bring together a bunch of disparate business processes into one system on a common business data graph, an object graph, a data lake with a consistent interface, you can do some pretty magical things.

**Matt MacInnis** (01:16:27):
So we do payroll, we do HCM, we do IT, we do spend. We are about to launch a new product in the category of business intelligence and data management. And there's a whole bunch of other stuff coming beyond that. And then you layer in AI on top of this, where we alone, where we alone have all of your business data organized into this nice, neat package with a beautiful semantic layer on top of it. The AI can work magic. And we have shipped nothing, nothing yet in this category that I would say gets anywhere near what we're going to show next year. And it is going to set the standard. It is going to be the most ... The back flips that the AI is doing reading and writing data for the user just on our internal use cases at Rippling is jaw dropping. So I'm super excited about the tailwind this is going to create for us.

**Matt MacInnis** (01:17:20):
You ask about what can I share about the success of the company? What I can say, there are tens of thousands of companies that now run on Rippling. We're less than 1% of the market. And the market cap at 16 billion, I think now undervalues where we are from a revenue perspective by a long shot. There is just so much upside to do what we're doing. And SaaS might be dead-ish in terms of point solutions, but long live SaaS in terms of what we're building.

**Lenny Rachitsky** (01:17:48):
Let me follow that thread in AI. There's been a lot of talk about AI is going to replace SaaS, as you maybe just said.

**Matt MacInnis** (01:17:55):
Yeah. People are going to vibe code their way to their payroll system, which I ... good luck to the employees of those companies.

**Lenny Rachitsky** (01:18:01):
And so what I'm hearing here is that you actually do believe a lot of SaaS companies that are selling individual solutions are in big trouble. The answer implied here is this kind of compound startup idea of you need to do a lot of things for people for them to continue to pay for your software. Is that the gist?

**Matt MacInnis** (01:18:18):
No. I think the gist is ... there's a really good quote. I forget who it's attributable to, but it's, "There's two ways to make money in software, bundling and unbundling." And you just got to get the timing right. So this is a period of bundling. So here's the problem; point solutions don't have enough data in the age of AI to be useful. You got to be able to provide the AI with a lot of context about a lot of data so it can do things. It can do joins. It can do correlations.

**Matt MacInnis** (01:18:49):
And so if you're a point solution, you're in hard water because you've got to now rely on data from other sources. You've got to integrate to third party systems. And when you integrate to a third party system, even the best ones are still sort of drinking their data through a straw, which is a real problem. I mean, the biggest HCM software company you can think of integrates to the other biggest payroll software company you can think of through flat files via SFTP. What are they going to do? What are they going to do? It's just never going to work. It's just no way. And so I literally have no idea what they're going to do. They're just not going to build AI software, I guess. Point SaaS is sort of in a rough spot, especially when you cleave two really important systems apart and say they have to integrate. It's very, very hard.

**Matt MacInnis** (01:19:39):
The other thing that I would say about the world of, even just like not SaaS, but AI software is that point solutions in the AI world are also in a rough spot for the same reason. It's like if you're selling the shovels like OpenAI and Google with Gemini, you can make money. And if you own the mine, like Rippling, with the data, you can make money. If you're somewhere in the middle, building AI software that then tries to use the shovels from the shovel provider, but then also needs to rent out the mine, or get the ore out of somebody else's mind and start refining it, you're in a very difficult place from an economic standpoint. Because you're not going to be permitted by either of those parties to build a big business on their backs.

**Matt MacInnis** (01:20:25):
That's just not how it's going to work. They're going to demand value capture that crushes your unit economics. So I look at the landscape of AI companies that I've seen and I think you have to have a really durable, interesting insight that gives you a shot at viable unit economics to be an interesting business. And that is going to kill off 80 or 90% of the stuff that I see right now as standalone AI businesses.

**Lenny Rachitsky** (01:20:53):
So what I'm hearing here ... I love that you correct me when I get these things wrong, and that's exactly what I want. What I'm hearing is it's less about how difficult it is to build the SaaS product. It's about, do you have first party data that allows you to build an incredible AI product on top of what you've got?

**Matt MacInnis** (01:21:09):
Yep. Because look, SaaS software is a bit flipping. All SaaS software applications are bit flippers. It's an interface changing-

**Lenny Rachitsky** (01:21:09):
Your database?

**Matt MacInnis** (01:21:18):
Yeah. Changing values in your database, that's what it does. It's a really hard problem. One of Rippling's superpowers is we're a coin sorter. You dump $20,000 for an employee in the top of the coin sorter, and it figures out what goes to the government, what goes to health insurance, what goes to your 401k, what goes to you. And it has to move all that money very, very reliably and seamlessly, very challenging software to build and manage. What's it doing? Even that is just flipping bits in a database. And so AI is a new way to flip bits. AI is just a new way to flip bits. Hopefully a way that abstracts us a little bit further from having to think because our future is Wally. It's going to be great.

**Lenny Rachitsky** (01:21:57):
Speaking of Wally, actually, I have this Matic Robot. You have one of these?

**Matt MacInnis** (01:22:04):
Uh-huh.

**Lenny Rachitsky** (01:22:05):
No. It's like a self-driving car robot, basically, a self-driving car. People built this robot that cleans your house. Maybe I didn't mention that. It's like a house cleaning robot that just goes around.

**Matt MacInnis** (01:22:15):
Oh, okay.

**Lenny Rachitsky** (01:22:15):
It's like a Roomba. You should have one. They're expensive, but incredibly cool. And actually in Wally, there's a scene where they actually have basically what these things look like. So it's not so far-fetched for that movie.

**Matt MacInnis** (01:22:26):
I definitely was actually quite prescient, perhaps with the exception of the gravity defying day beds.

**Lenny Rachitsky** (01:22:33):
Yeah. I don't know, but that's not good news. You've seen that movie. Oh, man. So on this AI stuff, which I'm hearing is we're going to see a lot more consolidation where these point solution companies realize they need this data. This is existential, and they're going to combine and merge and bundle, as you described?

**Matt MacInnis** (01:22:51):
It's possible.

**Lenny Rachitsky** (01:22:52):
Or for now.

**Matt MacInnis** (01:22:52):
I am not an investigator on this stuff. I think there's some really interesting investors out there who I think are thinking quite deeply about this. And in particular, the conviction, which is like Mike Fornell and Sarah Guo. I think those are two investors who are hyper focused on AI. And when they made the decision to take that approach, at the time I thought that's kind of narrow. Now I'm like, no, no, no, that was the right move. And it just means that they have a very deep, deep, deep set of hypotheses that they've formed over the course of seeing every AI deal in the valley. And there are better people to ask this question of than I am. And I think if you're an entrepreneur, I recommend them to you because I think they're really smart.

**Lenny Rachitsky** (01:23:35):
Awesome. I love those guys. Also, Sarah and Elad have a podcast called No Priors that I'd also recommend. We'll point it to you in the show notes. So on this AI note, I guess is there anything else that you think would be really helpful for founders that are working in this space building an AI startup to hear they think you were seeing of like, here's what you need to do to win in an AI company?

**Matt MacInnis** (01:23:58):
So much. I actually think that if I were to give you an answer to this question right now, it would be bullshit. Yeah. I don't have anything ... Back to my point earlier, I don't have enough relevant experience in the abstract to dole out on a podcast on that topic, but I wish them luck.

**Lenny Rachitsky** (01:24:17):
I love that. Circling back to your advice, don't ask for advice, ask for a relevant experience. And I love that that's where your mind immediately went. I'm going to take us to AI Quarter, which is a recurring segment on this podcast. And the question is just what's one way you've found AI to be useful in your work day-to-day? Is there something that you found it unlocked in how you work?

**Matt MacInnis** (01:24:37):
I mean, it's not a super exciting thing, but I'll say where I use ... So one of the most important functions that I perform as an executive is the synthesis of ideas, and the ability to communicate those ideas very clearly to people. So when I talk about the product quality list and the pickle, as something that we've come up with internally, I do turn to AI, ChatGPT and Gemini, where I take a really, let's say, angular view of some topic and I give ... really, I write the essay for the AI and I'm like, "Look, this is the crisp idea I want to communicate. Help me come up with pithy ways to articulate these things." And 80% of what it outputs is trash. It's just sort of middle of the road, average, low alpha junk.

**Matt MacInnis** (01:25:24):
But it is a thought partner, a non-judgemental thought partner where in 20% of the stuff it comes out with, I'm like, yeah, it's pretty good. That's a new word I didn't think of. That does kind of hit the nail on the head for this concept. And so if I believe that my job is to sort of bring brains along on the journey for some sort of change that I'm trying to bring about, then the most important tool is language. And I do find that the ChatGPT and Gemini do a great job of helping me refine how to articulate the concepts that I want to articulate. They are not useful in coming up with the ideas themselves.

**Lenny Rachitsky** (01:25:59):
That's an awesome tip. I don't know if you've played with Claude much, but I actually find Claude is better at writing and words and language.

**Matt MacInnis** (01:26:05):
I have not spent a lot of time with Claude. I have used it, but by virtue of this conversation, I'll probably go give it a go.

**Lenny Rachitsky** (01:26:12):
Right. Yeah. They're all great, but there's something about Claude that is just at writing specifically is much better, but they're all getting better all the time. There's always something new.

**Lenny Rachitsky** (01:26:23):
Matt, we've covered so much ground. We've touched on everything I was hoping to touch on. Before we get to our very exciting lightning round, is there anything else that you were hoping to share or anything else you wanted to touch on or leave listeners with?

**Matt MacInnis** (01:26:34):
Yeah. We've spent a lot of time talking about intensity and the grind, and the need to just always be operating at the 99th percentile. And I think if you listen to that in a vacuum, it's very easy to believe that that intensity is soul crushing, that it's a negative, that it's maybe not something that you want. And I think there's a backstop for me that I didn't talk about today that I think is important to share, which is that life is amazing. That the fact that we all exist on this blue marble drifting through space and time, that we are some weird instantiation of consciousness, each of us, and that you're here for such a short period of time whittling your stick, doing something, that if you remember how insignificant we are and all of this is, it brings this levity to what we do and to the work we put into building this.

**Matt MacInnis** (01:27:47):
Because Silicon Valley in 2025 is Florence and the Renaissance. It's crazy. The amount of creativity and insane invention and progress that's happening for our species right now in this place is absolutely unparalleled in all human history. You've got to zoom out and appreciate that magic. And so then you turn around and you're like, fuck, I've got to work on Friday night, right? I've got to go give it my all. I've got to go compete in the arena of business.

**Matt MacInnis** (01:28:22):
You have to never forget that number one, none of this matters. And number two, it is an absolutely beautiful and amazing phenomenon that we get to be alive doing this right now. So play the sport, play it with everything you've got, but never forget that it's just a sport and that none of it matters. I think it's super important as a counterpoint to the intensity that we talked about.

**Lenny Rachitsky** (01:28:48):
That is beautiful.

**Matt MacInnis** (01:28:50):
I think about Pale Blue Dot, Carl Sagan's whole thing. Just a stunning photograph that literally changed the way I think about who I am as a person when I saw it. Yeah.

**Lenny Rachitsky** (01:29:02):
Well, going in a completely different direction, with that, Matt, we reached our very exciting lighting round.

**Matt MacInnis** (01:29:07):
Okay.

**Lenny Rachitsky** (01:29:07):
All right, five questions for you. Are you ready?

**Matt MacInnis** (01:29:09):
Okay.

**Lenny Rachitsky** (01:29:11):
Here we go. What are two or three books that you find yourself recommending most to other people?

**Matt MacInnis** (01:29:16):
Okay. Two or three books. You give me a heads-up on this. So one book is Conscious Business. I don't have Conscious Business in front of me because it's actually at the office because we have Conscious Business Reading Club at Rippling. And every member of my current, my product leadership team is going through this right now, Conscious Business, Fred Kofman. It's been used in many businesses, LinkedIn most notably, as a framework for thinking about ... effectively, it's a user manual for human beings. So if you are a leader, a manager, an executive, whatever, particularly younger people in their 20s and 30s who are just sort of getting the hang of being a CEO or a product leader for the first time, this book is absolute fucking gold. It was recommended to me by Bryan Schreier at Sequoia when he was on my board at my previous company. I took way too long to take him up on the advice, wished I had read it sooner. Highly recommend Conscious Business. Changes your life.

**Matt MacInnis** (01:30:01):
Number two, Thinking in Systems, Donna Meadows. I always mispronounce her name, Donella Meadows. She died partway through writing this manuscript. Her fellow professors picked it up and finished it for her. It is the best generic framework for thinking about how systems work. You will extrapolate from this book to every aspect of your life after you read it.

**Matt MacInnis** (01:30:23):
And then the third is classic 1960s, The Effective Executive. It's an anachronism. It uses weird pronouns for the secretary and the executive. I'll let you guess which ones. But the book itself is so chock-full of simple enduring advice on how to be effective at leading teams. And the good shit is the stuff that's been in print for 70 years, and that's one of them.

**Lenny Rachitsky** (01:30:48):
Beautiful. I love the first one is you don't have it because you're using it with your team constantly. You mentioned at one point before we started recording, you have eight copies of that book that you just give out to everyone.

**Matt MacInnis** (01:30:57):
Yeah, and we handed out like candy.

**Lenny Rachitsky** (01:30:59):
Okay. Is there a favorite recent movie or TV show you've really enjoyed?

**Matt MacInnis** (01:31:03):
Yeah. So I'm a little embarrassed by this answer, but I'm going to be honest.

**Lenny Rachitsky** (01:31:07):
Please.

**Matt MacInnis** (01:31:08):
There's a new series called Heated Rivalry and it's about ... I'm Canadian and I'm gay. So it's about two hockey players. In Canada, rivals between the Bruins and the Maple Leafs, although they don't use those names, who are just heated rivals on the ice. And it's a huge thing that the world is watching, but actually they're secretly in love with each other and they start hooking up. And it's been labeled by the media as smutty but delightful. And I would say that's accurate. So it might not be for everybody, but it is a smash hit right now. It's on HBO Max and Crave, and it's only six episodes. But like I said, a little embarrassing because it's a little chintzy, but it's a lot of fun. It's also really fun to see gay people represented in the media as though they're normal, and it's fun.

**Lenny Rachitsky** (01:31:57):
And soon to be on Netflix with the acquisition if that goes through.

**Matt MacInnis** (01:32:00):
Oh, yeah.

**Lenny Rachitsky** (01:32:00):
It's crazy. Amazing. Okay. Is there a favorite product you recently discovered that you really love?

**Matt MacInnis** (01:32:06):
My Fellow coffee maker. I love my Fellow coffee maker. It's got an interface that lets you set light, medium, dark roast. It changes the temperature. It blooms the coffee. It tells you how many grams of coffee to put into the basket, slick interface, high quality coffee. It's definitely awesome. And so I have Ashley have one in the office and one at home and one in the garage.

**Lenny Rachitsky** (01:32:30):
Wow. That is a favorite product. You have three of them in the same cool space.

**Matt MacInnis** (01:32:30):
Yeah.

**Lenny Rachitsky** (01:32:35):
Oh no, in the office. Okay.

**Matt MacInnis** (01:32:35):
Fellow is also a Rippling customer and that's a nice side effect when we get to ...

**Lenny Rachitsky** (01:32:39):
Have they ever escalated anything to you?

**Matt MacInnis** (01:32:42):
No. If you're listening and you're from Fellow, I want to hear all your gripes.

**Lenny Rachitsky** (01:32:46):
Perfect. Two more questions. Do you have a favorite life motto that you find yourself coming back to often in work or in life?

**Matt MacInnis** (01:32:51):
It's a dark one, and I'll share this one sort of partially for the humor of it, but it's actually sometimes useful immediately. At least it's sort of a moment of smiling when it happens. The motto comes from my dad who said, "Matt, nothing's ever so bad in life that it couldn't get worse." And it's like we were going through some shit yesterday at work and we were like, "Fuck, now that happened." And I looked at the CTO and I'm like, "Dude, nothing's ever so bad that it couldn't get worse." And we had a good laugh and continued to brace for whatever might come next. So not exactly uplifting, but fun to use.

**Lenny Rachitsky** (01:33:26):
No, it is uplifting. I'm an optimist and I find myself thinking that often with my wife just like, "It could be worse. It could be worse than this. " Definitely. It's actually [inaudible 01:33:36].

**Matt MacInnis** (01:33:36):
And it might get there.

**Lenny Rachitsky** (01:33:38):
So let's enjoy this less worse version. Final question. Something you shared with me is that you were a DJ when you were a younger person. Can you just give us a little DJ voice to give people a sense of your skills?

**Matt MacInnis** (01:33:52):
Well, so first of all, it's not DJ, Lenny. It's radio personality. And yeah, I used to do the greatest hits of all time with hits from the '60s, the '70s, the '80s, and a little bit of the '90s, 101.5 The Hawk. Yeah. You can turn it on. It's very inauthentic, but it sounds good on the radio. It's cool. I did it when I was in high school. I ended up doing the midday segment right before I went to college. And what a gift. What a huge gift in my most formative years to have developed an ability to be in front of a microphone comfortably, because here we are.

**Lenny Rachitsky** (01:34:28):
I love for people that weren't watching this YouTube, you lean really close to the mic to get that radio personality voice.

**Matt MacInnis** (01:34:34):
Yeah, you got to be able to hear the saliva in the mouth.

**Lenny Rachitsky** (01:34:38):
Incredible, and it's like the same person talking. If you're not watching this, you're like, "Where did that guy come from?" That was great. You nailed it. Matt, this was incredible. I really appreciate being here. I really appreciate you sharing all this advice that I have not heard other people share. Two final questions. Is there something you want to plug, point people to, and how can listeners be useful to you?

**Matt MacInnis** (01:34:55):
Look, my life is rippling. I point people there, and this is my life's work. It's going to be a banger, so stay tuned. And the way that you can help me is that if you're a customer, you got to tell me when you have problems, because that's how we get better.

**Lenny Rachitsky** (01:35:10):
What's the way to get to you? Is there any place you want to point people to?

**Matt MacInnis** (01:35:13):
DM me on Twitter, is easy @stanine. You can email me my last name at rippling.com, and I'll go that far without giving out my phone number. How's that? Perfect.

**Lenny Rachitsky** (01:35:25):
A perfect boundary.

**Matt MacInnis** (01:35:26):
Yeah.

**Lenny Rachitsky** (01:35:27):
Matt, thank you so much for being here.

**Matt MacInnis** (01:35:29):
It's my pleasure. Thank you for having me, Lenny, and congrats on all the success with this podcast. It's been great.

**Lenny Rachitsky** (01:35:34):
Same to you. It's always a good sign at the end of a conversation when you're like, oh, I got to get me some of that stock and I got to get into that Rippling.

**Matt MacInnis** (01:35:41):
It's a good buy.

**Lenny Rachitsky** (01:35:41):
A great job.

**Matt MacInnis** (01:35:42):
Recommended by-

**Lenny Rachitsky** (01:35:43):
15 billion.

**Matt MacInnis** (01:35:44):
Yeah, but you're [inaudible 01:35:46]

**Lenny Rachitsky** (01:35:45):
You're hard to get [inaudible 01:35:46] All right, man. Thank you so much.

**Matt MacInnis** (01:35:50):
Thanks.

**Lenny Rachitsky** (01:35:52):
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

---


</div>