Leader Spotlight: The right and wrong approach to product-market fit, with Charlie Kaplan

---

Charlie Kaplan is VP of Product at Audiomack, a youth-driven, artist-first music streaming platform. He started his career at NPR and then became an editor at The Webby Awards. From there, Charlie joined Cymbal as Head of Growth before getting promoted to CEO. In his seven years at Audiomack, he spent his initial tenure as Director of Product Management and Growth before becoming VP of Product.

In our conversation, Charlie talks about assessing product-market fit and shares examples from his work of successful and unsuccessful product adoption. He emphasizes the importance of experimentation and studying churn to improve the product experience. Charlie also discusses times when testing and failure revealed important truths about his users.

---

‘No amount of thinking can replace doing’

Throughout your career, you’ve built and grown products from zero to scale. Based on your experience, what are the earliest signals that a product does or does not have product-market fit?

This is a seminal question in product and entrepreneurship. While the term product-market fit might not be familiar to people outside of product or technology, the core idea resonates with everyone: Do you need this or not? Sometimes, that need is actually a want that’s so strong that it turns into a need.

I categorize my career into two big chapters. The first one was with a company that did not have product-market fit, called Cymbal, and the other, Audiomack, greatly does. Cymbal was a music social network that I was the CEO of from 2016 through 2018. It was a lot like Instagram, but instead of photos and videos, it was songs drawn from Spotify, Apple Music, and SoundCloud. Now I’m at Audiomack, which is the biggest streaming music service in Sub-Saharan Africa. We currently have 41 million monthly active users.

The key differentiator between these two companies is that with Cymbal, people loved the idea more than they liked actually using it. We’d describe it, and they were all in, which is an early hint about what product-market fit really is (and isn’t). However, product-market fit is about the experience of using something, not how much you like the idea. And the key part of building a social network is, in my opinion, that you can’t start with just a social network anymore.

A social network’s value depends on the number of connections you have there. All the content in your feed has to come from people you follow, but if no one else you know or could be interested in is on it, the value of the idea instantly vaporizes. You need a way to sustain users while their friends aren’t yet there — when they’re in what we call “single-player mode.”

Audiomack, on the other hand, is an on-demand streaming service. You search for a song you like, find it, and play it. The need that a user might have expected going into Cymbal couldn’t always be satisfied because it depended on who else was on the platform, how relevant they were to you, and how well you were connected to them. By contrast, on Audiomack, as long as we have the music you’re looking for, we’re off to a good start.

As much as you can plan for something, it’s hard to gauge how it will do until it’s out there in the real world. How do you think about that as an element of predicting product-market fit?

The longer I’ve been working in entrepreneurship and product, the more I believe that you can only really rely on what you can measure and reproduce. Eric Ries’ book The Lean Startup talks about the scientific method as it applies to entrepreneurs. De facto, the hypothesis has to be disprovable to be valid and for you to then iterate upon your findings. Each revelation should lead you to the next observation. And one of the persistent issues in science — especially social science — is that there can be a big difference between an observed result and a reported result.

Putting something in front of people and seeing what they do, as opposed to surveying them and asking what they want, is much closer to the bone of what’s actually going to work in the real world. This led me to a radicalizing observation I had about myself as a product person and entrepreneur, which is that no amount of thinking or asking can replace doing. Getting out there and seeing how people genuinely respond to it — there’s no substitute for that.

Relying on experimentation rather than assumptions

People often fall into the trap of making assumptions, especially when they’re experienced in that area. How do you view the benefits vs. pitfalls of being a domain expert for the product you work on?

One of the greatest advantages you can have as an entrepreneur is personal experience with the subject matter, but it can also put you in a bubble. If you are building a product for people like you and where you can identify with their problems, you can skip a lot of the user research steps. It puts you in a position where you are in the groove with the user, and they’re not foreign or exotic to you — you can understand them because they represent your experience. But that also creates risk.

For example, I have worked in music technology for the last decade, but in addition to that, I’m an artist; you can find my own records online. I identify with Audiomack’s user story and can make educated guesses about what will work. That carries a lot of inherent risk as well as advantages. My vantage point creates blind spots. If you ask me what the user story is for an artist getting their music to fans, I can tell you every step in that journey and what it takes to make the music, but just from my perspective.

Back to the fact that Audiomack is the biggest streaming music service in Sub-Saharan Africa: That is a very different population and environment from where I grew up. Naming and knowing those differences between you and your users is critical because you can never allow your intuition to override what you see in these tests. Throughout my time at Audiomack, I’ve seen tons of things that make our users different from me as an artist, and I have to have a sense of perspective. This is why it’s so important to continually rely on experimentation and observation: it reveals the world as it is, not as you imagine it to be.

Have you ever gotten feedback from your user base that was surprising to you and challenged your assumptions?

Absolutely. The way that I make music is very different from others. I do it in a more conventional way — I write songs on a physical instrument, demo it on my phone, record voice memos, and play it into microphones with a band at a studio. There is a whole world of people who make music strictly on their phone, oftentimes with no instrument whatsoever.

One of the breakthrough experiences for me was in 2019, when Audiomack’s only upload function was on our website. My boss, Dave Macli, and I put our heads together and we said, “Maybe it’s time for us to have a mobile app where artists can not only see their music’s stats, but upload music directly from their phones.” Personally, that is not something that I ever do. In my world, when I finish a piece of music, it’s sent to me in an email, and I upload it to a laptop. Uploading from my phone felt like such a lightweight relationship to me. That was a total perceptual misunderstanding of how people make music. It’s no less serious than the way that I do it.

So, when we launched the creator app, it transformed the impact of user-generated content on Audiomack. Now, 80 to 90 percent of user-uploaded music on Audiomack comes from mobile. That’s incredible to me because it is so different from my relationship to music. I’m grateful that we were able to stay open to this idea and look beyond ourselves, because otherwise, we would have missed out on such a huge opportunity. We also wouldn’t have gotten such meaningful lessons about who our users are, which continue to inform our strategy every day.

How exactly did you get the insights from your audience that this is the kind of app and experience that they were looking for?

We had to begin with a hypothesis. Some of that was very practical from a backend perspective. We already had the upload function built, which limited the cost of the effort. We were seeing a lot of it happening on the mobile web, and that experience is comparatively flawed to an app because if you leave the page or you close the tab, it’s gone. On top of that, we can’t come back and communicate with you directly through a mobile web page in the way that we can with an app push notification.

We started to see that there was an intense body of user behavior on the mobile web that we could augment if we made a dedicated app for the device. It began there. We asked, “If we are seeing this amount of user activity on mobile web, does that mean that users only access us through a mobile device? If so, can upload, engagement, and retention be augmented? We launched, and the results were really strong. It really changed how we thought about artists. Building the creator app allowed us to create features that bring the artist and listener together.

Studying churn and avoiding hype cycles

As someone who has a lot of experience in the space, how do you help younger team members make sure they’re not mistaking hype for actual product-market fit?

The best way to do this is to study churn, in my opinion. Do you exist in a self-sustaining way? If you’re churning users but slowly growing, that’s good news. Or, as the old metaphor goes, is there a hole in the bucket? You’ll see major advantages if you can identify, with users themselves, why they’re churning. It does not happen randomly.

Chasing hype cycles is definitely something that I have seen in my own career. When Cymbal first launched, we were on the homepage of Forbes. Suddenly, tens of thousands of people were signing up for Cymbal. What did that tell us other than people really liked the idea of it? Well, they’d show up and say, “No one I know is here,” and they would leave. It was an incredibly instructive thing. That’s a hype cycle. You can make the most of it if you have product-market fit, but it is not a substitute for actually building something that people want. And the only way that you can tell that people want something is if they keep coming back.

On the other hand, at Audiomack, we launched a messaging feature where artists could message their fans. We accidentally launched it on the web to everyone instead of a cohort of users. Messages exploded. We didn’t tell users about it, we just put a button up by accident. It was released, and we didn’t mean to merge it, and we thought it was a terrible mistake. Actually, it was great news and got fantastic responses. We messed up, but we learned something incredibly valuable.

On the flip side, when something underperforms, how do you handle those situations? How do you decide when it’s worth iterating or just letting it go?

It’s incredibly important to do a deeper read of your failures than of your successes. Not just because it forces you to revise your thesis, but because there’s often something special in looking at what didn’t work.

A good example of this is when we launched a feature called Support three years ago. It’s still in the app, and it’s the ability for a listener to directly pay an artist. In exchange for sending the artist a certain amount of money, they might get a badge on their profile that distinguishes them as a supporter. The artist can target messages specifically to supporters and give them specific benefits.

When we launched, it was underwhelming. Money came in, but it really made us review what our core assumptions about user behavior were. We thought that there was some degree of altruism or super fandom that could be expressed through this type of feature, but it didn’t materialize in the volume we wanted. Every now and again, there would be this weird spike of supporters’ activity. People would come back, and all of a sudden, they’d be supporting a lot. We didn’t know why it was happening in spikes like that.

After looking at it, we realized people were supporting themselves. There is a section of our Discover page that shows the most supported projects, so people would pay to get on the homepage through that. It was like an ad campaign, and that eventually informed our Boost feature. Boost is essentially an ad unit in the second spot after a queue finishes that artists can bid on. It targets their music toward receptive listeners.

If we had gotten crickets, we might’ve just said, “We’re wrong about this.” We could have tinkered with it, but eventually backed off of it. In this case, there was smoke, we just didn’t know where the fire was. And when we found the fire, we built an entirely new feature around it. It was huge. This is why it’s so important when you roll something out to make it measurable and give it its due time in examining it.

Surfacing truths about users

You mentioned the relationship between listeners and artists. How do you balance those two perspectives when you are creating your strategy?

I find that when a marketplace fails, it’s usually because the two parties don’t have a ton to do with each other. At Cymbal, we couldn’t create strong enough relationships between the users who were there. Users were incompatibly different because what made those social interactions valuable to them was absent and hard to establish. We experimented with a ton of different ways, but it didn’t stick.

On a streaming service, however, an artist and the listener are two sides of the same coin in terms of what they want. The listener wants to find music they want to listen to, and the artist wants people to listen to their music, and so they meet in a place where, if they’re both matched well, they’re super happy. The ultimate job of the streaming service is to create matches and connect those parties.

Lastly, how do you balance the pressure to move quickly with the desire to take your time to test a hypothesis?

I would always take hard empiricism over speed, because I’ve been in situations where I was running very fast, but in the wrong direction. The only way that you can know if you’re going the wrong way is through experimentation.

I’m definitely a maximalist when it comes to measuring things, and I think that there is a granularity. Sometimes, it reveals truths about your users that a competitor of yours can only get through running the same tests on the same users. And every time you run a test, you refine and sharpen the image of your users. If you’re thoughtful about it, you can build things better for them.

My philosophy is to take things one day at a time, and if each one of those days, you’re fundamentally testing something and surfacing a truth about your user, you save time in the future.