ChatGPT has come on the scene in a big way over the past several months. It achieved rocket growth of 100 million users only a few months after its launch. It is the fastest-growing application of all time and is causing waves of disruption across multiple industries in a way that we just haven’t seen before.
I’ve been particularly intrigued by how it can help us in product. Can it transform the way that we work? I believe it is an assistant that will supercharge us. Today, it is still very young and limited in what it can do, but as you’ll see, the potential is huge.
In this article, we’ll take a quick look at ChatGPT and dive into various aspects of product management where we can start to leverage it. We’ll look at some very serious ethical concerns around ChatGPT and also explore some innovative tools in the generative AI space that can help us with our jobs.
ChatGPT is a generative AI chatbot from OpenAI that was trained on an extremely large set of data. It allows you to ask questions across a broad set of topics and instantly get responses. ChatGPT is conversational in nature, allowing you to chat back and forth within the tool. It is generative in the sense that it produces content, something that is not common for all AI tools.
You can use ChatGPT for a wide variety of things from writing articles and poems, explaining different topics, creating written proposals or contracts, and even generating and debugging code.
To try it out, you can sign up for either the free or paid version at chat.openai.com.
You can immediately start using it once you log in. Below, you can see I’ve asked ChatGPT to describe itself. It is truly remarkable how fast the answers come:
As I mentioned, I’ve been particularly curious about how product professionals can leverage ChatGPT in our day-to-day lives. I did some experimenting with the tool over the past few months and have been testing how it can help me with different types of tasks. I’ll go over how ChatGPT works in helping with the following categories of tasks:
As product managers, we’re always learning new topics and approaches — our craft is constantly evolving. I have found that ChatGPT has a large amount of knowledge of techniques. As an example, I asked it to describe the RICE model, and here is what it came back with:
This answer is very good and clearly describes the RICE model. How about the now/next/later roadmap?
This answer here is not so good. It is misleading and only partially correct. The now/next/later roadmap is not about tasks and projects, it is a thematic roadmap focused on customer problems and initiatives. It does indeed have now, next, and later sections, so at least that part is correct.
The very unsettling part here is that ChatGPT sounds very confident in its response, although it is wrong. That could be very damaging. I have had mixed results, quite honestly, when using ChatGPT in this fashion.
Sometimes it gives excellent answers and sometimes it is way off. Your mileage definitely may vary here. In the future, I imagine we’ll have more crowdsourced options to train ChatGPT when it gives us incorrect answers.
One of the places where ChatGPT will be able to help us is in gathering data. The data can help us make decisions on where to invest, hopefully saving time and resources. Its huge volume of data allows ChatGPT to extrapolate and assimilate across a large breadth of information.
To put it to the test, imagine you are a PM for a music player, similar to Apple Music and Spotify. You have received several requests for a new sharing feature that allows users to enable their friends to choose the music they will listen to. You need to weigh doing this feature against other possible work. Let’s see if ChatGPT can assist:
This is good data to help inform our decision. This information, as well as the awareness of other music players’ features, are both helpful. One area it lacks, though, is that it does not get into enough specifics about the feature we described. Fortunately, we can add additional context to get a more precise answer:
This is better! Now, ChatGPT is giving a more relevant answer to the specific feature we are describing and raising concerns we might not have considered.
My advice when using ChatGPT is if you don’t get the most helpful answer at first, consider adding context as we did here. In some cases, you may want to rephrase the prompt entirely. The ability to rephrase or add additional context is a remarkable aspect of this technology.
In the previous section, we used ChatGPT to gather data to help us determine if we should invest in building that music-sharing feature. Assuming we decide to develop a significant feature like this, it will have various components and ChatGPT can also help us determine what they are:
The list here looks pretty good, though some items seem unnecessary.
For example “Notifications and Updates” is somewhat redundant with “Music selection process.” Also, voting and ranking sound a little bizarre. However, the rest of the items are reasonable. We’ll probably want to wordsmith them, but it gives us something we can use as a starting point.
Seeing this list, this looks like a lot. Maybe this is a greater scope than we had first thought? Do we need to ship all of these for our MVP? Let’s see if ChatGPT can help us prioritize:
Very good. Now, we can see which components are the most important. Having the results in a table is also helpful so we can copy them into tools like Google Sheets.
Wait a minute, something looks wrong. Do you see it? Why does “Invitation System” have an importance of “Medium?” How can the users of this new feature use it unless they have a way to be invited? Let’s ask ChatGPT to explain its answers:
The response here shows us that ChatGPT’s replies may sometimes defy common sense. Although it is technically correct that being able to select music is more important than the invitation system, practically, that holds little value as the feature is not usable without being able to invite users. This is another illustration of why you can’t just blindly accept ChatGPT responses. Still, the response it gave us was helpful and something we could use as part of our planning.
Once our new sharing feature ships, we’re going to need metrics and KPIs to track how it performs. Let’s ask ChatGPT:
This answer is marginally useful. You could basically apply these same metrics to almost any feature and the response does not go into much depth about metrics that are feature-specific. For example, tracking the average number of songs that are added, the average number of songs played, the total number of songs played, and the total minutes played would all be more specific metrics that would help understand its usage.
We’re often creating light mockups to share with our customers to get validation on features. Can ChatGPT help? It can’t render mockups inline (yet), but it can generate HTML/CSS code for a mockup that shows our new feature:
Copying this into a tool like Replt, we can see the mockup:
Pretty cool, right?
No, this won’t win any design contests, but it gives us a preview of where things can go as the technology improves. We can easily imagine something much richer in the future.
One way to help get buy-in for developing the feature is to create a launch post that we can socialize. We can even share this with customers for validation. Why not just ask ChatGPT?
This is not bad. Granted, the content here is somewhat boilerplate and not the most exciting, but it does save time and give us something we can work with. We can take it and tweak it. I am sure there were many times in the past when I could have used this, especially for internal use.
ChatGPT has great potential. It also has not been without its pitfalls. Most importantly, some big ethical questions around its use are still up in the air. For one thing, as you’ve seen, sometimes its answers can be very wrong, and this could spread misinformation.
There have also been a host of articles raising concerns around racism and bias both in the kinds of answers ChatGPT gives and around how it was trained. There are also issues around access, which can result in increasing the digital divide.
Without care and intention, there is a very real fear of harm that this technology can cause.
Beyond ChatGPT, newer tools leveraging ChatGPT’s model have appeared and are taking things to the next level. Two that I’ve been impressed with — which I imagine becoming an important part of our toolbelt — are Kraftul and Galileo AI.
As PMs, we have feedback coming from all directions and we often spend tons of time wading through it to better understand our users. Often, this data sits and gets stale because we don’t have the time to mine it.
Kraftful uses ChatGPT’s model to greatly reduce this pain and takes care of this for us. You can connect it to different sources of user feedback, like Google Play or iOS app stores, Zendesk, and Intercom. Once the sources are connected to it, it mines the data and summarizes the top and most loved features, complaints, and competitors across all the sources.
It also provides a chat that allows you to ask free-form questions. I’ve been using it and I love the direction. It is going to make our lives a lot easier and help us be more attentive to our customer’s needs:
Earlier in the article, we looked at how we can use ChatGPT to generate very primitive mockups. I mentioned that hopefully in the future, there would be something richer. The future is now with Galileo AI.
As we can see below, you can literally tell it the kind of mockup you want and it will create it for you in Figma, and you can then edit it directly. Just imagine how powerful this will be to have at your fingertips!
As you’ve seen in this article, ChatGPT is a tool that has big potential to change the way we work in product. You’ve seen how generative AI can help us learn new approaches, do research, create content, strategize, create mockups, and more.
It’s not there YET, but it can already give us a big boost in the work we do. To get the benefit, we need to understand how to use it and be aware of its weaknesses. The ethical concerns mentioned are very troubling and hopefully, these will get addressed so that ChatGPT can provide the broadest benefit and not create more inequity.
I look forward to seeing the amazing things we can accomplish using this groundbreaking new technology.
LogRocket identifies friction points in the user experience so you can make informed decisions about product and design changes that must happen to hit your goals.
With LogRocket, you can understand the scope of the issues affecting your product and prioritize the changes that need to be made. LogRocket simplifies workflows by allowing Engineering, Product, UX, and Design teams to work from the same data as you, eliminating any confusion about what needs to be done.
Get your teams on the same page — try LogRocket today.
Deepika Manglani, VP of Product at the LA Times, talks about how she’s bringing the 140-year-old institution into the future.
Burnout often starts with good intentions. How product managers can stop being the bottleneck and lead with focus.
Should PMs iterate or reinvent? Learn when small updates work, when bold change is needed, and how Slack and Adobe chose the right path.
AI accuracy problems are often chunking problems. Learn how chunk size and structure impact cost, retrieval quality, and UX.
Learn more →
One Reply to "Can ChatGPT transform how we build products?"
Nice read! Thanks!!