Leader Spotlight: Making magic with the right SDLC model, with Trevor Riley

---

Trevor Riley is a product leader with extensive experience in the real estate and property management industries. He began his career at Vivint, a smart home company, where he worked in customer service, quality assurance, customer support, and workforce management. Trevor then transitioned to Entrata, where he climbed the ranks for over a decade to become SVP of Product. Most recently, he served as Head of Product at BiggerPockets, an online real estate investment resource and community.

In our conversation, Trevor talks about how adaptation is crucial to long-term success, and how you need to adapt your software development life cycle to fit the resources and dynamics of the company. He offers his “hot take” that the product owner function in a SaaS company actually decelerates growth and shares his preference for other product management models. Trevor also discusses the importance of acknowledging assumptions when linking a product team’s metrics with overall company success KPIs.

---

Calibrating scrum ceremonies and artifacts

To start, could you talk about ways that you’ve adapted the software development life cycle in terms of scaling a company to ensure that it’s supporting rapid growth?

I believe that adaptation is critical to success. In my experience, there’s no one-size-fits-all solution. You’ve got a set of outcomes that you want to achieve and a framework for various artifacts and ceremonies that you believe are going to work. That’s great, but the challenge is in the fine-tuning and calibration of those ceremonies and those artifacts — how you make adjustments to the software development life cycle (SDLC) to fit the specific dynamics of the business at the right time.

That process starts by asking questions that require you to look beyond the technology team itself. What is the mindset of the leadership team and CEO? What are their priorities? How can you adapt our SDLC based on their styles and expectations? You also need to look at your own team and ask about your strengths and weaknesses. Are they exceptionally technical or are they more strategic? How does the technology team fit with the rest of the organization?

Let the answers to those questions help you decide how you need to adapt your software development life cycle to fit the dynamics of the company.

For example, I worked in a company where a lot of unplanned work was constantly being thrown at us. We wanted to limit the amount of unplanned work that our team was getting, but, given the dynamics, we realized it was going to be easier for us to make a pivot and adapt our software development life cycle to fit the reality of the organization. We ended up moving to one-week sprints so if unplanned work came in we could simply include it in the following week’s sprint.

How the product owner function can decelerate growth

How do you define the role of a product owner within the SaaS environment, and, based on your experience, what do you feel are the pros and cons of having a dedicated product owner in that scenario?

To put it bluntly, I am not a fan of the product owner role. I’ve been a part of organizations that attempted to use it and others that didn’t. My perspective is that it was invented to fill a hole that existed because product managers and engineering managers couldn’t figure out how to work well together.

Sometimes, engineering managers want very specific, clear requirements and the product managers want to focus on discovery and strategy. The product owner role was invented to fill that gap. The problem is that it muddies accountability, and it also adds one more daisy to the daisy chain. This can create frustration with the engineering team not delivering exactly what the product manager had envisioned because it’s gone through two separate filters. You get these communication breakdowns.

The highest-performing teams that I’ve worked with have the engineering manager and product manager come together and share accountability for the outcomes of the entire team. They share some of the responsibilities that are traditionally on the product owner but do so even more efficiently. I’ve become a big believer in that model.

When you’re referring to the product owner, are you talking about that specific title or more how that role is structured within a company?

In my experience, it’s pretty inconsistent, and maybe that’s part of the problem. If you survey 10 different product leaders and ask them to define the product owner, you’re likely going to get different definitions. I had an experience where we believed we needed the business analyst role, but we weren’t quite sure how it was defined. The business analysts ended up being asked to perform a lot of the functions of the traditional product owner role; they essentially became that function.

They were very dissatisfied with their role. They didn’t understand what they were responsible for, didn’t have a lot of autonomy, and felt like they were just trying to interpret what the PMs wanted. In the end, we moved away from that model completely and instead ended up bringing on more product managers. We differentiated between an associate product manager, product manager, and senior product manager, and narrowed those scopes. That was a much better system for us — there was more accountability and job satisfaction across the board.

Some folks say that product management can be a lonely job in terms of community because, although they are part of cross-functional teams, each product manager is essentially on their own. Do you agree with that?

I think that’s an astute observation. It can be a bit of a lonely role, even though you’re constantly interfacing and collaborating with a lot of different members of the organization. I think of it as being the captain of a ship, but no one on the ship actually has to do anything that you say. That cross-functional component of the role is critical. People talk about product managers as being CEOs, but don’t have the organization set up in that way.

It’s important to have opportunities and spaces for product managers to collaborate and talk about what’s working and what’s not working with their respective delivery teams. In my last organization, it was really important that the product team had a chance to talk — even without me being there — to bounce ideas off each other, share stories, and offer support when there were frustrations.

An ideal 3-in-the-box model

Do you have a preference for a 2-in-a-box or 3-in-a-box model for product management and, if so, why?

When I talk about the two-in-the-box effect, it’s protecting the idea that the engineering manager and the product manager are both equally responsible for the same set of outcomes — even if there’s a little bit of differentiation between where the product and feature accountability lies. If you have engineering managers who are working with multiple product managers, or if you have product managers who are working with multiple engineering managers, you’re not going to get that two-in-the-box effect.

The three-in-the-box model is that same idea, just expanded to the product designer. In a perfect world, the three-in-the-box model is ideal. When you have an engineering manager, product manager, and product designer sharing equal accountability for the same outcomes and working with the same delivery teams, that’s where a lot of magic happens.

Organizationally, depending on where you’re at and how you’re scaling, it may not be feasible to always have that setup. Part of the reason is that you’re trying to figure out where the bottlenecks are. Sometimes, you can’t justify having as many designers as product managers because you don’t have a bottleneck in design or you might have a bottleneck in product management. You certainly have to adapt based on resources.

Do you feel that the 3-in-the-box model best supports outcomes vs. outputs?

I think so because as soon as you start building fences or silos, it almost becomes impossible for teams to be focused on outcomes. Instead, it becomes an issue of accountability. Design is doing design work and then they’re flipping that work over to product. Product’s doing product work and they’re flipping that over into engineering. Engineering is doing engineering work. In this scenario, the only realistic thing that any of those teams are going to be truly focused on are the outputs that they’re responsible for.

That’s a really hard thing to overcome. If you build that two-in-the-box or three-in-the-box model and create a charter that has everyone within that box equally responsible for the outcomes, you can support a culture that says, “We care a lot more about outcomes than we do about outputs, and that’s what we want to measure your success on.” There’s no hierarchy in that scenario — everyone really is a team.

The importance of acknowledging assumptions

Are there certain metrics or KPIs that you tend to prioritize? Of course, it can depend, but are there any that you tend to go back to when you’re looking at the impact of your product initiatives?

I’m a big fan of the HEART framework, which stands for happiness, engagement, adoption, retention, and task success. What’s nice about this framework is how it aligns with the natural product cycle. You don’t have to be focused on all five of those metric areas at once when you’re starting out. When you’re in the process of a zero-to-one build, you can just focus on happiness and engagement. Then, if you have a very mature product, you’re likely getting into the nuances of task success and retention.

The framework doesn’t prescribe specific KPIs. It gives you an idea of how to think about setting up a scorecard for your product, but then you still have to go through and say, “What does happiness mean for my particular product? How do I know if my users are happy or not?” That starting point makes it easy to have the right kind of conversations and build out metrics that are going to be most impactful.

How do you ensure those metrics are aligned with broader business goals?

This is where I think the VP of product role is critical. The most important thing in that role is linking the metrics that the product and technology teams use to define success for the product with the defined success for the business itself. I’ve found that there are actually more assumptions than you think to link those two pieces together. You need to acknowledge those assumptions when we’re coming up with these metrics so you can test whether they’re true or not.

For example, say I am building a HEART scorecard and I’m focused on engagement. I say, “My primary metric to tell me if my users are engaged will be daily active users (DAU).” I have a hypothesis that more active users are going to lead to greater retention and increased LTV. That assumption sounds right, but there are assumptions built into that scorecard that I need to check myself on. Maybe it’s not enough to have a certain number of daily active users, but it matters where those users are engaging in the platform. Or maybe which users and which types of users are engaging matters more.

If you don’t recognize some of those built-in assumptions or aren’t willing to test some of them out, you can end up in a situation where DAU is going up, but business objectives aren’t being achieved. That’s a frustrating situation. Building the bridge between the two is critical.

Trading off good and bad tech debt

Can you talk about your thought process when you’re considering trade-offs between investing in new features vs. focusing on performance improvement or scalability?

When you have these conversations. you need to understand whether you’re trading off good tech debt or bad tech debt. Tech debt is not equal. You want to reduce bad tech debt, and even then when your company’s scaling, you may consciously still be OK with accumulating some of that bad tech debt for the sake of focusing efforts on delivering important features. A lot of times, you may be willing to accumulate good tech debt for the sake of being able to accelerate some of your product roadmaps, but there is a trade-off.

The difference is that good debt doesn’t necessarily get bigger over time. You can pay that off at any time and the cost of paying it off doesn’t change. Bad debt is debt that is going to grow and become more costly over time, and sometimes even exponentially.

You have to acknowledge the cost of accumulating this debt and that there is a point at which it will have to be paid off. You can’t just pretend it doesn’t exist or won’t be costing the organization. You have to believe that the benefit of paying it off later outweighs the cost of missing out on some of the opportunities to accelerate the roadmap today. These are tough conversations, especially in organizations that are scaling quickly.

We talked earlier about outcomes over outputs, which is similar to impact over activity. How have you cultivated a culture that values the impact over activity with your teams, and are there any strategies that you’ve found helpful?

Impact versus activity is similar, if not the same thing, to outcomes versus outputs. Product teams, engineering teams, and delivery teams put a lot of blood, sweat, and tears into shipping a product or feature. When the release day happens, especially if it’s on schedule, you’ll want to celebrate. I’m not suggesting that you shouldn’t, but the challenge is recognizing that it’s not enough to just ship something. You sometimes celebrate the release but then quickly move on to the next thing and never properly evaluate the impact.

I think we tend to do this, at least in part, because of a timing problem. The release date is an unambiguous milestone, but figuring out when you should celebrate its impact is much less obvious. How long should you wait? Or perhaps the better question is how much data do you need to collect? We want to move fast and react appropriately, but some features and products inevitably need some time to bake. This is where institutionalizing high-quality retrospectives into your SDLC can help.

When I have a product manager present a proposal before we’ve committed to it on the roadmap, I expect them to have a hypothesis around the impact. That hypothesis becomes the basis for how we should think about measuring and celebrating outcome success, and should also guide the timeline for scheduling the retrospective. I want the product managers to think about how we’ll proceed if it’s positive, and what we will have learned and know how to proceed if the impact is not positive.

That doesn’t mean that it was a waste of an investment, necessarily. It means that we learned something and we should be able to pivot quickly to take advantage of that learning so that we can take the next appropriate steps.