Leader Spotlight: The evolution of DevEx with Vilas Veeraraghavan

Vilas Veeraraghavan is the Senior Director of Software Engineering, DevEx, at BILL. He previously held leadership roles at Comcast, Netflix, and Walmart building high-performing teams that drive automation, chaos engineering, and CI/CD at scale. Additionally, he worked to improve developer productivity across the enterprise by enabling engineers to do more with their dev tools.

Vilas joined BILL to help improve engineering efficiency and enable his team to deliver better results with less friction. We recently sat down with Vilas to learn more about how DevEx can improve the speed of innovation, his thoughts on how the concept of platform has evolved in relation to developer experience, and insights into how to determine the right time to make an investment in DevEx.

Our conversation has been edited lightly for brevity.

---

An opportunity to leave your signature

Could you talk a bit about what BILL does, your role within the company, and how your DevEx team contributes to BILL’s mission?

Sure. BILL’s mission is to help businesses succeed faster. The idea is to give them the tools they need so that they don’t need to worry about the day-to-day. Instead, they can focus on improving their business, like planning for an expansion or growing revenue.

From my perspective, my immediate customer is the engineering group within BILL — all the innovation and features that they are working on. My job is to remove obstacles so they can get to the end customer as fast as possible.

My focus is: Do we have the right tools? Are we enabling our developers? And, if not, what are the obstacles? Where are they stuck? What do we need to provide as improvements? Those are the questions I constantly ask.

You came to BILL around two years ago after engineering management stints at much larger companies — Comcast, Netflix, and Walmart. What has the experience been like for you managing a team at a smaller, earlier-stage company? What was unexpected or surprising to you?

The reason I wanted to work for a company like BILL is that it provides opportunities to truly leave your signature. BILL isn’t very small right now, but at the time, it was a smaller company than I was used to.

It gave me a chance to bring in the expertise that I gained at these larger companies. I knew what worked at scale, and I knew how large enterprises operated. Bringing that knowledge to a team that is doing mission-driven work was something I was looking for.

In terms of unexpected — and this is not necessarily just about BILL, but about every company — there’s so much good work happening within the company and, most of the time, the challenges are similar in each company, no matter its size. The challenge lies between the developer and the deploy. It’s always: How fast can we deploy?

Honestly, it’s not rocket science. It’s mostly about how to ensure these devs can get to production first and what they need to do so. Most of the time, it’s the same answer for everyone: we need CI/CD pipeline tools; we need a way for them to break this thing into smaller pieces; we need a way to get quick feedback; we need to get observability. It’s a standard set of answers.

I don’t know if that was unexpected, but it was a happy surprise for me. Like, oh, this seems very similar to what we just did before — it’s just in a different setting.

Helping each other and raising the bar

I noticed you’ve been sharing the Twitter hashtag #lovewhereyouwork and the inspirational story behind that. I’m curious, why do you love where you work?

I am genuinely passionate about BILL’s mission of making it simple to connect and do business. The people within the company represent and live it every day. I think one of the major reasons for me to join the company was the people I work with on a daily basis. It makes a huge difference. It helps you sleep at night because of the culture here.

Everyone is here to help each other and help SMBs succeed. That’s really cool. And internally, it’s still a really nice, fun environment, where everyone is helping each other, almost raising the bar, so that others can also continue to push themselves. The people are what keep me here and continue to make me feel excited about being here. That’s why I find BILL special.

It sounds like an important part of your role is making sure the developers have what they need to get to production. How do you go about evaluating development tools?

Some feel that DevEx is just about speeding up the delivery process, but I look at it a bit differently. I like to ask: How do we enable our developers to get to productivity themselves? So it’s more about the experience.

This is why there is a bit of a difference between DevEx and dev productivity. Productivity is almost like you’re taking something out of the person, whereas DevEx is about giving to the dev community and saying, hey, let’s focus on you. Let’s make sure that we give you the tools you need to enable yourself.

DevEx, as a term, has probably been around for ages, but only now are we truly defining it this way.

I think most of the discussions about tooling remain the same across different companies. This comes back to my statement about my happy surprise when I first joined BILL. The needs are the same for every company.

Especially today, there’s a lot of focus on getting something that:

1. Is a good product by itself;
2. Has a lot of community interaction around it — meaning, if we need help, if we need to understand how things work in different case studies, we can actually get that; and
3. Is from a company that’s making an investment in order to listen to its developers and users.

All three of these are important. There are many readily available open source tools that provide substantial community support, have good adoption, and offer online support as well.

Competing views around platform

Speaking of tooling, platform engineering is becoming a bit of a trend in DevOps. How has the concept of platform evolved in relation to DevEx?

In most cases, DevEx is tied to platform or platform engineering in terms of specifying the utilities that the company should have. I think that discussion tends to be either “platform is everything” on one side, or “platform is just this one thing” on the other. As an industry, that is a discussion that we are yet to have. We are yet to sit down and ask, what are the boundaries around platform?

Twenty years ago, platform used to be a very small, tiny contribution. It used to be common jars and utilities and things like that. Now, it’s transformed into: “Oh, platform is everything — you need infrastructure, you need this, you need that.” Everything is platform.

Within the industry there are a lot of competing views; there is no consensus. I feel this might be a topic around which there will be a lot of intelligent discussions, hopefully, in the next three to four years.

And what’s your opinion on the boundaries around platform?

I am all for having the platform and DevEx enable anything that the developer needs to do that is not business logic.

Everything that is business logic — what the developer is providing to their own end user — should be owned and governed by the developer, and they should also decide the policies. The last thing that DevEx or the platform should do is come in the way of delivery.

Listening to developers

Clearly, the role of DexEv continues to evolve. How was it impacted by the shift to a hybrid or fully remote work model?

Developer experience before the pandemic was all about tooling. During the pandemic, it also became about the day-to-day and the dev community. Is the developer actually able to do what they want to do on a given day? Are they accomplishing enough? If they can’t, are they speaking to someone? Do they have the kind of support they need to be able to push themselves harder or faster?

Also, I believe the pandemic accelerated standardization. A lot of standardization occurred in build processes in terms of how you deliver something, how you partner with someone, and how you make the right calls.

Let’s say you’re making API calls. What is the contract for that? How do you test it? Post-pandemic, there was less of the richer, high-bandwidth communication. People were forced to figure things out in an easy-to-communicate way.

I switched jobs during the pandemic, so I saw multiple companies go through this. Developers felt it was challenging for them to take on a lot of responsibility. They needed help.

Pre-pandemic, developer experience was mostly about focusing on things like business logic. Now, we care more about helping developers with their day-to-day work.

Helping with that day-to-day work probably involves a high level of communication. How do you handle that feedback loop?

The feedback process varies, and it comes from different areas. I haven’t yet found one single way to collect feedback. There is feedback on channels like Slack, some folks come and give personal feedback face-to-face, and others send emails.

I know some companies implement internal surveys for developers, but there can be a lot of recency bias to them. A developer might say, “I faced this yesterday, and it was such a big problem.” But it might be something that won’t reoccur for another six months.

Ultimately, what matters is that we keep listening to developers and strive to continually improve the DevEx experience. If we aren’t listening to developers, then we’re not really serving them.

Choosing metrics carefully

When it comes to developer quality, performance, and efficiency, what data or metrics do you find most helpful?

Metrics have a tendency to be abused. They can always be gamed as well. It’s tricky because the metrics that you measure are generally the ones that people will focus on and actually deliver.

The metrics we use change regularly, depending on what the business is looking for. The definition of the metrics that you’re measuring will pivot in any given business because of the nature of the problems you’re solving.

If the business is looking for speed, we have to measure things around speed. If the business is looking for quality, then we have to measure around quality. If the business wants to improve both speed and quality, then we measure different metrics.

The industry has used Dora metrics a lot, which are essentially four key metrics: change failure rate, speed of delivery, lead time for changes, and mean time to recover. But I feel that there’s a nuance to them that has to be adjusted internally.

You do not want to take those metrics and just plug them into your systems. You want to be very careful about how you motivate the team to do things, and you also have to educate teams on how to read them. Otherwise, teams could use the metrics in a way that’s counter-productive.

Could you give me an example?

Take change failure rate. It measures the number of times that you encounter failure after you’ve deployed into production. At that point, you have to either revert it or fix it. You could have a change failure rate of zero, but you might be delivering completely useless features. You do not want to be there.

You want to be able to say, I’m delivering something of value. So you want to make sure the entire value stream has been tested and checked.

Ask yourself: Am I doing something that’s valuable to the customer and will actually add to our business output? Those things are a little different, so we have a lot of discussions about these topics.

Investing in DevEx

Without stealing your thunder for your upcoming conference talk, do you have any examples of how a company can better invest in DevEx to improve the speed of quality innovation?

There is always valuable information in the interactions that developers have with the different teams they work with. Usually, a developer will go to X number of teams and say, “Hey, I want to get this thing to production. What do I need to do?”

Every one of those interactions is an opportunity for us to get better because when they ask for something, it means it isn’t available. Or, if it is available, it’s probably not meaningfully useful to them. From my perspective, this is about the investment.

Let’s imagine a developer spends 30 minutes a day debugging things that could be easily solved with a tool. Even with just 100 developers, that’s a lot of developer dollars per day that could be used for something else. If even just a fraction of that was spent developing an innovative feature, you’d probably take that any day. That’s the ROI.

To succeed in this investment idea, we can bring in a tool, or create a utility, or provide a service that is cheaper than having the developer spend that time and not really get to what they need to do.

This is just one example, but that’s how every single thing has to be measured. How much time is the developer spending on the task, and how much is the context switch penalty?

Quality is a classic case for this concept. If you push something out and there’s an issue that isn’t found until production, the amount of churn will be pretty significant. When it comes back, the dev will need to recall the details of working on it earlier, reset that context, get back into it, and deliver. That’s expensive, so you need to catch those issues early.

Your investment becomes: How quickly do I want to catch it? Your ROI essentially evolves from that.

What advice do you have for companies that are just starting to consider investing in DevEx?

I think there’s a threshold at which developer experience is something that starts becoming a challenge. I don’t recommend immediately investing in DevEx if your company is a small startup that’s still trying to make sure there’s a good product market fit and that it’s providing value to the consumer.

At that stage, I would not put DevEx at the top of the priority list. But it becomes really significant as you hit scale and as you hit challenges where you’re trying to improve the productivity of the overall delivery system. If there is a demand for your products but you’re not able to deliver fast enough, you’re already too late in investing in DevEx.

Investing in DevEx doesn’t mean you have to start a large team of a number of people. It’s more about ensuring that you have observability into your own metrics, into how people are working, and into how they are feeling hindered or helped by specific tools, and then making sure you are investing in the right tools. That’s a start.

Then, there might eventually be a community model of DevEx where you have volunteers from different areas who are DevEx champions who can provide support. Next, it probably transforms into a DevEx team. That is what I’ve seen at most companies — it starts off mostly as a grassroots-level thing and then eventually that team becomes more and more empowered to deliver for its developers.

What matters most is having open lines of communication with your developers and really listening to their feedback. Otherwise, your day just comes along as a series of meetings. It’s better for everyone to be able to say, “I met a few people, I exchanged a few ideas, I got some ideas for myself.” It makes your day feel different.