Leader Spotlight: Building with grit with Ken Pickering

Ken Pickering is the Senior Vice President of Engineering at Starburst. He has over 20 years of experience working for both B2B and B2C orgs and has held leadership roles at a number of companies, including Hopper and Rue La La.

In our conversation, Ken talked about the challenges associated with uncertainty when working on a project. He also shared insights on scaling data structures and engineering teams and reflected on qualities that separate good engineers from great engineers.

---

Handling uncertainty

You’ve held a number of interesting roles in your career. Could you share an example of an especially challenging project, either from Starburst or a prior company, and what made it so challenging?

The most challenging projects are those that don’t have an existing solution or that have a lot of uncertainty. Maybe you’re working on a problem that you’re not sure you can solve. Maybe you’re building something that you’re not certain will work or that the market will like.

When I was working at Hopper, sales of flights were surging, but it took us about a year to figure out how to sell hotels well. We probably tried 50 different iterations in that timeframe, just pushing new code out every day, but nothing really connected. When you’re dealing with the general market and have limited feedback and insights, grinding through that until you find a fit is hard.

At Starburst we talk about grit. I think that’s an important attribute to have if you work for a startup. You have to understand that not everything you do is going to work. Not everything you do is going to be successful. Some days are going to be super hard and super frustrating, but the magic happens when you keep at it and eventually find something.

In hindsight, would you have approached the hotel problem differently?

It can be easy to fall into a trap of iterating on a path that won’t be fundamentally successful. You have an idea of what the product should be, but maybe you don’t have the best intelligence or information. Still, you continue on the path even though it’s not working.

In this case, it was a matter of going back to first principles: taking a step back and rethinking why people reserve hotels. We threw out everything we had done, went back to the beginning, and rethought the problem.

This is a very common second-product problem that a lot of businesses run into. Just because you have one successful product, that doesn’t mean you can easily build a second one. The product or the market may be completely different.

That’s a good point — it can be difficult to replicate success in the same way. You’ve worked in some very high-growth environments, like Hopper and Rue La La. How do you maintain high technical quality when everything is moving 100 miles a minute?

Investing in automated deployment and having an easy-to-use build system with plenty of automated test cases is very helpful. This includes all levels, from unit testing to integration to smoke testing. To maintain a really high bar, you need to make them reliable and have a zero-tolerance policy to test failures.

You also need to understand the risk around deployment and what could impact customers negatively. Then, do everything you can to mitigate that risk, whether it’s feature toggling or deploying software in a way that enables you to ramp it up slowly or ramp it back if there are issues.

We had to do this a lot at both Hopper and Rue La La because they were app companies. An app is basically a thick client. Once you deploy the app, it’s hard to deploy a new one because you have to go through the app store again, get it approved, and get all users to download it on their phones.

A bad push could translate to a week of downtime. We had to figure out how to do more on the server side to make sure we didn’t have stability issues with new features, and also roll out versions very slowly so we could see if there was a crash issue. It’s really about understanding the risk to your business and mitigating it with control mechanisms to reduce impact down the line.

For a SaaS business, it’s important to be able to roll forward with releases as quickly as possible so that if there’s an incident, you can deal with it quickly. Keeping time to resolution super low and being able to roll forward is pretty critical. Honestly, though, it really is around the speed of build and the speed of being able to actually get new code out into wherever you’re going.

Scaling strategically

What are some pitfalls that you’ve observed when scaling a data structure? Maybe something that junior engineers should be aware of?

A lot of the time, especially in modern engineering, it’s about choosing the wrong technology or structure or schema for something you’re trying to do. It’s painful because you usually have to do a re-architecture and migration of the underlying data at some point. Those are often the most costly mistakes.

I’m not saying that people need to over-engineer their data strategy, but sometimes we overload a data store because we’re already using it in production. Then, six or 12 months later, we realize it was not the most awesome idea. Maybe it got us some quick wins, but at the end of the day, it’s going to take us four months to migrate off of it, so how much time did it ultimately save us?

When you’re designing things, it’s important to consider what it will look like a year from now. Forecast it out. Consider if it’s really the appropriate approach or if you should be thinking differently about it; if you’re choosing it for convenience or because it’s the right thing to do.

Data has become one of the largest commodities that engineering collects and produces as an organization to fuel the company. Adding delays to the process due to poor choices is a lot more costly than it used to be.

Engineers today are dealing with volumes and scales of data that they may not have had experience with in their careers. Also, there are also different technologies, so it’s important to understand how to best navigate a roadmap. But the mistake I see most often is people just overloading something into something they probably shouldn’t.

When you’re looking at building for scalability, do you think about it differently based on industry?

Well, if you’re an IoT company, you’re generating a lot more data than other forms of business. But there are also different orders of scale. At Starburst, our open source project is used at the internet scale — it’s used by Facebook, Netflix, Uber, Lyft, LinkedIn, and Apple, for instance, and they deal with data at a much different level than Starburst does.

Understanding what stage your business is in and how to forecast that effectively is really important. Just think of how much security information a bank like Capital One collects on a daily basis — it’s in the petabytes. As an engineer, the last thing you want is failing to predict that your data is going to be in the petabytes scale and then being woefully unprepared for it.

We talked about scaling data structures, but what about your team? In your experience, what’s the most difficult aspect of scaling up an engineering team?

The hardest part often comes down to culture. If you have 20 engineers and grow to 40 in a year, then 50 percent of your team is brand new, with different work culture experiences. Let’s say you grow by another 40 engineers in year two. Now 75 percent of your engineering team has been there less than a year and a half.

The key is to build an organization where the culture meshes with the company values and is accessible as part of everything from the interview process, through onboarding, and then through employment. This could be anything from how you design, write, and test code to the things you value as human beings and how you expect people to act inside the company.

Without this, you can lose control of your value system, of how things are done. That’s very risky. That can cause huge amounts of cultural skew internally. Your quality might drop, you could end up with a lot of internal turmoil with teams doing things differently, and there could be strife between organizations. Those things are the hardest to unwind.

---

Finding great engineers

Are there some indicators that you look for in the interview to know whether someone is the right fit?

I look for people who aren’t afraid to admit mistakes or admit that they’re wrong. I’ll ask a hard question about workplace conflict or turmoil or ask them to share a time when they were wrong. I like to work with people who can self-reflect and are humble. Intellectual honesty is hugely important.

When I’m hiring leaders, which is most of the hiring that I do at this point, I try to understand the ownership aspects of their leadership. How much did they drive versus their boss or their team? What did they push for? What do they believe in? This tells me a lot about how they’re going to come in and hopefully add value.

At Starburst, we have a saying: “We want drivers, not passengers.” Ownership is something that’s very important to us. We don’t want people implying that something is someone else’s job and those sorts of things.

If something goes wrong, we want a leader who will intellectually own it, fix it, and admit the mistake. When you work in a fast-paced ecosystem, you have to understand you’re going to make mistakes. The worst thing you could do is revert to politics rather than just correcting the issue and moving on.

What’s your approach to promoting a culture of innovation within your teams?

There’s innovation in the product roadmap, in terms of letting people be a bit creative with how they implement features. We also do hackathons periodically to let people work their creative muscle, and we’ve actually gotten several productizable ideas from them. We sell products for engineers, and sometimes engineers have great ideas for quality-of-life features for the product.

Lastly, I do believe in funding innovation. If an engineer has a great idea, I’ll work with product to carve out aspects of the roadmap that are R&D and can be driven. We’ll spend some time working on a plan and developing a proof of concept.

Starburst’s natural-language SQL stuff was all engineering-driven. It was a “let’s hack something together and try to get some of it to work” type thing.

We have a number of features that have come up that way. I kind of approach it in the way VC-backed companies work. Engineers come up with a pitch, and if it’s accepted, I’ll give them a month to figure it out.

I try to have gates for research projects so that we’re driving toward something. It’s not just a blank-check research thing. We have agreed-upon deliverables, and if we hit them, then we keep going.

In your opinion, what separates good engineers from great engineers?

I’ve always approached engineering from a problem-solving methodology. For me, part of the magic of engineering is taking a different set of requirements and making something that didn’t exist before.

Great engineers love creating and building things, exercising some creativity, and using technology to solve real problems for people. Being a very smart person gets you pretty far in this industry, but loving it and wanting to invest more of yourself into it is what makes a great engineer.