The increasing nature of frontend complexity

Frontend web development has never been a hotter or more controversial topic.

Where once it was the server-side community that was embroiled in arguments over seemingly obscure topics, like whether ORMs were useful abstractions, now you can’t ask whether CSS belongs in JavaScript without raising everyone’s hackles.

The proliferation of frameworks and buzzwords in the JavaScript ecosystem has gotten so extreme that Medium thought pieces on JavaScript fatigue are the new lightning rod of the development world. We can’t even stare at our own navels without sparking off a ‘navel-gazing considered harmful’ trend on hackernews.

What is the source of all of this frustration and angst? Is this the standard cycle of new technologies sparking backlash by everyone invested in prior generations of technology, or is there an underlying root cause?

Controversy follows change and uncertainty. Where there are real problems and no accepted right answer, we argue over possibilities. And when we disagree about what the problems even are, we run into real challenges.

Controversy tends to spike in the development community during periods of change. And, surprise, we are in the midst of a massive and rapid period shift in complexity from the backend to the front.

Front end software development is:

– real-time (instant load, 60fps)
– distributed, incremental (synchronize remote data as needed)
– asynchronous
– reactive (react to user actions in realtime)

Front end is the hardest kind of dev I do. The folks who do it every day are heroes.

— Yehuda Katz 🥨 (@wycats) November 14, 2017

Conservation of complexity

Not everywhere in the web world are things increasing in complexity. It’s never been easier to put together a database-driven API. For simple examples, you can fire up Firebase and never touch a line of code. The tooling and automation has reached an incredible level of sophistication, and as a result many tasks are simpler than ever before.

Sophistication in tooling has not made our job as engineers go away.

There seems to be a universal axiom that the amount of complexity we have to account for never goes down, it simply moves around. In this case, the reduction of complexity in creating server-side applications has co-occurred with a dramatic shift of complexity towards the frontend, a shift we are very much still in the midst of dealing with.

Origins of the shift: Not one megatrend, but three

The rise in frontend complexity stems from the confluence of three megatrends, each with their own reasons and impacts, coming together to create a perfect storm.

Diversification of devices

The first of these megatrends is the incredible diversification of network connected devices that run web browsers. When I started in the web, you assume your users were on desktop devices running on dial-up or DSL. The variation in screen size, latency, and bandwidth that we had to deal with were minimal. Today’s frontend developers need to account for everything from tiny budget phones with only 320px wide screens on 3G connections up to massive desktop screens running on high speed corporate ethernet.

The bar for quality is higher

At the same time, the quality bar has risen dramatically.

The web is in the midst of a shift from the most widely used document platform in the world to the most widely used application platform. Users expect web applications to feel as snappy and responsive as native applications. Increasingly, design is a differentiator.

Massive market growth

Finally, the number of consumers able to access the web has grown dramatically. According to the ICT report on internet usage, since 2010 alone we’ve gone from approximately 2 BILLION people online to over 4 BILLION, a massive increase in the audience that web developers are targeting.

Evidence of the shift

It is this combination of trends — the increased complexity of environments, higher quality bar, and the market pressure of a massive increase in audience that has led to the frantic and often chaotic explosion of activity in the frontend web space.

Evidence of this explosion is everywhere. With HTML, CSS, and JavaScript as the primary tools used to address the frontend, we can see this looking at those ecosystems.

Ecosystem growth

The number of JavaScript packages registered on npm is on a fundamentally different growth curve than packages in every other programming language.

Frontend HTML and CSS frameworks have also multiplied. Five years ago you might have had three choices — the venerable grid960 or one of the newly hot full featured frameworks, Bootstrap or Foundation. Today, a quick search for “css frameworks” turns up a list of over 100!

New programming models

Beyond the explosion of tooling in the existing language ecosystem, we’re also seeing a fundamental shift in the programming models & possibilities available to us. To handle increasingly complex user interfaces, we’re seeing a shift towards new programming models.

The first efforts work within the existing languages to use models that are shown to better scale with complexity — component-based architectures, functional programming mechanisms, and carefully encapsulated state.

The next generation is going farther, thinking beyond the limits of the JavaScript programming model with new languages that have fundamentally different programming models like Elm and Web Assembly.

What does this all mean?

One of the most obvious impacts of this massive shift in complexity is visible in the stress and fatigue that frontend developers have been feeling and complaining about.

When your field is changing so quickly that best practices are obsolete within two years, you have to run just to keep up.

The same thing happened a decade ago in server technology — keeping up with the latest and greatest in Ruby on Rails or Django was a full-time job. But today, that technology stack has matured.

This isn’t to say stagnated. Not at all. There is still serious progress being made. But, one can make a good living as an API developer using Ruby on Rails without spending half of your work life racing to keep up. In fact, some of the biggest differences in recent releases have been not in the core technology, but in integrating the newest frontend tools like webpack.

You can’t escape tooling

Another major impact of the massive shift in complexity towards the frontend, is that frontend tooling is no longer a “nice to have.” It is both a necessity and a glaring pain point.

Tools exist for a purpose. Humans can only comprehend and handle so much complexity at a time. Tools allow us to encapsulate blocks of complexity into simpler pieces, letting us abstract away problems and create increasingly complex solutions.

As our requirements on the frontend have become complex, an entire ecosystem of frontend tools has germinated out of necessity. Tools like gulp, webpack, brocolli, rollup, babel, flow, and more exist to help us harness the chaos and make it manageable.

However, for all of the frontend tooling that has been developed, it’s actually one of the glaring holes in the ecosystem. Compared to back-end tooling, the frontend tooling ecosystem is incredibly raw and immature.

Browsers are moving rapidly to improve debugging tools, but logging is still primitive. Build tools are also improving rapidly but they too are still in their infancy. Furthermore, configuring them is overly complex (have you tried configuring webpack by hand recently?. Plus, community support and funding continue to lag behind backend tooling.

Tremendous opportunity

The combination of massive market demand and glaring holes in the ecosystem creates an incredible opportunity for individuals and companies alike.

At the company level, the inadequacy of the tooling available for the frontend creates business opportunities to plug those gaps. LogRocket is attacking the problem of frontend logging, but there are a multitude of other opportunities here.

Who will create a better frontend focused IDE? Better frontend performance monitoring? Better systems for testing and validation? While versions of these tools all exist, they all feel primitive compared to the level of tooling available to us in the backend.

At the individual level, the opportunity is even larger. New tooling and frameworks create tremendous power, allowing individuals to solve problems and create products that previously took entire teams to implement (or weren’t even possible).

And the field is expanding far faster than the number of developers can keep up. Obviously, tons of demand and not enough supply equals jobseeker’s market.

But when you combine that with exponentially increasing power to create value you get a market where a skilled frontend developer can practically dictate their terms.

Where is this going?

Making predictions in a field changing as rapidly as this one is an exercise in misestimation. But, there are a few things I can say with certainty.

1. First, it’s not going to slow down anytime soon. The forces that have driven the rise in frontend complexity are not going away, and the current solutions are just not good enough for the pace of innovation to drop.

2. Second, “frontend web development” will expand into more and more previously native domains.

The rise of WebVR creates an opportunity to create powerfully immersive media experiences that previously might have required massively custom solutions. And with the web, you can seamlessly and progressively engage someone into a VR experience, adjusting for the power of their device, and linking out to non-VR resources where they are more appropriate.

New specifications like the Web Payments API support seamless access to capabilities that previously were restricted to native devices only. Imagine a world where paying for things online is as simple as buying things from an app store — a simple thumb ID and you’re done. What new experiences will move to the web? What new opportunities will this unlock?

The complexity will continue to increase, but the supporting tooling ecosystems will also mature. It’s already happening. Look no further than the way we’re working on the core underlying technologies. There’s TC39 working on the JavaScript language and the CSS WG working on the CSS specification.

Both have dramatically improved the speed and consistency of new spec releases. The spec communities have figured out how to iteratively improve the specifications, and the browser vendors are increasingly able to coordinate and release improvements on an ongoing basis.

Astoundingly, we have reached a point where every major browser is an evergreen browser. Now we can imagine a future where almost every user has instant access to new web platform features as they are developed.

Lastly, we’re also seeing the non-browser tooling starting to improve. Webpack and Babel are increasingly being supported by the community, and tooling vendors like LogRocket are beginning to carve out supportable businesses and support frontend developers.

The future’s so bright…

This is a hard time to be a frontend developer.

The technology is moving so fast it’s hard to keep up, the ecosystem is fragmented, and there is tremendous pressure for even individuals to meet and exceed the user experience standards set by massive billion-dollar companies.

All of that is true, and yet I could not be more excited about the future of frontend development. We’re in an incredible time of innovation and change, with the browser transitioning to become the platform for application delivery and execution for an audience that is 4 billion strong and growing. There have never been more opportunities, and there has never been more powerful tooling.

This is an amazing time to be a frontend developer.

Kevin Ball has been a web developer for over a decade, co-founded and served as CTO for 2 web startups, and has given talks all over the world. He is currently president of ZenDev, LLC, a frontend consulting and training company, and publishes the Friday Frontend newsletter.

---