Monolith: An application stored in a single codebase; low complexity
Microservices: Multiple applications loosely coupled; high complexity
The Spectrum: The evolution from 2 architecture choices to a range
TL;DR: The options for IT architecture have evolved dramatically from the binary choice of monolithic versus microservices, to a spectrum with various options to extend these core approaches or even smash them together. Each of these approaches comes with pros and cons impacting complexity, cost, stability, and user experience. AI is going to increase the rate at which software flows through these systems, testing their limits. But AI can also help teams navigate this by migrating across the spectrum faster, refactoring legacy systems, and helping to manage complexity.
The Evolution of IT Architecture
A lot of discussion in AI today revolves around software development and our ability to write or review code. AI is certainly accelerating how fast we can write software. But IT architecture determines how safely and efficiently we can run it. This week we want to quickly look at where this architecture began (monolithic), how that has changed (microservices) and then explore where it could go from here.
The Monolith Era
Software architecture in the middle of the 20th century was dominated by monolithic architecture. This is a single, self-contained application where all features live in one codebase. In the monolith era, your shopping cart, authentication, payment logic, database access, user interface and everything else ran as one deployable application.
Monolithic architecture is most renowned for its simplicity. Because the application lives in a single codebase, initial development tends to be simpler and require less planning. This simplicity also makes it easier to secure, debug, and maintain.
However, as cloud computing became more widely available, applications scaled in size, and the adoption of agile coding practices encouraged teams to push for more frequent releases, the downsides of monolithic architecture became more apparent.
The downsides of monolithic architecture:
Deployment Speed: The smallest changes to the app still require that you release the whole application each time. This is cumbersome and encourages infrequent releases rather than continuous updates. Agile best practices prefer short releases in order to allow flexibility and better cater to customer demands.
Tech Lock-In: Monoliths are typically built around a single tech stack, making it very difficult to introduce a new language. If a team wanted to use Python for AI tasks, they would likely need to rewrite the relevant portions of the application or rely on awkward interoperability rather than simply spinning up a separate service.
Scalability: Monolithic applications must be scaled as a single unit. If one specific function (e.g., the search) consumes a lot of resources, the entire application must be replicated to support the traffic, leading to inefficient infrastructure utilization.
Single Point of Failure: If one function goes down, the entire application crashes.
The Microservices Shift
As these pain points became more obvious, the microservices architecture emerged. Instead of one central codebase, the microservices architecture breaks out applications into their own tech stack. This means that search, shopping cart, payments, and authentication are no longer interdependent and instead can be managed, deployed, and monitored by individual teams.
The benefits of microservices architecture directly address the pitfalls of the monolith:
Deployment Speed: Services can be deployed independently, allowing organizations to ship features multiple times a day without coordinating a single massive release.
Tech Agnostic: By allowing each team to manage their own application, they can now also use their own tech stack.
Scalability: If an application is seeing a lot of customer traffic, that application can now be scaled up independently from the larger application. This tends to be cost efficient at scale.
Isolated Failure: If one application fails, it no longer impacts the other applications.
However the adoption of microservices created problems of its own:
Harder Observability & Debugging: Bugs now span different teams and technology which makes finding and solving these problems increasingly difficult.
Operational Complexity: Multiple continuous integration / continuous delivery (CI/CD) pipelines, monitoring, and constant need for communication between applications increases operational overhead.
Data Management: Each application now has its own database, adding complexity to understanding the data.
Microservices has been broadly adopted by the industry, with ~74% of companies reportedly using this architecture. However, companies are now trying to solve the new problems that it has created. Platform teams have emerged to help solve the complexity, new platforms have emerged to help centralize sources of truth or provide observability, and some companies have begun to explore entirely new options.
The IT Architecture Spectrum
As the industry evolves, the idea of two solutions has given way to the idea of IT infrastructure as a spectrum, with each point having tradeoffs impacting cost, stability, and user experience, and most importantly, complexity.
Types of architectures:
1) Modular Monoliths: While we are unlikely to revert back to the original monolith, there have been some advocates for a “modular monolith”, which is a variation of the monolithic architecture that maintains all of the code in a single codebase, but has clear internal boundaries between modules (Logiciel Report).
Theoretically, this could provide some of the benefits of the monolith, like reduced operational complexity, while avoiding some of the pitfalls of a massive repository of code that is difficult to navigate. The biggest problem with this approach is that you lose the ability to independently scale applications, which could increase costs at scale. This opens the door for potential hybrid structures that could combine monolith cores with microservices for high-scale components.
Complexity (Low): Because the application is maintained in one codebase, the level of complexity here is the lowest and is managed by the team.
2) Serverless & Function-as-a-Service (FaaS): While some companies are gravitating back to less distributed systems, others are taking them to the extremes. With microservices, the application is broken down into multiple components that are loosely coupled. FaaS breaks these down even further into tiny pieces called functions. These functions are so small that you do not need to keep servers up and running at all times to execute them. If microservices are like operating a full restaurant, FaaS is like operating a food truck that only appears for a moment to serve a single purpose. Instead of having to manage a server, the cloud provider manages the allocation of resources, allowing companies to just pay for the milliseconds of compute time used rather than for idle server capacity.
Because these can scale quickly without having to spin up new servers, this can be helpful for short-lived but high-intensity workloads — think Black Friday sales. It can also make it easier to push processing to the “edge”. Instead of spinning up a new server, you just buy compute in a local system for a brief moment in time.
Complexity (High): This is the most extreme form of a distributed system, and while it is operationally complex, much of the complexity is offloaded to the cloud vendor.
3) Composable Architecture: Another idea that has emerged that is tangential to microservices is the composable architecture. Now instead of breaking an application down into smaller technical components, they are broken down based on business functionality (e.g. inventory management). This allows the entire business system to be organized in a way that makes it easy to replace or upgrade.
Complexity (Medium): This is more of a derivative of traditional microservices and comes with similar levels of complexity that must be managed by the individual teams running each application.
The Role Of AI
The adoption of AI into the enterprise workflows is influencing the way that architectures are built. AI operates best when it has readable code and context. Building architectures with these two ideas in mind will increase the productivity of AI systems. For example, a monolithic architecture may help reduce the number of disparate applications that AI needs to understand, but jumbled spaghetti code can negate some of these benefits.
The ability to transition to different architectures will also be made easier by AI. For example, you could have an agent help refactor a monolithic codebase into microservices in a fraction of the time it would have taken with manual human coding. Or on the other hand, if you wanted to migrate from microservices to monolithic, it could help with that and then monitor the code to make sure it does not turn into a black box of spaghetti code.
A common thread as you move further away from a centralized monolithic system is complexity. While you can gain benefits to scale, composability, fault tolerance, and deployment speed, this tends to come with operational overhead, more observability, and a plethora of systems that are causing developers to burn out. AI could both dramatically increase complexity or simplify distributed systems in a way that allows developers to capture the benefits while negating the downsides. We are specifically excited to meet with teams that are creating tools looking to address this problem.
Takeaway: The options for IT architecture have evolved dramatically from the binary choice of monolithic versus microservices, to a spectrum with various options to extend these core approaches or even smash them together. Each of these approaches comes with pros and cons impacting complexity, cost, stability, and user experience. AI is going to increase the rate at which software flows through these systems, testing their limits. But AI can also help teams navigate this by migrating across the spectrum faster, refactoring legacy systems, and helping to manage complexity.
Have a great weekend,
Josh