Couple weeks ago an article was brought to my attention. An article with a catchy title – “The Death of Microservices Madness in 2018”. Since I am greatly interested in software architecture for distributed software systems such statements are definitely too bold to be ignored.

One of the biggest misconceptions about Microservices I’ve observed is that people accept the idea as something holistic, self-contained and self-sufficient. In reality, Microservices is just an implementation approach for distributed systems architected and designed according to much wider concepts – Service Oriented Architecture, Event Driven Architecture, Domain Driven Design. Looking at microservices in a disconnect from these architectural styles is a mistake that will become costly and may lead to an ultimate failure. More on this here.

The article I’m referring to has quickly raised many questions in my mind, which ultimately led me to the opposite side from it’s author on many aspects of what was being presented. Most of the problems and complexities raised in the article are stemming from one root cause – misunderstanding (or not knowing) of fundamentals of SOA, EDA and DDD. Reading the article you may feel that microservices approach is too complex, full of failures and really something that may not work for you. I would caution readers from jumping to conclusions before analyzing the causes of such failures and complexities.

The following are just some issues and complexities raised in the article as almost inevitable and unavoidable:

• high coupling of components that otherwise should really be autonomous
• lack of data ownership and encapsulation
• transactional boundaries leading to the nightmare of distributed transactions
• understanding of communication patterns and dependencies
• and many more – complexity of development, deployment and running, etc.

If you are attempting to design and build a complex software system (and you are past the concept of “lets throw everything in and have a big monolith”) you better be considering such fundamentals from SOA, DDD, EDA as Bounded Context, Aggregate, Service, autonomy, messaging, etc. Without understanding the details of these concepts you will sooner rather than later corner yourself and will experience a lot of pain and frustration. I would highly recommend anyone interested in distributed system design to keep grokking these architectural styles, the concepts they postulate, the solutions they bring.

Key misconceptions and questionable statements

Now I want to comment on the most prominent statements made by the author of the original article. I’ll address each of the sections that I believe act as a source of confusion or misinformation. You’d want to refer to the original article to understand where, in my mind, the author has been missing some concepts that would help him to deal with issues and complexities.

Real world systems and poorly defined boundaries

I disagree with this statement – as a matter of fact, real world systems have much better defined boundaries than most software. And this is where author’s misunderstanding (or lack of knowledge) of DDD starts to play a crucial role in his thinking. The boundaries are always there, however discovering them and defining them is never obvious or easy from the start. Discovering and defining the boundaries (bounded contexts) may become difficult at times, and it is really a never ending process – the boundaries change as the system is better analyzed and developed, as new business details emerge and enrich our understanding of the business and the processes we need to model.

Next, the author quickly projects the complexity of defining the boundaries onto the implementation and deployment aspects. We should ask ourselves – why would the implementation and deployment be complex for this sort of architecture? High degree of coupling between microservices is the most likely cause, which really means that microservices start depending on each other for their functionality. Such coupling is the red flag indicating that the boundaries are being crossed, that you’ve mixed concerns and are no longer able to satisfy transactional consistency.

If you have coupling between microservices that don’t belong to the same DDD aggregate then you are doing something wrong. Step back and review your aggregate and its implementation, understand what’s missing and what’s required to ensure its autonomy and independence.

Communication complexities are great and get ignored

One of the key points the author makes is communication complexity, which involves more and more services as the system progresses. The author throws many concepts in the same pile – cross-service communication, transaction management, network related issues and dealing with those, etc.

This point really is closely related to the system boundaries one we’ve just touched on – communication becomes essential and its complexity rises if we need to satisfy a business transaction spread between multiple microservices. Cross service communication is a sign of invalid boundaries and wrong implementation of a DDD aggregate. The whole purpose of a DDD aggregate is to provide transactional consistency boundaries, ensuring the state consistency of the system. As soon as your code is engaging in a distributed transaction you must step back and reassess the boundaries, understand the dependencies and fix those.

I believe that complexities associated with messaging style of communication are grossly over exaggerated. The complexities that author tries to attribute to messaging are simply a sign of poor implementation that he had to work with, as there is nothing inherently wrong or overly complex in messaging, including versioning.

Complexities of state are ignored

I don’t event know where to start with this statement. The author mixes concepts that have very little or nothing to do with each other – stateless vs stateful, single data store for different services, no ownership of data, serverless, no-SQL, etc.

I believe the cause for seeing such complexities lies in misunderstanding of SOA and DDD – the key concepts of services in SOA and bounded context in DDD postulate the ownership of state (data). Not understanding this concept leads towards many services relying on the same data, crossing each others boundaries, affecting state and, ultimately, breaking encapsulation and system operation. And this is what SOA and DDD help you to deal with.

On the subject of stateful vs stateless microservices – I believe that all services require state since the very definition of service is the authority over state and behavior. The code that does not need state is called functions, not service. With this said, needing state does not equal keeping that state in memory, does not necessarily equal needing the same bit of changing data for multiple parallel processing – we don’t need stateful microservices for many, many use cases. In fact, having stateful microservices may be an indication that design needs to be carefully reviewed to confirm the decision before proceeding further – having stateful services infringes on your ability to scale and be resilient.

Distributed Transactions

The subject that burnt many and something that you definitely want to stay away from. I always tell that to teams I’m working with – there are patterns developed for managing distributed transactions, but trust me you don’t want to go there, you don’t want to try those. What is the solution? I’ve mentioned it already – it’s a concept of Aggregate coming from DDD. That’s your friend, that’s a part of the bounded context which your team owns. You want to make sure your aggregate implementation is scoped to a single service, so any operation on it always ensures that your state is consistent.

Increased complexity for developers

What is the base level for complexity? What is easy, what is complex? What are we comparing this to? I believe complexity needs to be properly described, as developing an enterprise level software system is not a walk in the park no matter which architectural style one picks.

For any business domain, developing a large enterprise software system requires properly resourced teams – you need to have enough senior technical people to lead the pack, to mentor and help others. Developing distributed systems requires somewhat different thinking and there is some complexity in that, I’ll give you that.

However, when done right, the actual implementation in most aspects becomes much more trivial than that in a regular monolyth/big-ball-of-mud system. To do it “right” all that is required is understanding one single key concept from DDD – Bounded context. Knowing what Bounded Context is all about, and projecting that onto very similar concept of Service in SOA it becomes very clear, that implementation of any complex software system adhering to SOA/DDD not only will be easier for any single engineering team, but will not become exponentially more complex with system growth.

What is required for success is a proper team organization with senior dev leads who can coach and mentor others, teaching the thinking behind distributed, decoupled, highly autonomous system components.

Increased complexity for operators (or devops)

This area has a potential for complexity, which is greatly associated with miscommunication and disconnect between implementation and supporting teams. Operations teams need to be involved early on, be part of solution architecture, understand the nature of distributed systems, provide non-technical requirements and influence the decisions that will affect the operations. The “throwing over a wall” culture will greatly increase the chances of friction, dissatisfaction, conflict and failure. However I do not buy the argument of high chance of failure when development and operations are separate teams – it’s all about communication and company culture – highly political, unhelpful and war-like culture will result in failures regardless.

There should be a well articulated company wide high level strategy of how such system is run and maintained. With that in mind wide automation should be considered to support the efforts.

Expertise is required

I do agree that expertise is needed, but why only in this case? Don’t we need expertise when developing or running any large software system? Perhaps having incompetent people in key roles attributes a great deal to high failure rate in software industry in general?

Any organization who has a need to develop a large software system needs to critically asses it’s capabilities, it’s workforce and act accordingly. And this is a business problem which should be at the forefront of management decision making.

Microservices is not architecture

This is where I completely agree with the author. Simply trying to build many runtime components, each of which is small in size will never provide any benefit and most likely will lead to a great failure. If you decide that your complex system needs to be distributed, scalable, resilient, then you really need to start looking into SOA, EDA, DDD, CQRS. Simply adopting “microservices” will not get you far – it’s an implementation approach in the end, not an architectural style.

Don’t underestimate the complexities

Another point made by the original author that is easy to agree with. It seems foolish to think that people will get into a complex problem solving without understanding many of the potential complexities, however I completely agree that this happens all too often – the teams are asked to deliver something they may not be ready for; developers, in general, are way too much attracted by anything new and shiny; technical leads may be forced into agreeing to something they do not know well, but they cannot afford to say ‘no’; and so on.

With all I said above – there are still lots of things one need to really know to be more or less comfortable starting on this path. As always – be pragmatic, but don’t be afraid of doing something simply because you found some negative feedback.

Conclusion

The article in question has prompted me to actually get onto the Internet and make couple of posts about distributed software architecture. I am by no means an expert who’s got all the answers, but I believe I’ve got some knowledge that’s worth sharing. Up until now I’ve been mostly sharing with engineering teams I’m working with, running internal training sessions, creating documentation, helping with implementation, etc. Hopefully some of the bits of knowledge will be useful to others. You can find the first post here and, hopefully, more to come very soon.