Explanation of Event-Driven Architecture

Do you know how Amazon or Netflix handle millions of users at the same time? The secret lies in event-driven architecture. And today, I'm going to tell you everything you need to know about this approach. Also, if you want to dive even deeper, some time ago I created the only completely free course on distributed systems, which are built precisely on top of this type of architecture. So, if you find this post useful, I invite you to check it out:

• Link to the distributed systems course.

1 - What is event-driven architecture?

As the name suggests, this is an architecture where communication is based on events. But not within a single application, rather between different applications.

In these kinds of applications, actions are triggered by an event detected in the system, typically in a different application.

The internal details of each individual application are irrelevant when considering the global architecture; this means that if you have multiple applications, one could follow hexagonal, another clean architecture, and others could be core driven, and you could also have serverless components.

When we talk about system architectures, we usually have three main types:

• Monolithic architecture, where all the code and business logic reside in a single application.
• Microservices, where we have small applications that interact with each other.
• Event-driven architecture, which is technically a microservices architecture, but commonly, when people talk about microservices, they imply direct communication between services, whereas in event-driven architectures, there is no direct communication.

NOTE: I have a post with more details here.

In this post, we will talk about event-driven architecture, where events are messages sent by different services and other services or applications react to those events.

For example, on an e-commerce website, when someone makes a purchase, an event called OrderCreated is generated, which contains the items bought and their quantities.

Once the event is created, the order service forgets about it; it is up to the inventory service to be able to read this event. Also, it is possible that this is not the only service that reads the message, so each consumer is responsible for being able to process the message.

The communication of messages does not occur directly; instead, it is carried out via an intermediary, called a message broker. Whether it is a queue or a message stream, its job is to propagate the information, in this case, events, to its consumers, which are services in our system.

This approach is called the producer-consumer pattern and it's the pattern upon which event-driven architectures are based.

When it comes to implementing this queue, you have many options. In the channel we've seen RabbitMQ, but I've also worked professionally with Kafka, Kinesis Streams, Azure Service Bus, SQS, etc. The options are practically endless, but the way they work is very similar and they all have the same goal: to move a message from point A to point B .. N.

2 - The impact of event-driven architecture

When we have a single application in a monolith, we run it on a server and that's it, there is a single point of connection to the database. By using event-driven architecture, things change, whether you have a single database or a database per application, you will have multiple points where your system needs such configuration.

And just as I'm including the database, don't forget the messaging system which the whole system depends on. If, for any reason, this system fails, we won't have producers or consumers of messages.

As a result, our system won't work as it should; ultimately, having event-driven architecture forces you to have greater infrastructure and of course greater control over that infrastructure.

Separating each service gives a great level of decoupling and freedom for each system, which is very beneficial, but on the downside, you need a well-configured monitoring system because when an error occurs, it tends to be much harder to find where the error is. Process automation is also crucial in these kinds of systems, because it's not the same to manually deploy a monolith once a month as it is to deploy 20 applications daily.

Doing basic things inside a monolith, like a transaction, becomes a rather convoluted process with the SAGA pattern when using event-driven architecture, which is costly not only in terms of infrastructure but also in human resources for development.

At the same time, it gives each system/service/team freedom to use the best technology to solve the problem that needs to be solved.

For instance, if you need to use cache in a specific application, you can include it only in that application; you don't need the entire system to access this cache.

And technically, it doesn't matter if a process takes 500 ms or 10 seconds because events occur asynchronously, meaning there is no user or system waiting for an immediate response.

2.1 - Scaling in event-driven architecture

Many will tell you, "Monoliths don't scale"; this is completely wrong. The correct statement would be, "Monoliths don't scale economically", because, my friends, with unlimited money, anything scales.

One of the key features of microservice environments (and of course of event-driven architectures) is that you can scale applications individually. In a monolith, you'd have to scale the entire monolith, which is obviously not ideal, since you usually only want to scale part of it. But it is possible.

Although, obviously, it's much better in an event-driven architecture because with smaller applications we can scale only those applications as needed.

Let me give you two cases to illustrate this. Amazon: the store makes the vast majority of its sales in the evening, while in the morning, people are browsing to see what to buy.

During the morning, they need multiple instances of the product service, since everyone is browsing and there's much more load.

By the evening, there are fewer people browsing; people go to their carts and already know what they want. So we can remove a couple of product service instances, leaving only the minimum required, and increase the order service instances so all those orders are processed without any issues.

The same thing happens if you work at a trucking or vehicle rental company, your vehicles have a device that sends notifications about location, speed, etc. every 30 seconds, but if the car is off, sending every 10 minutes is enough.

This means that at night, or outside business hours, we will have much less load than during the day.

Therefore, we will scale so that during work hours, we have a higher number of consumer application instances than at night.

The same trend can be seen at lunchtime, when the number of instances needed drops as vehicles are stopped.

Of course, all these increases and decreases are (or should be) carried out automatically.

2.2 - Do I need an event-driven system?

If you're asking yourself this question, most likely the answer is no.

Managing a distributed system or event-driven system isn't easy; in fact, it's very difficult. So if you're just starting to develop your application, I'd recommend starting with a single monolith, since you will have time to migrate later, or, if you don't want to migrate, you can develop new parts as separate services. But at the start, go with a monolith. Don't make the mistake of using distributed systems when you have five clients and ten calls per minute.

Distributed systems are a vast field. I have a completely free course you can check out here.