Message Brokers : what? why?
How does our world work? For this question there can be many different answer but in all of the answers there would be an important word that everyone uses, Connection. Just like the world, most of the applications that we use nowadays are connected to each other. Various different applications are interconnected and depended on each other. These applications may need to send data between each other for their operations. So who keeps track of all of this.
Lets take a look at a model that most of you may have been familiar with. Point to point model. Look at this diagram below.
We have 3 applications that are directly in connection with each other, now lets add two more nodes to this. Now the above diagram would look like this,
See what happened? The amount of connections between the applications more than doubled from what it initially was. This will increase its complexity and also makes it harder to maintain in the long run.
And also these applications will be connecting via HTTP protocol for data transfer, if one system is not available at that time the request cannot be completed(Synchronous). Basically a constant connection between the applications must exist for them to communicate. For these reasons there had to be an intermediator who will be taking care of storing and sending the messages whenever the other person is not available at the moment.
For the above mentioned reasons message brokers were created. Message brokers work just like the real life courier service. If you want to send a courier to your friend or family member who lives far away from you, you simply drop your parcel at the courier. The courier will make sure to deliver it to your friend and your friend will receive it when he goes back home. This is exactly how message brokers also work. The sender will send the message and message broker will keep the message. When the receiver comes back online it will forward the message.
You guys must be familiar with RESTful applications. A key term that people throw around when talking about RESTful applications is that it is Asynchronous. This however is not true. REST works on top of HTTP and most used HTTP version (HTTP/1.1) is not Asynchronous. In order to make REST asynchronous, it must be implemented with message brokers for it to truly become asynchronous.
Another great advantage of message brokers is that there is no need for two applications that are communicating to each other’s locations. All of these applications almost act independently (loosely coupled). The message broker will taker care of the connection and the communication.
Now you all got a basic understanding of what a message broker is and why there is a need for message brokers. There used to be two basic model that almost all of the message broker used. Those are;
- Point-to-Point Messaging Domain
- Publish/Subscribe Messaging Domain
Lets take a look into these models to get a better basic understanding about messaging brokers.
Point-to-Point Messaging Model
In point-to-point messaging model the sender will send the message, after the message broker receives the message it will hold the message in a queue. Once the receiver comes back online the message broker will dequeue the message and send it to the receiver. This happens in a First In First Out (FIFO) manner.
The messages will be received in the order that they are sent. The example that I explained at the start of this article about message broker can be taken as a real life example.
This model’s main characteristics are;
- Each message has only one consumer.
- The sender and receiver does not need to have connection at the same time for communication.
- The receiver will acknowledge once the message is received.
- The queue is durable.(Messages are saved even if both parties are not connected)
Systems that may use this type of message broker are depositing money into your account and a booking system also uses this type of model.
Publish/Subscribe Messaging Model
The above point-to-point model is good for communicating with a single node. But if a sender has to communicate with multiple node point-to-point cannot be used. That is where Publish/Subscribe model shines the most.
I will give a real life example first for easier understanding. Imagine that you are in a conference call, you will be speaking and everyone on that call will be hearing your message. If one person leaves the call and then joins in a little while, that person would have missed the words that you had spoken at that time frame.
As you can see from the above diagram, in this model the topic will receive messages from the publisher (sender) and then it will broadcast it to the subscribers (receivers) who are currently subscribed to the Topic (message broker). The sender doesn’t need to know about the number of receivers. In this model if a user joins in late they will receive the previous messages.
This model’s key characteristics are;
- Each message can have 0, 1 or more subscribers.
- Publishers and subscribers are dependent on the time. Both should be subscribers should be connected at the same time to receive the messages.
- By default Topic is nondurable which means if a subscriber is not connected to the topic during broadcast it will not receive the message later.
Note: According to JMS(Java Messaging Specification) API topic can be made non durable so that subscribers that join in late will also be able to receive the content that they missed.
The message brokers nowadays follow the JMS which is Java Messaging Specification. The example message broker that use JMS is Kafka.
So in conclusion I hope you all understood what is a message broker and why we need message brokers as well as when REST truly becomes asynchronous.