How microservice architecture can benefit from Async API

Good first slide.

I think that's the talk and I think looking at microservices everyone can relate to this one very easily.

Okay, let's see.

Yes, we are back on track.

Hi everyone.

I'm not much of a talker, so pardon me if I fumble anywhere, but try to be on track.

The topic of my talk is, microservices, and I have been building APIs for quite a while, and for front end mobile apps.

And in the last course of action, or last couple of years, I've seen that micro APIs have taken its way off with the introduction of GraphQL, and other different, similar technologies.

AsynchAPI is pretty much on the same track, just with a slight deviation around how do you actually interact with the backend APIs.

It should not always be synchronous.

Something which has been taken care by, different industries like Customer Data Platform or martech, where everything is all events, so an event is a customer action, so a customer comes onto the platform and basically takes a certain action, we look at that action and then we quantify it and then basically do an action in return.

So basically it can be basically returning back some kind of offers, or it can be basically sending an email to a customer.

So let's look at the agenda.

So looking at microservices, what microservices really mean, I think pretty much everyone knows about it.

Monolith versus distributed monolith versus microservices.

That's.

It's quite interesting, understanding our domain, and then looking at AsynchAPI, and then how it is different from Open API.

And then looking at what other toolings are available around it, and then looking at a demo use case.

So this is taken directly from Martin Flower, who was a brilliant man in ThoughtWorks and has written books around microservices.

So according to Martin Flower, microservice architecture is basically, if we go by definition, is basically an independent service which can basically perform a certain action, but has certain traits around it.

Those traits are basically, it has some kind of business logic.

Inside it, it can be deployed in silo and it can interact easily, using different protocols.

And then it is very language agnostic, so every microservice can be basically handled independently.

Now, these four parts are quite important looking at microservices, right?

It's the business capability, what kind of business logic is there.

How do you deploy your microservices?

How do they deploy in scale?

And how do you interact with them?

And then basically, how do they store data or how do they interact with other different services?

Looking at distributed monolith, look at, so if you really look at it, so monolith has basically, that's where everything started, right?

This is our big, giant application, which basically interacts with a user interaction.

User says, okay, I want to place an order.

You basically take that request and then you do a bunch of business logic around it, store some data, and then basically return a response back.

Then we became very clever about it because our monolith started expanding as the business grows, so over a period of time, it grows, you have more and more business logic inside it reaches to a point where it becomes like, quite cumbersome to maintain it.

So what happens is, if you bring change in a certain part of the service within the monolith, you have to basically ship the whole monolith itself.

What we did is, we basically started breaking it down.

If you really look at the difference between microservices and distributed microservices, the biggest difference you will find is, if your service needs another service to be deployed alongside with it, it is not a microservice.

It is basically a distributed monolith.

If it has a dependency on a certain record, which is going to affect another service to play its part, it is not a microservice.

It is a monolith.

They should be independent of each other.

And if they are, then it is a true microservice in nature.

This has happened with me in the beginning of the days.

You create microservices, you think that you are creating the ideal microservices, and you reach to a point where you break down every single thing, and it reaches to a point where you don't know where your data is going, and how do you basically track?

How do you log information?

How do you trace what is happening with it?

And you have created this giant distributed monolith, which is very hard to maintain.

So we don't want to get into that situation.

So in order to design microservices, I think the biggest thing we challenge for businesses are the idea of microservices.

Okay, we're going to break it down.

We're going to either expose it as an internal API or we're going to expose it behind some kind of messaging queue, or we're going to do some kind of logic and that's going to be independent of each other.

But one thing we don't understand is breaking down a service, it does not mean that you are creating a microservice.

What do you really need to understand is whether it really means value within your organization.

So taking this example of ordering itself, if you look at the biggest bounding context around it, so I'm just looking at the, two parts of it.

So if you go to a shop, what are you really doing?

You basically give, hand over your money.

You like a product and you get the product back.

That product will, either you are in store, then you basically get the product straight away, you shop online, you have a delivery, it will get delivered through a courier service, or there are other platforms now available where you can basically order your groceries as well, and then someone comes and delivers it for you.

Understanding the bounding context of your high level domain is very important.

So if you really look at from the ordering of a product domain, what you really have is there is a financial aspect of a product and then there is a delivery aspect of a product.

So we separate them out.

You will see basically I have products.

So there is a product which I am trying to sell, there will be a customer who is going to come and basically buy my product and then I will bind them together into a cart, a shopping cart, or you can call it an order.

And then in all that context, the customer is going to pay me, basically pay me through a card information.

If it is a recurring subscription, then I'm going to keep that kind of financial information along with it.

So, you can really see that.

There are certain identifiable domains in every single business, and that is basically what a microservice really means, right?

I can basically dissect an order.

I'll take order as an example again.

If you look at an order has If you want to break it down into microservices, there are certain contexts around it.

Ordering goes through a number of different steps around it, right?

The first step of the order will be your product validation or your procurement itself.

That is the first part of it.

You can make it itself as a microservice itself.

The second step is basically your validation around the financial aspects of it.

Whether the user, what are the various ways users are willing to pay for it.

And then the third part is the delivery aspect of it, right?

If you really look at it, these are three different aspects of an ordering flow.

Do you really need microservices for all the three different aspects?

Or whether it can be just one single microservice?

It depends on business to business, so I'm It's no true solution for every single product.

So every single product can have different aspects of it.

So if I basically look at the bounding context for my example here, so I can easily say that I have three different microservices which I'm building here: a product, a customer, so any customer comes, first do a registration with us, or he will basically log in, so there is a customer service which is going to provide me information about the customer, then there is an ordering, and then there is a product around it.

So how does asynchronous interaction works?

If you really look at it, your interface, ordering interface, is basically every user comes and connect with it and we're going to look at it in a minute.

How does that work?

So there is an interface which every single user connect to and then basically it provides some kind of request, and then we take that request, and then we acknowledge the request.

We are not returning a reply back.

What we are doing is just acknowledging, yes, we have got the request, and then we will basically further process it down the line between different microservices.

Now, these services can be independent of each other.

Now if you really want to look at the real life example of this is Amazon.

Amazon cards are basically our biggest example of it.

If you go through the buying process of, Amazon, you will see that you will never get a, you get that your order has been placed successful, but it never says that yes, your product has been shipped back to you.

The delivery.

Sometimes you will get an email saying that your payment might have failed, or we, the product is no longer available.

The reason for that is the consumer does not want that to get the acknowledgment around when my product is going to be delivered.

The consumer wants the acknowledgment, and this is basically a buying behavior, is that when you have got my request that I want to buy a product, it can be a boot, and then you acknowledged it, that yes, I will accept your order, and then I will do something about it, and then I will basically be constantly returning back an update on your order, that what is actually where your order is at.

Whether it has been product fulfilled, whether the payment has gone through, whether the delivery guy has accepted your order, and whether where it is en route to you from the delivery aspect of it.

So that's where AsynchAPI comes in.

When it comes to microservices, yes, we are interacting with a lot of microservices we created.

So you create like 10, 15 odd microservices.

So where, what does async is basically playing a role inside it.

So AsynchAPI is nothing but a specification which was written, by this guy, Frank Mendes.

And his use case was basically he wanted to, track down what are the different events happening around, a certain, use case he had and, in Hitch at the time when he was working with Hitch.

And, he basically said that, okay, I'm creating all those, microservices and I'm talking to all these microservices.

But every time a developer comes in or the, service owner of that service comes in and they are basically constantly improving the service and they're bringing in new changes.

What happens is that the on the other side who is the person or the client who is consuming the microservice has no understanding about it and what happens is it might break.

Microservices tend to break very easily if there is no contract in between them.

So what we did is there should be a way to establish a contract between the client, as well as the service itself.

And if we can establish that, we can help the client to basically interact with the service itself.

And that's where the specification comes in play.

So the specification actually talks about that how can you interact with the microservice, on what protocols you can interact with it.

It's pretty much as same as Open API spec.

So let's look at what the spec looks like.

So If you look at it, on one side is the Open API spec, which we all know about it, and then the other side is the AsynchAPI.

If you look at it, the spec is pretty much similar to how we are used to understanding the Open API spec.

So you have basically a server information, you have a way to basically authenticate yourself, whether you are the right person for interacting with the service, and then basically you have a bunch of, path around it, what is the path looks like, what is the HTTP method you are using to talk to it.

There will be a request and a response.

The difference between this and if you look at the Open API, and AsyncAPI is that it's basically asynchronous in nature.

But the standardization of how you're going to interact with it is pretty much the same.

Where Open API spec has a context around path, API Asynch, because we are talking about, messaging brokers and all the, WebSockets.

So they have a context around channels and then soon to be topics as well in terms of if you go into SMS or other, other services.

You have a producer.

The producer is the one who is producing the messages and then basically you have a consumer who is consuming it.

They can be, you can have multiple consumers or you can have a single consumer at a given time.

And then there is a whole, payload, which floats in between, which is basically what is being delivered to the services itself.

In between the, producer and the consumer is your message broker.

The message broker is basically the system which is responsible for delivering the message to the right consumer, and they will do it in different ways.

There, there are ways of fan in, fan out in the AWS SQS terms, or you can do it, basically you can have multiple subscribers to it, and then you can have publish and send.

And then on top of it, we have added some metadata around it, which is basically talks about what are the tags, message IDs, we call it correlation IDs as well, which can help us identify each individual request which is going from one microservice to another.

Let's look at the spec itself.

So this is the spec, this is sample spec for AsynchAPI and I'm not sure how much it is visible around it.

So you got, if you really look at, at Open API spec, it's pretty much similar to open API spec.

You have an, the version.

For your specification, it is right now 2.6.0 and there is a pre release of 3.0 as well lined up.

And then you got some server information, you can basically, it supports MQTT, it supports WebSockets, it supports HTTP, and there are talks going on actually supporting some of the AWS services where you can basically publish a message to a SNS topic.

And from SNS topic, it can go into a queue and then a service is listening from the queue.

So you can do a lot of magic around it, right?

So you got the WebSocket, and then you can have different kind of service.

So I have defined WebSocket and MQTT, which is your, mosquito is used as an example, but you can use pretty much any of the other messaging services.

You define what protocol you want to use and then basically there is a, you define, describe your servers.

Once you have defined the servers, you have basically a channel.

Channel can be basically where you're, putting the message in, and channels basically correspond to the microservices which you are creating, right?

So you basically, if we say in an ordering domain, you can have three different microservices.

One is the order fulfillment, the other one is your payments, and the third one is delivery.

So you can have three different channels alongside it.

So there is no endless possibilities around it.

And then basic, pretty much same as how you see in Open API spec, you got a component has a message element around it, message element basically tells you what kind of message you can expect.

And then you can basically define different kind of protocols around it, whether you're publishing, whether you're subscribing, and then the schemas around it.

So there is nothing new in this, if you really look at it, it's very easy to understand, and it basically, boost the developer efficiency as well because now you're not doing the guesswork of understanding what your microservices require.

You practically have a spec and then you can basically use that as a template to basically generate code or you can basically write some documentation around it for the rest of the team to understand.

There are a lot of tooling which has been now developed around AsynchAPIs and there is a nice CLI, which is used to develop, basically build a project.

There are generators now available, pretty much it can generate React clients, NodeJS clients.

You can basically take a spec and then you can basically generate documents around it.

There is a lot of tooling which has been provided now around AsynchAPI, which can help you basically gain that momentum in building your microservices.

Pretty much bootstrapping it.

There is a studio also in place, similar to editor.swaggerhub.com.

You can have studio.asyncapi.com and you can pretty much see and feel the documentation around it.

You can generate documentation inside markdown as well as HTML.

There are frameworks now coming in place.

There are initiatives going on where, they are integrating now SNS as well as SQS into the plat, into the AsynchAPI spec as well.

And there are different, frameworks around it, which can basically bootstrap your journey of creating those microservices.

Validator and parsers are already in place, so you can basically create your spec and you can validate it, and then there are other things around GitHub Actions.

So, looking at the use case, basically we are looking at is how about we create a client and then define, so client can be anyone, it can be a front end app, it can be an iOS or an Android app, and then we generate a WebSocket, and that WebSocket basically is ingesting whatever the requests are coming and doing it in an asynchronous fashion and then return certain kind of message.

That message can be an acknowledgement or it can be just a request.

And then, basically, there is some kind of, queuing system behind it.

On the back of the queuing system, basically, we send a notification back.

We can send the notification back to the customer, or we can send it back to the fulfillment center, saying that, yeah, the product need to be dispatched to the end user, or we can send it to another, different departments.

Let's look at the code itself.

For the sanity's sake, what I have done is I have already created A sample app.

Let me just bring it up.

What I'm going to do is, so let me just show you the spec itself.

So we got a WebSocket, which is already running.

And what it is saying is that we created a channel called order and basically it takes our order request and then it returns the order response back.

So let me just bring it up here.

So that's the studio.

So you can see here, I have defined two of the servers.

There is a WebSocket, which clients can connect to.

Then there is a message broker in between, a queuing system, and then I have an order.

Order basically takes a set of request parameters, which is basically a product ID and order priority.

And a store ID itself, so store ID and a product ID, and a quantity.

And then basically it goes through it, and in return, it returns back an order ID, and it'll return the list of the remaining products, and so that you can go and update the DOM at the same time.

So all it is doing is basically, the objective of this, my interaction is that you tell me what product you want, what store you need it for, and then tell me the quantity you want, and then I'm gonna basically take that quantity away from the, my inventory, and I'm gonna return back an order ID back.

And then you can basically, I will provide you a subscription endpoint where you can subscribe and you can get the order status back.

So that all, that's all it is doing.

And then down the line, I can do multiple things.

I can basically go and call a validation service, and I'm gonna say, okay, check with the store whether the product is available or not with them.

I can easily go to the delivery department, and I'm gonna say dispatch this order to this particular consumer.

I can call another service, and I'm gonna say, these are the basically the payment details.

Just go and take the money out of customer's account.

So Yeah, so practically you can have all the request and response defined here and let's look at it, how it looks like in real life.

So I'm going to connect to the server, which is already in place.

So it's connected and all I'm saying is, I have a product and that product I need for that store and I need the quantity to be one and on the back of it, all I'm doing is I have this small JSON file as a local data store.

If I can see where I am, so practically think this as a small data store.

I got bunch of orders, which is this one.

So I got orders, which is basically all my incoming orders.

I got bunch of stores, and then I got products.

And then products are basically I got quantity, so I don't have one product there.

I have product ID 11, so I can use that, so I go back here, and I'm gonna say product ID 11 for store ID 1, and I practically send the request, and what in the background it has done is, it has gone through and now added a product into the orders, so I have a new order coming through, a new order ID has been generated, And if we look at the downstream, it returned me product ID 11 and now the quantity of that has been reduced to 2.

So all it has done is it has blocked that one product which I ordered for and has generated an order ID and returned back with an acknowledgment saying, Hey, I got your order and I am on the on the process of fulfilling it.

And you will basically get those notifications if you subscribe to those endpoints.

And eventually the client can basically hit the subscription, subscribe to the other endpoint and basically says, okay, now tell me what is the status of my order looks like.

Also, down the line, what it has done is, it has called a message queue system and it has basically sent a message to a notification service.

Notification service says, okay, I'm going to go and tell A service department, so just use Slack as an example, saying there is a, has a new order has been received and that's basically where you're going to go and look at the order.

So practically that Slack webhook can be represented as sending a message to the client saying okay your order has been fulfilled, check the order status on a certain page.

User can then go on to that order page and has a nice experience where he can see whether his order has been packaged, whether it has been dispatched, or whether it has been delivered.

So they can basically do that.

The idea of, behind, this is mostly around having the mindset of changing our user experience.

User does not want to know that what happened with my order?

He doesn't, when need the assurance around it.

They know that there is a business there.

I'm gonna pay the money and now I'm gonna get the product.

What they really need is an acknowledgement saying, yes, you have received my request and now I need to have an experience where you will tell me, where my order is at.

So you break it down, that experience into successive steps.

To basically provide updated information to the user rather than giving a single message back saying, okay, we have received your order.

This is the order ID, check back with us later, right?

And the order will be dispatched to you in, say, three days down the line, whereas the other experiences mostly around that, okay, we have received the order, how about I will give you all successively, I will send you all the notifications around.

Where your order is at.

That is a different experience where user has been acknowledged with and given the assurance that yes, we have got your back.

Your order is at successive steps.

So you will basically get different notifications along the way or you can basically have a status page with an order which basically tells you where your order is sitting.

So that's just an example of it So changing the mindset from having a synchronous experience to an asynchronous experience can be really worthwhile and I think that's where AsynchAPI is really shining well because it actually helps the developer to basically build those kind of experience at ease.

Thank you so much.