Recently, I tried to use SAGA Pattern on Microservices architecture
Last Day, I tried to make simple microservices app, this app is actually a small project that I build for me to deploy into a cloud platform. because I never deploy microservices app into a cloud. I don’t know how the best practices to deploy microservices to the cloud, and I don’t know how to communicate each service when the services had been deployed. My last project that had been deployed into Cloud Platform is a simple blog project, and it is also not a microservices and I just used the cloud platform like when I use a VPS. so I decided to my self to learn about Cloud Technology and Cloud Infrastructure, Also learn about Kubernetes. So I build a simple online store with a microservices pattern
When I decided to make this thing, I feel so lazy. actually, I’m a lazy person when building projects that aren’t my task from my company or my precious projects, I feel lazy and I think playing Duolingo for learning English and Basic Japanese from it more a good day for me 😅 So because I’m also lazy, rather than build this thing with Golang and using awesome technology. I decided to use NestJs because it has awesome code generation. and also currently my implementation of Saga Pattern also still under the top of the HTTP Layer. maybe next iteration I’ll migrate into messaging or event-based. And also maybe I’ll update this article with the event-based
The problems
First time I know about microservices (it’s around 2018) I think microservices is only a technique to separate the services from big monolith apps. like When I have e-commerce, so I can make separate like User-service, product-service, etc. and I think only this approach that known as microservices. but When I learn more deeply there is another approach about microservices, like separating the database, so every service has a single database. Then After I tried this approach as an e-commerce app that needs a transactional in the database I got the problems. “How can I implement transactions when I used a different database in my services since this approach cannot implement an ACID”. and I know about the Saga Pattern.
The Saga
Saga Design Pattern is a way to keep the consistency of data across microservices in distributed transactional. since ACID is not supported with distributed transactional. saga is a sequence of transactions that will send an event/ message to the next step of the transaction when the transaction is successful, but if the transactions fail the saga also will send an event/ message that informs a failure of a transaction to each of the service.
in monolith app or microservices but using a single database, we can keep the data consistency in the transaction by using a two-phase commit. in a two-phase commit every database event like create, read, update, and delete in the transactions should run the commit or rollback at the same time. so the data will be inserted into all of the tables or not at the same time. but in multi-database two-phase commit is not a good choice
In Saga Pattern, there are two different approaches:
- Choreography
- Orchestration
Choreography
In the Choreography pattern, there is no central service for handling all of the transaction flow. all of the services that hold on their business process has their event consumer and event publisher
In Choreography, because there is no centralized to ensure the transactions work, so it’s actually sent the event in the service. like when the user creates an order, after the local transaction on order service successfully it sent the event/ message to the next services.
Orchestration
in the orchestration pattern, there is a central for managing all of the services. so the business logic will be placed on the orchestrator and the service that consumes from the orchestrator doesn’t have any business logic process. because the business process has been placed on the orchestrator. I mean, it’s like the other service just consumes the event from the orchestrator then does a local transaction.
After I try to learn the concept of Saga Pattern as far as I can. I decided to use an orchestration pattern because it’s more simple rather than choreography. In my home opinion, the orchestration saga is more simple to build and debug. because there was an orchestrator to manage the business logic. rather than in choreography because the service is individual and has its business logic and the service should ensure to continue to the next process or nor. and also in choreography is more difficult for you to know about the app flow because you should read the process one by one in all of the services that have any relations
so here is my diagram for the saga on my app
The flow:
- First, the order service will act as an orchestrator here.
- Create Order in order service based on cart_id, if it has succeeded then continue to remove user cart based on the cart_id
- Order Service will call Cart Service to delete the user cart, before deleting the cart, I save the cart as cache using Redis. and if it has succeeded. Then
- The order Service will call the Product Service to decrease the product’s qty. but before I decreased the product’s qty, I also save the product qty into Redis
- If all of the flow has been successful, run the commit
- do a commit on the cart service and the product service will remove cache from Redis
So How do I do a rollback if an error occurred
The flow:
- First, the order service will act as an orchestrator here.
- Create Order in order service based on cart_id, if it has succeeded then continue to remove user cart based on the cart_id
- Order Service will call Cart Service to delete the user cart, and there an error occurs, so
- rollback order
- rollback cart
- rollback product
And I also use Redis to keep my data if fails happens. so in the cart service, before I deleted the user’s cart I search the cart and save it into Redis and then delete the cart, and vice versa for the product. so in Order Service, I just do commit to the local order service transaction, Cart Service, and Product Service. In the Cart Service and Product Service, the commit function will remove the data from Redis. and the Rollback function for the product and cart service will insert the previous data that has been saved into Redis to MySQL Database, and also remove the Redis data
Implementation in Code
Order Service for adding a new order
here is my service file under the Order-Service for creating an Order.
like what you have seen. my service has a local transaction for handling order transactions, if it is successful the transaction will be committed, and if it failed the rollback will be executed. in lines 59–62, I call cart service. where my order is based on the cart, so to make a new order I will make a request to the order endpoint and also I’ll send it with a request.url, where the request.url is a cart_id. so here I remove the cart.
in the line of 65–68, I also call a product service. where the product service is responsible for managing the product’s stock. so I’ll patch product stock from product service from this line.
if all of the flow is successful without any errors or issues. the transaction will do a commit. but if it failed, the transaction will do a rollback.
Cart Service for removing user’s cart
Here is a transaction handling on the cart service. the first gist is used for deleting the user’s cart. the second is used for commit and the last is used to rollback.
before I deleted the user’s cart I will search the cart and the result from it will be saved to Redis.
So when I want to commit the transaction I will delete the Redis cache
But, If I want to do a rollback. I’ll get the data from the Redis and save it again into my primary database where I decided to use MySQL.
Product Service for handling product’s stock
Here is a transaction handling on the product service, the first gist is used for patching the product’s quantity. the second is used for commit and the last is used to rollback.
First, I search for the product by the ID. if I didn’t get the result from DB just throw an error. and I just saved the data from the query’s result into Redis, and I also updated the quantity.
so it’s the commit process. just remove the cache
and here is the flow for the rollback of the patch product flow.
I learn this concept as far and as fast that I can. Actually, I didn’t try this concept into real-world apps. hopefully, this article can improve your understanding of the Saga Pattern. And I’m sorry if there is a mislead about my article. Thanks for reading it 😁
Link to Github
Order Service: https://github.com/nurcahyaari/learn-microservice-ecommerce-order-service
Cart Service: https://github.com/nurcahyaari/learn-microservice-ecommerce-cart-service
Product Service: https://github.com/nurcahyaari/learn-microservice-ecommerce-product-service
References:
