Microservice Patterns - Managing transactions with sagas - 4.3 Handling the lack of isolation

Reference

https://www.amazon.com/Microservices-Patterns-examples-Chris-Richardson/dp/1617294543

• The isolation property of ACID transactions ensures that the outcome of executing multiple transactions concurrently is the same as if they were executed in some serial order
• Isolation makes it a lot easier to write business logic that executes concurrently
• The challenge with using sagas is that they lack the isolation property of ACID transactions.
• Problem
• other sagas can change the data accessed by the saga while it’s executing
• other sagas can read its data before the saga has completed its updates, and consequently can be exposed to inconsistent data
• saga to be ACD:
• Atomicity—The saga implementation ensures that all transactions are executed or all changes are undone.
• Consistency—Referential integrity within a service is handled by local databases. Referential integrity across services is handled by the services.
• Durability—Handled by local databases.
• Strategy to deal with the lack of isolation:  countermeasures

Overview of anomalies

• Lost updates: Update a data and another Saga override the same data right away
• Dirty reads: Read a data while another Saga update that data but not completed
• Fuzzy/nonrepeatable reads: Different step of Saga read same data get different result because other saga has made update

Countermeasures for handling the lack of isolation

• Semantic lock: application level lock
• Commutative updates: Design update operations to be executable in any order
• Pessimistic view: Reorder of saga steps to minimize business risk due to dirty read
• Reread value: Reread data to verify it's unchanged before overwriting it
• Version file: Record the update to a record so that they can be reordered. Ex. record the operations as they arrive and then execute them in the correct order
• By value: Use request's business risk to dynamically to select the concurrency mechanism. Ex. transfer money, use distributed transaction. Simple status update, use Sagas.

THE STRUCTURE OF A SAGA

a saga consists of three types of transactions (One complete flow may have every kinds of transactions):

• Compensatable transactions:  can potentially be rolled back. Ex. Delete a created Order
• Pivot transaction:  The go/no-go point in a saga. Ex. verify before proceed, go or no-go after verify.
• Retriable transactions: Follow pivot transaction but guarantee to succeed. Ex. Can retry when fail.

Microservice Patterns - Managing transactions with sagas - 4.2 Coordinating sagas

Reference

https://www.amazon.com/Microservices-Patterns-examples-Chris-Richardson/dp/1617294543

Sagas process

1. Saga init by system command
2. Coordination logic select and tell the first Saga participant to execute local transaction
3. After finish, Saga's sequencing coordination select and invoke next Saga participant
4. Continue until all steps executed
5. If any local transaction fail, Saga must execute the compensating transaction in reverse order.

Ways to construct Saga's Coordination logic

• Choreography - Distribute decision making and sequence among the Saga participants. Primary communicate by exchanging events.
• Orchestration - Centralize coordination logic in a Saga Orchestration class. Saga orchestrator send command to Saga participants to tell them which operation to perform.

Choreography-based sagas

• saga participants subscribe to each other’s events and respond accordingly

IMPLEMENTING THE CREATE ORDER SAGA USING CHOREOGRAPHY

Each participant updates its database and publishes an event that triggers the next participant

RELIABLE EVENT-BASED COMMUNICATION

Need consider following items when implementing choreography-based sagas:

1. Ensure saga participant update database and publish event as part of database transaction => Atomically.
   To communicate reliably, the saga participant must use transactional messaging.
2. Saga participant must be able to map each event that it receives to its own data.
   => When Order Service receives a event, it must be able to look up the corresponding Order.
   => Solution: Publish event containing a correlation id, used to enable other participants to perform mapping.
   => Ex. Order Id

BENEFITS AND DRAWBACKS OF CHOREOGRAPHY-BASED SAGAS

• Benefit
• Simplicity: Service publish its own events
• Loose Coupling: Each service communicate through events, don't have direct knowledge of each other
• Drawback
• Difficult to understand: Unlike orchestration, no single place to define Saga. Difficult for a developer to understand how Sagas work.
• Cyclic dependencies between Services
• Risk of tight coupling
• Because of these drawbacks, more complex Sagas use orchestration.

Orchestration-based sagas

1. orchestrator class whose sole responsibility is to tell the saga participants what to do
2. orchestrator communicates with the participants using command/ async reply-style interaction
3. it sends a command message to a participant telling it what operation to perform.
4. After the saga participant has performed the operation, it sends a reply message to the orchestrator
5. orchestrator then processes the message and determines which saga step to perform next

MODELING SAGA ORCHESTRATORS AS STATE MACHINES

• A good way to model a saga orchestrator is as a state machine

BENEFITS AND DRAWBACKS OF ORCHESTRATION-BASED SAGAS

• Simpler dependencies
• Less coupling
• Improves separation of concerns and simplifies the business logic

DRAWBACK

the risk of centralizing too much business logic in the orchestrator

Microservice Patterns - 4.1 Managing transactions with sagas - Transaction management in a microservice architecture

Reference

https://www.amazon.com/Microservices-Patterns-examples-Chris-Richardson/dp/1617294543

Challenges

In a monolithic system, a createOrder operation only need to rely on a relational database with transactional annotation.

In a microservice system, a createOreder operation need make sure operations in Kitchen Service, Customer Service, Order Service, Accounting Service all success. Because each service has its own database.

Traditional Approach

Simply to say: two-phase commit. Ex. Java EE can complete multiple service transaction by JTA.

Problem

1. Modern technology such as No-SQL database or Message broker such as Kafka doesn't support it.
2. Traditional approach is synchronous operation with lower availability (all services must be available)
   2 Services with 99.5% aviliability, overall availability become 99%

Using the Saga pattern to maintain data consistency

Saga: Maintain data consistency across services using a sequence of local transactions that are coordinated using asynchronous messaging. See http://microservices.io/ patterns/data/saga.html.

Each local transaction updates data within a single service using the familiar ACID transaction frameworks and libraries mentioned earlier

1. System operation init the first step of Saga
2. Completion of local transaction triggers the execution of next local transaction
3. a saga must be rolled back using compensating transactions

Example: Create Order Saga

1. Init by external createOrder request
2. Other five transactions are triggered after previous done

Challenges

1. Lack of isolations between Sagas
2. Rollback when error occurs

SAGAS USE COMPENSATING TRANSACTIONS TO ROLL BACK CHANGES

When a step fail, previous sagas need undo changes (must be undone)