What is Scatter-Gather in Mule 4?
Scatter-Gather is a routing event processor that process the different business logic on same given payload in different routes and aggregates all the individually processed mule events into new mule event.
Scatter-Gather is very useful when you want to process a mule event simultaneously across multiple processors independently.
The following diagrams details how Scatter-Gather works:
Behind the scenes:
Scatter-Gather router receives a mule event and sends a reference/copy of mule event to each route simultaneously.
In our case, Scatter-Gather sends mule event to route-1, route-2 and route-3 parallel.
All the routes starts processing the mule event in parallel. After all routes finish the processing then returns a mule event.
The event can be a same mule event or a new event created by the processors in the route.
After all the routes finish their processing, the Scatter-Gather router combines the all resulting Mule events from each route and create a new mule event then passes the mule event to next processor.
Scatter-Gather Example
Now let’s create a sample flow with Scatter-Gather router and see how the router works in detail.
Step 1: Add HTTP Listener and configure as shown below or with your own parameters.
Global configuration
General configuration
Step 2: Add a Set Variable and set a variable called name with value Vanchiv. This is to see how variable propagation works in Scatter-Gather.
Step 3: Now add the Scatter-Gather router next after set variable node.
Scatter-Gather router config:
First let’s configure the global HTTP Requester which we will be calling inside the Scatter-Gather router.
We have added only one parameter Host for demo purpose but in your case the configuration might be different.
Route-1:
Add HTTP Requester node and refer the global HTTP Requester configuration and then config the General parameters.
Add Set Variable and override the variable the we set before the Scatter-Gather router.
We will see how the variable propagation works later.
Route-2:
Add HTTP Requester node and refer the global HTTP Requester configuration and then config the General parameters.
Add Set Variable and override the variable the we set before the Scatter-Gather router.
The Scatter-Gather router should look like:
Step 4: Add Transform Message and simply transform the object to JSON model.
Step 5: At last add the logger and log the variable “name“.
That’s it. Now run the mule application and trigger a request to http://localhost:8089/scatterGather.
Response Payload
Scatter-Gather returns the payload as an Object. The sample output would look like:
{
"0": {
"inboundAttachmentNames": [],
"exceptionPayload": null,
"inboundPropertyNames": [],
"outboundAttachmentNames": [],
"payload": {
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
},
"outboundPropertyNames": [],
"attributes": {
"headers": {
"date": "Sun, 21 Feb 2021 05:42:24 GMT",
"content-type": "application/json; charset=utf-8",
"content-length": "83",
"connection": "keep-alive"
},
"reasonPhrase": "OK",
"statusCode": 200
}
},
"1": {
"inboundAttachmentNames": [],
"exceptionPayload": null,
"inboundPropertyNames": [],
"outboundAttachmentNames": [],
"payload": {
"userId": 1,
"id": 2,
"title": "quis ut nam facilis et officia qui",
"completed": false
},
"outboundPropertyNames": [],
"attributes": {
"headers": {
"date": "Sun, 21 Feb 2021 05:42:24 GMT",
"content-type": "application/json; charset=utf-8",
"content-length": "99"
},
"reasonPhrase": "OK",
"statusCode": 200
}
}
}Note: We have removed few headers to keep the response simple.
The above response contains the attributes and the payload but most of the times the attributes are not required.
In such cases, to extract the only payload you can use flatten function to merge the both the responses.
To do that, replace the above dataweave script as shown below:
The response would look like after the above transformation:
Point to Rememeber:
Scatter-Gather returns the mule event as an Object.
Use flatten function to merge the both responses.
Variable Propagation
Now let’s see what happens to variables when we override the variables inside Scatter-Gather router.
Did you observe the variable propagation here?
Every routes start with the initial values and modifications to a variable within a specific route do not affect other routes.
So, if a variable is added or modified in one route, then, after aggregation, the value is defined by that route.
Concurrency Control
By default the Scatter-Gather process the mule event in parallel but sometimes we may need to process the mule event sequentially.
To achieve the sequential process, we can use two methods:
- Using maxConcurrency element
- Using Try scope
Using max Concurrency
This feature is very handy and can implement easily.
Click on Scatter-Gather router and set max Concurrency is equal to 1 as shown below:
Using Try Scope
This method is little tricky where you will need to wrap all the processors inside Try scope and set Transaction action to ALWAYS_BEGIN.
In this case, irrespective of maxConcurrency element the processing is always sequentially.
We hope this tutorial helped you to learn and understand the concept of Scatter-Gather in Mule 4 with few tips and tricks. For detailed information, read the official documentation.