A Splitter is useful to break out a single message into a sequence of sub-messages that can be processed individually. Likewise, a Recipient List or a Publish-Subscribe Channel is useful to forward a request message to multiple recipients in parallel in order to get multiple responses to choose from. In most of these scenarios, the further processing depends on successful processing of the sub-messages. For example, we want to select the best bid from a number of vendor responses or we want to bill the client for an order after all items have been pulled from the warehouse. How do we combine the results of individual, but related messages so that they can be processed as a whole?
Use a stateful filter, an Aggregator, to collect and store individual messages until a complete set of related messages has been received. Then, the Aggregator publishes a single message distilled from the individual messages. The Aggregator is a special Filter that receives a stream of messages and identifies messages that are correlated. Once a complete set of messages has been received (more on how to decide when a set is 'complete' below), the Aggregator collects information from each correlated message and publishes a single, aggregated message to the output channel for further processing. Unlike most of the previous routing patterns, the Aggregator is a stateful component. Simple routing patterns like the Content-Based Router are often stateless, which means the component processes incoming messages one-by-one and does not have to keep any information between messages. After processing a message, the component is in the same state as it was before the message arrived. Therefore, we call such a component stateless. The Aggregator cannot be stateless since it needs to store each incoming message until all the messages that belong together have been received. Then, it needs to distill the information associated with each message into the aggregate message. The Aggregator does not necessarily have to store each incoming message in its entirety. For example, if we are processing incoming auction bids, we may only need to keep the highest bid and the associated bidder ID without having to keep the history of all individual bid messages. Still, the Aggregator has to store information across messages and is therefore stateful. When designing an Aggregator, we need to specify the following items:
Example: Serverless Loan Broker on AWSNEWThe modern implementation of the Loan Broker using AWS serverless constructs includes an implementation of a Aggregator with Lambda and DynamoDB. The Aggregator uses a simple completeness condition of waiting for a minimum number of answers. The surrounding workflow, implemented using AWS Step Functions, adds a time-out component. Example: Serverless Loan Broker on GCPNEWThe modern implementation of the Loan Broker using GCP includes an implementation of a Aggregator with a Cloud Function and Datastore. The implementation stores all elements of a specific Aggregate under the same key and can therefore fail when inserting the record (Datastore doesn't seem to like heavy contention on one record). It therefore configures retry logic on the incoming messages. The Aggregator signals back to the Process Manager on completion. Related patterns: Scatter-Gather, Introduction to Composed Messaging Examples, Content-Based Router, Control Bus, Correlation Identifier, Composed Message Processor, Event-Driven Consumer, Event Message, Guaranteed Delivery, Message Expiration, Point-to-Point Channel, Process Manager, Publish-Subscribe Channel, Recipient List, Resequencer, Splitter, Transactional Client Integration Pattern Language
|
Want to keep up-to-date? Follow My Blog.
Want to read more in depth? Check out My Articles.
Want to see me live? See where I am speaking next.
 Find the full description of this pattern in: |  From Enterprise Integration to Enterprise Transformation: My new book describes how architects can play a critical role in IT transformation by applying their technical, communication, and organizational skills with 37 episodes from large-scale enterprise IT. |
 Parts of this page are made available under the Creative Commons Attribution license. You can reuse the pattern icon, the pattern name, the problem and solution statements (in bold), and the sketch under this license. Other portions of the text, such as text chapters or the full pattern text, are protected by copyright. |
| © 2003, 2019 • Bobby Woolf • All rights reserved. |
