Enterprise Integration Patterns
Conversation Patterns
Subscribe-NotifyConversation Patterns » Basic Conversations

Multi-responses, Streamed responses

A participant wants to retrieve information from another participant.

How can one participant receive information from another participant if that information cannot easily be packaged into a single message?

  • A participant could try to aggregate as many individual messages into a larger message as possible, but most messaging systems have size limits for messages.
  • Large messages can overburden the receiver who has to parse the message -- an operation typically performed in memory. Especially parsing large XML messages, which are still in widespread use, tax the receiving system's memory and CPU resources.
  • The individual data to be transmitted may be time sensitive. Storing and aggregating it into a single message would reduce the data's value for the recipient.
  • The response data may not be a finite set of data but a continuous stream of data, which would make it impossible to know when to bundle up messages.

The interested participant, or Subscriber, initiates a Subscribe-Notify conversation by sending a message to the Provider that expresses its interest in data the Provider can supply. The Provider subsequently keeps sending notification messages to the Subscriber until a stop condition is reached.

The Subscribe-Notify conversation involves the following participants:

  • The Subscriber initiates the conversation by sending a Subscribe message that expresses interest in receiving notification messages. The interest may be defined by the provider's address or the Subscribe message can contain information as to what messages the Subscriber would like to receive, e.g. through a list of Topics.
  • The Provider waits for incoming Subscribe messages and delivers a stream of Notify messages to the Subscriber at the pace in which data becomes available or events occur.

Subscribe-Notify requires the originator, i.e. the Subscriber to receive inbound messages, which can be a challenge in situations where it sits behind a firewall that allows only outbound connections. In such cases, the conversation has to fall back to the initiator using Polling.

Subscribe-Notify makes more efficient use of network resources than Polling. Even if the Subscriber knows the appropriate interval to poll, Subscribe-Notify transmits only half as many messages as a repeated Request-Response, ignoring administrative messages, such as Lease or Renewal Reminder.

Subscribe-Notify is particularly efficient if the Subscriber cannot judge the interval at which interesting events occur, or if this frequency varies widely. As long as no noteworthy events occur, no messages have to be sent.

Subscribe-Notify forces the Subscriber to wait for an update rather than to actively request a status. For slow-changing data streams, new subscribers may want to request the current state immediately instead of waiting for the next regular update. For this purpose, the Subscribe message may be acknowledged with an immediate Notify message to the new subscriber.

Most systems establish an ontology to express interest in specific topics and subtopics, as described in Publish-Subscribe Channel

The Provider may acknowledge the subscription immediately using a Quick Acknowledgment. Some Providers may also provide a list of active subscriptions to the Subscriber.

The Subscribe-Notify conversation does not have a natural stopping point -- notification messages could continue indefinitely. Therefore, a stopping condition is needed. Typical stopping conditions are:

  • The Subscriber sends a stop request in form of an Unsubscribe message.
  • The conversation uses a Lease, causing the conversation to stop after a deadline is reached without the subscriber renewing interest.
  • If the channel is connection-oriented (such as an HTTP connection), the provider stops sending messages when the subscriber does not accept a connection.
  • The provider notifies the subscriber of the end of the transmission.
  • The provider may simply stop sending messages, leaving the subscriber to realize this via a time-out condition.

Reference: http://kodu.ut.ee/~dumas/ServiceInteractionPatterns/Multitransmission.html#pattern:multi-responses

Example: Streaming Results from a Long-running Query

When an application invokes a query that takes a long time to process, some of the results may be available before the query completes. In this case, the provider can start sending partial results before the whole operation completes. The client can start processing the result data sooner, possible providing a better user experience. This approach does not work well if a global operation such as sorting is required, unless of course the data that is fetched is already in the desired order, for example because the database has a clustered index on the data. NoSQL column data stored tend to cluster by the primary key as they use it to distribute data across nodes and because their data sets tend to be too large for global sorts. Therefore, using Subscribe-Notify to retrieve data incrementally works particularly well.

Example: Stock Ticker Subscription

A service may want to receive frequent updates to the price of a traded security. Instead of repeatedly polling, the service should subscribe and receive the price updates as Fire-and-Forget messages.

Example: WS-Eventing

Different approaches: WS-Eventing uses single URL, not meant for pub-sub topics. WS-Notification more complex, includes topic hierarchy.

Example: WS-Notification

Example: JMS

Related patterns: Lease, Polling, Publish-Subscribe Channel, Quick Acknowledgment, Renewal Reminder

Table of Contents
Describing Conversations
Hypermedia State
Conversation Vocabulary
Dynamic Discovery
Advertise Availability
Consult Directory
Leader Election
Starting a Conversation
Three-Way Hand Shake
Acquire Token First
Rotate Tokens
Verify Identity
User Grants Access
Basic Conversations
Asynchronous Request-Response
Request-Response with Retry
Quick Acknowledgment
Load Balancer
Managing Distributed Systems
Resource Management
Incremental State Build-up
Renewal Reminder
Ensuring Consistency
Ignore Error
Compensating Action
Tentative Operation
Coordinated Agreement