Limiting Concurrent Requests

Concurrency Throttling for incoming requests

Use Case Description

An organization wishes to spread processing across two servers. One is a high end server with a very fast processing and other is a low end server with relatively slow processing. However high end server can only support a low amount of concurrent requests compared to the low end server which can support a large amount of concurrent requests. Ideally the organization want to provide results to the end user in lowest possible latency.

Proposed Solution

Since the organization wants the processed results to be returned in the least possible time, they should try to use fast processing sever for all the requests but they should also make sure that fast processing server is not overloaded. They decided to use an ESB to fulfill this requirements.

External application will forward the processing requests to the HTTP web service exposed by the ESB those will go through a Concurrency Throttle. The throttle which monitors the number concurrent messages will only allow a configured number of messages to go through to the fast processing endpoint and rest will be directed to slow processing endpoint.