Proxying and Load Balancing requests to Tomcat

Sample Number

111

Level

Intermediary

Description

This sample demonstrates the usage of UltraESB to proxy and load balance requests to a cluster of Tomcat servers

Use Case

Traditionally the Apache web server has been used to proxy requests to Tomcat servers using mod_jk or mod_proxy etc. This usually entails configuring multiple configuration files at different locations after making the necessary modules available to Apache. This sample describes how the UltraESB can be used to load balance and fail-over requests to Tomcat servers, and compares the configuration with a similar configuration with Apache2 using mod_jk.

The requirement is to load balance requests from clients between multiple Tomcat instances that host distributable web applications performing session replication. However, sticky sessions are desired so that a session fail-over will only take place on an unexpected failure of a Tomcat node.

A typical deployment using Apache2 Web Server with mod_jk
mod jk
An equivalent deployment using the UltraESB
mod jk with ultraesb

Sample Configuration

Configuration of Tomcat instances

For this example, we used two Tomcat 6.X servers on an Ubuntu platform, and configured session replication and clustering as per article: http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html

The Tomcat1 server listens for HTTP on port 8080 and for AJP on port 8009, while Tomcat2 listens for HTTP on port 8081 and for AJP on port 8010. The server.xml for the Tomcat1 can be found here, while the server.xml for Tomcat2 can be found here. Note that we specified "tomcat1" as the jvmRoute for the first server, and "tomcat2" as the jvmRoute for the second server.

<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

The above appends this "jvmRoute" to the JSESSIONID cookie or the jsessionid path parameter when using URL re-writing, to allow stick sessions. Then we configured the Tomcat example application to use session replication by specifying the <distributable/> tag into the end of the web.xml (found at webapps/examples/WEB-INF) as follows:

<web-app .....
    .....
    <distributable/>
</web-app>

You could now start the Tomcat instances, and directly access them at http://localhost:8080/examples To test session replication, the following scenario could help.

Access http://localhost:8080/examples/servlets/servlet/SessionExample this will return a page where you could set a session attribute. If you set attribute "a" as say "100", you will be returned to a page such as shown below. Note that the URL now contains the jvmRoute id of the Tomcat instance that handled the request and established the session. Since we accesses Tomcat1 over port 8080, we can see ".tomcat1" at the end of the session id as shown below.

http://localhost:8080/examples/servlets/servlet/SessionExample;jsessionid=711442A37BE8AF883ED0237BF10219CF.tomcat1

If you now access Tomcat2 at http://localhost:8081/examples/servlets/servlet/SessionExample you will still see the attribute a=100 since the session has been replicated between the instances. Note that configuring details for Tomcat is out of scope for this article.

Configuration of the UltraESB

The example scenario can be found in the configuration file at samples/conf/ultra-sample-111.xml, and the salient point of this configuration is shown below. The Proxy service "web-proxy" defines the URL pattern it accepts as "*" and thus handles any request arriving over its transport "http-8280" configured to listen on HTTP port 8280. Once a request arrives, the mediation.getJvmRoute() method extracts the jvmRoute of a JSESSIONID cookie or jsessionid path segment, or returns null if one is not found. We then use conditional routing to send the request to endpoint definition "tc1-failover" or "tc2-failover" depending on the instance holding the active session, and if a session is not found, we use round-robin load balancing to select between one of the instances.

 1<u:proxy id="web-proxy">
 2    <u:transport id="http-8280">
 3        <u:property name="ultra.transport.url" value="*"/>
 4    </u:transport>
 5    <u:target>
 6        <u:inSequence>
 7            <u:java><![CDATA[
 8                String jvmRoute = mediation.getHTTPSupport().getJvmRoute(msg);
 9                logger.debug("JVM Route : {}", jvmRoute);
10                if ("tomcat1".equals(jvmRoute)) {
11                    mediation.sendToEndpoint(msg, "tc1-failover");
12                } else if ("tomcat2".equals(jvmRoute)) {
13                    mediation.sendToEndpoint(msg, "tc2-failover");
14                } else {
15                    mediation.sendToEndpoint(msg, "round-robin-loadbalance");
16                }
17            ]]></u:java>
18        </u:inSequence>
19        <u:outDestination>
20            <u:address type="response"/>
21        </u:outDestination>
22    </u:target>
23</u:proxy>
24
25<!--Always forwards to http://localhost:8080 and if the endpoint fails, forwards to http://localhost:8081  -->
26<u:endpoint id="tc1-failover" type="fail-over">
27    <u:address type="prefix">http://localhost:8080</u:address>
28    <u:address type="prefix">http://localhost:8081</u:address>
29    <u:property name="ultra.endpoint.switch_location_headers_to" value="http://localhost:8280/"/>
30</u:endpoint>
31
32<!--Always forwards to http://localhost:8081 and if the endpoint fails, forwards to http://localhost:8080  -->
33<u:endpoint id="tc2-failover" type="fail-over">
34    <u:address type="prefix">http://localhost:8081</u:address>
35    <u:address type="prefix">http://localhost:8080</u:address>
36    <u:property name="ultra.endpoint.switch_location_headers_to" value="http://localhost:8280/"/>
37</u:endpoint>
38
39<!--Forwards to either http://localhost:8080 or http://localhost:8081 using a round-robin policy -->
40<u:endpoint id="round-robin-loadbalance" type="round-robin-with-fail-over">
41    <u:address type="prefix">http://localhost:8081</u:address>
42    <u:address type="prefix">http://localhost:8080</u:address>
43    <u:property name="ultra.endpoint.switch_location_headers_to" value="http://localhost:8280/"/>
44</u:endpoint>

The "tc1-failover" endpoint performs fail-over processing, and always selects the first available address to forward the message to. If this endpoint fails, the next available address in the list is selected. Thus the "tc1-failover" always prefers Tomcat1 server first, while "tc2-failover" always prefers Tomcat2 server first. If the preferred instance fails, the request is sent to the other. The "round-robin-loadbalance" endpoint is a round-robin load balancer with fail-over, and is used to allocate a request to a node using round-robin policy. If an address on this fails, the request fails-over to the next. The UltraESB supports weighted, and random load balancing with and without failover. Finally, the switchLocationHeadersTo property of the endpoints instruct the UltraESB to re-write any Location headers (e.g. when used with REST or RESTful calls etc) to point to the UltraESB, instead of the individual Tomcat instances.

Advanced options

Although not shown in this example, the UltraESB allows HTTP level failures, SOAP faults etc from a server etc to cause a fail-over to another instance; as well as validation of a successful response by a custom ResponseValidator - before accepting it as a valid response to be sent back to the client. This allows the UltraESB to detect for example that a HTTP 200 response that states "Service is not active" to be identified as a failure and fail-over the request to another instance. Additionally, the UltraESB could easily read or write cookies, HTTP headers etc much easily.

Configuration of the Apache2 Web Server

On Ubuntu 9.04, installing Apache2 with mod_jk required the following commands:

sudo apt-get install apache2
sudo apt-get install libapache2-mod-jk

Next we created a mod_jk.conf and workers.properties file at /etc/apache2, and edited the /etc/apache2/apache2.conf to comment Virtual hosts, and include the mod_jk.conf at the very end as shown below.

#Include /etc/apache2/sites-enabled/
Include /etc/apache2/mod_jk.conf

Testing the scenarios

Both the Apache2 proxying, and the UltraESB proxying performs load balancing and failover with sticky sessions as expected. To test against the Apache2 frontend, access http://localhost/examples/servlets/servlet/SessionExample and to test against the UltraESB, test against the URL http://localhost:8280/examples/servlets/servlet/SessionExample In both cases, killing one of the Tomcat servers owning the session will failover to the other instance without loss of session contents.

To start the sample # 111 configuration of the UltraESB, start it as follows, and start the two Tomcat servers as described above.

user@host:~/java/ultraesb-1.7.0/bin$ ./ultraesb.sh -sample 111

To make the testing more meaningful, you may edit the SessionExample Servlet at webapps/examples/WEB-INF/classes/SessionExample.java to report the Tomcat instance name in the heading as follows:

sessions

Now, if the original request was handled by "tomcat1", kill that instance, and retry the request - and you will be taken to the Tomcat2 instance as follows:

sessions2
In this topic
In this topic
Contact Us