HTTP and HTTPS Transport Guide

This guide describes the HTTP and HTTPS transport configuration options.

HttpNIOListener

The HttpNIOListener is the HTTP transport listener based on the Apache HttpComponents - HttpCore NIO project. It uses Java Non-Blocking IO (NIO) to process thousands of concurrent connections efficiently by breaking away from the thread-per-socket model used by Servlet containers etc. Hence, a very few dedicated IO threads will read/write messages from thousands of concurrent users, and delegate to a worker thread pool for actual message processing through the UltraESB. This way the worker threads are guaranteed not to block on IO - be it waiting for responses from remote servers, or reading/writing against slow clients etc. The HttpNIOListener constructor accepts a reference to the FileCache implementation used. For production systems the RAM disk based file cache is recommended, while for testing purposes, the older Memory mapped file cache could be used. The supported properties for configuration are as follows:

Name Description

port

The HTTP listen port

bindAddress

Specify the specific bind address over which to listen for requests. By default the transport will bind on all interfaces

replaceOriginServer

When set to false, will prevent the UltraESB from writing its product name and version as the HTTP origin server

defaultContextPath

The default context path for services exposed over the HTTP transport. This defaults to "/service"

noCompression

When set to true, turns off response GZip compression even when the client user agent indicates its supported

compressionMinSize

The minimum response size to turn on response GZip compression if client supports it (defaults to 2048L)

requestFilters

A List of request filters to intercept messages (e.g. ServiceResourceRequestFilter, BasicAuthenticationFilter, DigestProcessingFilter or any custom implementation of the RequestFilter interface)

properties

Additional HTTP listener properties such as "http.socket.timeout" which specifies the default listen socket timeout

zeroCopyEnabled

When set to true, enables Zero-Copy proxying (Turned off by default)

enableConnectionDebug

Turn on very detailed connection debugging information on connection level failures

connectionDebugHeaders

A list of HTTP headers to extract into log files (’all’ - dump all headers, 'none' - skip all headers)

stopNewConnectionsAt

Limit total number of concurrent connections (for the prevention of a DOS attack - defaults to 4096)

resumeNewConnectionsAt

After a DOS attack is detected and new connections are stopped, wait till the total active connections reduce to this level before new connections are resumed (default 3073)

maxPayloadSize

Maximum payload size accepted (for the prevention of a DOS attach - defaults to 1048576L)

continueOnRuntimeExceptions

Continue execution even on RuntimeExceptions or restart IO reactor (Usually these are caused by user written code - defaults to true)

continueOnCheckedExceptions

Continue execution on checked exceptions or restart IO reactor (Usually caused by serious IO errors - defaults to false)

autoRestartAttempts

Number of attempts to auto-restart the IO reactors on failures before giving up

The HttpNIOListsner optionally allows the following advanced properties to be tuned - however any property other than the 'http.socket.timeout' should only be altered after specific confirmation from AdroitLogic.

  • http.socket.timeout - TCP socket timeout (defaults to 120000 ms)

  • http.lst_io_threads - defaults automatically to the number of CPU cores available on the system (this is strongly recommended)

  • http.socket.buffer-size - socket buffer size (defaults to 8K)

  • http.tcp.nodelay - control the use of the Nagle algorithm (defaults to true)

  • http.connection.stalecheck - check connections before re-use (defaults to false)

  • http.socket.linger

  • http.connection.max-line-length

  • http.connection.max-header-count

HttpsNIOListener

The HttpsNIOListener is the HTTPS variant of the HTTP protocol transport listener. It extends the HttpNIOListener, and thus inherits all of its properties, and exposes the following additional properties for configuration.

Name Description

sslVerifyClient

One of 'none', 'optional' or 'require'. Defaults to 'none', which implies that client authentication (i.e. 2-way SSL authentication) is disabled. If set to 'optional' the client may authenticate itself, and if set to 'require' a request will not be accepted from an unauthenticated client.

identityStorePath

The path to the identity keystore to be used. See 'conf/keys/identity.jks' for a sample identity keystore

identityStoreType

The Java keystore type of the identity store. Defaults to "JKS"

identityKeyPassword

The private key password for the identity certificate

identityStorePassword

The keystore password for the identity keystore

trustStorePath

The path to the trust keystore to be used. See 'conf/keys/trust.jks' for a sample trust keystore. The trust store is used to validate the credentials of connecting clients when 2-way authentication is used. The default trust store is a copy of the JDK 1.8.x trust store. If a trust store is not specified, it will default to the JDK trust store.

trustStoreType

The Java keystore type of the trust store. Defaults to "JKS"

trustStorePassword

The keystore password for the trust keystore

nonProductionNoRemoteCertValidation

If set to true, this will disable remote certificate validation - but this setting should NEVER be used in a production deployment.

tlsProtocolsToEnable

TLS protocol names to enable eg: TLS, TLSv1

cipherSuitesToEnable

Cipher suites to enable. If this value is set then all other cipher suites will be disabled, otherwise default

HttpNIOSender

The HttpNIOSender is the HTTP request sender of the UltraESB. It too accepts a FileCache instance in its constructor, similarly to the HttpNIOListener (described above) and exposes the following configuration options.

Name Description

keepalive

The default keepalive time for re-usable outgoing connections, in ms (defaults to 30000 - usually overridden by Endpoint/Address)

keepAliveSafetyThreshold

The safety threshold in ms to leave at the end of the keep-alive duration before a connection would be re-used (defaults to 5000 - usually overridden by Endpoint/Address)

replaceUserAgent

Will not use the UltraESB product identifier as the HTTP User Agent if this flag is set to false. By default this is set to true.

unzipResponseEntities

Default to true. When turned off, the transport will not unzip compressed response entities, and could be enabled when the response is not read/used in the outSequence to further improve performance.

proxyHost

The HTTP Proxy server hostname to use

proxyPort

The HTTP Proxy server port to use

proxyBypassList

A list of hostnames for which the HTTP Proxy should be bypassed

continueOnRuntimeExceptions

Continue execution even on RuntimeExceptions or restart IO reactor (Usually these are caused by user written code - defaults to true)

continueOnCheckedExceptions

Continue execution on checked exceptions or restart IO reactor (Usually caused by serious IO errors - defaults to false)

enableConnectionDebug

Turn on very detailed connection debugging information on connection level failures

connectionDebugHeaders

A list of HTTP headers to extract into log files (’all’ - dump all headers, 'none' - skip all headers)

zeroCopyEnabled

When set to true, enables Zero-Copy proxying (Turned off by default)

properties

Additional HTTP sender properties such as "http.socket.timeout" which specifies the default send socket timeout

The HttpNIOSender optionally allows the following advanced properties to be tuned - however any property other than the ’http.socket.timeout’ should only be altered after specific confirmation from AdroitLogic.

  • http.socket.timeout - TCP socket timeout (defaults to 120000 ms)

  • http.connection.timeout -TCP connection establishment timeout (defaults to 10 s) - i.e. maximum time until a SYN packed is acknowledged by the remote server

  • http.snd_io_threads - defaults automatically to the number of CPU cores available on the system (this is strongly recommended)

  • http.socket.buffer-size - socket buffer size (defaults to 8K)

  • http.tcp.nodelay - control the use of the Nagle algorithm (defaults to true)

  • http.connection.stalecheck - check connections before re-use (defaults to false)

  • http.socket.linger

HttpsNIOSender

The HttpsNIOSender is the HTTPS variant of the HTTP protocol transport sender. It extends the HttpNIOSender, and thus inherits all of its properties, and exposes the following additional properties for configuration

Name Description

hostnameVerifier

One of 'Strict', 'AllowAll', 'DefaultAndLocalhost' or 'Default' (i.e. the default). See below for more information.

nonProductionNoRemoteCertValidation

If set to true, this will disable remote certificate validation - but this setting should NEVER be used in a production deployment.

identityStorePath

Same as per the HttpsNIOListener

identityStoreType

Same as per the HttpsNIOListener

identityStorePassword

Same as per the HttpsNIOListener

identityKeyPassword

Same as per the HttpsNIOListener

trustStorePath

Same as per the HttpsNIOListener

trustStoreType

Same as per the HttpsNIOListener

trustStorePassword

Same as per the HttpsNIOListener

tlsProtocolsToEnable

TLS protocol names to enable eg: TLS, TLSv1

cipherSuitesToEnable

Cipher suites to enable. If this value is set then all other cipher suites will be disabled, otherwise default

The 'Strict' HostnameVerifier works the same way as java.net.URL in Sun Java 1.4, Sun Java 5, Sun Java 6. It’s also pretty close to IE6. The hostname must match either the first CN, or any of the subject-alts. A wildcard can occur in the CN, and in any of the subject-alts. A wildcard such as "*.foo.com" matches only subdomains in the same level, for example "a.foo.com". It does not match deeper subdomains such as "a.b.foo.com".

The 'Default' HostnameVerifier works the same way as Curl and Firefox. The hostname must match either the first CN, or any of the subject-alts. A wildcard can occur in the CN, and in any of the subject-alts. The only difference between DEFAULT and STRICT is that a wildcard (such as "*.foo.com") with DEFAULT matches all subdomains, including "a.b.foo.com".

The 'AllowAll' HostnameVerifier essentially turns hostname verification off completely, while the 'DefaultAndLocalhost' HostnameVerifier works like the 'Default' one with one additional relaxation: a host of "localhost", "localhost.localdomain", "127.0.0.1", "::1" will always pass, no matter what is in the server’s certificate.

See the Javadocs of the org.adroitlogic.ultraesb.transport.http.util.HostnameVerifier for more details or refer the original code by Julius Davies at [http://juliusdavies.ca/commons-ssl/src/java/org/apache/commons/ssl/HostnameVerifier.java]

HttpBlockingSender

The HttpBlockingSender is a HTTP transport sender using apache http client. This uses the PoolingClientConnectionManager to manage connection pooling and provides statistics for each connection based on the PoolingClientConnectionManager’s internal stats

To use HttpBlockingSender you have to define a blocking sender bean as follows, and when you define an address in the in-destination section of a proxy service or in an endpoint, you have you use that HttpBlockingSender bean id in-front of the address in curly brackets.

Defining HttpBlockingSender

<bean id="blocking-sender" class="org.adroitlogic.ultraesb.transport.http.HttpBlockingSender">
    <constructor-arg ref="fileCache"/>
</bean>

Using HttpBlockingSender in Addresses

<u:inDestination>
    <u:address>{blocking-sender}http://localhost:8281/service/sample01</u:address>
</u:inDestination>
<u:endpoint id="sample-endpoint">
    <u:address>{blocking-sender}http://localhost:8281/service/sample02</u:address>
</u:endpoint>
Once you have a blocking http sender defined along with a non-blocking http sender, in all the addresses you have to define which sender to be used in curly braces as shown above.
Name Description

zeroCopyEnabled

When set to true, enables Zero-Copy proxying (Turned off by default)

replaceUserAgent

Will not use the UltraESB product identifier as the HTTP User Agent if this flag is set to false. By default this is set to true.

enableConnectionDebug

Turn on very detailed connection debugging information on connection level failures

connectionDebugHeaders

A list of HTTP headers to extract into log files (’all’ - dump all headers, 'none' - skip all headers)

unzipResponseEntities

Default to true. When turned off, the transport will not unzip compressed response entities, and could be enabled when the response is not read/used in the outSequence to further improve performance.

continueOnRuntimeExceptions

Continue execution even on RuntimeExceptions or restart IO reactor (Usually these are caused by user written code - defaults to true)

continueOnCheckedExceptions

Continue execution on checked exceptions or restart IO reactor (Usually caused by serious IO errors - defaults to false)

maxConnectionPerRoute

Maximum connections per route

maxConnectionTotal

Maximum connections in total

tlsProtocolsToEnable

TLS protocol names to enable eg: TLS, TLSv1

cipherSuitesToEnable

Cipher suites to enable. If this value is set then all other cipher suites will be disabled, otherwise default

HttpsBlockingSender

The HttpsBlockingSender is the HTTPS variant of the HTTP blocking transport sender. It extends the HttpBlockingSender, and thus inherits all of its properties, and exposes the following additional properties for configuration

Name Description

hostnameVerifier

Same as per the HttpsNIOSender

nonProductionNoRemoteCertValidation

Same as per the HttpsNIOSender

identityStorePath

Same as per the HttpsNIOListener

identityStoreType

Same as per the HttpsNIOListener

identityStorePassword

Same as per the HttpsNIOListener

identityKeyPassword

Same as per the HttpsNIOListener

trustStorePath

Same as per the HttpsNIOListener

trustStoreType

Same as per the HttpsNIOListener

trustStorePassword

Same as per the HttpsNIOListener

Message Properties
Constant HttpConstants. Value Description

REMOTE_ADDRESS

ultra.http.conn_remote_addr 

Remote IP address

REMOTE_PORT

ultra.http.conn_remote_port 

Remore port

LOCAL_ADDRESS

ultra.http.conn_local_addr 

Local Interface IP address

LOCAL_PORT

ultra.http.conn_local_port 

Local listen port

USERNAME

ultra.http.username 

Authenticated user name

USERROLES

ultra.http.userroles 

User roles as a String

USERROLES_ARRAY

ultra.http.userroles.array 

User roles as an array

SSL_CLIENT_DN

ultra.http.ssl_client_dn 

SSL client certificate DN

SSL_CLIENT_CERTS

ultra.http.ssl_client_certs 

SSL client certificates

METHOD

ultra.http.method 

HTTP method

SERVICE_URL

ultra.http.service_url 

Service URL for the incoming message

QUERY_STRING

ultra.http.query_string 

Query string for the incoming message

QUERY_PARAM_MAP_WDUPS

ultra.http.query_param_map_wdups 

See Javadoc

FORWARD_URL_POSTFIX

ultra.http.forward_url_postfix 

See Javadoc

HEADER_SIZE

ultra.http.header_size 

Size of the received HTTP header request or response

ENTITY_SIZE

ultra.http.entity_size 

Size of the received HTTP entity request or response

MESSAGE_SIZE

ultra.http.message_size 

Size of the received HTTP message request or response

RESPONSE_STATUS_CODE

ultra.http.response_status_code 

Response status code received

RESPONSE_TIME

ultra.http.response_time 

Last response time

AUTH_USERNAME

ultra.http.auth_username 

Authentication username to be used for an outgoing message

AUTH_PASSWORD

ultra.http.auth_password 

 

In this topic
In this topic
Contact Us