FIX Message Proxying

Version: 17.07

Supported Since: 17.01

Use Case Description

I want to proxy FIX messages without allowing direct communication between the actual FIX initiator and acceptor.

Proposed Solution

Instead of direct communication between FIX agents, UltraESB-X can be made to act as the proxy for all FIX messages.

UltraStudio Configuration

UltraESB-X Configuration

Implementation of the Solution

We shall configure a sample compatible with the default example applications (Banzai and Executor) provided with QuickFIX/J distributions. Hence for testing the sample you would need a copy of the QF/J binary distribution. (It should be possible to integrate the sample with other FIX implementations as well, with appropriate configuration changes on the initiator and acceptor sides.)

Prerequisites

FIX Ingress and Egress Connectors should be selected when creating the Ultra project (or added to the project later via Tools → Ultra Studio → Component Registry).

For testing, the QF/J binary distribution can either be downloaded from here or built from sources available via the same.

Implementation

Configurations required by the underlying QF/J socket initiators, acceptors and sessions are exposed at different levels in the UltraESB-X FIX support implementation. [default] configurations used by the initiator and acceptor are usually specified at connector level whereas session-specific ([session]) configurations have to be specified as a hierarchy of Project-X resources. Additional support resources are required for QF/J message storage and logging.

Add the following resource definitions under project.xpml of the created Ultra project:

  1. The message store is used by FIX agents to store and retrieve messages for resend purposes. QF/J allow to store messages in file base (FileMessageStoreFactory) or in a database (JDBCMessageStoreFactory). Here we shall use file-based message stores for both cases.

    For acceptor (ingress connector),

    Type

    Bean

    Class Name

    org.adroitlogic.x.transport.fix.util.FileMessageStoreFactoryImpl

    Resource ID

    acc-fileMsgStore

    fileStorePath

    /tmp/fixTest

    For initiator (egress connector),

    Type

    Bean

    Class Name

    org.adroitlogic.x.transport.fix.util.FileMessageStoreFactoryImpl

    Resource ID

    ini-fileMsgStore

    fileStorePath

    /tmp/fixTest

  2. Similarly, QF/J logging can be file-based or database-backed. Log factories (FileLogFactory or JDBCLogFactory respectively) should be provisioned for each initiator and acceptor. Here we shall use file-based logging.

    For acceptor,

    Type

    Bean

    Class Name

    org.adroitlogic.x.transport.fix.util.FileLogFactoryImpl

    Resource ID

    acc-fileLogFactory

    fileLogPath

    /tmp/fixTest

    For initiator,

    Type

    Bean

    Class Name

    org.adroitlogic.x.transport.fix.util.FileLogFactoryImpl

    Resource ID

    ini-fileLogFactory

    fileLogPath

    /tmp/fixTest

  3. In addition to basic configurations, additional properties that fall under [default] category can be provided to FIX connectors in the form of Map resources.

    For acceptor,

    Type

    Map

    Resource ID

    acc-qfSettings

    Entries

    AcceptorTemplate

    Y

    UseDataDictionary

    N

    DefaultApplVerID

    FIX.5.0

    For initiator,

    Type

    Map

    Resource ID

    ini-qfSettings

    Entries

    LogonTimeout

    6

    DefaultApplVerID

    FIX.5.0

  4. Similarly, the initiator also requires a Map of additional endpoint settings ([session] category) that would be used in establishing the outbound connection:

    Type

    Map

    Resource ID

    ini-endpointSettings

    Entries

    DefaultMarketPrice

    12.30

  5. The acceptor requires a List resource containing FIXSession entries with session-specific properties for each FIX session that has to be established with the initiator (in our case, Banzai). Since we have only one session, we shall define a single FIXSession entity and enclose it in a List resource:

    Type

    Bean

    Class Name

    org.adroitlogic.x.transport.fix.FIXSession

    Resource ID

    acc-BANZAISession1

    beginString

    FIXT.1.1

    targetCompId

    BANZAI

    Type

    List

    Resource ID

    acc-fixSessionList

    Entries

    acc-BANZAISession1

Now we shall implement our use case in a separate integration flow fix-proxy.

  1. Add a FIX Ingress Connector from the connectors list, to accept FIX messages sent out by the initiator. The connector should be configured as per the following table.

    socketAcceptPort

    12000

    senderCompID

    UESB

    heartBeatIntervalSec

    30

    startTime

    00:00:00

    endTime

    00:00:00

    otherSettings

    acc-qfSettings

    configuredSessions

    acc-fixSessionList

    messageStoreFactory

    acc-fileMsgStore

    logFactory

    acc-fileLogFactory

  2. Add a FIX Egress Connector from the connectors list, to send out accepted FIX messages to the acceptor. The connector should be configured as per the following table.

    heartBeatIntervalSec

    30

    startTime

    00:00:00

    endTime

    00:00:00

    reconnectInterval

    20

    messageStoreFactory

    ini-fileMsgStore

    logFactory

    ini-fileLogFactory

    otherSettings

    ini-qfSettings

    socketConnectHost

    localhost

    socketConnectPort

    9879

    beginString

    FIXT.1.1

    senderCompID

    UESB

    targetCompID

    EXEC

    endpointConfigurations

    ini-endpointSettings

  3. Connect the Processor port of the ingress connector to the Input port of the egress connector (passing the accepted message directly into the egress connector).

  4. Connect the Response Processor port of the egress connector to the Input port of the ingress connector (passing the response message from the acceptor back to the initiator).

The final flow configuration should resemble the following:

fix flow

Testing

  1. Extract the QF/J binary distribution at a known location of your filesystem.

  2. Create the following custom configuration files for Banzai and Executor sample applications (which are based on the default configurations available inside the respective JARs, modified to forward and receive messages to/from UESB, which acts as the message proxy):

    banzai.cfg

    [default]
FileStorePath=target/data/banzai
ConnectionType=initiator
SenderCompID=BANZAI
TargetCompID=UESB
SocketConnectHost=localhost
StartTime=00:00:00
EndTime=23:00:00
HeartBtInt=30
ReconnectInterval=5

[session]
BeginString=FIXT.1.1
DefaultApplVerID=FIX.5.0
SocketConnectPort=12000

    executor.cfg

    [default]
FileStorePath=target/data/executor
ConnectionType=acceptor
StartTime=00:00:00
EndTime=23:00:00
HeartBtInt=30
ValidOrderTypes=1,2,F
SenderCompID=EXEC
TargetCompID=UESB
UseDataDictionary=Y
DefaultMarketPrice=12.30

[session]
BeginString=FIXT.1.1
DefaultApplVerID=FIX.5.0
SocketAcceptPort=9879

  3. Create a new UltraESB Server run configuration in Ultra Studio and run the project.

    Successful startup of the project with the FIX transport elements would be indicated by ESB logs:

    2016-12-18T12:43:46,925 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [500101I001]  INFO Project Initializing project : fix-1.0-SNAPSHOT
2016-12-18T12:43:47,088 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [100220I001]  INFO IntegrationFlow Initializing the integration flow : fix-proxy
2016-12-18T12:43:47,163 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXEgressConnector Initializing egress connector : 86971553-2ed3-cc46-3492-bb6ad653e269
2016-12-18T12:43:47,168 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender FIX Transport Sender Initialized
2016-12-18T12:43:47,169 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender FIX Transport Sender started
2016-12-18T12:43:47,172 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXIngressConnector Initializing ingress connector : 2a8be6ab-903d-7862-92e5-357d1428186c
2016-12-18T12:43:47,175 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportListener FIX transport Listener initialized
2016-12-18T12:43:47,177 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [500101I002]  INFO Project Starting project : fix-1.0-SNAPSHOT
2016-12-18T12:43:47,178 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [100220I020]  INFO IntegrationFlow Starting the integration flow : fix-proxy
2016-12-18T12:43:47,178 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportListener FIX transport listener started
2016-12-18T12:43:48,981 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXSessionFactory Created new socket acceptor for ingress config : 2a8be6ab-903d-7862-92e5-357d1428186c
2016-12-18T12:43:49,018 [127.0.1.1-janaka-ENVY] [main] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportListener Socket acceptor for fix ingress connector: 2a8be6ab-903d-7862-92e5-357d1428186c started

  4. Start the Banzai example application with the custom configuration:

    java -jar /path/to/quickfixj-examples-banzai-<version>.jar /path/to/banzai.cfg

    The Banzai UI will be displayed:

    fix banzai ui

    At startup, Banzai logs will indicate that a session is established successfully with the acceptor UESB and initial message exchange (including login):

    Dec 18, 2016 12:41:44 PM quickfix.mina.initiator.InitiatorIoHandler sessionCreated
INFO: MINA session created for FIXT.1.1:BANZAI->UESB: local=/127.0.0.1:46242, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=localhost/127.0.0.1:12000
<20161218-07:11:45, FIXT.1.1:BANZAI->UESB, outgoing> (8=FIXT.1.19=7235=A34=149=BANZAI52=20161218-07:11:45.67556=UESB98=0108=301137=710=123)
<20161218-07:11:45, FIXT.1.1:BANZAI->UESB, event> (Initiated logon request)
<20161218-07:11:45, FIXT.1.1:BANZAI->UESB, incoming> (8=FIXT.1.19=7235=A34=149=UESB52=20161218-07:11:45.72556=BANZAI98=0108=301137=710=119)
Dec 18, 2016 12:41:45 PM quickfix.mina.initiator.InitiatorIoHandler processMessage
INFO: Setting DefaultApplVerID (1137=7) from Logon
<20161218-07:11:45, FIXT.1.1:BANZAI->UESB, event> (Received logon)

  5. Start the Executor example application with the custom configuration:

    java -jar /path/to/quickfixj-examples-executor-<version>.jar /path/to/executor.cfg

    The only observation will be the Executor startup logs, as UESB (initiator) is not currently attempting to to establish a session with it.

    INFO: [FIX.4.2:EXEC->UESB] daily, 00:00:00-UTC - 23:00:00-UTC
<20161218-07:25:32, FIX.4.2:EXEC->UESB, event> (Session FIX.4.2:EXEC->UESB schedule is daily, 00:00:00-UTC - 23:00:00-UTC)
<20161218-07:25:32, FIX.4.2:EXEC->UESB, event> (Session state is not current; resetting FIX.4.2:EXEC->UESB)
<20161218-07:25:32, FIX.4.2:EXEC->UESB, event> (Created session: FIX.4.2:EXEC->UESB)
<20161218-07:25:32, FIX.4.2:EXEC->UESB, event> (Valid order types: [1, 2, F])
Dec 18, 2016 12:55:32 PM quickfix.mina.SessionConnector startSessionTimer
INFO: SessionTimer started
Dec 18, 2016 12:55:32 PM quickfix.mina.NetworkingOptions logOption
INFO: Socket option: SocketTcpNoDelay=true
Dec 18, 2016 12:55:32 PM quickfix.mina.NetworkingOptions logOption
INFO: Socket option: SocketSynchronousWrites=false
Dec 18, 2016 12:55:32 PM quickfix.mina.NetworkingOptions logOption
INFO: Socket option: SocketSynchronousWriteTimeout=30000
Dec 18, 2016 12:55:32 PM quickfix.mina.acceptor.AbstractSocketAcceptor startAcceptingConnections
INFO: Listening for connections at 0.0.0.0/0.0.0.0:9879 for session(s) [FIX.4.2:EXEC->UESB]
press <enter> to quit
Dec 18, 2016 12:55:32 PM quickfix.mina.SingleThreadedEventHandlingStrategy$1 run
INFO: Started QFJ Message Processor

  6. Send a request (e.g. a stock buy request, as in the figure below) over the BANZAI→UESB session.

    Banzai logs will indicate the successful sending of the request to UESB and reception of the response, accompanied by displaying of the execution on the UI:

    <20161218-07:27:06, FIXT.1.1:BANZAI->UESB, outgoing> (8=FIXT.1.19=12935=D34=3449=BANZAI52=20161218-07:27:06.10356=UESB11=148204602605621=138=140=154=155=ADRT59=060=20161218-07:27:06.10110=149)
<20161218-07:27:06, FIXT.1.1:BANZAI->UESB, incoming> (8=FIXT.1.19=11635=834=3249=UESB52=20161218-07:27:06.11056=BANZAI11=148204602605614=017=537=539=054=155=ADRT150=2151=110=199)

    Executor logs will indicate the reception of the request from UESB and successful sending of the response:

    <20161218-07:27:06, FIXT.1.1:EXEC->UESB, incoming> (8=FIXT.1.19=12635=D34=549=UESB52=20161218-07:27:06.10556=EXEC11=148204602605621=138=140=154=155=ADRT59=060=20161218-07:27:06.10110=210)
<20161218-07:27:06, FIXT.1.1:EXEC->UESB, outgoing> (8=FIXT.1.19=11335=834=749=EXEC52=20161218-07:27:06.10756=UESB11=148204602605614=017=537=539=054=155=ADRT150=2151=110=012)
<20161218-07:27:06, FIXT.1.1:EXEC->UESB, outgoing> (8=FIXT.1.19=13835=834=849=EXEC52=20161218-07:27:06.10856=UESB6=12.311=148204602605614=117=631=12.332=137=638=139=254=155=ADRT150=2151=010=101)

    ESB logs will indicate creation of an initiator for EXEC (executor), and forwarding of the request:

    2016-12-18T12:57:06,106 [127.0.1.1-janaka-ENVY] [pool-2-thread-3] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender Started initiator for sender compID: UESB with session ID: FIXT.1.1:UESB->EXEC
2016-12-18T12:57:06,106 [127.0.1.1-janaka-ENVY] [pool-2-thread-3] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender Fix message sent to target over session: FIXT.1.1:UESB->EXEC

By increasing the log level of FIX (org.adroitlogic.x.transport.fix) to TRACE, it is possible to view the full content of FIX messages being exchanged, in addition to message reception and response sending logs, on the ESB log:

log4j2.xml (src/test/resources/)

<Configuration ...>
    ...
    <Loggers>
        ...
        <AsyncLogger name="org.adroitlogic.x.transport.fix" level="TRACE"/>

in which case, logs resulting from the above transaction would resemble:

2016-12-18T13:04:58,888 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->BANZAI] [system-] [000000D000] DEBUG FIXApplication Received FIX message from BANZAI Message Sequence Number: 42 Receiver ID: UESB
2016-12-18T13:04:58,889 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->BANZAI] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=12935=D34=4249=BANZAI52=20161218-07:34:58.84556=UESB11=148204649879221=138=140=154=155=ADRT59=060=20161218-07:34:58.84310=204
2016-12-18T13:04:58,916 [127.0.1.1-janaka-ENVY] [pool-2-thread-1] [fix-1.0-SNAPSHOT] [000000D000] DEBUG FIXTransportSender Starting initiator for for sender compID: UESB with session ID: FIXT.1.1:UESB->EXEC
2016-12-18T13:04:58,950 [127.0.1.1-janaka-ENVY] [pool-2-thread-1] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender Started initiator for sender compID: UESB with session ID: FIXT.1.1:UESB->EXEC
2016-12-18T13:04:58,950 [127.0.1.1-janaka-ENVY] [pool-2-thread-1] [fix-1.0-SNAPSHOT] [000000D000] DEBUG FIXApplication Sending application level FIX message to EXEC Message Type: D Message Sequence Number: 12 Sender ID: UESB
2016-12-18T13:04:58,950 [127.0.1.1-janaka-ENVY] [pool-2-thread-1] [fix-1.0-SNAPSHOT] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=12735=D34=1249=UESB52=20161218-07:34:58.95056=EXEC11=148204649879221=138=140=154=155=ADRT59=060=20161218-07:34:58.84310=052
2016-12-18T13:04:58,951 [127.0.1.1-janaka-ENVY] [pool-2-thread-1] [fix-1.0-SNAPSHOT] [000000I000]  INFO FIXTransportSender Fix message sent to target over session: FIXT.1.1:UESB->EXEC
2016-12-18T13:04:59,951 [127.0.1.1-janaka-ENVY] [QFJ Timer] [system-] [000000D000] DEBUG FIXApplication Sending administrative FIX message to: EXEC Message Type: A Message Sequence Number: 13 Sender ID: UESB
2016-12-18T13:04:59,952 [127.0.1.1-janaka-ENVY] [QFJ Timer] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=7135=A34=1349=UESB52=20161218-07:34:59.95056=EXEC98=0108=301137=710=035
2016-12-18T13:05:00,209 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Sending administrative FIX message from: UESB Message Type: A Message Sequence Number: 15 Sender ID: EXEC
2016-12-18T13:05:00,209 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=7135=A34=1549=EXEC52=20161218-07:35:00.09456=UESB98=0108=301137=710=023
2016-12-18T13:05:00,210 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Sending administrative FIX message from: UESB Message Type: 2 Message Sequence Number: 16 Sender ID: EXEC
2016-12-18T13:05:00,210 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=6235=234=1649=EXEC52=20161218-07:35:00.09856=UESB7=1216=010=096
2016-12-18T13:05:00,211 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Sending application level FIX message to EXEC Message Type: D Message Sequence Number: 12 Sender ID: UESB
2016-12-18T13:05:00,211 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=15835=D34=1243=Y49=UESB52=20161218-07:35:00.21056=EXEC122=20161218-07:34:58.95011=148204649879221=138=140=154=155=ADRT59=060=20161218-07:34:58.84310=047
2016-12-18T13:05:00,213 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Sending administrative FIX message to: EXEC Message Type: 4 Message Sequence Number: 13 Sender ID: UESB
2016-12-18T13:05:00,213 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=9535=434=1343=Y49=UESB52=20161218-07:35:00.21256=EXEC122=20161218-07:35:00.21236=14123=Y10=222
2016-12-18T13:05:00,296 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Received FIX message from EXEC Message Sequence Number: 17 Receiver ID: UESB
2016-12-18T13:05:00,296 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=11435=834=1749=EXEC52=20161218-07:35:00.26656=UESB11=148204649879214=017=137=139=054=155=ADRT150=2151=110=073
2016-12-18T13:05:00,297 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000D000] DEBUG FIXApplication Received FIX message from EXEC Message Sequence Number: 18 Receiver ID: UESB
2016-12-18T13:05:00,298 [127.0.1.1-janaka-ENVY] [QF/J Session dispatcher: FIXT.1.1:UESB->EXEC] [system-] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=13935=834=1849=EXEC52=20161218-07:35:00.27156=UESB6=12.311=148204649879214=117=231=12.332=137=238=139=254=155=ADRT150=2151=010=157
2016-12-18T13:05:00,298 [127.0.1.1-janaka-ENVY] [pool-2-thread-2] [fix-1.0-SNAPSHOT] [000000D000] DEBUG FIXApplication Sending application level FIX message to BANZAI Message Type: 8 Message Sequence Number: 40 Sender ID: UESB
2016-12-18T13:05:00,298 [127.0.1.1-janaka-ENVY] [pool-2-thread-2] [fix-1.0-SNAPSHOT] [000000T000] TRACE FIXApplication FIX Message : 8=FIXT.1.19=11635=834=4049=UESB52=20161218-07:35:00.29756=BANZAI11=148204649879214=017=137=139=054=155=ADRT150=2151=110=219
In this topic
FIX Message Proxying
Use Case Description
Proposed Solution
Implementation of the Solution
Prerequisites
Implementation
Testing
In this topic
FIX Message Proxying
Use Case Description
Proposed Solution
Implementation of the Solution
Prerequisites
Implementation
Testing
(c) Copyright 2016-2017 AdroitLogic. All rights reserved.
Terms of Use
Contact Us