File Polling

Version: 17.07

Supported Since: 17.01

Use Case Description

Gringotts is one of the major banks in Western Europe. They provide financial support for large number of small time business institutions. For these institutions they are required to send monthly business invoices in a secure manner. Currently they do have two separate systems to utilize for this purpose. The invoice generation needs to be done using one of their legacy backend processing systems while the generated invoices needs to be sent out to relevant parties using a separate secure file gateway. The requirement is after generating the invoices it is required to notify the file gateway about the generated invoices and the relevant information about the recipient through a JMS queue.

The legacy invoice generation system will generate the invoice file and a XML file that contains the required meta information about the invoice recipient. The generated files will be placed into a predefined locations of local file system. The secure file gateway is supposed to utilize the meta information file to extract the relevant information and send the invoices securely to the relevant parties.

Proposed Solution

Gringotts bank has decided to integrate the secure file gateway with the legacy invoice generator using an ESB. Their requirement of ESB is to poll the predefined local file system directories in-order to pick up the generated meta information XML files and validate the XML content before forwarding it to the file gateway through a JMS queue. For this implementation they have decided to use Apache ActiveMQ as the JSM provider.

file polling 1

UltraStudio Configuration

UltraESB-X Configuration

Implementation of the Solution

Prerequisite

In order to implement above use case you must first select following dependencies when you are creating a new Ultra project.

  • Local File Connector from the connector list

  • Spring JMS Connector from the connector list

  • XML Operation from the processor list

If you have already created a project, you can add above dependencies via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list and Processors list, select above dependencies.

For this sample let’s use ActiveMQ and if you have not already installed ActiveMQ please follow the installation guide.

First open the project.xpml file of the project and right click on it (on the XML content of the file). From the context menu, select Resource Template as shown in below figure

jms egress connector 5

After that from the shown dialog box, select ActiveMQ JMS template.

jms egress connector 6

Next you need to specify the required parameters as shown below. It is mandatory to specify a bean prefix and for that you can specify any value. The only other mandatory field is the ActiveMQ Broker URL.

jms egress connector 7
Implementation

To implement above use case, first create an integration flow named “file-polling-with-xml-validation-flow”, and then add required components by going through following steps in order.

  1. Add a File Ingress Connector to obtain the generated invoice files from the predefined directory as request messages to the ESB. For the input field File Path it is required to enter a base directory location whether the invoice meta-information files will be put into by the legacy invoice generator. For the input field File Name Pattern user should provide a regular expression to filter the invoice meta-information files by their name. Instead of using this property user can use the "File Path Pattern* property by inserting a regex pattern for sub directory path to access meta-information files in the sub directories within the base directory.

There are some advanced properties that the user will be able to configure for the file ingress connector element of the ESB message flow. To make sure that the meta information file has been completely written to the file system before injecting it to the ESB, user can define the property Wait After Modification (in seconds). It defines the minimum time that the File Ingress Connector will wait after the last modified time of the file before the processing is started. By marking the property Remove Original File user will be able to command the ESB to delete the original file after the processing has been completed. Otherwise the same file will be processed repeatedly. The properties Move After Process and Move After Failure respectively defines the file paths where the original file should be copied after processing in the success case and in the failure case.

User can define the file polling schedule of the File Ingress Connector under the Scheduling configurations section. The property Polling Start Delay defines the time that the File Ingress Connector should wait after the ESB is started before it starts processing files. Polling Repeat Interval defines the time between two polling cycles for the File Listener in milli seconds. User can define the number of repetitions that the File Ingress Connector should poll by defining the property Polling Repeat Count. By entering a negative value for this property, it will imply that the polling process should continue indefinitely.

  1. Add an XSD Validator to validate the XML content of the meta-information file and select the metaValidator.xsd file as the XSD file path.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.adroitlogic.org/gringotts"
            targetNamespace="http://www.adroitlogic.org/gringotts" elementFormDefault="qualified">
  <xsd:element name="invoice-meta-info">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="subject" type="xsd:string" minOccurs="0" maxOccurs="1" />
        <xsd:element name="receiver-id" type="xsd:string" minOccurs="0" maxOccurs="1" />
        <xsd:element name="receiver-address" type="xsd:string" minOccurs="0" maxOccurs="1" />
        <xsd:element name="payload" type="Payload" minOccurs="1" maxOccurs="1" />
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
  <xsd:complexType name="Payload">
    <xsd:sequence>
      <xsd:element name="path" type="xsd:string" minOccurs="1" maxOccurs="unbounded" />
    </xsd:sequence>
  </xsd:complexType>
</xsd:schema>
  1. After that send the validated message to destination JMS queue using a Spring JMS Egress Connector. Specify ActiveMq-jmsTemplate as the JMS template and the destination queue name as well.

  2. In case of XSD validation failure of the XML payload the meta information file should be moved to an error directory location. To handle that scenario it is required to add File Egress Connector to the XSD Validator elements on failed validation out port.

  3. After that add an Exception flow end element to exceptionally complete the flow

The completed integration flow should look like below.

file polling complete flow

Configuration for each element is as below. The numbering corresponds to the numbers shown in above diagram.

Design View

Text View

.

1. File Ingress Connector (Basic Tab)