File Polling
Version: 17.01
Supported Since: 17.01
Use Case Description
Gringotts is one of the major banks in Western Europe. They provide financial support for large number of small time business institutions. For these institutions they are required to send monthly business invoices in a secure manner. Currently they do have two separate systems to utilize for this purpose. The invoice generation needs to be done using one of their legacy backend processing systems while the generated invoices needs to be sent out to relevant parties using a separate secure file gateway. The requirement is after generating the invoices it is required to notify the file gateway about the generated invoices and the relevant information about the recipient through JMS queue.
The legacy invoice generation system will generate the invoice file and a XML file that contains the required meta information about the invoice recipient. The generated files will be placed into a predefined locations of local file system. The secure file gateway is supposed to utilize the meta information file to extract the relevant information and send the invoices securely to the relevant parties.
Proposed Solution
Gringotts bank has decided to integrate the secure file gateway with the legacy invoice generator using an ESB. Their requirement of ESB is to poll the predefined local file system directories inorder to pick up the generated meta information XML files and validate the XML content before forwarding it to the file gateway through a JMS queue. For this implementation they have decided to use Apache ActiveMQ as the JSM provider.
Implementation
In-order to develop the above mentioned flow, first you need to add File Listener, File Sender and Spring JMS Sender from connector list and XSD Validator dependency from processors list.
| If you have already created a project, you can add above dependencies via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list and Processors list, select above dependencies |
For this sample let’s use ActiveMQ and if you have not already installed ActiveMQ please follow the installation guide.
After that, in the project.xpml file select the ActiveMQ JMS template and fill the necessary fields. After that following resources will be added to the project.xpml file
In-order to develop the flow, first let’s create an integration flow named “file-mediation-integration” and follow the below steps
-
Add a File Ingress Connector to obtain the generated invoice files from the predefined directory as request messages to the ESB. For the input field File Path it is required to enter a base directory location whether the invoice meta-information files will be put into by the legacy invoice generator. For the input field File Name Pattern user should provide a regular expression to filter the invoice meta-information files by their name. Instead of using this property user can use the "File Path Pattern* property by inserting a regex pattern for sub directory path to access meta-information files in the sub directories within the base directory.
There are some advanced properties that the user will be able to configure for the file ingress connector element of the ESB message flow. To make sure that the meta information file has been completely written to the file system before injecting it to the ESB, user can define the property Wait After Modification (in seconds). It defines the minimum time that the File Listener will wait after the last modified time of the file before the processing is started. By marking the property Remove Original File user will be able to command the ESB to delete the original file after the processing has been completed. Otherwise the same file will be processed repeatedly. The properties Move After Process and Move After Failure respectively defines the file paths where the original file should be copied after processing in the success case and in the failure case.
User can define the file polling schedule of the File Listener under the Scheduling configurations section. The property Polling Start Delay defines the time that the File Listener should wait after the ESB is started before it starts processing files. Polling Repeat Interval defines the time between two polling cycles for the File Listener in milli seconds. User can define the number of repetitions that the File Listener should poll by defining the property Polling Repeat Count. By entering a negative value for this property, it will imply that the polling process should continue indefinitely.
-
Next, add a XSD Validator to validate the XML content of the meta-information file and select the metaValidator.xsd file as the XSD file path.
-
After that send the validated message to destination JMS queue using a Spring JMS Egress Connector. Specify ActiveMq-jmsTemplate as the JMS template and the destination queue name as well.
-
In case of XSD validation failure of the XML payload the meta information file should be moved to an error directory location. To handle that scenario it is required to add File Egress Connector to the XSD Validator elements on failed validation out port.
-
After that add an Exception flow end element to exceptionally complete the flow
The complete flow should look as below
Now after running the project, send the sample message by placing a meta information file into the defined location. A sample content of a meta information file is provided below. The payload will be sent to the JMS queue after successful validation.
<?xml version="1.0" encoding="UTF-8"?>
<invoice-meta-info xmlns="http://www.adroitlogic.org/gringotts" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.adroitlogic.org/gringotts_metainfo.xsd">
<subject>monthly receipt</subject>
<receiver-id>Hogsmeade</receiver-id>
<receiver-address>https://hogsmeade.com/service/finance-receipt</receiver-address>
<payload>
<path>/Users/dammina/code/internal-projects/project-x/file/invoices/hogsmeade-invoice-january-15.txt</path>
<path>/Users/dammina/code/internal-projects/project-x/file/invoices/hogsmeade-invoice-january-30.txt</path>
</payload>
</invoice-meta-info>If you remove or change any field, then the XSD validation will be failed and the invoice meta information file will be moved to the user defined error file destination.
Now you can run the integration flow and check the file mediating working as expected. If you put a input file with .xml file name extension to the configured directory source location, your integration flow should fetch the file validate it over provided XSD and upload it to the target message queue.
Property Configuration
When running the sample in the UltraESB-X distribution, you can override following properties in order for the sample to work with your local configurations.
| Refer to Managing Project Properties documentation on how to override properties. |
|
Directory path to listen and poll files from |
|
Schedule expression to poll for files from configured directory |
|
File name pattern to be used in polling files |
|
Directory path to copy input files after complete processing |
|
Directory path to copy input files after processing failure |
|
Whether to keep or remove input file after processing from the input directory |
|
Active MQ broker URL (e.g: tcp://localhost:61616) |
|
Active MQ username |
|
Active MQ password |
|
File path to move the file upon validation failure |
|
JMS queue name to be used at egress end |
In this topic
In this topic