azure file ingress connector

Azure File Ingress Connector

Version: 17.07

Supported Since: 17.07

What is an Azure File Ingress Connector?

Azure File Ingress Connector can be used to pull Azure objects from Azure containers. The default behaviour of this connector is that it will pull objects from the container and persist temporarily in the local file system and inject it to the UltraESB-X engine. After successfully completing the integration flow, it will then delete the original object from the container.

Azure credentials associated with an account which has write permission to the source bucket will be necessary when configuring this connector.
In order to use the Azure File Ingress Connector, you must first select the Azure Connector dependency from the connector list when you are creating an empty Ultra project. If you have already created a project, you can add the Azure Connector dependency via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list, select the Azure Connector dependency.
azure file ingress outports

Out Ports

Processor

The message will be emitted from this out port after an Object has been obtained from the file share. The payload of the message will be the Object which has been pulled from the container.

On Exception

The message will be emitted from this out port if the processing element fails to prepare a message from the object pulled from the container.

Parameters

* marked fields are mandatory

Azure Account Name

Basic

Azure Account Name is required.

Azure Account Key

Basic

Azure Account Key is required.

Default Endpoints protocol *

Basic

Name of the endpoints protocol to be used, which is located in the configured Azure storage account.

Source share name *

Basic

Name of the Azure container, which is located in the configured Azure storage account.

Source share prefix *

Basic

If searches are conducted across shares, the prefix can be used to select multiple shares.

File prefix name *

Basic

If there are multiple files to be extracted using the prefix, this can be used.

Temporary Buffer location *

Basic

Name of the temporary buffer location, on the local file system

Scheduler Configuration

Scheduling

Bean reference of the scheduler configuration bean which should be declared as a resource in the project.xpml file. By default there is internal scheduler configuration within the framework which will be shared by all the polling connectors. If you need to configure higher level of concurrent processing threads which will fetch the objects from the Azure share, you can configure your own thread pool configuration while declaring the parameters of the scheduler configuration bean as below.

<x:resources>
    <x:resource id="custom-scheduler-config">
        <bean id="schedulerConfigBean" class="org.adroitlogic.x.base.trp.SchedulerConfig">
            <constructor-arg name="name" value="my-custom-scheduler"/>
            <property name="schedulerThreadCount" value="4"/>
            <property name="pollingThreadCoreSize" value="4"/>
            <property name="pollingThreadMaxSize" value="10"/>
            <property name="pollingQueueSize" value="25"/>
            <property name="pollingKeepAliveTime" value="5000"/>
        </bean>
    </x:resource>
</x:resources>

In this configuration,

  • schedulerThreadCount - is the number of threads which will be used to schedule the number of threads to be used to handle scheduled polling tasks in this Azure Ingress Connector. Generally this value should be smaller value since polling threads don’t do heavy task in this, rather it’s just initiating the polling task for the scheduling iteration and handover the file fetching and processing task to a separate executor service. This executor service can be configured by using next four parameters of above scheduling configuration bean.

  • pollingThreadCoreSize - is the core size of the Azure object fetching thread pool

  • pollingThreadMaxSize - is the maximum number of threads of the Azure object fetching thread pool

  • pollingQueueSize - is the queue size of the Azure object fetching thread pool

  • pollingKeepAliveTime - is the keep alive time of the Azure object fetching thread pool

Polling Cron Expression

Scheduling

Cron expression for the Azure object polling schedule. Cron expression should be a valid Quartz cron expression since the Framework is underneath using Quartz to extract the schedule from the cron expression.

Polling Start Delay

Scheduling

Delay in milliseconds to start the polling schedule. Any iteration which comes within this time period from the startup time of the framework, won’t be considered as a valid file polling iteration.

Polling Repeat Interval

Scheduling

Interval in milliseconds for the next iteration of the polling schedule. This will be considered if there isn’t a configured cron schedule.

Polling Repeat Count

Scheduling

Number of iterations which should go through the polling schedule. If this is set to 1 which means only the first iteration of the polling schedule will be considered as a valid file polling iteration and all other iterations of the schedule will be ignored. By default, this value is set to -1 which means it will consider all the iterations of the polling schedule as a valid iteration.

Concurrent Polling Count

Scheduling

Maximum number of concurrent threads which can be used to poll the configured Azure share to fetch objects. By default, this value is 1.

Concurrent Execution Count

Scheduling

Maximum number of concurrent threads which can be used to process the fetched objects from the Azure share. By default, this value is 4.

Sample Use Case

azure file ingress flow

In this scenario, assume that an Agency allows remote branches to upload their files to an Azure File storage and they want to distribute those files immediately to various 3rd parties etc. Assume, 3rd parties have provided their SFTP credentials to the Agency.

azure file ingress listener
Figure 1. Configuring Azure file ingress connector to accept files with a given prefix.

Now the 3rd parties will receive the required files into their SFTP destinations.

Product and company names and marks mentioned are the property of their respective owners and are mentioned for identification purposes only.

In this topic
In this topic
(c) Copyright 2016-2017 AdroitLogic. All rights reserved.
Terms of Use
Contact Us