file nio ingress connector

NIO File Ingress Connector

Version: 17.07

Supported Since: 17.01

What is NIO File Ingress Connector?

NIO file transport acts as a non-polling transport which will trigger an event if and only if there is a file or a directory is created or modified. In a polling transport, it scans for a directory or set of directories by a given time period of interval. That will be an overhead and insufficient use of system resources since it scans for set or directories and files periodically even though there are no files in those.

The JDK 7 provided a special package called java.nio. Java NIO (New IO) which is an alternative IO API for Java (from Java 1.4), meaning alternative to the standard Java IO and Java Networking APIs. Java NIO offers a different way of working with IO than the standard IO APIs. This package provided a package called java.nio.file and it has a file change notification API, called the WatchService API. This API allows to register a directory (or directories) with the watch service. When the service get started it detects all the events of creation or modification of files and directories.

In order to use the NIO File Ingress Connector, you must first select the NIO File Connector dependency from the connector list when you are creating an empty Ultra project. If you have already created a project, you can add the NIO File Connector dependency via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list, select the NIO File Connector dependency.
nio file ingress connector 3

Out Ports

Processor

The message will be emitted from this out port when a file modification is detected.

On Exception

The message will be emitted from this out port if the processing element failed to create the payload from modified file

Parameters

* marked fields are mandatory

Root Path *

Basic

This is the base path of the file system which you need to scan the files for. You can specify any file path pattern later at the 'Path pattern' field and that will be checked within this root path.

Pattern Syntax *

Basic

There are 2 types of pattern syntax which we can match with the file paths of the local file system. This will be related to the 'Path pattern' field.

GLOB
REGEX

File Path pattern *

Basic

File path pattern which we need to scan for the files. This can be just a file pattern or file pattern with the directory pattern.

The path pattern should match with the pattern syntax selected in Pattern Syntax property.

Remove original file

Advanced

Specify whether to remove original file after fetching it for processing. If this is set to false, input file won’t be removed by the framework automatically. By default, this value is true.

Wait after modification

Advanced

This is a time period in seconds which will be used to decide whether to fetch a file or not based on its last modified time of the file. Framework will not fetch a given fill until current time exceeds, the summation of the last modified time of the file and this value. By default, this value is zero.

Move after process

Advanced

File path to move input file in the file system, after completing the processing successfully. This path will be considered only if it’s successfully processed without any errors.

If the move after process directory matches the path pattern mentioned above this file will be fetched again by the watch service. Then this will be happen recursively without stopping.

Move after failure

Advanced

File path to move the fetched input file in the file system, after completing the processing with failures. This path will be considered only if the connector couldn’t process the file.

If the move after failure directory matches the path pattern mentioned above this file will be fetched again by the watch service. Then this will be happen recursively without stopping.

Move timestamp format

Advanced

Set to attach the current timestamp in this format to the file name as a suffix, while moving files both in success and failure scenarios. If this is not set, original file name will be used as it is while moving.

Sample Use Case

For this scenario, let’s use the NIO File Ingress Connector to pick up XML files and log the content of the files to the log file. The complete flow will look as below.

nio file ingress connector 4

As for the pattern syntax, select glob and specify the file path pattern as shown below

nio file ingress connector 5
The set of file paths which should be scanned for the files are generated using the above specified root path and file path pattern. See the example below.

Root Path

Pattern Syntax

Path Pattern

Description

/tmp

glob

*.xml

All the XML files inside the tmp directory

/tmp

glob

/*/.xml

All the XML files inside any level of directories in tmp directory

/tmp

glob

/a/.xml

All the XML files inside the directories which names ends with a in the tmp directory

/tmp

regex

\d.xml

All the XML files which name is a single digit from 0-9 in the tmp directory

Now after running the project, you will be able to see that all the files which match the path pattern have been successfully picked up from the any directory which matches the root directory and the path pattern you specified.

In this topic
In this topic
Contact Us