NIO File Ingress Connector
Version: 17.07
Supported Since: 17.01
What is NIO File Ingress Connector?
NIO file transport acts as a non-polling transport which will trigger an event if and only if there is a file or a directory is created or modified. In a polling transport, it scans for a directory or set of directories by a given time period of interval. That will be an overhead and insufficient use of system resources since it scans for set or directories and files periodically even though there are no files in those.
The JDK 7 provided a special package called java.nio. Java NIO (New IO) which is an alternative IO API for Java (from Java 1.4), meaning alternative to the standard Java IO and Java Networking APIs. Java NIO offers a different way of working with IO than the standard IO APIs. This package provided a package called java.nio.file and it has a file change notification API, called the WatchService API. This API allows to register a directory (or directories) with the watch service. When the service get started it detects all the events of creation or modification of files and directories.
| The current version of NIO File Ingress Connector is only compatible with Linux-based systems. |
| In order to use the NIO File Ingress Connector, you must first select the NIO File Connector dependency from the connector list when you are creating an empty Ultra project. If you have already created a project, you can add the NIO File Connector dependency via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list, select the NIO File Connector dependency. |
Out Ports
Processor |
The message will be emitted from this out port when a file modification is detected. |
On Exception |
The message will be emitted from this out port if the processing element failed to create the payload from modified file |
Parameters
* marked fields are mandatory
Root Path * |
Basic |
This is the base path of the file system which you need to scan the files for. You can specify any file path pattern later at the 'Path pattern' field and that will be checked within this root path. |
||
Pattern Syntax * |
Basic |
There are 2 types of pattern syntax which we can match with the file paths of the local file system. This will be related to the 'Path pattern' field. GLOB |
||
File Path pattern * |
Basic |
File path pattern which we need to scan for the files. This can be just a file pattern or file pattern with the directory pattern.
|
||
Remove original file |
Advanced |
Specify whether to remove original file after fetching it for processing. If this is set to false, input file won’t be removed by the framework automatically. By default, this value is true. |
||
Wait after modification |
Advanced |
This is a time period in seconds which will be used to decide whether to fetch a file or not based on its last modified time of the file. Framework will not fetch a given fill until current time exceeds, the summation of the last modified time of the file and this value. By default, this value is zero. |
||
Move after process |
Advanced |
File path to move input file in the file system, after completing the processing successfully. This path will be considered only if it’s successfully processed without any errors. Unless this value is an absolute path, it will be computed relative to the current working directory of the process.
|
||
Move after failure |
Advanced |
File path to move the fetched input file in the file system, after completing the processing with failures. This path will be considered only if the connector couldn’t process the file. Unless this value is an absolute path, it will be computed relative to the current working directory of the process.
|
||
Move timestamp format |
Advanced |
Set to attach the current timestamp in this format to the file name as a suffix, while moving files both in success and failure scenarios. If this is not set, original file name will be used as it is while moving. |
Sample Use Case
For this scenario, let’s use the NIO File Ingress Connector to pick up XML files and log the content of the files to the log file. The complete flow will look as below.
As for the pattern syntax, select glob and specify the file path pattern as shown below
| The set of file paths which should be scanned for the files are generated using the above specified root path and file path pattern. See the example below. |
Root Path |
Pattern Syntax |
Path Pattern |
Description |
/tmp |
glob |
*.xml |
All the XML files inside the tmp directory |
/tmp |
glob |
/*/.xml |
All the XML files inside any level of directories in tmp directory |
/tmp |
glob |
/a/.xml |
All the XML files inside the directories which names ends with |
/tmp |
regex |
\d.xml |
All the XML files which name is a single digit from 0-9 in the tmp directory |
Now after running the project, you will be able to see that all the files which match the path pattern have been successfully picked up from the any directory which matches the root directory and the path pattern you specified.
In this topic
In this topic