s3 egress connector

S3 Egress Connector

Version: 17.07

Supported Since: 17.07

What is S3 Egress Connector?

S3 Egress Connector can be used to push files to Amazon AWS S3 buckets.

When sending messages to a S3 bucket, UltraESB engine expects the file content as the message payload and file name as a variable. However, this connector can be configured to override the file name of the original file with an arbitrary file name or connector can also be configured to append timestamp to the file name.

S3 egress connector can be configured to attach additional S3 metadata to the file which will be uploaded to the S3 bucket.

AWS S3 credentials associated with an account which has write permission to the target bucket will be necessary when configuring this connector.
In order to use the S3 Egress Connector, you must first select the AWS Connector dependency from the connector list when you are creating an empty Ultra project. If you have already created a project, you can add the AWS Connector dependency via Component Registry. From Tools menu, select Ultra Studio → Component Registry and from the Connectors list, select the AWS Connector dependency.
s3 egress 5

Out Ports

On Exception

The message will be emitted from this out port if the processing element fails to push the message content to target S3 bucket.

Response Processor

The message will be sent to this out port if the Complete Flow property is disabled and the message has been sent to the S3 bucket successfully

Side Ports

Connector Operation

This port is used to connect operational elements to the Egress Connector. By-default, user does not have to connect any operational element and the default connector operation will be used.

Parameters

* marked fields are mandatory

S3 Region *

Basic

The region of the target S3 Bucket

Destination bucket name *

Basic

Name of the AWS S3 bucket, which is located in the configured AWS S3 region.

Use profile Credentials

Basic

If selected, profile credentials will be used, credentials will be picked from ~/.aws/credentials (Linux/Mac) or C:\Users\USER_NAME\.aws\credentials (Windows)

AWS Access Key Id

Basic

AWS Access Key Id is required only if profile credentials are not going to be used.

AWS Access Secret Key

Basic

AWS Access Secret Key is required only if profile credentials are not going to be used.

File Name

Basic

File name for the payload to be saved (if there is an original name to the file, it will be preserved if this property is not provided)

Append timestamp to file name

Basic

If this option is selected, file names will be prefixed with current timestamp. This option will be useful if there is a requirement to sort the files chronologically.

For this to work, ultra.file.timestamp_format transport header should be present in the incoming message, else this option will be ignored.

Custom Object metadata

Basic

This field accepts a Java HashMap. Setting this will add new metadata to the S3 object, any existing or default metadata will be overridden.

Sample Use Case

In this case, assume that a Photography Agency wants to backup all their photos to amazon S3, in-order to save their local storage space for new photos.

s3 egress 1

File Listener will be configured as below to pick only jpeg images from the source folder

s3 egress 2
Figure 1. File Listener configuration

S3 Egress Connector will be configured as below to push S3 objects by appending backup date to the original file name

s3 egress 3
Figure 2. S3 Egress Connector configuration

In-order to append the timestamp to the object, transport header ultra.file.timestamp_format defining the expected date time format should be present in the incoming message context. Add New Transport Header processing element is used to set the time format of the timestamp.

s3 egress 4
Figure 3. Configuration of Add New Transport Header

Product and company names and marks mentioned are the property of their respective owners and are mentioned for identification purposes only.

In this topic
In this topic
Contact Us