Advanced Job Configuration

When creating or editing a job, you can use the Advanced tab to configure a job's contents, improve job performance, specify file management tasks, change networking options, and set job scheduling.

Note: Advanced options vary by job type. Some configuration options may not be available for certain jobs.

Drop Box/Source Specification

Inclusion/Exclusion

  • Set Include File Patterns and Exclude File Patterns to specify which file types to include in transfer jobs.

  • Choose to include specific file extensions, or set * to include all files in the job source.

You can also use expression characters to provide granular specificity to transfer jobs. Separate expression searches must be separated by a comma (,).

ExpressionAction
*Match zero or more characters
?Match a single character in a file name
[...]Match enclosed characters (e.g. [ch] matches files with c and h)
[A-z] or [1-9]Match a range of letters or numbers
[^A-z] or [!A-z]Do not match a range of letters or numbers

Note: To search for files that contain expression characters in their name, escape the character using \.

You can also exclude subdirectories if necessary. To exclude a subdirectory at a specific level, use the @ character:

Example PathDescription
C:\Example\SubdirectoryInclude Subdirectory
C:\Example\@SubdirectoryExclude Subdirectory

File Readiness

File Readiness checks apply to MediaDropBox, MediaAggregator, and MediaDistributor jobs.

  • Set a readiness check type:

    • None: No checks are made.
    • Stream Growing Files: Starts a transfer as files are being copied to the source, and attempts the transfer as data is written to the source.
    • Consecutive Seek (Recommended): Reads the last few bytes of the file, then after a specified number of delay, reads the same bytes again. If the content matches, the file is transferred.
    • MD5 Hash: Uses an MD5 Hash to compute a hash on files. After a specified delay, the MD5 hash of a file is checked. Only files with identical hashes are transferred. Job initialization can take a long time for larger file sizes.
    • Date/Time and Size: Checks the date, time, and size of a file twice. Files without changes are transferred.

Note: If the local or network file system supports file locking, the readiness check is not required.

  • When using Stream Growing Files, specify header options:

    • Header Check Delay: The delay in seconds that the transfer waits, after the end of the file is detected, before the header is resent. The default is 10 seconds.

    • Header Rewrite Size: The number of bytes to transfer in the header of the file. The default is 1 MB.

  • Select a Readiness Check Delay to set the pause, in seconds, between the two phases of MD5 Hash, Consecutive Seek, or Date/Time and Size file readiness checks. The default delay is 10 seconds. When choosing to Stream Growing Files, this is the time that must pass without file changes for the file to be declared complete.

Data Actions

Media Mover jobs can be configured to move and delete files and folders as part of a transfer action.

  • Set a list of Source Directories to Exclude from Deletion Scan if required for deletion actions.

  • Set a Source Data Location After Successful Transfer to select a directory to move transferred files into.

File Skipping

  • Specify whether to Skip Source File Not Found On Send, which will skip files that were initially read as part of the job, but were removed before they were transferred, and to Skip Files With 0 Byte Size, to skip files with no content.

  • Set a Source File Data filter to skip files by date or time, based on when the file was last modified, or accessed, or when the file was created.

Destination Specification

MediaDropBox, MediaAggregator, and MediaDistributor allow you to set options for transfer destinations.

  • Enable one of the Directory Mapping Options to transfer files to a structure that matches the source within the destination directory, or to transfer files to the Agent in a directory named for the job time or date.

  • Flatten Directory Structure to write all transferred files to the destination directory without using the folder structure as the origin.

  • Select a Job Date/Time and choose the Maximum Job Date-Time Directories to maintain on the target Agent. The default is 10. When the value is exceeded, the oldest job date/time directory is deleted.

  • Specify file removal tasks by enabling Remove Expired Files/Directories on Target and setting an Expiration Period.

Note: MediaReplicator jobs automatically create an identical directory structure based on the replication job.

Transport Options

Job Transport Options allow you to control the way data transfers during a job run, including the number of Simultaneous Transfer Streams, as well as to allow UDP firewall traversal by enabling Use UDP Control Channel.

WAN Acceleration

  • Select whether to Use WAN Accelerator. This setting uses all available bandwidth up to the user-specified bandwidth maximum. Use this option on high-latency, high-bandwidth networks where throughput is a top priority. If the throughput is less than 57200 Bps, the transfer will switch to TCP. WAN Acceleration is enabled by default.

  • Set the WAN Accelerator Aggressiveness to configure the job's sensitivity to other network traffic. The higher the level of aggressiveness, the less the job accommodates other network traffic. The WAN Accelerator Aggressiveness setting also specifies how aggressively the transfer attempts to reclaim bandwidth after packet loss.

    There are three levels of aggressiveness:

    • High - When 3% packet loss is detected, the transfer rate slows to 94% of the current transfer rate. The transfer waits 20ms before trying to increment the transfer rate by 5%.
    • Medium - When 2% packet loss is detected, the transfer rate slows down to 93% of the current transfer rate. The transfer waits 40ms before trying to increase the transfer rate by 4%.
    • Low - When 1% packet loss is detected, the transfer rate slows to 91% of the current transfer rate. The transfer waits 80ms before trying to increment the transfer rate by 3%.

Note: When Use UDP Control Channel is set to Yes, file transfers automatically use UDP WAN acceleration, which may override the Use WAN Accelerator setting. If the throughput is less than 57200 Bps, the transfer will switch to TCP.

Bandwidth Configuration

Configuring a job to use limited bandwidth can improve network performance for other activities.

You can set job-specific resource options:

  • Set the Bandwidth Ceiling to configure the maximum throttle rate for the transfer. Typically, this is the maximum speed of the network. Enter 0 to enable auto-detection for the best bandwidth rate.

  • Set the Bandwidth Floor to configure the minimum rate at which data should be sent. Enter 0 to enable auto-detection for the best bandwidth rate.

Note: If WAN Accelerator Aggressiveness is set to Medium or Low, the rate specified here will be used.

  • Set the Bandwidth Throttle by Time of Day to configure bandwidth to reduce network usage during periods of higher demand. Click + to add additional time parameters, or click x to remove them.

A bandwidth throttle specifies the rate at which the source will make the data available to the network, regardless of how fast the machine is sending. The Bandwidth Throttle is the rate at which the source reads the data from disk and queues data for transfer.

A throttle is split between the concurrent target Agents. For example, if your throttle is 50Mb/sec and you specify two target Agents, the bandwidth be divided equally at 25Mb/sec per target.

Note: Once a job has started, all bandwidth throttles account for Daylight Savings Time (DST). If DST changes while the job is running, bandwidth time of day changes may be off by an hour.

Advanced Options

Encryption

Adding encryption to a transfer reduces the overall performance of a transfer, but can guarantee of the integrity of the data stream. Without encryption specified, the underlying TCP protocol can guarantee the integrity of messages across each single network hop, but has no facilities for detecting a man-in-the-middle attack.

  • Encryption On encrypts transferred data using an AES256-SHA cypher.
  • No encryption – signed transfers unencrypted data, but includes the SSL protocol message digest calculation and signing to ensure data stream integrity.

Note: Mutual authentication is always used, regardless of the encryption level specified. Encryption is done in-stream and not on disk prior to sending.

Delivery Mode

  • Select a Delivery Mode to define the performance level and details recorded when conducting transfers.

  • Level 1 - No Checkpoint Restart: Does not create temporary work files. This mode is optimized for speed.

  • Level 2 - Checkpoint Restart: Uses temporary work files during transfer. Temporary files allow an interrupted transfer to resume from where the interruption occurred. The creation of temporary files adds time to the transfer, but provides resiliency.

  • Level 3 - Checkpoint Restart, Filename Logging: Logs file names transferred as part of the job log.

  • Level 4 - Checkpoint Restart, Filename Logging, Certified Delivery: Creates an Agent certified delivery log of files transferred, and allows File Search on the Manager Dashboard.

Transient File Naming Modes

Configuring a transfer's Transient File Naming allows you to set whether the #work_file# and #chkpt_file# tags used during a transfer as a prefix or suffix to a file name before the job completes.

You can also configure jobs to not apply the #work_file# tag, so that it can be opened immediately, regardless of transfer progress.

File System Options

  • Enable Compress Files to have the source Agent compress each file before sending it, and to decompress files automatically on the target Agents.

  • Specify a job to Follow Symbolic Links to allow your jobs to transfer files that are linked in your file system through symbolic links.

  • If you are running over a low speed WAN link, enable Incremental Transfer to transfer only the changed bytes of a file instead than the entire file.

  • Select whether to Verify for sufficient disk space before Job transfers start and/or Verify if target directory exists. These verifications ensure that the transfer is not aborted due to insufficient disk space or a nonexistent directory.

  • In MediaReplicator jobs, enable Synchronize Target with Source to remove files from the target directory if they are not found in the source folder.

Note: Use caution when enabling the Synchronize Target with Source option. Any content deleted on the target Agent is not recoverable via Signiant.

Notification And Logging

  • Select the Number of consecutive failures before notification is sent. The default is 1.

  • Select the Log Detail Level. More detailed logs can result in slower transfer performance.

    • Error: Report job errors. Recommended for highest performance.
    • Warn: Report non-critical issues as well as errors.
    • Info: Report job status, details, errors, and non-critical issues.
    • Debug: Report all available job details. Debug logs provide the most detail, but can significantly reduce transfer performance.

SNMP

  • Enter SNMP Trap Receivers. These are host names or IP addresses of systems that are listening for SNMP traps. If required, enter the SNMP Trap Community String and select SNMP Trap Types, which are job completion statuses on which SNMP traps are sent.

Schedule

  • Select additional time parameters such as time zone and start/end times, choose whether a transfer should be interrupted upon failure, and set the job's queuing priority based on other jobs.
Was this page helpful?
About SigniantSigniant’s intelligent file movement software helps the world’s top content creators and distributors ensure fast, secure delivery of large files over public and private networks. Built on Signiant’s patented technology, the company’s on-premises software and SaaS solutions move petabytes of high-value data every day between users, applications and systems with proven ease.LEARN MORE