Object Storage Profiles

Object Mover organizes storage endpoints into storage profiles. Storage profiles allow you to save storage credentials for storage locations allowing you to create multiple jobs without needing to re-enter storage credentials for each new job. Storage profiles also allow you to easily reconfigure jobs to send files to another storage location if required by editing the storage profile.

After creating a Storage Profile, you can use it to create Object Mover jobs.

Object profile configuration can vary depending on your object storage type.

Cloud object storage through Amazon AWS, Microsoft Azure, or Google Cloud uses Signiant Flight to provide accelerated transfers to claimed object storage endpoints via Flight Gateway.

Agents can also transfer to other low-latency, unaccelerated S3 compatible object storage.

Creating Storage Profiles

The Administration > Object Storage > List menu controls your Object Storage Profiles, and allows you to create and edit existing profiles. You can create Storage Object Profiles for AWS S3 Storage, Microsoft Azure Blob Storage, Google Cloud Storage, or S3 Compatible storage.

Claiming Endpoints

Before you create a profile for an AWS S3 bucket, Azure Blob container, or Google Storage bucket, you must claim the endpoint in your Flight Administration Console. If your endpoint is not claimed, you will not be able to transfer files to or from your storage location.

AWS S3 Storage

Ensure that the user associated with the AWS access key has the following object and bucket operation permissions:

• AbortMultipartUpload
• DeleteObject
• GetObject
• ListBucket
• ListMultipartUploadParts
• PutObject
• GetBucketLocation

Note: Amazon S3 Glacier is not supported.

To create a new AWS S3 Storage Profile:

1. In your Manager, navigate to Administration > Storage Profiles > List.
2. Click Add AWS S3 Storage.
3. On the Definition tab, enter your storage details:
   • Name: The profile display name.
   • Bucket: Your AWS bucket name.
   • SubFolder: The target subfolder.
   • Access Key: Your AWS access key.
   • Secret Key: Your AWS secret key.
4. Click OK to save the profile.

Azure Blob Storage

To create a new Azure Object Storage Profile:

1. In your Manager, navigate to Administration > Storage Profiles > List.
2. Click Add Azure Storage.
3. On the Definition tab, enter your storage details:
   • Name: The profile display name.
   • Storage Account: Your Azure storage account name.
   • Container: Your Azure storage account container name.
   • SubFolder: The target subfolder.
   • Access Key: Your Azure access key.
4. Click OK to save the profile.

Google Cloud Storage

In order to transfer files to and from Google Cloud Storage, you must create a Service Account. Create the account using the Google Cloud Console, by navigating to IAM & Admin > Service Accounts.

Create a new Service Account with Storage Object Admin, Storage Object Creator and Storage Object Viewer account permissions.

Once you have created a storage account, create and download the Service Account Key from the Service accounts list in JSON format, and create the storage profile for Object Mover.

To create a new Google Cloud Storage Profile:

1. From the Manager navigation menu, select Administration > Storage Profiles > List.
2. Click Add Google Cloud Storage.
3. On the Definition tab, enter your storage details:
   • Name: The profile display name.
   • Bucket: Your Google Storage Bucket name.
   • SubFolder: The target subfolder.
   • JSON Credentials File: Your Service Account Key obtained from Google.
4. Click Choose File and select your Service Account Key JSON file.
5. Click Open to upload the file.
6. Click OK to save the profile.

S3 Compatible Storage

S3 compatible storage can be any third-party object storage that is not hosted by Amazon. S3 compatible storage profiles do not require you to claim the endpoints in Flight.

To create a new Local S3 Object Storage Profile:

1. In your Manager, navigate to Administration > Storage Profiles > List.
2. Click Add S3 Compatible Storage.
3. On the Definition tab, enter your storage details:
   • Name: The profile display name.
   • Storage Server: Your S3 Storage Endpoint.
   • Bucket: Your S3 Bucket name.
   • SubFolder: The target subfolder
   • Bucket Access Style: Your S3 storage access style
     • Virtual Host Style
     • Path Style
   • Access Key: Your S3 Access Key
   • Secret Key: Your S3 Secret Key
4. Click OK to save the profile.

Advanced Settings

S3 compatible storage can often require additional configuration settings to ensure compatibility. Click the Advanced tab to view additional options:

• Endpoint Qualifier is used to provide a specific region or signature for S3 compatible storage locations that cannot be determined via us-east-1.
• Enable Checkpoint enables the ability for transfers to resume from a certain point if they are interrupted.
• Use SSL sets whether or not the storage profile uses SSL encryption when connecting to a storage location.
  • You can configure whether to Verify Hostnames to add additional security if your storage supports this feature.
  • Certain storage locations require you to enable Allow Self Signed Certificate to use them as SSL secured storage.

Setting Storage Profile Permissions

Permissions allow administrators to control user and group access to management objects. For information on enabling permissions, see Understanding Users.

Deleting Storage Profiles

Before you can delete a storage profile, you must remove it from any jobs in your Manager.

To delete a storage profile:

1. From the Manager navigation menu, select Administration > Storage Profiles > List.
2. Select the profile you want to delete and click Delete.
3. Click Yes to confirm.