Configuring Aspera On-Demand on AWS for Huge File transfers

The Aspera On-Demand Server contains the core features from Aspera's Connect Server,  Aspera’s management application, Aspera On-Demand Console for Amazon Web Services (AWS). The On-Demand Server is stored as an Amazon Machine Image (AMI) in AWS account.
Note: Aspera Inc will only share private Amazon Machine Image (AMI) once you have placed a purchase or evaluation order with them.

Aspera On Demand Core Features:

• fasp Transport Server: A transport server that accepts incoming Aspera connections.
• ascp command-line:  A command-line transfer program to initiate Aspera transfers.
• Simple Sync:  A command-line synchronization program.
• Connect Server Web UI:  A web-based transfer server.
• On-Demand Console Features:
• Transfer monitoring and control: View transfers, pause / resume / cancel, change transfer rates
• Transfer Initiation: Initiate and schedule transfer jobs remotely.
• Email Notification: Notify users of transfer events with customizable messages.
• Reporting: Create detail and summary reports of transfer activity.

Aspera On Demand Server Overview:
The Aspera On-Demand Server is packaged as an Amazon Machine Image (AMI). The Aspera AMI contains the Connect Server, and the Aspera On-Demand Console for AWS. When booting up the Aspera AMI, you can use the Connect Server for fasp™ file transfers, and the On-Demand Console for AMI's transfer server management.

How to access Aspera On-Demand Console ?

Type the following on the browser :
Browser : http://<EC2 instance's Public DNS>/aspera/console 
Login: admin/(instance id)
When logged in to On-Demand Console as an administrator, we should see the following items in the Console menu:

• Dashboard: Overview of current and recent transfers, and simple Aspera EC2 node monitoring
• Activity: Lists transfer activities involving at least one managed nodes
• Nodes: Manage the On-Demand Server (managed node) and the unmanaged nodes
• Users: Create and manage On-Demand Console users
• Groups: Create and manage On-Demand Console groups
• Transfer: Initiate a file transfer between Aspera EC2 nodes
• Reports: Generate reports on transfers
• Preferences: Configure your account preferences
• Config: Configure the On-Demand Console in this module
• Search: Search for the most recent transfers that associate to key word(s) entered by users. Search Results shows up to 20 of the most recent transfers.

Instructions to set up Aspera On-Demand server in AWS:


Step 1: Register the Aspera On demand server
Before using the Aspera On-Demand Server, make sure you have registered the Aspera On-Demand for AWS. You can find the registration page here: http://asperasoft.com/ondemand/reg

Step 2: Configure the AWS security groups for File Transfer
create a Security Group that opens the necessary ports on firewall settings for the Aspera On-Demand Server. 
>Go to the Security Groups in the AWS Console Navigation sidebar, click Create Security Group.
> Create a new group for the Apsera On-Demand Server, open the following ports:
HTTP - TCP/80
HTTPS - TCP/443
Custom (SSH) - TCP/33001
Custom (fasp) UDP/33001
You can use your existing key pair for the Aspera AMI. If you wish to generate a new key pair, go to the Navigation sidebar, click Create Key Pair. Enter the key pair name and click Create, the private key will be downloaded to your computer.

Step 3: Launch the Aspera AMI:
In the AWS Management Console, go to the AMIs in the Navigation sidebar, search for the AMI with the keyword aspera, or the exact name referred in the subscription-confirmation page. Launch the AMI. Choose the EC2 instance type according to your transfer needs. Note: If you need large amounts of file to be transferred in short time period then larger instance type is recommended.

Step 4: Log in the On-Demand Console:
When the Aspera AMI is launched, locate it under the Instances in the AWS Console Navigation sidebar. You can obtain the following On-Demand Server information:
With Login & Password 
Address:  Public DNS

Step 5: Transfer Files with the Aspera On-Demand Server:
When the On-Demand Server is set up, and at least one system/transfer account is created, you can transfer files with the On-Demand Server through the Web UI, fasp™ Transfer Server, or the On-Demand Console. 
If you have added the un-managed nodes into the On-Demand Console, it can be used to initiate transfers between nodes when the Console user has the permission to start transfers. It provides two types of transfer methods:
A) Simple Transfer:  one-time transfer session that requires entering all transfer info.
B) Smart Transfer: A transfer template that stores all the transfer settings and can easily be reused.

Simple Transfer:To start a Simple Transfer in the Transfer screen, click Simple Transfer. Fill out the Start a Simple Transfer form. When finished, click Transfer to initiate the transfer.

Description: Enter the name and comment of the transfer. This can be helpful if you want to search for this transfer later.
Source: Select the source endpoint and files to transfer.
Selected Source Items: The selected files and folders to transfer will be added into the Selected Source Items panel.

Destination:  Select the destination endpoint and the directory to receive the file.
Destination Directory:  The selected folder to receive the files will be added into this panel.

Console Transfer Options: In both the Simple and Smart Transfer, we can find more transfer settings in the More Options panel:
Connection: fasp™ Port (UDP)  Specify the UDP port for fasp file transfers.
Transfer >
Target rate         : Specify the transfer target rate.
Minimum rate      : Set the transfer minimum rate
Bandwidth policy : Choose a transfer policy among fixed/high/fair/low.
Retry policy        :  Check the option to enable the retry policy, and specify the number of attempts and the duration.

Session Details: We can view the session details in the console once the transfer is over.

Aspera On-Demand vs SCP Comparison Matrix:

The following table illustrates the comparison (time taken) between Aspera On-demand and SCP For transfering data between Amazon EC2 USA-EAST and  US-West region.

Note :

• 50 GB Single compressed file when uncompressed translates to 500 GB file size. 
• Default settings of Aspera On-demand on AWS was used 

Source : US-East  | Target: US-West
Instance Type	File Size / Type	Aspera On-Demand (time)	SCP (time)
m1.large	5 GB Single File Compressed	3m 21s	-
m1.xlarge	50 GB Single File Compressed	35m 32s	-
m2.4xlarge	50 GB Single File Compressed	33m 58s	2hr 1min

 References :

http://cloud.asperasoft.com/aspera-on-demand/

http://asperasoft.com