Server Setup in Google Cloud Storage
If you have an Aspera consumption-based entitlement, the server can be installed on an instance in Google Cloud Storage and run as a self-managed, cloud-based server that enables high-speed transfers with your Google Cloud Storage.
About this task
Prerequisites:
- You have a Google Cloud account.
- You have a bucket in Google Cloud Storage.
- You can SSH into your Google instance as root. For instructions, see https://cloud.google.com/compute/docs/instances/connecting-to-instance.
- Your Linux Kernel is 2.6.34 or higher. Run the following
command to confirm:
$ uname -a
Create a VM Instance
Procedure
- Log in to Google Developers Console.
- Select the relevant project or create a new one.
-
Generate an SSH key pair on the remote computer that will be used to
connect to the instance.
# ssh-keygen -t rsa
-
Create a new instance.
Click Compute Engine > VM Instances > Create Instance. For more details, see https://cloud.google.com/compute/docs/instances/creating-and-starting-an-instance.
-
In the VM instances dialog, enter the details of
your instance.
Enter the name, zone, machine type, boot disk, and firewall, and select the HTTP or HTTPS boxes. Select a Service account from the dropdown menu.
-
Click Identity and API access.
Select Set access for each API, scroll down to Storage, and select Full.
-
Click Management, disk, networking, SSH keys.
- Click SSH Keys and add the SSH key that you created.
- Click Create to create your instance.
Configure the Instance to Support Aspera
Procedure
-
SSH into your instance as root.
# ssh username@gc_instance_ip_address
When the connection is established, elevate to root privileges:
$ sudo -i
-
Connect to your instance as an administrator by using Remote Desktop
Protocol or Powershell.
For instructions, see https://cloud.google.com/compute/docs/instances/windows/connecting-to-windows-instance.
-
Update sshd_config to enable port 33001 and password
authentication.
Edit /etc/ssh/sshd_config and make the following changes:
- Enable TCP/33001 by adding the text
Port 33001
. For example,... Port 22 Port 33001
- Enable password authentication by uncommenting the line
#PasswordAuthentication no
and changing the value toyes
. For example,... PasswordAuthentication yes
Save your changes.
- Enable TCP/33001 by adding the text
-
Restart the sshd service to activate the changes.
# systemctl sshd restart
-
Configure the local firewall.
Ensure that the local firewall is configured to support Aspera. Allow inbound connections on TCP/22, TCP/33001, and UDP/33001.
-
Verify that /etc/hosts contains an
entry for
127.0.0.1 localhost
. -
Disable SELinux.
For instructions, see Disabling SELinux.Warning: If this procedure is done incorrectly, you system might be unable to boot.
-
Confirm that the Google firewall allows connections on the ports used by
Aspera FASP (TCP/22, TCP/33001, and UDP/33001).
If the ports are not open, you can create firewall rules in the Google Cloud Platform console by going to App Engine > Firewall Rules. Allow TCP/33001, UDP/33001, and, if you need access to the Node API, TCP/9092.You can also use gcloud, an internal tool built into the CentOS image, to configure the firewall. For more information on using gcloud, see https://cloud.google.com/compute/docs/gcloud-compute/. For more information on configuring Google firewall, see https://cloud.google.com/compute/docs/networking#firewalls.
To open TCP/33001 for SSH connections, run the following command:
# gcloud compute firewall-rules create allow-fasp-ssh --description "fasp ssh channel" --allow tcp:33001
To open UDP/33001 for FASP transfers, run the following command:
# gcloud compute firewall-rules create allow-fasp-data --description "fasp data channel" --allow udp:33001
If you need to access the Node API, you need to open TCP/9092 by running:
# gcloud compute firewall-rules create allow-node-api --description "node api channel" --allow tcp:9092
Install, Configure, and Enable the Aspera Server
Procedure
- Install HSTS on your VMI.
-
Create Aspera system user accounts and set passwords.
HSTS uses the system accounts to authenticate connections and these must be in place before you can transfer.
-
Set the MTU size on the instance to 1460.
# asconfigurator -x "set_node_data;transfer_protocol_options_datagram_size,1460"
-
Enable your entitlement and register by running the following commands:
# /opt/aspera/bin/asalee-config.sh enable # systemctl asperanoded restart # /opt/aspera/bin/alee-admin register customer_ID entitlement_ID
To entitle Faspex (v.3.7.8+), Shares (v.1.7.3+), or Console (v.2.3.2+), run the corresponding command.- To entitle
Faspex:
# export RAILS_ENV=production # asctl faspex:rake entitlement:config_license_server EL_KEY="entitlement_id" EL_CUSTOMER_ID="customer_id"
- To entitle
Shares:
# /opt/aspera/shares/bin/run bash -c 'cd /opt/aspera/shares/u/shares && RAILS_ENV=production bundle exec rake aspera:ami:entitlement:config_license_server EL_KEY="entitlement_id" EL_CUSTOMER_ID="customer_id"
- To entitle Console:
# cd /opt/aspera/console/ # export RAILS_ENV=production # export PATH=/opt/aspera/common/ruby/bin:$PATH # aspera:ami:entitlement:license_mode_on* # rake aspera:ami:entitlement:config_license_server EL_KEY="entitlement_id" EL_CUSTOMER_ID="customer_id"
- To entitle
Faspex:
-
Enable the Aspera Trapd service by running the following command:
# /opt/aspera/bin/astrap-config.sh enable
-
Set the transfer user's docroot to Google Cloud Storage.
# asconfigurator -x "set_user_data;user_name,username;absolute,
gs:///my_bucket/my_path
"# service asperanoded restart
-
Verify the docroot for the transfer user.
Run the following command to view the settings associated with the transfer user:
# /opt/aspera/bin/asuserdata -u username
The output for
docroot option set
should be:... docroot option set: canonical_absolute=
gs:///my_bucket/my_path
/ canonical_show_as=/ absolute: "gs:///my_bucket/my_path
/" -
Run a test transfer.
-
Install a free Aspera client application if you do not have one on your local computer.
You can download the IBM Aspera Desktop Client from https://www.ibm.com/products/aspera/downloads.
-
Run a test transfer:
# ascp -P 33001 --policy=fair -l 10000 local_filepath username@gc_instance_ip_address:/
Where local_filepath is a directory on the local machine with the files you want to transfer to Google Cloud. If your set up and transfer command are successful, files appear in your Google Cloud Storage bucket.
-
Install a free Aspera client application if you do not have one on your local computer.