Setting up Aspera Cluster Manager

You must set up Aspera Cluster Manager (ACM) as part of a high availability installation.

For overview information about ACM, see Aspera Cluster Manager (Overview).

Installing ACM Software on the Shared Storage

  1. Obtain the latest ACM software from IBM Aspera and place it in the shared /mnt/shared/orchestrator/acm_data/ directory.
  2. Extract the ACM software on the dedicated shared volume by running the following command, as root:
    $ cd /mnt/shared/orchestrator/acm_data/
    $ tar xzvf acm4orchestrator-0-9.tar.gz
    Note: You only need to run this step on one node; the /mnt/shared/orchestrator/acm_data/ directory is shared by both orchestrator servers.
  3. Create a symbolic link by executing the following commands on both Orchestrator servers:
    $ cd /opt/aspera
    $ ln –s /mnt/shared/orchestrator/acm_data/ ./acm
  4. Confirm that the file database.yml is present in the following directory:
    /opt/aspera/orchestrator/config/
    If you are using a distributed architecture, with MySQL running on servers that are separated from the Orchestrator servers, copy the database.yml file from the /opt/aspera/orchestrator/config/ directory on any Orchestrator server into the /opt/aspera/acm/config/ directory on any MyQL server.
  5. Confirm that the MySQL user and password defined in database.yml match the MYSQL_USER and MYSQL_PWD values as defined in the following file:
    /opt/aspera/acm/bin/acmcommon
  6. Configure the services to run in the following directory:
    /opt/aspera/acm/bin/acmcommon
    By default, all services run together, as in the following example:
    SERVICES=('APACHE' 'ORCHESTRATOR' 'MYSQL')
    If you are running Orchestrator in a distributed architecture (MySQL and Orchestrator running on different servers), follow these additional steps.
    • To run Orchestrator + Apache services only, set the following:
      SERVICES=('APACHE' 'ORCHESTRATOR')
    • To run MySQL services only, set the following:
      SERVICES=('MYSQL')
    Note: In a distributed architecture, the ACM mount points on /opt/aspera/acm/ must point to different folders on the shared systems for the Orchestrator + Apache services and the MySQL systems. This is required because SERVICES must be set differently on those systems.
    For example, in /etc/fstab on the Orchestrator + Apache server, you could have the following:n
    10.0.75.10:/home/mnt/shared/acm_data_orchestrator/ /mnt/shared/orchestrator/acm_data/ nfs4 rw,noac,sync,hard,intr 0 0
    On the server for MySQL, you could have the following:
    10.0.75.10:/home/mnt/shared/acm_data_mysql/ /mnt/shared/orchestrator/acm_data/ nfs4 rw,noac,sync,hard,intr 0 0
    
  7. To configure the ACM log file to be /var/log/aspera.log, managed by the rsyslog service, on each node, modify /etc/rsyslog.conf with the following steps.
    1. Add these lines at the end of the file (note that # Aspera Logging is text, not a command):
      # Aspera Logging
      local2.* -/var/log/aspera.log
    2. Replace all cron.none occurrences with the following:
      cron.none;local2.none
    3. Replace /var/log/messages occurrence with -/var/log/messages
    4. Restart rsyslog:
      $ /etc/init.d/rsyslog restart
  8. Set up your system to rotate the logs.

    You may find that your log file, /var/log/aspera.log is growing too quickly. In that case, there are several ways to rotate Aspera logs:

    • "Option A": Add /var/log/aspera.log to the following directory:
      /etc/logrotate.d/syslog
    • "Option B": Create an entry for aspera.log in the following file:
      /etc/logrotate.conf
    • "Option C": Create a separate configuration file for aspera.log in the following directory:
      /etc/logrotate.d/

    Option A will rotate your logs with the system logs (usually once a week, compressed, and saving the last 10 logs). However, on some servers, there is so much traffic that the logs need to be rotated more often than once a week; in that case, use Option B or C.

    Option A: Add /var/log/aspera.log to the entries in /etc/logrotate.d/syslog, as follows:

    /var/log/messages 
    /var/log/secure 
    /var/log/maillog 
    /var/log/spooler 
    /var/log/boot.log 
    /var/log/cron 
    /var/log/aspera.log { 
    sharedscripts 
    postrotate 
    /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true 
    /bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> /dev/null || true 
    endscript 
    }

    Option B: Edit /etc/logrotate.conf by adding the configuration after the line # system-specific logs may also be configured here.

    The following example compresses and rotates 10 logs whenever /var/log/aspera.log reaches 100MB. After log rotation is complete, it will run whatever scripts are specified by postrotate ... endscript.

    /var/log/aspera.log {
    rotate 10
    size 100M
    create 664 root
    postrotate
    /usr/bin/killall -HUP syslogd
    endscript
    compress
    }

    The following example compresses and rotates 10 logs once daily. Instead of moving the original log file and creating a new one, the copytruncate option tells logrotate to first copy the original log file, then truncate it to zero bytes.

    /var/log/aspera.log {
    daily
    rotate 10
    copytruncate
    compress
    }

    Option C: Create a separate /etc/logrotate.d/aspera configuration file containing the same information as Option B.

    If you find that logs are being overwritten before long transfers of many files are complete, you can increase the log size. For more information, see the IBM Aspera High-Speed Transfer Server Admin Guide.

  9. Now that you have completed this procedure, you can review the steps in Setting up Aspera Cluster Manager.

Turning Off Automatic Restart for the MySQL, Apache, and Orchestrator Services

The chkconfig command turns off the automatic restart process for the Aspera services after reboot. Disabling automatic restart is essential in a high availability environment so that ACM can control the restart process as needed to maintain high availability.
  1. Turn off the MySQL, Apache, and Orchestrator services by running the following:
    # chkconfig aspera_mysqld off
    # chkconfig aspera_httpd off
    # chkconfig AsperaOrchestrator off
  2. If you are using a Centos 7.x platform, run the following additional steps after turning off the services.
    1. After reboot, turn off (disable) the automatic restart of the orchestrator service:
      # systemctl disable orchestrator_asctl.service
    2. Remove the symbolic link:
      # /etc/systemd/system/multi-user.target.wants/orchestrator_asctl.service
    3. To confirm that it is disabled, run the following:
      # systemctl list-unit-files | grep orchestrator_asctl.service
      orchestrator_asctl.service                  disabled

Configuring the crontab Entry to Run ACM

Configure ACM services in crontab on both nodes so that the acm4orchestrator script is launched every minute.

Use the crontab –e command to configure your entry as follows.
$ crontab –e
* * * * * /opt/aspera/acm/bin/acm4orchestrator ip_address > /dev/null 2>&1
One parameter must be entered in crontab: the IP address of the host where the script is running. This parameter is passed to the acm4orchestrator script.
In the example below, the IP address is 10.0.71.21.
$ crontab –e
* * * * * /opt/aspera/acm/bin/acm4orchestrator 10.0.71.21 device_ID > /dev/null 2>&1
Find the device ID:
$ stat -c "%d" mount_point
For example:
$ stat -c "%d" /mnt
In the example above, /mnt is the shared storage location.
Create the mount to the shared storage:
$ mount -v -o vers=3,proto=tcp,port=2049 shared_storage mount_point
For example:
$ mount -v -o vers=3,proto=tcp,port=2049 ha_storage.aspera.us /mnt

Once configured in crontab, the acm4orchestrator script runs regularly to determine the active node and start the required Orchestrator services on both the active and passive nodes, depending on their current status (active or passive).

Obtaining the crontab Parameter Values

To list the IP addresses available on a system, run the following command:
$ ip addr | grep "inet"
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
inet 10.0.75.21/16 brd 10.255.255 scope global eth0

Running an ACM Sanity Check

The acmctl command has an option to check that the necessary configurations have been made that allow the acm4orchestrator script to run appropriately. You should make sure that each server passes the sanity test.

  1. Run the acmctl command with the -s option on both nodes to verify the basic ACM prerequisites.

    Note that the status of the Checking if an entry for ACM seems to exist in crontab displays as KO, not OK, in the example below, because the user has not yet created the crontab entry that will run the ACM software on each server.

    $ /opt/aspera/acm/bin/acmctl –s
    
    ACM sanity check
    ----------------
    Checking if an entry for ACM seems to exist in the crontab     OK 
    Checking that the orchestrator master service is disabled in chkconfig     OK 
    Checking that SE Linux mode is not set to enforcing     OK
    
  2. Correct any task that does not pass the sanity check (except for tasks with crontab status; those tasks are addressed in the next section).
  3. Run the following command, where ip_address is the IP address of the host where the script is running:
    /opt/aspera/acm/bin/acm4orchestrator ip_address
    Examine the output to make sure there are no displayed errors.

Identifying the Status of ACM on Each Orchestrator Server

The following command can be used to identify which Orchestrator server is active and which is passive:
/opt/aspera/acm/bin/acmctl -i
For more information about using this command, see Disabling and Re-enabling ACM on One Node

Connecting to Orchestrator with the VIP

If a load balancer is providing a virtual IP address (VIP) in front of the Orchestrator servers, and the services are running properly, you can now connect to the Orchestrator application using the VIP assigned to the ACM cluster.