Testing and optimizing transfer performance

To verify that your system's FASP® transfer is reaching the target rate and can use the maximum bandwidth capacity, prepare a client to connect to an Aspera® server. For these tests, you can transfer an existing file or file set, or you can transfer uninitialized data in place of a source file, which you can delete at the destination, eliminating the need to read from or write to disk and saving disk space.

Using faux:/// as a test source or destination

You can use faux:/// as the argument for the source or destination of an ascp session to test data transfer without reading from disk on the source and writing to disk on the target. The argument takes different syntax that depends on if you are using it as a mock source file or mock source directory.

Note: If you set large file sizes (> PB) in a faux:/// source, use faux:// as a target on the destination because most computers do not have enough system memory available to handle files of this size, and your transfer might fail.

Faux Source File

To send random data in place of a source file (do not read from the source), you can specify the file as faux:///fname?fsize.fname is the name assigned to the file on the destination and fsize is the number of bytes to send. fsize can be set with modifiers (k/K, m/M, g/G, t/T, p/P, or e/E) to a maximum of 7x260 bytes (7 EiB).

For example,

$ ascp --mode=send --user=username --host=host_ip_address faux:///fname?fsize target_path

Faux source directory

In some cases, you might want to test the transfer of an entire directory, rather than a single file. Specify the faux source directory with the following syntax:

faux:///dirname?file=file&count=count&size=size&inc=increment&seq=sequence&buf_init=buf_option

Where:

  • dirname is a name for the directory (required).
  • file is the root for file names. The default is file (optional).
  • count is the number of files in the directory (required).
  • size is the size of the first file in the directory, default 0 (optional). size can be set with modifiers (k/K, m/M, g/G, t/T, p/P, or e/E) to a maximum of 7x260 bytes (7 EiB).
  • increment is the increment of bytes to use to determine the file size of the next file, default 0 (optional).
  • sequence is how to determine the size of the next file: "sequential" or "random". Default is "sequential" (optional). When set to "sequential", file size is calculated as:
    size + ((N - 1) * increment)

    Where N is the file index; for the first file, N is one.

    When set to "random", file size is calculated as:

    size +/- (rand * increment)

    Where rand is a random number between zero and one. If necessary, increment is automatically adjusted to prevent the file size from being negative.

    For both options, increment is adjusted to prevent the file size from exceeding 7x260 bytes.

  • buf_option is how faux source data is initialized: "none", "zero", or "random". Default is "zero". "none" is not allowed for downloads (Ascp run with --mode=recv).

When the defaults are used, Ascp sends a directory that is named dirname and that contains count number of zero-byte files that are named file_count.

For example, to transfer a faux directory (mydir) that contains 1 million files to /tmp on 10.0.0.2, and the files in mydir are named "testfile" and file size increases sequentially from 0 - 2 MB by an increment of 2 bytes:

$ ascp --mode=send --user=username --host=10.0.0.2 faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential /tmp

Faux target

To send data but not save the results to disk at the destination (do not write to the target), specify the target as faux://.

For example, to send a real file to a faux target, run the following command:

$ ascp --mode=send --user=username --host=host_ip_address source_file1 faux://

To send random data to a faux target, run the following command:

$ ascp --mode=send --user=username --host=host_ip_address faux:///fname?fsize faux://

Testing transfer performance

Start a transfer by using the command line. The following tests (except for the sender read test) are using faux:/// so the transfer won't read from the disk. The 5G is the size of the file (for example, 5GB). The argument accepts P (Petabyte), T (Terabyte), G (Gigabyte), or M (Megabyte).

  • Network bandwidth test

    The network bandwidth capacity can be isolated by using the faux:/// and --no-write command line options. The faux:/// implies the --no-read argument, therefore you don't need to specify it in the command.

    These arguments cause ascp to not read from the storage nor to write to the storage, but rather to simulate reading and writing. The faux:/// option is useful because it does not require the specified source file to exist.

    The following example is for a 500 Mbps transfer. Replace 500M with your network bandwidth. The test is only for the network and not for the storage.
    ascp -l 500M --no-write --user user --host destination faux:///bigfile.txt?5G /
  • Destination write test

    You can verify for slowdowns in reading and writing from storage after validating network rates and ensuring that they match what is possible. To test if writing is slow, run an ascp command with and without the --write argument. Use the --no-read or faux:/// argument for both tests to isolate slow writes on the receiver.

    To test the destination write speed, run:

    ascp -l 1G --no-write --user user --host destination --mode send faux:///bigfile.txt?5G /
    Then, compare the results with the ones that you get when you run the following command:
    ascp -l 1G --user user --host destination --mode send faux:///bigfile.txt?5G /

    If there is a noticeable lag between the two commands, the receiver's storage performance might be the cause of the slowdown.

  • Sender read test

    By using the --no-read argument and comparing the results, you might determine whether your sender is responsible for the performance issues. Because you need to be able to test reading a file from disk, the faux:// option can't be used in this situation. Use the --no-write argument in these instances to ensure that you are isolating for the sender.

    To test the sender read, run:

    ascp -l 1G --no-read --no-write --user user --host destination --mode send /path/to/bigfile.txt /
    Then, compare the results with what you get when you run the following command:
    ascp -l 1G --no-write --user user --host destination --mode send /path/to/bigfile.txt /

    If there is a noticeable slowdown between the two commands, it can be an indication that the sender's storage is performing slow.

  • CPU endpoints speed test

    Run the following command with and without the -T argument, then compare the results.

    ascp -T -l 500M --no-write faux:///bigfile.txt?5G user@destination:/

    From the results, if you find a noticeable difference, it would indicate that there is a CPU bottleneck around the encryption, which might be uncommon and indicates the use of CPUs without accelerated encryption instructions. If you find that there is a limitation with the transfer performance, contact the Aspera Technical Support for assistance.

    Note: In production, -T must never be used as it disables in-transit encryption. Use only -T when running a test for a CPU-based bottleneck.

Hardware upgrades for better performance

To improve the transfer speed, you can also upgrade the related hardware components:

Component Description
Hard disk The I/O throughput, the disk bus architecture (such as RAID, IDE, SCSI, ATA, and Fibre Channel).
Network I/O The interface card, the internal bus of the computer.
CPU Overall CPU performance affects the transfer, especially when encryption is enabled.