Sync has more than 80 options that can be used when composing an
async session, but only a few are required, and for security by using several
others.
For a complete list and descriptions of available options, see the Async command reference. For configuration and option usage that is required to synchronize with AWS S3 storage, see
Synchronizing with AWS S3 Storage.
Note: These instructions describe how to compose a bidirectional async session
between a Windows client and a Linux server. They include the required and the recommended options
in the correct order. You can use the short form or long form (POSIX) option tags. The complete set
of commands by using both tag formats are summarized at the end of the instructions.
-
Confirm that both endpoints have Sync-enabled licenses and that the remote endpoint is running
an Aspera transfer server application (HSTS or HSTE).
Run ascp -A in the command line and look for sync2
in the
Enabled settings
section.
-
Begin by starting async.
-
Enter instance options.
Instance options are used to configure the local (client) computer for the
async session and are mostly optional. Include
-L
log_dir
(or
--alt-logdir=log_dir
) to
set client-side logging to a directory that you can access because you might not have permission to
access the log in its default location (see
Logging). The logging directory
must not be in the directory that is being synchronized.
For example, if the Windows
client's username is
morgana, morgana can use
-L
to log to a
directory in the home folder:
async -L "C:\Users\morgana\Aspera jobs\log"
In
this example, the path must be in quotation marks because the path includes a folder name that
contains a space. For more information about path formatting, see Async command reference.
-
Name the session by using the
-N
option (or
--name=pair
).
-N pair
is required in
async commands. The value for
pair is a name that uniquely
identifies the
Aspera Sync session and is visible in
IBM Aspera Console.
-N pair
must follow any instance options and must precede all
session arguments. Names can use only standard alphanumeric characters, plus "
_
"
and "
-
" characters.
Note: If your remote host is an Aspera cluster, ensure that your
session name is unique by naming the session with a descriptive string followed by the UUID of the
local host, such as
"cluster-sync-ba209999-0c6c-11d2-97cf-00c04f8eea45".
For example, name the session job1:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1
Once you name the session, you enter the session options. Session options define the transfer
parameters including authentication, transfer rate and policy, database storage, and the folders to
synchronize.
-
Provide authentication credentials.
Sync supports three methods of authenticating to the server: SSH key, password, and basic
token. Use SSH keys, unless your server requires a basic token.
- SSH key: To use SSH key authentication, your SSH public key must be configured on the
remote server. For instructions on creating keys and setting them up on the server, see the IBM Aspera High-Speed
Transfer Server. Specify the path to your private key file by using the
-i
file
(or --private-key-path=file
)
option.
- Password: The password is the one associated with the Aspera transfer user account on the
server. You can provide the password as an environment variable (ASPERA_SCP_PASS) or when prompted
after starting the command.
- Basic token: Basic tokens are used for synchronizing to Aspera products that require
access key authentication, such as IBM Aspera on Cloud transfer service (AoCts) or IBM Aspera Transfer Cluster Manager (ATC Manager). For instructions on creating the basic token,
see Sync with basic token authorization. You can provide the token as an environment variable
(ASPERA_SCP_TOKEN) or in the command line by using the
-W
token_string
(or
--token=token_string
) option.
For example, use -i
and specify the path to morgana's SSH private key in their
home folder:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa
In this case, the path to the SSH key can use platform independent of path separators
(/
) and be entered without quotation marks around it because it does not have a
space in it.
-
If the local data is stored on a mount or object storage, specify the locations for the local
snapshot database.
The snapshot database cannot be located on CIFS, NFS, or other shared file systems mounted on
Linux. If the local files and directories that are specified in the previous step are on a mount,
you must specify a local location by using -b db_dir
(or
--local-db-dir=db_dir
). The database must not be in the
directory that is being synchronized.
For example, use -b
to store the local snapshot database in morgana's
"Aspera jobs
" folder:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db"
-
Set transfer parameters.
The same transfer rate and transfer policy options that are used to control
ascp transfers can be applied to async sessions. Set a target
rate that is based on your available bandwidth and system capabilities. Set the target (maximum)
rate by using -l rate
(or
--target-rate=rate
).
For example, use
-l
to set the target rate to 500
Mbps:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m
-
Specify the local directory for synchronization.
Enter the local directory by using -d ldir
(or
--local-dir=ldir
).
For example, use -d
to set the local directory to morgana's
data folder:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m -d c:/users/morgana/data
-
Specify the transfer username, remote host, and remote directory for synchronization.
Unlike previous options for which one short option flag was equivalent to one long option
flag, when specifying the username, remote host, and remote directory, the short flag option is the
equivalent of one to three long option flags. For example, if the username is
morgana, the remote host IP address is
10.0.0.1, and
the remote directory is
/data, then the following options are equivalent to
each other:
-r morgana@10.0.0.1:/data
--remote-dir=morgana@10.0.0.1:/data
--user=morgana --remote-dir=10.0.0.1:/data
--user=morgana --host=10.0.0.1 --remote-dir=/data
If the name of your remote directory
contains an "@", use the --user
option so that the "@
" is not
treated specially in the argument for --remote-dir
.
For example,
use
-r
to set the username, remote host, and remote
directory:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m -d c:/users/morgana/data -r morgana@10.0.0.1:/data
-
If a source directory is on an NFS or CIFS mount, require Sync to use the mount signature
file.
Warning: If you do not use the mount signature file and the NFS or CIFS mount is
unreachable, Sync considers those files as deleted and deletes them from the other
endpoint.
If the local endpoint is on an NFS or CIFS mount and the Sync is push or
bidirectional, use
--local-mount-signature
. If the remote endpoint is on an NFS or
CIFS mount and the Sync is pull or bidirectional, use
--remote-mount-signature
.
-
Specify the locations for the remote Sync log and database.
On the server, Sync logs to the default location (see
Logging) if no
location is specified for
<async_log_dir>
in the server's configuration file.
Use
-R
(or
--remote-logdir
) to specify a logging location to which
you have access. The location must be within your docroot on the server, unless you are
synchronizing with AWS S3 object storage.
-R
is overridden by the server's
configuration file. If you are restricted to aspshell on the server, you cannot use this
option.
Also, you can use -B
(or --remote-db-dir
) to specify a
location for the remote Sync database. As with the log file, the location must be within your
docroot, it is overridden by <async_db_dir>
in the server's configuration file,
and you cannot use this option if you are restricted to aspshell.
As on the local computer,
the Sync log and database must not be in a directory that is being synchronized.
For example, to
set the remote log and snapshot database files to morgana's home
folder:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m -d c:/users/morgana/data -r morgana@10.0.0.1:/data -R /morgana/async/log -B /morgana/async/db
-
Specify the synchronization mode.
Sync can be run in three modes:
push
: The contents of ldir are synchronized to
rdir, with the ldir content that overwrites the
rdir content, by default (unless the overwrite options are specified otherwise,
such as to overwrite only if rdir is older, or never overwrite).
pull
: The contents of rdir are synchronized to
ldir, with the rdir content that overwrites the
ldir content, by default.
bidi
(bidirectional): The contents of ldir and
rdir are synchronized, with newer versions of files and directories that
overwrite older versions in either ldir or rdir, by
default.
To synchronize the remote folder with the local folder use
-K push
(or
--direction=push
).
For example, use
-K bidi
to do
a bidirectional
sync:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m -d c:/users/morgana/data -r morgana@10.0.0.1:/data -R /morgana/async/log -B /morgana/async/db -K bidi
-
Preserve file attributes.
When a file or directory is transferred between computers, the file is written to the
destination as the transfer user and the file modification time (and creation time on Windows) are
reset. Most users prefer to preserve timestamps from the source to the destination by using the
-t
option.
For example, use
-t
timestamps:
async -L "C:\Users\morgana\Aspera jobs\log" -N job1 -i c:/users/morgana/.ssh/id_rsa -b "C:\Users\morgana\Aspera jobs\db" -l 500m -d c:/users/morgana/data -r morgana@10.0.0.1:/data -R /morgana/async/log -B /morgana/async/db -K bidi -t
Note: When
synchronizing between Unix-like operating systems, you can also preserve the user IDs (uid) and
group IDs (gid) from the source to the destination by using the options -u -j
(equivalent to --preserve-uid --preserve-gid
).
Extended file attributes
and ACLs can also be preserved; see the Async command reference. When using
--dedup
, file metadata preservation is supported for
copy
.
SummaryThe instructions created the following Sync
session, which is shown by using short option flags and POSIX (long) flags. Each option is shown on
a separate line for clarity, but must be entered in the command line as a single line.
Warning: This example does not include the option to make Sync check for a mount signature
file. If a source is on an NFS or CIFS mount, include
--local-mount-signature
and
--remote-mount-signature
to prevent Sync from deleting files on an endpoint if a
mount becomes unavailable. For instructions, see
Configuring Sync endpoints.
Using
short-format option flags:
async
-L "C:\Users\morgana\Aspera jobs\log"
-N job1
-i c:/users/morgana/.ssh/id_rsa
-b "C:\Users\morgana\Aspera jobs\db"
-l 500m
-d c:/users/morgana/data
-r morgana@10.0.0.1:/data
-R /morgana/async/log
-B /morgana/async/db
-K bidi
-t
Using long-format option flags:
async
--alt-logdir="C:\Users\morgana\Aspera jobs\log"
--name=job1
--private-key-path=c:/users/morgana/.ssh/id_rsa
--local-db-dir="C:\Users\morgana\Aspera jobs\db"
--target-rate=500m
--local-dir=c:/users/morgana/data
--user=morgana
--host=10.0.0.1
--remote-dir=/data
--remote-logdir=/morgana/async/log
--remote-db-dir=/morgana/async/db
--direction=bidi
--preserve-time
If the session is between Linux computers, it also includes the
following session options:
-u
-j
Or using long-format option flags:
--preserve-uid
--preserve-gid