Running distcp in Kerberized and non-Kerberized cluster test

  1. Run distcp to copy a sample file from the local native HDFS to the remote HDFS Transparency in a non-Kerberized cluster:
    [hdfs@c16f1n07 root]$ hadoop distcp -skipcrccheck -update 
hdfs://c16f1n07.gpfs.net/tmp/redhat-release hdfs://c16f1n03.gpfs.net:8020/tmp

[hdfs@c16f1n07 root]$ hadoop fs -ls -R hdfs://c16f1n03.gpfs.net:8020/tmp
-rw-r--r--   1 hdfs      root         52 2018-03-19 23:26 hdfs://c16f1n03.gpfs.net:8020/tmp/redhat-release
  2. Run distcp to copy a sample file from the remote HDFS Transparency to the local native HDFS in a Kerberized cluster:
    [hdp-user1@c16f1n07 root]$ klist
Ticket cache: FILE:/tmp/krb5cc_11015
Default principal: hdp-user1@IBM.COM
Valid starting       Expires              Service principal
03/19/2018 22:54:03  03/20/2018 22:54:03  krbtgt/IBM.COM@IBM.COM

[hdp-user1@c16f1n07 root]$ hadoop distcp -pc 
hdfs://c16f1n03.gpfs.net:8020/tmp/redhat-release hdfs://c16f1n07.gpfs.net:8020/tmp

[hdp-user1@c16f1n07 root]$ hadoop fs -ls hdfs://c16f1n07.gpfs.net:8020/tmp/redhat-release
-rw-r--r--   3 hdp-user1 hdfs         52 2018-03-20 01:30 hdfs://c16f1n07.gpfs.net:8020/tmp/redhat-release