Configure a Datadrop connection

Datadrop is the Apptio Secure File Transfer Protocol (SFTP) server. You can push your data files to the Datadrop server and not have to worry about provisioning or maintaining the server.

On Datalink, the Datadrop connector includes the ability for you to self-provision your SFTP server. Once requested, the server will be provisioned instantaneously. In contrast, on Datalink, you must ask the TBMA to provision an SFTP server, which often takes 1-2 days.

The Datadrop connector supports only r12 as a destination.

You will generate a public-private key pair to provision the server and access your files. Every customer has their own public-private key pair and thus you have access only to the folders that store your files. You can generate ten private keys and add them to Datadrop. This means that ten users can use Datadrop without needing to share key pairs.

You can use the Datadrop viewer to view the files present on the SFTP server. You cannot use it to transfer files to the server. Once the SFTP server has been provisioned, any files created or transferred to the server will be immediately visible in the viewer without the need for any additional settings.

After the instance has been provisioned, the user can be deleted from the same page. Once deleted, the old username and private key can no longer be used to access the server.

Datalink Datadrop migration to AWS is in progress. Migration will move the data and it will be reconciled. Customers can raise a request if it is not reconciled after migration.

Any file that gets uploaded to Datadrop is stored until customer delete the files via SFTP client. Same applies in case of archive.

Configure a Datadrop connection

If you use an existing extract, transform, load (ETL) tool, you can push data files to Datadrop, as shown in the following image. Customer Data , shown in the image, may require multiple connections and data sources. Datadrop allows customers to control the data and the time the data is pushed externally.

The Apptio -hosted Datalink agent pulls data files from Datadrop and transfers them to your Apptio instance.

Datadrop uses public/private SSH keys for authentication.

To implement automated transport using SFTP through datadrop.apptio.com, you need to generate an SSH encryption key pair (private key and corresponding public key), then supply the public key for authentication to datadrop.apptio.com. To maintain security, you must not share your private key with Apptio .

Datalink accepts ssh-rsa key types.

Create a public/private key pair

This example shows you how to use PuTTYgen to generate and configure a public/private SSH key pair for Datadrop authentication.

You can find more information about PuTTYgen here.

1. Download and install a key generator such as the PuTTY Key Generator (PuTTYgen): https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
2. Launch the key generator and set the following parameters:
   • Type of key to generate: RSA
   • Number of bits in the generated key: 4096
3. Select Generate .
4. In Key comment , enter a comment that describes the key's usage. We recommend you include the string DataDrop in this comment.
5. In Key passphrase , enter a passphrase. Use a strong password and keep it secret, including from Apptio
6. Copy the entire text from Public key for pasting into OpenSSH authorized keys file to a text file (scroll to select the entire key). The key begins with ssh-rsa and ends with your comment string.
7. Select Save public key .
8. Select Save private key . Do not share the private key or password with anyone, including Apptio.
9. Close the key generator.

Generate keys using iOS, Linux, or Unix

To generate keys in iOS, Linux, or Unix:

1. Initiate a terminal session.
2. Enter the following:

   cd ~/.ssh
   
   ssh-keygen -t rsa -b 4096 -f <keyname> -C "<comment>"

   You will be prompted for a passphrase and confirmation. The following two files are generated:
   • keyname : this is the private key
   • keyname.pub : this is the public key, which is the file you need to send to Apptio

For example:

~/.ssh$ ssh-keygen -t rsa -b 4096 -f testkey -C "testkey1"

Generating public/private rsa key pair.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in testkey.

Your public key has been saved in testkey.pub.

The key fingerprint is:

SHA256:RtgrTu+WeRn7IrU42YLRRYh3yW0ddI1+kPL4ewP9egw testkey1

The key's randomart image is:

+---[RSA 4096]----+

| . o o oo+o|

| .oo = + =..|

| ..oo . = . |

| . .. . o .|

| o.S. ... |

| o.+. o .E. |

| .o.B = .+.|

Configure file encryption

You can optionally use PGP encryption for files you transfer with Datadrop.

The PGP encryption feature adds an extra layer of security to the connector by allowing you to upload encrypted versions of your data. This prevents attackers from viewing the data while it is being uploaded into Apptio .

The expected file name format for using PGP encryption is fileToUpload.xyz.pgp , where xyz is the original extension of any file, such as .csv, .tsv, .txt, .dat, or .xml.

Generate a public key for PGP encryption

1. In Datalink , navigate to Settings > PGP .
2. Select Download Public Key .

   

3. Use a FIPS compliant PGP encryption tool or use FIPS approved cyphers (e.g. RSA, ECDSA, ECDH, DH etc.), along with the downloaded public key, to encrypt your files.
   Tip: A link to a PGP encryption tool is provided in the PGP settings page, however, you can use any FIPS compliant PGP encryption tool.

   If connector still fails during decryption, then renew the pgp key.

You now need to transfer the files to Apptio with the Datadrop connector.

Provision the Datadrop server

1. In Datalink , navigate to Settings > Datadrop .
2. Enter the authentication details:
   1. Provide a user name of your choice. The username should not contain spaces.
   2. Copy the public SSH key you generated into Public Key .
   3. Select Add Key .

      Learn more about configuring SSH keys

      

3. Select Provision to set up the Datadrop server.

   

   Datalink provisions the server and displays the server ID:

   

Transfer files to Datadrop using an SFTP client

This example shows you how to use FileZilla to transfer files to Datadrop.

1. Download and install SFTP tool FileZilla : https://filezilla-project.org/download.php?type=client FileZilla.
2. Configure the following SFTP client settings:
   • Protocol : SFTP
   • Host : server ID, from step 3 of Provision the Datadrop server
   • Port : 22 (ensure you have firewall access to this port before attempting to connect to Datadrop)
   • Logon type : Key file
   • Username : user name chosen in step 2 of Provision the Datadrop server
   • Key file location : path to private key file
3. Connect to the server and make the transfer.

View the Datadrop files

Once you have set up the Datadrop server and transferred the files using an SFTP client, you can view the uploaded files using the Datadrop viewer.

1. In Datalink , navigate to Datadrop Viewer .

   In the Datadrop viewer, you can do the following:

   • View folders and file lists present in the Datadrop account.
   • Use the breadcrumbs links at the top of the page to navigate folders.
   • Use the Search at the top of the page to search for specific data. \
   • Select the icon next to the file in Copy Path to copy the file path to the clipboard.

Configure a Datadrop connection to move the uploaded files to Apptio

1. In Datalink , navigate to Connections .
2. Select New Connection .
3. Enter your connection name.
4. Select Datadrop as the data type, then select Next .

   

5. Choose either an on-premises agent or a cloud-based default agent.
6. In the Apptio section, enter the destination settings. This is the location in Apptio where the data will be transferred to.
7. Choose the time period, then select Next .
8. In the Source section:
   1. Provide the location of the file or folder.
      Note:

      You can copy the Remote Directory path from the Datadrop viewer. Ensure you only enter the path to the file, not the file name.

   2. (Optional) Enter a Destination Table Name or choose to have the file name as the table name.

      

   3. Select Next .
9. In the Schedule section, set up the connector to run on a schedule. You can set up and run schedules on a daily, weekly, or monthly cadence. The Datadrop connector supports Gregorian and non-Gregorian calendar types.

   Learn more about connector chaining

10. Select Complete .

    Datalink adds the new connector to the list of connections on the homepage.