Data ingestion examples

Use the following example steps for data ingestion in the following scenarios.

You must consider data ingestion and access:
  • You can have data ingestion through object interface and access through file interface.
  • You can have data ingestion through file interface and access through object interface.
  • You can have data ingestion and access through object and file interfaces concurrently.
  1. This step is for the standard REST client. Get proper authentication token from the Authentication URL by using proper credentials to authorize on further requests.
  2. This step is for the standard REST client. Use the token that is obtained in the previous step to do the PUT, POST, DELETE, COPY (object only), or HEAD operations for objects that are under container that is created with unified file and object access storage policy.
  3. This step is for the standard file client step. Mount SMB or NFS exports on respective NFS or SMB clients with regular mount commands or interface available with file clients:
    mount -t cifs -o username=STORAGE5TEST\\fileuser1,password=Passw0rd5,vers=3.0 //192.0.2.4/unified_access /mnt/unified_access

For the following data ingestion example steps that are done by using the curl command, this setup is assumed:

  • The user is "fileuser".
  • The password is "Password6".
  • The account name is "admin".
  • The host is specscaleswift.example.com.
  1. Run the following command to obtain the authentication token:
    curl -s -i -H "Content-Type: application/json" 
-d '{ "auth": {"identity": {"methods": ["password"],"password": {"user": {"name": "fileuser","domain": 
{ "name": "Default" },"password": "Passw0rd6"}}},"scope": {"project": {"name": "admin","domain": { "name": "Default" }}}}}' 
http://specscaleswift.example.com:35357/v3/auth/tokens

    The auth token that is obtained in this step must be stored in the $AUTH_TOKEN variable.

  2. Run the following command to obtain the project list:
    curl -s -H "X-Auth-Token:  $AUTH_TOKEN" http://specscaleswift.example.com:35357/v3/projects

    The project ID obtained in this step must be stored in the $AUTH_ID variable.

  3. Run the following command to do a PUT operation:
    curl -i -s -X PUT --data @/tmp/file.txt -H "X-Auth-Token:
$AUTH_TOKEN" "http://specscaleswift.example.com:8080/v1/AUTH_$AUTH_ID/RootLevelContainer/TestObj.txt"

    This command uploads the /tmp/file.txt file.

  4. Run ht following command to set up the metadata age of the uploaded object:
    curl -i -s -X POST -H "X-Auth-Token: $AUTH_TOKEN" -H 
X-Container-Meta-Age:21 http://specscaleswift.example.com:8080/v1/AUTH_$AUTH_ID/RootLevelContainer/TestObj.txt
  5. Run the following command to read the metadata:
    curl -i -s --head -H  "X-Auth-Token: $AUTH_TOKEN"
http://specscaleswift.example.com:8080/v1/AUTH_$AUTH_ID/RootLevelContainer/TestObj.txt