Inline file validation with URI

Inline file validation with URI can be customized to filter which files are validated.

Validation requests and returned responses

During the inline validation process, ascp automatically generates a JSON-based request. The call is made with the URL defined in aspera.conf. For example:

POST URL/validation/files HTTP/1.1
Content-type: application/json
Draft comment: jowen
Commenting this out because it's "under the hood" content.

Sample Request Body

The following is an example request body sent by the HTTP post.

{
"startstop" : "start",
"xfer_id" : "AAAA-BBBB",             
"session_id" : "1111-2222",
"host" : "10.0.258.12",
"client_ip" : "10.0.125.04",
"user" : "admin",
"userid" : 24,
"direction" : "send",
"target_rate_kbps" : 0,
"min_rate_kbps" : 0,
"rate_policy" : "fair",
"cipher" : "aes-128",
"cookie" : "xyz",
"manifest_file" : "/data/manifests/aspera-transfer-1234.txt",
"file" : "/data/home/luke/test.mpg",
"size" : 1000000,
"start_byte" : 0,
"bytes_written" : 0,
"tags" : { tags },
"file_name_encoding" : "utf8",
"file_csum" : "a1000abf882"
"file_csum_type" : "sha2-256" 
 }
Field Description Values
"startstop" Sets when the validation should be called: at the start of the transfer, at the end of the transfer, or repeatedly, as the transfer progresses. start, stop, or running
"xfer_id" Value used to identify a transfer session String
"session_id" Value used to identify a validation session String
"host" Server hostname or IP address Hostname or IP address
"client_ip" Client IP address IP address
"user" SSH account login name String
"user_id" Value used to identify the user String
"direction" Direction of transfer (send or receive) send or recv
"target_rate_kbps" Target rate (in kbps) for file transfer with ascp Integer
"min_rate_kbps" Minimum rate (in kbps) for file transfer with ascp Integer
"rate_policy" Defines the ascp rate policy. This value is taken from the default configuration in the GUI or aspera.conf, if not defined here. fixed, fair, high, or low
"cipher" The encryption cipher for file data. String; aes-128, any, or none
"cookie" The cookie sent to the client system String
"manifest_file" Path to manifest file, which contains a list of transferred files. The command for this in ascp is --file-manifest-path=file_path Filepath
"file" Path to file being validated Filepath
"size" Allowable file size in bytes Integer (up to 64-bit)
"start_byte" Integer
"bytes_written" Integer
"tags" The JSON request passes the supplied tag values to ascp, which in turn passes the tags to the validator.
"file_name_encoding" String
"file_csum" File checksum String
"file_csum_type" File checksum type md5, sha1, sha-256, sha-384, sha-512, or any

The system then generates a JSON accepted or error response (OK or Bad Request). If a file validation fails, it terminates the session with an error message from the URI.

  • Sample JSON accepted response: The "file_encryption" field is only returned if server-side EAR is present.
    HTTP 200 OK
    {
        "id" : "1111-2222-333",
        "file_encryption" : {               
            "passphrase" : "supersecret"
        }
        "aspera_response_object_name" : {
            "startstop" : "start"
            "xfer_id" : "AAAA-BBBB",             
            .  .  .
            "file_csum" : "a1000abf882",
            "file_csum_type" : "sha2-256" 
        }
    }
  • Sample JSON error response:
    HTTP 400 Bad Request
    {  
      "error" : {
        "code" : "1022",
        "message" : "The file fails validation"
      }
    }

Custom code for including and excluding files

Administrators can include or exclude files by enabling allowlisting, or another method of their own design. You can do this by creating custom code in the programming language of your choice, using a web server that runs a REST service. (HSTS users have the option to use the web server associated with that installation).

The following is an example of custom code that creates a file blocklist, using a Java™ servlet deployed on an Apache web server. Note that this code uses the servlet name SimpleValidator, which was defined in web.xml above.
package aspera.validation;

import com.google.gson.Gson;
import com.google.gson.JsonObject;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.BufferedReader;
import java.io.IOException;

@WebServlet(name = "SimpleValidator")
public class SimpleValidator extends HttpServlet {
    protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        StringBuilder fileRequestJSON = new StringBuilder();
        BufferedReader reader = request.getReader();
        String line = "";
        Gson gson = new Gson();

        System.out.println("Got Validation request...");
        while (line != null) {
            line = reader.readLine();
            if (!(line == null)) {
                fileRequestJSON.append(line).append("\n");
            }
        }

        ValidationInput validationInput = gson.fromJson(fileRequestJSON.toString(), ValidationInput.class);

        System.out.println("FileData JSON: " + fileRequestJSON.toString());

        if (validationInput.file != null && validationInput.file.endsWith(".sh")
           || validationInput.file.endsWith(".exe")) {

            JsonObject innerObject = new JsonObject();
            innerObject.addProperty("message", "Cannot transfer executable file!!");
            innerObject.addProperty("code", 1);

            JsonObject jsonObject = new JsonObject();
            jsonObject.add("error", innerObject);

            response.getOutputStream().println(jsonObject.toString());

            response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
        }
        else {

            JsonObject jsonObject = new JsonObject();
            jsonObject.addProperty("success", true);
            jsonObject.addProperty("data", "File is ok to transfer");
            jsonObject.addProperty("code", 1);
            response.getOutputStream().println(jsonObject.toString());

            response.setStatus(HttpServletResponse.SC_OK);
        }
        return;
    }
}