Installing File preview

Generates a preview on the user files (videos, images, and PDFs) when they are uploaded to S3.

Important: This feature is available for Aspera on Cloud Premium subscribers. To enable the use of the IBM Aspera File preview, you must build a container image with FFmpeg packaged inside. FFmpeg is a free open-source utility, that is not provided or managed by IBM and is subject to third party’s terms and conditions. FFmpeg contains codecs for encoding and decoding various video coding formats. Certain codecs contained within FFmpeg may be covered by patents and require a license to use. The Dockerfile provided by IBM will enable all codecs in FFMPEG by default; however, you must assess your licensing needs in connection with your use of FFmpeg and adjust which codecs to enable accordingly. IBM makes no warranties or conditions, express or implied, and IBM will have no liability to Client, regarding the FFmpeg utility when used with the Cloud Service.

For a list of file types that are available for preview, see File preview supported file types.

Prerequisites

Before you begin, fulfill these requirements:

Configuring File preview

Update the ./terraform-aws/variables.tf configuration file depending on your use case.

  • preview_duration: adjusts the duration of a preview for video files.
    • Default value is set to 15 seconds
    • You can update this value in the AWS Lambda page, inside the Configuration > Environment variables section .
  • preview_audio: Set this to true to enable the audio of preview videos.
  • There will be 2 instances of File Preview in AWS Lambda:
    • One with high resources used for the video preview processing
    • One with low resources for the image thumbnail processing.
    • Depending on the file extension, it will invoke either one of them to reduce costs of the running AWS Lambda Instances.
    • You do not need to change the environment variables for high_resource_lambda_name nor low_resource_lambda_name unless the names for the lambda functions are manually changed outside of Terraform.

Installing File preview

Important: You must wait a few minutes before uploading or transferring new files after the installation is complete.

To install File preview, do the following:

  1. Create an AWS ECR private repository (if one does not exist already).
    aws ecr create-repository \
    --repository-name {repo_name} \
    --image-scanning-configuration scanOnPush=true \
    --region {region}
    
  2. Provide Docker access to push docker images in AWS ECR private repositories.
    aws ecr get-login-password --region {region} | docker login --username AWS --password-stdin {account_id}.dkr.ecr.{region}.amazonaws.com
    You can obtain the {account_id} value running:
    aws sts get-caller-identity | grep Account

Build the docker images:

  1. Preview: The main core function to generate the previews for the uploaded files.
    Note: There are four different encoders to replace the encoder value with: vp9, av1, x264, or openh264.
    The following code represents an example using vp9:
    cd previews && docker build --build-arg encoder=vp9 -t {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:previews .
    Note: The value {repo_name} has to match the name of the AWS ECR private repository you created in step one.
  2. Previews-checker: Check existing items in the bucket on a particular path. previews-checker calls File Preview on each file that does not have a preview.
    cd ../previews-checker && docker build -t {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:checker .
  3. Previews-filter: Filters out S3 upload triggers by file extensions using a low resource lambda.
    cd ../previews-filter && docker build -t {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:filter .
  4. Push all docker images to your ECR repository.
    docker push {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:previews
    docker push {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:checker
    docker push {account_id}.dkr.ecr.{region}.amazonaws.com/{repo_name}:filter
    
  5. Update both aws_secret_key and aws_access_key in ./terraform-aws/previews/variables.tf. Make sure to use a security_token when using short-term credentials. Your account must be able to install the following resources:
    • Bucket
    • IAM roles and policies
    • Lambda functions
    Note: You can modify the variables in the variables.ft file as needed.
  6. Initialize a working directory containing Terraform configuration files.
    cd terraform-aws/previews
    terraform init
    
    The Terraform script will create a new bucket. You can also import an existing bucket, to do this make sure the bucket_names value in variables.tf is properly updated and run ./import.sh. Do the following to find the bucket_names for your existing AWS S3 bucket:
    • Sign in to your AWS account, and search for the keyword S3.
    • In the Buckets list, choose the names of the buckets that you want to select, and use those names as the list for the bucket_names variable.
  7. Create an execution plan that lets you preview the changes that Terraform plans to make to your infrastructure.
    terraform plan
  8. Validate the configuration files.
    transform validate
  9. Execute the actions proposed in the Terraform plan.
    terraform apply

Using Previews-checker

Previews-checker generates previews for existing files without previews (more specifically for files that were in the bucket before installing File Preview).
  1. Define the following parameters to invoke the AWS Lambda function:
    • path: Location in the bucket where you want to check for previews, and which can also be empty to invoked Lambda function from the bucket root level.
    • bucket: Bucket name. The associated role should have access to it.
  2. Invoke previews-checker lambda function with the previously defined parameters.
    $ aws lambda invoke \
        --function-name {checker_function_name} \
        --invocation-type Event \
        --payload '{ "path": {my_path}, "bucket": {bucket_name} }' \
        --region {region} \
        response.json
  3. Optional: You can delete previews-checker if you don't have files without previews.
    $ terraform destroy --target aws_cloudwatch_log_group.terraform_log_checker  --target aws_lambda_function.terraform_lambda_checker
  4. Optional: Delete the docker image from your ECR private repository.
    $ aws ecr batch-delete-image \
         --repository-name {repo_name} \
         --region {region} \
         --image-ids imageTag=checker