AWS Environments

Create an IAM policy in AWS that grants limited access to the provisioned EC2 instances. Use the sample policy provided below. You can make compatible changes to the policy as needed.

1. In the AWS Console, go to the IAM service.
2. In the navigation pane, click Policies, and then click Create policy.
3. In the policy editor, click the JSON tab and paste the contents of the following sample policy:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "0",
               "Effect": "Allow",
               "Action": "ssm:GetParameter",
               "Resource": "arn:aws:ssm:*:<ACCOUNT_ID>:parameter/StreamSets-Deployment-Token-*"
           },
           {
               "Sid": "2",
               "Effect": "Allow",
               "Action": "s3:*Object*",
               "Resource": [
                   "arn:aws:s3:::cloudformation-waitcondition-*"
               ]
            }
       ]
   }

4. Replace the <ACCOUNT_ID> parameter in the sample policy with your 12-digit AWS account ID.
5. When Amazon EC2 deployments managed by this environment are configured to use an external resource archive file stored in a private Amazon S3 bucket, add the following additional statement to the IAM policy:

            {
                "Sid": "1",
                "Effect": "Allow",
                "Action": "s3:GetObject*",
                "Resource": [
                     "arn:aws:s3:::*"
                ]
            }

   Or, instead of granting read access to all Amazon S3 buckets, restrict access to a specific bucket, as follows:

   "arn:aws:s3:::<bucket_name>/*"

   For more information about configuring external resources for deployments, see External Resources.

6. Make additional compatible changes to the policy as needed.
7. Click Next: Tags.
8. Click Review policy.
9. On the Review page, enter a policy name.

   For example, to designate that this policy is for the instance profiles used by IBM StreamSets, you might enter the name StreamSets-Instance-Profile-Policy.

10. Click Create policy.

In AWS, create an instance profile to associate with the provisioned EC2 instances. You can create a single default instance profile to use for all deployments belonging to the parent environment, or you can create a unique instance profile for each deployment.

1. In the AWS Console, go to the IAM service.
2. In the navigation pane, click Roles, and then click Create role.
3. Select AWS service as the type of trusted entity, and then choose EC2 as the use case.
4. Click Next: Permissions.
5. Search for and then select the policy that you created for IBM StreamSets instance profiles.

   For example, select StreamSets-Instance-Profile-Policy.

6. Attach any additional policies required by the StreamSets engines running on the EC2 instances.

   For example, if your pipelines process Amazon S3 data, attach an S3 policy to the role as well.

7. Click Next: Tags.
8. Click Next: Review.
9. On the Review page, enter a role name.

   For example, to designate that this role is for IBM StreamSets EC2 deployments, you might enter the name StreamSets-Deployment-EC2-Instance-Profile.

10. Click Create role.
11. Click the name of the role to view its details, and then copy the instance profile ARN.

    Be sure to copy the instance profile ARN, and not the role ARN. You will enter this instance profile ARN when you create the AWS environment or Amazon EC2 deployment in Control Hub.

12. To use a unique instance profile for each deployment, simply repeat these steps to create another instance profile and attach the same IAM policy to it.

For either authentication method, create an IAM policy in AWS that delegates limited access to Control Hub. Use the sample policy required for the feature version you plan to use. You can make compatible changes to the policy as needed.

1. Copy one of the following sample policies, based on the feature version that you plan to use for the AWS environment:
   • IAM Policy for AWS_2023_12_15
   • IAM Policy for AWS_2021_06_01
   Note: When creating a new environment, use the latest feature version.
2. In the AWS Console, go to the IAM service.
3. In the navigation pane, click Policies, and then click Create policy.
4. In the policy editor, click the JSON tab and paste the contents of the sample IAM policy that you copied in step 1.
5. Replace the following parameters in the sample policy:

   Parameter	Replacement Value
   <ACCOUNT_ID>	Your 12-digit AWS account ID.
   <INSTANCE_PROFILE_NAME>	Name of the instance profile that you created for EC2 instances provisioned by StreamSets. For example, enter StreamSets-Deployment-EC2-Instance-Profile.

   You can make compatible changes to the policy as needed.

6. If you created multiple instance profiles, add each instance profile to the sample policy as follows, replacing the <INSTANCE_PROFILE_NAME_n> parameters with the name of each profile that you created:

   ...
          {
               "Sid": "4",
               "Effect": "Allow",
               "Action": "iam:PassRole",
               "Resource": [
                   "arn:aws:iam::<ACCOUNT_ID>:role/<INSTANCE_PROFILE_NAME_1>",
                   "arn:aws:iam::<ACCOUNT_ID>:role/<INSTANCE_PROFILE_NAME_2>"
                ]
           },
   ...

   For example, if you created a default instance profile for the environment and then created a second instance profile that one deployment will use to override the default profile, enter both profile names here.

7. Click Next: Tags.
8. Click Review policy.
9. On the Review page, enter a policy name.

   For example, to designate that this policy is for IBM StreamSets environments, you might enter the name StreamSets-Environment-Policy.

10. Click Create policy.

For cross-account role authentication, create a cross-account role and attach the IAM policy that you created to this role. Control Hub assumes this role to perform tasks in your AWS account.

1. To create a cross-account role, first retrieve the unique external ID generated for your Control Hub organization.
   Important: Using the generated external ID prevents the confused deputy problem and ensures that Control Hub can assume this role only when acting on behalf of your organization.
   1. Log in to IBM StreamSets.
   2. In the Control Hub Navigation panel, click Set Up > Environments and then click the Create Environment icon: .
   3. Enter a name for the environment, and then select Amazon Web Services (AWS) for the Environment Type.
   4. Click Save & Next.
   5. Select Manual Cross-Account Role for the Credential Type.
   6. Copy the External ID value.

      You can leave this page open in the browser, cancel the environment creation, or click Save & Exit to save an incomplete environment that you finish configuring after completing the prerequisites.

2. In the AWS Console, go to the IAM service.
3. In the navigation pane, click Roles, and then click Create role.
4. Select Another AWS account as the type of trusted entity.

5. In the Account ID property, enter the IBM StreamSets AWS account ID based on the Control Hub instance that you use:
   • For the ap102.hub.streamsets.com, eu101.hub.streamsets.com, or na100.hub.streamsets.com instances, use 156041411084 as the ID.
   • For all other instances, use 632637830840 as the ID.
6. Select Require external ID and paste the external ID that you retrieved from Control Hub.
7. Click Next: Permissions.
8. Search for and then select the policy that you created for IBM StreamSets environments.

   For example, select StreamSets-Environment-Policy.

9. Click Next: Tags.
10. Click Next: Review.
11. On the Review page, enter a role name.

    For example, to designate that this role is for IBM StreamSets environments, you might enter the name StreamSets-Environment-Role.

12. Click Create role.
13. Click the name of the role to view its details, and then copy the role ARN.

    You will enter this ARN when you create the AWS environment in Control Hub.

For access keys authentication, create an IAM user with programmatic access that uses access keys, and attach the IAM policy that you created to this user. Control Hub uses these access keys to perform tasks in your AWS account.

1. In the AWS Console, go to the IAM service.
2. In the navigation pane, click Users, and then click Add user.
3. Enter a user name.

   For example, to designate that this user is for IBM StreamSets environments, you might enter the name StreamSets-Environment-User.

4. For Access type, select Programmatic access.
5. Click Next: Permissions.
6. Select Attach existing policies directly.
7. Search for and then select the policy that you created for IBM StreamSets environments.

   For example, select StreamSets-Environment-Policy.

8. Click Next: Tags.
9. Click Next: Review.
10. Click Create User.
11. Click Download .csv to download a CSV file that contains the access key ID and secret access key.

    You will enter these access keys when you create the AWS environment in Control Hub.