AWS S3 Storage Connections
Requirements
An AWS S3 storage connection must have access to an AWS account that includes the following components:
- An S3 bucket
- An Identity and Access Management (IAM) policy with permissions for Element
- An IAM role or an IAM user with the IAM policy attached
For stronger security, Element recommends that you use an IAM role.
AWS Resources
The following AWS documentation provides instructions for meeting AWS account requirements. Supplement the user guides with the recommendations and requirements in subsequent sections.
Task | AWS User Guide |
---|---|
Create an AWS S3 bucket | Create Bucket Overview |
Create an IAM policy | Creating IAM Policies |
Create an IAM user | Creating IAM Users |
Create an IAM role | Creating an IAM Role |
Creating an AWS S3 Bucket
Element recommends the following AWS S3 bucket settings:
- ACLs disabled
- Public access blocked
- Default encryption enabled
Consult your IT representative to confirm the appropriate settings for your lab and determine appropriate encryption. Default encryption protects the run, which includes genomic data. Bucket versioning and tags are not necessary for uploading runs. You cannot rename buckets. Selecting a region close to you increases the data transfer speed.
Creating an IAM Policy
Use one of the following JavaScript Object Notation (JSON) policy templates to create the IAM policy for an IAM role or an IAM user. The templates include all required permissions, but you must update the template for your bucket. For instructions, see Update the JSON Policy Template.
To learn more about the required permissions in the policy template, see JSON Policy Required Permissions.
JSON Policy Templates
- Template for IAM Role
- Template for IAM User
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ObjectWrite",
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::BUCKET_NAME/OPTIONAL_PREFIX/*"
},
{
"Sid": "S3ObjectListing",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::BUCKET_NAME",
"Condition": {
"StringLike": {
"s3:prefix": [
"OPTIONAL_PREFIX/*"
]
}
}
},
{
"Sid": "S3BucketLevelOperations",
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::BUCKET_NAME"
},
{
"Sid": "STSOperations",
"Effect": "Allow",
"Action": [
"sts:GetCallerIdentity"
],
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ObjectWrite",
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::BUCKET_NAME/OPTIONAL_PREFIX/*"
},
{
"Sid": "S3ObjectListing",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::BUCKET_NAME",
"Condition": {
"StringLike": {
"s3:prefix": [
"OPTIONAL_PREFIX/*"
]
}
}
},
{
"Sid": "S3BucketLevelOperations",
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::BUCKET_NAME"
},
{
"Sid": "STSOperations",
"Effect": "Allow",
"Action": [
"sts:GetCallerIdentity",
"sts:GetFederationToken"
],
"Resource": "*"
}
]
}
Update the JSON Policy Template
- Copy the template policy into a text editor.
- In all three
"Resource"
sections, replaceBUCKET_NAME
with your bucket name. - Update the
OPTIONAL_PREFIX
string.- If you use a prefix, replace
OPTIONAL_PREFIX
with your prefix name in the"Condition"
section and the first"Resource"
section. - If you do not use a prefix, delete the entire
"Condition"
section and/OPTIONAL_PREFIX/*
from the first"Resource"
section.
- If you use a prefix, replace
- Copy the updated template into the JSON section when creating an IAM policy.
JSON Policy Required Permissions
Permission | Purpose |
---|---|
S3:GetBucketLocation | Determines the region a bucket resides in. |
S3:ListBucket | Lists objects in the bucket as needed by the upload mechanism. |
S3:PutObject | Performs single and multipart uploads. |
STS:GetCallerIdentity | Verifies credentials by looking up the current user or role name. |
STS:GetFederationToken | Allows for the creation of limited temporary credentials for an IAM user. |
Using an IAM user requires the STS:GetFederationToken
permission as a security measure.
Creating an IAM Role
As a more secure alternative to the IAM user, the IAM role serves as the credentials and permissions for Elembio Cloud to transfer data to the AWS S3 bucket.
- Give the IAM role a name that clearly indicates the access of Element instruments, such as
Element-ServiceUser
. - When associating the IAM role with an IAM policy, make sure to select the policy you created for Elembio Cloud.
- Set up the role for a trusted entity AWS account and enter the account ID
588258415937
. - Select the option to require an external ID, and then enter the ID of your choice. The external ID can include alphanumeric characters and the special characters
@:,=-./_
. Spaces are not permitted. - After creating the IAM role, use the following AWS Command Line Interface (AWS CLI) command to set a maximum session duration of 12 hours (43,200 seconds):
aws iam update-role {rolearn} --max-session-duration 43200
Creating an IAM User
Creating the IAM user enables the creation of Access Key and Secret Key credentials. The credentials grant Elembio Cloud access to the AWS S3 bucket as the associated IAM policy permits.
- Give the IAM user a name that clearly indicates the access of Element instruments, such as
Element-ServiceUser
. - When associating the IAM role with an IAM policy, select the policy you created for Elembio Cloud.
- Leave the option for AWS management console access unselected.
- When you create the access key, select third-party service for the access key use case. Copy the access and secret keys to use for adding the storage connection.
Add an AWS S3 Storage Connection
- Review the requirements for an AWS S3 storage connection.
- On the Storage page, select Add Storage.
- Select S3.
- In the Storage Name field, enter a unique name for the storage connection.
- Enter the bucket name.
- If applicable, enter a prefix.
- In the Region list, select the region associated with the bucket.
If you do not see the region you need, contact Element Technical Support.
- Select the applicable credential type.
- For Role, enter the role Amazon Resource Name (ARN) and external ID.
- For Access Keys, enter the access key and secret key.
- Select Add Storage to add the storage connection.
Edit an AWS S3 Storage Connection
To make sure files successfully transfer from the instrument to one location, Elembio Cloud restricts editing to certain fields. You can only update the storage connection name or credentials. When updating credentials, you must update all fields for the credential.
- On a card for the storage connection, select More, and then select Edit.
- Update the storage connection information as needed.
- Select Save.