s3:ListBucket
and s3:GetObject
for that bucket. Learn how.
s3:PutObject
for that bucket. Learn how.
--key
, --secret
, and --token
(CLI) or key
, secret
, and token
(Python) in your command or code for the
corresponding S3 source or destination connector.protocol://bucket/
(for example, s3://my-bucket/
).
If the target files are in a folder, the path to the target folder in the S3 bucket, formatted as protocol://bucket/path/to/folder/
(for example, s3://my-bucket/my-folder/
).
root
to that
specific username.
In this policy, replace the following:
<my-account-id>
with your AWS account ID.<my-bucket-name>
in two places with the name of your bucket.create-s3-bucket.yaml
. To change
the following bucket policy to restrict it to a specific user in the AWS account, change root
to that
specific username.
create-s3-bucket.sh
.
To change the following bucket policy to restrict it to a specific user in the AWS account, change root
to that
specific username.
In this script, replace the following:
<my-account-id>
with your AWS account ID.<my-unique-bucket-name>
with the name of your bucket.<us-east-1>
with your AWS Region.x-amz-meta-
and is
followed by a unique name.
For more information about how to add or replace user-defined metadata for a file in S3, see the following:
Unstructured outputs any user-defined metadata that it finds for a file into the metadata.data_source.record_locator.metadata
field of
the document elements’ output for the corresponding file. For example, if Unstructured processes a file with the user-defined metadata
x-amz-meta-mymetadata
name set to the value myvalue
, Unstructured outputs the following into the metadata.data_source.record_locator.metadata
field of
the document elements’ output for the corresponding file:
<name>
(required) - A unique name for this connector.
<key>
- The AWS access key ID for the authenticated AWS IAM user (required).<secret>
- The AWS secret access key corresponding to the preceding AWS access key ID (required).<token>
- The AWS STS session token for temporary access (required).<endpoint-url>
- A custom URL, if connecting to a non-AWS S3 bucket.
<remote-url>
(required) - The S3 URI to the bucket or folder, formatted as s3://my-bucket/
(if the files are in the bucket’s root) or s3://my-bucket/my-folder/
.
recursive
(source connector only), set to true
to access subfolders within the bucket. The default is false
if not otherwise specified.