> ## Documentation Index
> Fetch the complete documentation index at: https://docs.unstructured.io/llms.txt
> Use this file to discover all available pages before exploring further.

# VAST

> Ingest your files into Unstructured from [VAST](https://www.vastdata.com/).

<Note>
  First time creating a connector? [Read this first](/pipelines/connector-first-time-reqs).
</Note>

## Requirements

You will need:

* A [VAST](https://www.vastdata.com/) user account with permissions to the bucket to which you want to connect.

* A bucket that contains the objects to which you want to connect. A bucket is a storage resource that uses Amazon's Simple Storage Service (S3) as an access protocol.

  To create a bucket, see [Creating S3 Buckets](https://kb.vastdata.com/documentation/docs/creating-s3-buckets-via-vms-1) in the *VAST Cluster Administrator's Guide*.

* The path to the files in the bucket.

  If the target files are in the root of the bucket, the path should be formatted as `vast://bucket/` (for example, `vast://my-bucket/`). If the target files are in a folder, format the path to the target folder as `vast://bucket/path/to/folder/` (for example, `vast://my-bucket/my-folder/`).

* The URL for the VAST region in which the bucket is located (for example, `https://vast-cluster.my-company.com`).

* An enabled access and secret key pair that grants the authenticated user permission to the VAST bucket.

  To grant a user access to a bucket, and attach identity policies that define their permissions on that bucket, see [Managing S3 User Access from the VAST Web UI](https://kb.vastdata.com/documentation/docs/managing-s3-user-access-from-the-vast-web-ui-1) or [Managing S3 Access from the VAST CLI](https://kb.vastdata.com/documentation/docs/managing-s3-access-from-the-vast-cli) in the *VAST Cluster Administrator's Guide*.

  <Note>
    VAST recommends using identity policies and bucket policies to control S3 bucket access.

    * *Identity policies* are attached to users and groups, and defines the actions the users or groups can perform against buckets.
    * *Bucket policies* are attached to a view, and define actions that particular users or groups can perform against the view and files or directories under the view.

    For more information, see [Overview of S3 Access Management](https://kb.vastdata.com/documentation/docs/overview-of-s3-access-management) in the *VAST Cluster Administrator's Guide*.

    Identity and bucket policies support a subset of elements listed in [Amazon's IAM JSON Policy Reference](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies.html). For information about the required JSON format, supported elements and identity and bucket policy examples, see [Identity and Bucket Policy Reference](https://kb.vastdata.com/documentation/docs/identity-and-bucket-policy-reference-1) in the *VAST Cluster Administrator's Guide*.
  </Note>

  For source connectors, the authenticated user must have bucket permissions for the following S3 actions:

  * `s3:ListBucket`
  * `s3:GetObject`

  For destination connectors, the authenticated user must have bucket permissions for the following S3 actions:

  * `s3:PutObject`

VAST buckets are private by default. [Anonymous access](https://kb.vastdata.com/documentation/docs/overview-of-vast-cluster-s3-implementation-1#support-for-authenticated-or-anonymous-requests) can be explicitly enabled using bucket policies, but is not recommended.

## Create the connector

To create the source connector:

1. On the sidebar, click **Connectors**.
2. Click **Sources**.
3. Cick **New** or **Create Connector**.
4. Give the connector some unique **Name**.
5. In the **Provider** area, click **VAST**.
6. Click **Continue**.
7. Follow the on-screen instructions to fill in the fields as described later on this page.
8. Click **Save and Test**.

Fill in the following fields:

* **Name** (*required*): A unique name for this connector.
* **Remote URL** (*required*): The path to the files in the bucket.

  If the target files are in the root of the bucket, the path should be formatted as `vast://bucket/`. For example: `vast://my-bucket/`

  If the target files are in a folder, format the path to the target folder as `vast://bucket/path/to/folder/`. For example: `vast://my-bucket/my-folder/`

  <Note>
    For destination connectors, the relative directory structure of files is preserved when files are uploaded. Also, files are overwritten if they already exist at the destination.
  </Note>
* **Recursive** (source connector only): Check this box to access sub-folders within the bucket.
* **Max Objects** (source connector only): The maximum number of objects to index. This is useful for large buckets.
* **Endpoint URL** (*required*): The VAST storage endpoint URL. For example, `https://vast-cluster.my-company.com`.
* **Key** (*required*): The access key ID for the authenticated user.
* **Secret** (*required*): The secret access key corresponding to the preceding access key ID.