Google Cloud Storage

​

Requirements

• A Google Cloud service account. Create a service account.
• A service account key for the service account. See Create a service account key in Create and delete service account keys. To ensure maximum compatibility across Unstructured service offerings, you should give the service account key information to Unstructured as a single-line string that contains the contents of the downloaded service account key file (and not the service account key file itself). To print this single-line string without line breaks, suitable for copying, you can run one of the following commands from your Terminal or Command Prompt. In this command, replace <path-to-downloaded-key-file> with the path to the service account key file that you downloaded by following the preceding instructions.
  • For macOS or Linux:
    tr -d '\n' < <path-to-downloaded-key-file>
    
  • For Windows:
    (Get-Content -Path "<path-to-downloaded-key-file>" -Raw).Replace("`r`n", "").Replace("`n", "")
    
• The URI for a Google Cloud Storage bucket. This URI consists of the target bucket name, plus any target folder within the bucket, expressed as gs://<bucket-name>[/folder-name]. Create a bucket. This bucket must have, at minimum, one of the following roles applied to the target Google Cloud service account:
  • Storage Object Viewer for bucket read access.
  • Storage Object User for bucket write access.
  • The Storage Object Admin role provides read and write access, plus access to additional bucket operations.
  To apply one of these roles to a service account for a bucket, see Add a principal to a bucket-level policy in Set and manage IAM policies on buckets.

​

Examples

import os

from unstructured_client import UnstructuredClient
from unstructured_client.models.operations import CreateDestinationRequest
from unstructured_client.models.shared import CreateDestinationConnector

with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as client:
    response = client.destinations.create_destination(
        request=CreateDestinationRequest(
            create_destination_connector=CreateDestinationConnector(
                name="<name>",
                type="gcs",
                config={
                    "remote_url": "<remote-url>",
                    "service_account_key": "<service-account-key>"
                }
            )
        )
    )

    print(response.destination_connector_information)

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/destinations" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    "type": "gcs",
    "name": "<name>",
    "config": {
        "remote_url": "<remote-url>",
        "service_account_key": "<service-account-key>"
    }
}'

​

Configuration settings