Google Drive

Connect Google Drive to your preprocessing pipeline, and use the Unstructured Ingest CLI or the Unstructured Ingest Python library to batch process all your documents and store structured outputs locally on your filesystem.

You will need:

The Google Drive prerequisites:

1. A Google Cloud service account and its related credentials.json key file or its contents in JSON format. Learn how.

2. Note the local path to the credentials.json key file. Or, copy the key file’s contents into a compatible string—including properly escaped quotes—as required. To make converting a JSON object into a compatible string easier, you can search the Internet by using a search phrase such as “tools for converting a JSON object into a string.” Before using any tool, check to make sure that the tool does not share the key file’s contents with anyone you do not trust.

3. Give the service account’s email address access to the Google Drive folder. Learn more.

4. Note the folder ID. This is a part of the URL for your Google Drive folder represented in the following URL as {folder_id}: https://drive.google.com/drive/folders/{folder-id}.

The Google Drive connector dependencies:

pip install "unstructured-ingest[google-drive]"

You might also need to install additional dependencies, depending on your needs. Learn more.

The following environment variables:

• GOOGLE_DRIVE_FOLDER_ID - The folder ID, represented by --drive-id (CLI) or drive_id (Python).
• GCP_SERVICE_ACCOUNT_KEY_FILEPATH - The path to the credentials.json key file, represented by --service-account-key-path (CLI) or service_account_key_path (Python), or
• GCP_SERVICE_ACCOUNT_KEY_STRING - The contents of the credentials.json key file as a string, represented by --service-account-key (CLI) or service_account_key (Python).

These environment variables:

• UNSTRUCTURED_API_KEY - Your Unstructured API key value.
• UNSTRUCTURED_API_URL - Your Unstructured API URL.

Now call the Unstructured Ingest CLI or the Unstructured Ingest Python library. The destination connector can be any of the ones supported. This example uses the local destination connector:

​

Output a key file’s contents as a string

If you need to convert the contents of a credentials.json file into a string, you could use a Python script such as the following. This script takes the local path to the key file as input and outputs the key file’s contents as a string.

# Filename: json_file_to_string.py

import json
import sys

def json_file_to_string(file_path):
    try:
        # Read the JSON file.
        with open(file_path, 'r') as file:
            # Load the JSON data.
            data = json.load(file)
        
        # Convert the JSON data back to a string, with no whitespace.
        json_string = json.dumps(data, separators=(',', ':'))
        
        return json_string
    
    except FileNotFoundError:
        print(f"Error: File '{file_path}' not found.")
        return None
    except json.JSONDecodeError:
        print(f"Error: '{file_path}' is not a valid JSON file.")
        return None
    except Exception as e:
        print(f"An unexpected error occurred: {e}")
        return None

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python json_file_to_string.py <path_to_json_file>")
        sys.exit(1)
    
    file_path = sys.argv[1]
    result = json_file_to_string(file_path)
    
    if result:
        print(result)