If you’re new to Unstructured, read this note first.

Before you can create a source connector, you must first sign in to your Unstructured account:

  • If you do not already have an Unstructured account, go to https://unstructured.io/contact and fill out the online form to indicate your interest.
  • If you already have an Unstructured account, sign in by using the URL of the sign in page that Unstructured provided to you when your Unstructured account was created. If you do not have this URL, contact Unstructured Sales at sales@unstructured.io.

After you sign in, the Unstructured user interface (UI) appears, which you use to get your Unstructured API key. To learn how, watch this 40-second how-to video.

After you create the source connector, add it along with a destination connector to a workflow. Then run the worklow as a job. To learn how, try out the hands-on Workflow Endpoint quickstart, go directly to the quickstart notebook, or watch the two 4-minute video tutorials for the Unstructured Python SDK.

You can also create source connectors with the Unstructured user interface (UI). Learn how.

If you need help, reach out to the community on Slack, or contact us directly.

You are now ready to start creating a source connector! Keep reading to learn how.

Ingest your files into Unstructured from Box.

The requirements are as follows.

  1. Access to the Developer Console from your Box enterprise account or Box developer account.

  2. A Box Custom App in your Box account, set up to use Server Authentication (with JWT). See Setup with JWT.

  3. The appropriate application scopes and advanced features set up for the Box Custom App, as follows:

    • In the Box Custom App, on the Configuration tab, under Application Scopes, check the box titled Write all files and folders stored in Box.
    • In the Box Custom App, on the Configuration tab, under Advanced Features, check the box titled Make API calls using the as-user header.
    • Click Save Changes before continuing.

  4. Authorization by a Box Admin in your Box account for the Box Custom App. See App Authorization.

  5. Access by your Box account’s source or target folder to your Box Custom App, as follows:

    • In the Box Custom App, on the General Settings tab, copy the Service Account ID (which takes the form AutomationUser_<your-app-service-id>_<a-random-string@boxdevedition.com).
    • Share your Box account’s target folder with the copied service account’s email address as a Co-owner or Editor.
    • Note the remote URL to the target folder, which takes the format box://<path/to/folder/in/account>.

  6. The private key configuration JSON file for the Box Custom App, or a string that contains this file’s contents.

    • To download this file, in the Box Custom App, on the Configuration tab, under Add and Manage Public Keys, click Generate a Public/Private Keypair. Store the downloaded private key configuration JSON file in a secure location.

    • To ensure maximum compatibility across Unstructured service offerings, you should give the private key configuration JSON file information to Unstructured as a single-line string that contains the contents of the downloaded private key configuration JSON file (and not the file itself). To print this single-line string, suitable for copying, you can run one of the following commands from your Terminal or Command Prompt. In this command, replace <path-to-downloaded-key-file> with the path to the private key configuration JSON file that you downloaded by following the preceding instructions.

      • For macOS or Linux:

        tr -d '\n' < <path-to-downloaded-key-file>

      • For Windows:

        (Get-Content -Path "<path-to-downloaded-key-file>" -Raw).Replace("`r`n", "").Replace("`n", "")

To create an Box source connector, see the following examples.

import os

from unstructured_client import UnstructuredClient
from unstructured_client.models.operations import CreateSourceRequest
from unstructured_client.models.shared import (
    CreateSourceConnector,
    SourceConnectorType,
    BoxSourceConnectorConfigInput
)

with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as client:
    response = client.sources.create_source(
        request=CreateSourceRequest(
            create_source_connector=CreateSourceConnector(
                name="<name>",
                type=SourceConnectorType.BOX,
                config=BoxSourceConnectorConfigInput(
                    remote_url="<remote-url>",
                    recursive=<True|False>,
                    box_app_config="<box-app-config>"
                )
            )
        )
    )

    print(response.source_connector_information)

Replace the preceding placeholders as follows:

  • <name> (required): A unique name for this connector.
  • <remote-url> (required): The URL to the target Box folder. This URL must take the format box://<path/to/folder/in/account>.
  • For recursive, set to true to recursively access files from subfolders within the taget Box folder. The default is false if not otherwise specified.
  • <box-app-config> (required): The contents of the private key configuration JSON file for the Box Custom App with access to the target Box folder. This file’s contents must be provided as a single string.