Unstructured

Sign up for the Platform beta.

1. Use the sign-in URL in the welcome email that Unstructured sends you.
2. Click Google or GitHub to sign in with your Google or GitHub account. Or, enter your email address and then click Sign In.
3. If you entered your email address, check your email inbox for a message from Unstructured. In that email, click the Sign In link.
4. The first time you sign in, read the terms and conditions, and then click Accept.

1. In the sidebar, click Sources.
2. Click New Source.
3. In the Type dropdown list, select the source location type that matches yours.
4. Fill in the rest of the fields with the appropriate settings. Learn more.
5. Click Save and Test.
6. Click Close.

1. In the sidebar, click Destinations.
2. Click New Destination.
3. In the Type dropdown list, select the destination location type that matches yours.
4. Fill in the rest of the fields with the appropriate settings. Learn more.
5. Click Save and Test.
6. Click Close.

1. In the sidebar, click Worfklows.

2. Click New Workflow.

3. Enter a Name for the new workflow.

4. Expand Connect.

5. In the Sources dropdown list, select your source location from Step 3.

6. In the Destination dropdown list, select your destination location from Step 4.

   You can select multiple source and destination locations. Files will be ingested from all of the selected source locations, and the processed data will be delivered to all of the selected destination locations.

7. Fill in the rest of the fields with any non-default settings. Learn more.

8. Click Save.

1. In the sidebar, click Jobs.
2. Click Run Job.
3. In the Select a Workflow dropdown list, select your workflow from Step 5.
4. Click Run.

1. In the list of Jobs, click the ID link for your new job.
2. When the Status shows Completed, go to the next Step.

Go to your destination location to view the processed data.

Sign up for a 14-day trial of the Unstructured Serverless API at https://app.unstructured.io.

1. Enter your email address and then click Sign In to receive a magic link to sign in to your personalized dashboard, or authenticate yourself with your Google or GitHub account.
2. Once you authenticate with the magic link or with your Google or GitHub account, your dashboard appears.

1. On the sidebar, click API Keys, if it is not already selected.
2. To get your API key, click the copy icon in the Actions column for your API key, and then click Key Only. Store your copied API key in a secure location. Do not share it with others.
3. To get your API URL, click the copy icon next to the URL next to API URL. Store your copied API URL in a secure location. Do not share it with others.

1. Set an environment variable named UNSTRUCTURED_API_KEY to the value of your Unstructured API key.
2. Set another environment variable named UNSTRUCTURED_API_URL to the value of your Unstructured API URL.

Run the following command:

pip install unstructured-ingest

You might also need to install additional dependencies, depending on your needs. Learn more.

Run the following code, replacing:

• <path/to/input> with the source (input) path to the directory on your local machine that contains the compatible files for Unstructured to process on its hosted compute resources.
• <path/to/output> with the destination (output) path to the directory on your local machine that will contain the processed data that Unstructured returns from its hosted compute resources.

import os

from unstructured_ingest.v2.pipeline.pipeline import Pipeline
from unstructured_ingest.v2.interfaces import ProcessorConfig
from unstructured_ingest.v2.processes.connectors.local import (
    LocalIndexerConfig,
    LocalDownloaderConfig,
    LocalConnectionConfig,
    LocalUploaderConfig
)
from unstructured_ingest.v2.processes.partitioner import PartitionerConfig

if __name__ == "__main__":
    Pipeline.from_configs(
        context=ProcessorConfig(),
        indexer_config=LocalIndexerConfig(input_path=os.getenv("LOCAL_FILE_INPUT_DIR")),
        downloader_config=LocalDownloaderConfig(),
        source_connection_config=LocalConnectionConfig(),
        partitioner_config=PartitionerConfig(
            partition_by_api=True,
            api_key=os.getenv("UNSTRUCTURED_API_KEY"),
            partition_endpoint=os.getenv("UNSTRUCTURED_API_URL"),
            strategy="hi_res",
            additional_partition_args={
                "split_pdf_page": True,
                "split_pdf_allow_failed": True,
                "split_pdf_concurrency_level": 15
            }
        ),
        uploader_config=LocalUploaderConfig(output_dir=os.getenv("LOCAL_FILE_OUTPUT_DIR"))
    ).run()

Go to your destination location to view the processed data.