> ## Documentation Index
> Fetch the complete documentation index at: https://docs.unstructured.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Overview
To start using the Unstructured API's workflow operations right away, skip ahead to the [quickstart](#quickstart) or
[walkthrough](#walkthrough),
which uses the Unstructured Python SDK from remotely hosted Google Colab notebooks. These notebooks require no local setup.
The [Unstructured Pipelines](/pipelines/overview) features a no-code user interface for transforming your unstructured data into data that is ready
for retrieval-augmented generation (RAG).
The workflow operations in the [Unstructured API](/api-reference/overview) enable a full range of partitioning, chunking, embedding, and
enrichment options for your files and data. They are designed to batch-process files and data in remote locations; send processed results to
various storage, databases, and vector stores; and use the latest and highest-performing models. These workflow operations have built-in logic
to deliver the highest quality results at the lowest cost.
This page provides an overview of the Unstructured API's workflow operations. These operations enable Unstructured Pipelines automation usage
scenarios as well as for documentation, reporting, and recovery needs.
## Getting started
Choose one of the following options to get started with the Unstructured API's workflow operations:
* Follow the [quickstart](#quickstart) or [walkthrough](#walkthrough), which use the Unstructured Python SDK from remotely hosted Google Colab notebooks. These notebooks require no local setup.
* Start using the [Unstructred Python SDK](#unstructured-python-sdk). This option requires you to set up a local Python virtual environment.
* Start using a local [REST](#rest-endpoints) client, such as `curl` or Postman.
## Quickstart
The following quickstart shows how to use the [Unstructured Python SDK](/api-reference/workflow/overview#unstructured-python-sdk)
to have Unstructured process local files by using the Unstructured API's *on-demand jobs* functionality. This functionality
is part of the Unstructured API's collection of [workflow operations](https://docs.unstructured.io/api-reference/workflow/overview).
This quickstart is available in two options:
* [Use a remote notebook](https://colab.research.google.com/github/Unstructured-IO/notebooks/blob/main/notebooks/Unstructured_API_On_Demand_Jobs_Quickstart.ipynb) - This option uses a remotely hosted Google Colab notebook. There are no additional setup steps required.
* [Use your local machine](/api-reference/quickstart/overview) - This option requires you to install the Unstructured Python SDK on your local machine.
The on-demand jobs functionality is designed to work *only by processing local files*.
To process files (and data) in remote file and blob storage, databases, and vector stores, you must use other
workflow operations in the Unstructured API. To learn how, see for example the notebook
[Dropbox-To-Pinecone Connector API Quickstart for Unstructured](https://colab.research.google.com/github/Unstructured-IO/notebooks/blob/main/notebooks/Dropbox_To_Pinecone_Connector_Quickstart.ipynb).
## Walkthrough
[This walkthrough](https://colab.research.google.com/github/Unstructured-IO/notebooks/blob/main/notebooks/Unstructured_API_On_Demand_Jobs_Walkthrough.ipynb) builds upon the [quickstart](#quickstart).
Like the quickstart, this walkthrough shows how to use the [Unstructured Python SDK](/api-reference/workflow/overview#unstructured-python-sdk)
to have Unstructured process local files by using the Unstructured API's *on-demand jobs* functionality. This walkthrough goes further by
also showing in depth how to use Unstructured's [chunking](/concepts/chunking), [enriching](/concepts/enriching/overview), and [embedding](/concepts/embedding) features.
This walkthrough is available as a remotely hosted Google Colab notebook. This notebook requires no local setup.
The on-demand jobs functionality is designed to work *only by processing local files*.
To process files (and data) in remote file and blob storage, databases, and vector stores, you must use other
workflow operations in the Unstructured API. To learn how, see for example the notebook
[Dropbox-To-Pinecone Connector API Quickstart for Unstructured](https://colab.research.google.com/github/Unstructured-IO/notebooks/blob/main/notebooks/Dropbox_To_Pinecone_Connector_Quickstart.ipynb).
## Unstructured Python SDK
Watch the following 4-minute video to learn how to use the Python SDK to call the Unstructured API's workflow operations to
create [connectors](#connectors) in Unstructured Pipelines.
Watch the following 4-minute video to learn how to use the Python SDK to call the Unstructured API's workflow operations to
create [workflows](#workflows) and [jobs](#jobs) in Unstructured Pipelines.
Open a related [notebook](https://colab.research.google.com/github/Unstructured-IO/notebooks/blob/main/notebooks/Dropbox_To_Pinecone_Connector_Quickstart.ipynb) that covers many of
the concepts that are shown in the preceding videos.
The [Unstructured Python SDK](https://github.com/Unstructured-IO/unstructured-python-client), beginning with version 0.30.6,
allows you to call the Unstructured API's workflow operations through standard Python code.
To install the Unstructured Python SDK, run the following command from within your Python virtual environment:
```bash theme={null}
pip install "unstructured-client>=0.30.6"
```
If you already have the Unstructured Python SDK installed, upgrade to at least version 0.30.6 by running the following command instead:
```bash theme={null}
pip install --upgrade "unstructured-client>=0.30.6"
```
The Unstructured Python SDK code examples, shown later on this page and on related pages, use the following environment variable, which you can set as follows:
```bash theme={null}
export UNSTRUCTURED_API_KEY=""
```
This environment variable enables you to more easily run the following Unstructured Python SDK examples and help prevent
you from storing scripts that contain sensitive API keys in public source code repositories.
To get your Unstructured API key, do the following:
1. If you do not already have an Unstructured account, [sign up for free](https://unstructured.io/?modal=try-for-free).
After you sign up, you are automatically signed in to your new Unstructured **Let's Go** account, at [https://platform.unstructured.io](https://platform.unstructured.io).
To sign up for a **Business** account instead, [contact Unstructured Sales](https://unstructured.io/?modal=contact-sales), or [learn more](/api-reference/overview#pricing).
2. If you have an Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business SaaS** account and are not already signed in, sign in to your account at [https://platform.unstructured.io](https://platform.unstructured.io).
For other types of **Business** accounts, see your Unstructured account administrator for sign-in instructions,
or email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
3. Get your Unstructured API key:
a. After you sign in to your Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business** account, click **API Keys** on the sidebar.
For a **Business** account, before you click **API Keys**, make sure you have selected the organizational workspace you want to create an API key
for. Each API key works with one and only one organizational workspace. [Learn more](/pipelines/account/workspaces#create-an-api-key-for-a-workspace).
b. Click **Generate API Key**.
c. Follow the on-screen instructions to finish generating the key.
d. Click the **Copy** icon next to your new key to add the key to your system's clipboard. If you lose this key, simply return and click the **Copy** icon again.
Calls made by the Unstructured Python SDK's `unstructured_client` functions for creating, listing, updating,
and deleting connectors, workflows, and jobs in Unstructured Pipelines all use the Unstructured API's workflow operations URL.
This URL was provided to you when your Unstructured account was created.
If you do not have this URL, email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
The default URL for the Unstructured API's workflow operations is `https://platform.unstructuredapp.io/api/v1`.
However, you should always use the URL that was provided to you when your Unstructured account was created.
To specify an API URL in your code, set the `server_url` parameter in the `UnstructuredClient` constructor to the target API URL.
The Unstructured API's workflow operations enable you to work with [connectors](#connectors),
[workflows](#workflows), and [jobs](#jobs) in Unstructured Pipelines.
* A *source connector* ingests files or data into Unstructured from a source location.
* A *destination connector* sends the processed data from Unstructured to a destination location.
* A *workflow* defines how Unstructured will process the data.
* A *job* runs a workflow at a specific point in time.
For general information about these objects, see:
* [Connectors](/pipelines/connectors)
* [Workflows](/pipelines/workflows)
* [Jobs](/pipelines/jobs)
Skip ahead to start learning about how to use the Unstructured Python SDK to work with
[connectors](#connectors),
[workflows](#workflows), and [jobs](#jobs) programmatically.
## REST endpoints
The Unstructured API's workflow operations are callable from a set of Representational State Transfer (REST) endpoints, which you can call through standard REST-enabled
utilities, tools, programming languages, packages, and libraries. The examples, shown later on this page and on related pages, describe how to call the Unstructured API's workflow operations with
`curl` and Postman. You can adapt this information as needed for your preferred programming languages and libraries, for example by using the
`requests` library with Python.
You can also use the [Unstructured API's workflow operations - Swagger UI](https://platform.unstructuredapp.io/docs) to call the REST endpoints
that are available through the default Unstructured API's workflow operations URL: `https://platform.unstructuredapp.io`. To use the Swagger UI, you must provide your Unstructured API key with each call. To
get this API key, see the [quickstart](#quickstart), earlier on this page.
Note that you should always use the URL that was provided to you when your Unstructured account was created.
If you do not have this URL, email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
### curl and Postman
The following `curl` examples use the following environment variables, which you can set as follows:
```bash theme={null}
export UNSTRUCTURED_API_URL="https://platform.unstructuredapp.io/api/v1"
export UNSTRUCTURED_API_KEY=""
```
For the API URL, this URL was provided to you when your Unstructured account was created.
If you do not have this URL, email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
The default URL for the Unstructured API's workflow operations is `https://platform.unstructuredapp.io/api/v1`.
However, you should always use the URL that was provided to you when your Unstructured account was created.
These environment variables enable you to more easily run the following `curl` examples and help prevent
you from storing scripts that contain sensitive URLs and API keys in public source code repositories.
To get your Unstructured API key, do the following:
1. If you do not already have an Unstructured account, [sign up for free](https://unstructured.io/?modal=try-for-free).
After you sign up, you are automatically signed in to your new Unstructured **Let's Go** account, at [https://platform.unstructured.io](https://platform.unstructured.io).
To sign up for a **Business** account instead, [contact Unstructured Sales](https://unstructured.io/?modal=contact-sales), or [learn more](/api-reference/overview#pricing).
2. If you have an Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business SaaS** account and are not already signed in, sign in to your account at [https://platform.unstructured.io](https://platform.unstructured.io).
For other types of **Business** accounts, see your Unstructured account administrator for sign-in instructions,
or email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
3. Get your Unstructured API key:
a. After you sign in to your Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business** account, click **API Keys** on the sidebar.
For a **Business** account, before you click **API Keys**, make sure you have selected the organizational workspace you want to create an API key
for. Each API key works with one and only one organizational workspace. [Learn more](/pipelines/account/workspaces#create-an-api-key-for-a-workspace).
b. Click **Generate API Key**.
c. Follow the on-screen instructions to finish generating the key.
d. Click the **Copy** icon next to your new key to add the key to your system's clipboard. If you lose this key, simply return and click the **Copy** icon again.
The following Postman examples use variables, which you can set as follows:
1. In Postman, on your workspace's sidebar, click **Environments**.
2. Click **Globals**.
3. Create two global variables with the following settings:
* **Variable**: `UNSTRUCTURED_API_URL`
* **Type**: `default`
* **Initial value**: The Unstructured API's workflow operations URL that was provided to you when your Unstructured account was created.
* **Current value**: The Unstructured API's workflow operations URL that was provided to you when your Unstructured account was created.
* **Variable**: `UNSTRUCTURED_API_KEY`
* **Type**: `secret`
* **Initial value**: ``
* **Current value**: ``
4. Click **Save**.
These variables enable you to more easily run the following examples in Postman and help prevent you from storing
Postman collections that contain sensitive URLs and API keys in public source code repositories.
Unstructured offers a [Postman collection](https://learning.postman.com/docs/collections/collections-overview/) that you can import into Postman to make Unstructured API's workflow operations requests through a graphical user interface.
1. [Install Postman](https://learning.postman.com/docs/getting-started/installation/installation-and-updates/).
2. [Sign in to Postman](https://learning.postman.com/docs/getting-started/installation/postman-account/#signing-in-to-postman).
3. In your workspace, click **Import**.
4. In the **Paste cURL, Raw text or URL** box, enter the following URL, and then press `Enter`:
For all workflow operations:
```text theme={null}
https://raw.githubusercontent.com/Unstructured-IO/docs-samples/main/postman/Unstructured-REST-API-Workflow-Endpoint.postman_collection.json
```
For [on-demand job](#run-an-on-demand-job) related operations only:
```text theme={null}
https://raw.githubusercontent.com/Unstructured-IO/docs-samples/main/postman/Unstructured-REST-API-On-Demand-Jobs.postman_collection.json
```
Each on-demand job is limited to 10 files, and each file is limited to 50 MB in size.
If you need to launch a series of on-demand jobs in rapid succession, you must wait at least one second between launch
requests. Otherwise, you will receive a rate limit error.
A maximum of 5 on-demand jobs can be running in your Unstructured account. If you launch a new on-demand job
but 5 existing on-demand jobs are still running, the new on-demand job will remain in a scheduled state until one of the 5
existing on-demand jobs is done running.
5. On the sidebar, click **Collections**.
6. Expand **Unstructured REST API - Workflow Operations**.
7. Select the request that you want to use.
8. As applicable, modify the URL as needed to specify any required resource IDs for the request.
9. On the **Headers** tab, next to `unstructured-api-key`, enter your Unstructured API key in the **Value** column.
As applicable, add, remove, or modify any other required headers for the request.
10. As applicable, on the **Params** tab, add, remove, or modify any required parameters for the request.
11. As applicable, on the **Body** tab, add, remove, or modify the required request body for the request.
12. Click **Send**.
13. To save the response, in the response area, click the ellipses, and then click **Save response to file**.
To get your Unstructured API key, do the following:
1. If you do not already have an Unstructured account, [sign up for free](https://unstructured.io/?modal=try-for-free).
After you sign up, you are automatically signed in to your new Unstructured **Let's Go** account, at [https://platform.unstructured.io](https://platform.unstructured.io).
To sign up for a **Business** account instead, [contact Unstructured Sales](https://unstructured.io/?modal=contact-sales), or [learn more](/api-reference/overview#pricing).
2. If you have an Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business SaaS** account and are not already signed in, sign in to your account at [https://platform.unstructured.io](https://platform.unstructured.io).
For other types of **Business** accounts, see your Unstructured account administrator for sign-in instructions,
or email Unstructured Support at [support@unstructured.io](mailto:support@unstructured.io).
3. Get your Unstructured API key:
a. After you sign in to your Unstructured **Let's Go**, **Pay-As-You-Go**, or **Business** account, click **API Keys** on the sidebar.
For a **Business** account, before you click **API Keys**, make sure you have selected the organizational workspace you want to create an API key
for. Each API key works with one and only one organizational workspace. [Learn more](/pipelines/account/workspaces#create-an-api-key-for-a-workspace).
b. Click **Generate API Key**.
c. Follow the on-screen instructions to finish generating the key.
d. Click the **Copy** icon next to your new key to add the key to your system's clipboard. If you lose this key, simply return and click the **Copy** icon again.
The Unstructured API's workflow operations enable you to work with [connectors](#connectors),
[workflows](#workflows), and [jobs](#jobs) in Unstructured Pipelines.
* A *source connector* ingests files or data into Unstructured from a source location.
* A *destination connector* sends the processed data from Unstructured to a destination location.
* A *workflow* defines how Unstructured will process the data.
* A *job* runs a workflow at a specific point in time.
For general information about these objects, see:
* [Connectors](/pipelines/connectors)
* [Workflows](/pipelines/workflows)
* [Jobs](/pipelines/jobs)
Skip ahead to start learning about how to use the REST endpoints to work with
[connectors](#connectors),
[workflows](#workflows), and [jobs](#jobs) programmatically.
## Restrictions
The following Unstructured SDKs, tools, and libraries do *not* work with the Unstructured API's workflow operations:
* The [Unstructured JavaScript/TypeScript SDK](/api-reference/legacy-api/partition/sdk-jsts)
* [Local single-file POST requests](/api-reference/legacy-api/partition/sdk-jsts) to the legacy Unstructured Partition Endpoint
* The [Unstructured open source Python library](/open-source/introduction/overview)
* The [Unstructured Ingest CLI](/open-source/ingestion/ingest-cli)
* The [Unstructured Ingest Python library](/open-source/ingestion/python-ingest)
The following Unstructured API URL is also *not* supported: `https://api.unstructuredapp.io/general/v0/general` (the default legacy Unstructured Partition Endpoint URL).
4. In the **Paste cURL, Raw text or URL** box, enter the following URL, and then press `Enter`:
For all workflow operations:
```text theme={null}
https://raw.githubusercontent.com/Unstructured-IO/docs-samples/main/postman/Unstructured-REST-API-Workflow-Endpoint.postman_collection.json
```
For [on-demand job](#run-an-on-demand-job) related operations only:
```text theme={null}
https://raw.githubusercontent.com/Unstructured-IO/docs-samples/main/postman/Unstructured-REST-API-On-Demand-Jobs.postman_collection.json
```