Unstructured API on Azure
Follow these steps to deploy the Unstructured API service into your Azure account.
Log in to the Azure Portal
Go to https://portal.azure.com.
Access the Azure Marketplace
Go to the Unstructured Data Preprocessing - Customer Hosted API offering in the Azure Marketplace.
Start the deployment process
- Click Get It Now and fill out the form.
- Read the terms and click Continue.
- Click Create.
Configure the deployment options
-
On the Create a virtual machine page, click the Basics tab.
-
In the Project details section, select an existing Subscription, and select an Resource group from the dropdown menus, or create a new resource group by clicking Create new.
-
In the Instance details section, enter a name in the Virtual machine name field. Note this name, as you will need it later steps.
-
Select a Region from the dropdown menu.
-
For Image, select Unstructured Customer Hosted API Hourly - x64 Gen2 (default).
-
For Size, select a VM size from the dropdown menu, or leave the default VM size selection. To learn more, see Azure VM comparisons.
-
In the Administrator account section, for Authentication type, select SSH public key or Password.
-
Enter the credential settings, depending on the authentication type.
Set up the load balancer
-
Click the Networking tab.
-
In the Networking interface section, fill out the following fields:
- Virtual network: Click Create new, or select an existing virtual network from the dropdown menu. To learn more, see Quickstart: Use the Azure portal to create a virtual network.
- Subnet: Click Manage subnet configuration, or select a new or existing subnet from the dropdown menu. To learn more, see Add, change, or delete a virtual network subnet.
- Configure network security group: Click Create new, or select an existing security group from the dropdown menu. To learn more, see Create, change, or delete a network security group.
-
In the Load balancing section, fill out the following fields:
-
Load balancing options: Select Azure load balancer.
-
Select a load balancer: Click Create a load balancer and fill out the following fields in the pop-up window, or select an existing load balancer from the dropdown menu and note this name as you will need it in later steps:
- Enter a Load balancer name. Note this name, as you will need it in later steps.
- For Type, select Public or Internal.
- For Protocol, select TCP or UDP.
- Set both Port and Backend port to 80.
-
-
Click Create.
Finalize and deploy
-
Click Review + create.
-
Wait for validation.
-
Click Create.
Post-deployment: additional load balancer configuration
- Go to your load balancer: in the Azure portal’s Search resources, services and docs box, enter and then select Load balancers.
- Search for and open the new load balancer that you created earlier, or the existing load balancer that you chose earlier.
- Make any necessary settings updates to your new or existing load balancer, based on the recommended configurations in the Load balancer network settings section, later on this page.
Post-deployment: get the deployed endpoint URL
-
Go to your virtual machine: in the Azure portal’s Search resources, services and docs box, enter and then select Virtual machines.
-
Search for and open the new virtual machine that you created earlier, using the name that you entered earlier.
-
On the Overview tab, under Properties, note the Public IP address for the Load balancer.
-
The deployed endpoint URL is http://<load-balancer-public-IP-address>/general/v0/general. Note this endpoint URL, as you will need it later to call the Unstructured API.
Post-deployment: set API environment variables
Note the API environment variables in the API environment variables section, later on this page. If you need to set any of these in the Docker container on the virtual machine, do the following:
-
If the virtual machine is not already running from earlier, click the Start icon.
-
After the virtual machine starts, click the Connect icon, and then click Connect from the drop-down list.
-
Follow the on-screen directions for one of the available options to connect to the virtual machine and display a connected terminal.
-
Stop the running container in the virtual machine, so that you can restart it later with the environment variables set: In the connected terminal, run the following command:
sudo docker container ls
. -
Note the
CONTAINER ID
value for the running container. -
Run the following command, replacing
<CONTAINER ID>
with theCONTAINER ID
value:sudo docker container rm --force <CONTAINER ID>
-
Now run the container again, setting the environment variables at the same time: Run the following command:
sudo docker image ls
. -
Note the
REPOSITORY
andTAG
value for the Docker image. -
Run the following command, replacing
<REPOSITORY>
and<TAG>
with theREPOSITORY
andTAG
values for the Docker image, and replacing<VAR1>=<value1>
,<VAR2>=<value2>
and so on with the environment variable name and value pairs:sudo docker run -d --restart unless-stopped \ -p 80:5000 \ -e <VAR1>=<value1> -e <VAR2>=<value2> -e <VAR3>=<value3> \ <REPOSITORY>:<TAG>
-
Verify that the environment variables were set correctly: Run the following command:
sudo docker container ls
-
Note the
CONTAINER ID
value for the running container. -
Run the following command, replacing
<CONTAINER ID>
with theCONTAINER ID
value:
sudo docker exec <CONTAINER ID> bash -c 'printenv'
- The environment variables should be in the list that appears.
Call the Unstructured API
You can now use the running virtual machine to call the Unstructured API. For example, run one of the following, setting the following environment variables to make your code more portable:
- Set
UNSTRUCTURED_API_URL
tohttp://
, followed by your load balancer’s public IP address, followed by/general/v0/general
. - Set
LOCAL_FILE_INPUT_DIR
to the path on your local machine to the files for the Unstructured API to process. If you do not have any input files available, you can download any of the ones from the example-docs folder in GitHub. - Set
LOCAL_FILE_OUTPUT_DIR
to the path on your local machine for Unstructured API to send the processed output in JSON format.
Load balancer network settings
Unstructured recommends the following load balancer settings, which you should set on your deployment’s load balancer soon after you finalize and deploy it.
On the load balancer’s Overview tab in the Azure portal:
- SKU: Standard
On the load balancer’s Settings tab in the Azure portal:
-
Frontend IP configuration: Private IP
-
Backend pools: VMSS
-
Health probes:
- Protocol: HTTP, or HTTPS (this requires setting up a reverse proxy on the VMSS set to do TLS termination)
- Port:
80
or443
(this can be any port that the backend VMs are listening on) - Path:
/healthcheck
- Interval (seconds):
5
-
Load balancing rules:
- Protocol: TCP
- Port:
443
for HTTPS, or80
for HTTP - Backend port:
443
for HTTPS, or80
for HTTP - Idle timeout (minutes):
60
- Enable TCP Reset box: Checked
-
Inbound NAT rules:
- Frontend Port:
443
for HTTPS, or80
for HTTP - Backend port:
443
for HTTPS, or80
for HTTP - Protocol: TCP
- Enable TCP Reset box: Checked
- Idle timeout (minutes):
60
- Frontend Port:
API environment variables
Unstructured supports the following environment variables, which you can set in the Docker image on the virtual machine, as needed:
ALLOW_ORIGINS
: CORS-allowed origins.UNSTRUCTURED_ALLOWED_MIMETYPE
: The list of allowed MIME types, if you want to limit the file types that can be processed.UNSTRUCTURED_API_KEY
: The default Unstructured API key to use.UNSTRUCTURED_MEMORY_FREE_MINIMUM_MB
: The minimum amount of free memory in MB to allow for processing a file. If this memory is too low, the server will return a503
error.UNSTRUCTURED_PDF_HI_RES_MAX_PAGES
: The maximum number of pages in a PDF file that the Unstructured API will not reject, if thehi_res
strategy is used. The default is300
.UNSTRUCTURED_REDIRECT_ROOT_URL
: If this is set, redirect aGET
request to the Unstructured API to use this URL instead.
Was this page helpful?