Unstructured API on Azure

Go to https://portal.azure.com.

Go to the Unstructured Data Preprocessing - Customer Hosted API offering in the Azure Marketplace.

1. Click Get It Now and fill out the form.
2. Read the terms and click Continue.
3. Click Create.

1. On the Create a virtual machine page, click the Basics tab.

2. In the Project details section, select an existing Subscription, and select an Resource group from the dropdown menus, or create a new resource group by clicking Create new.

3. In the Instance details section, enter a name in the Virtual machine name field. Note this name, as you will need it later steps.

4. Select a Region from the dropdown menu.

5. For Image, select Unstructured Customer Hosted API Hourly - x64 Gen2 (default).

6. For Size, select a VM size from the dropdown menu, or leave the default VM size selection. To learn more, see Azure VM comparisons.

7. In the Administrator account section, for Authentication type, select SSH public key or Password.

8. Enter the credential settings, depending on the authentication type.

1. Click the Networking tab.

2. In the Networking interface section, fill out the following fields:

   • Virtual network: Click Create new, or select an existing virtual network from the dropdown menu. To learn more, see Quickstart: Use the Azure portal to create a virtual network.
   • Subnet: Click Manage subnet configuration, or select a new or existing subnet from the dropdown menu. To learn more, see Add, change, or delete a virtual network subnet.
   • Configure network security group: Click Create new, or select an existing security group from the dropdown menu. To learn more, see Create, change, or delete a network security group.

3. In the Load balancing section, fill out the following fields:

   • Load balancing options: Select Azure load balancer.

   • Select a load balancer: Click Create a load balancer and fill out the following fields in the pop-up window, or select an existing load balancer from the dropdown menu and note this name as you will need it in later steps:

     • Enter a Load balancer name. Note this name, as you will need it in later steps.
     • For Type, select Public or Internal.
     • For Protocol, select TCP or UDP.
     • Set both Port and Backend port to 80.

4. Click Create.

1. Click Review + create.

2. Wait for validation.

3. Click Create.

1. Go to your load balancer: in the Azure portal’s Search resources, services and docs box, enter and then select Load balancers.
2. Search for and open the new load balancer that you created earlier, or the existing load balancer that you chose earlier.
3. Make any necessary settings updates to your new or existing load balancer, based on the recommended configurations in the Load balancer network settings section, later on this page.

1. Go to your virtual machine: in the Azure portal’s Search resources, services and docs box, enter and then select Virtual machines.

2. Search for and open the new virtual machine that you created earlier, using the name that you entered earlier.

3. On the Overview tab, under Properties, note the Public IP address for the Load balancer.

4. The deployed endpoint URL is http://<load-balancer-public-IP-address>/general/v0/general. Note this endpoint URL, as you will need it later to call the Unstructured API.

Note the API environment variables in the API environment variables section, later on this page. If you need to set any of these in the Docker container on the virtual machine, do the following:

1. If the virtual machine is not already running from earlier, click the Start icon.

2. After the virtual machine starts, click the Connect icon, and then click Connect from the drop-down list.

3. Follow the on-screen directions for one of the available options to connect to the virtual machine and display a connected terminal.

4. Stop the running container in the virtual machine, so that you can restart it later with the environment variables set: In the connected terminal, run the following command: sudo docker container ls.

5. Note the CONTAINER ID value for the running container.

6. Run the following command, replacing <CONTAINER ID> with the CONTAINER ID value:

   sudo docker container rm --force <CONTAINER ID>
   

7. Now run the container again, setting the environment variables at the same time: Run the following command: sudo docker image ls.

8. Note the REPOSITORY and TAG value for the Docker image.

9. Run the following command, replacing <REPOSITORY> and <TAG> with the REPOSITORY and TAG values for the Docker image, and replacing <VAR1>=<value1>, <VAR2>=<value2> and so on with the environment variable name and value pairs:

   sudo docker run -d --restart unless-stopped \
   -p 80:5000 \
   -e <VAR1>=<value1> -e <VAR2>=<value2> -e <VAR3>=<value3> \
   <REPOSITORY>:<TAG>
   

10. Verify that the environment variables were set correctly: Run the following command:

    sudo docker container ls
    

11. Note the CONTAINER ID value for the running container.

12. Run the following command, replacing <CONTAINER ID> with the CONTAINER ID value:

sudo docker exec <CONTAINER ID> bash -c 'printenv'

14. The environment variables should be in the list that appears.

You can now use the running virtual machine to call the Unstructured API. For example, run one of the following, setting the following environment variables to make your code more portable:

• Set UNSTRUCTURED_API_URL to http://, followed by your load balancer’s public IP address, followed by /general/v0/general.
• Set LOCAL_FILE_INPUT_DIR to the path on your local machine to the files for the Unstructured API to process. If you do not have any input files available, you can download any of the ones from the example-docs folder in GitHub.
• Set LOCAL_FILE_OUTPUT_DIR to the path on your local machine for Unstructured API to send the processed output in JSON format.