Transform a JSON file into a different schema

In your local Python code project, install the json-converter package.

pip install json-converter

1. Find the local source JSON file that you want to transform.

2. Note the JSON field names and structures that you want to transform. For example, the JSON file might look like the following (the ellipses indicate content omitted for brevity):

   [
     {
       "type": "...",
       "element_id": "...",
       "text": "...",
       "metadata": {
         "filetype": "...",
         "languages": [
           "eng"
         ],
         "page_number": 1,
         "filename": "..."
       }
     },
     {
       "type": "...",
       "element_id": "...",
       "text": "...",
       "metadata": {
         "filetype": "...",
         "languages": [
           "eng"
         ],
         "page_number": 1,
         "filename": "..."
       }
     },
     {
       "...": "..."
     }
   ]
   

1. Decide what you want the JSON schema in the transformed file to look like. For example, the transformed JSON file might look like the following (the ellipses indicate content omitted for brevity):

   [
     {
       "content_type": "...",
       "content_id": "...",
       "content": "...",
       "content_properties": {
         "page": 1
       }
     },
     {
       "content_type": "...",
       "content_id": "...",
       "content": "...",
       "content_properties": {
         "page": 1
       }
     },
     {
       "...": "..."
     }
   ]
   

2. Create the JSON field mappings file, for example:

   {
     "content_type": ["type"],
     "content_id": ["element_id"],
     "content": ["text"]
     "content_properties.page": ["metadata.page_number"]
   }
   

   This file declares the following mappings:

   • The type field is renamed to content_type.
   • The element_id field is renamed to content_id.
   • The text field is renamed to content.
   • The page_number field nested inside metadata is renamed to page and is nested inside content_properties.
   • All of the other fields (filetype, languages, and filename) are dropped.

   For more information about the format of this JSON field mappings file, see the Project Description in the json-converter page on PyPI or the README in the json-converter repository in GitHub.

1. Set the following local environment variables:

   • Set LOCAL_FILE_INPUT_PATH to the local path to the source JSON file.
   • Set LOCAL_FILE_OUTPUT_PATH to the local path to the target JSON file.
   • Set LOCAL_FIELD_MAPPINGS_PATH to the local path to the JSON field mappings file.

2. Add the following Python code file to your project:

   import os, json
   from json_converter.json_mapper import JsonMapper
   
   # Converts one JSON file with one schema to another 
   # JSON file with a different schema. 
   # Provide the path to the input (source) and output 
   # (target) JSON files and the path to the JSON schema 
   # field mappings file.
   def convert_json_with_schemas(
       input_file_path: str,
       output_file_path: str,
       field_mappings_path: str
   ) -> None:
   
       output_data = []
   
       try:
           with open(field_mappings_path, 'r') as f:
               element_mappings = json.load(f)
       except FileNotFoundError:
           print(f"Error: Input JSON schema field mappings file '{input_file_path}' not found.")
       except IOError as e:
           print(f"I/O error occurred: {e}")
       except Exception as e:
           print(f"An unexpected error occurred: {e}")
   
       try:
           with open(input_file_path, 'r') as f:
               input_data = json.load(f)
               
               for item in input_data:
                   converted_data = JsonMapper(item).map(element_mappings)
                   output_data.append(converted_data)
       except FileNotFoundError:
           print(f"Error: Input JSON file '{input_file_path}' not found.")
       except IOError as e:
           print(f"I/O error occurred: {e}")
       except Exception as e:
           print(f"An unexpected error occurred: {e}")
   
       try:
           with open(output_file_path, 'w') as f:
               json.dump(output_data, f, indent=2)
       
           print(f"Transformation complete. Output written to '{output_file_path}'.")
       except IOError as e:
           print(f"I/O error occurred: {e}")
       except Exception as e:
           print(f"An unexpected error occurred: {e}")
   
   if __name__ == "__main__":
       convert_json_with_schemas(
           input_file_path=os.getenv("LOCAL_FILE_INPUT_PATH"),
           output_file_path=os.getenv("LOCAL_FIELD_MAPPINGS_PATH"),
           field_mappings_path=os.getenv("LOCAL_FILE_OUTPUT_PATH")
       )
   

3. Run the Python code file.

4. Check the path specified by LOCAL_FILE_OUTPUT_PATH for the transformed JSON file.