0

My restraints are that I have to use: Django and MongoDB

I am trying to build a website that will read some JSON input and then parse all the properties and objects in the JSON and display results. This is so that I can check which properties in the JSON are missing and what the actual values are to the properties. In the JSON is roughly 300 properties and I am only interested in half. This is why making this tool will help so that I don't have to look manually through it all.

The issue I am running into is the amount of time it takes to search. It takes roughly 5-9 seconds each time I parse the JSON. I don't know if this is a fast, slow, or normal amount of time. This is how the code works:

  1. I paste all the JSON and select submit. This JSON is a "event". Basically this JSON was created after doing a single action in one of the apps. For example if I begin to watch a video in the app then the event "Video Started" would generate and this is what I am looking at. I have over 400 different events that all contain different requirements for properties and expected values.
  2. JSON is loaded with Python.
  3. I determine which device (11 difference devices) and which app (6 different apps) this JSON applies to based on a menu selected by the user.
  4. I then loop through all the properties that I have saved in my database which is roughly 350-400. While doing so, I check to see if that property applies to this device, app, and event and store into an array.
  5. Then I loop through the JSON and check against each of the properties that I expect to have that is in my array I created above. If the property is present, I copy the name and value into another array which will then get sent to the front end to display. If the property is missing, I add the property to the array but I put "Missing" as the value.

The way I do this is because each property has its own requirements based on the app and device. And I am trying to design a way that if a change is made, I only have to make the change to the property that is stored versus editing each instance of that property for each device, app, and event setup.

This is all complicated without actually showing but hopefully someone will be able to understand and can offer some sort of feedback.

Improve this question
3
  • 3
    Multiple seconds sounds pretty slow. Do you have timings that indicate which of these steps is leading to the biggest slowdown? I suspect that step 4 is the slow part if you're doing 400 separate database requests. Can you batch that into a single query returning multiple documents in one go?
    – amon
    Apr 11, 2022 at 14:25
  • Interesting, I never thought about running some time checks against the different steps. I am learning Django still so querying all at once might or might not be possible. I will look into the timing and then see where the slowest portion is.
    – xf900
    Apr 11, 2022 at 14:36
  • "Timing each step" is critical when tracking down any performance problem in any technology (see also "benchmarking an application"). You can often narrow things down quickly.
    – Greg Burghardt
    Apr 11, 2022 at 19:10

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.