3

I'm looking for a conceptual approach to a problem.

I'm building a web app that imports data from a source I don't control. The data quality is pretty high, but occasionally, the data does not make sense. So I want to expose the exceptions (based on data rules) to users in a modal so that they can accept the funky data or change it before committing.

The app has been developed so far to behave like this:

  • while importing data, transactions are validated
  • transactions that do not validate are accumulated in a php array (aka "exceptions_arr")
  • in some cases intelligence in the code proposes a solution to "bad" data fields (based on related data) and adds these suggestions to the "exception" record (but does not replace the original imported data)
  • then before completing the import, the exception records (and proposed fixes) are exposed to the user in a modal, one at a time
  • for each record, the user decides: skip it, accept the original data or edit the record and save it (which will, of course, be validated again for security reasons)

Goal:

Create a user experience IN THE MODAL: display an exception, submit the user's decision and/or edits, record that data on the backend, send a confirming message, then serve up the next exception by replacing the body of the modal. Bonus if the underlying page is updated (with some counter stats) based on the decisions made.

Question: how to do this?

It's the looping through the exceptions that I'm having trouble with. My approach so far has been to: a) set and display the modal with an empty structure (done, no issues) b) create the tbody of the table with the first exception data (done, no issues) c) capture the appropriate data (none, old, proposed/edited) when the user submits his/her decision (done, no issues)

Current approach:

Currently I am POSTing the user's decision data to the host page via ajax, and handling the resulting user decision/POST just fine - once. The problem is that this POST back to the page that launched the modal blows away the accumulated exception_arr.

I have searched how to loop through the data one record at a time and halt the process when each exception is found (and pop open a fresh modal and collect the user's decision), but alas jquery/javascript/php code isn't easily halted and resumed. When the user decision is POSTed, the launching page still restarts (from the beginning).

Is using SESSION storage a good approach? Read all exceptions into the DOM and then process?

2
  • Could you import all the acceptable records, and then present a page with all the exceptions (possibly a table with one exception per row), and have the user choose what to do with each row? Would at least save you having to navigate back and forth between the import and exception handling pages.
    – Dan1701
    Dec 29, 2015 at 4:34
  • Yes, that's possible. The advantage of the persistent modal is less clicking and rendering time, but architecturally, the row display would work. I'll keep that as a strong Plan B if I can't get the modal route to work. Dec 29, 2015 at 6:00

1 Answer 1

1

In my experience the most stable solution for this scenario is to still persist the failing records rather than keeping them in memory or in the session. What if the user gets distracted in the middle and then closes their browser and goes home? What if the server crashes or you need to restart? Then you loose all the records currently being processed.

So, I would persist the error records anyway. Depending on your data, you can stick them into the same tables as any other records and just flag them for review. Alternatively, you can store them into a separate set of tables or just as a file to the file system.

Then your screen can just ask the database for the next failing record for that user and display that record. Once the record gets handled it gets deleted and inserted properly.

An additional benefit to this is that this keeps your web server stateless. Which means you can set-up clustering and don't need to worry which of your web-servers the user is accessing.

1
  • Great advice. I was able to get the interaction working between the modal and the backend, but your suggestion about persisting the errant records makes a ton of sense for all of the reasons stated. Jan 2, 2016 at 11:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.