0

I am working on a script to get data from excel spreadsheets to a database. The data is from surveys conducted by our office where the data comes in with very little formatting. At the moment, the data is manipulated with a lot of copy and paste, then analysed 'by hand' (ie. someone clicks and drags in excel and does pivots and copy pastes into other software).This leads to messy file structures and missing or incorrect/incomprehensible data. enter the need for a database.

I have got a solution working which will accept csv files. It will parse the data and insert it correctly as long as each column header is correct. But my superior is pushing me to accept xml files with an xsd schema so I can validate the data.

My argument would be that whether or not I use xml, the user will have to save the original file as another file type and I can validate the data within my script based on column headers anyway.

The counter argument is that if they decide to include a new data set (ie, new table layout) my script might break.

Either way, if the data type is incorrect, the user will have to go back to the file and edit it before any solution will work. So the answer I'm looking for is whether or not I should bother to set up xml functionality.

[note, I am using php to script as I am not familiar with vba and I'm on a student placement so I do not have enough time to learn a new language.] [ASIDE - am I taking the wring approach to this?]

Improve this question
5
  • if you import only those columns with headers you recognize, (and hopefully, independent of their position), why would your importer break if they change the table layout?
    – keppla
    Commented Jul 12, 2011 at 14:35
  • I don't think it will, and it shouldn't but that is not the general consensus. CSV seemed to me like the natural choice for this sort of data.
    – Aido
    Commented Jul 12, 2011 at 14:40
  • if youre forced to do design by comittee ('its not the general consensus') the problem does not seen to be a technical one (i.e., whats the best solution), but a political one (i.e., what do they want to be the 'best solution')
    – keppla
    Commented Jul 13, 2011 at 6:26
  • @keppla I understand, but I would like to know the best approach regardless of political undertones if you get me.
    – Aido
    Commented Jul 13, 2011 at 8:41
  • I think, Jack M. has this best approach.
    – keppla
    Commented Jul 13, 2011 at 8:45

1 Answer 1

Reset to default
2

If I were you, I would create the importer to work with the CSV, but include stub functionality for the XML. Release the tool with the CSV functionality and then begin working on the XML.

As far as your superior is concerned, I would point out that working with CSV is drastically simpler than working with XML, so you can get a deliverable faster, and then work on the XML portion. This means that the rest of the team can begin using this tool, and then later on if they do decide to change, the XML functionality will be there.

As to your final question, sort of. You have an idea for something good, and your superior is trying to provide an augmentation to make that better in their mind. This isn't a bad thing, as you get paid the same, but have more hours of work to do. As a result, I would suggest that you try and massage their idea into yours, and make it an even better idea instead of fighting the change.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.