I have a function which sends data to a third-party's server using a library they've supplied. The function is very simple. It just takes a list of objects to send, loops through each item in the list, and calls the third-party library's function to upload an object. The problem I'm running into is that my list of objects is fairly big (~22,000 entries, though each entry is only about 240 bytes, so only ~5MB in total), and I always get exceptions part way through uploading the list.
I get a variety of errors, typically Internal Server Errors, but also errors related to missing pieces at the level where their library is working.
A solution I've just written (and tested and had work!) is to catch any exception during the upload process, and then recall my upload function, but pass it only the objects in my list which haven't made it through the upload process yet.
This worked when I tested it, though it caught eleven exceptions before making it all the way through my 22,000 entries. Something about writing code this way makes me feel like an abuser.
I've got a couple of ideas to improve my exception handling:
- Catch only certain exceptions - I've only seen three or four different exceptions come up. If I limit my catch to those same three or four, the program will be more predictable (I won't catch something unrelated like a lost network connection on my end, and try to keep sending my data up)
- Limit the number of times I try recalling my upload function - Supposing their server is down, I could end up in an infinite loop. If I limit it to say 50 or 100 tries, that should hopefully be far more than I need.
Is there a standard best practice for this sort of situation? Is this a misuse of exceptions?