This question is actually a bunch of issues with your data model rolled up into one. You need to start untangling them, one at a time. More natural, intuitive solutions will drop out as you try to simplify each piece of the puzzle.
Problem 1: You can't depend on DB Order
Your descriptions of sorting your data is not clear.
- The biggest potential problem is that you're not specifying an explicit sort in your database, via an
ORDER BY clause. If you're not because it seems too expensive, your program has a bug. Databases are allowed to return results in any order if you don't specify one; you can't depend on it coincidentally returning data in the order just because you ran the query a few times and it looks that way. The order might change because rows get rearranged on disk, or some are deleted and new ones take their place, or an index is added. You must specify an
ORDER BY clause of some kind. Speed is worthless without correctness.
- It's also not clear what you mean by insertion order mattering. If you're talking about the database itself, you must have a column that actually tracks this, and it must be included in your
ORDER BY clause. Otherwise, you have bugs. If such a column doesn't exist yet, then you need to add one. Typical options for columns like this would be an insertion timestamp column or an auto-incrementing key. The auto-incrementing key is more reliable.
Problem 2: Making in memory sort efficient
Once you make sure it's guaranteed to be returning data in the order you expect, you can leverage this fact to make in memory sorts much more efficient. Just add a
dense_rank() column (or your database's equivalent) to your query's result set. Now each row has an index that will give you a direct indication of what the order is supposed to be, and you can sort by this in memory trivially. Just make sure you give the index a meaningful name (like
Viola. Now you don't have to depend on the database result set order anymore.
Problem 3: Do you even need to do this processing in code?
SQL is actually really powerful. It's an amazing declarative language that lets you do a lot of transformations and aggregations on your data. Most DBs even support cross row operations nowadays. They're called window or analytic functions:
Do you even need to pull your data into memory like this? Or could you do all the work in the SQL query by using window functions? If you can do all (or maybe even just a significant part) of the work in the DB, fantastic! Your code problem goes away (or gets a lot simpler)!
Problem 4: You're doing what to that
Assuming you can't do it all in the DB, let me get this straight. You're taking the data as map (which is keyed by things you don't want to sort by), then you're iterating over it in insertion order, and modifying the map in place by replacing the value of some keys and adding new ones?
I'm sorry, but what the heck?
Callers should not have to worry about all this. The system you've created is extremely fragile. It only takes one dumb mistake (maybe even made by yourself, like we have all done) to make one little wrong change and the whole thing collapses like a deck of cards.
Here's maybe a better idea:
- Have your function accept a
- There are a couple ways you can handle the ordering problem.
- Apply Fail Fast. Throw an error if the list in not in the order the function requires. (Note: You can use the sort index from Problem 2 to tell if it is.)
- Create a sorted copy yourself (again using the index from problem 2).
- Figure out a way to build up the map itself in order.
- Construct the map you need internally to the function, so the caller doesn't have to care about it.
- Now iterate over whatever in order representation you have and do what you have to.
- Return the map, or transform it into an appropriate return value
A possible variation could be to construct a sorted representation and then create a map of key to index. This would let you modify your sorted copy in place, without accidentally creating duplicates.
Or maybe this makes more sense: get rid of the
data parameter and make
processData actually fetch its own data. You can then document that you're doing this because it has very specific requirements on the way in which the data is fetched. In other words, make the function own the entire process, not just one piece of it; the inter-dependencies are too strong to split the logic into smaller chunks. (Change the name of the function in the process.)
Maybe these won't work for your situation. I don't know without full details of the problem. But I do know a fragile and confusing design when I hear one.
I think the problem here is ultimately that the devil is in the details. When I start running into trouble like this, it's usually because I have an inappropriate representation of my data for the problem I'm trying to actually solve. The best solution is find a better representation, and then my problem becomes simple (maybe not easy, but straightforward) to solve.
Find someone who gets that point: your job is to reduce your problem to a set of simple, straightforward ones. Then you can build robust, intuitive code. Talk to them. Good code and good design make you think that any idiot could have thought them up, because they're simple and straightforward. Maybe there's a senior developer who has that mindset you can talk to.