-3

I am making a Java and Spring webapp that scrapes data from a web and then publishes it through an API. Some of the raw scrapped data is in the form of Set<SomeObject> which I then convert to a Set<ConvertedObject>. The thing is, there is a field that has a repeating value in a lot of SomeObject and I want to avoid converting it more than once - it isn't an expensive operation but still.

So far I have come up with two solutions:

  • Extract that field from SomeObject to a Map<String, Set<SomeObject>>, which is easy to do from the web I am scrapping (like 5 lines of code).
  • Keep the field inside SomeObject and make an internal service with cache to convert that field, which is also easy to do with Spring.

The possible values are not so many; 7 values cover 95%+ of the entities, but some times there are some other values apart from those which I am able to parse.

I like the second solution in that the code becomes cleaner, no complicated Map and Set structure, but it is also a more sophisticated one in my opinion that isn't really necessary.

Which one do you think is better?

To better clarify, I am most concerned with clean code rather than efficiency. Plus, I am also interested in a general answer regarding Spring web applications design. Should I use components everywhere I can or sometimes vanilla solutions are better?

  • To some extent, algorithms and data structures are used with most things especially with OOP (object-oriented programming), so it makes your question seem off topic or in need of some clarification into what you mean by "avoid". With just the main content at a glance, what are you meaning with "a field that has repeating value in a lot of SomeObject"? Is it that over each SomeObject collection item this field is typically the same value, or that the actual value held in a field is also seen held in other fields of SomeObject? – eparham7861 Jan 18 at 22:53
  • 1
    The question in the title makes no sense. Your scraper is an algorithm, your classes are data structures. All code is either an algorithm (function/method/operation) or data (fields/properties/variables). If you had neither, you wouldn't have built anything. – Flater Jan 18 at 23:28
  • 1
    "it isn't an expensive operation but still" This implies you're doing it just because, which sets you off on the wrong foot from the start. Solutions exist to solve problems. If you can't point out the problem, there's no point in trying to provide a solution (the only exception to this is teaching exercises - but then the question itself becomes moot) – Flater Jan 18 at 23:31
  • It means this field has tipically the same value. I know the wording of the question is not the best, I accept any suggestions you can make. But I think you can get what I'm asking. In a way the question is whether you would solve this problem in a "vanilla" way or rely on more complex concepts like Spring components with cache. The operation involves parsing a string with a 10 to 20 length. It's not expensive but I will do it many times. I considered calculating it many times but then I could just do the map from solution nr 1 – Hertzu Jan 19 at 3:29
  • To better clarify, I am most concerned with getting clean code. Making a map to avoid converting the same value many times or just making a set are both my decision, it's not like making the map involves a lot of trouble for a problem that isn't there. They involve roughly the same amount of lines of code. – Hertzu Jan 19 at 3:51
0

A cache is the correct solution. I would keep the signature of the cache the same as the conversion function. If a conversion function converts a string to a string, I would keep the cache a Map.

You'll want to pick a fixed size LRU cache so infrequently used items don't clog the cache.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.