9

I'm currently at the design stage of a component-based architecture in C++.

My current design includes the usage of features such as:

  • std::vectors of std::shared_ptrs to hold the components
  • std::dynamic_pointer_cast
  • std::unordered_map<std::string,[yada]>

Components will represent data and logic of various items that are needed in a game-like software, such as Graphics, Physics, AI, Audio, etc.

I've read all over the place that cache misses are hard on performance, so I ran some tests, which led me to believe that, indeed, it can slow down an application.

I haven't been able to test the aforementioned language features, but it is said in many places that these tend to cost a lot and should be avoided if possible.

Since I'm at the design stage of the architecture, and these will be included in the core of the design, should I try to find ways to avoid them now, since it's going to be very hard to change it later if there are performance issues?

Or I'm just caught in doing premature optimization?

6
  • 3
    I would be very reluctant to settle on a design that made it very hard to change later, regardless of the performance issues. Avoid that if you can. There are many designs that are both flexible and fast. Jul 27, 2016 at 3:07
  • 1
    Without even knowing the details, the answer to this question is almost always a resounding "YES !!". Jul 27, 2016 at 17:34
  • 2
    @Mawg "...Yet we should not pass up our opportunities in that critical 3%." Since this is the core of the design, how could I know whether I'm working on this 3%? Jul 27, 2016 at 17:43
  • 1
    An excelelnt points, Alexandre (+1), and, yes, I do know that last half the quote, whcih almost never gets mentioned :-) But, to go back to my comment before that (which is reflected in the acecpted answer), the answer to this question is almost always a resounding "YES !!". I still feel that it is better to get it working first & optimize later, but YMMV, everyone has his opinion, all of which are valid, and only the OP can really answer his own - subjective - question. Jul 28, 2016 at 6:53
  • 1
    @AlexandreVaillancourt Keep reading Knuth's paper (PDF, quote comes from right side of page labeled 268, page 8 in a PDF reader). "...he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail." (emphasis his)
    – 8bittree
    Jul 28, 2016 at 15:00

4 Answers 4

26

Without reading anything but the title: Yes.

After reading the text: Yes. Though it is true that maps and shared pointers etc. do not perform well cache-wise, you will most certainly find that what you want to use them for — as far as I understand — is not the bottleneck and will not be held in or use cache efficiently regardless of the data structure.

Write the software avoiding the most stupid mistakes, then test, then find the bottlenecks, then optimize!

Fwiw: https://xkcd.com/1691/

3
  • 3
    Agreed. Make it work right first, because it won't matter how fast it fails to work. And always remember that the most effective optimizations don't involve tweaking the code, they involve finding a different, more efficient algorithm.
    – Todd Knarr
    Jul 26, 2016 at 21:50
  • 10
    I would like to point that the first line is not true because optimization is always premature, but rather because optimization is only not premature if you know you need it, in which case you wouldn't be asking about it. So, the first line is only true because the very fact that you are asking a question about whether or not optimization is premature means that you are not sure that you need optimization, which by definition makes it premature. Phew. Jul 27, 2016 at 0:49
  • @JörgWMittag: agreed.
    – steffen
    Jul 27, 2016 at 10:03
3

I'm not familiar with C++, but in general It depends.

You do not need to prematurely optimize the isolated algorithms where you can easily optimize when it come to that.

However you need to get overall design of the application to achieve desired key performance indicators.

For example if you need to design an application to serve millions of requests per second, you need to think about the scalability of the application when designing it, rather than getting the application working.

3

If you have to ask, then yes. Premature optimization means optimization before you are sure there is a significant performance problem.

1

ECS? I'll actually suggest that it may not be premature if so to put a lot of thought into the data-oriented side of the design and benchmark different reps because it could impact your interface designs, and the latter is very costly to change late in the game. Also ECS just demands a lot of work and thought upfront and I think it's worth utilizing some of that time to make sure it's not going to give you design-level performance grief further down the line given how it's going to be at the heart of your entire freaking engine. This part glares out to me:

unordered_map<string,[yada]>

Even with small string optimizations, you have a variable-sized container (strings) inside another variable-sized container (unordered_maps). In fact, the small string optimizations could actually be as harmful as helpful in this case if your table is very sparse, since the small string optimization would imply that each unused index of the hash table will still use more memory for the SS optimization (sizeof(string) would be much larger) to the point where the total memory overhead of your hash table might cost more than whatever you're storing into it, especially if it's a simple component like a position component, in addition to incurring more cache misses with the huge stride to get from one entry in the hash table to the next.

I'm assuming the string is some kind of key, like a component ID. If so, this already makes things dramatically cheaper:

unordered_map<int,[yada]>

... if you want the benefits of being able to have user-friendly names that scripters can use, e.g., then interned strings can give you the best of both worlds here.

That said, if you can map the string to a reasonably low range of densely-used indices, then you might just be able to do this:

vector<[yada]> // the index and key become one and the same

The reason I don't consider this premature is because, again, it could impact your interface designs. The point of DOD shouldn't be to try to come up with the most efficient data representations imaginable in one go IMO (that should generally be achieved iteratively as needed), but to think about them enough to design interfaces on top to work with that data that leave you enough breathing room to profile and optimize without cascading design changes.

As a naive example, a video processing software which couples all of its code against this:

// Abstract pixel that could be concretely represented by
// RGB, BGR, RGBA, BGRA, 1-bit channels, 8-bit channels, 
// 16-bit channels, 32-bit channels, grayscale, monochrome, 
// etc. pixels.
class IPixel
{
public:
    virtual ~IPixel() {}
    ...
};

Isn't going to get far without a potentially epic rewrite, since the idea of abstracting at the single pixel level is already extremely inefficient (the vptr itself would often cost more memory than the entire pixel) compared to abstracting at the image level (which will often represent millions of pixels). So put enough thought into your data representations in advance so that you don't have to face such a nightmare scenario, and ideally no more, but here I do think it's worth thinking about this stuff upfront since you don't want to build an intricate engine around your ECS and find that the ECS itself is the bottleneck in ways that require you to change things at the design level.

As for ECS cache misses, in my opinion developers often try too hard to make their ECS cache-friendly. It starts to yield too little bang for the buck to try to access all your components in a perfectly contiguous fashion, and will often imply copying and shuffling data all over the place. It's usually good enough to, say, just radix sort component indices prior to accessing them so that you are accessing them in a way where you at least aren't loading a memory region into a cache line, only to evict it, and then load it all over again in the same loop just to access a different part of the same cache line. And an ECS doesn't have to provide amazing efficiency all across the board. It's not like an input system benefits from that as much as a physics or rendering system, so I recommend aiming for "good" efficiency across the board and "excellent" just in the places where you really need it. That said, use of unordered_map and string here are easy enough to avoid.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.