You have the fundamental problem of working with multi-purpose "framework" code. The choice of a CSS foundation like Bootstrap vs Pure vs roll-your-own are not much different from Flask vs Django vs do-it-yourself, AngularJS vs Backbone.js vs home-grown, or dojo vs jQuery vs adhoc.
The fundamental questions are always ones like: What will get me there fastest, cheapest, most efficiently, and with the greatest ease? What will be the simplest and most flexible? What will be the most maintainable? What will require the least rework and force-fitting?
The fundamental problem is that those frameworks are always trying to support a large and diverse set of applications, and they are generally prescriptive. They assume that there are "right ways" to do things, and that most applications should do things those ways. Some are more emphatically normative, trying to impose the "right way," while others are merely conventional or suggestive. You see the same kind of differences in language design, where Python and Java for example clearly suggest "There is a right way to do it! Do it that way!" while Perl, PHP, and Ruby are more "Hey, man! Whatever floats your boat! Tim Toady!"
In your case, you seem to already have a good amount of experience with the options, and to understand the kind of diversions and patches that will be required for your fiddly custom projects. Absent examples of those designs, we probably cannot say whether building up from Pure or adjusting back from Bootstrap will be less work.
I have personally chosen the build-up route more often. It's challenging to discover how complex frameworks operate under the covers, and the best ways to extend them. I hate the feeling of constantly fighting my framework. If I find just the right way to subclass or adjust the framework, it can be awesome and elegant. But if I don't find that elegant adjustment, and then have to hack at it crudely, the process of figuring out work-arounds, and then debugging them and making them reliable, is onerous. There's also a gamble involved that you will be able to find the right work-arounds. Occasionally, it is not even feasible (at least, not in the allotted time/budget). So I would rather start with a smaller platform and build up. That has proven more reliable and effective for me.
So that argues for the Pure route. But you, good sir, seem much closer to the specific job/framework tradeoffs involved, and therefore are best positioned to make this choice. In the end, they're both great foundations, and either one will probably get you what you need.