43

This is a general question on a subject I've found interesting as a gamer: CPU/GPU bottlenecks and programming. If I'm not mistaken, I've come to understand that both CPU and GPU calculate stuff, but that one is better in some calculations than the other due to the difference in architecture. For example, cracking hashes or cryptocurrency mining seems way more efficient on GPUs than on CPUs.

So I've wondered: is having a GPU at 100% load while the CPU is at 50% (for example) inevitable?

Or, more precisely: Can some calculations that are normally done by the GPU be done by the CPU if the first one is at 100% load, so that both reach a 100% load?

I've searched a bit about the subject, but have come back quite empty-handed. I think and hope this has its place in this subsection and am open to any documentation or lecture you might give me!

  • 53
    It's trivially possible to have both CPU and GPU both run an infinite loop of NO-OPs at the same time, which will lead to both having a load of 100%. – Jörg W Mittag Jun 24 '16 at 10:43
  • 17
    Following @Jörg's point, the only thing measured by CPU % is what fraction of time is not spent waiting for other processors. 100% can be a good thing if the program is efficient, or a bad thing if the the program is inefficient. Too much of the time, people focus on CPU % as if it's a measure of performance - it is not. – Mike Dunlavey Jun 24 '16 at 12:08
  • 22
    The original Crysis did this just fine. – CubicleSoft Jun 24 '16 at 12:41
  • 5
    @MikeDunlavey you bring up a good point. With cars we don't measure their performance by the RPM, we measure speed. – Captain Man Jun 24 '16 at 13:50
  • 1
    @JörgWMittag: The CPU, maybe. But OS's and GPUs have halting problem solvers to deal with infinite loops. Namely, if a shader doesn't complete in a reasonable amount of time, it dies and the GPU resets. – Nicol Bolas Jun 25 '16 at 0:15
62

Theoretically yes, but practically it's rarely worth it.

Both CPUs and GPUs are turing-complete, so any algorithm which can be calculated by one can also be calculated by the other. The question is how fast and how convenient.

While the GPU excels at doing the same simple calculations on many data-points of a large dataset, the CPU is better at more complex algorithms with lots of branching. With most problems the performance difference between CPU and GPU implementations is huge. That means using one to take work from the other when it is stalling would not really lead to a notable increase in performance.

However, the price you have to pay for this is that you need to program everything twice, once for the CPU and once for the GPU. That's more than twice as much work because you will also have to implement the switching and synchronization logic. That logic is extremely difficult to test, because its behavior depends on the current load. Expect very obscure and impossible to reproduce bugs from this stunt.

  • 1
    You mentioned that With most problems the performance difference between CPU and GPU implementations is huge, I'm actually quite interested to which extent the performance gap goes. Would you have any numbers or articles about this (for example, on the example of texture 3D-rendering)? Thanks for your answer and for your time! – MadWard Jun 24 '16 at 9:25
  • 2
    You may want to add that there are performance costs for synchronization between the CPU and GPU, so you generally want to minimize the number of transfers between the two. Also, naively adding in branches for "don't execute on the elements the CPU already worked on" wouldn't buy you anything, since the GPU threads operate in lockstep. – Ethan Jun 24 '16 at 15:40
  • 3
    @gardenhead Nothing in the universe supports unbounded recursion, because the universe is finite size and has finite information density. "Turing-completeness" of a system is generally a discussion of what would be possible with such constraints removed. – Random832 Jun 24 '16 at 19:02
  • 3
    I have little doubt that a modern GPU is technically at least as close to Turing completeness as an 80's PC... however, if you try to run general algorithms on a GPU it will usually degenerate into a sequential processor that also won't be faster than an 80's PC, so the Turing-completeness of a GPU is in practice hardly more useful than the Turing-completeness of Brainfuck. – leftaroundabout Jun 24 '16 at 21:43
  • 7
    @leftaroundabout Modern GPUs are trivially Turing complete as any CPU. Turing completeness has nothing to do with: 1) performance 2) readability of source. 80's CPU were as close to TC has everything else: either they were TC or they weren't (the latter option being nonsense). – Margaret Bloom Jun 25 '16 at 16:27
36

It is not related to game programming. Some scientific code can also use both the GPU and the CPU.

With careful -and painful- programming, e.g. by using OpenCL or CUDA, you could load both your GPU and your CPU near 100%. Very probably you'll need to write different pieces of code for the GPU (so called "kernel" code) and for the CPU, and some boring glue code (notably to send into the GPU the compiled kernel code).

However, the code would be complex, and you probably need to tune it to the particular hardware you are running on, in particular because data transmission between GPU & CPU is costly.

Read more about heterogeneous computing.

See also OpenACC, supported by recent versions of GCC (e.g. GCC 6 in june 2016)

  • 1
    You're right, my tags and title were misleading, removed games and added performance/optimization. I didn't mean that it was exclusive to games, but that's where I noticed it. I thought it would have to be very hardware-specific too. Thanks for your answer and links! – MadWard Jun 24 '16 at 9:20
  • 3
    This would pretty much end up with two algorithms. I tried it once: whole image at once for GPU, and multiple images at once for CPU (to abuse large cache). It is indeed painful, especially to maintain. – PTwr Jun 24 '16 at 12:18
11

From a supercomputing viewpoint it is better not to think in CPU/GPU load in percentage but rather determine how many operations your problem at hand needs and then compare that to the peak performance of the system.

If you get 100% CPU utilization it does not necessarily mean that you get all the performance out of the system. CPUs can often do multiple different things at the same time, say a division and an addition. If you can start the division early, it can possibly be overlapped with the addition. Your desktop CPU most likely has an out of order unit which will reorder the statements in order to benefit from such overlaps. Or if you have the following program:

if (expr1)
    expr2;
else
    expr3;

A reordering CPU will try to compute the three expressions at the same time and then throw away the result of one of them. This makes it faster overall. If you have some blocker in your program and you cannot reorder, then you are utilizing less lanes in the CPU, but it will probably still show 100%.

Then you have SIMD features in the CPUs which are vector operations. It is like GPGPU-light in the sense that you usually only have four or eight operations at the same time, GPUs do like 32 or 64. Still you have to use that to crank out the FLOPS.

Stuff like false sharing can lead so a heavy synchronization cost which usually shows up as kernel load in Linux. The CPU is completely used but you do not have much useful throughput.

I have done some programming on an IBM Blue Gene/Q machine. It is has many hierarchy levels (schematic of outdated Blue Gene/L) and is therefore hard to program efficiently. You will have to use the full hierarchy down to SIMD and SMT (Intel calls this HyperThreading) in order to get the performance out.

And then the network often limits you. Therefore it turns out that it is faster in (wall clock) time to compute things at multiple CPUs at the same time instead of communicating it over the network. This will put more load on the CPUs and make the program run faster. But the actual program throughput is not as good as it seems from the raw numbers.

If you add GPUs to the mix, it will become even harder to orchestrate this whole thing to yield performance. That will be one of the things I'll start to do in my Lattice QCD Master Thesis in a couple months.

1

You might be interested in checking out the Servo browser engine being developed at Mozilla Research, and more specifically its Web Render (video).

While shifting a task from CPU to GPU dynamically might be impractical, as mentioned in other answers (notably @Philip's), it can be practical to study the load of CPU/GPU on typical workloads in advance and switch some tasks to the generally less loaded one.

In the case of Web Render, the novelty is that traditionally browsers do most of their rendering work on the CPU (ie, the CPU is used to compute which objects to display, where to cut, etc...). The GPU is normally better at it... except that not all usecases are trivial to implement (partial culling, shadows, ... and text).

An initial version of Web Render proved highly successful in the performance increase, but did no try to address the issue of text rendering (and had a few other limitations). Mozilla Research is now working on a second version which is intended to have fewer limitations, and notably to support text rendering.

The goal, of course, is to off-load as much as possible of the rendering process to the GPU, leaving the CPU free to execute Javascript, update the DOM, and all the other tasks.

So, while not as extreme as your suggestion, it does go into the direction of designing a computation strategy with both CPU and GPU in mind.

0

With a focus on games (since you mentioned it specifically in your post), there are some ways you can balance the load. One example is "skinning", i.e. animating a model. For each frame to be rendered, you have to generate the transformation matrices for each frame of animation and apply it to the vertices of the model to transform it into the pose it needs to be in. You also have to interpolate frames to get smooth movement, unless you want your animation to look like the original Quake (i.e. jerky).

In this situation, you could do it either on the CPU and upload the results to the GPU for rendering, or do the calculation and rendering on the GPU. I believe nowadays it is done on the GPU (known as "hardware skinning"): it makes sense to do so given you have relatively simple calculations that have to be performed thousands of times over, and each vertex can be calculated concurrently since the result of vertex A has no bearing on the result of vertex B.

In theory however, you could dynamically switch between doing it on the CPU or GPU depending on how overloaded the GPU and CPU are.

The main blocker to doing this across all calculations however is that the CPU and GPU have different strengths and weaknesses. Massively parallel jobs are better done on the GPU, while intensive linear tasks with branching are better done on the CPU. Only a few jobs could realistically be done on both without a serious performance hit.

Overall, the major issue with GPU programming (at least with OpenGL and DirectX 11 and under) is that you have little control over how the GPU interprets your shader code. Branching within a shader is risky because if you accidentally create a dependency between calculations, then the GPU may decide to start rendering your pixels one-by-one, turning 60fps to 10fps in an instant despite the actual data to be rendered being identical.

0

One real world example is the open source LuxRender rendering engine, which is capable of fully loading a CPU and GPU at the same time. In addition, it can load multiple GPUs at the same time and can also distribute across multiple computers.

LuxRender uses OpenCL to facilitate this, although builds without OpenCL also exist.

This is practical because the algorithms that LuxRender uses are highly parallelizable. The most common algorithm LuxRender uses is path tracing, where many individual light paths may be computed independently of each other—an ideal situation for GPU computing and one which requires no complex synchronization between compute nodes. However, limitations of GPUs (lower amounts of memory, lack of support for some complex rendering features, and general lack of availability to some artists) ensure that CPU support is still essential.

  • what's the point of showing this image, how is it relevant to the question asked? – gnat Jun 25 '16 at 23:07
  • 1
    Ehh fine. I'll delete it. I was thinking it would easily demonstrate what kind of software it is. But perhaps it's just really distracting. (There many different kinds of rendering engines; this one is targeted at photorealistic stills.) – PythonNut Jun 25 '16 at 23:08
0

Yes, it's certainly possible.

Any computation that a CPU can do, a GPU can also do, and vice versa.

But it's uncommon because:

  • Engineering complexity While it is possible to run the same code on a CPU and GPU (e.g. CUDA), the processors have different abilities and performance characteristics. One is MIMD; the other, SIMD. What is fast on one is slow on the other (e.g. branching), so you make need to write separate code to maximize performance.

  • Cost efficiency GPUs are in aggregate a lot more powerful than CPUs. The whole idea of GPUs is to use cheaper, slower, but more numerous processors to perform computations far faster than CPUs could for the same cost. GPUs are more efficient cost-wise by one or two orders of magnitude.

If you get your algorithm to run on GPUs, it just makes more sense to optimize for those, and add as many as you need.

protected by gnat Jun 25 '16 at 23:05

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.