Does the games industry use automated testing for visual parts of games/rendering? How?

Question

Some parts of a game are easy to test in an automated way (logic, maths, input handling); but there's also a lot that's purely visual and not easily testable.

I would be surprised if the games industry left all of this to manual testing; there's enough money in it that I would guess that effort has been put into being able to regression test at least some visual aspect of games.

Is this true? If so, what are the possible ways that rendering of games can be tested? Capturing output and comparing images (can this be reliable?)? Intercepting data from the graphics card at a low level? Capturing vertex information (etc.) on its way to the graphics card? It seems like there are lots of possibilities; but I can't find any information on this :(

Note: This question was marked as a dupe of this one, but I'm not asking about specific tech/frameworks/tools on how to do this, but more broadly the ideas around this practice, and what the actual games industry do (if they do at all).

Answer 1

Any company will do in different ways, even different games will be tested differently.

• Maybe the lack of information about graphics is because rendering is indeed quite "repetitive" (I cannot think a better word, sorry) and in a typical complex scene there will be enough primitive combinations so most persons can usually see easily most of the glitches (with the exception of less used shaders, but a stress-shader special level/demo can be used). Remember that humans were hunters for thousands of years :) so probably we are engineered to see glitches.
• Also the freedom of movement present in most games and the randomization of other elements on a typical game that makes to feel it as more "realistic" are usually a nightmare for applying pure automated tests: in fact a human tester will find more quickly and more errors than any automated test. You can code a logic for record timings, stats and positions/angles/other variables whenever a tester is playing... so even on no-automated tests you will have feedback like-automated-tests. It's usually useful to be capable of making a playback of any beta tester recorded session (and usually is a good feature for the final game as well).
• About rendering stats, for example you can get: the draw-call count stats for each one of the shaders (as you will make the calls, it is enough to maintain some counters), the upload counts (the same, you will control when gpu buffers are updated), the timings (not only fps, also the lapses between each call), etc. If necessary (but I believe that is not recomended except when hunting for a specific error) you can get stats from the graphics card itself (for example you can get fill rates and others by using occlusion queries).
• Finally, other reason to use beta testers is that different graphic cards will have probably different behaviors... you may need to have a farm of testing machines o_O. The beloved human beta testers will soft all that mess.

Hope that it helps!

Disclaimmer: I'm not a betatester, I'm a developer XDDD.