Not distributing the binary of the library does not shield you from legal risk. The question you should be asking is not "can I legally distribute a GPL v3 licensed library with my proprietary app", but "can I legally distribute a proprietary app compiled with a GPL v3 licensed library?"
This question does not yet have a definitive answer. What, exactly constitutes a derivative work under copyright law as applied to software has not yet been fully nailed down. According to the FSF, what you want to do is not permitted by the GPL.
I personally agree with Linus Torvalds' opinion that in some cases, the FSF's view that linking GPL code always requires releasing both parts under the GPL is overbroad. I believe the definition of "derivative work" is in practice narrower than the FSF's definition, due in part to the purely functional (legal term, not programming term) nature of much software, especially software that implements or describes standardized APIs. However, while I find Linus' argument persuasive, a judge or jury may not.
If you want to avoid legal risk, you should not distribute a proprietary app linked with GPL code.
This also depends on the language in question. In C and C++, and probably other languages as well, the bodies of preprocessor macros are copied into your source code before it is submitted to the compiler. Does the library contain macros? Does your program use them? Are these macros copyrightable material? Does using those macros constitute a derivative work? In C++, if a module uses templates provided by a library, the compiled code for those templates appears in the binary of the module. Are those templates copyrightable material? Does using those templates constitute a derivative work? In many languages, certain short functions/subroutines can be explicitly or automatically in-lined. Does your binary contain in-lined code from a library? Is that code copyrightable material? Does using it constitute a derivative work? In languages with inheritance, does inheriting from a class create a derivative work of that class?
The precise details of how, exactly, your program makes use of the library matter. If your program's binary contains code from the library, which it very well might, depending on the language and compiler details I just mentioned, it is very likely a derivative work.
Furthermore, the precise terms of the library's license matter. Does the license explicitly permit the use of short macros and templates, as the Gnu LGPL v3 does, but the Gnu GPL v3 does not?
Response to edit 1:
There is a difference between using an external program via command-line invocation and linking a program with a library. If XCode executes GDB without linking to it, and Apple provides the corresponding source code of their version of GDB, then Apple is in compliance with the terms of the GPL. It does not matter that XCode contains proprietary components, as long as those components are not combined with GPL code in a single program. See the GPL FAQ entry on "mere aggregation".
On the other hand, if a company were to create a debugger by linking a proprietary binary with GDB, and not releasing the source code of the combined program under the GPL, they would not (according to the FSF, see above) be in compliance with the terms of the GPL.