According to the License compatibility matrix from GNU, if you link your GPLv2 (or later) software with an LGPLv3 library, the effective license for the product becomes GPLv3.
The copyleft nature of the GPL licenses means that everything your software depends on must be available under the same terms as the part that actually carries the GPL license. That includes libraries you link to (with an explicit exception for major OS components and standard libraries).
The code doesn't have to be all licensed under the GPL, but the GPL does require that you have the freedom to modify all parts of an application that contains GPL code and that you have the freedom to distribute the application with or without modifications.
What it means for a license X to be compatible with the GPL is that the license X gives you, the user, at least as much freedom as the GPL does, so that in a product that contains both GPL code and code that uses license X, the distribution of that product doesn't violate any terms and conditions of either license.
GPLv3 and GPLv2 are not compatible with each other, because GPLv3 has put additional requirements in place, which is not allowed under GPLv2. This is also the reason why a combination of GPLv2 (or later) and LGPLv3 effectively 'upgrades' the GPLv2 code to GPLv3, which is possible under the "or later" permission.