Not possible. By including C++ dependencies in your Python module, you are trading platform independence for performance. You have to decide whether that tradeoff is worth it.
If so, you really only have the choice between two Python packaging formats:
- source packages that the end user must compile themselves, but are potentially portable
- precompiled wheels that are specific to a platform (Python version, OS, snd CPU architecture)
There is no magical in-between format that combines the advantages of both (unless you count currently highly unusual techniques such as compiling native code to WebAssembly, but Python does not provide a suitable platform for this).
What you can do is using hosted CI services to easily generate a large matrix of packages in order to account for different OSes and Python versions.
Wheels only work when installed directly, e.g. via a package index (doesn't have to be the public PyPI though) or when directly installing a .whl file. Installing a package from git will always need a compile because it's effectively a source package, though you could move the native parts into separate packages.
By the way, if your software is Windows-specific (why else would Visual Studio be a hard dependency) then platform independence doesn't matter, because Windows effectively only runs on x86-64 processors.