What we do is major.minor.patch-build. So 1.3.2-589. We think it's a fairly common approach. It's also compatible with rpms, which we use for distributing some of our software.
So if I just use it as a normal build, then version precedence will always favor 1.3.2 over 1.3.2-build.589. This seems ugly (and prone to errors).
If you never create a 1.3.2, then ...
To make it compatible with SemVer, your workflow needs some changes:
When a new release is started, you start with <major>.<minor>.<patch>-build.0 (or ...1 if you're more of positive person).
The Dev/Testing cycle continues as normal.
When Testing clears a build for release, the relevant version <major>.<minor>.<patch>-...
So your key problem here to my mind is this : "And our CI/CD pipeline tools do automatic sorting and picking based on SemVer V2 precedence"
Obviously if you are going to do more than one build of a patch version your CI/CD must understand that the build number affects precedence in some way.
If you can alter this to look at the build number then ...