Generally, such speed-ups can only be achieved by splitting the project into modules that are then compiled separately.
But doing that in a way that doesn't reduce maintainability and development speed often requires re-structuring and re-architecting, because the common modules of "Core", "Model", "UI", etc.. are not suited for this kind of compilation.
So instead of "horizontal slices" you need "vertical slices". Modules that contain all logic for specific feature, that can be compiled separately from others. Or maybe some kind of plugin system, where plugins can be compiled separately. But that requires creation of stable and solid API for those plugins to use.
If you don't wan't to go as far as creating separate modules, then re-architecting to increase capacity of parallel compilation is best option. Most project that I saw had really "tall" dependency trees of modules. Instead, you should strive for "shallow" and "wide" dependency tree. Which maximizes amount of modules that can be compiled in parallel.