It is not scalable to maintain one fork of your software per client, regardless of whether you try to maintain this as multiple repositories or as multiple branches in one repository. You will be unable to apply cross-cutting changes to all your clients, except with extraordinary effort. Common cross-cutting changes are refactorings, redesigns, or security fixes.
The solution is twofold:
Recognize that having an individual deployment for a client is not the same as running an individual project for that client. You can deploy with different configurations from the same codebase.
Create client-specific variants through feature toggles, build-time or run-time configuration, plugin systems, and dependency injection.
Do not hardcode modifications. If you have more than one client, you will regret this. Instead, make the engine configurable so that client-specific plugins can be loaded. When you want to modify the behavior, refactor your core engine to support a plugin there, for example by introducing a new interface. Then provide an implementation for that interface as customer-specific code. When you see that multiple clients might need that functionality you can move that code into the core, but possibly disable it for clients that don't need it.
The shared core of your software is crucial for making modifications with low effort. As it evolves, make sure that it stays well-designed. This is your framework for building client-specific variants.
For the repository layout, this hinges on whether you need to give access to the development repository to clients. If not, consider keeping all work in a single monorepo. For example:
crm/
build-tools/
core-engine/ (contains source, tests, default assets)
client-a/ (contains config and client-specific code, tests, assets)
client-b/
...
If clients need access to their repos, then it might be better to create separate repos for the core engine and the client-specific adaptions. It may be best to treat the core engine as a library that is used by client-specific apps. However, separate repositories make it more difficult to apply backwards-incompatible changes to your core, like changing a method's signature or renaming a class.