This is an open ended questions with lots of possible responses that depend on what you are trying to do. Nevertheless I'll add a few things as an answer, since a comment won't be large enough.
The service would act as a database connection pool (I think 2000+ connections on a database would cause problems);
Yes, that's a good idea. You keep a smaller number of connections opened and you reuse them as requests arrive to the service. But you need to know how fast requests will be served and how much each request uses the
database, otherwise a small pool can easily be depleted and other requests will get blocked while waiting for a database connection to be released.
Caching can help there, to return already fetched data (like I said, depends
on what you are trying to do - if queries are unique you can't cache much).
Also note that the pool size will get multiplied by the number of services you put in place. A few services and you can use large database pool sizes; more services and you need to decrease the pool size, so that you have the same number of connections opened to the database, overall.
It is possible to have a database with log shipping to other read-only database to serve some queries;
The database can easily become your bottleneck. Too many connections and/or too many queries and you can break it. At that point it doesn't matter that you can horizontally scale your services to any number. All requests will
eventually reach the same database.
There are various ways to protect it: caching I already mentioned (depends on your use case), duplicate some info on other servers to serve some queries as you mention, CQRS (depends on your use case), use a relational vs non-relational (depends on your use case again), etc.
Note though that when you distribute data like that, the CAP theorem starts to apply. Be aware of that as you might need to compromise between consistency and availability.
It would scale better as we can add more machines to run the REST services;
Yes, the REST service will scale, but as I mentioned above, if you don't protect the database, that can easily become a bottleneck.
It is possible to use HTTPS with compression for security and bandwidth saving reasons;
Yes, as well as other things... maybe you want authentication/authorization later, etc.
It is possible to make some centralized changes on business entities without redeploying the 2000+ machines;
Yes, but up to a certain extent and not all kind of changes. If you make a breaking change you will need to update the clients too. So think about a strategy to update the clients to the latest version or if you allow older clients to still work and use the application.
It integrates better with other systems (just point it to the REST service).
Yes, but that means clients for your service that maybe you can't control.
If you do a breaking change and you have a good strategy to update your 2000+ JavaFX clients then no problem. But if other clients exist and you don't have
control over them, you might need to implement versioning on the REST service and support more than one version until everyone can update to the latest.
Like I said, it depends on what you are trying to do. Overall, yours is a good idea. But be aware that stuff will not come for free just because you stick a REST service in front of a database.
Just my 2 cents!