It makes a lot of sense to use a HTTP API for the public API, simply because tooling and security of such APIs are very well-understood.
But of course it also makes sense to use a message queue so that the tasks can be processed asynchronously.
This is not a contradiction. You can use the message queue purely internally. You mention this approach but discard it because “when thousands of PDF's are sent to the server as a batch process by different clients, I feel this will be too much”. But why would that be too much? A couple of thousand messages is not a lot.
If you are concerned about the storage size of the message queue, then you might consider storing parts of the data out-of-band. For example, you might store the uploaded PDFs in an S3-like object store or other database, and then only put a link to this PDF into the message that is sent via the message queue.
If you are concerned that one user's bulk uploads could block other users' tasks, then you can take appropriate steps to ensure fair scheduling of the work. Rate limits alone might not be sufficient. If the rate limit is so high that it is possible to accumulate a backlog of work, then all tasks submitted at the same time will have equivalent latency. In contrast, it might be desirable to increase latency for bulk uploads, but still allow speedy responses for other users. You can achieve this using a priority queue, where the priority for one user's messages decreases as the number of their pending tasks increases. I would expect a function of the type prio = maxprio - log(n_pending_items, b) to provide some useful properties, but I'm not that deep into Quality-of-Service (QoS) algorithms.
An alternative way to deal with large queues of pending tasks is to dynamically scale your compute resources as necessary, which is feasible in a cloud setting. In case the tasks are processed by a serverless Function-as-a-Service offering, such scaling might also happen automatically.
