we have a monolith that basically reaches close to a 100k IoT devices/routers and tries to gather some information about them.
As you can imagine, this can take close up to 40 hours for the process to complete(due to badly designed monolith + slow network that reaches these devices)
Now I am trying to break this IoT device syncing part out from the monolith, and make it it's own service(with multiple handlers to manage the load to make things faster).
But all the data/credentials required to connect to these devices are still in the Old Monolith. This data changes rarely, probably once a month.
What's the best way to sync data from the old monolith? Should it be a REST API in the old monolith that returns a giant JSON with all the data which the new micro-service can call like say every week?
I know about things like event based architecture which works great for a large team, but I am a lone developer(so limited resources) so I am wondering some practical engineering solution that works and which can avoid making the data syncing part it's own project.