I'm currently working on an architecture that have the following structure:
(central server)
/ | \
(local server) (local server) (local server)
/ | | | | \
(PC) ... (PC) (PC) ... (PC) (PC) ... (PC)
There is one central server that talk to multiple local servers (more than 100) and every local server talk to multiple PCs.
The central server receive a big amount of data every day multiple time a day and after some manipulation synchronize the data to the local server, after that the local servers spread the data on their PCs.
All this machines talk to each other via HTTP requests, every exchange of data is done with an HTTP POST of a json file.
Is critical that every information that the central server receive is correctly received and stored on the local server before and on the PCs after.
I have now to test if the synchronization work correctly and I want to automate the testing so that a script will run continuously on the central server and check if the newly arrived data are synchronized with the machines below.
So my first question is: it makes sense to test every time ALL the data the central server receive? (We are talking about tens of thousands of database entry for every single local server, so hundreds of thousand in total.)
Also for the PCs the performance are a big issue, I can't steal too many CPU or RAM resources from them.
If the answer is no, it makes sense to test just a subset of the data taken randomly?
If no, what is the best way to act?
UPDATE THe data are passed to the local server in this way: A service on the central server receive the data and store them on a database, another service is called, this one take the data from the database in COPY format and place the COPY in a file. The file is sended to the local server via an http POST request.