We have been in the process of scaling up my companies infrastructure, we have graduated from single servers and moved everything off to scalable AWS Ec2 servers that auto scale, there is one little item that has not scaled well though
Before, we had the server OS firing off a script that sent out emails via a cron task (ubuntu linux) unfortunately now because there are multiple concurrent servers, these cron tasks are firing concurrently, instead of sending 1 email they now send 2+ emails depending on how many servers are running at that given moment
Originally I tried creating something using one of our databases to check if the task had run in the last 5 minutes before allowing the task to run, but if they are both starting and hitting the database at the exact same time without latency then they will both run continue. This seems to be the case because multiple emails are still going out. I havent found a good solution yet, are there any ideas on what I can do to scale a task runner outside of moving this task to something serverless?
sorry let me further explain what the app is doing (its not an email server) every week a group of people at my company is to receive a report of their progress, the application generates a report and saves it to the database, then emails the link to the user. Because these were being generated by a command line script using cron, at 6AM the computer runs the command line script, where it generates all 40+ reports and emails a link to each user. Because now there are multiple concurrent servers, each one runs the same script at the same time - generating an independent report, and sending the link to the user. From the users perspective they are receiving two emails with the same data.