2

I have a node js app, in which I need to perform background jobs. My plan is to use a data structure which will contain all the jobs needs done. My design also suggests that this data structure needs to persist between application restarts. To achieve that I can think of two options to store the datastructure:

a) use a database

b) use a file

Solutions like external queues might probably be an overkill for my app(the data structure will contain 5 - 10 entries(jobs) at any time).

Currently, I am inclined towards using a log file, which I think will be faster and plus I do not need to query that info(therefore need for a database).

How do you evaluate this approach? Is using a log file for runtime persistence a good practice? Are there any other benefits/drawbacks to it? Is there an alternative I should consider in my scenario?

Improve this question
5
  • It sounds like you're confusing concepts. A log is an append-only thing, it records events. What you apparently want is some registry of scheduled jobs, that's not a log file. You can put this registry in a file but then you need to handle potential parallel accesses and modifications yourself, which is error-prone. Or you could use a really small database such as SQLite which handles all the locking stuff and is supported in many programming languages. I'd prefer the second route.
    – Hans-Martin Mosner
    Commented Dec 11, 2019 at 9:48
  • Nodejs is single threaded, so maybe I will not face parallel access issues. Unless this(file IO) is handled by the operating system which will then become troublesome. Right? Thanks, but I cannot get it out of my mind that a database is not an overkill for this as I will need to constantly update it (each time a job finishes or a new job is added). Maybe though it is the only reasonable solution !?
    – Cap Barracudas
    Commented Dec 11, 2019 at 9:56
  • Sqlite persists after system reboot right ? But it is polled on the memory for faster access ?
    – Cap Barracudas
    Commented Dec 11, 2019 at 10:06
  • 2
    Back in the day, batch jobs were done with single files in a directory. All jobs were done, when the files were moved from one directory to another. In case of errors it was moved to another directory. I would call that poor-man's-queue but it works. So a file approach could be viable under circumstances.
    – Thomas Junk
    Commented Dec 11, 2019 at 11:36
  • @Hans-MartinMosner under certain circumstances a log-file could do the trick: If you know beforehand which jobs have to be done, reading the logfile could give you a clue of what is left to do, But that is - admittedly - not the best of all solutions.
    – Thomas Junk
    Commented Dec 11, 2019 at 11:38

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.