I'm a hobbyist coder; never worked professionally. I am not looking for anyone to write code for me, but I need to know how to approach this problem and, perhaps, ideas for further research. This problem is an outgrowth of my kid's science project that I wanted to toy with.
Bottom line is: I have a sensor that feeds data objects at irregular and unpredictable intervals (temp, air pressure, other things). Each new "update" feeds another data object with all the pertinent info...and each object has a time stamp. The unpredictability is the core of the problem, and that will not change. As my code gathers new data objects at irregular intervals, I need to run simple arithmetic operations on the data objects for the last n minutes. I need to output these arithmetic operations continuously on each new "update" of fresh data. The sensor could put out as many as, say, 20 updates per second...or fewer. CPU and RAM usage are a mild concern, but I want to focus on getting working design first.
At first, I was thinking circular array or queue, but that's no good since I do not know how many objects will cover the last n minutes.
Next, I considered a doubly linked list. The problem with this approach is that it will have to consume heavy CPU by iterating over the entire list each new "update" in order to remove outdated objects from the list or will have to consume a lot of ram if I do not remove old objects from the list each time.
I am wondering what design patterns (and data structures) might fit this problem, and what other items I could research to learn more to solve this problem.
I realize I am not giving much information here, but I want to keep simple and I believe I have given the gist of the problem.
I really appreciate any help. Btw, I'm using C# and the CLR for now. Python might be a better option since this is data-science'ish. I believe I could write and/or consume a Python library/class. I'm not real fluent with Python.
UPDATE - 8/30
I've been thinking in response to everyone's responses--which are great and I really appreciate.
I'm thinking, use the ConcurrentQueue class from the .NET library as my core data structure. I did not know it was re-sizable until @amon mentioned it. A queue seems perfect because I can iterate from the head of the queue and peek at the next timestamp, using a while loop (i.e., while the next "peek" is outside the n-minute time window then dequeue). Since all of the data objects must necessarily be enqueue'd in temporal order, this should work with the head always being the oldest and tail always being the newest data object. This mitigates my concern about CPU usage in keeping the queue "current" (i.e., only containing data objects inside the n-minute time window).
As to updating the user's view and mitigating CPU usage there, I could update the view every x seconds as mentioned by @JohnWu. I would probably use a Thread Timer which would update the data objects underlying the user's view on a separate thread at a fixed interval.
If this uses too much CPU, I will investigate saving pieces of state as discussed by @ErikEidt. But since I'm not calculating only averages, that will get a little complicated. I hoping the above deals with any resource problems.
Just wanted to say thanks for the insight.
Next step is to learn more about threading.
SELECT AVG(pressure) WHERE <condition>
), but the database organizes the data and you're not stuck having to write new code when your questions change. SQLite is more than capable enough for an application like yours and gives you the option of running in memory if you don't care about persistence or on disk if you do.