I am currently attempting to create a gravitational n-body simulation using a modified Barnes-Hut algorithm, to be more amicable to GPU computation. This is primarily as a learning project. My goal is to simulate a number of stars comparable to that in real galaxies, meaning on the order of hundreds of billions to tens of trillions, but even a few million would be useful. It is very unlikely that I will be able to compute this at a speed amicable to display, meaning that I must pre-compute the data and look at it after the computation finishes. To do this, my first idea for how to store the data is to create a file that has the locations of all the stars concatenated together for each moment of discretized time, and then the next moment concatenated to that, to make something like the following, where the data in each bracket represents a single frame:
{x₁, y₁, z₁, x₂, y₂, z₂, …. xₙ, yₙ, zₙ}, {x₁, y₁, z₁, x₂, y₂, z₂, …. xₙ, yₙ, zₙ}, {x₁, y₁, z₁, x₂, y₂, z₂, …. xₙ, yₙ, zₙ}
Alternatively, this format can be described in C++ pusedo-code (portability between C++ implementations is not important):
void writeData(std::vector<Frame> frames, std::ostream &out){
//decoding knows how many points there are in each frame as a property of the file format, so it can read an entire frame at a time until EOF
for(const Frame &frame : frames){
for(Point &point : frame.points()){
float x = point.x();
float y = point.y();
float z = point.z();
out.write(&x, sizeof(float));
out.write(&y, sizeof(float));
out.write(&z, sizeof(float));
}
}
}
The size of the data given by this format in bytes is n*3*2*60*25 for n particles, one minute of data, 25 frames per second, and half precision floats. For one billion particles, this works out to 16 terabytes for one minute of video with one billion particles, something that I physically don’t have even close to enough hard drives to store (and I have a lot of hard drives). I also doubt that this data will compress well with standard lossless compression algorithms such as zlib, as it is fairly unstructured from a binary perspective. I also can’t think of any reasonably simple compression algorithms that would work well for this data.
Of course, I can encode a rendered frame of the particles in a 2-dimensional image and create a video with any of the many modern compression algorithms, but this sacrifices the ability to change the camera’s location while observing the data, something essential to gaining a good understanding of the three dimensional layout of the simulated particles (ie. consider that constellations appear to be completely different as Earth moves through the galaxy, over geologic time scales). Encoding multiple videos from different perspectives somewhat mitigates this, but nothing compares to the ability to control the camera’s location and angle during playback.
How can I enable camera movement with pre-computed data and a large number of particles without having thousands of dollars to drop on massive hard drives? I think that this is possible because regular, two dimensional, video is a similar problem that is now solved enough to be practically useful, and because other people have done n-body simulations (as I have seen videos of what can only be n-body simulations in news publications).