Timeline for Dictionary Application: Suggestions for serialization format and data structure
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 10, 2018 at 22:33 | comment | added | McBear Holden | Wow the ESR's book seems to be a treasure trove! Thanks again man! | |
| Nov 10, 2018 at 22:12 | comment | added | amon | @McBearHolden I'm not much of a book-learning person so I don't have any recommendations. System programming might include efficient I/O or low-level tricks to avoid parsing, but it's a very broad term that also includes lots of boring stuff. Learning a bit of C is useful, I guess – if only to understand POSIX APIs or to read source code of systems-level software. While writing my discussion of file formats in the answer, I had the Textuality chapter from ESR's The Art of Unix Programming (2003) in mind. | |
| Nov 10, 2018 at 20:41 | comment | added | McBear Holden | what programming books discuss stuff like.this? System programming? | |
| Nov 10, 2018 at 20:24 | comment | added | amon | @McBearHolden Usually the program has to first parse the complete file into a data structure before performing any queries. With a suitable binary format, the on-disc format is identical to the in-memory data structure. I can skip the parsing step. That means I also don't have to load the complete file first, but it's sufficient to only load relevant pages of the file. This is much more complicated than ordinary parsing and there can be important portability and security concerns, but when done correctly (as is the case for most databases) then we get significant performance improvements. | |
| Nov 10, 2018 at 19:54 | vote | accept | McBear Holden | ||
| Nov 10, 2018 at 19:54 | comment | added | McBear Holden | You've mentioned effecicient to load binary file format. What makes a binary file efficient to load? Do you mean the parsing complexity? | |
| Nov 10, 2018 at 19:49 | comment | added | McBear Holden | Really appreciate your.detailed explanation. I learn a lot from the description of your thought process. | |
| Nov 10, 2018 at 19:08 | history | answered | amon | CC BY-SA 4.0 |