I have large list of strings. Each string is of different length, just like sentence in text. Average length is below 255 chars. Please, propose file structure on disk to enable fast picking of random sentence. This means I need fast: * read total count * read N-th string. There's no requirements of writing time. It's better if structure would be simple and waste of disk space is minimum.
I come with following format:
<N, number of strings> <S1, fixed size first chunk of string 1, right padded with zeroes if size of chunk is less than size of string> ... <SN, string N chunk, ends with index of next string N chunk, if string N dont fits in single chunk> <SN+1, string N additional chunk 1> <SN+2, string N additional chunk 2>
Additional chunks of the same string I can put together to minimize seek and reads. This way I pick every string with equal probability and read it with minimum of 1 seek and 1 read and at most 2 seek and 2 reads. I can pick average string length as size of a chunk.