CSV files and many other text based formats encode information in a potentially variable size. Due to the variable record size, there is no way to directly jump to a random record if you don’t know its precise position in the file.

To get some randomness in your app, you could imagine to go to a random offset in the file and move forward or backward to the next record by looking for a line separator. However, you will not know which record it is. Moreover, if you use the CSV variant defined in RFC4180, this will not work, because line separators can be part of a field if it is enclosed between quotes, and you wouldn’t know if there wasn’t such an unclosed opening quote before. The only way is then to read the file from the start and parse it record after record.

What you need to do is to construct an index. , as suggested by Killian and Amon in the comments. For example:

• at start of your programme by parsing your data file, and build in memory an array with the starting position of the record in the file.
• or, use a separate indexer that parses the file, and created an index file. Each index record is of fixed size and indicates the position of the next record in the main data file. Since the size of index record is fixed, you can locate easily any random record n by going to position n*sizeOfRecord in the index file, read the position of where to find the record in the data file, and start reading the data at this offset.

Remark: this indexing approach works with any variable length encoding. It could be multiline files, or even complex formats such as json or xml, provided that the indexer can parse the expected format to identify the boundaries of each record.

CSV files and many other text based formats encode information in a potentially variable size. Due to the variable record size, there is no way to directly jump to a random record if you don’t know its precise position in the file.

To get some randomness in your app, you could imagine to go to a random offset in the file and move forward or backward to the next record by looking for a line separator. However, you will not know which record it is. Moreover, if you use the CSV variant defined in RFC4180, this will not work, because line separators can be part of a field if it is enclosed between quotes, and you wouldn’t know if there wasn’t such an unclosed opening quote before. The only way is then to read the file from the start and parse it record after record.

What you need to do is to construct an index. , as suggested by Killian and Amon in the comments. For example:

• at start of your programme by parsing your data file, and build in memory an array with the starting position of the record in the file.
• or, use a separate indexer that parses the file, and created an index file. Each index record is of fixed size and indicates the position of the next record in the main data file. Since the size of index record is fixed, you can locate easily any random record n by going to position n*sizeOfRecord in the index file, read the position of where to find the record in the data file, and start reading the data at this offset.

Remark: this indexing approach works with any variable length encoding. It could be multiline files, or even complex formats such as json or xml, provided that the indexer can parse the expected format to identify the boundaries of each record.

CSV files and many other text based formats encode information in a potentially variable size. There is no way to directly jump to a random record if you don’t know its precise offset in the file.

You can also imagine to go to a random offset in the file and move forward or backward to the next record, looking for a line separator. However, you will not know which record it is. Moreover, if you use the CSV variant defined in RFC4180, this will not work, because line separators can be part of a field if it is enclosed between quotes. The only way is then to read the file from the start and parse it record after record.

What you need to do is to construct an index. For example:

• at start of your programme, parse your file, and build a table with the starting position of the record in the file.
• use a separate indexer that parses the file, and created an index file. Each record is of fixed size and indicates the position of the next record in the file. Since the size of index record is fixed, you can locate easily any random record n by going to position n*sizeOfRecord in the index file, read the position of the record in the data file, and start reading the line at this offset.

CSV files and many other text based formats encode information in a potentially variable size. Due to the variable record size, there is no way to directly jump to a random record if you don’t know its precise position in the file.

To get some randomness in your app, you could imagine to go to a random offset in the file and move forward or backward to the next record by looking for a line separator. However, you will not know which record it is. Moreover, if you use the CSV variant defined in RFC4180, this will not work, because line separators can be part of a field if it is enclosed between quotes, and you wouldn’t know if there wasn’t such an unclosed opening quote before. The only way is then to read the file from the start and parse it record after record.

What you need to do is to construct an index. , as suggested by Killian and Amon in the comments. For example:

• at start of your programme by parsing your data file, and build in memory an array with the starting position of the record in the file.
• or, use a separate indexer that parses the file, and created an index file. Each index record is of fixed size and indicates the position of the next record in the main data file. Since the size of index record is fixed, you can locate easily any random record n by going to position n*sizeOfRecord in the index file, read the position of where to find the record in the data file, and start reading the data at this offset.

Remark: this indexing approach works with any variable length encoding. It could be multiline files, or even complex formats such as json or xml, provided that the indexer can parse the expected format to identify the boundaries of each record.