Loading in and Autocompleting a CSV File React

Question

I'm currently working on a React Application with Firebase that involves a ton of data manipulation and reading. One specific feature is the ability to autofill a form based on a very large CSV file (about 500 MB).

I am wondering what would be the best option to go about this using React to read the CSV file? I doubt just putting the CSV in the project folder and reading it that way is the best, seeing how that would increase my project by 500 MB. Should I somehow just throw it into a Firestore collection?

Also, if I were to throw it inside the src project of React, what is a good limit to place on the size of CSV to keep locally?

Cheers in advance

Answer 1

By “autofill a form,” you mean give the suggestions for autocompletion to the user when the user is typing in a given field of the form, right?

CSV format is not particularly convenient when the task is to search for matches. Especially if the CSV file is not sorted, you have to walk through the entire file in order to find all the matches, and even if it's sorted, you have to write your own algorithm (or find an existent one) to do the job more cleverly. This is just too difficult, and both disk usage and application performance would suffer.

Instead, extract the entities from the CSV file and import it to a database. Databases are ideal for just that: searching for a small number of elements in large (sometimes huge, several terabytes and more) sets. Then it's up to you to do a simple query which would perform the actual search.

Since you're talking about 500 MB of text data, this is a relatively small size for virtually any database. This means that you can go with whatever database system you're most comfortable with.