Looking for the correct architecture -
I have a serverless app, hosted at AWS. It uses Lambdas and several DynamoDB tables for most of the BE logic (managed with aws-amplify).
I want to add a feature where users can upload CSVs, see them as a table on the app, and create a simple public API to fetch one row, based on ID (no need for more complex queries). Structure of the CSV (columns) varies with each upload.
Each users will add about 0-10 CSVs, each CSV will contain 3-20 columns and around 1k-100k rows.
How should I build the database part? (not limited to Lambdas/DynamoDB)
Thanks
EDIT
The solutions I had in mind is:
1.
Create a new table (sql/document) for each CSV upload, and save the name of the table under user.csvs[]
.
This way I'll have huge amount of tables. Is that a reasonable solution?
- Add all CSV data to a document db, e.g. --
user {
name: "john",
csvs: {
csv123: {
id345: {col1: 'x', col2: 'y'}
}
}
}
What should I index in this solution for best performance?