I have some questionnaire data in CSV files for different projects. I created a function that takes a specific subset of columns and calculates aggregated values. The problem is that across these different projects, the column names are different but they still need to be aggregated in the same way
The way I'm handling this right now is as follows...
Each project uses a different python script where I use a dictionary to map keys to specific columns in my dataframe/csv file:
import pandas as pd
df = pd.read_csv("data.csv")
q_map = {'q1': df['question1'],
'q2': df['question2'],
'q3_h': df['question3_hours'],
'q3_m': df['question3_minutes']}
A different q_map
is needed for each projects because the column names will vary. For example, here q1
is mapped to df['question1']
, but in another project it might be called df['q1_1']
I then pass q_map
into my aggregation function:
def aggregate(q_map):
if len(q_map) != 4:
raise Exception("Incorrect number of items")
total_a = q_map['q1'] + q_map['q2']
total_minutes = q_map['q3_h']*60 + q_map['q3_m']
return total_a, total_minutes
total, minutes = aggregate(q_map)
So in essence the dictionary is used as a way to ensure that the column names are always the same within the function, that way the function itself doesn't need to care if columns are named differently across projects, everything will still be aggregated in the same way
This isn't very user-friendly for (at least) 2 reasons:
- The end user needs to pass in an exact number of columns for the aggregation to work. I'm handling this right now with the
Exception
but theres no intuitive way for the user to know exactly how many columns need to be passed in without reading documentation. - The keys need to be the same as what is used internally by the function (e.g.
q1
,q3_h
). Again, its difficult for the user to know exactly how to name their keys when creating the dictionary. An incorrectly named key will cause problems.
I feel the natural solution is just to use named arguments in my signature like:
def aggregate(q1, q2, q3_h, q3_m):
pass
That way the user doesnt need to care about naming or how many columns are passed in. However, in reality this function uses 42 different columns for aggregation, and I feel like a function signature of that length would get unwieldy and easy to pass columns in the wrong order
Is there a more sensible way (other than named arguments) to handle this type of situation, where you need to enforce a specific number and specific name of arguments going into a function?
q_map
.