I process a lot of tweets in real time using python and for each tweet I need to assign it in to a specific bucket. I have about 50 buckets, each with their own rules. The majority of them are simple like so
bucket_one = ['test', 'foo'] if any(x in tweet['text'] for x in bucket_one): tweet['bucket_one'] = 1
So the way I designed it is by having a dictionary where the key is the bucket name and the value is a list of terms that would need to be present. This way I can have 1 generic method that would assign the tweet to a bucket by looping through the dict.
My problem is that some buckets have more complicated rules like:
if 'boo' in tweet['text'] and not any(y in tweet['text'] for y in no_words): tweet['bucket_two'] = 1
So as you see I have a neat and elegant way (in my opinion and please challenge me if you can think of a better way) of assigning the easy buckets however there are about 10 buckets with more complex rules that differ.
Is it ok to have 1 generic method for 40 buckets and then have an individual method for each other more complex bucket? or should I just have 50 individual methods?