0

So I am using Stanford CoreNLP in my project. I have data which consists of reviews of products on a forum. I need to be able to assign a sentiment value to a given review. CoreNLP allows you to predict a sentiment class of a given sentence. The class varies from 0 - very negative to 4 - very positive. How do I combine the sentiment values of sentences into a final value which gives the sentiment for the whole review.

Is weighted averaging the correct way to do it? Is exponentially weighted averaging an option? Or are there other methods like averaging which provide a more comprehensive way to summarize a list of predicted classes?

1

Take an hypothetic review of an hypothetic question:

Interesting question ! You made my day ! I have never seen this before. This is such an opinion based crap that it should never ever been asked here ...

So sentence by sentence, the evaluation could look like: 1->4, 2->3, 3->2, 4->0. The average would be 2,25 which is a little better than neutral, which obviously is not the case.

The problem is that you apply the rules of continuous related values to something utterly discrete looking, considering a relation where there might not be one.

Unfortunately, there is no good answer here and no magic formula. Here are some guessing mechanics:

  • average as you propose
  • weighted average giving the sentence more weight depending on their position (Here I'd put more weight on last sentences, as in a conversation, the first sentences are introductory, but for a review it could be possible that the contrary rule is better: the user gives first his spontaneous impression and then try to add more perspective but less gut feeling).
  • optimistic or pessimistic approach, taking the either the max or the min of all the sentences.
  • averaging positive aspects, average negative aspects, and take the maximum absolute value of either positive or negative, and reduce it by a factor of the value for the opposite sentiment. So 10 minor negative sentiments could be outweighed by 2 extremely positive one.
  • same principle, but look at the most frequent positive and the most frequent negative

There are a lot of other alternatives as well, so the first thing that you have to do is to look at 50 reviews, and do your own overall rating, and simulate the different approaches trying to understand why they differ/converge with your own feeling.

If you have enough ressources, instead of trying to find a formula, use some ML approach to let your system learn how to combine the ratings based on your samples.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.