How to implement a system to hide spammed user content?

Question

I'm trying to work out a good way to manage user submitted content and create a method to determine when spammed/poor content should be hidden.

The aim is to create comment voting similar to youtubes like/dislike comments and if the score is good show the comments, else they're invisible.

Approach

My thought is to do this via the comments table, you could create two columns rateup, ratedown.

For each rate up the comment gets the rateup colulmn is: rateup = rateup + 1

and the same for the ratedown: ratedown = ratedown + 1

You can then put a clause into your database query to only select comments if their rateup ratio is greater than a number, say -5:

WHERE (rateup - ratedown) > -5 

but this would need a lot of voting up and down if a user posts many spam/poor content posts over the site?

To strengthen the approach I could add the same two columns to the user table and also update these columns to give an overlal view of the users combined content submission ratio and then display comments like so:

WHERE ((comment.rateup - comment.ratedown) > -5 ) && ((user.rateup - user.ratedown) > -5 )

Issues

Would this approach lead to instances of users creating mass fake accounts to attack good users and ruin their reputation?

What other methods could I use to help hide spammed/poor content without moderators having to manually go through everything?

Answer 1

I'd recommend using a multi-step process to reduce the ability to spam.

First, require registration to submit content. In the registration process use CAPTCHA or another similar (but less intrusive) process to block auto-spam scripts along with email response verification. This will prevent most automated attempts and slow down a lot of manual or semi-manual spamming.

Also, record IP addresses. You can use this to compare against known spamming proxies and to catch multiple account creation. I wouldn't flag or ban on this basis alone but put such accounts on a moderator monitoring list.

Next, use a content grader algorithm to watch for common spammer techniques ("Great Post!!!"), certain keywords related to certain products, common curse words and for the insertion of links from sites other than recognized ones that you want to white list (Wikipedia, CNN, etc.). Flag offending content for further examination.

You can also use a participation scoring system like Stack Exchange uses. Add features as users participate. Spammers generally won't participate in a site but will move on to easier targets.

After that, use a user spam scoring system to allow users to flag inappropriate content. Don't act on this alone but put flagged content on a moderator list for further action.

Answer 2

Have a separate "spam" flag that any user can use.

When the number of spam flags hits 5 (as in Stack Exchange say), automatically delete/hide the post and put a "spammer" flag against the user. When the user gets "N" spammer flags (where this could be as low as 1 or as high as you like) then stop the user posting anything.

This is independent of the like/dislike flags and this can carry a higher weight.

Have the spam flags listed for moderators to review so that they can spot someone abusing the flags.

Answer 3

Another approach would be to use flagged posts as data points to train a neural network. Then the program could screen content as it is posted. The book Algorithms of the Intelligent Web is kind of heavy reading, but it outlines the theory behind spam filters nicely.

Answer 4

I think your problem needs to be split in 3 sub-problems:

1) rating the user-submitted content

2) avoid people "cheating the algorithm"

3) spam detection

Solutions:

1) Simply subtracting upvotes and downvotes won't help much. For example, if a few people downvote it as the content gets out, it will be quickly hidden - other people will have no chance to upvote it. You could prevent this setting a minimum amount of votes, but this brings to another problem: Content with 1 upvote and 6 downvotes has a total of -5. Content with 1000 upvotes and 1005 downvotes also has a total of -5, but its ratio is more balanced, and although it's probably something controversial, you might not want to hide that. This is an article I have found on how to rank entities: http://evanmiller.org/how-not-to-sort-by-average-rating.html . Probably the article goes in a direction that you don't really want to take, but it might help your decision.

2) You would need some kind of system that detects "vote fraud". I would suggest you to look into Support Vector Machines, or Artificial Neural Networks. You will need to identify patterns in the voting that are not natural. (IP blocking can be done without SVM or ANN, but that might not be enough)

3) Although I really like the Neural Network answer, probably is not really feasible due to the huge amount of data you'd need to train it. The organisation behind Wordpress, (most likely) already trained some machine learning algorithm for you: http://akismet.com/ . It is a popular anti-spam service, and it works pretty well

Hope this helps