Wednesday, April 07, 2010

You've just been drafted.

Pretty soon you're going to start seeing a pink box like this at the top of the front page every once in a while:


This "reddit's spam filter needs your help!" box is the first step in a major overhaul that's been in the works for the better part of the last year and has been in our dreams since forever (reddit admins have weird dreams).

We've never been comfortable with the collateral damage caused when our anti-spam and anti-cheating mechanisms catch an innocent victim, but until now, the alternative would be to let spam and cheating take over the site.

Like most components of reddit, this is a story of outgrowing:
  • In the earliest days, there was no spam.
  • Then, there was some spam, but users would downvote it right away.
  • Then, the New queue was so flooded with spam that it became unreadable, which ultimately starved the front page of good submissions. So we (the admins) started removing it manually.
  • Then we asked you guys to report spam so we at least didn't have to go looking for it.
  • Then, even sorting through the reports got overwhelming, and we had to turn the job over to moderators.
  • Then, the moderators were overwhelmed and an automated spam filter had to be set up for each reddit community.
  • Then, traffic grew so much that the spam filter's tiny false positive rate started accumulating into a constant stream of stories about poor souls who were unfairly blocked. Most redditors are understandably sympathetic to these stories, and so there have been numerous prominent submissions that inevitably end with us being accused of censorship -- or at best, being a police state. And that makes us feel terrible.
Adding to the problem is the fact that the spam filter only really works when it's fed a constant stream of training data -- "This is spam." "This is not spam." ... it has a really voracious appetite for this training data, and moderators simply can't keep up anymore. So the malnourished spam filter starts acting crazy, and in a vicious cycle, the moderators get more work to do.

So now, in the hopes of solving this problem once and for all, we're drafting people like you to help out. We call this new system "deputy moderation" and will be putting up instructions here just as soon as we get around to it. (It's a wiki, so you can jump in and help too.)

As you flood us with useful training data, our spam filter will get better and better. Soon your feedback will -- when there's a quorum and a clear consensus -- be able to stand in for moderators when they're asleep or otherwise not available to tend their queues. We expect Australian and New Zealic redditors will especially love this new feature, for reasons enumerated quite bracingly over here.

If this stuff works, we'll be able to decommission a lot of our sneakiest anti-spam measures, which while extremely powerful at stopping spam seem to also be the ones with the worst collateral damage.

P.S. The first two words of this blog post are a lie -- it won't be "pretty soon" unless everything goes according to plan, and it never does. We're going to deploy this change very slowly and carefully, since it could kill the site in about seventy-three different complex ways. We'll probably enable users for this in order of seniority. Or maybe descending karma. But we will get it out as quickly as we safely can.

Thanks for volunteering being volunteered!
discuss this post on reddit