Got a forum? Noticed a lot of spam since November of 2006? Me, too. Turns out that November is when forum spamming using spambots got really, really popular. It's not just you. Everybody has the same problem.
So how do you keep a forum free of spam? Simple: moderate. Take a look at every post made. Check its IP to see if our friends at XRumer have used a proxy to get around our IP ban lists. Check for suspicious URLs. Check for reference to popular spam topics. Sure -- simple! Except that the spammers are legion and they cheat -- they use spambots. The obvious next step is to have a, well, a modbot to block spam as fast as the spambots can shovel it at you.
The modbot presented here (note: still working on documentation) is a simple, very configurable set of scripts to (1) examine pending posts, (2) apply rules you specify against them and tools anybody can write, to determine whether a post looks like spam, and (3) reject obvious spam and approve obvious legitimate posts, leaving only the marginal cases for you and your human help to deal with. And its best feature is its cost: nothing. Not a red cent. Welcome to the good part of the Internet.
I know there are a lot of you out there who don't remember the days when there were three servers and we had to set up our Websites using the toggle switches on the front panel (and who can forget Internet chats back in the days when you had to batch-process your replies on punch cards?) But in point of fact, the Internet used to be a pretty friendly place. I remember that. Spammers want to take that away from us, and that's not nice. And so spam fighting is a hobby of mine, and has been, on a serious basis, since 1999. Oh, I'm not going to complain if you throw me a shekel or two on Paypal. But if you don't have the money, no problem. You still shouldn't have the spam problem, because if you have a spam problem, we all have a spam problem.
So welcome. Bear in mind that installation and configuration of the modbot is still not for the faint of heart, but it'll get better as I get some more exposure. Right now, I can install it for you and configure it to match your forum/bulletin board setup -- it'll work with anything with judicious application of a pipewrench and a good sledgehammer. Or you can download it from SourceForge and configure it yourself, just as soon as I get organized enough to put together a proper distro (sigh). If you do, please drop me a note with any configuration techniques you found useful.
And remember: if you're interested in my help despamming your own forum, drop me a line and we'll talk.
I'm going to keep a few notes on spam categories here (these notes were the proximate reason for me to develop the Wiki code I'm using, actually.)
- Thoughts on trackback spam
- Thoughts on MediaWiki spam
- Some thoughts on text analysis
- More thoughts on text plausibility metrics
- An example spam trail
Here is a log of specific actions I'm taking in response to various attacks or insights for the forums I'm actually automodding.