Trackback as a Casualty in the Spam Wars

Regular visitors to this blog may have noticed that for a few weeks now, there no longer is a trackback facility here. I'm a strong believer in trackbacks; they're an important tool for better connecting the distributed conversations which take place across different sites in the blogosphere. Unfortunately, however, trackback as it exists today remains a highly vulnerable technology; because of its extremely lightweight protocol, there's no reliable way to protect against spammers trying to game the Google PageRank of their Websites by posting thousands of trackbacks with links to their sites all over the Web.

Yes, there are trackback spam filters or general pre-publication approval functions which will at least ensure that such spam trackbacks are never visibly posted to my site; all that's left for me to do is to delete the spam from my trackback queue and to publish the small number of legitimate trackbacks buried in all the spam. But as I found out the hard way, by the time the spam has arrived, the damage is already done.

Like many Australian users, I'm on a shaped plan - once my monthly incoming traffic allowance is exhausted, my broadband speed is shaped down from ADSL2+ speeds to a meagre 256kbps. I'm a heavy user, so this does happen pretty regularly, but usually towards the end of the monthly period, and I monitor my traffic to stretch it for as long as I can. So, I was all the more surprised when in July, just two days into the new period, I had already exhausted 60% of my allowance - on closer examination, in fact, it turned out that I had received roughly 30GB of incoming traffic per day - all this while I was still in Boston, and certainly not using my Australian broadband particularly heavily...

As I found out from my server logs, virtually all of this traffic had been generated by a handful of persistent trackback spammers hitting my Website over and over again, usually multiple times each second (perhaps their spam scripts had gotten stuck on my blog for some reason). Trackbacks are relatively short messages in and of themselves, so you can imagine the massive amount of individual trackbacks these spammers must have sent me - and of course, even switching off trackbacks on the Website doesn't necessarily keep the spammers from trying to send their messages (even now, with trackback functions switched off for several weeks, I'm still receiving spam...).

Ultimately, I ended up resorting to some harsher measures: first off, I created a list of all IPs showing up in the Apache access log as having sent trackbacks, and sorted the list by the number of trackbacks sent (in descending order). On Linux, the following script creates this list:

cat /var/log/httpd/access_log* | grep -i post.*trackback | cut -f 1 -d - | sort | uniq -c -d | sort -b -g -r > /var/log/httpd/trackback-spam

Then I picked out the worst offenders (anyone who'd sent more than 100 trackbacks over the period covered by the access logs), and banned access to my Apache Webserver (below). This was remarkably successful - while I'm still getting some rogue trackbacks, they're now only a very small component of all incoming traffic, and no longer affect my traffic allowance.

I also keep an eye on new developments: running

tail -f /var/log/httpd/access_log | grep -i post.*trackback

in a shell window continually shows any incoming trackbacks, and I manually add any further repeat offenders to my Apache ban list.

For anyone else in a similar position: you might want to add the following ban list to your Apache configuration file (probably in /etc/httpd/conf/httpd.conf) - these have been the worst offenders for me:

<Directory "/var/www/html">

#
# Possible values for the Options directive are "None", "All",
# or any combination of:
# Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# Note that "MultiViews" must be named *explicitly* --- "Options All"
# doesn't give it to you.
#
# The Options directive is both complicated and important. Please see
# http://httpd.apache.org/docs/2.2/mod/core.html#options
# for more information.
#
Options Indexes FollowSymLinks

#
# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
# Options FileInfo AuthConfig Limit
#
AllowOverride All

#
# Controls who can get stuff from this server.
#
Order allow,deny
Allow from all
Deny from 24.243.59.44
Deny from 24.61.227.66
Deny from 38.114.204.15
Deny from 38.114.204.43
Deny from 58.147.0.227
Deny from 62.231.243.136
Deny from 62.231.243.137
Deny from 62.231.243.138
Deny from 63.223.73.31
Deny from 64.111.110.10
Deny from 64.111.117.7
Deny from 64.111.212.66
Deny from 64.141.108.29
Deny from 64.151.124.32
Deny from 64.202.161.130
Deny from 64.202.165.131
Deny from 64.202.165.132
Deny from 64.202.165.133
Deny from 64.202.165.201
Deny from 64.22.107.90
Deny from 64.22.117.2
Deny from 64.22.79.33
Deny from 64.22.92.20
Deny from 64.34.204.49
Deny from 64.34.204.8
Deny from 64.34.66.29
Deny from 64.72.116.195
Deny from 64.72.127.155
Deny from 64.85.160.107
Deny from 65.254.224.22
Deny from 65.254.224.35
Deny from 65.254.224.37
Deny from 66.232.113.242
Deny from 66.232.120.191
Deny from 66.45.237.219
Deny from 66.7.193.187
Deny from 66.79.163.173
Deny from 66.79.165.4
Deny from 66.79.167.238
Deny from 66.79.168.43
Deny from 66.79.168.98
Deny from 66.90.101.70
Deny from 66.90.103.170
Deny from 66.90.103.188
Deny from 66.90.118.234
Deny from 66.90.73.212
Deny from 66.90.73.236
Deny from 66.96.95.200
Deny from 67.15.24.45
Deny from 67.15.250.11
Deny from 67.159.30.71
Deny from 67.159.44.136
Deny from 67.159.44.252
Deny from 67.159.5.240
Deny from 67.159.5.246
Deny from 67.168.39.103
Deny from 67.18.95.35
Deny from 68.113.166.254
Deny from 68.195.34.242
Deny from 69.13.29.43
Deny from 69.46.6.146
Deny from 69.50.210.8
Deny from 69.56.136.2
Deny from 69.59.21.25
Deny from 69.64.64.139
Deny from 69.73.182.91
Deny from 69.89.25.184
Deny from 69.89.25.192
Deny from 69.90.159.238
Deny from 70.84.110.130
Deny from 70.84.205.66
Deny from 70.84.21.226
Deny from 70.85.107.66
Deny from 70.85.147.194
Deny from 70.85.147.66
Deny from 70.85.202.2
Deny from 70.86.157.90
Deny from 70.86.222.122
Deny from 70.86.96.58
Deny from 72.22.71.65
Deny from 72.232.150.250
Deny from 72.232.189.218
Deny from 72.232.200.10
Deny from 72.232.219.210
Deny from 72.232.226.38
Deny from 72.232.229.50
Deny from 72.232.249.138
Deny from 72.232.31.82
Deny from 72.232.35.130
Deny from 72.232.86.218
Deny from 72.232.90.186
Deny from 72.249.25.172
Deny from 72.249.33.235
Deny from 72.249.44.148
Deny from 72.249.57.36
Deny from 72.36.154.178
Deny from 72.36.164.42
Deny from 72.36.207.162
Deny from 72.36.219.130
Deny from 72.36.236.186
Deny from 72.52.168.12
Deny from 72.8.121.40
Deny from 72.9.156.118
Deny from 72.9.159.180
Deny from 72.9.254.71
Deny from 74.200.197.82
Deny from 74.208.16.21
Deny from 74.208.16.27
Deny from 74.208.16.66
Deny from 74.208.9.221
Deny from 74.220.207.80
Deny from 74.50.3.205
Deny from 74.52.158.146
Deny from 74.53.25.18
Deny from 74.53.81.114
Deny from 75.126.130.51
Deny from 75.126.139.26
Deny from 75.126.151.122
Deny from 75.126.175.154
Deny from 75.126.210.26
Deny from 75.126.3.84
Deny from 75.126.42.198
Deny from 75.126.89.211
Deny from 77.232.68.39
Deny from 77.232.68.46
Deny from 80.33.150.202
Deny from 81.84.241.234
Deny from 81.95.146.227
Deny from 82.146.53.67
Deny from 82.197.131.40
Deny from 84.243.241.92
Deny from 85.204.3.181
Deny from 85.234.144.215
Deny from 85.234.150.197
Deny from 88.198.53.215
Deny from 90.149.68.224
Deny from 125.181.228.9
Deny from 193.201.54.39
Deny from 193.86.238.12
Deny from 195.200.82.110
Deny from 195.242.99.80
Deny from 195.76.80.122
Deny from 198.145.45.185
Deny from 199.203.56.248
Deny from 203.146.251.107
Deny from 203.22.204.159
Deny from 205.234.222.81
Deny from 205.234.98.111
Deny from 206.225.81.184
Deny from 207.58.179.168
Deny from 207.58.179.71
Deny from 208.101.35.52
Deny from 208.109.211.150
Deny from 208.109.233.13
Deny from 208.110.218.201
Deny from 208.116.53.138
Deny from 208.122.14.114
Deny from 208.122.32.62
Deny from 208.131.138.245
Deny from 208.53.131.69
Deny from 208.53.138.212
Deny from 208.53.138.22
Deny from 208.53.138.228
Deny from 208.53.170.164
Deny from 208.53.170.38
Deny from 208.75.148.79
Deny from 208.97.142.16
Deny from 208.97.183.12
Deny from 208.97.184.16
Deny from 209.11.242.250
Deny from 209.126.144.83
Deny from 209.139.208.178
Deny from 209.190.27.114
Deny from 209.200.36.163
Deny from 210.193.49.194
Deny from 210.94.178.29
Deny from 211.30.203.77
Deny from 213.248.63.238
Deny from 213.251.189.201
Deny from 213.251.189.203
Deny from 216.110.60.3
Deny from 216.32.74.106
Deny from 216.69.168.92
Deny from 217.198.114.13
Deny from 220.232.130.88

Deny from 218.191.16.36
Deny from 222.111.99.81
Deny from 69.74.165.136
Deny from 220.194.47.69
Deny from 69.50.221.102
Deny from 66.90.73.213
Deny from 214.3.118.210
Deny from 70.87.244.242
Deny from 72.36.140.10
Deny from 74.52.29.146
Deny from 66.90.101.11
Deny from 208.74.171.142
Deny from 67.159.44.214
Deny from 222.231.15.169
Deny from 203.162.27.93
Deny from 203.162.27.92
Deny from 203.162.27.90
Deny from 203.162.27.94
Deny from 203.162.27.95
Deny from 203.162.27.91
Deny from 61.152.145.19
Deny from 161.53.232.46
Deny from 200.83.4.6
Deny from 61.233.41.218
Deny from 209.200.52.73
Deny from 83.149.19.6
Deny from 220.215.21.87
Deny from 209.200.52.71
Deny from 209.200.52.78
Deny from 209.200.52.82
Deny from 89.33.60.35
Deny from 222.236.111.32
Deny from 200.83.4.5
Deny from 124.47.123.61
Deny from 85.125.80.94
Deny from 203.166.160.74
Deny from 210.205.32.163
Deny from 72.36.180.66
Deny from 66.128.38.34
Deny from 203.69.39.250
Deny from 221.122.59.2
Deny from 74.208.15.177
Deny from 158.203.31.128
Deny from 70.86.222.122
Deny from 202.101.105.172
Deny from 70.86.222.122
Deny from 80.108.114.153
Deny from 211.189.26.81
Deny from 207.234.131.237
Deny from 210.42.140.5
Deny from 24.87.55.20
Deny from 203.211.130.48
Deny from 62.231.243.136
Deny from 89.108.90.33
Deny from 203.69.39.251
Deny from 202.29.54.250
Deny from 60.190.79.18
Deny from 210.205.32.173
Deny from 203.69.39.250
Deny from 200.83.4.3
Deny from 200.83.4.4
Deny from 62.231.243.138
Deny from 200.83.4.6
Deny from 217.141.106.201
Deny from 205.234.129.157
Deny from 72.232.168.114
Deny from 85.234.150.249
Deny from 220.232.130.49
Deny from 72.232.182.234
Deny from 217.141.108.200
Deny from 217.141.250.204
Deny from 217.141.107.201
Deny from 66.7.192.123
Deny from 38.119.53.130
Deny from 217.141.249.203
Deny from 216.41.24.3
Deny from 217.141.251.202
Deny from 209.250.226.82
Deny from 67.159.45.97
Deny from 210.17.215.228
Deny from 80.253.80.117
Deny from 125.7.203.168
Deny from 83.149.19.6
Deny from 208.255.68.245
Deny from 64.22.107.90
Deny from 217.141.105.203
Deny from 72.249.57.37
Deny from 125.46.36.223
Deny from 68.198.108.220
Deny from 90.227.74.118
Deny from 203.229.247.2
Deny from 59.188.27.17
Deny from 221.232.159.112
Deny from 205.247.24.5
Deny from 222.66.48.253
Deny from 203.69.39.251
Deny from 24.1.218.238
Deny from 81.74.236.38

</Directory>

Technorati : Apache, blogs, spam, trackback
Del.icio.us : Apache, blogs, spam, trackback