Noticed a lot more junk in your inbox? Danny Bradbury reports on the increasingly sophisticated methods being used to pump out millions of unwanted emails.
For David Hart, monitoring spam is a matter of atonement. The technician, formerly a consultant for a spammer, reformed a couple of years ago and started his own DNS blackhole list - a list of Internet addresses that have been identified as spam senders.
Volunteers at TQM3, including Hart, watch email traffic for likely spammers and constantly update the list, which is then available to systems administrators across the Internet.
"There's a certain amount of guilt," says Hart. "I was wrong, and this is part of making amends for a profound misjudgment on my part."
Recently he's been getting plenty of opportunities to redeem himself, thanks to the "tail" - a screen on his email server that shows all the IP addresses on the Internet that are being added to his spam list. Normally, the tail twitches along at a steady rate, but lately, activity has exploded. "It's been scrolling so fast that it's been hard to keep up with it," says Hart. "Since the early summer we have seen a significant increase, and it has been reported elsewhere."
The growth in spam is also showing up at companies such as Postini, which analyses Internet traffic using its filtering system before delivering it to clients. "We're seeing growth in overall spam in terms of volume and relative percentage," says Postini founder Scott Petry. The company monitored nearly 70bn emails in September and October, and spam levels soared by 59% during that period. The company says that 91% of emails are now spam.
Why? There is no single cause, say experts, but rather a confluence of unfortunate events. One of the biggest problems is that the main vehicle for spam - vast networks of home computers infected with malware, known as "botnets" - have been growing in size.
These botnets have existed for about five years. They are created when worms or Trojan horse programs are used to infect a PC, taking control of it and forcing it to accept commands sent by a central controller. The commands, traditionally sent via a real-time online chat protocol called Internet Relay Chat (IRC), were initially used to force large numbers of bots to attack a target web site, flooding it with traffic in a distributed denial of service (DDoS) attack.
DDoS attacks still happen, but bots are increasingly used to send spam emails, in effect acting as their own mail servers. In the 1990s, spam was mostly channelled via unprotected email servers online, used to send thousands of unsolicited emails anonymously. But then network administrators began locking them down.
Now, spammers send email directly from home machines thanks to botnet operators who program them to become email servers and then sell their processing power and bandwidth. These money-motivated operators are honing their product, warns David Watson, who works on the UK Honeynet project, a non-profit research effort to track spammers' activities.
"Because of the high risk/reward ratio, the major perpetrators are upping their game and producing more professional and more effective cyber-scams on an almost daily basis," he says.
Today's bots are infected with modular, kit-based software that can easily be upgraded and reconfigured to take advantage of new vulnerabilities discovered in Windows; botnets are almost exclusively targeted at Windows PCs.
"There are changes in command and control capabilities and a move to shorter-lived botnets that deliver their spam very quickly," warns Watson. "And the adoption of peer-to-peer communication will also make it harder for system defenders to keep on top." Marvel for a moment: botnet operators are people who have worked out how to upgrade the software on thousands of machines almost simultaneously without their owners' knowledge. It would be impressive if the effects weren't so deleterious.
Botnet operators have in the past controlled infected machines from a single point. Today, advanced malware such as the Sinit worm uses the same techniques as peer-to-peer file distribution networks such as Kazaa, with infected machines forwarding instructions and software updates to each other. This makes botnets even harder to take down. And now that some botnets use encrypted communications, it's even harder for experts to monitor their controllers.
Botnet operators are becoming smarter too about how they use the infected machines, explains Simon Heron, a director at security company Network Box. "In the old days, a compromised machine would start cranking out thousands of emails per hour." That would slow the PC down. "It became very easy to realise that you'd been violated," he says. But nowadays, operators send out smaller flurries of email from each machine, keeping them below the ISPs' (and users') radar. "Now that they're doing it in a more subtle fashion, it's difficult to realise that they've been violated," notes Heron.
Rise of the botnets
How can botnet operators maintain the overall volume of email while reducing the number of emails per bot? By increasing the number on the network. According to Dean Turner, senior manager of Symantec's Security Response team in the US, the company saw almost 4.7m new active bot network machines in the first half of this year. And Johannes Ullrich, chief research officer at the SANS Internet Storm Center, which monitors online threats, saw the number of attacking client machines rocket from 770,000 on October 15 to 1,845,000 six days later.
But why are botnets expanding so much now? One reason could be Stration, one of the most aggressively distributed pieces of malware that Graham Cluley, senior technology consultant at anti-virus company Sophos, has seen. The worm, which persuades users to install it by claiming to be a system patch, accounted for half of all malware seen online on some days, accoording to Sophos.
"It's constantly being refined to get past antivirus products," Cluley says, adding that there have been over a hundred new versions of the worm created in a single day. It mostly targets English-language speakers, Cluley says. "So, we're not seeing as many bots created in non-English-speaking parts of the world as we used to."
But TQMCube's Hart is. According to his figures, large numbers of new bots are coming from Poland. "The spammers have really focused in on these developing technology countries that have a sudden burst of broadband," he says.
Countries just beginning to roll out broadband often have large numbers of computers with older, unpatched operating systems ripe for infection. Take China, which is the largest source of botnet machines after the US. Research group Ovum says that broadband penetration in China has been growing at 79% a year for the past three years. Within the next 12 months it will become a bigger broadband market than the US, says Ovum.
The types of spam being sent are also changing, warns Network Box's Heron. In particular, spam mail promoting cheap, penny stocks in obscure companies have grown in volume. Called "pump and dump" spams (It's the name on everyone's screen, Guardian Money, November 4), they work by sending false or outdated "inside information" to large numbers of people urging them to buy the stock, promising a leap in price. The spammers, who have bought the stock cheap, dump it on the market and pocket the difference.
'Pump and dump' spam often uses embedded images, rather than text or HTML links, making it harder to spot, says Cluley, who believes many are being sent from Stration-infected computers.
"We've begun to see that overtaking the traditional spams" that sell performance-enhancement drugs, Heron says. "It must be giving a better return to the spammers." It is also a bad deal for victims, as the site spamstocktracker.com illustrates.
Hart wishes that ISPs would simply block all unauthorised traffic on port 25, which computers use to send email.
He argues that any port 25 traffic not destined for an ISP's own mail server and accompanied with an authorised user name and password should be rejected. However, neither of the UK's most popular ISPs, BT Retail and NTL, block this port, although they do scan for bot-like activities on their own network.
But if botnet operators continue to send fewer mails from each bot, scanning for telltale activities may become more difficult - and experts worry that the mails could become more effective.
"Lower-volume target attacks are on the rise," says Watson. "Just like in the legitimate world, better market demographics and more targeted sales techniques can sometimes yield better results, and cybercriminals understand this."
Building a database of more targeted information about an individual, such as where they work, and sending mail specifically to them will enable spammers to increase their per-spam yield.
The trade-off for more focused spam will be the effort involved in gathering information about their targets, explains Heron. But just as legitimate markets evolve, so do illegitimate ones, concludes Watson. "One of the common opinions in the botnet tracking community is that in this particular arms race, the black hats currently have the upper hand."
• Stop the bots: how to keep a clean machine
You can check whether your machine has been listed on the Internet's blacklists as a possible source of spam. First, find your present IP address at http://www.freeourdata.org.uk/blog/?p=55 then try the set of four "triples" (eg 192.168.0.1) that yields at blacklist sites such as spamhaus.org/xbl/ or robtex.com/rbls.html.
Another check for outgoing spam, if you have a router with a built-in firewall, is to turn on its log and set it temporarily to block outgoing traffic on port 25. Don't try to send ordinary email (though webmail and Gmail will still work, as they use different outgoing ports). If the router's log shows multiple attempts to send, you know something's amiss. If you suspect your machine has been infected by a virus or worm and is being used to send spam (or, as can also happen, to store stolen or illicit files or even to host phishing websites), the only surefire way to get rid of it is to wipe your machine clean and start again from your original system disks and most recent data backup. Try to get the latest system updates on CD and install them before you go online again; an unprotected PC can be re-infected before it has time to download the update files.
Antivirus software isn't enough; the variety of bots and their rapid evolution means no single piece of software can root them out if you let them on your machine. That's why Windows users, especially those on broadband, should at the very least have a firewall, install Windows XP's Service Pack 2 and turn on Automatic Updates in Windows. Microsoft also offers the free Malicious Software Removal Tool. Millions of people have benefited; will you join them?
No responsibility can be taken for the content of external Internet sites.